Compare commits

..
16 Commits
Author SHA1 Message Date
dtourolle a2c7789007 Sign the Android build with a real key, and let package.sh use it too
Benchmarks / CPU and I/O (per commit) (push) Successful in 2m52s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Successful in 44m13s
Build and test / Layer separation (push) Successful in 56s
🐳 Android image / Build and push (push) Successful in 17m16s
Build and test / android-image (push) Successful in 17m17s
Traceability / Requirement traces (push) Successful in 1m6s
Build and test / Android (aarch64) (push) Successful in 23m37s
The release keystore now exists and its four secrets are loaded into
Gitea, so CI produces an APK a device can update in place. Until now
every build, CI and local alike, was signed with a throwaway debug key
-- CI's fresh per run, the local one exactly as durable as the cache
directory it lived in -- and the night that cache was cleared, no build
anywhere could install over the tablet's copy.

package.sh forwards KEYSTORE_PASS, KEY_PASS and KEY_ALIAS into the
container and copies the keystore under the mounted target directory
for the build, so a local release-signed build is one environment line.
The doc records where the local copy of the key lives.
2026-09-11 23:22:28 +02:00
dtourolle 7c44740d9f Skip the read-only-directory test where modes are not enforced
Benchmarks / CPU and I/O (per commit) (push) Successful in 3m41s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Successful in 1h44m9s
Build and test / Layer separation (push) Successful in 48s
🐳 Android image / Build and push (push) Successful in 2s
Build and test / android-image (push) Successful in 2s
Traceability / Requirement traces (push) Successful in 36s
Build and test / Android (aarch64) (push) Successful in 1h1m51s
`a_failed_overwrite_puts_the_original_back` makes the presets directory
read-only and expects the overwrite to fail. CI's Desktop job runs in a
container as root, and root is not refused by a mode: the write succeeds,
the assertion fails, and build-and-test has been red on every push to
master since the test arrived.

The test now probes the refusal it depends on -- one write into the
directory it just locked -- and skips where that write goes through.
Probed rather than keyed on the uid, because what the test needs is the
refusal itself, and a filesystem mounted without permission checks would
pass a uid test and fail this one all the same.
2026-09-11 22:22:46 +02:00
dtourolle 4f31123b0c Let the user choose which SCRFD finds their faces
faces.md §12.3 measured what the cheapest detector costs: the small
faces in every group shot, and a dog embedded a dozen times. Which
trade is right depends on the machine doing the sweep — a desktop left
overnight and a tablet on a battery want different answers — so the
detector is now a per-device setting, Fast / Balanced / Thorough on
the settings page beside the indexing button, persisted with the rest
of the settings file.

A detector is half of a model id. Every face, marker, shard and
calibration is keyed on faces.model_id precisely so that a model change
is a new id and a re-index rather than a silent change under existing
data, and a detector change is a model change: it decides which faces
exist and where the landmarks that align them land. So each choice
names its own pipeline. 500M keeps the bare "w600k_mbf" every existing
library was written under, so an upgrade disturbs nothing; the others
are qualified. Choosing one restarts coverage from zero under the new
id, the sweep re-detects, confirmed names carry across by box overlap,
and the sync shards are keyed by the same id so a peer on another
setting neither adopts nor pollutes them. The library controller
carries the id into the sync the same way it carries the cache budget,
because the sync starts from places that have no settings in reach.

All three shape-fixed exports ship — APK, Arch, Flatpak — since a
tablet has no other way to obtain the one it was not installed with;
the APK grows by twenty megabytes for the choice.
2026-09-11 22:12:53 +02:00
dtourolle adf5d6cdd9 Drop a rival pipeline's marker when an image is re-indexed
record_detections replaces every face on an image whatever model found
them, but left the other models' face_index rows standing. With one
model that was unobservable. With a second pipeline it leaves an image
marked "done" under the first with none of its faces behind the marker
— the state the V12 repair existed to undo — and a user who switched
back would find those photographs permanently empty.

An image now holds the faces of whichever pipeline looked at it last,
and only that pipeline's marker. Confirmed names still carry across by
box overlap, since they were read before the replacement.
2026-09-11 22:12:40 +02:00
dtourolle 9d35addd86 Measure what the cheapest SCRFD actually costs in faces
§1 chose scrfd_500m on FLOPs and never measured the recall it gave up.
A dr-ui example now runs several detectors over the same sample of
stored proxies, matches boxes by IoU against the first, buckets the
result by face size, times each, and writes contact sheets of the
disagreements in both directions — because a count of extra faces says
nothing until someone has looked at whether they are faces.

Over 400 proxies from the reference library: 2.5G finds 14% more faces
for 12% more time, 10G a further 12% for 3.1× the time. The extras are
small real faces. The 86 faces only 500M found are a dog a dozen times,
a stop sign, a wheel and the backs of heads. Recorded in faces.md §12.3.
2026-09-11 22:12:39 +02:00
dtourolle 3d6d69ec90 Wrap the face-sweep repair match the way rustfmt wants it
Benchmarks / CPU and I/O (per commit) (push) Successful in 4m5s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 39m17s
Build and test / Layer separation (push) Successful in 1m24s
🐳 Android image / Build and push (push) Successful in 10s
Build and test / android-image (push) Successful in 10s
Traceability / Requirement traces (push) Successful in 55s
Build and test / Android (aarch64) (push) Successful in 27m8s
CI's Desktop job failed at the Format step on 16f3fb4: rustfmt puts the
`match faces_without_proxy(...)` on its own line under the `let` and
re-indents its arms, and the commit was written without running it. No
code changes; only the layout of that one match in library.rs.
2026-09-11 22:00:38 +02:00
dtourolle 16f3fb41a3 Measure the faces already found rather than finding them again
Benchmarks / CPU and I/O (per commit) (push) Successful in 3m13s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 54s
Build and test / Layer separation (push) Failing after 1s
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 2s
Traceability / Requirement traces (push) Successful in 52s
Build and test / Android (aarch64) (push) Successful in 30m6s
Every face stored before its quality was kept holds a unit vector, and
V14 forgot the run marker of each image holding one so that the next
sweep would look again. Looking again meant detecting again: a whole
re-detection per image, with every suggestion on it thrown away and the
confirmations carried across by box overlap, to recover one number.

The sweep now has a measuring pass between the proxy repair and the
un-indexed images. It lists every image holding an unmeasured face,
fetches the original once, warps each stored face from the landmarks it
already has, embeds it, and writes the raw vector and its length over
the old row. Ids, boxes and identities are untouched; the marker is
re-written fresh so the sync exports the measured vectors. A face whose
landmarks no longer make a warp is dropped, as detection would have
refused to store it. `faces_unindexed` leaves those images to the
measuring pass, so the V14 deletion no longer costs a second detection.
2026-09-11 21:50:12 +02:00
dtourolle 8b3abdb787 Keep each face's quality, and never compare against a poor one
The embedder's raw output has a length, and the length is a reading of
how recognisable the crop was: a blur, an occlusion or a hard profile
comes out short. Normalising threw it away. A short vector sits near
the middle of the sphere and matches a little of everyone, which is how
one bad crop bridges two people in a grouping pass.

So the length is kept — the store now holds the raw vector, re-normalised
on load, with the length beside it as `faces.quality` — and a face under
MIN_GALLERY_QUALITY (14) is a probe: measured against the gallery and
placed where it fits, but never what another face is measured against.
Two probes are never paired, and a probe is nobody's evidence for a
confidence. The People screen shows the number as "Quality 17.3", dimmed
below the floor.

Faces indexed before this stored unit vectors and have no reading; they
are admitted to the gallery, and schema V14 forgets the run marker of
every image holding one so the next indexing pass measures them. A
peer's unmeasured shard faces are not adopted, or a sync would write
that marker back.
2026-09-11 21:50:12 +02:00
dtourolle a87139b838 Give every mask an eye and a colour, and put the brush where the mask is
The first build of seeing a mask showed the selected layer's, in one global
style, from a strip at the top of the panel. It answered the wrong question and
answered it somewhere nobody looked. What a photographer asks of two masks is
how they meet — where the sky's edge sits against the building's — and that
needs both on screen at once, in colours that can be told apart.

So each row of the stack has an eye, drawn in the colour its mask is shown in,
and each mask has six swatches to choose that colour from. Several can be open
at once; a new one comes up open, in the first colour nothing else is using.
The style — tint, alpha, outline — is the one setting that stays global, above
the stack, because three styles at once are three pictures that cannot be read
against each other. Alpha now draws every shown mask, each in its colour, on
black. In the pipeline a `Reveal` is a list of `(layer, colour)` rather than
one layer, and every reveal block carries its own colour.

The brush moves too. Select, Paint and Erase and the three sliders under them
sat at the top of the panel, appeared only once a row was selected, and said
nothing about which mask they acted on — so "how do I paint" and "how do I
correct the model's outline" both had the same answer and nobody found it.
They sit under the selected mask's parts now, beside the swatches, and on a
subject or a category the hint says what a stroke there does: it becomes a
part of this mask, joined to the model's, and can be taken out again.

Eyes and colours are viewing state, on the session and not on the layer, so a
photograph reopened has every eye closed — the stored-mask round-trip test
asserts it.
2026-09-11 19:03:51 +02:00
dtourolle 936490880b Release 0.12.0
Benchmarks / CPU and I/O (per commit) (push) Successful in 12m48s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 37m37s
Build and test / Layer separation (push) Successful in 46s
🐳 Android image / Build and push (push) Successful in 2s
Build and test / android-image (push) Successful in 3s
Traceability / Requirement traces (push) Successful in 1m4s
Build and test / Android (aarch64) (push) Successful in 53m59s
2026-09-11 09:33:36 +02:00
dtourolle d8b9b5a4bb Let the release script write the release commit it was never trusted with
Every release commit in the history reads `Release X.Y.Z` and none of them
carries the message this script would have written, so nobody has ever
passed it `--commit` — and the reason is in the message it wrote: a
Co-Authored-By trailer naming an assistant, which no commit in this repository
carries and none should.

The trailer goes, and so does the paragraph above it: the script's own header
already says why it exists, and a release commit is the one place a one-line
subject is the whole convention.
2026-09-11 09:33:28 +02:00
dtourolle ac0aea70ec Show a mask as soon as it is made
Benchmarks / CPU and I/O (per commit) (push) Successful in 3m52s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 39m53s
Build and test / Layer separation (push) Successful in 1m0s
🐳 Android image / Build and push (push) Successful in 4s
Build and test / android-image (push) Successful in 4s
Traceability / Requirement traces (push) Successful in 46s
Build and test / Android (aarch64) (push) Successful in 25m19s
Choosing a category is asking what it selected, and for a subject or a
category that question had no other answer on screen: the model's outline is
not derivable from anything visible, a fresh layer carries no adjustment to
judge it by, and the list it was chosen from says "architecture 23%" without
saying which 23%. The control that draws the mask existed but had to be found
and pressed, a panel's height away from the list the choice was made in.

So a new layer arrives with its mask showing, from the resting position only.
Somebody who has chosen the alpha or the outline keeps it, and nothing re-arms
in the background — every caller is a press that asked for a new mask.

That makes the canvas depend on how a layer arrived, which is correct and
worth stating: a session that has just made a mask draws a frame that a
session which read the same mask out of a sidecar does not. Viewing state is
not edit state and does not travel in a file, and
`a_stored_mask_renders_exactly_what_the_model_rendered` now says so at both
ends.
2026-09-10 20:56:11 +02:00
dtourolle 5d175cc668 Let a press on the photograph reach the tool that was armed for it
The brush did nothing, and neither did three other things nobody had tried
lately: clicking a subject on the photograph to select it, placing a repair,
and sampling a neutral. All four are TouchAreas over the canvas, and all four
sat behind the pan/zoom area, which is full-canvas and enabled for everything
but a crop. It took every press in the viewport and they were never offered
one.

Slint hit-tests siblings front-to-back (`send_mouse_event_to_item` visits
children `TraversalOrder::FrontToBack`), a TouchArea answers `GrabMouse` on any
press it is enabled for, and the first grab aborts the traversal. Front means
*last declared*. Each of the four carried a comment saying it sat "above the
pan/zoom area so a click reaches it first" — true of the order they were
written in, and backwards.

Nothing about the geometry decides this, so nothing about the geometry could
have fixed it. The pan area is declared first now, as the backstop it always
meant to be, and the rule it leaves behind is that the general case goes above
the specific ones. `GradientHandles` is the other end of that rule and is why
dragging a handle has worked all along while everything between it and the pan
area did not.

The order is asserted in a test, because this is a fault that compiles, passes
every other test, and silently removes four tools at once.
2026-09-10 20:56:10 +02:00
dtourolle 76ad667fd6 Offer a mask that is nothing but a hand
Every route to a layer began with a selection — a gradient, a band, a subject,
a category — and painting was reachable only by making one of those and
joining a painted part to it. So the answer to "brush a correction onto this
corner of the sky" was "add a radial gradient you do not want, then paint into
that", which is not an answer.

Paint sits beside Linear and Radial and makes a layer whose base is a brush.
It covers nothing until a stroke lands in it, so pressing it arms the brush
and shows the mask as well: a row that appeared and changed no pixel, with the
pointer still in "select", is indistinguishable from a button that did nothing.
2026-09-10 20:28:09 +02:00
dtourolle c045702a47 Show the photographer the mask they are shaping
Nobody can refine an edge they are not being shown. The only thing drawn on
the canvas was the region overlay — a false-coloured picture of what the model
*detected* — which knows nothing of a layer's feather, its falloff, its
morphology, its invert or its opacity, and nothing at all about a gradient, a
range or a stroke. Every control added for mask editing therefore acted on
something invisible, which is why the whole feature reads as absent rather
than as unfinished.

A layer's finished mask now draws over the photograph in one of three styles:
a tint for whether the right thing is selected, an alpha for where the edge
is, an outline for whether that edge is registered against the detail the
other two hide.

The hard part is not the shader. A selection with no adjustment on it changes
no pixel, so it is not active, so it holds no slice of the mask array and is
never rasterised — and that is exactly the layer somebody wants to look at,
for the whole of the time between choosing a subject and deciding what to do
to it. So `MaskStack::rendered` is `active()` plus the layer being looked at,
and the rasteriser, the composer and the distance-field builder all index by
position in it. Which is also why the design's "two uniforms, no recompile" is
not available: a uniform can select a slot, it cannot conjure one.

The reveal is never on the graph. It reaches the pipeline as an argument to
`compose_revealing`, and `compose_for` — which the exporter, the thumbnail and
the neutral probe all call — has no way to ask for one. A flag on the graph
would have been shorter, would have type-checked, and would have been one
forgotten reset away from a red tint baked into an exported file.

And the tools that shape a mask now arm. `Masking.tool` is an `in` property
only Rust may write, and the handler wrote nothing back, so the strip reported
"Select" however many times Paint was pressed and the paint area was never
enabled — the brush, the parts and the whole of FR-DEV-19b reachable from no
control in the application.

The region overlay stands down while a mask is being shown, and its button now
says what it hides: two overlays that look alike and mean different things is
worse than either.
2026-09-10 20:28:06 +02:00
dtourolle 193b35a249 Start a category mask where the photograph can bear it
Clicking "architecture" made a layer whose mask was gone. Every category layer
began at STRICTNESS_DEFAULT, and that constant was fitted on the synthetic sky
the refine tests build — its own note warns that a real photograph's noise
"moves every crossing down together", which turns out to be a considerable
understatement. Measured over seven ordinary frames, half scale removes 76% to
99.5% of `architecture`, 36% to 93% of `ground` and 18% to 91% of
`vegetation`. Only sky, the category the number was calibrated against,
survives it.

An empty mask is indistinguishable from a broken one: the layer is listed, the
adjustment moves, and no pixel changes. So what this looks like from outside
is that the segmentation does not make masks at all.

No smaller constant fixes it either, because a nat of evidence means different
things over a smooth sky and over a stone facade — the useful position is
above 5 on one frame and below 1 on the next. So the frame is asked instead:
`Refinement::gentle` walks down from half scale and takes the first rung whose
gate removes no more than a sixth of the category's weight, and the model's
own outline when none of them does. One `apply` on a friendly photograph and
four on an unfriendly one, paid when a layer is made rather than for eight
categories nobody masked.

The slider's reset went to 4 as well, so taking the control back to its
"default" emptied the mask. It goes to zero now, which is the one position
documented to mean something: exactly what the model weighted.
2026-09-10 20:27:40 +02:00
59 changed files with 4650 additions and 538 deletions
Generated
+23 -23
View File
@@ -1221,7 +1221,7 @@ checksum = "f27ae1dd37df86211c42e150270f82743308803d90a6f6e6651cd730d5e1732f"
[[package]] [[package]]
name = "darkroom-android" name = "darkroom-android"
version = "0.11.0" version = "0.12.0"
dependencies = [ dependencies = [
"android_logger", "android_logger",
"dr-plat", "dr-plat",
@@ -1234,7 +1234,7 @@ dependencies = [
[[package]] [[package]]
name = "darkroom-desktop" name = "darkroom-desktop"
version = "0.11.0" version = "0.12.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"dr-plat", "dr-plat",
@@ -1407,7 +1407,7 @@ checksum = "d8b14ccef22fc6f5a8f4d7d768562a182c04ce9a3b3157b91390b52ddfdf1a76"
[[package]] [[package]]
name = "dr-bench" name = "dr-bench"
version = "0.11.0" version = "0.12.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"dr-catalog", "dr-catalog",
@@ -1424,7 +1424,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-catalog" name = "dr-catalog"
version = "0.11.0" version = "0.12.0"
dependencies = [ dependencies = [
"dr-face", "dr-face",
"dr-plat", "dr-plat",
@@ -1439,7 +1439,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-decode" name = "dr-decode"
version = "0.11.0" version = "0.12.0"
dependencies = [ dependencies = [
"dr-types", "dr-types",
"env_logger", "env_logger",
@@ -1453,7 +1453,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-export" name = "dr-export"
version = "0.11.0" version = "0.12.0"
dependencies = [ dependencies = [
"dr-decode", "dr-decode",
"dr-gpu", "dr-gpu",
@@ -1471,7 +1471,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-face" name = "dr-face"
version = "0.11.0" version = "0.12.0"
dependencies = [ dependencies = [
"env_logger", "env_logger",
"log", "log",
@@ -1484,7 +1484,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-film" name = "dr-film"
version = "0.11.0" version = "0.12.0"
dependencies = [ dependencies = [
"log", "log",
"serde", "serde",
@@ -1493,7 +1493,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-gpu" name = "dr-gpu"
version = "0.11.0" version = "0.12.0"
dependencies = [ dependencies = [
"bytemuck", "bytemuck",
"dr-decode", "dr-decode",
@@ -1510,7 +1510,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-ingest" name = "dr-ingest"
version = "0.11.0" version = "0.12.0"
dependencies = [ dependencies = [
"dr-plat", "dr-plat",
"dr-types", "dr-types",
@@ -1522,7 +1522,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-lens" name = "dr-lens"
version = "0.11.0" version = "0.12.0"
dependencies = [ dependencies = [
"lensfun", "lensfun",
"log", "log",
@@ -1530,7 +1530,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-pipeline" name = "dr-pipeline"
version = "0.11.0" version = "0.12.0"
dependencies = [ dependencies = [
"dr-types", "dr-types",
"log", "log",
@@ -1539,7 +1539,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-plat" name = "dr-plat"
version = "0.11.0" version = "0.12.0"
dependencies = [ dependencies = [
"android-native-keyring-store", "android-native-keyring-store",
"dr-types", "dr-types",
@@ -1555,7 +1555,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-preset-xmp" name = "dr-preset-xmp"
version = "0.11.0" version = "0.12.0"
dependencies = [ dependencies = [
"dr-pipeline", "dr-pipeline",
"log", "log",
@@ -1565,7 +1565,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-segment" name = "dr-segment"
version = "0.11.0" version = "0.12.0"
dependencies = [ dependencies = [
"env_logger", "env_logger",
"log", "log",
@@ -1578,7 +1578,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-sync" name = "dr-sync"
version = "0.11.0" version = "0.12.0"
dependencies = [ dependencies = [
"async-trait", "async-trait",
"dr-plat", "dr-plat",
@@ -1592,7 +1592,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-sync-folder" name = "dr-sync-folder"
version = "0.11.0" version = "0.12.0"
dependencies = [ dependencies = [
"async-trait", "async-trait",
"dr-sync", "dr-sync",
@@ -1604,7 +1604,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-sync-nextcloud" name = "dr-sync-nextcloud"
version = "0.11.0" version = "0.12.0"
dependencies = [ dependencies = [
"async-trait", "async-trait",
"dr-decode", "dr-decode",
@@ -1626,7 +1626,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-thumbs" name = "dr-thumbs"
version = "0.11.0" version = "0.12.0"
dependencies = [ dependencies = [
"dr-types", "dr-types",
"jpeg-encoder", "jpeg-encoder",
@@ -1638,7 +1638,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-types" name = "dr-types"
version = "0.11.0" version = "0.12.0"
dependencies = [ dependencies = [
"serde", "serde",
"serde_json", "serde_json",
@@ -1647,7 +1647,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-ui" name = "dr-ui"
version = "0.11.0" version = "0.12.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"async-trait", "async-trait",
@@ -1686,7 +1686,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-xmp" name = "dr-xmp"
version = "0.11.0" version = "0.12.0"
dependencies = [ dependencies = [
"dr-types", "dr-types",
"log", "log",
@@ -6987,7 +6987,7 @@ checksum = "8df9b6e13f2d32c91b9bd719c00d1958837bc7dec474d94952798cc8e69eeec3"
[[package]] [[package]]
name = "traceability" name = "traceability"
version = "0.11.0" version = "0.12.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"serde", "serde",
+1 -1
View File
@@ -27,7 +27,7 @@ members = [
] ]
[workspace.package] [workspace.package]
version = "0.11.0" version = "0.12.0"
edition = "2021" edition = "2021"
rust-version = "1.92" rust-version = "1.92"
license = "GPL-3.0-or-later" license = "GPL-3.0-or-later"
+14 -5
View File
@@ -269,11 +269,13 @@ fn android_main(app: slint::android::AndroidApp) {
/// thousand of them is an ANR by definition — the system puts "DarkRoom isn't /// thousand of them is an ANR by definition — the system puts "DarkRoom isn't
/// responding" over a window that has never painted, and offers to kill it. /// responding" over a window that has never painted, and offers to kill it.
/// ///
/// This copies **41 MB** on the first launch after an install: 24.9 MB of scene /// This copied **41 MB** on the first launch after an install: 24.9 MB of scene
/// model, 13.6 MB of embedder, 2.5 MB of detector, each read whole out of the /// model, 13.6 MB of embedder, 2.5 MB of detector, each read whole out of the
/// APK and written to `/data`. v0.10.0 added the scene model, which is 60% of /// APK and written to `/data`. v0.10.0 added the scene model, which was 60% of
/// that total; v0.10.0 is the release the ANR appeared in, and the 8,010 minor /// that total; v0.10.0 is the release the ANR appeared in, and the 8,010 minor
/// faults in its report are what 41 MB of freshly touched pages looks like. /// faults in its report are what 41 MB of freshly touched pages looks like.
/// The two further detectors the settings page offers since have made it
/// 61 MB, which is the same argument with a larger number.
/// ///
/// So it runs on a worker (NFR-ARCH-1: nothing blocking on the UI executor) and /// So it runs on a worker (NFR-ARCH-1: nothing blocking on the UI executor) and
/// this function returns as soon as the thread is running. Nothing on the /// this function returns as soon as the thread is running. Nothing on the
@@ -317,8 +319,15 @@ fn unpack_bundled_models(app: &slint::android::AndroidApp) {
// decodes to 150 anonymous channels — `library::scene_model` wants the // decodes to 150 anonymous channels — `library::scene_model` wants the
// vocabulary and the category descriptor beside it, and requires all three // vocabulary and the category descriptor beside it, and requires all three
// before it reports the tab available. // before it reports the tab available.
const BUNDLED: [(&std::ffi::CStr, &str); 5] = [ //
// Three detectors, because which one runs is a setting
// (`FaceDetector`, docs/faces.md §12.3) and a tablet has no other way to
// obtain the one it was not shipped with. Twenty megabytes of APK for
// the choice; the embedder is the same for all three.
const BUNDLED: [(&std::ffi::CStr, &str); 7] = [
(c"models/scrfd_500m_640.onnx", "scrfd_500m_640.onnx"), (c"models/scrfd_500m_640.onnx", "scrfd_500m_640.onnx"),
(c"models/scrfd_2.5g_640.onnx", "scrfd_2.5g_640.onnx"),
(c"models/scrfd_10g_640.onnx", "scrfd_10g_640.onnx"),
(c"models/arcface_mbf_b1.onnx", "arcface_mbf_b1.onnx"), (c"models/arcface_mbf_b1.onnx", "arcface_mbf_b1.onnx"),
(c"models/yolo26s-sem-ade20k.onnx", "yolo26s-sem-ade20k.onnx"), (c"models/yolo26s-sem-ade20k.onnx", "yolo26s-sem-ade20k.onnx"),
( (
@@ -334,8 +343,8 @@ fn unpack_bundled_models(app: &slint::android::AndroidApp) {
for (asset_path, name) in BUNDLED { for (asset_path, name) in BUNDLED {
let dest = dir.join(name); let dest = dir.join(name);
// Already unpacked. Not re-read on every launch: this is 41 MB of // Already unpacked. Not re-read on every launch: this is 61 MB of
// copying across the five entries, and the file does not change without // copying across the seven entries, and the file does not change without
// the APK changing, at which point the install wiped it anyway. It // the APK changing, at which point the install wiped it anyway. It
// matters more now than it did — a launch that skips every entry here // matters more now than it did — a launch that skips every entry here
// costs nothing at all, which is what makes the second launch after an // costs nothing at all, which is what makes the second launch after an
+23 -15
View File
@@ -34,6 +34,10 @@ struct Known {
crop_px: f32, crop_px: f32,
} }
/// What the catalog holds per face, decoded: photograph, vector, size,
/// quality.
type Decoded = (u64, Vec<f32>, f32, Option<f32>);
fn main() { fn main() {
let args: Vec<String> = std::env::args().skip(1).collect(); let args: Vec<String> = std::env::args().skip(1).collect();
let Some(path) = args.first() else { let Some(path) = args.first() else {
@@ -59,10 +63,10 @@ fn main() {
let model = dr_face::ModelId::new(MODEL_ID.to_string()); let model = dr_face::ModelId::new(MODEL_ID.to_string());
let stored = faces::embeddings(conn, MODEL_ID).expect("embeddings"); let stored = faces::embeddings(conn, MODEL_ID).expect("embeddings");
let mut embedding_of = HashMap::new(); let mut embedding_of: HashMap<faces::FaceId, Decoded> = HashMap::new();
for (id, image, blob, crop_px) in stored { for f in stored {
if let Some(e) = dr_face::Embedding::from_f16_bytes(model.clone(), &blob) { if let Some(e) = dr_face::Embedding::from_f16_bytes(model.clone(), &f.embedding) {
embedding_of.insert(id, (image.0, e.v.to_vec(), crop_px)); embedding_of.insert(f.face, (f.image.0, e.v.to_vec(), f.crop_px, f.quality));
} }
} }
println!("faces with embeddings: {}", embedding_of.len()); println!("faces with embeddings: {}", embedding_of.len());
@@ -82,7 +86,7 @@ fn main() {
if !f.confirmed { if !f.confirmed {
continue; continue;
} }
if let Some((image, embedding, crop_px)) = embedding_of.get(&f.id) { if let Some((image, embedding, crop_px, _)) = embedding_of.get(&f.id) {
mine.push(Known { mine.push(Known {
image: *image, image: *image,
person: p.id, person: p.id,
@@ -288,19 +292,22 @@ fn band(label: &str, v: &[f32]) {
/// The whole library through the real clusterer, for the numbers it would /// The whole library through the real clusterer, for the numbers it would
/// actually write. /// actually write.
fn full_library( fn full_library(
embedding_of: &HashMap<faces::FaceId, (u64, Vec<f32>, f32)>, embedding_of: &HashMap<faces::FaceId, Decoded>,
confirmed: &HashMap<faces::FaceId, u64>, confirmed: &HashMap<faces::FaceId, u64>,
cal: &dr_face::Calibration, cal: &dr_face::Calibration,
) { ) {
let mut candidates: Vec<dr_face::Candidate> = embedding_of let mut candidates: Vec<dr_face::Candidate> = embedding_of
.iter() .iter()
.map(|(id, (image, embedding, crop_px))| dr_face::Candidate { .map(
face: id.0, |(id, (image, embedding, crop_px, quality))| dr_face::Candidate {
image: *image, face: id.0,
embedding: embedding.clone(), image: *image,
crop_px: *crop_px, embedding: embedding.clone(),
confirmed_person: confirmed.get(id).copied(), crop_px: *crop_px,
}) quality: *quality,
confirmed_person: confirmed.get(id).copied(),
},
)
.collect(); .collect();
candidates.sort_by_key(|c| c.face); candidates.sort_by_key(|c| c.face);
@@ -317,11 +324,13 @@ fn full_library(
.collect(); .collect();
let crop_px: Vec<f32> = candidates.iter().map(|c| c.crop_px).collect(); let crop_px: Vec<f32> = candidates.iter().map(|c| c.crop_px).collect();
let images: Vec<u64> = candidates.iter().map(|c| c.image).collect(); let images: Vec<u64> = candidates.iter().map(|c| c.image).collect();
let gallery: Vec<bool> = candidates.iter().map(|c| c.in_gallery()).collect();
let view = dr_face::neighbours::Faces { let view = dr_face::neighbours::Faces {
embeddings: &flat, embeddings: &flat,
dim, dim,
crop_px: &crop_px, crop_px: &crop_px,
images: &images, images: &images,
gallery: &gallery,
}; };
let t = std::time::Instant::now(); let t = std::time::Instant::now();
@@ -335,8 +344,7 @@ fn full_library(
let agglomerate = t.elapsed().as_secs_f64() - scan; let agglomerate = t.elapsed().as_secs_f64() - scan;
let t = std::time::Instant::now(); let t = std::time::Instant::now();
let _ = let _ = dr_face::identity_shares(&gallery, &clusters, &evidence, dr_face::TOP_MATCHES);
dr_face::identity_shares(candidates.len(), &clusters, &evidence, dr_face::TOP_MATCHES);
println!( println!(
" scan {scan:.2}s ({} evidence pairs) · agglomerate {agglomerate:.2}s · score {:.2}s", " scan {scan:.2}s ({} evidence pairs) · agglomerate {agglomerate:.2}s · score {:.2}s",
evidence.len(), evidence.len(),
+89 -17
View File
@@ -76,6 +76,9 @@ pub struct SharedFace {
pub confidence: f32, pub confidence: f32,
pub embedding: Vec<u8>, pub embedding: Vec<u8>,
pub crop_px: f32, pub crop_px: f32,
/// See `faces::DetectedFace::quality`. `None` from a shard written before
/// the number was kept.
pub quality: Option<f32>,
/// The face cut out and encoded, or empty where none was kept. /// The face cut out and encoded, or empty where none was kept.
/// ///
/// Travels with the face rather than in the catalog snapshot, which is the /// Travels with the face rather than in the catalog snapshot, which is the
@@ -224,8 +227,8 @@ impl FaceShardStore {
tx.execute( tx.execute(
"INSERT INTO faces "INSERT INTO faces
(file_id, model_id, x, y, w, h, landmarks, confidence, (file_id, model_id, x, y, w, h, landmarks, confidence,
embedding, crop_px, crop) embedding, crop_px, crop, quality)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11)", VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12)",
rusqlite::params![ rusqlite::params![
f.file_id as i64, f.file_id as i64,
f.model_id, f.model_id,
@@ -238,6 +241,7 @@ impl FaceShardStore {
f.embedding, f.embedding,
f.crop_px as f64, f.crop_px as f64,
(!f.crop.is_empty()).then_some(f.crop.as_slice()), (!f.crop.is_empty()).then_some(f.crop.as_slice()),
f.quality.map(f64::from),
], ],
)?; )?;
} }
@@ -442,9 +446,10 @@ impl FaceShardStore {
} }
let mut fq = src.prepare(&format!( let mut fq = src.prepare(&format!(
"SELECT f.file_id, f.model_id, f.x, f.y, f.w, f.h, f.landmarks, "SELECT f.file_id, f.model_id, f.x, f.y, f.w, f.h, f.landmarks,
f.confidence, f.embedding, f.crop_px, {} f.confidence, f.embedding, f.crop_px, {}, {}
FROM faces f WHERE f.file_id = ?1 AND f.model_id = ?2", FROM faces f WHERE f.file_id = ?1 AND f.model_id = ?2",
crop_column(&src) column_or_null(&src, "crop"),
column_or_null(&src, "quality"),
))?; ))?;
let faces: Vec<SharedFace> = fq let faces: Vec<SharedFace> = fq
.query_map(rusqlite::params![file_id, &model_id], read_shared_face)? .query_map(rusqlite::params![file_id, &model_id], read_shared_face)?
@@ -484,7 +489,8 @@ impl FaceShardStore {
let Some(edge) = edge else { return Ok(None) }; let Some(edge) = edge else { return Ok(None) };
let mut q = conn.prepare( let mut q = conn.prepare(
"SELECT file_id, model_id, x, y, w, h, landmarks, confidence, embedding, crop_px, crop "SELECT file_id, model_id, x, y, w, h, landmarks, confidence, embedding, crop_px,
crop, quality
FROM faces WHERE file_id = ?1 AND model_id = ?2", FROM faces WHERE file_id = ?1 AND model_id = ?2",
)?; )?;
let faces: Vec<SharedFace> = q let faces: Vec<SharedFace> = q
@@ -541,6 +547,7 @@ fn upgrade_shard(conn: &Connection) -> Result<(), CatalogError> {
for (table, column, decl) in [ for (table, column, decl) in [
("faces", "crop", "BLOB"), ("faces", "crop", "BLOB"),
("indexed", "indexed_at", "INTEGER"), ("indexed", "indexed_at", "INTEGER"),
("faces", "quality", "REAL"),
] { ] {
if !has_column(conn, table, column)? { if !has_column(conn, table, column)? {
conn.execute_batch(&format!("ALTER TABLE {table} ADD COLUMN {column} {decl}"))?; conn.execute_batch(&format!("ALTER TABLE {table} ADD COLUMN {column} {decl}"))?;
@@ -555,16 +562,19 @@ fn has_column(conn: &Connection, table: &str, column: &str) -> Result<bool, Cata
Ok(stmt.exists(rusqlite::params![table, column])?) Ok(stmt.exists(rusqlite::params![table, column])?)
} }
/// `f.crop`, or a `NULL` standing in for it. /// `f.<column>`, or a `NULL` standing in for it.
/// ///
/// A shard downloaded from a peer is opened **read-only** and cannot be /// A shard downloaded from a peer is opened **read-only** and cannot be
/// upgraded, so one written before crops existed has to be read as it is rather /// upgraded, so one written before a column existed has to be read as it is
/// than repaired. Selecting a literal keeps the column count the same, which is /// rather than repaired. Selecting a literal keeps the column count the same,
/// what lets [`read_shared_face`] stay a single function. /// which is what lets [`read_shared_face`] stay a single function.
fn crop_column(conn: &Connection) -> &'static str { ///
match has_column(conn, "faces", "crop") { /// `column` is one of this module's own names, never anything read from
Ok(true) => "f.crop", /// outside, which is what makes formatting it into SQL acceptable.
_ => "NULL", fn column_or_null(conn: &Connection, column: &'static str) -> String {
match has_column(conn, "faces", column) {
Ok(true) => format!("f.{column}"),
_ => "NULL".to_string(),
} }
} }
@@ -637,7 +647,8 @@ pub fn export_to_shards_reporting(
continue; continue;
} }
let mut fq = conn.prepare( let mut fq = conn.prepare(
"SELECT x, y, w, h, landmarks, detector_confidence, embedding, crop_px, crop "SELECT x, y, w, h, landmarks, detector_confidence, embedding, crop_px, crop,
quality
FROM faces WHERE image_id = ?1 AND model_id = ?2", FROM faces WHERE image_id = ?1 AND model_id = ?2",
)?; )?;
let faces: Vec<SharedFace> = fq let faces: Vec<SharedFace> = fq
@@ -654,6 +665,7 @@ pub fn export_to_shards_reporting(
embedding: r.get(6)?, embedding: r.get(6)?,
crop_px: r.get::<_, f64>(7)? as f32, crop_px: r.get::<_, f64>(7)? as f32,
crop: r.get::<_, Option<Vec<u8>>>(8)?.unwrap_or_default(), crop: r.get::<_, Option<Vec<u8>>>(8)?.unwrap_or_default(),
quality: r.get::<_, Option<f64>>(9)?.map(|q| q as f32),
}) })
})? })?
.collect::<Result<_, _>>()?; .collect::<Result<_, _>>()?;
@@ -710,6 +722,14 @@ pub fn import_from_shards(
let Some((faces, edge)) = store.get_image(file_id as u64, model_id)? else { let Some((faces, edge)) = store.get_image(file_id as u64, model_id)? else {
continue; continue;
}; };
// A peer that embedded before the quality was kept has done work this
// device cannot finish: the number exists only at embedding time, and
// adopting the faces would write the run marker that keeps them from
// ever being measured (schema V14). Left for this device's own pass —
// or for the peer's, whose re-export replaces these.
if faces.iter().any(|f| f.quality.is_none()) {
continue;
}
let local: Vec<crate::faces::DetectedFace> = faces let local: Vec<crate::faces::DetectedFace> = faces
.into_iter() .into_iter()
.map(|f| crate::faces::DetectedFace { .map(|f| crate::faces::DetectedFace {
@@ -721,6 +741,7 @@ pub fn import_from_shards(
confidence: f.confidence, confidence: f.confidence,
embedding: f.embedding, embedding: f.embedding,
crop_px: f.crop_px, crop_px: f.crop_px,
quality: f.quality,
model_id: f.model_id, model_id: f.model_id,
// A peer that indexed before crops existed sends none, and the // A peer that indexed before crops existed sends none, and the
// reader falls back to the proxy exactly as it does for a face // reader falls back to the proxy exactly as it does for a face
@@ -766,6 +787,7 @@ fn read_shared_face(r: &rusqlite::Row<'_>) -> rusqlite::Result<SharedFace> {
embedding: r.get(8)?, embedding: r.get(8)?,
crop_px: r.get::<_, f64>(9)? as f32, crop_px: r.get::<_, f64>(9)? as f32,
crop: r.get::<_, Option<Vec<u8>>>(10)?.unwrap_or_default(), crop: r.get::<_, Option<Vec<u8>>>(10)?.unwrap_or_default(),
quality: r.get::<_, Option<f64>>(11)?.map(|q| q as f32),
}) })
} }
@@ -849,7 +871,11 @@ CREATE TABLE IF NOT EXISTS faces (
crop_px REAL NOT NULL, crop_px REAL NOT NULL,
-- The face, cut out. NULL where the face was found before crops were kept, -- The face, cut out. NULL where the face was found before crops were kept,
-- or adopted from a peer that did not have one. -- or adopted from a peer that did not have one.
crop BLOB crop BLOB,
-- Length of the raw embedding (`faces::DetectedFace::quality`). NULL from
-- a build that did not keep it, and a face the receiving device will not
-- adopt -- see `import_from_shards`.
quality REAL
); );
CREATE INDEX IF NOT EXISTS faces_file ON faces(file_id, model_id); CREATE INDEX IF NOT EXISTS faces_file ON faces(file_id, model_id);
@@ -908,6 +934,7 @@ mod tests {
confidence: 0.87, confidence: 0.87,
embedding: vec![seed; 1024], embedding: vec![seed; 1024],
crop_px: 180.0, crop_px: 180.0,
quality: Some(17.5),
crop: vec![seed; 64], crop: vec![seed; 64],
} }
} }
@@ -925,6 +952,7 @@ mod tests {
assert_eq!(edge, 1024); assert_eq!(edge, 1024);
assert_eq!(faces[0].embedding.len(), 1024); assert_eq!(faces[0].embedding.len(), 1024);
assert!((faces[0].crop_px - 180.0).abs() < 1e-3); assert!((faces[0].crop_px - 180.0).abs() < 1e-3);
assert_eq!(faces[0].quality, Some(17.5));
} }
/// The case the run marker exists for, carried across the wire: an image /// The case the run marker exists for, carried across the wire: an image
@@ -1115,6 +1143,7 @@ mod catalog_round_trip {
confidence: 0.9, confidence: 0.9,
embedding: vec![seed; 1024], embedding: vec![seed; 1024],
crop_px: 180.0, crop_px: 180.0,
quality: Some(20.0),
model_id: "w600k_mbf".into(), model_id: "w600k_mbf".into(),
crop: vec![seed; 64], crop: vec![seed; 64],
} }
@@ -1162,9 +1191,47 @@ mod catalog_round_trip {
let got = faces::for_image(&b, dr_types::ImageId(90)).unwrap(); let got = faces::for_image(&b, dr_types::ImageId(90)).unwrap();
assert_eq!(got.len(), 1); assert_eq!(got.len(), 1);
assert!((got[0].crop_px - 180.0).abs() < 1e-3); assert!((got[0].crop_px - 180.0).abs() < 1e-3);
assert_eq!(got[0].quality, Some(20.0));
assert!((got[0].landmarks[2].0 - 0.15).abs() < 1e-5); assert!((got[0].landmarks[2].0 - 0.15).abs() < 1e-5);
let emb = faces::embeddings(&b, "w600k_mbf").unwrap(); let emb = faces::embeddings(&b, "w600k_mbf").unwrap();
assert!(emb.iter().any(|(_, _, blob, _)| blob[0] == 1)); assert!(emb.iter().any(|e| e.embedding[0] == 1));
}
/// A face a peer embedded without measuring it is work this device
/// cannot finish, and adopting it would write the marker that stops it
/// ever being measured. The image stays outstanding instead.
#[test]
fn a_peers_unmeasured_faces_are_left_for_this_device_to_index() {
let b = device(&[(90, 5001), (91, 5002)]);
let mut store = FaceShardStore::open(&tempdir("unmeasured")).unwrap();
let shared = |file_id: u64, quality: Option<f32>| SharedFace {
file_id,
model_id: "w600k_mbf".into(),
x: 0.1,
y: 0.2,
w: 0.15,
h: 0.2,
landmarks: vec![1; 40],
confidence: 0.87,
embedding: vec![1; 1024],
crop_px: 180.0,
quality,
crop: Vec::new(),
};
store
.put_image(5001, "w600k_mbf", 2560, &[shared(5001, None)])
.unwrap();
store
.put_image(5002, "w600k_mbf", 2560, &[shared(5002, Some(19.0))])
.unwrap();
assert_eq!(import_from_shards(&b, &store, "w600k_mbf").unwrap(), 1);
let cov = faces::coverage(&b, "w600k_mbf").unwrap();
assert_eq!(cov.indexed, 1);
assert_eq!(cov.outstanding(), 1, "the unmeasured image was adopted");
assert!(faces::for_image(&b, dr_types::ImageId(90))
.unwrap()
.is_empty());
} }
#[test] #[test]
@@ -1187,7 +1254,7 @@ mod catalog_round_trip {
"a peer's copy replaced work this device had already done" "a peer's copy replaced work this device had already done"
); );
let emb = faces::embeddings(&b, "w600k_mbf").unwrap(); let emb = faces::embeddings(&b, "w600k_mbf").unwrap();
assert_eq!(emb[0].2[0], 9, "B's own embedding was overwritten"); assert_eq!(emb[0].embedding[0], 9, "B's own embedding was overwritten");
} }
/// A device holding a subset of the library takes only its own part. /// A device holding a subset of the library takes only its own part.
@@ -1281,6 +1348,7 @@ mod catalog_round_trip {
confidence: 0.87, confidence: 0.87,
embedding: vec![seed; 1024], embedding: vec![seed; 1024],
crop_px: 180.0, crop_px: 180.0,
quality: None,
crop: vec![seed; 64], crop: vec![seed; 64],
} }
} }
@@ -1430,6 +1498,10 @@ mod catalog_round_trip {
let (faces, _) = store.get_image(77, "w600k_mbf").unwrap().unwrap(); let (faces, _) = store.get_image(77, "w600k_mbf").unwrap().unwrap();
assert_eq!(faces.len(), 1); assert_eq!(faces.len(), 1);
assert!(faces[0].crop.is_empty(), "a crop was invented from nowhere"); assert!(faces[0].crop.is_empty(), "a crop was invented from nowhere");
assert_eq!(
faces[0].quality, None,
"a quality was invented from nowhere"
);
let _ = std::fs::remove_dir_all(&dir); let _ = std::fs::remove_dir_all(&dir);
} }
+256 -19
View File
@@ -61,10 +61,22 @@ pub struct DetectedFace {
/// Five `(x, y)` pairs, normalised the same way. /// Five `(x, y)` pairs, normalised the same way.
pub landmarks: [(f32, f32); 5], pub landmarks: [(f32, f32); 5],
pub confidence: f32, pub confidence: f32,
/// 512 × f16, L2-normalised — `dr_face::Embedding::to_f16_bytes`. /// 512 × f16, the raw model output — `dr_face::Embedded::to_f16_bytes`.
///
/// Raw rather than unit length, so the length ([`Self::quality`]) is in
/// the blob and not only beside it. Readers re-normalise on load.
pub embedding: Vec<u8>, pub embedding: Vec<u8>,
/// Source pixels across the aligned crop (docs/faces.md §7). /// Source pixels across the aligned crop (docs/faces.md §7).
pub crop_px: f32, pub crop_px: f32,
/// Length of the raw embedding before normalisation — the model's own
/// reading of how recognisable the crop was, and the gate on whether
/// this face may be compared *against* (`dr_face::MIN_GALLERY_QUALITY`).
///
/// `None` where it was never measured: a face indexed, here or by a peer,
/// before raw vectors were stored. The unit vector those builds kept has
/// no length left to read, so the only way to measure one is to embed it
/// again (schema V14).
pub quality: Option<f32>,
/// Which model produced the embedding. Comparing across models is the one /// Which model produced the embedding. Comparing across models is the one
/// mistake that yields plausible garbage rather than an error. /// mistake that yields plausible garbage rather than an error.
pub model_id: String, pub model_id: String,
@@ -94,6 +106,9 @@ pub struct Face {
pub landmarks: [(f32, f32); 5], pub landmarks: [(f32, f32); 5],
pub confidence: f32, pub confidence: f32,
pub crop_px: f32, pub crop_px: f32,
/// See [`DetectedFace::quality`]. `None` for a face indexed before it was
/// recorded.
pub quality: Option<f32>,
pub model_id: String, pub model_id: String,
/// `None` when the face belongs to no one yet. /// `None` when the face belongs to no one yet.
pub person: Option<PersonId>, pub person: Option<PersonId>,
@@ -143,6 +158,16 @@ pub use dr_face::Calibration;
/// part of the frame: a re-index with a better model must not discard the /// part of the frame: a re-index with a better model must not discard the
/// user's labelling (FR-CULL-10). Matching is by box overlap, since the face is /// user's labelling (FR-CULL-10). Matching is by box overlap, since the face is
/// in the same place even when the box moves a little. /// in the same place even when the box moves a little.
///
/// **Every model's faces are replaced, and every other model's marker goes
/// with them.** An image holds the faces of whichever pipeline looked at it
/// last, never a mixture — two detectors drawing boxes over the same face is
/// not two opinions but a duplicate. So the replacement is unconditional on
/// `model_id`, and the run markers of the pipelines whose faces were just
/// removed are dropped too: a marker that says "done" over an image with
/// none of that model's faces is exactly the state that made the V12 repair
/// necessary, and a user who switches their detector back would otherwise
/// find those photographs permanently empty.
pub fn record_detections( pub fn record_detections(
conn: &Connection, conn: &Connection,
image_id: ImageId, image_id: ImageId,
@@ -177,6 +202,10 @@ pub fn record_detections(
} }
tx.execute("DELETE FROM faces WHERE image_id = ?1", [image_id.0 as i64])?; tx.execute("DELETE FROM faces WHERE image_id = ?1", [image_id.0 as i64])?;
tx.execute(
"DELETE FROM face_index WHERE image_id = ?1 AND model_id != ?2",
rusqlite::params![image_id.0 as i64, model_id],
)?;
let now = now_secs(); let now = now_secs();
let mut ids = Vec::with_capacity(faces.len()); let mut ids = Vec::with_capacity(faces.len());
@@ -184,8 +213,8 @@ pub fn record_detections(
tx.execute( tx.execute(
"INSERT INTO faces "INSERT INTO faces
(image_id, x, y, w, h, landmarks, detector_confidence, (image_id, x, y, w, h, landmarks, detector_confidence,
embedding, crop_px, model_id, detected_at, crop) embedding, crop_px, model_id, detected_at, crop, quality)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12)", VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, ?13)",
rusqlite::params![ rusqlite::params![
image_id.0 as i64, image_id.0 as i64,
f.x as f64, f.x as f64,
@@ -202,6 +231,7 @@ pub fn record_detections(
// the database rather than two the readers each have to know // the database rather than two the readers each have to know
// about. // about.
(!f.crop.is_empty()).then_some(f.crop.as_slice()), (!f.crop.is_empty()).then_some(f.crop.as_slice()),
f.quality.map(f64::from),
], ],
)?; )?;
let id = FaceId(tx.last_insert_rowid() as u64); let id = FaceId(tx.last_insert_rowid() as u64);
@@ -251,6 +281,107 @@ pub fn record_detections(
Ok(ids) Ok(ids)
} }
/// A face embedded again from its stored landmarks: the new vector and its
/// length. What the measuring pass hands back per face.
#[derive(Debug, Clone, PartialEq)]
pub struct Measurement {
pub face: FaceId,
/// 512 × f16, raw — `dr_face::Embedded::to_f16_bytes`.
pub embedding: Vec<u8>,
pub quality: f32,
}
/// Write fresh embeddings over faces that were found before their quality was
/// kept, and re-mark the image as indexed.
///
/// The cheaper half of what `record_detections` does, for the case schema V14
/// created: the boxes and landmarks are right, the identities are the user's
/// work, and only the vector needs doing again. Updating in place is what
/// keeps `face_person` and the face ids exactly as they were -- a
/// re-detection would carry confirmations across by box overlap and lose
/// every suggestion, for no gain.
///
/// `dropped` are faces whose landmarks turned out to be degenerate -- the warp
/// could not be built from them. Deleted here, as detection would have refused
/// to store them (`dr_ui::faces::index_proxy`), and because a face left with
/// no reading would put its image back on the measuring pass's list on every
/// sweep, at the cost of an original each time.
///
/// The run marker is re-written with a fresh time, and that is not
/// bookkeeping: `face_shard::export_to_shards` re-exports an image whose
/// marker is newer than the store's copy, which is how the measured vectors
/// reach the other devices.
pub fn record_measurements(
conn: &Connection,
image_id: ImageId,
model_id: &str,
source_edge: u32,
measured: &[Measurement],
dropped: &[FaceId],
) -> Result<(), CatalogError> {
let tx = conn.unchecked_transaction()?;
for m in measured {
tx.execute(
"UPDATE faces SET embedding = ?2, quality = ?3 WHERE id = ?1",
rusqlite::params![m.face.0 as i64, m.embedding, f64::from(m.quality)],
)?;
}
for f in dropped {
tx.execute("DELETE FROM faces WHERE id = ?1", [f.0 as i64])?;
}
let remaining: i64 = tx.query_row(
"SELECT COUNT(*) FROM faces WHERE image_id = ?1 AND model_id = ?2",
rusqlite::params![image_id.0 as i64, model_id],
|r| r.get(0),
)?;
tx.execute(
"INSERT INTO face_index (image_id, model_id, indexed_at, faces_found, source_edge)
VALUES (?1, ?2, ?3, ?4, ?5)
ON CONFLICT(image_id, model_id) DO UPDATE SET
indexed_at = excluded.indexed_at,
faces_found = excluded.faces_found,
source_edge = excluded.source_edge",
rusqlite::params![
image_id.0 as i64,
model_id,
now_secs(),
remaining,
source_edge as i64,
],
)?;
tx.commit()?;
Ok(())
}
/// The faces on one image that have no quality reading yet.
///
/// The measuring pass's per-image work: every face this model found whose
/// vector was stored as a unit one (schema V14), with the landmarks the
/// warp is rebuilt from.
pub fn unmeasured_on_image(
conn: &Connection,
image_id: ImageId,
model_id: &str,
) -> Result<Vec<Face>, CatalogError> {
Ok(for_image(conn, image_id)?
.into_iter()
.filter(|f| f.model_id == model_id && f.quality.is_none())
.collect())
}
/// How many of a model's faces have no quality reading.
///
/// What the measuring pass has left to do, for a screen that wants to say so.
pub fn faces_unmeasured(conn: &Connection, model_id: &str) -> Result<u64, CatalogError> {
conn.query_row(
"SELECT COUNT(*) FROM faces WHERE model_id = ?1 AND quality IS NULL",
[model_id],
|r| r.get::<_, i64>(0),
)
.map(|n| n as u64)
.map_err(Into::into)
}
/// How much of the library has been through face detection. /// How much of the library has been through face detection.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] #[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub struct Coverage { pub struct Coverage {
@@ -366,7 +497,7 @@ pub fn for_image(conn: &Connection, image_id: ImageId) -> Result<Vec<Face>, Cata
let mut q = conn.prepare( let mut q = conn.prepare(
"SELECT f.id, f.image_id, f.x, f.y, f.w, f.h, f.landmarks, "SELECT f.id, f.image_id, f.x, f.y, f.w, f.h, f.landmarks,
f.detector_confidence, f.crop_px, f.model_id, f.detector_confidence, f.crop_px, f.model_id,
fp.person_id, fp.probability, fp.confirmed fp.person_id, fp.probability, fp.confirmed, f.quality
FROM faces f FROM faces f
LEFT JOIN face_person fp ON fp.face_id = f.id LEFT JOIN face_person fp ON fp.face_id = f.id
WHERE f.image_id = ?1 WHERE f.image_id = ?1
@@ -393,9 +524,18 @@ pub fn unassigned(conn: &Connection, model_id: &str) -> Result<Vec<FaceId>, Cata
/// One face's stored embedding, as the clustering pass consumes it. /// One face's stored embedding, as the clustering pass consumes it.
/// ///
/// A named type rather than a tuple because it crosses a crate boundary and /// A struct rather than a tuple because it crosses a crate boundary and "the
/// "the third element" is not a thing anyone should have to remember. /// fourth element" is not a thing anyone should have to remember.
pub type StoredEmbedding = (FaceId, ImageId, Vec<u8>, f32); #[derive(Debug, Clone, PartialEq)]
pub struct StoredEmbedding {
pub face: FaceId,
pub image: ImageId,
/// 512 × f16 — `dr_face::Embedding::from_f16_bytes` reads it.
pub embedding: Vec<u8>,
pub crop_px: f32,
/// See [`DetectedFace::quality`].
pub quality: Option<f32>,
}
/// Embeddings for clustering, oldest first so the pass is deterministic. /// Embeddings for clustering, oldest first so the pass is deterministic.
/// ///
@@ -404,16 +544,17 @@ pub type StoredEmbedding = (FaceId, ImageId, Vec<u8>, f32);
/// would double the memory of the one operation that holds them all at once. /// would double the memory of the one operation that holds them all at once.
pub fn embeddings(conn: &Connection, model_id: &str) -> Result<Vec<StoredEmbedding>, CatalogError> { pub fn embeddings(conn: &Connection, model_id: &str) -> Result<Vec<StoredEmbedding>, CatalogError> {
let mut q = conn.prepare( let mut q = conn.prepare(
"SELECT id, image_id, embedding, crop_px FROM faces "SELECT id, image_id, embedding, crop_px, quality FROM faces
WHERE model_id = ?1 ORDER BY id", WHERE model_id = ?1 ORDER BY id",
)?; )?;
let rows = q.query_map([model_id], |r| { let rows = q.query_map([model_id], |r| {
Ok(( Ok(StoredEmbedding {
FaceId(r.get::<_, i64>(0)? as u64), face: FaceId(r.get::<_, i64>(0)? as u64),
ImageId(r.get::<_, i64>(1)? as u64), image: ImageId(r.get::<_, i64>(1)? as u64),
r.get::<_, Vec<u8>>(2)?, embedding: r.get::<_, Vec<u8>>(2)?,
r.get::<_, f64>(3)? as f32, crop_px: r.get::<_, f64>(3)? as f32,
)) quality: r.get::<_, Option<f64>>(4)?.map(|q| q as f32),
})
})?; })?;
rows.collect::<Result<_, _>>().map_err(Into::into) rows.collect::<Result<_, _>>().map_err(Into::into)
} }
@@ -643,7 +784,7 @@ pub fn for_person(
let mut q = conn.prepare( let mut q = conn.prepare(
"SELECT f.id, f.image_id, f.x, f.y, f.w, f.h, f.landmarks, "SELECT f.id, f.image_id, f.x, f.y, f.w, f.h, f.landmarks,
f.detector_confidence, f.crop_px, f.model_id, f.detector_confidence, f.crop_px, f.model_id,
fp.person_id, fp.probability, fp.confirmed fp.person_id, fp.probability, fp.confirmed, f.quality
FROM faces f FROM faces f
JOIN face_person fp ON fp.face_id = f.id JOIN face_person fp ON fp.face_id = f.id
WHERE fp.person_id = ?1 AND (?2 OR fp.confirmed = 1) WHERE fp.person_id = ?1 AND (?2 OR fp.confirmed = 1)
@@ -894,6 +1035,7 @@ fn read_face(r: &rusqlite::Row<'_>) -> rusqlite::Result<Face> {
landmarks: blob_to_landmarks(&r.get::<_, Vec<u8>>(6)?), landmarks: blob_to_landmarks(&r.get::<_, Vec<u8>>(6)?),
confidence: r.get::<_, f64>(7)? as f32, confidence: r.get::<_, f64>(7)? as f32,
crop_px: r.get::<_, f64>(8)? as f32, crop_px: r.get::<_, f64>(8)? as f32,
quality: r.get::<_, Option<f64>>(13)?.map(|q| q as f32),
model_id: r.get(9)?, model_id: r.get(9)?,
person: person.map(|p| PersonId(p as u64)), person: person.map(|p| PersonId(p as u64)),
probability: r.get::<_, Option<f64>>(11)?.unwrap_or(0.0) as f32, probability: r.get::<_, Option<f64>>(11)?.unwrap_or(0.0) as f32,
@@ -1016,6 +1158,7 @@ mod tests {
confidence: 0.9, confidence: 0.9,
embedding: vec![seed; 1024], embedding: vec![seed; 1024],
crop_px: 180.0, crop_px: 180.0,
quality: Some(10.0 + f32::from(seed)),
model_id: "w600k_mbf".into(), model_id: "w600k_mbf".into(),
crop: Vec::new(), crop: Vec::new(),
} }
@@ -1041,6 +1184,80 @@ mod tests {
assert!((got[0].crop_px - 180.0).abs() < 1e-3); assert!((got[0].crop_px - 180.0).abs() < 1e-3);
assert!((got[0].landmarks[2].1 - 0.2).abs() < 1e-5); assert!((got[0].landmarks[2].1 - 0.2).abs() < 1e-5);
assert!(got[0].person.is_none()); assert!(got[0].person.is_none());
// Both readers carry the quality, and the one for the grouping pass
// carries it as the option it is.
let mut qualities: Vec<Option<f32>> = got.iter().map(|f| f.quality).collect();
qualities.sort_by(|a, b| a.partial_cmp(b).unwrap());
assert_eq!(qualities, vec![Some(11.0), Some(12.0)]);
let stored = embeddings(&c, "w600k_mbf").unwrap();
assert_eq!(stored.len(), 2);
assert_eq!(stored[0].quality, Some(11.0), "oldest first");
assert_eq!(stored[1].quality, Some(12.0));
}
/// The measuring pass writes over the vector and nothing else: the face
/// keeps its id, its box and whoever the user said it was.
#[test]
fn measuring_replaces_the_vector_and_keeps_the_identity() {
let c = db();
let img = image(&c, 1);
let unmeasured = DetectedFace {
quality: None,
..face(1)
};
let ids = record_detections(
&c,
img,
"w600k_mbf",
1024,
&[unmeasured.clone(), unmeasured],
)
.unwrap();
let person = create_person(&c, "Anna").unwrap();
confirm(&c, ids[0], person).unwrap();
assert_eq!(faces_unmeasured(&c, "w600k_mbf").unwrap(), 2);
assert_eq!(unmeasured_on_image(&c, img, "w600k_mbf").unwrap().len(), 2);
let marked_at: i64 = c
.query_row("SELECT indexed_at FROM face_index", [], |r| r.get(0))
.unwrap();
c.execute("UPDATE face_index SET indexed_at = indexed_at - 100", [])
.unwrap();
record_measurements(
&c,
img,
"w600k_mbf",
6000,
&[Measurement {
face: ids[0],
embedding: vec![9; 1024],
quality: 21.5,
}],
&[ids[1]],
)
.unwrap();
let got = for_image(&c, img).unwrap();
assert_eq!(got.len(), 1, "the degenerate face was kept");
assert_eq!(got[0].id, ids[0]);
assert_eq!(got[0].quality, Some(21.5));
assert_eq!(got[0].person, Some(person));
assert!(got[0].confirmed);
let e = embeddings(&c, "w600k_mbf").unwrap();
assert_eq!(e[0].embedding[0], 9);
assert_eq!(faces_unmeasured(&c, "w600k_mbf").unwrap(), 0);
// The marker says one face at the native edge, and is fresh — which
// is what makes the sync export it again.
let (found, edge, at): (i64, i64, i64) = c
.query_row(
"SELECT faces_found, source_edge, indexed_at FROM face_index",
[],
|r| Ok((r.get(0)?, r.get(1)?, r.get(2)?)),
)
.unwrap();
assert_eq!((found, edge), (1, 6000));
assert!(at >= marked_at, "the marker was not refreshed");
} }
/// Re-detection is coalesced per image, so it must replace rather than /// Re-detection is coalesced per image, so it must replace rather than
@@ -1405,6 +1622,25 @@ mod tests {
assert_eq!(edge, 2048, "the marker should record the newer proxy"); assert_eq!(edge, 2048, "the marker should record the newer proxy");
} }
/// A user who switches detector and back must not find the images the
/// second pipeline visited reported as done under the first with no
/// faces behind the marker.
#[test]
fn re_indexing_under_another_model_drops_the_first_models_marker() {
let c = db();
let img = image(&c, 1);
record_detections(&c, img, "w600k_mbf", 2048, &[face(1)]).unwrap();
record_detections(&c, img, "scrfd_2.5g+w600k_mbf", 2048, &[face(2), face(3)]).unwrap();
assert!(is_indexed(&c, img, "scrfd_2.5g+w600k_mbf").unwrap());
assert!(
!is_indexed(&c, img, "w600k_mbf").unwrap(),
"the first pipeline's marker outlived its faces"
);
assert_eq!(for_image(&c, img).unwrap().len(), 2);
assert_eq!(coverage(&c, "w600k_mbf").unwrap().outstanding(), 1);
}
#[test] #[test]
fn clearing_a_marker_puts_that_image_back_in_the_queue() { fn clearing_a_marker_puts_that_image_back_in_the_queue() {
let c = db(); let c = db();
@@ -1450,10 +1686,11 @@ mod tests {
record_detections(&c, img, "w600k_mbf", 1024, &[face(7)]).unwrap(); record_detections(&c, img, "w600k_mbf", 1024, &[face(7)]).unwrap();
let e = embeddings(&c, "w600k_mbf").unwrap(); let e = embeddings(&c, "w600k_mbf").unwrap();
assert_eq!(e.len(), 1); assert_eq!(e.len(), 1);
assert_eq!(e[0].1, img); assert_eq!(e[0].image, img);
assert_eq!(e[0].2.len(), 1024); assert_eq!(e[0].embedding.len(), 1024);
assert_eq!(e[0].2[0], 7); assert_eq!(e[0].embedding[0], 7);
assert!((e[0].3 - 180.0).abs() < 1e-3); assert!((e[0].crop_px - 180.0).abs() < 1e-3);
assert_eq!(e[0].quality, Some(17.0));
} }
// ── stored crops ────────────────────────────────────────────────────── // ── stored crops ──────────────────────────────────────────────────────
+1 -1
View File
@@ -61,7 +61,7 @@ pub use collections::{Collection, CollectionKind, TreeRow};
pub use dedup::{seen_by_content, seen_by_metadata, set_content_hash}; pub use dedup::{seen_by_content, seen_by_metadata, set_content_hash};
pub use error::CatalogError; pub use error::CatalogError;
pub use face_shard::{FaceShardStore, SharedFace}; pub use face_shard::{FaceShardStore, SharedFace};
pub use faces::{Calibration, DetectedFace, Face, FaceId, Person, PersonId}; pub use faces::{Calibration, DetectedFace, Face, FaceId, Measurement, Person, PersonId};
pub use jobs::{Job, JobKind, Priority}; pub use jobs::{Job, JobKind, Priority};
pub use keywords::{Coverage, Keyword, KeywordId, SelectionKeyword}; pub use keywords::{Coverage, Keyword, KeywordId, SelectionKeyword};
pub use merge::MergeReport; pub use merge::MergeReport;
+134 -3
View File
@@ -15,7 +15,7 @@ use rusqlite::Connection;
use crate::error::CatalogError; use crate::error::CatalogError;
/// Schema version this build writes and understands. /// Schema version this build writes and understands.
pub const SCHEMA_VERSION: i64 = 13; pub const SCHEMA_VERSION: i64 = 14;
/// Apply migrations up to [`SCHEMA_VERSION`]. /// Apply migrations up to [`SCHEMA_VERSION`].
/// ///
@@ -119,6 +119,23 @@ pub fn migrate(conn: &Connection) -> Result<i64, CatalogError> {
tx.commit()?; tx.commit()?;
} }
if from < 14 {
let tx = conn.unchecked_transaction()?;
// `ALTER TABLE ... ADD COLUMN` has no `IF NOT EXISTS`, and NFR-R5
// wants this re-enterable: a catalog whose `user_version` was rewound
// by a rollback already has the column, and would otherwise fail its
// next open on it.
let has_quality: bool = tx
.prepare("SELECT 1 FROM pragma_table_info('faces') WHERE name = 'quality'")?
.exists([])?;
if !has_quality {
tx.execute_batch("ALTER TABLE faces ADD COLUMN quality REAL;")?;
}
tx.execute_batch(V14)?;
tx.pragma_update(None, "user_version", 14)?;
tx.commit()?;
}
Ok(from) Ok(from)
} }
@@ -251,7 +268,8 @@ pub fn for_attached(schema_name: &str) -> String {
format!( format!(
"{}\n{}\n{}\n\ "{}\n{}\n{}\n\
ALTER TABLE {schema_name}.people ADD COLUMN ignored INTEGER NOT NULL DEFAULT 0;\n\ ALTER TABLE {schema_name}.people ADD COLUMN ignored INTEGER NOT NULL DEFAULT 0;\n\
ALTER TABLE {schema_name}.faces ADD COLUMN crop BLOB;", ALTER TABLE {schema_name}.faces ADD COLUMN crop BLOB;\n\
ALTER TABLE {schema_name}.faces ADD COLUMN quality REAL;",
rewrite_for_attached(V1, schema_name), rewrite_for_attached(V1, schema_name),
rewrite_for_attached(V6, schema_name), rewrite_for_attached(V6, schema_name),
rewrite_for_attached(V8, schema_name), rewrite_for_attached(V8, schema_name),
@@ -569,6 +587,61 @@ CREATE TABLE IF NOT EXISTS sidecars (
); );
"#; "#;
const V14: &str = r#"
-- TRACES: FR-CULL-9 | FR-CULL-10
-- How recognisable the model found each face, and a second look at the faces
-- it was never asked about.
--
-- The embedder's raw output has a length, and the length is a quality
-- reading: it grows with how much of a face the model could make out, and a
-- blur, an occlusion or a hard profile comes out short (dr_face::embedding,
-- `MIN_GALLERY_QUALITY`). Normalising threw it away. A short vector sits
-- near the middle of the sphere and matches a little of everyone, which is
-- how one bad crop bridges two people in a grouping pass -- so a face below
-- the floor is compared against the others and never compared *against*.
--
-- Nullable, and NULL means "never measured": every face indexed before this
-- version stored the unit vector, whose length is one whatever the crop was.
-- A face with no reading is admitted to the gallery, because a rule that
-- cannot be checked should admit rather than exclude -- but it is also a
-- face this rule is not yet protecting anyone from, and the only way to
-- measure it is to embed it again.
--
-- The sweep's measuring pass is what does that: `dr_ui::library::
-- faces_unmeasured` lists every image holding a face with no reading, and
-- each face is embedded again from the native render with the landmarks it
-- already has, the raw vector written over the old one (`record_measurements`)
-- and nothing else touched -- not the id, not the box, not who the user said
-- it was. The faces keep drawing the People screen throughout.
--
-- The run markers of those images are forgotten too, exactly as V12 forgot
-- the runs made against too small a proxy. The build this shipped in had no
-- measuring pass yet, and a marker is the one thing that stops a face ever
-- being looked at again; with the pass in place `faces_unindexed` leaves
-- these images to it rather than detecting them from scratch, so the
-- deletion costs nothing -- and an image that was examined and found empty
-- keeps its marker, since there is nothing on it to measure.
--
-- The cost is a re-fetch of every image with a face on it, on the next pass
-- the user starts. That is a whole-library transfer (FR-NC-6), and it starts
-- when they say so, not here.
--
-- From this version the `embedding` blob is the **raw** model output rather
-- than the unit vector V8 describes -- the length is the quality, and a store
-- that kept only the direction had thrown it away. Readers re-normalise on
-- load, so a unit blob from before and a raw blob from now compare alike;
-- `quality` is that length kept beside the blob for the readers that never
-- load the vector, and NULL rather than 1.0 for the old rows, because a unit
-- vector reads as a length of one and one is not "unmeasured".
--
-- The column itself is added in `migrate`, guarded, because ALTER has no
-- IF NOT EXISTS and this step has to be re-enterable (NFR-R5).
DELETE FROM face_index
WHERE EXISTS (SELECT 1 FROM faces f
WHERE f.image_id = face_index.image_id
AND f.model_id = face_index.model_id);
"#;
const V9: &str = r#" const V9: &str = r#"
-- TRACES: FR-CULL-8 -- TRACES: FR-CULL-8
-- A record that face detection has *run* on an image, distinct from what it -- A record that face detection has *run* on an image, distinct from what it
@@ -648,7 +721,7 @@ CREATE TABLE faces (
x REAL NOT NULL, y REAL NOT NULL, w REAL NOT NULL, h REAL NOT NULL, x REAL NOT NULL, y REAL NOT NULL, w REAL NOT NULL, h REAL NOT NULL,
landmarks BLOB NOT NULL, -- 5 x (x, y) f32, normalised likewise landmarks BLOB NOT NULL, -- 5 x (x, y) f32, normalised likewise
detector_confidence REAL NOT NULL, detector_confidence REAL NOT NULL,
embedding BLOB NOT NULL, -- 512 x f16, L2-normalised embedding BLOB NOT NULL, -- 512 x f16; unit length until V14, raw since
-- Source pixels across the aligned 112x112 crop (docs/faces.md §7). -- Source pixels across the aligned 112x112 crop (docs/faces.md §7).
-- --
-- Not cosmetic: it is the honest quality signal for the UI, a feature in -- Not cosmetic: it is the honest quality signal for the UI, a feature in
@@ -1405,6 +1478,64 @@ mod tests {
assert_eq!(kept, vec![3, 4]); assert_eq!(kept, vec![3, 4]);
} }
#[test]
fn v14_forgets_runs_that_found_faces_but_never_measured_them() {
let c = mem();
c.pragma_update(None, "user_version", 0).unwrap();
migrate(&c).unwrap();
c.execute(
"INSERT INTO roots(id, kind, label) VALUES (1, 'local', 'test')",
[],
)
.unwrap();
c.execute(
"INSERT INTO images(id, root_id, source_ref, added_at)
VALUES (1,1,'a',0),(2,1,'b',0),(3,1,'c',0)",
[],
)
.unwrap();
// Image 1 was examined and holds a face; 2 was examined and found
// empty; 3 holds a face found by a different model.
for (image, model) in [(1, "m"), (2, "m"), (3, "m")] {
c.execute(
"INSERT INTO face_index(image_id, model_id, indexed_at, faces_found, source_edge)
VALUES (?1, ?2, 0, 0, 2560)",
rusqlite::params![image, model],
)
.unwrap();
}
for (image, model) in [(1, "m"), (3, "other")] {
c.execute(
"INSERT INTO faces
(image_id, x, y, w, h, landmarks, detector_confidence, embedding,
crop_px, model_id, detected_at)
VALUES (?1, 0.1, 0.1, 0.2, 0.2, X'00', 0.9, X'00', 180.0, ?2, 0)",
rusqlite::params![image, model],
)
.unwrap();
}
c.pragma_update(None, "user_version", 13).unwrap();
migrate(&c).unwrap();
let kept: Vec<i64> = c
.prepare("SELECT image_id FROM face_index ORDER BY image_id")
.unwrap()
.query_map([], |r| r.get(0))
.unwrap()
.map(Result::unwrap)
.collect();
// 1 goes: it has a face with no quality. 2 stays: nothing on it to
// measure. 3 stays: its face belongs to a run this marker does not
// describe.
assert_eq!(kept, vec![2, 3]);
// And the faces themselves are untouched.
let faces: i64 = c
.query_row("SELECT count(*) FROM faces", [], |r| r.get(0))
.unwrap();
assert_eq!(faces, 2);
}
#[test] #[test]
fn job_uniqueness_coalesces_rather_than_duplicating() { fn job_uniqueness_coalesces_rather_than_duplicating() {
let c = mem(); let c = mem();
+3 -2
View File
@@ -63,15 +63,16 @@ fn main() {
let embed_ms = t.elapsed().as_secs_f64() * 1e3; let embed_ms = t.elapsed().as_secs_f64() * 1e3;
println!( println!(
" [{i}] conf {:.3} box {:.0},{:.0} {:.0}×{:.0} crop_px {:.0} embed {embed_ms:.0} ms", " [{i}] conf {:.3} box {:.0},{:.0} {:.0}×{:.0} crop_px {:.0} quality {:.1} embed {embed_ms:.0} ms",
d.confidence, d.confidence,
d.bbox.0, d.bbox.0,
d.bbox.1, d.bbox.1,
d.width(), d.width(),
d.height(), d.height(),
aligned.source_px(), aligned.source_px(),
emb.quality,
); );
all.push((path.clone(), i, emb)); all.push((path.clone(), i, emb.embedding));
} }
} }
+2
View File
@@ -46,11 +46,13 @@ fn main() {
); );
for n in sizes { for n in sizes {
let (embeddings, crop_px, images) = population(n); let (embeddings, crop_px, images) = population(n);
let gallery = vec![true; n];
let faces = Faces { let faces = Faces {
embeddings: &embeddings, embeddings: &embeddings,
dim: EMBEDDING_DIM, dim: EMBEDDING_DIM,
crop_px: &crop_px, crop_px: &crop_px,
images: &images, images: &images,
gallery: &gallery,
}; };
let start = std::time::Instant::now(); let start = std::time::Instant::now();
+48 -11
View File
@@ -136,11 +136,24 @@ pub const RIVAL_FLOOR: f32 = 0.5;
/// ///
/// A face in no group, or one with no evidence for anybody, scores 0. /// A face in no group, or one with no evidence for anybody, scores 0.
/// ///
/// `gallery` is one flag per face — which faces may be evidence at all
/// ([`crate::embedding::MIN_GALLERY_QUALITY`]). Its length is the face count.
/// A pair is evidence *about* either face but only *from* a gallery one: a
/// probe learns from the references it matched, and a reference learns nothing
/// from a probe that happened to match it, however well. Without that, the one
/// short vector in a group would be the strongest match every face in it had.
///
/// `pairs` must be the *evidence* list — scanned at [`RIVAL_FLOOR`], not at the /// `pairs` must be the *evidence* list — scanned at [`RIVAL_FLOOR`], not at the
/// merge threshold. Passing the merge list still works but silently removes /// merge threshold. Passing the merge list still works but silently removes
/// every rival weaker than a merge, which is most of them, and every uniqueness /// every rival weaker than a merge, which is most of them, and every uniqueness
/// collapses to 1. /// collapses to 1.
pub fn identity_shares(faces: usize, clusters: &[Cluster], pairs: &[Pair], top: usize) -> Vec<f32> { pub fn identity_shares(
gallery: &[bool],
clusters: &[Cluster],
pairs: &[Pair],
top: usize,
) -> Vec<f32> {
let faces = gallery.len();
// An identity is a *person*, not a group. One person routinely holds // An identity is a *person*, not a group. One person routinely holds
// several anchored groups — the same reason they hold several unnamed ones // several anchored groups — the same reason they hold several unnamed ones
// — and keying this by group had Catherine competing with Catherine, which // — and keying this by group had Catherine competing with Catherine, which
@@ -174,15 +187,16 @@ pub fn identity_shares(faces: usize, clusters: &[Cluster], pairs: &[Pair], top:
} }
// A pair is evidence in both directions: j's identity hears about i, // A pair is evidence in both directions: j's identity hears about i,
// and i's identity hears about j. The pair list holds each unordered // and i's identity hears about j. The pair list holds each unordered
// pair once, so both have to be recorded here. // pair once, so both have to be recorded here — each only where the
// face doing the telling is in the gallery.
let (gi, gj) = (group_of[p.i], group_of[p.j]); let (gi, gj) = (group_of[p.i], group_of[p.j]);
if gj != usize::MAX { if gj != usize::MAX && gallery[p.j] {
evidence[p.i] evidence[p.i]
.entry(key_of[gj]) .entry(key_of[gj])
.or_default() .or_default()
.push(p.probability); .push(p.probability);
} }
if gi != usize::MAX { if gi != usize::MAX && gallery[p.i] {
evidence[p.j] evidence[p.j]
.entry(key_of[gi]) .entry(key_of[gi])
.or_default() .or_default()
@@ -255,6 +269,11 @@ mod tests {
Pair { i, j, probability } Pair { i, j, probability }
} }
/// `n` faces, every one of them fit to be compared against.
fn all(n: usize) -> Vec<bool> {
vec![true; n]
}
/// The failure the module exists to fix: face 0 matches its own group's /// The failure the module exists to fix: face 0 matches its own group's
/// three members strongly, and the group has forty more it is unrelated to. /// three members strongly, and the group has forty more it is unrelated to.
/// The old within-group mean reported ~0.07 for this. /// The old within-group mean reported ~0.07 for this.
@@ -264,7 +283,7 @@ mod tests {
let clusters = vec![cluster(&members)]; let clusters = vec![cluster(&members)];
let pairs = vec![pair(0, 1, 0.99), pair(0, 2, 0.97), pair(0, 3, 0.95)]; let pairs = vec![pair(0, 1, 0.99), pair(0, 2, 0.97), pair(0, 3, 0.95)];
let shares = identity_shares(44, &clusters, &pairs, TOP_MATCHES); let shares = identity_shares(&all(44), &clusters, &pairs, TOP_MATCHES);
assert!( assert!(
(shares[0] - 0.97).abs() < 1e-6, (shares[0] - 0.97).abs() < 1e-6,
"the mean of its three real matches, undiluted: {}", "the mean of its three real matches, undiluted: {}",
@@ -284,7 +303,7 @@ mod tests {
pair(0, 4, 0.90), pair(0, 4, 0.90),
]; ];
let shares = identity_shares(5, &clusters, &pairs, TOP_MATCHES); let shares = identity_shares(&all(5), &clusters, &pairs, TOP_MATCHES);
// Coherent at 0.90, and only half of the evidence is its own. // Coherent at 0.90, and only half of the evidence is its own.
assert!( assert!(
(shares[0] - 0.45).abs() < 1e-6, (shares[0] - 0.45).abs() < 1e-6,
@@ -298,9 +317,9 @@ mod tests {
#[test] #[test]
fn a_rival_too_weak_to_merge_still_lowers_the_confidence() { fn a_rival_too_weak_to_merge_still_lowers_the_confidence() {
let clusters = vec![named(&[0, 1], 1), named(&[2, 3], 2)]; let clusters = vec![named(&[0, 1], 1), named(&[2, 3], 2)];
let sure = identity_shares(4, &clusters, &[pair(0, 1, 0.95)], TOP_MATCHES); let sure = identity_shares(&all(4), &clusters, &[pair(0, 1, 0.95)], TOP_MATCHES);
let contested = identity_shares( let contested = identity_shares(
4, &all(4),
&clusters, &clusters,
&[pair(0, 1, 0.95), pair(0, 2, 0.60)], &[pair(0, 1, 0.95), pair(0, 2, 0.60)],
TOP_MATCHES, TOP_MATCHES,
@@ -321,7 +340,7 @@ mod tests {
fn an_unnamed_group_is_not_treated_as_competition() { fn an_unnamed_group_is_not_treated_as_competition() {
let clusters = vec![cluster(&[0, 1]), cluster(&[2, 3])]; let clusters = vec![cluster(&[0, 1]), cluster(&[2, 3])];
let shares = identity_shares( let shares = identity_shares(
4, &all(4),
&clusters, &clusters,
&[pair(0, 1, 0.95), pair(0, 2, 0.90)], &[pair(0, 1, 0.95), pair(0, 2, 0.90)],
TOP_MATCHES, TOP_MATCHES,
@@ -343,7 +362,7 @@ mod tests {
let mut pairs: Vec<Pair> = (1..11).map(|j| pair(0, j, 0.90)).collect(); let mut pairs: Vec<Pair> = (1..11).map(|j| pair(0, j, 0.90)).collect();
pairs.extend((11..62).map(|j| pair(0, j, 0.55))); pairs.extend((11..62).map(|j| pair(0, j, 0.55)));
let shares = identity_shares(62, &clusters, &pairs, TOP_MATCHES); let shares = identity_shares(&all(62), &clusters, &pairs, TOP_MATCHES);
// Ten at 0.90 against ten at 0.55 — not fifty-one at 0.55. // Ten at 0.90 against ten at 0.55 — not fifty-one at 0.55.
assert!( assert!(
(shares[0] - 0.90 * (9.0 / 14.5)).abs() < 1e-5, (shares[0] - 0.90 * (9.0 / 14.5)).abs() < 1e-5,
@@ -352,11 +371,29 @@ mod tests {
); );
} }
/// A probe learns from the references it matched; a reference learns
/// nothing from a probe. The pair is the same pair — what differs is who
/// is doing the telling.
#[test]
fn a_face_outside_the_gallery_is_nobody_s_evidence() {
let clusters = vec![named(&[0, 1, 2], 1)];
let gallery = vec![true, true, false];
let pairs = vec![pair(0, 1, 0.80), pair(0, 2, 0.99), pair(1, 2, 0.99)];
let shares = identity_shares(&gallery, &clusters, &pairs, TOP_MATCHES);
// Faces 0 and 1 hear only from each other: the 0.99 the probe offered
// them is not counted.
assert!((shares[0] - 0.80).abs() < 1e-6, "{}", shares[0]);
assert!((shares[1] - 0.80).abs() < 1e-6, "{}", shares[1]);
// The probe hears from both references.
assert!((shares[2] - 0.99).abs() < 1e-6, "{}", shares[2]);
}
/// A face nothing has any evidence about claims nothing. /// A face nothing has any evidence about claims nothing.
#[test] #[test]
fn a_face_with_no_evidence_reports_no_confidence() { fn a_face_with_no_evidence_reports_no_confidence() {
let clusters = vec![cluster(&[0, 1])]; let clusters = vec![cluster(&[0, 1])];
let shares = identity_shares(2, &clusters, &[], TOP_MATCHES); let shares = identity_shares(&all(2), &clusters, &[], TOP_MATCHES);
assert_eq!(shares, vec![0.0, 0.0]); assert_eq!(shares, vec![0.0, 0.0]);
} }
} }
+307 -3
View File
@@ -19,6 +19,23 @@
//! and clustering never moves it. Two groups holding confirmations of //! and clustering never moves it. Two groups holding confirmations of
//! *different* people cannot merge, whatever their similarity says. //! *different* people cannot merge, whatever their similarity says.
//! //!
//! # The gallery, and the faces that are only ever compared against it
//!
//! A third defence, and the cheapest of all: **a short embedding is never a
//! reference.** The length of the raw vector is the model's own reading of
//! how recognisable the crop was ([`crate::embedding::MIN_GALLERY_QUALITY`]),
//! and a short one sits near the centre of the sphere, matching a little of
//! everybody. One of those in a group is a bridge to the next group over.
//!
//! So the population is split. Faces at or above the floor are the
//! **gallery**, and they cluster exactly as described below. Faces under it
//! are **probes**: each is measured against the finished groups and joins the
//! one it fits, by the same average-link rule and under the same constraints
//! — but it is measured against the gallery members only, never against
//! another probe, and once placed it is never part of what the next face is
//! measured against. A blurred photograph of a known person is still named;
//! it just cannot vouch for anyone else.
//!
//! # Average link, not single link //! # Average link, not single link
//! //!
//! Single-link chains: one bad edge welds two identities together, and it is //! Single-link chains: one bad edge welds two identities together, and it is
@@ -117,6 +134,11 @@ pub struct Candidate {
pub embedding: Vec<f32>, pub embedding: Vec<f32>,
/// Source pixels across the aligned crop, for the calibration's size term. /// Source pixels across the aligned crop, for the calibration's size term.
pub crop_px: f32, pub crop_px: f32,
/// Length of the raw embedding, where it was recorded
/// ([`crate::embedding::MIN_GALLERY_QUALITY`]). `None` for a face indexed
/// before it was kept, which is admitted to the gallery — see
/// [`Candidate::in_gallery`].
pub quality: Option<f32>,
/// The person this face is *confirmed* to be, if any. /// The person this face is *confirmed* to be, if any.
/// ///
/// Suggestions are deliberately not passed here. They are this function's /// Suggestions are deliberately not passed here. They are this function's
@@ -125,6 +147,13 @@ pub struct Candidate {
pub confirmed_person: Option<u64>, pub confirmed_person: Option<u64>,
} }
impl Candidate {
/// Whether this face may be compared *against*, as well as compared.
pub fn in_gallery(&self) -> bool {
crate::embedding::in_gallery(self.quality)
}
}
/// One group of faces the clusterer believes are one person. /// One group of faces the clusterer believes are one person.
#[derive(Debug, Clone, PartialEq)] #[derive(Debug, Clone, PartialEq)]
pub struct Cluster { pub struct Cluster {
@@ -202,7 +231,7 @@ pub fn cluster_scored(faces: &[Candidate], cal: &Calibration, min_probability: f
let clusters = build(faces, cal, min_probability, &merges); let clusters = build(faces, cal, min_probability, &merges);
let confidence = crate::assign::identity_shares( let confidence = crate::assign::identity_shares(
faces.len(), &columns.gallery,
&clusters, &clusters,
&evidence, &evidence,
crate::assign::TOP_MATCHES, crate::assign::TOP_MATCHES,
@@ -225,6 +254,7 @@ struct Columns {
dim: usize, dim: usize,
crop_px: Vec<f32>, crop_px: Vec<f32>,
images: Vec<u64>, images: Vec<u64>,
gallery: Vec<bool>,
} }
impl Columns { impl Columns {
@@ -245,6 +275,7 @@ impl Columns {
dim, dim,
crop_px: faces.iter().map(|f| f.crop_px).collect(), crop_px: faces.iter().map(|f| f.crop_px).collect(),
images: faces.iter().map(|f| f.image).collect(), images: faces.iter().map(|f| f.image).collect(),
gallery: faces.iter().map(Candidate::in_gallery).collect(),
} }
} }
@@ -254,22 +285,171 @@ impl Columns {
dim: self.dim, dim: self.dim,
crop_px: &self.crop_px, crop_px: &self.crop_px,
images: &self.images, images: &self.images,
gallery: &self.gallery,
} }
} }
} }
/// Agglomerate the gallery over its pairs, then place the probes.
///
/// `pairs` is what [`neighbours::above_threshold`] returned: every pair has a
/// gallery side, but a pair with a probe on the other side is not a merge —
/// it is the evidence [`place_probes`] works from. Only the gallery-to-gallery
/// pairs reach the engine, so a probe enters it as a singleton with no edges
/// and comes out exactly as it went in.
fn build( fn build(
faces: &[Candidate], faces: &[Candidate],
cal: &Calibration, cal: &Calibration,
min_probability: f32, min_probability: f32,
pairs: &[neighbours::Pair], pairs: &[neighbours::Pair],
) -> Vec<Cluster> { ) -> Vec<Cluster> {
let gallery: Vec<bool> = faces.iter().map(Candidate::in_gallery).collect();
let (merges, probe_pairs): (Vec<_>, Vec<_>) = pairs
.iter()
.copied()
.partition(|p| gallery[p.i] && gallery[p.j]);
let mut engine = Engine::new(faces, cal, min_probability); let mut engine = Engine::new(faces, cal, min_probability);
let parts = components(faces.len(), pairs); let parts = components(faces.len(), &merges);
for (component, edges) in parts.members.iter().zip(&parts.edges) { for (component, edges) in parts.members.iter().zip(&parts.edges) {
engine.agglomerate(component, edges); engine.agglomerate(component, edges);
} }
engine.finish() let dot = engine.dot;
let clusters = engine.finish();
if probe_pairs.is_empty() {
return clusters;
}
place_probes(
faces,
cal,
min_probability,
dot,
&gallery,
clusters,
&probe_pairs,
)
}
/// Put each probe into the finished group it fits, or leave it alone.
///
/// The same decision the engine makes for a singleton — average link over the
/// group, at or above `min_probability`, subject to [`Engine::can_link`]'s two
/// constraints — with one difference that is the whole point: the average is
/// over the group's **gallery** members. A probe already placed is not part of
/// what the next one is measured against, so a run of short vectors cannot
/// pull each other in one after another.
///
/// Probes are placed in index order and each placement is final, which is
/// what keeps this deterministic. The group a probe joins gains its
/// photograph, so a second face from the same frame cannot follow it — the
/// co-occurrence rule, applied exactly as the engine applies it.
fn place_probes(
faces: &[Candidate],
cal: &Calibration,
min_probability: f32,
dot: neighbours::DotFn,
gallery: &[bool],
mut clusters: Vec<Cluster>,
probe_pairs: &[neighbours::Pair],
) -> Vec<Cluster> {
// Where each face sits, and what each group's photographs and gallery
// members are. The probe's own singleton is here too, and is dropped once
// it has moved.
let mut group_of = vec![usize::MAX; faces.len()];
for (g, c) in clusters.iter().enumerate() {
for &m in &c.members {
group_of[m] = g;
}
}
let mut images: Vec<HashSet<u64>> = clusters
.iter()
.map(|c| c.members.iter().map(|&m| faces[m].image).collect())
.collect();
let references: Vec<Vec<usize>> = clusters
.iter()
.map(|c| c.members.iter().copied().filter(|&m| gallery[m]).collect())
.collect();
// Which groups each probe has any above-threshold pair into. Only those
// can average above the threshold — the argument the module note makes
// for the engine holds here unchanged.
let mut candidates: Vec<Vec<usize>> = vec![Vec::new(); faces.len()];
for p in probe_pairs {
let (probe, reference) = if gallery[p.i] { (p.j, p.i) } else { (p.i, p.j) };
candidates[probe].push(group_of[reference]);
}
let mut moved: Vec<usize> = Vec::new();
for probe in 0..faces.len() {
if gallery[probe] || candidates[probe].is_empty() {
continue;
}
let mut groups = std::mem::take(&mut candidates[probe]);
groups.sort_unstable();
groups.dedup();
let face = &faces[probe];
let mut best: Option<(f32, usize)> = None;
for g in groups {
let target = &clusters[g];
if let (Some(mine), Some(theirs)) = (face.confirmed_person, target.person) {
if mine != theirs {
continue;
}
}
if images[g].contains(&face.image) {
continue;
}
let (mut sum, mut count) = (0.0_f64, 0.0_f64);
for &r in &references[g] {
let cos = dot(&face.embedding, &faces[r].embedding);
let min_crop = face.crop_px.min(faces[r].crop_px);
sum += cal.probability(cos, min_crop, 0.0) as f64;
count += 1.0;
}
if count == 0.0 {
continue;
}
let p = (sum / count) as f32;
// Strictly better wins; on a tie the lowest group index, which is
// the engine's own tiebreak.
if p >= min_probability && best.is_none_or(|(bp, _)| p > bp) {
best = Some((p, g));
}
}
let Some((_, g)) = best else { continue };
let own = group_of[probe];
clusters[g].members.push(probe);
clusters[g].members.sort_unstable();
clusters[g].person = clusters[g].person.or(face.confirmed_person);
images[g].insert(face.image);
group_of[probe] = g;
moved.push(own);
}
if moved.is_empty() {
return clusters;
}
// The singletons the probes left behind, then the order `Engine::finish`
// promises: largest first, lowest member first among equals.
let mut vacated = vec![false; clusters.len()];
for g in moved {
vacated[g] = true;
}
let mut out: Vec<Cluster> = clusters
.into_iter()
.zip(vacated)
.filter(|(_, gone)| !gone)
.map(|(c, _)| c)
.collect();
out.sort_by(|x, y| {
y.members
.len()
.cmp(&x.members.len())
.then(x.members[0].cmp(&y.members[0]))
});
out
} }
/// Split one person's faces into the groups a raised threshold separates them /// Split one person's faces into the groups a raised threshold separates them
@@ -726,10 +906,19 @@ mod tests {
image, image,
embedding: at_cosine(identity, cosine), embedding: at_cosine(identity, cosine),
crop_px: 150.0, crop_px: 150.0,
quality: None,
confirmed_person: None, confirmed_person: None,
} }
} }
/// A face too short to be a reference: compared, never compared against.
fn probe(face: u64, image: u64, identity: usize, cosine: f32) -> Candidate {
Candidate {
quality: Some(crate::embedding::MIN_GALLERY_QUALITY - 5.0),
..candidate(face, image, identity, cosine)
}
}
/// A calibration steep enough that the test's cosines are unambiguous: /// A calibration steep enough that the test's cosines are unambiguous:
/// 0.6 is near-certain, 0.1 is near-impossible. /// 0.6 is near-certain, 0.1 is near-impossible.
fn cal() -> Calibration { fn cal() -> Calibration {
@@ -1099,6 +1288,7 @@ mod tests {
image, image,
embedding: at_cosine(p, cosine), embedding: at_cosine(p, cosine),
crop_px: 60.0 + ((out.len() % 11) as f32) * 25.0, crop_px: 60.0 + ((out.len() % 11) as f32) * 25.0,
quality: None,
confirmed_person: None, confirmed_person: None,
}); });
image += 1; image += 1;
@@ -1182,6 +1372,7 @@ mod tests {
image: 5_000, image: 5_000,
embedding: at_cosine(200, 1.0), embedding: at_cosine(200, 1.0),
crop_px: 150.0, crop_px: 150.0,
quality: None,
confirmed_person: None, confirmed_person: None,
}); });
let out = cluster(&faces, &cal(), DEFAULT_MERGE_PROBABILITY); let out = cluster(&faces, &cal(), DEFAULT_MERGE_PROBABILITY);
@@ -1191,4 +1382,117 @@ mod tests {
"the outlier was absorbed" "the outlier was absorbed"
); );
} }
// ── the gallery ───────────────────────────────────────────────────────
/// A short vector is still somebody: it joins the group it matches.
#[test]
fn a_probe_joins_the_group_it_matches() {
let faces = vec![
candidate(1, 10, 0, 1.0),
candidate(2, 11, 0, 0.95),
probe(3, 12, 0, 0.92),
];
let out = cluster(&faces, &cal(), DEFAULT_MERGE_PROBABILITY);
assert_eq!(out.len(), 1);
assert_eq!(out[0].members, vec![0, 1, 2]);
}
/// Two short vectors that resemble each other are noise agreeing with
/// noise, and there is nothing in the gallery for either to be measured
/// against.
#[test]
fn two_probes_are_never_grouped_with_each_other() {
let faces = vec![probe(1, 10, 0, 1.0), probe(2, 11, 0, 0.98)];
let out = cluster(&faces, &cal(), DEFAULT_MERGE_PROBABILITY);
assert_eq!(out.len(), 2, "two probes were grouped: {out:?}");
}
/// The point of measuring against the gallery only: a probe that has been
/// placed is not a stepping stone for the next one.
#[test]
fn a_placed_probe_is_not_what_the_next_probe_is_measured_against() {
let mut first = probe(2, 11, 0, 0.6);
// 0.6 along identity 0 and 0.8 along its perpendicular: near enough to
// the reference to join it, and much nearer to the face below.
first.embedding = at_cosine(0, 0.6);
let mut second = probe(3, 12, 0, 0.0);
second.embedding = at_cosine(0, 0.0);
let faces = vec![candidate(1, 10, 0, 1.0), first, second];
let out = cluster(&faces, &cal(), DEFAULT_MERGE_PROBABILITY);
let group = out.iter().find(|c| c.members.contains(&0)).unwrap();
assert_eq!(
group.members,
vec![0, 1],
"the first probe should have joined"
);
assert!(
out.iter().any(|c| c.members == vec![2]),
"the second probe reached the group through the first: {out:?}"
);
}
/// A confirmation on a probe is still the user's word: the group it joins
/// becomes that person, and a group already someone else's is closed to it.
#[test]
fn a_probe_carries_its_confirmation_and_respects_others() {
let mut anchored = probe(3, 12, 0, 0.92);
anchored.confirmed_person = Some(7);
let faces = vec![
candidate(1, 10, 0, 1.0),
candidate(2, 11, 0, 0.95),
anchored,
];
let out = cluster(&faces, &cal(), DEFAULT_MERGE_PROBABILITY);
assert_eq!(out.len(), 1);
assert_eq!(out[0].person, Some(7));
let mut theirs = candidate(1, 10, 0, 1.0);
theirs.confirmed_person = Some(8);
let faces = vec![theirs, candidate(2, 11, 0, 0.95), {
let mut a = probe(3, 12, 0, 0.92);
a.confirmed_person = Some(7);
a
}];
let out = cluster(&faces, &cal(), DEFAULT_MERGE_PROBABILITY);
assert!(
out.iter()
.any(|c| c.members == vec![2] && c.person == Some(7)),
"a probe confirmed as one person joined another's group: {out:?}"
);
}
/// The co-occurrence rule follows a probe in: once it has joined, its
/// photograph is the group's.
#[test]
fn a_probe_cannot_join_a_group_holding_a_face_from_its_own_photograph() {
let faces = vec![
candidate(1, 10, 0, 1.0),
candidate(2, 11, 0, 0.95),
probe(3, 10, 0, 0.92),
];
let out = cluster(&faces, &cal(), DEFAULT_MERGE_PROBABILITY);
assert!(out.iter().any(|c| c.members == vec![2]), "{out:?}");
}
/// A probe's placement is scored like anyone else's, from the references
/// it matched — and the references' own scores do not hear from it.
#[test]
fn a_probe_is_scored_but_is_not_evidence() {
let gallery_only = vec![candidate(1, 10, 0, 1.0), candidate(2, 11, 0, 0.95)];
let without = cluster_scored(&gallery_only, &cal(), DEFAULT_MERGE_PROBABILITY);
let mut with_probe = gallery_only.clone();
with_probe.push(probe(3, 12, 0, 0.99));
let with = cluster_scored(&with_probe, &cal(), DEFAULT_MERGE_PROBABILITY);
assert_eq!(with.clusters[0].members, vec![0, 1, 2]);
assert!(with.confidence[2] > 0.9, "{}", with.confidence[2]);
assert_eq!(
&with.confidence[..2],
&without.confidence[..],
"a probe changed what the references were sure of"
);
}
} }
+43 -5
View File
@@ -16,6 +16,41 @@ use crate::align::{Aligned112, ALIGNED_EDGE};
use crate::embedding::{normalise, Embedding, ModelId, EMBEDDING_DIM}; use crate::embedding::{normalise, Embedding, ModelId, EMBEDDING_DIM};
use crate::{install_backend, FaceError}; use crate::{install_backend, FaceError};
/// What one pass of the embedder produces: the direction, and the length.
///
/// Two fields rather than a `quality` on [`Embedding`], because every other
/// holder of an `Embedding` relies on it being unit length and compares by
/// dot product; the length is a separate fact about the same face, and it is
/// stored separately too.
#[derive(Debug, Clone, PartialEq)]
pub struct Embedded {
pub embedding: Embedding,
/// L2 norm of the raw model output.
///
/// The model's own opinion of how recognisable the crop was — see
/// [`crate::embedding::MIN_GALLERY_QUALITY`] for what it means and where
/// it is used.
pub quality: f32,
}
impl Embedded {
/// Storage form: the **raw** vector, `512 × f16`.
///
/// Not the unit vector. The length is the quality, and a store that held
/// only the direction would have thrown it away at the one moment it could
/// be known — which is what this crate used to do. Readers re-normalise
/// ([`Embedding::from_f16_bytes`]), so every comparison is still a dot
/// product, and [`crate::embedding::read_f16_bytes`] gives the length back
/// to a reader that wants it.
///
/// f16 costs nothing extra at this scale: its precision is relative, so a
/// component of a vector of length 20 is kept to the same three figures as
/// the same component scaled to length 1.
pub fn to_f16_bytes(&self) -> Vec<u8> {
self.embedding.to_f16_bytes_scaled(self.quality)
}
}
/// A loaded ArcFace graph. /// A loaded ArcFace graph.
pub struct Embedder { pub struct Embedder {
session: ort::session::Session, session: ort::session::Session,
@@ -63,7 +98,7 @@ impl Embedder {
} }
/// Embed one aligned face. /// Embed one aligned face.
pub fn embed(&mut self, face: &Aligned112) -> Result<Embedding, FaceError> { pub fn embed(&mut self, face: &Aligned112) -> Result<Embedded, FaceError> {
// `(x·255 − 127.5) / 128` — see the `/128` note in `detect::Letterbox`. // `(x·255 − 127.5) / 128` — see the `/128` note in `detect::Letterbox`.
let px = face.pixels(); let px = face.pixels();
let mut input = Array4::<f32>::zeros((1, 3, ALIGNED_EDGE, ALIGNED_EDGE)); let mut input = Array4::<f32>::zeros((1, 3, ALIGNED_EDGE, ALIGNED_EDGE));
@@ -95,11 +130,14 @@ impl Embedder {
let mut v = Box::new([0.0_f32; EMBEDDING_DIM]); let mut v = Box::new([0.0_f32; EMBEDDING_DIM]);
v.copy_from_slice(&data[..EMBEDDING_DIM]); v.copy_from_slice(&data[..EMBEDDING_DIM]);
normalise(&mut v); let quality = normalise(&mut v);
Ok(Embedding { Ok(Embedded {
model: self.model.clone(), embedding: Embedding {
v, model: self.model.clone(),
v,
},
quality,
}) })
} }
} }
+119 -18
View File
@@ -12,6 +12,43 @@
/// Embedding dimensionality. Fixed by the model family, not a parameter. /// Embedding dimensionality. Fixed by the model family, not a parameter.
pub const EMBEDDING_DIM: usize = 512; pub const EMBEDDING_DIM: usize = 512;
/// The shortest raw embedding a face may be *compared against*.
///
/// # What the length of the vector says
///
/// ArcFace is trained on the direction of its output and nothing else, and
/// the length it leaves behind turns out to be a free quality signal: the
/// magnitude grows with how recognisable the crop was to the model, and a
/// blurred, occluded, badly lit or hard-profile face comes out short. MagFace
/// (Meng et al., CVPR 2021) made that the training objective; the plain
/// ArcFace heads this crate runs already show it, weaker but usable, which is
/// why it is worth keeping the number the normalisation discards.
///
/// # Why it gates the gallery and not the face
///
/// A short vector is a bad *reference*: it sits nearer the centre of the
/// sphere than a real identity does and matches a little of everyone, which
/// is exactly the face that welds two people together in a clustering pass.
/// It is not a bad *probe* — the face is still real, still somebody, and
/// comparing it against good references is the only way it will ever be named.
/// So a face below this floor is compared against the gallery and never
/// becomes part of it: see `cluster::Candidate::in_gallery`.
///
/// 14 is the operating point for `w600k_mbf`, whose norms on the reference
/// library run from about 8 on a blur to the high 20s on a clean portrait. A
/// face whose quality was never recorded — indexed before the number was kept
/// — is not gated, because a rule that cannot be checked should admit, not
/// exclude.
pub const MIN_GALLERY_QUALITY: f32 = 14.0;
/// Whether an embedding of this quality may serve as a reference.
///
/// `None` is "not measured", and is admitted: the rule is about a number that
/// was read and found short, not about a number that is missing.
pub fn in_gallery(quality: Option<f32>) -> bool {
quality.is_none_or(|q| q >= MIN_GALLERY_QUALITY)
}
/// Which model produced an embedding. /// Which model produced an embedding.
/// ///
/// Embeddings from different models are not comparable, and this is the one /// Embeddings from different models are not comparable, and this is the one
@@ -58,10 +95,19 @@ impl Embedding {
} }
/// Storage form: `512 × f16`, 1 KB per face (catalog.md §10.1). /// Storage form: `512 × f16`, 1 KB per face (catalog.md §10.1).
///
/// This writes the unit vector. What the catalog stores is the raw one —
/// `embed::Embedded::to_f16_bytes` — because the length is the quality
/// and a unit vector has none left to read.
pub fn to_f16_bytes(&self) -> Vec<u8> { pub fn to_f16_bytes(&self) -> Vec<u8> {
self.to_f16_bytes_scaled(1.0)
}
/// The unit vector scaled by `length`, as `512 × f16`.
pub(crate) fn to_f16_bytes_scaled(&self, length: f32) -> Vec<u8> {
let mut out = Vec::with_capacity(EMBEDDING_DIM * 2); let mut out = Vec::with_capacity(EMBEDDING_DIM * 2);
for &x in self.v.iter() { for &x in self.v.iter() {
out.extend_from_slice(&f32_to_f16_bits(x).to_le_bytes()); out.extend_from_slice(&f32_to_f16_bits(x * length).to_le_bytes());
} }
out out
} }
@@ -71,25 +117,42 @@ impl Embedding {
/// The f16 round-trip perturbs a unit vector by ~1e-3 in cosine — three /// The f16 round-trip perturbs a unit vector by ~1e-3 in cosine — three
/// orders below the separation between a match and a non-match — but the /// orders below the separation between a match and a non-match — but the
/// drift is free to remove and invisible if left, so it is removed here /// drift is free to remove and invisible if left, so it is removed here
/// rather than remembered at every call site. /// rather than remembered at every call site. The same pass is what turns
/// a stored raw vector back into the unit one every comparison expects.
pub fn from_f16_bytes(model: ModelId, bytes: &[u8]) -> Option<Self> { pub fn from_f16_bytes(model: ModelId, bytes: &[u8]) -> Option<Self> {
if bytes.len() != EMBEDDING_DIM * 2 { read_f16_bytes(model, bytes).map(|(e, _)| e)
return None;
}
let mut v = Box::new([0.0_f32; EMBEDDING_DIM]);
for (i, chunk) in bytes.chunks_exact(2).enumerate() {
v[i] = f16_bits_to_f32(u16::from_le_bytes([chunk[0], chunk[1]]));
}
normalise(&mut v);
Some(Self { model, v })
} }
} }
/// Read a stored vector back, with the length it was stored at.
///
/// The length is the quality where the blob is a raw one, and ~1 where it is
/// a unit vector from before raw vectors were stored — which is why the
/// catalog keeps the quality beside the blob rather than deriving it from
/// this: a unit vector reads as a quality of 1, not as "unmeasured".
pub fn read_f16_bytes(model: ModelId, bytes: &[u8]) -> Option<(Embedding, f32)> {
if bytes.len() != EMBEDDING_DIM * 2 {
return None;
}
let mut v = Box::new([0.0_f32; EMBEDDING_DIM]);
for (i, chunk) in bytes.chunks_exact(2).enumerate() {
v[i] = f16_bits_to_f32(u16::from_le_bytes([chunk[0], chunk[1]]));
}
let length = normalise(&mut v);
Some((Embedding { model, v }, length))
}
fn dot(a: &[f32; EMBEDDING_DIM], b: &[f32; EMBEDDING_DIM]) -> f32 { fn dot(a: &[f32; EMBEDDING_DIM], b: &[f32; EMBEDDING_DIM]) -> f32 {
a.iter().zip(b.iter()).map(|(x, y)| x * y).sum() a.iter().zip(b.iter()).map(|(x, y)| x * y).sum()
} }
pub(crate) fn normalise(v: &mut [f32; EMBEDDING_DIM]) { /// Scale `v` to unit length, and return the length it had.
///
/// The length is the one thing about the raw output that survives being
/// thrown away by everything downstream, and it is a quality signal
/// ([`MIN_GALLERY_QUALITY`]) — so it comes back out rather than being lost
/// here.
pub(crate) fn normalise(v: &mut [f32; EMBEDDING_DIM]) -> f32 {
// Clamped rather than checked: a zero-norm embedding is a broken model, // Clamped rather than checked: a zero-norm embedding is a broken model,
// not a runtime condition worth an error path, and dividing by 1e-6 keeps // not a runtime condition worth an error path, and dividing by 1e-6 keeps
// the NaN out of the catalog. // the NaN out of the catalog.
@@ -97,6 +160,7 @@ pub(crate) fn normalise(v: &mut [f32; EMBEDDING_DIM]) {
for x in v.iter_mut() { for x in v.iter_mut() {
*x /= norm; *x /= norm;
} }
norm
} }
// ── f16 ─────────────────────────────────────────────────────────────────── // ── f16 ───────────────────────────────────────────────────────────────────
@@ -113,9 +177,10 @@ fn f32_to_f16_bits(x: f32) -> u16 {
let mant = bits & 0x007f_ffff; let mant = bits & 0x007f_ffff;
if exp >= 0x1f { if exp >= 0x1f {
// Overflow, inf, or NaN. Embeddings are unit-norm so this is the // Overflow, inf, or NaN. No component of an embedding exceeds its
// broken-model path; infinity is the honest answer, not a clamp that // length, and the lengths this model produces are in the tens, so
// hides it. // this is the broken-model path; infinity is the honest answer, not a
// clamp that hides it.
return sign return sign
| 0x7c00 | 0x7c00
| if mant != 0 && exp == 0x1f + 112 { | if mant != 0 && exp == 0x1f + 112 {
@@ -125,9 +190,9 @@ fn f32_to_f16_bits(x: f32) -> u16 {
}; };
} }
if exp <= 0 { if exp <= 0 {
// Subnormal or underflow. A component of a unit 512-vector is ~0.04, // Subnormal or underflow. A component of a unit 512-vector is ~0.04
// nowhere near here, so this branch exists for correctness rather than // and a stored one is that times the length, nowhere near here, so
// for traffic. // this branch exists for correctness rather than for traffic.
if exp < -10 { if exp < -10 {
return sign; return sign;
} }
@@ -221,6 +286,42 @@ mod tests {
} }
} }
#[test]
fn normalising_reports_the_length_it_removed() {
let mut v = Box::new([0.0_f32; EMBEDDING_DIM]);
v[0] = 3.0;
v[1] = 4.0;
let norm = normalise(&mut v);
assert!((norm - 5.0).abs() < 1e-6, "norm {norm}");
assert!((v[0] - 0.6).abs() < 1e-6 && (v[1] - 0.8).abs() < 1e-6);
}
/// The gate admits what it cannot measure: a face from before the number
/// was kept is not a face that was found wanting.
#[test]
fn an_unmeasured_quality_is_admitted_to_the_gallery() {
assert!(in_gallery(None));
assert!(in_gallery(Some(MIN_GALLERY_QUALITY)));
assert!(in_gallery(Some(27.5)));
assert!(!in_gallery(Some(MIN_GALLERY_QUALITY - 0.01)));
assert!(!in_gallery(Some(8.0)));
}
/// The storage form carries the length, and the length comes back out —
/// without touching the direction every comparison is made on.
#[test]
fn a_raw_vector_round_trips_with_its_length() {
let e = unit(3);
let raw = e.to_f16_bytes_scaled(21.5);
let (back, length) = read_f16_bytes(e.model.clone(), &raw).unwrap();
assert!((length - 21.5).abs() < 0.05, "length {length}");
assert!(e.cosine(&back).unwrap() > 0.9999);
// A unit vector from an older store reads as length 1, not as an
// error — see `read_f16_bytes` on why that is not "unmeasured".
let (_, one) = read_f16_bytes(e.model.clone(), &e.to_f16_bytes()).unwrap();
assert!((one - 1.0).abs() < 1e-2, "length {one}");
}
#[test] #[test]
fn f16_round_trip_rejects_a_wrong_length_blob() { fn f16_round_trip_rejects_a_wrong_length_blob() {
assert!(Embedding::from_f16_bytes(ModelId::new("m"), &[0u8; 100]).is_none()); assert!(Embedding::from_f16_bytes(ModelId::new("m"), &[0u8; 100]).is_none());
+4 -2
View File
@@ -75,8 +75,10 @@ pub use cluster::{
#[cfg(feature = "inference")] #[cfg(feature = "inference")]
pub use detect::{DetectOptions, Detection, Detector}; pub use detect::{DetectOptions, Detection, Detector};
#[cfg(feature = "inference")] #[cfg(feature = "inference")]
pub use embed::Embedder; pub use embed::{Embedded, Embedder};
pub use embedding::{Embedding, ModelId, EMBEDDING_DIM}; pub use embedding::{
in_gallery, read_f16_bytes, Embedding, ModelId, EMBEDDING_DIM, MIN_GALLERY_QUALITY,
};
pub use naming::{name_for_instance, name_instances, NamedFace}; pub use naming::{name_for_instance, name_instances, NamedFace};
/// What can go wrong between an image and a face. /// What can go wrong between an image and a face.
+47 -2
View File
@@ -117,6 +117,17 @@ pub struct Faces<'a> {
/// Which photograph each face came from. Two faces in one frame are not /// Which photograph each face came from. Two faces in one frame are not
/// the same person, so those pairs are never returned (docs/faces.md §9). /// the same person, so those pairs are never returned (docs/faces.md §9).
pub images: &'a [u64], pub images: &'a [u64],
/// Which faces may be compared *against* — the gallery
/// ([`crate::embedding::MIN_GALLERY_QUALITY`]).
///
/// A pair needs at least one gallery side: a probe measured against a
/// reference is a comparison, two short vectors measured against each
/// other is noise agreeing with noise, and those pairs are never returned.
/// Filtered here rather than by the caller for the same reason
/// co-occurrence is: what this module leaves out of the list stays out of
/// the graph, the components and the merge order, so nothing downstream
/// has to remember the rule.
pub gallery: &'a [bool],
} }
impl Faces<'_> { impl Faces<'_> {
@@ -272,8 +283,9 @@ fn scan_rows(scan: &Scan, from: usize, to: usize, out: &mut Vec<Pair>) {
let a = faces.row(i); let a = faces.row(i);
let crop_a = faces.crop_px[i]; let crop_a = faces.crop_px[i];
let image_a = faces.images[i]; let image_a = faces.images[i];
let gallery_a = faces.gallery[i];
for j in start..tile_end { for j in start..tile_end {
if image_a == faces.images[j] { if image_a == faces.images[j] || !(gallery_a || faces.gallery[j]) {
continue; continue;
} }
let cos = dot(a, faces.row(j)); let cos = dot(a, faces.row(j));
@@ -552,6 +564,7 @@ mod tests {
embeddings: Vec<f32>, embeddings: Vec<f32>,
crop_px: Vec<f32>, crop_px: Vec<f32>,
images: Vec<u64>, images: Vec<u64>,
gallery: Vec<bool>,
} }
impl Set { impl Set {
@@ -561,6 +574,7 @@ mod tests {
dim: DIM, dim: DIM,
crop_px: &self.crop_px, crop_px: &self.crop_px,
images: &self.images, images: &self.images,
gallery: &self.gallery,
} }
} }
@@ -588,10 +602,12 @@ mod tests {
} }
} }
let crop_px = vec![150.0; embeddings.len()]; let crop_px = vec![150.0; embeddings.len()];
let gallery = vec![true; embeddings.len()];
Set { Set {
embeddings: embeddings.concat(), embeddings: embeddings.concat(),
crop_px, crop_px,
images, images,
gallery,
} }
} }
@@ -601,7 +617,7 @@ mod tests {
let mut out = Vec::new(); let mut out = Vec::new();
for i in 0..n { for i in 0..n {
for j in i + 1..n { for j in i + 1..n {
if faces.images[i] == faces.images[j] { if faces.images[i] == faces.images[j] || !(faces.gallery[i] || faces.gallery[j]) {
continue; continue;
} }
let cos: f32 = faces let cos: f32 = faces
@@ -750,6 +766,35 @@ mod tests {
assert!(above_threshold(&s.faces(), &cal(), 0.9).is_empty()); assert!(above_threshold(&s.faces(), &cal(), 0.9).is_empty());
} }
/// A probe against a reference is a comparison; two probes against each
/// other is not. The rule lives here so that nothing downstream sees the
/// pair at all.
#[test]
fn two_faces_outside_the_gallery_are_never_paired() {
let mut s = population(1, 3, 1.0);
s.gallery = vec![false, false, true];
let pairs = above_threshold(&s.faces(), &cal(), 0.9);
assert!(
!pairs.iter().any(|p| p.i == 0 && p.j == 1),
"two probes were paired with each other"
);
// Each probe is still measured against the one reference.
assert!(pairs.iter().any(|p| p.i == 0 && p.j == 2));
assert!(pairs.iter().any(|p| p.i == 1 && p.j == 2));
}
#[test]
fn the_gallery_rule_matches_the_reference_at_scale() {
let mut s = population(60, 8, 0.97);
for (i, g) in s.gallery.iter_mut().enumerate() {
*g = i % 3 != 0;
}
let f = s.faces();
let got = above_threshold(&f, &cal(), 0.9);
let want = reference(&f, &cal(), 0.9);
assert!(same_pairs(&got, &want), "{} vs {}", got.len(), want.len());
}
#[test] #[test]
fn pairs_come_back_in_index_order() { fn pairs_come_back_in_index_order() {
let s = population(300, 8, 0.97); let s = population(300, 8, 0.97);
+28 -2
View File
@@ -708,10 +708,36 @@ impl MaskPass {
source: Option<&DemosaicedImage>, source: Option<&DemosaicedImage>,
width: u32, width: u32,
height: u32, height: u32,
) -> Result<&MaskArray, GpuError> {
self.render_revealing(stack, labels, subjects, source, width, height, None)
}
/// TRACES: FR-DEV-19c
/// [`Self::render`], also drawing the layer being looked at.
///
/// A selection with no adjustment on it changes no pixel, so it is not
/// active and has no slice — which is right until somebody asks to *see*
/// it, and that is the state a photographer is in from choosing a subject
/// until deciding what to do to it.
///
/// `reveal` has to be the same one the shader was composed with and the
/// same one the distance fields were built for: all three index this array
/// by position in [`MaskStack::rendered`], and two of them disagreeing
/// shows as an adjustment applied through another layer's mask.
#[allow(clippy::too_many_arguments)]
pub fn render_revealing(
&mut self,
stack: &MaskStack,
labels: Option<&LabelField>,
subjects: Option<&SubjectMasks>,
source: Option<&DemosaicedImage>,
width: u32,
height: u32,
reveal: Option<&dr_pipeline::mask::Reveal>,
) -> Result<&MaskArray, GpuError> { ) -> Result<&MaskArray, GpuError> {
// At least one layer, because a zero-layer texture array is invalid // At least one layer, because a zero-layer texture array is invalid
// and the shader binds this slot unconditionally. // and the shader binds this slot unconditionally.
let active = stack.active_count().clamp(1, MAX_LAYERS) as u32; let active = stack.rendered_count(reveal).clamp(1, MAX_LAYERS) as u32;
self.ensure_array(width, height, active)?; self.ensure_array(width, height, active)?;
let mut encoder = self let mut encoder = self
@@ -721,7 +747,7 @@ impl MaskPass {
label: Some("mask-encoder"), label: Some("mask-encoder"),
}); });
for (slot, layer) in stack.active().enumerate().take(MAX_LAYERS) { for (slot, layer) in stack.rendered(reveal).enumerate().take(MAX_LAYERS) {
// **The path a mask with one part takes is the path every mask // **The path a mask with one part takes is the path every mask
// took before parts existed**: drawn straight into the layer's // took before parts existed**: drawn straight into the layer's
// slice, cleared by the draw itself. Nothing about an unedited // slice, cleared by the draw itself. Nothing about an unedited
+235 -2
View File
@@ -12,8 +12,8 @@
use dr_gpu::{AdjustPass, DemosaicedImage, GpuContext, LabelField, MaskPass}; use dr_gpu::{AdjustPass, DemosaicedImage, GpuContext, LabelField, MaskPass};
use dr_pipeline::descriptor::ParamId; use dr_pipeline::descriptor::ParamId;
use dr_pipeline::mask::{Join, MaskLayer, MaskPart, MaskSource, MaskStack}; use dr_pipeline::mask::{Join, MaskLayer, MaskPart, MaskSource, MaskStack, Reveal, RevealStyle};
use dr_pipeline::operation::compose_full; use dr_pipeline::operation::{compose_full, compose_full_revealing};
use dr_pipeline::spot::SpotSet; use dr_pipeline::spot::SpotSet;
use dr_pipeline::{ops, EditGraph, Framing}; use dr_pipeline::{ops, EditGraph, Framing};
use dr_types::ColourSpace; use dr_types::ColourSpace;
@@ -789,3 +789,236 @@ fn the_order_parts_are_joined_in_is_the_mask() {
"and subtracting after an addition takes it away again" "and subtracting after an addition takes it away again"
); );
} }
// --- seeing the mask (FR-DEV-19c) ------------------------------------------
/// A radial that covers the middle of the frame and nothing near the corners.
fn middle() -> MaskSource {
MaskSource::Radial {
centre: (0.5, 0.5),
radii: (0.3, 0.3),
angle: 0.0,
feather: 0.05,
}
}
/// [`render`], with one layer's mask drawn over the result.
fn render_revealing(ctx: &GpuContext, stack: &MaskStack, reveal: &Reveal) -> Vec<u8> {
let source = grey(ctx);
let shader = compose_full_revealing(
&ops::chain(),
&Framing::new(),
ColourSpace::Srgb,
stack,
&SpotSet::new(),
&[],
Some(reveal),
);
let mut masks = MaskPass::new(ctx).expect("mask pass");
let array = masks
.render_revealing(stack, None, None, None, SIZE, SIZE, Some(reveal))
.expect("rasterise");
let mut adjust = AdjustPass::new(ctx);
adjust
.render_masked(&source, &shader, SIZE, SIZE, Some(array))
.expect("render");
adjust.export_pixels().expect("readback").0
}
/// TRACES: FR-DEV-19c
/// The state every mask is in for its first few seconds: chosen, and not yet
/// used for anything.
///
/// Such a layer changes no pixel, so it is not active, so it occupied no mask
/// slot and was never rasterised — and the reveal drew nothing. That is the
/// whole of "I clicked the category and nothing happened": there was a mask,
/// and no way to see that there was.
#[test]
fn a_selection_with_no_adjustment_can_still_be_seen() {
let Some(ctx) = ctx() else {
eprintln!("no adapter; skipping");
return;
};
let mut stack = MaskStack::new();
stack.push(MaskLayer::new("m1", middle()));
assert!(
stack.is_neutral(),
"the fixture must be a selection with nothing done to it"
);
let pixels = render_revealing(&ctx, &stack, &Reveal::one("m1", RevealStyle::Alpha));
assert!(
luma_at(&pixels, SIZE / 2, SIZE / 2) > 200,
"the middle is inside the mask and should read white"
);
assert!(
luma_at(&pixels, 1, 1) < 40,
"the corner is outside it and should read black"
);
}
/// And with nobody looking, the same stack changes nothing at all.
///
/// The other half of the property above: a layer renders *because* it is being
/// revealed, so it must stop when the reveal does — otherwise a selection with
/// no adjustment would leave a slice in the array for ever.
#[test]
fn a_mask_nobody_is_looking_at_draws_nothing() {
let Some(ctx) = ctx() else {
eprintln!("no adapter; skipping");
return;
};
let mut stack = MaskStack::new();
stack.push(MaskLayer::new("m1", middle()));
let pixels = render(&ctx, &stack, None);
assert_eq!(
luma_at(&pixels, SIZE / 2, SIZE / 2),
128,
"flat grey, exactly as it went in"
);
}
/// TRACES: FR-DEV-19c
/// A tint has to leave the photograph visible, or it cannot be judged against
/// it — which is the one thing an overlay exists for.
#[test]
fn a_tint_colours_the_mask_and_leaves_the_rest_alone() {
let Some(ctx) = ctx() else {
eprintln!("no adapter; skipping");
return;
};
let mut stack = MaskStack::new();
stack.push(MaskLayer::new("m1", middle()));
let pixels = render_revealing(&ctx, &stack, &Reveal::one("m1", RevealStyle::Tint));
let at = |x: u32, y: u32| {
let i = ((y * SIZE + x) * 4) as usize;
(pixels[i], pixels[i + 1], pixels[i + 2])
};
let (r, g, _) = at(SIZE / 2, SIZE / 2);
assert!(r > g + 40, "the mask should read red, got r={r} g={g}");
assert!(
g > 20,
"and not opaque — the photograph under it is what the tint is judged \
against, got g={g}"
);
let (r, g, b) = at(1, 1);
assert!(
(120..=136).contains(&r) && r == g && g == b,
"outside the mask the photograph is untouched, got ({r}, {g}, {b})"
);
}
/// TRACES: FR-DEV-19c
/// An outline draws where the mask stops and nowhere else — which is the
/// point of it, since the other two styles cover the detail the boundary has
/// to be judged against.
#[test]
fn an_outline_draws_the_boundary_and_not_the_interior() {
let Some(ctx) = ctx() else {
eprintln!("no adapter; skipping");
return;
};
let mut stack = MaskStack::new();
stack.push(MaskLayer::new("m1", middle()));
let pixels = render_revealing(&ctx, &stack, &Reveal::one("m1", RevealStyle::Edge));
// Where the line landed, along the row through the centre. Searched
// rather than sampled at one place: the radial's edge crosses this row
// about 9.6 pixels out from the middle on a 32px frame, and asserting a
// particular pixel would be asserting the rounding.
let (at, brightest) = (SIZE / 2..SIZE)
.map(|x| (x, luma_at(&pixels, x, SIZE / 2)))
.max_by_key(|&(_, v)| v)
.expect("the row is not empty");
assert!(
brightest > 160,
"there should be a line somewhere on this row, brightest was {brightest}"
);
assert!(
(SIZE / 2 + 7..=SIZE / 2 + 12).contains(&at),
"and it should be on the mask's boundary, not somewhere else: x={at}"
);
assert_eq!(
luma_at(&pixels, SIZE / 2, SIZE / 2),
128,
"the picture inside the mask is untouched"
);
assert_eq!(
luma_at(&pixels, 1, 1),
128,
"and so is the picture outside it"
);
}
/// TRACES: FR-DEV-19c
/// Two masks shown at once come out in two colours, each where its own mask
/// is — which is what makes "where do these meet" a question the screen can
/// answer.
#[test]
fn two_shown_masks_are_drawn_each_in_its_own_colour() {
use dr_pipeline::mask::RevealedLayer;
let Some(ctx) = ctx() else {
eprintln!("no adapter; skipping");
return;
};
// A left half and a right half, as two brush layers with one fat dab each.
let half = |id: &str, x: f32| {
let mut layer = MaskLayer::new(id, MaskSource::brush());
paint(&mut layer, 0, false, &[(x, 0.5)]);
layer
};
let mut stack = MaskStack::new();
stack.push(half("left", 0.2));
stack.push(half("right", 0.8));
let reveal = Reveal {
layers: vec![
RevealedLayer {
layer: "left".into(),
colour: [1.0, 0.0, 0.0],
},
RevealedLayer {
layer: "right".into(),
colour: [0.0, 0.0, 1.0],
},
],
style: RevealStyle::Alpha,
};
let pixels = render_revealing(&ctx, &stack, &reveal);
let at = |x: u32| {
let i = ((SIZE / 2 * SIZE + x) * 4) as usize;
(pixels[i], pixels[i + 1], pixels[i + 2])
};
let (r, _, b) = at(SIZE / 5);
assert!(
r > 200 && b < 40,
"the left mask reads red, got r={r} b={b}"
);
let (r, _, b) = at(SIZE * 4 / 5);
assert!(
b > 200 && r < 40,
"the right mask reads blue, got r={r} b={b}"
);
let (r, g, b) = at(SIZE / 2);
assert!(
r < 40 && g < 40 && b < 40,
"between them, alpha shows black: ({r}, {g}, {b})"
);
}
+29
View File
@@ -849,6 +849,35 @@ impl EditGraph {
) )
} }
/// TRACES: FR-DEV-19c
/// [`Self::compose_for`], with one layer's mask drawn over the picture.
///
/// **The screen's composition, and only the screen's.** The reveal is not
/// on the graph and cannot be: it is how a photographer is looking at an
/// edit, not part of one, so it arrives as an argument to the one call
/// that draws the canvas. Every other path through this type composes
/// without it and could not ask for it if it wanted to.
///
/// The revealed layer renders whether or not it carries an adjustment —
/// which is the whole point, since a fresh selection carries none — so the
/// mask array must be rasterised for the same `reveal`. See
/// [`crate::mask::MaskStack::rendered`] for what the two have to agree on.
pub fn compose_revealing(
&self,
output: dr_types::ColourSpace,
reveal: Option<&crate::mask::Reveal>,
) -> ComposedShader {
crate::operation::compose_full_revealing(
&self.ops,
&self.framing,
output,
&self.masks,
&self.spots,
&self.warps,
reveal,
)
}
/// TRACES: FR-DSP-1 /// TRACES: FR-DSP-1
/// How this render relates to the file it stands for. /// How this render relates to the file it stands for.
/// ///
+325 -10
View File
@@ -1760,6 +1760,89 @@ impl MaskLayer {
} }
} }
/// TRACES: FR-DEV-19c
/// How a mask is drawn when the photographer asks to see it.
///
/// Three, because they answer three different questions and no one of them
/// answers all three — which is the argument for offering a choice rather than
/// picking the best one.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum RevealStyle {
/// The mask over the photograph in a flat colour. What every editor's
/// photographers already expect, and the only style that answers "is this
/// selecting the right thing" while the picture is still visible.
Tint,
/// The mask alone, white on black. For judging an edge, which a tint over
/// a busy photograph cannot be read against.
Alpha,
/// The boundary outlined over the untouched picture. For checking
/// registration against detail the other two hide — the same reasoning the
/// region overlay's white outline already carries.
Edge,
}
impl RevealStyle {
/// In the order the interface offers them.
pub const ALL: [RevealStyle; 3] = [Self::Tint, Self::Alpha, Self::Edge];
}
/// TRACES: FR-DEV-19c
/// One layer whose mask is being shown, and the colour it is shown in.
#[derive(Debug, Clone, PartialEq)]
pub struct RevealedLayer {
/// Which layer, by id. By id rather than by slot because the slot is
/// derived from which layers render, and that is decided *by* this — see
/// [`MaskStack::rendered`].
pub layer: String,
/// Linear RGB in the output space's primaries. Each shown mask has its
/// own, because two masks in one colour are one mask as far as the eye
/// can tell, and telling a sky from the building in front of it is what
/// showing them together is for.
pub colour: [f32; 3],
}
/// TRACES: FR-DEV-19c
/// The layers whose masks are being shown, and how.
///
/// Several at once, each in its own colour, one style for all of them: a tint
/// beside an outline beside an alpha would be three pictures that cannot be
/// read against each other, where three tints in three colours are one.
///
/// **Never part of an edit.** It is not stored on [`MaskStack`] and it does
/// not travel with the graph: it is passed to the one composition that draws
/// the screen, so an export, a thumbnail and the neutral probe are
/// structurally unable to reveal anything. A flag on the stack would have been
/// fewer parameters and would have tinted every exported file red.
#[derive(Debug, Clone, PartialEq)]
pub struct Reveal {
/// In no particular order; the stack's order is what they draw in.
pub layers: Vec<RevealedLayer>,
pub style: RevealStyle,
}
impl Reveal {
/// One layer, shown red — what a test wants and what nothing else does.
pub fn one(layer: impl Into<String>, style: RevealStyle) -> Self {
Self {
layers: vec![RevealedLayer {
layer: layer.into(),
colour: [0.85, 0.10, 0.15],
}],
style,
}
}
/// The colour `id` is shown in, or `None` when it is not shown.
pub fn colour_of(&self, id: &str) -> Option<[f32; 3]> {
self.layers.iter().find(|l| l.layer == id).map(|l| l.colour)
}
/// Whether anything at all would be drawn.
pub fn is_empty(&self) -> bool {
self.layers.is_empty()
}
}
/// The ordered stack of local adjustments. /// The ordered stack of local adjustments.
#[derive(Debug, Clone, Default, PartialEq)] #[derive(Debug, Clone, Default, PartialEq)]
pub struct MaskStack { pub struct MaskStack {
@@ -1838,6 +1921,34 @@ impl MaskStack {
self.active().count() self.active().count()
} }
/// TRACES: FR-DEV-19c
/// [`Self::active`], plus the layer being looked at.
///
/// A selection with no adjustment on it yet is not active — it changes no
/// pixel, so it occupies no mask slot and the rasteriser never draws it.
/// That is right for rendering and exactly wrong for *showing* the mask,
/// which is the state a photographer is in for the whole of the time
/// between choosing a subject and deciding what to do to it.
///
/// So this is the sequence both halves walk whenever a reveal is in play,
/// and the index within it is the texture-array slot — the same contract
/// [`Self::active`] carries, and the reason the rasteriser, the composer
/// and the field builder must all be given the same `reveal` or none of
/// them. Two of them disagreeing shows as an adjustment applied through
/// another layer's mask.
pub fn rendered<'a>(
&'a self,
reveal: Option<&'a Reveal>,
) -> impl Iterator<Item = &'a MaskLayer> {
self.layers
.iter()
.filter(move |l| l.is_active() || reveal.is_some_and(|r| r.colour_of(&l.id).is_some()))
}
pub fn rendered_count(&self, reveal: Option<&Reveal>) -> usize {
self.rendered(reveal).count()
}
/// Whether any layer changes any pixel. /// Whether any layer changes any pixel.
pub fn is_neutral(&self) -> bool { pub fn is_neutral(&self) -> bool {
self.active_count() == 0 self.active_count() == 0
@@ -1858,25 +1969,54 @@ pub(crate) struct LayerShader {
pub uniform_values: Vec<f32>, pub uniform_values: Vec<f32>,
pub body: String, pub body: String,
pub helpers: Vec<crate::operation::Helper>, pub helpers: Vec<crate::operation::Helper>,
/// TRACES: FR-DEV-19c
/// The block that draws one layer's mask over the finished picture, empty
/// when nothing is being revealed.
///
/// Kept apart from `body` because it belongs at the other end of the
/// shader. Everything in `body` runs on scene-referred colour in the
/// working space, where a flat tint would then be pushed through the base
/// curve and the camera matrix and arrive as some other colour, and a
/// white-on-black alpha would arrive as neither. This runs after the
/// output transform, so what is written is what is seen.
pub reveal: String,
} }
/// Emit the WGSL for every active layer. /// Emit the WGSL for every layer that renders, and for the mask being looked
/// at.
/// ///
/// `slot` is the layer's index in the mask texture array, matching /// `slot` is the layer's index in the mask texture array, matching
/// [`MaskStack::active`]. /// [`MaskStack::rendered`] — the revealed layer renders whether or not it has
pub(crate) fn compose_layers(stack: &MaskStack) -> LayerShader { /// an adjustment on it, which is why the two are one sequence and why every
/// other half of the pipeline has to be given the same `reveal` for the slots
/// to mean the same thing.
pub(crate) fn compose_layers_revealing(stack: &MaskStack, reveal: Option<&Reveal>) -> LayerShader {
let mut out = LayerShader { let mut out = LayerShader {
uniform_fields: String::new(), uniform_fields: String::new(),
uniform_values: Vec::new(), uniform_values: Vec::new(),
body: String::new(), body: String::new(),
helpers: Vec::new(), helpers: Vec::new(),
reveal: String::new(),
}; };
if stack.active().next().is_some() { if stack.rendered(reveal).next().is_some() {
out.helpers.push(MASK_SAMPLER); out.helpers.push(MASK_SAMPLER);
} }
for (slot, layer) in stack.active().enumerate() { for (slot, layer) in stack.rendered(reveal).enumerate() {
if let Some((r, colour)) = reveal.and_then(|r| Some((r, r.colour_of(&layer.id)?))) {
// Alpha begins from black, once, before the first mask lands on
// it: the style is "the masks alone", and the photograph is what
// it leaves out.
if out.reveal.is_empty() && r.style == RevealStyle::Alpha {
out.reveal.push_str(
"\n // ==== showing masks alone: the photograph goes first ====\n c = vec3<f32>(0.0);\n",
);
}
out.reveal
.push_str(&reveal_block(slot, layer, r.style, colour));
}
let prefix = format!("mask{slot}"); let prefix = format!("mask{slot}");
let _ = writeln!( let _ = writeln!(
@@ -1973,6 +2113,84 @@ pub(crate) fn compose_layers(stack: &MaskStack) -> LayerShader {
out out
} }
/// TRACES: FR-DEV-19c
/// The WGSL that draws one layer's mask over the finished picture.
///
/// # Why this is not two uniforms
///
/// The slot and the style are written into the source, so turning the reveal
/// on, off, or onto another layer recompiles the fused shader. That is a
/// button press rather than a frame — and the alternative costs more than it
/// saves: a uniform can select a slot, but it cannot conjure one for a layer
/// that is not rendering, and the whole reason this exists is that a selection
/// with no adjustment on it yet is exactly that layer. So the composition
/// changes either way, and a uniform would only have added a branch per pixel
/// on top of it.
///
/// **Everything below runs after the output transform.** `c` is already in the
/// output space's primaries and still linear — the clip and the encode come
/// after — which is what makes a stated colour arrive as itself.
fn reveal_block(slot: usize, layer: &MaskLayer, style: RevealStyle, colour: [f32; 3]) -> String {
let prefix = format!("mask{slot}");
let colour = format!(
"vec3<f32>({:.4}, {:.4}, {:.4})",
colour[0], colour[1], colour[2]
);
let mut out = format!(
"\n // ==== showing mask {slot}: {} ====\n //\n // Not part of the\
\n // photograph: this is the mask itself, drawn because someone asked to\
\n // see it. Nothing downstream of the screen composes this shader.\n {{\n",
layer.display_name()
);
// The shaped mask, exactly as the layer above applied it — the same two
// uniforms, in the same order. A reveal that showed the raw slice would
// draw a different mask from the one doing the work, which is worse than
// showing none: it would send a photographer to fix an edge that is
// already where they want it.
let _ = writeln!(out, " var m = sample_mask(uv_src, {slot});");
let _ = writeln!(
out,
" m = select(m, 1.0 - m, u.{prefix}_invert > 0.5);"
);
let _ = writeln!(out, " m = clamp(m * u.{prefix}_opacity, 0.0, 1.0);");
let body = match style {
// A bit over half strength. Half is the strength every editor settled
// on for the same reason: past it the tint is opaque enough to hide
// the thing being judged, and below it a mask over a bright sky
// cannot be seen at all.
RevealStyle::Tint => " c = mix(c, COLOUR, m * 0.55);",
// Onto the black the section began with, at full strength: this is
// the mask itself, and where two overlap the later one lands on top,
// which is the order they composite in.
RevealStyle::Alpha => " c = mix(c, COLOUR, m);",
// The gradient's magnitude, over the untouched picture. Central
// differences one texel apart in the *mask's* own grid, so the outline
// is one mask texel wide however far the view is zoomed in — the
// boundary's position is the thing being checked, and a line that grew
// with the zoom would hide it.
RevealStyle::Edge => {
" let texel = 1.0 / vec2<f32>(textureDimensions(masks));\n\
\x20 let dx = sample_mask(uv_src + vec2<f32>(texel.x, 0.0), SLOT)\n\
\x20 - sample_mask(uv_src - vec2<f32>(texel.x, 0.0), SLOT);\n\
\x20 let dy = sample_mask(uv_src + vec2<f32>(0.0, texel.y), SLOT)\n\
\x20 - sample_mask(uv_src - vec2<f32>(0.0, texel.y), SLOT);\n\
\x20 // Doubled so a soft edge, whose gradient is spread over many\n\
\x20 // texels and therefore shallow everywhere, still draws a line.\n\
\x20 let edge = clamp(2.0 * sqrt(dx * dx + dy * dy), 0.0, 1.0);\n\
\x20 c = mix(c, COLOUR, edge);"
}
};
let _ = writeln!(
out,
"{}",
body.replace("SLOT", &slot.to_string())
.replace("COLOUR", &colour)
);
let _ = writeln!(out, " }}");
out
}
/// A stable fingerprint of a segmentation, for [`MaskSource::Regions`]. /// A stable fingerprint of a segmentation, for [`MaskSource::Regions`].
/// ///
/// Built from the things that change what a region id *means* — the proxy /// Built from the things that change what a region id *means* — the proxy
@@ -2021,7 +2239,7 @@ mod tests {
let mut stack = MaskStack::new(); let mut stack = MaskStack::new();
stack.push(layer); stack.push(layer);
assert!(stack.is_neutral()); assert!(stack.is_neutral());
assert_eq!(compose_layers(&stack).body, ""); assert_eq!(compose_layers_revealing(&stack, None).body, "");
} }
#[test] #[test]
@@ -2107,7 +2325,7 @@ mod tests {
stack.push(lit_layer("m1", 1.0)); stack.push(lit_layer("m1", 1.0));
stack.push(lit_layer("m2", -1.0)); stack.push(lit_layer("m2", -1.0));
let shader = compose_layers(&stack); let shader = compose_layers_revealing(&stack, None);
assert!(shader.body.contains("sample_mask(uv_src, 0)")); assert!(shader.body.contains("sample_mask(uv_src, 0)"));
assert!(shader.body.contains("sample_mask(uv_src, 1)")); assert!(shader.body.contains("sample_mask(uv_src, 1)"));
assert!(shader.body.contains("u.mask0_opacity")); assert!(shader.body.contains("u.mask0_opacity"));
@@ -2124,7 +2342,7 @@ mod tests {
stack.push(off); stack.push(off);
stack.push(lit_layer("m2", -1.0)); stack.push(lit_layer("m2", -1.0));
let shader = compose_layers(&stack); let shader = compose_layers_revealing(&stack, None);
assert!( assert!(
shader.body.contains("sample_mask(uv_src, 0)"), shader.body.contains("sample_mask(uv_src, 0)"),
"the one active layer must use slot 0, not slot 1" "the one active layer must use slot 0, not slot 1"
@@ -2132,13 +2350,110 @@ mod tests {
assert!(!shader.body.contains("sample_mask(uv_src, 1)")); assert!(!shader.body.contains("sample_mask(uv_src, 1)"));
} }
/// TRACES: FR-DEV-19c
/// The slot the reveal is given has to be the slot the layer renders
/// through, and a revealed layer renders even with nothing done to it.
///
/// Both halves in one assertion because the failure is the pair coming
/// apart: a reveal pointed at a slot the rasteriser did not draw shows
/// whatever was last in that slice, which reads as the mask being wrong
/// rather than as the reveal being wrong.
#[test]
fn a_revealed_layer_takes_a_slot_of_its_own() {
let mut stack = MaskStack::new();
stack.push(lit_layer("m1", 1.0));
// No adjustment, so this changes no pixel and would ordinarily render
// through no slot at all.
stack.push(MaskLayer::new("m2", MaskSource::brush()));
let reveal = Reveal::one("m2", RevealStyle::Alpha);
let shader = compose_layers_revealing(&stack, Some(&reveal));
assert_eq!(
stack.rendered_count(Some(&reveal)),
2,
"the layer being looked at renders alongside the active one"
);
assert!(
shader.reveal.contains("sample_mask(uv_src, 1)"),
"the reveal must read slot 1, which is where m2 renders"
);
assert!(
shader.reveal.contains("u.mask1_opacity"),
"and shape it with that layer's own uniforms, not another's"
);
}
/// Two masks shown together are drawn each in its own colour, in stack
/// order — which is what makes a sky and the building in front of it two
/// things on screen rather than one red shape.
#[test]
fn each_shown_mask_is_drawn_in_its_own_colour() {
let mut stack = MaskStack::new();
stack.push(MaskLayer::new("sky", MaskSource::brush()));
stack.push(MaskLayer::new("wall", MaskSource::brush()));
let reveal = Reveal {
layers: vec![
RevealedLayer {
layer: "wall".into(),
colour: [0.0, 0.0, 1.0],
},
RevealedLayer {
layer: "sky".into(),
colour: [1.0, 0.0, 0.0],
},
],
style: RevealStyle::Tint,
};
let shader = compose_layers_revealing(&stack, Some(&reveal));
let sky = shader
.reveal
.find("vec3<f32>(1.0000, 0.0000, 0.0000)")
.expect("the sky's red is in the shader");
let wall = shader
.reveal
.find("vec3<f32>(0.0000, 0.0000, 1.0000)")
.expect("the wall's blue is in the shader");
assert!(
sky < wall,
"drawn in stack order, not in the order they were asked for"
);
assert!(
shader.reveal.contains("sample_mask(uv_src, 0)")
&& shader.reveal.contains("sample_mask(uv_src, 1)"),
"each reads its own slot"
);
}
/// A composition nobody asked to see a mask through draws none.
#[test]
fn nothing_is_revealed_unless_it_was_asked_for() {
let mut stack = MaskStack::new();
stack.push(lit_layer("m1", 1.0));
assert!(compose_layers_revealing(&stack, None).reveal.is_empty());
}
/// A reveal aimed at a layer that is not in the stack is not a slot, and
/// must not become one.
#[test]
fn a_reveal_naming_no_layer_reveals_nothing() {
let mut stack = MaskStack::new();
stack.push(lit_layer("m1", 1.0));
let reveal = Reveal::one("gone", RevealStyle::Tint);
assert_eq!(stack.rendered_count(Some(&reveal)), 1);
assert!(compose_layers_revealing(&stack, Some(&reveal))
.reveal
.is_empty());
}
#[test] #[test]
fn each_layer_gets_its_own_uniforms() { fn each_layer_gets_its_own_uniforms() {
let mut stack = MaskStack::new(); let mut stack = MaskStack::new();
stack.push(lit_layer("m1", 1.0)); stack.push(lit_layer("m1", 1.0));
stack.push(lit_layer("m2", -1.0)); stack.push(lit_layer("m2", -1.0));
let shader = compose_layers(&stack); let shader = compose_layers_revealing(&stack, None);
assert!(shader.uniform_fields.contains("mask0_exposure_")); assert!(shader.uniform_fields.contains("mask0_exposure_"));
assert!(shader.uniform_fields.contains("mask1_exposure_")); assert!(shader.uniform_fields.contains("mask1_exposure_"));
assert_eq!( assert_eq!(
@@ -2156,7 +2471,7 @@ mod tests {
fn the_inner_block_shadows_c_and_copies_back() { fn the_inner_block_shadows_c_and_copies_back() {
let mut stack = MaskStack::new(); let mut stack = MaskStack::new();
stack.push(lit_layer("m1", 1.0)); stack.push(lit_layer("m1", 1.0));
let body = compose_layers(&stack).body; let body = compose_layers_revealing(&stack, None).body;
assert!(body.contains("var masked = c;")); assert!(body.contains("var masked = c;"));
assert!(body.contains("var c = masked;")); assert!(body.contains("var c = masked;"));
+77 -2
View File
@@ -551,6 +551,28 @@ pub fn compose_full(
masks: &MaskStack, masks: &MaskStack,
spots: &crate::spot::SpotSet, spots: &crate::spot::SpotSet,
warps: &[Box<dyn crate::lens::Warp>], warps: &[Box<dyn crate::lens::Warp>],
) -> ComposedShader {
compose_full_revealing(ops, framing, output, masks, spots, warps, None)
}
/// TRACES: FR-DEV-19c
/// [`compose_full`], with one layer's mask drawn over the finished picture.
///
/// Separate from [`compose_full`] rather than an argument on it, and that is
/// the safety property rather than a convenience: the reveal is a thing the
/// screen does, and every other consumer of the pipeline — the exporter, the
/// thumbnail, the neutral probe — calls the function that has no way to ask
/// for it. A flag reachable from the graph would have been one forgotten reset
/// away from a red tint baked into an exported file.
#[allow(clippy::too_many_arguments)]
pub fn compose_full_revealing(
ops: &[Box<dyn Operation>],
framing: &Framing,
output: ColourSpace,
masks: &MaskStack,
spots: &crate::spot::SpotSet,
warps: &[Box<dyn crate::lens::Warp>],
reveal: Option<&crate::mask::Reveal>,
) -> ComposedShader { ) -> ComposedShader {
// The lens corrections, composed into one coordinate transform. Beside // The lens corrections, composed into one coordinate transform. Beside
// `framing` because they are the other half of the same stage: framing // `framing` because they are the other half of the same stage: framing
@@ -715,7 +737,13 @@ pub fn compose_full(
// from. Their uniforms follow the global ops' in the block for the same // from. Their uniforms follow the global ops' in the block for the same
// reason those follow framing's — slot order is emission order, and // reason those follow framing's — slot order is emission order, and
// nothing addresses a slot by number. // nothing addresses a slot by number.
let layers = crate::mask::compose_layers(masks); let layers = crate::mask::compose_layers_revealing(masks, reveal);
// TRACES: FR-DEV-19c
// Held apart from the body, because it belongs after the output transform
// rather than among the operations — see `mask::LayerShader::reveal`.
// Empty for every composition nobody is looking at a mask through, which
// is all of them but the screen's.
let reveal_block = layers.reveal.clone();
uniform_fields.push_str(&layers.uniform_fields); uniform_fields.push_str(&layers.uniform_fields);
uniform_values.extend_from_slice(&layers.uniform_values); uniform_values.extend_from_slice(&layers.uniform_values);
body.push_str(&layers.body); body.push_str(&layers.body);
@@ -975,7 +1003,7 @@ fn main(@builtin(global_invocation_id) gid: vec3<u32>) {{
c = mix(c, neutral, clipped); c = mix(c, neutral, clipped);
}} }}
{body} {body}
{rendering_tail}{to_output} {rendering_tail}{to_output}{reveal_block}
{store} {store}
}} }}
", ",
@@ -1569,6 +1597,53 @@ mod tests {
); );
} }
/// TRACES: FR-DEV-19c
/// **The property that keeps a reveal off an exported file.**
///
/// `compose_full` is the entry point the exporter, the thumbnail and the
/// neutral probe all use, and it has no argument that could ask for a
/// mask overlay. Only `compose_full_revealing` does, and only the canvas
/// calls it. Asserted rather than left to the type signature because the
/// tempting simplification — a flag on the graph — would type-check, be
/// shorter, and bake a red tint into every file the photographer sold.
#[test]
fn an_ordinary_composition_cannot_draw_a_mask_over_the_picture() {
use crate::mask::{MaskLayer, MaskSource, Reveal, RevealStyle};
let mut stack = MaskStack::new();
let mut layer = MaskLayer::new("m1", MaskSource::brush());
layer.set_param("exposure", crate::descriptor::ParamId("exposure"), 1.0);
stack.push(layer);
let plain = compose_full(
&crate::ops::chain(),
&Framing::new(),
ColourSpace::Srgb,
&stack,
&crate::spot::SpotSet::new(),
&[],
);
assert!(
!plain.source.contains("==== showing mask"),
"an export must never carry the overlay"
);
let shown = compose_full_revealing(
&crate::ops::chain(),
&Framing::new(),
ColourSpace::Srgb,
&stack,
&crate::spot::SpotSet::new(),
&[],
Some(&Reveal::one("m1", RevealStyle::Tint)),
);
assert!(shown.source.contains("==== showing mask"));
assert_ne!(
plain.structure_hash, shown.structure_hash,
"two different shaders must not share a pipeline cache entry"
);
}
/// Distortion alone samples once; chromatic aberration samples three times. /// Distortion alone samples once; chromatic aberration samples three times.
/// ///
/// `splits_channels` is the whole reason for this test. Lateral CA fetches /// `splits_channels` is the whole reason for this test. Lateral CA fetches
+171
View File
@@ -299,6 +299,13 @@ pub const STRICTNESS_MAX: f32 = 8.0;
/// smooth enough to sit on [`VARIANCE_FLOOR`], where a real one has noise and /// smooth enough to sit on [`VARIANCE_FLOOR`], where a real one has noise and
/// therefore a real spread, which moves every crossing down together. The /// therefore a real spread, which moves every crossing down together. The
/// ordering survives that; the exact placement is what the slider is for. /// ordering survives that; the exact placement is what the slider is for.
///
/// **Not where a new layer starts.** That warning turned out to be an
/// understatement — on a real photograph this position removes most of every
/// category that is not sky, and the measurements are in
/// [`Refinement::gentle`], which is what a layer starts at instead. What this
/// constant still names is the midpoint of the control's travel, and the
/// synthetic frame the tests hold it against.
pub const STRICTNESS_DEFAULT: f32 = 4.0; pub const STRICTNESS_DEFAULT: f32 = 4.0;
/// Why a refinement could not be built. /// Why a refinement could not be built.
@@ -773,8 +780,88 @@ impl Refinement {
.map(|&w| (w * 255.0).round().clamp(0.0, 255.0) as u8) .map(|&w| (w * 255.0).round().clamp(0.0, 255.0) as u8)
.collect() .collect()
} }
/// The strongest strictness this photograph can be started at without the
/// category disappearing.
///
/// # Why a constant could not do this job
///
/// [`STRICTNESS_DEFAULT`] was measured on the synthetic frame the tests
/// build, and its own note warns that a real photograph's noise "moves
/// every crossing down together". It moves them a great deal further than
/// that reads. Measured on seven ordinary frames, `4.0` — half scale, the
/// position a control would naturally start at — removes:
///
/// | category | weight removed at 4.0 |
/// |----------|----------------------|
/// | sky | 0% – 6% |
/// | vegetation | 18% – 91% |
/// | ground | 36% – 93% |
/// | architecture | 76% – **99.5%** |
///
/// So a layer created at the constant is an *empty mask* on most
/// photographs that are not mostly sky, and empty is indistinguishable
/// from broken: the adjustment moves and no pixel changes. That is the
/// whole of the fault, and it cannot be fixed by choosing a smaller
/// constant — the useful position is 5.0 on one frame and below 1.0 on the
/// next, because a nat of evidence means different things over a smooth
/// sky and over a stone facade.
///
/// # What this does instead
///
/// It asks the photograph. Walking down from half scale, the first rung
/// whose gate takes no more than [`GENTLE_CUT`] of the category's weight
/// is the answer, and [`STRICTNESS_OFF`] is the answer when none of them
/// does — the model's own outline, which is never wrong about *where the
/// category is*, only about where it stops.
///
/// Downwards rather than upwards because the friendly case is the common
/// one and it exits on the first rung: a sky costs one `apply`, and only a
/// frame the refinement disagrees with pays for all four.
///
/// This is a starting position and not a limit. The slider still offers
/// the whole range, and it is the control's job to let a photographer go
/// past what this considered safe.
pub fn gentle(&self, coverage: &[u8]) -> f32 {
let total: f64 = coverage.iter().map(|&c| c as f64).sum();
if total <= 0.0 {
return STRICTNESS_OFF;
}
for &strictness in GENTLE_LADDER {
let kept: f64 = self
.apply_coverage(coverage, strictness)
.iter()
.map(|&c| c as f64)
.sum();
if (total - kept) / total <= GENTLE_CUT as f64 {
return strictness;
}
}
STRICTNESS_OFF
}
} }
/// How much of a category's weight a *starting* strictness may take.
///
/// A sixth, and the number is doing one job: separating "the gate tidied the
/// edge" from "the gate ate the category". A twenty-proxy-pixel boundary
/// around a subject covering a fifth of the frame is a few percent of its
/// weight, so a refinement doing what it is for lands well under this; the
/// failures measured on [`Refinement::gentle`]'s table are all at 76% and
/// above. Nothing sits near the line, which is what makes it safe to state as
/// a constant rather than fit.
const GENTLE_CUT: f32 = 1.0 / 6.0;
/// The rungs [`Refinement::gentle`] tries, strongest first.
///
/// Whole nats, because that is the unit the evidence is denominated in and the
/// spacing the sweep in `examples/scene.rs` is read at. Stopping at half scale
/// rather than at [`STRICTNESS_MAX`]: past there a category's own dominant
/// colours have gone, and a photograph on which 8.0 removed under a sixth of
/// the weight would be one where the gate is finding nothing to cut and the
/// starting position may as well be gentler.
const GENTLE_LADDER: &[f32] = &[4.0, 3.0, 2.0, 1.0];
/// The eight-neighbourhood, as offsets. /// The eight-neighbourhood, as offsets.
/// ///
/// Eight rather than four because a watershed on a four-neighbourhood produces /// Eight rather than four because a watershed on a four-neighbourhood produces
@@ -1478,6 +1565,90 @@ mod tests {
); );
} }
/// The coverage buffer as the application holds it: one byte a pixel.
fn quantised(weights: &[f32]) -> Vec<u8> {
weights
.iter()
.map(|&w| (w.clamp(0.0, 1.0) * 255.0).round() as u8)
.collect()
}
/// What fraction of a category's weight a strictness takes away.
fn removed(r: &Refinement, coverage: &[u8], strictness: f32) -> f32 {
let total: f64 = coverage.iter().map(|&c| c as f64).sum();
let kept: f64 = r
.apply_coverage(coverage, strictness)
.iter()
.map(|&c| c as f64)
.sum();
((total - kept) / total) as f32
}
/// The promise a starting position has to keep: whatever it chooses, the
/// category is still there afterwards.
///
/// This is the fault it exists to fix, stated as an assertion. A fixed
/// strictness removed 76% to 99.5% of `architecture` on real photographs
/// — a mask that is empty on arrival, and indistinguishable from a broken
/// one, because the adjustment moves and no pixel changes.
#[test]
fn a_starting_strictness_never_empties_the_category() {
let opts = RefineOptions::default();
for (rect, colour, what) in [(FLAG, RED, "flag"), (CLOUD, WHITE, "cloud")] {
let (rgb, weights) = with(rect, colour);
let refinement = Refinement::compute(&weights, &rgb, EDGE, EDGE, CELL, &opts)
.expect("the frame has both sides");
let coverage = quantised(&weights);
let start = refinement.gentle(&coverage);
let cut = removed(&refinement, &coverage, start);
assert!(
cut <= GENTLE_CUT + 1e-3,
"the {what} frame starts at {start}, which takes {:.1}% of the category",
cut * 100.0
);
}
}
/// And it must not answer with zero out of caution.
///
/// A starting position that is always "off" would be a safe way of not
/// having the feature. On the frame the module was built for — a red flag
/// inside the sky — there *is* a strictness that takes the flag and leaves
/// the sky, and this has to find it.
#[test]
fn a_gentle_start_still_takes_the_flag_out_of_the_sky() {
let (rgb, weights) = with(FLAG, RED);
let refinement =
Refinement::compute(&weights, &rgb, EDGE, EDGE, CELL, &RefineOptions::default())
.expect("the flag frame has both sides");
let coverage = quantised(&weights);
let start = refinement.gentle(&coverage);
assert!(start > STRICTNESS_OFF, "gave up rather than choosing");
let refined = refinement.apply(&weights, start);
let inside = mean(&refined, EDGE, FLAG_CORE);
assert!(inside < 0.2, "the flag should be cut out, got {inside}");
let sky = mean(&refined, EDGE, (8, 8, 60, 60));
assert!(sky > 0.8, "the sky around it should survive, got {sky}");
}
/// A category nothing has claimed has nothing to judge, and asking must
/// not divide by its zero total.
#[test]
fn an_empty_category_starts_at_off() {
let (rgb, weights) = with(FLAG, RED);
let refinement =
Refinement::compute(&weights, &rgb, EDGE, EDGE, CELL, &RefineOptions::default())
.expect("the flag frame has both sides");
assert_eq!(
refinement.gentle(&vec![0u8; EDGE * EDGE]),
STRICTNESS_OFF,
"nothing to cut back"
);
}
/// Zero strictness is exactly the model's own weighting. /// Zero strictness is exactly the model's own weighting.
/// ///
/// The control's off position has to be the old behaviour bit for bit, or /// The control's off position has to be the old behaviour bit for bit, or
+2 -2
View File
@@ -19,8 +19,8 @@ pub use place::{Place, PlaceScope, Screen, StoredFilter};
pub use selector::{ColourLabel, DateSelector, FlagState, Selector, Tier}; pub use selector::{ColourLabel, DateSelector, FlagState, Selector, Tier};
pub use settings::{ pub use settings::{
CacheSettings, CollisionPolicy, ColourSpace, DevelopSettings, ExportFormat, ExportSettings, CacheSettings, CollisionPolicy, ColourSpace, DevelopSettings, ExportFormat, ExportSettings,
ExportTarget, GroupNavigation, ImportSettings, LibrarySettings, OutputSharpening, ScreenSize, ExportTarget, FaceDetector, GroupNavigation, ImportSettings, LibrarySettings, OutputSharpening,
Settings, SizingMode, ScreenSize, Settings, SizingMode,
}; };
pub use time::{ pub use time::{
civil_from_unix, civil_from_unix_at, format_date, parse_date, unix_from_civil, Civil, civil_from_unix, civil_from_unix_at, format_date, parse_date, unix_from_civil, Civil,
+127
View File
@@ -157,6 +157,96 @@ pub struct FaceSettings {
/// dropped by this, whatever their size: those are the user's judgements and /// dropped by this, whatever their size: those are the user's judgements and
/// a display preference does not overrule them (FR-CULL-12). /// a display preference does not overrule them (FR-CULL-12).
pub min_group_size: u32, pub min_group_size: u32,
/// Which graph finds the faces. See [`FaceDetector`].
pub detector: FaceDetector,
}
/// TRACES: FR-CULL-8
/// Which SCRFD graph the indexing pass detects with.
///
/// Three exports of one architecture, differing only in how much computation
/// they spend, and docs/faces.md §12.3 is the measurement that made this a
/// choice rather than a constant: over the same photographs the cheapest one
/// misses the small faces in a group and reports a dog a dozen times, the
/// middle one finds 14% more faces for 12% more time, and the largest a
/// further 12% for three times the cost. Which of those is the right trade
/// depends on the machine doing the sweep — a desktop left running overnight
/// and a tablet on a battery want different answers — so it is a setting,
/// per device, like the rest of this file.
///
/// # A detector is half of a model id
///
/// Every face row, run marker, sync shard and calibration is keyed by
/// `faces.model_id` (catalog.md §10.1), and the schema's whole reason for
/// carrying that column is that *a model change is a new id and a re-index*
/// rather than a silent change under existing data. A detector change is a
/// model change: it decides which faces exist and where the landmarks that
/// align them land. So each variant names its own pipeline, and choosing
/// another one puts every image back in the queue and shows the People screen
/// for the new pipeline — empty until the sweep has run, with confirmed names
/// carried across by `record_detections`' box overlap.
///
/// The first variant's id is the bare embedder name, because that is the id
/// every library indexed before this setting existed was written under;
/// making it `scrfd_500m+w600k_mbf` would have told those libraries they had
/// never been indexed.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Serialize, Deserialize)]
pub enum FaceDetector {
/// 2.5 GFLOPs. The measured sweet spot: nearly the same speed, and a
/// cleaner set of faces.
#[serde(rename = "scrfd_2.5g")]
Scrfd2_5g,
/// 10 GFLOPs. The most faces, at three times the time per image.
#[serde(rename = "scrfd_10g")]
Scrfd10g,
/// 500 MFLOPs. What every library was indexed with until now.
///
/// Last, because `other` has to be: a file written by a build that knows
/// a fourth detector loads as this one rather than throwing every other
/// setting away with it. [`FaceDetector::ALL`] is the display order.
#[default]
#[serde(rename = "scrfd_500m", other)]
Scrfd500m,
}
impl FaceDetector {
/// In the order the settings page offers them: cheapest first.
pub const ALL: [FaceDetector; 3] = [
FaceDetector::Scrfd500m,
FaceDetector::Scrfd2_5g,
FaceDetector::Scrfd10g,
];
/// The shape-fixed export's file name, as `tools/fix-face-model-shapes.sh`
/// writes it and every packager installs it.
pub fn file_name(self) -> &'static str {
match self {
FaceDetector::Scrfd500m => "scrfd_500m_640.onnx",
FaceDetector::Scrfd2_5g => "scrfd_2.5g_640.onnx",
FaceDetector::Scrfd10g => "scrfd_10g_640.onnx",
}
}
/// The `faces.model_id` this detector's pipeline writes under.
///
/// The embedder is the same `w600k_mbf` in every case; see the type note
/// for why the first is bare and the others are qualified.
pub fn model_id(self) -> &'static str {
match self {
FaceDetector::Scrfd500m => "w600k_mbf",
FaceDetector::Scrfd2_5g => "scrfd_2.5g+w600k_mbf",
FaceDetector::Scrfd10g => "scrfd_10g+w600k_mbf",
}
}
/// What the picker calls it.
pub fn label(self) -> &'static str {
match self {
FaceDetector::Scrfd500m => "Fast",
FaceDetector::Scrfd2_5g => "Balanced",
FaceDetector::Scrfd10g => "Thorough",
}
}
} }
impl FaceSettings { impl FaceSettings {
@@ -195,6 +285,7 @@ impl Default for FaceSettings {
// a setting, so an existing library regroups identically until the // a setting, so an existing library regroups identically until the
// user moves it. // user moves it.
min_group_size: 2, min_group_size: 2,
detector: FaceDetector::default(),
} }
} }
} }
@@ -1537,6 +1628,42 @@ mod tests {
assert_eq!(s.faces, FaceSettings::default()); assert_eq!(s.faces, FaceSettings::default());
} }
/// The detector every existing library was indexed with must keep the id
/// those libraries were written under, or an upgrade would report every
/// one of them un-indexed.
#[test]
fn the_default_detector_keeps_the_legacy_model_id() {
assert_eq!(FaceDetector::default(), FaceDetector::Scrfd500m);
assert_eq!(FaceDetector::default().model_id(), "w600k_mbf");
}
/// Two pipelines must never share an id: the whole point of the column is
/// that their faces are not interchangeable.
#[test]
fn every_detector_has_its_own_model_id_and_file() {
let ids: std::collections::HashSet<_> =
FaceDetector::ALL.iter().map(|d| d.model_id()).collect();
assert_eq!(ids.len(), FaceDetector::ALL.len());
let files: std::collections::HashSet<_> =
FaceDetector::ALL.iter().map(|d| d.file_name()).collect();
assert_eq!(files.len(), FaceDetector::ALL.len());
}
#[test]
fn the_detector_round_trips_and_an_unknown_one_falls_back() {
let mut s = Settings::default();
s.faces.detector = FaceDetector::Scrfd10g;
let text = serde_json::to_string(&s).unwrap();
assert!(text.contains(r#""detector":"scrfd_10g""#));
let back: Settings = serde_json::from_str(&text).unwrap();
assert_eq!(back.faces.detector, FaceDetector::Scrfd10g);
let newer = r#"{"faces":{"detector":"scrfd_99g"}}"#;
let s: Settings =
serde_json::from_str(newer).expect("unknown detector should not refuse the file");
assert_eq!(s.faces.detector, FaceDetector::Scrfd500m);
}
#[test] #[test]
fn sanitise_leaves_a_real_destination_alone() { fn sanitise_leaves_a_real_destination_alone() {
let mut s = Settings::default(); let mut s = Settings::default();
+25
View File
@@ -68,11 +68,36 @@ SO="${CACHE}/target/jniLibs/${ABI}/libdarkroom.so"
# at /work and the cache's target directory at /work/target-android, so every # at /work and the cache's target directory at /work/target-android, so every
# default in that script already points at the right place. # default in that script already points at the right place.
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
#
# Release signing, when asked for. assemble-apk.sh selects it by the presence
# of KEYSTORE_PASS (see its header), and the keystore has to be reachable from
# inside the container, so a host path in KEYSTORE is copied under the mounted
# target directory for the duration of the build and removed after. The
# passwords travel as environment, never as arguments -- docs/android-signing.md
# has the incantation.
# ---------------------------------------------------------------------------
echo "==> packaging APK" echo "==> packaging APK"
SIGNING_ENV=()
CONTAINER_KEYSTORE=""
if [[ -n "${KEYSTORE_PASS:-}" ]]; then
[[ -f "${KEYSTORE:-}" ]] || { echo "error: KEYSTORE_PASS is set but KEYSTORE is not a file" >&2; exit 1; }
install -m 600 "${KEYSTORE}" "${CACHE}/target/release.keystore"
CONTAINER_KEYSTORE="${CACHE}/target/release.keystore"
SIGNING_ENV=(
KEYSTORE=/work/target-android/release.keystore
KEYSTORE_PASS="${KEYSTORE_PASS}"
KEY_PASS="${KEY_PASS:-${KEYSTORE_PASS}}"
KEY_ALIAS="${KEY_ALIAS:?KEY_ALIAS is required when KEYSTORE_PASS is set}"
)
fi
"${HERE}/build.sh" env \ "${HERE}/build.sh" env \
ABI="${ABI}" RUST_TARGET="${RUST_TARGET}" \ ABI="${ABI}" RUST_TARGET="${RUST_TARGET}" \
DARKROOM_DEBUGGABLE="${DARKROOM_DEBUGGABLE:-}" \ DARKROOM_DEBUGGABLE="${DARKROOM_DEBUGGABLE:-}" \
"${SIGNING_ENV[@]}" \
/work/docker/android/assemble-apk.sh /work/docker/android/assemble-apk.sh
if [[ -n "${CONTAINER_KEYSTORE}" ]]; then
rm -f "${CONTAINER_KEYSTORE}"
fi
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# 3. Install from the host. # 3. Install from the host.
+16
View File
@@ -35,6 +35,22 @@ beyond telling everybody to uninstall and reinstall.
line, which keeps them out of shell history. Back the `.jks` up somewhere that line, which keeps them out of shell history. Back the `.jks` up somewhere that
is not this repository and not the machine that builds it. is not this repository and not the machine that builds it.
**The key exists, since 2026-09-11.** It was made as above, with a random
password, and the four secrets are loaded. The local copy is at
`~/.config/darkroom/signing/` on the development desktop — `darkroom-release.jks`
beside `storepass` and `keypass`, all mode 600 in a mode 700 directory. That
copy is what `package.sh` can sign with locally:
D=~/.config/darkroom/signing
KEYSTORE="$D/darkroom-release.jks" KEYSTORE_PASS="$(cat "$D/storepass")" \
KEY_ALIAS=darkroom ./docker/android/package.sh --install
Before it existed, every build — CI and local alike — was signed with a
throwaway debug key, and a debug key is exactly as durable as the cache
directory it lives in: the local one was regenerated the night the cache was
cleared, at which point no build anywhere could install over the device's copy.
Any device that received a build from before this date has to uninstall once.
## Loading the secrets ## Loading the secrets
base64 -w0 darkroom-release.jks > /tmp/ks.b64 base64 -w0 darkroom-release.jks > /tmp/ks.b64
+6 -1
View File
@@ -742,7 +742,12 @@ CREATE TABLE faces (
x REAL NOT NULL, y REAL NOT NULL, w REAL NOT NULL, h REAL NOT NULL, x REAL NOT NULL, y REAL NOT NULL, w REAL NOT NULL, h REAL NOT NULL,
landmarks BLOB, -- 5 × (x, y) f32, the alignment input landmarks BLOB, -- 5 × (x, y) f32, the alignment input
detector_confidence REAL NOT NULL, detector_confidence REAL NOT NULL,
embedding BLOB NOT NULL, -- 512 × f16, L2-normalised embedding BLOB NOT NULL, -- 512 × f16, the raw model output; re-normalised on load
-- Length of that vector: the model's own reading of how recognisable the
-- crop was, and the gate on whether this face may be compared *against*
-- (faces.md §6, §9). NULL for a face stored as a unit vector before it
-- was kept.
quality REAL,
-- Which model produced this. An embedding is only comparable to others -- Which model produced this. An embedding is only comparable to others
-- from the same model; mixing them silently yields nonsense similarities. -- from the same model; mixing them silently yields nonsense similarities.
model_id TEXT NOT NULL, model_id TEXT NOT NULL,
+87 -4
View File
@@ -447,9 +447,28 @@ every measured number in §1's table was produced with `/128`. The difference is
almost certainly immaterial, but "almost certainly" is not a reason to pick silently — write `/128` to almost certainly immaterial, but "almost certainly" is not a reason to pick silently — write `/128` to
match the numbers we have, and settle it with one back-to-back run in §12. match the numbers we have, and settle it with one back-to-back run in §12.
Output is 512 floats; **L2-normalise before storing**, so every downstream comparison is a dot product Output is 512 floats. Every downstream comparison is a dot product over the **unit** vector, and the
and no code path has to remember to normalise. The reference clamps the norm at 1e-6 before dividing, reference L2-normalises before storing so that no code path has to remember to. The reference clamps
which costs nothing and removes a NaN path. the norm at 1e-6 before dividing, which costs nothing and removes a NaN path.
**Keep the length.** The norm the normalisation divides out is not noise. ArcFace trains the
direction of its output and nothing else, and the magnitude it leaves behind grows with how much of a
face the model could make out — MagFace (Meng et al., CVPR 2021) made that the training objective,
and the plain ArcFace heads this crate runs already show it, weaker but usable. A blur, an occlusion,
a hard profile or a badly lit crop comes out short. On the reference library `w600k_mbf`'s norms run
from about 8 on a blur to the high 20s on a clean portrait.
So the store holds the **raw** vector, not the unit one — `dr_face::Embedded::to_f16_bytes` — and
readers re-normalise on load, which they had to do anyway (below). f16 keeps the same three figures
of a component whatever the vector's length, so this costs nothing in precision. The length is also
kept beside the blob as `faces.quality`, for the readers that never load the vector (the People
screen), and it is `NULL` for a face stored as a unit vector before this — a unit vector reads as a
length of one, and one is not "unmeasured".
What the number does is in §9: a face whose quality is under **`MIN_GALLERY_QUALITY` = 14** is still
placed, but is never what another face is compared *against*. The screen shows it as "Quality 17.3",
dimmed below the floor, so a user asking why a group did not gather the rest of a person can see that
none of its members can vouch for anyone.
Some exports of these graphs are fp16 in, fp16 out. The reference detects this from the graph's Some exports of these graphs are fp16 in, fp16 out. The reference detects this from the graph's
declared element type rather than from the filename; worth porting, because the alternative failure is declared element type rather than from the filename; worth porting, because the alternative failure is
@@ -461,7 +480,8 @@ between a match and a non-match, and the halving matters because these rows are
contemplates optionally syncing. contemplates optionally syncing.
Re-normalise on load after the f16 widen. It is one pass over 512 floats and it removes a class of Re-normalise on load after the f16 widen. It is one pass over 512 floats and it removes a class of
drift that is otherwise invisible. drift that is otherwise invisible — and, since the blob is raw, it is what turns the stored vector
back into the unit one every comparison expects.
--- ---
@@ -785,6 +805,22 @@ here, and it is the phone and tablet story that should decide whether it gets bu
- **Confirmed faces are anchors.** A confirmation is user data (FR-CULL-12) and clustering never - **Confirmed faces are anchors.** A confirmation is user data (FR-CULL-12) and clustering never
moves it. Two clusters each containing confirmations of *different* people cannot merge; a cluster moves it. Two clusters each containing confirmations of *different* people cannot merge; a cluster
containing confirmations of one person absorbs suggestions but never reassigns the confirmed. containing confirmations of one person absorbs suggestions but never reassigns the confirmed.
- **A short embedding is never a reference.** The length of the raw vector is the model's own
reading of the crop (§6), and a short one sits near the middle of the sphere, matching a little of
everybody — one of those in a group is a bridge to the next group over. So the population is
split: faces at or above `MIN_GALLERY_QUALITY` are the **gallery** and cluster as described below;
faces under it are **probes**, each measured against the finished groups and placed in the one it
fits by the same average-link rule under the same two constraints — but measured against gallery
members only, never against another probe, and once placed never part of what the next face is
measured against. Two probes are never paired at all, and `neighbours` drops those pairs before
anything downstream sees them. A probe's confidence (§9.1) is computed from the references it
matched; a reference's confidence hears nothing from a probe. A face whose quality was never
recorded is admitted to the gallery — a rule that cannot be checked admits rather than excludes —
and the next indexing pass **measures** it: `faces_unmeasured` lists every image holding one, and
each such face is embedded again from the native render with the landmarks it already has, the raw
vector written over the old one and its id, box and identity untouched
(`faces::record_measurements`). No detector runs and no suggestion is lost — the cost is the
original fetched once more, since the length exists only at the moment of embedding.
**The algorithm.** Constrained average-link agglomeration over the probability graph, merging while **The algorithm.** Constrained average-link agglomeration over the probability graph, merging while
the average pairwise probability exceeds **0.9** and no cannot-link is violated. Average-link rather the average pairwise probability exceeds **0.9** and no cannot-link is violated. Average-link rather
@@ -1026,6 +1062,53 @@ profile rather than the pipeline. Any future timing of this subsystem is a relea
Still unmeasured: the same pass on a phone (NFR-RES-2), which does not follow from this one. Still unmeasured: the same pass on a phone (NFR-RES-2), which does not follow from this one.
### 12.3 SCRFD-2.5G and 10G against 500M · 2026-09-11
§1 chose `500M` on cost; the recall it gives up was never measured. `examples/face_detectors`
in `dr-ui` runs several detectors over the same 400 proxies, spaced evenly through the reference
library, matches boxes at IoU ≥ 0.5 against the first, and writes contact sheets of the
disagreements — because a count of extra faces says nothing until someone has looked at whether
they are faces. Both size gates off, confidence at the production 0.5, release build, reference
desktop. Sizes are the box's shorter edge in proxy pixels; the network sees 0.625 of that.
| Detector | File | Mean ms | Faces | <16 | 16–32 | 32–64 | ≥64 |
|---|---|---|---|---|---|---|---|
| `scrfd_500m` | 2.5 MB | 157 | 1,490 | 153 | 730 | 399 | 208 |
| `scrfd_2.5g` | 3.3 MB | 177 | 1,698 | 268 | 801 | 418 | 211 |
| `scrfd_10g` | 17 MB | 488 | 1,896 | 357 | 886 | 438 | 215 |
| Against `500M` | Both | Candidate only | Baseline only |
|---|---|---|---|
| `2.5G` | 1,404 | **294** (242 of them under 32 px) | 86 |
| `10G` | 1,419 | **477** (411 under 32 px) | 71 |
Two things the sheets settled that the counts cannot:
- **The candidates' extras are faces.** The hundred smallest `2.5G`-only tiles are people —
soft, small, a few motion-blurred, and almost none of them anything else. These are the group
shots and the figures in the background that §7's table predicted `500M` would lose at 640.
- **`500M`'s "extras" are mostly not.** Of the 86 faces only `500M` found, the sheet shows the
same dog a dozen times, a stop sign, a wheel, two hands, the backs of several heads and one face
upside down. The larger models did not miss these; they declined them. So `500M` is paying
twice — for the faces it cannot find *and* for the non-faces it embeds, which is exactly the
garbage-embedding-bridges-two-clusters failure `DetectOptions::confidence` is set against.
**`2.5G` is the right detector.** 12% more time for 14% more faces and a cleaner set, in a file
0.8 MB larger; `10G` finds a further 12% for 3.1× the time, which is a desktop-only price and this
is not a desktop-only feature (NFR-RES-2). It loads in tract with the same fix as `500M`
(`--input input.1=1,3,640,640`; its outputs are declared dynamic and tract infers them) and
decodes through the same nine-output path unchanged. Same licence, same `buffalo_m` release page.
**Which detector runs is a setting** — `FaceSettings::detector`, per device, on the settings page
beside the indexing button as Fast / Balanced / Thorough. All three files ship. Each detector is
its own `faces.model_id` (`w600k_mbf` for `500M`, unchanged, so nothing already indexed is
disturbed; `scrfd_2.5g+w600k_mbf` and `scrfd_10g+w600k_mbf` for the others), which is the
mechanism §2.1 always intended for a model change: the coverage figure restarts at zero under the
new id, the sweep re-detects, `record_detections` carries confirmed names across by box overlap and
drops the previous pipeline's marker for each image it revisits, and the sync shards are keyed by
the same id so a peer on another setting neither adopts nor pollutes them. The default stays `500M`
so that an upgrade changes nothing until the user chooses; the recommendation is `2.5G`.
--- ---
## 13. Order ## 13. Order
+34 -25
View File
@@ -5,7 +5,7 @@
Every entry here is extracted from the comment beside the code that implements it, so this file cannot describe a gesture the application does not have. Add one by writing a `GESTURE:` block next to the implementation; there is nowhere else to write it. Every entry here is extracted from the comment beside the code that implements it, so this file cannot describe a gesture the application does not have. Add one by writing a `GESTURE:` block next to the implementation; there is nowhere else to write it.
36 gestures, in 4 places. 37 gestures, in 4 places.
## Develop ## Develop
@@ -25,16 +25,7 @@ Sampling a neutral is the first move of the tonal pass — every colour judgemen
Anchored on the fingers' midpoint, and on the pointer, so the gesture reads as magnifying the picture rather than sliding it about. Double-tap is the way to an exact 1:1; this is the way to everything in between. Anchored on the fingers' midpoint, and on the pointer, so the gesture reads as magnifying the picture rather than sliding it about. Double-tap is the way to an exact 1:1; this is the way to everything in between.
<sub>`ui/dr-ui/ui/app.slint:1941`</sub> <sub>`ui/dr-ui/ui/app.slint:1950`</sub>
### Paint a mask by hand
- **Touch** — Choose Paint or Erase, then drag on the photograph
- **Pointer** — Choose Paint or Erase, then drag
A model's mask stops inside a shoulder and leaks into the hair, and no single edge control fixes two errors that go opposite ways. The whole stroke is one step in the history, so taking a mark back costs one press however long it took to make.
<sub>`ui/dr-ui/ui/app.slint:2014`</sub>
### Move a magnified photograph about ### Move a magnified photograph about
@@ -43,7 +34,16 @@ A model's mask stops inside a shoulder and leaks into the hair, and no single ed
Only once there is something outside the viewport to reach, which is why the cursor becomes a hand exactly then. The view is clamped to the frame: panning past the edge would show undefined area beside the photograph, and that reads as a rendering fault rather than as the end of the picture. Only once there is something outside the viewport to reach, which is why the cursor becomes a hand exactly then. The view is clamped to the frame: panning past the edge would show undefined area beside the photograph, and that reads as a rendering fault rather than as the end of the picture.
<sub>`ui/dr-ui/ui/app.slint:2147`</sub> <sub>`ui/dr-ui/ui/app.slint:2041`</sub>
### Paint a mask by hand
- **Touch** — Choose Paint or Erase, then drag on the photograph
- **Pointer** — Choose Paint or Erase, then drag
A model's mask stops inside a shoulder and leaks into the hair, and no single edge control fixes two errors that go opposite ways. The whole stroke is one step in the history, so taking a mark back costs one press however long it took to make.
<sub>`ui/dr-ui/ui/app.slint:2128`</sub>
### Take back the last change ### Take back the last change
@@ -53,7 +53,7 @@ Only once there is something outside the viewport to reach, which is why the cur
A whole drag is one step, so undo takes back a decision rather than a frame of a gesture. The list is there because arriving six steps back costs what arriving from one does. A whole drag is one step, so undo takes back a decision rather than a frame of a gesture. The list is there because arriving six steps back costs what arriving from one does.
<sub>`ui/dr-ui/ui/app.slint:2314`</sub> <sub>`ui/dr-ui/ui/app.slint:2349`</sub>
### Do it again after taking it back ### Do it again after taking it back
@@ -61,7 +61,7 @@ A whole drag is one step, so undo takes back a decision rather than a frame of a
- **Pointer** — Click it, or press Redo in the History header - **Pointer** — Click it, or press Redo in the History header
- **Keyboard** — Ctrl+Shift+Z - **Keyboard** — Ctrl+Shift+Z
<sub>`ui/dr-ui/ui/app.slint:2327`</sub> <sub>`ui/dr-ui/ui/app.slint:2362`</sub>
### Copy the settings from this photograph ### Copy the settings from this photograph
@@ -71,7 +71,7 @@ A whole drag is one step, so undo takes back a decision rather than a frame of a
The panel is the copy that has to work: a tablet has no modifier key to hold and no menu bar to hang the action from. The shortcut is an accelerator for a control that is on screen either way. The panel is the copy that has to work: a tablet has no modifier key to hold and no menu bar to hang the action from. The shortcut is an accelerator for a control that is on screen either way.
<sub>`ui/dr-ui/ui/app.slint:2360`</sub> <sub>`ui/dr-ui/ui/app.slint:2395`</sub>
### Paste the settings onto this photograph ### Paste the settings onto this photograph
@@ -81,7 +81,7 @@ The panel is the copy that has to work: a tablet has no modifier key to hold and
The button names what would be pasted — "3 adjustments", and whether the crop is coming with it — which the shortcut cannot say. Both paste the same scope. The button names what would be pasted — "3 adjustments", and whether the crop is coming with it — which the shortcut cannot say. Both paste the same scope.
<sub>`ui/dr-ui/ui/app.slint:2372`</sub> <sub>`ui/dr-ui/ui/app.slint:2407`</sub>
### Change which group of adjustments is on screen ### Change which group of adjustments is on screen
@@ -91,7 +91,7 @@ The button names what would be pasted — "3 adjustments", and whether the crop
The groups are whatever the operation set declares itself to be about, so there are as many as the pipeline has and no key can be assigned to one of them by name. Stepping is the binding that survives a node being added. The groups are whatever the operation set declares itself to be about, so there are as many as the pipeline has and no key can be assigned to one of them by name. Stepping is the binding that survives a node being added.
<sub>`ui/dr-ui/ui/app.slint:2400`</sub> <sub>`ui/dr-ui/ui/app.slint:2435`</sub>
### Look at the photograph at 1:1 ### Look at the photograph at 1:1
@@ -101,7 +101,7 @@ The groups are whatever the operation set declares itself to be about, so there
Noise reduction and capture sharpening are judgements about single pixels, and a fitted view averages several of the file's into each one on screen — so the frame looks softer than it is and the correction goes too far. The point and the magnification survive opening the next photograph, which is what makes checking the same eye across forty portraits forty keystrokes rather than forty pans. Noise reduction and capture sharpening are judgements about single pixels, and a fitted view averages several of the file's into each one on screen — so the frame looks softer than it is and the correction goes too far. The point and the magnification survive opening the next photograph, which is what makes checking the same eye across forty portraits forty keystrokes rather than forty pans.
<sub>`ui/dr-ui/ui/app.slint:2435`</sub> <sub>`ui/dr-ui/ui/app.slint:2470`</sub>
### Move to the next or previous photograph ### Move to the next or previous photograph
@@ -111,7 +111,7 @@ Noise reduction and capture sharpening are judgements about single pixels, and a
The edit on screen is saved on the way out, so stepping through a folder is as much a departure as going back to the grid and loses nothing. The edit on screen is saved on the way out, so stepping through a folder is as much a departure as going back to the grid and loses nothing.
<sub>`ui/dr-ui/ui/app.slint:2465`</sub> <sub>`ui/dr-ui/ui/app.slint:2500`</sub>
### See the photograph before you edited it ### See the photograph before you edited it
@@ -121,7 +121,7 @@ The edit on screen is saved on the way out, so stepping through a folder is as m
Held rather than toggled, and no split screen: a split halves the working image on the tablet the column was sized for, and the comparison photographers describe making is a flick back and forth. It takes no history step, so checking whether a frame is overcooked costs nothing to undo afterwards. Held rather than toggled, and no split screen: a split halves the working image on the tablet the column was sized for, and the comparison photographers describe making is a flick back and forth. It takes no history step, so checking whether a frame is overcooked costs nothing to undo afterwards.
<sub>`ui/dr-ui/ui/app.slint:2589`</sub> <sub>`ui/dr-ui/ui/app.slint:2624`</sub>
### Put one control back to its default ### Put one control back to its default
@@ -139,7 +139,16 @@ The column is 280px wide and the colour mixer alone puts thirty-six of these in
Disabling a layer is the before-and-after a local edit constantly wants, so it is one press away rather than inside the row. It is an edit and does take a history step, unlike holding "Before" — the layer really is off until it is switched back on. Disabling a layer is the before-and-after a local edit constantly wants, so it is one press away rather than inside the row. It is an edit and does take a history step, unlike holding "Before" — the layer really is off until it is switched back on.
<sub>`ui/dr-ui/ui/masks.slint:196`</sub> <sub>`ui/dr-ui/ui/masks.slint:209`</sub>
### Show or hide one mask on the photograph
- **Touch** — Tap the eye on its row
- **Pointer** — Click the eye on its row
A mask is judged by seeing where it falls, and two are judged by seeing where they meet — so each row has its own eye rather than the panel having one, and the eye is drawn in the colour the mask shows in, so the row says which shape on the picture is its. Nothing about the edit changes: this is how the photograph is looked at, and takes no history step.
<sub>`ui/dr-ui/ui/masks.slint:274`</sub>
## Collections sidebar ## Collections sidebar
@@ -188,7 +197,7 @@ Double-click is what a file manager and a Lightroom panel use for the same thing
Grouping over-merges on siblings, on parents and children, and on the same person a decade apart, so splitting is as prominent as merging. A tool that can only merge makes its own errors permanent. Grouping over-merges on siblings, on parents and children, and on the same person a decade apart, so splitting is as prominent as merging. A tool that can only merge makes its own errors permanent.
<sub>`ui/dr-ui/ui/identity.slint:130`</sub> <sub>`ui/dr-ui/ui/identity.slint:161`</sub>
### Rule on a suggested face ### Rule on a suggested face
@@ -197,7 +206,7 @@ Grouping over-merges on siblings, on parents and children, and on the same perso
A face is either the system's guess or the user's judgement, and the two are never conflated. A rejection is remembered, so the face is not suggested for that person again. The gesture note above is the whole label: a tick and a cross are only "confirm" and "reject" to someone who can see the suggestion they sit beside, and `IconButton`'s fallback would announce them as "check" and "cross" — two icon names that say nothing about which person is being ruled on. A face is either the system's guess or the user's judgement, and the two are never conflated. A rejection is remembered, so the face is not suggested for that person again. The gesture note above is the whole label: a tick and a cross are only "confirm" and "reject" to someone who can see the suggestion they sit beside, and `IconButton`'s fallback would announce them as "check" and "cross" — two icon names that say nothing about which person is being ruled on.
<sub>`ui/dr-ui/ui/identity.slint:150`</sub> <sub>`ui/dr-ui/ui/identity.slint:181`</sub>
### See a person's photographs ### See a person's photographs
@@ -206,7 +215,7 @@ A face is either the system's guess or the user's judgement, and the two are nev
This is the point of having identified anybody. Without it the screen is a filing cabinet with no drawer handles. This is the point of having identified anybody. Without it the screen is a filing cabinet with no drawer handles.
<sub>`ui/dr-ui/ui/identity.slint:571`</sub> <sub>`ui/dr-ui/ui/identity.slint:602`</sub>
### Change how faces are grouped ### Change how faces are grouped
@@ -215,7 +224,7 @@ This is the point of having identified anybody. Without it the screen is a filin
The right match confidence is a property of your library, not of the model. "What would this do?" answers for this library without writing anything; names, confirmations and the groups you have set aside are kept whatever the dials say. The right match confidence is a property of your library, not of the model. "What would this do?" answers for this library without writing anything; names, confirmations and the groups you have set aside are kept whatever the dials say.
<sub>`ui/dr-ui/ui/identity.slint:608`</sub> <sub>`ui/dr-ui/ui/identity.slint:639`</sub>
## Library grid ## Library grid
+108 -20
View File
@@ -36,12 +36,18 @@ the *whole* boundary. When the model's coverage stops two pixels inside the
shoulder and leaks four pixels into the hair, no global number fixes both, and shoulder and leaks four pixels into the hair, no global number fixes both, and
that is the ordinary case rather than a corner one. that is the ordinary case rather than a corner one.
**And you cannot see the mask.** The overlay on the canvas is **And you cannot see the mask.** *(Built — see §6.)* The overlay on the canvas
[`overlay_rgba`](../ui/dr-ui/src/segmentation.rs) — a CPU-built false-colour was [`overlay_rgba`](../ui/dr-ui/src/segmentation.rs) and nothing else — a
picture of *what the model detected*, at proxy resolution. It is not the CPU-built false-colour picture of *what the model detected*, at proxy
layer's alpha: it knows nothing of the layer's feather, its falloff, its resolution. It is not the layer's alpha: it knows nothing of the layer's
morphology, its invert, or its opacity. Nobody can refine an edge they are not feather, its falloff, its morphology, its invert, or its opacity. Nobody can
being shown. refine an edge they are not being shown.
That one turned out to be load-bearing for the other three rather than the
last of four. With no way to see a mask, choosing a category produced a layer
whose extent was invisible and whose adjustment had not been touched yet — so
the correct behaviour and the broken one look identical, and "the segmentation
does not make masks" is what it reads as from the outside.
Those four are one feature, and this is its specification. Those four are one feature, and this is its specification.
@@ -390,31 +396,100 @@ usable along hair.
## 6. Seeing the mask ## 6. Seeing the mask
The mask array is **already bound to the composed adjust shader** — this is **Status: built.** `MaskStack::rendered`, `Reveal` as a list of
almost free, and it is the first thing to build, because every other tool here `(layer, colour)`, `RevealStyle`, `EditGraph::compose_revealing`,
`MaskPass::render_revealing`; on the panel, an eye and a colour per row and one
"Show masks as" strip above the stack.
The mask array is **already bound to the composed adjust shader**, so this is
almost free, and it is the first thing to build because every other tool here
is unusable without it. is unusable without it.
Two uniforms in the composed shader: which layer to reveal (−1 for none) and ### 6.1 One correction to this section, found in the building
which style. The block is always emitted, guarded by the uniform, so switching
the overlay on is a uniform write rather than a shader recompile. The draft said "two uniforms — which layer to reveal (−1 for none) and which
style — always emitted and guarded by the uniform, so switching the overlay on
is a uniform write rather than a shader recompile". The second half of that is
not available, and the reason is the case the feature exists for.
A uniform can *select* a slot. It cannot conjure one. A layer with no
adjustment on it changes no pixel, so it is not `is_active`, so it occupies no
slice of the mask array and the rasteriser never draws it — and that is
precisely the layer a photographer wants to look at, for the whole of the time
between choosing a subject and deciding what to do to it. Revealing it means
*rendering* it, which changes the sequence of layers, which changes the
uniform block. The composition moves either way.
So the slot and the style are written into the source, and turning the reveal
on, off, or onto another layer recompiles the fused shader. That is a button
press rather than a frame, and the uniform would only have added a branch per
pixel on top of a recomposition that was happening anyway.
`MaskStack::rendered(reveal)` is the one sequence this rests on: `active()`
plus the layer being looked at. The rasteriser, the composer and the distance
field builder all index by position in it, so all three must be given the same
`reveal` — two of them disagreeing shows as an adjustment applied through
another layer's mask, which is why they take it as an argument rather than
reading a flag.
### 6.2 Where the block runs, and why not with the others
After the output transform, immediately before the clip and the encode — not
among the layer blocks. Everything there runs on scene-referred colour in the
working space, where a flat tint would be pushed through the base curve and
the camera matrix and arrive as some other colour, and an alpha's white on
black would arrive as neither.
### 6.3 Not on the graph
The reveal is an argument to `EditGraph::compose_revealing`, and
`compose_for` — which the exporter, the thumbnail and the neutral probe all
call — has no way to ask for one. A flag on the graph would have been fewer
parameters, would have type-checked, and would have been one forgotten reset
away from a red tint baked into an exported file.
Three styles, all read from the same alpha: Three styles, all read from the same alpha:
- **Tint** — the mask over the picture in a flat colour at ~50%. The default, - **Tint** — the mask over the picture in its colour at ~50%. The default,
and what every editor's photographers already expect. Red by default and and what every editor's photographers already expect. The colour is the
configurable, since a red tint over a red dress shows nothing. mask's own, chosen from the swatches on its row — which is what answers a
red tint over a red dress.
- **Alpha** — the mask alone, white on black. For judging an edge, where a - **Alpha** — the mask alone, white on black. For judging an edge, where a
tint over a busy picture cannot be read. tint over a busy picture cannot be read.
- **Edge** — the boundary outlined over the untouched picture. For checking - **Edge** — the boundary outlined over the untouched picture. For checking
registration against detail the other two hide, and the same reasoning the registration against detail the other two hide, and the same reasoning the
region overlay's white outline already carries. region overlay's white outline already carries.
**When it appears.** Automatically while a mask tool is armed, while a part **Per mask, not per selection.** The first build of this showed the *selected*
row is selected, and for ~1s after a shaping slider is released; manually from layer's mask in one global style, and it answered the wrong question. What a
a control in the Local panel. The existing `overlay-hidden` property is the photographer asks of two masks is how they meet — where the sky's edge sits
precedent and the trap it documents applies unchanged: the photographer's against the building's — and that needs both on screen at once, in colours that
switch and the automatic reveal are separate questions, and an automatic can be told apart. So each row of the stack has an eye, and each mask a colour
reveal must never silently re-arm a switch the photographer turned off. from a six-entry palette (`MASK_COLOURS` in `develop.rs`); the eye is drawn in
that colour so the row says which shape on the picture is its. The style is
the one thing that stays global, because a tint beside an outline beside an
alpha would be three pictures that cannot be read against each other. Alpha
therefore draws every shown mask, each in its colour, on black.
**When it appears.** A new layer arrives with its eye open, in the first colour
nothing else is using — making a mask is asking what it selected, and for a
subject or a category that question has no other answer on screen. Arming
Paint or Erase opens the selected layer's eye if it was closed, on the same
argument `on_part_added` makes: a stroke into an invisible mask is
indistinguishable from a tool that did nothing. Pressing a swatch opens the
eye too, since colouring a mask nobody can see would change no pixel.
Only ever *this* layer's eye, and only on an explicit action. Every other eye
keeps whatever it was set to, and nothing re-arms in the background. The
existing `overlay-hidden` property is the precedent and the trap it documents
applies unchanged: an automatic reveal that re-arms a switch somebody turned
off is worse than no automatic reveal at all.
Viewing state and not edit state: eyes and colours are on the session, not on
the layer, and a photograph reopened has every eye closed.
The ~1s reveal after a shaping slider is released, from the draft, is not
built. It is a timer rather than a decision.
The region overlay stays exactly what it is — a picture of what the model The region overlay stays exactly what it is — a picture of what the model
detected — and gains a name in the interface that says so, because two detected — and gains a name in the interface that says so, because two
@@ -597,6 +672,19 @@ the mask pass beyond the blend variants; no change to distance fields, because
a painted part needs none. *This is the whole of the user-visible ask except a painted part needs none. *This is the whole of the user-visible ask except
push and intersect,* and it is deliberately the milestone that stands alone. push and intersect,* and it is deliberately the milestone that stands alone.
Done: parts with `Union` and `Subtract`, painted parts, the tool strip, the
canvas gesture, and the overlay (§6). Outstanding: the brush HUD and cursor —
radius, hardness and flow are sliders with no ring drawn on the photograph,
so the size of the brush is a number rather than a thing you can see — and
the incremental raster of §5.4, without which a layer's whole stroke history
is redrawn per dab.
One lesson from the order it was actually built in, since §6 said it and the
build did not listen: the overlay is not the last quarter of M1, it is the
first. Parts and painting shipped without it and the result was a feature
nobody could tell was working — the panel listed a mask, the photograph showed
nothing, and every report of it came back as "the masks do not work".
**M2 — Combine properly.** `Intersect`, model and gradient and range parts, **M2 — Combine properly.** `Intersect`, model and gradient and range parts,
per-part distance fields (§5.5), the part list with its join chips, folding per-part distance fields (§5.5), the part list with its join chips, folding
two layers. two layers.
+16
View File
@@ -517,6 +517,22 @@ hardness and flow the photographer sets. Strokes are stored as normalised source
rasterised on the device, so a correction stays on what it was painted on through a crop, a zoom and rasterised on the device, so a correction stays on what it was painted on through a crop, a zoom and
an export at any size. A whole stroke is one step in the history. an export at any size. A whole stroke is one step in the history.
**FR-DEV-19c — Seeing the mask.** Each layer's mask shall be drawable over the photograph, switched
per layer by an eye on its row and drawn in a colour of that layer's own, so that several can be
shown at once and told apart; the style — a tint, an alpha, or an outline — is one setting for all
of them. What is shown is the finished mask — every part folded, with the layer's feather, falloff,
morphology, invert and opacity applied — and it is shown for a layer that carries no adjustment
yet, which is the state every mask is in for its first few seconds. No rendered output ever carries
it: the reveal is a property of looking at an edit, not of the edit, so it reaches the pipeline
through the composition that draws the canvas and through no other, and it does not travel in a
sidecar.
Nobody can refine an edge they are not being shown. Before this the only thing drawn on the
photograph was FR-DEV-3's region overlay — a false-coloured picture of what the *model detected*,
which knows nothing of a layer's shaping and nothing at all about a gradient, a range or a stroke —
so choosing a subject or a category produced a layer whose extent was invisible, and every control
in FR-DEV-19a and FR-DEV-19b acted on something the photographer could not see.
### 3.4 Display and interaction ### 3.4 Display and interaction
**FR-DSP-1 — Proxy-resolution rendering.** The develop view renders at the resolution actually **FR-DSP-1 — Proxy-resolution rendering.** The develop view renders at the resolution actually
+79 -78
View File
File diff suppressed because one or more lines are too long
+13 -6
View File
@@ -1,7 +1,7 @@
# Face models # Face models
The shape-fixed SCRFD detector and ArcFace/MobileFaceNet embedder, in LFS. One copy, packaged by The shape-fixed SCRFD detectors and the ArcFace/MobileFaceNet embedder, in LFS. One copy, packaged
every platform: by every platform:
| Platform | How it ships | Where it lands | | Platform | How it ships | Where it lands |
|---|---|---| |---|---|---|
@@ -11,18 +11,25 @@ every platform:
`library::face_models` searches the account's own directory, then the shared user directory, then `library::face_models` searches the account's own directory, then the shared user directory, then
`$XDG_DATA_DIRS` — so a pair the user placed by hand always outranks the packaged one. `$XDG_DATA_DIRS` — so a pair the user placed by hand always outranks the packaged one.
scrfd_500m_640.onnx 2.5 MB scrfd_500m_640.onnx 2.5 MB "Fast" in settings — what every library was indexed with until 0.12
scrfd_2.5g_640.onnx 3.3 MB "Balanced" — 14% more faces for 12% more time (faces.md §12.3)
scrfd_10g_640.onnx 17 MB "Thorough" — a further 12% for 3× the time
arcface_mbf_b1.onnx 13 MB arcface_mbf_b1.onnx 13 MB
Which detector runs is a per-device setting (`FaceSettings::detector`); all three are installed so
the choice exists on every platform. Each is its own `faces.model_id`, so changing it re-indexes.
**A clone without git-lfs gets a ~130-byte pointer where each model should be.** Both packagers check **A clone without git-lfs gets a ~130-byte pointer where each model should be.** Both packagers check
for exactly that and refuse, rather than shipping the pointer and failing inside tract on the user's for exactly that and refuse, rather than shipping the pointer and failing inside tract on the user's
machine. Fix it with `git lfs pull`. machine. Fix it with `git lfs pull`.
These are not what InsightFace ships. They came from `buffalo_sc.zip` and `buffalo_s.zip` on the These are not what InsightFace ships. They came from `buffalo_sc.zip`, `buffalo_m.zip`, `buffalo_l.zip`
InsightFace v0.7 release with their input dimensions pinned, because tract cannot parse either graph and `buffalo_s.zip` on the InsightFace v0.7 release with their input dimensions pinned, because tract
while they are dynamic: cannot parse any of the graphs while they are dynamic:
./tools/fix-face-model-shapes.sh det_500m.onnx models/face/scrfd_500m_640.onnx --input input.1=1,3,640,640 ./tools/fix-face-model-shapes.sh det_500m.onnx models/face/scrfd_500m_640.onnx --input input.1=1,3,640,640
./tools/fix-face-model-shapes.sh det_2.5g.onnx models/face/scrfd_2.5g_640.onnx --input input.1=1,3,640,640
./tools/fix-face-model-shapes.sh det_10g.onnx models/face/scrfd_10g_640.onnx --input input.1=1,3,640,640
./tools/fix-face-model-shapes.sh w600k_mbf.onnx models/face/arcface_mbf_b1.onnx --dim None=1 ./tools/fix-face-model-shapes.sh w600k_mbf.onnx models/face/arcface_mbf_b1.onnx --dim None=1
The weights carry a non-commercial research-only grant. They are here because this is a private The weights carry a non-commercial research-only grant. They are here because this is a private
Binary file not shown.
Binary file not shown.
+2 -2
View File
@@ -4,7 +4,7 @@
# makes `makepkg -si` in this directory install what you are actually working # makes `makepkg -si` in this directory install what you are actually working
# on. Swap `source` for a tagged tarball when there is something to release. # on. Swap `source` for a tagged tarball when there is something to release.
pkgname=darkroom pkgname=darkroom
pkgver=0.11.0 pkgver=0.12.0
# Back to 1 with the version: a new pkgver is a new archive name, so there is # Back to 1 with the version: a new pkgver is a new archive name, so there is
# nothing for makepkg to reuse and nothing for a release number to disambiguate. # nothing for makepkg to reuse and nothing for a release number to disambiguate.
pkgrel=1 pkgrel=1
@@ -61,7 +61,7 @@ package() {
# These live in LFS; a checkout without `git lfs pull` has ~130-byte # These live in LFS; a checkout without `git lfs pull` has ~130-byte
# pointers here. Installing one produces a package whose face indexing # pointers here. Installing one produces a package whose face indexing
# fails inside the graph loader on the user's machine, so refuse instead. # fails inside the graph loader on the user's machine, so refuse instead.
for _m in scrfd_500m_640.onnx arcface_mbf_b1.onnx; do for _m in scrfd_500m_640.onnx scrfd_2.5g_640.onnx scrfd_10g_640.onnx arcface_mbf_b1.onnx; do
_src="models/face/${_m}" _src="models/face/${_m}"
if [[ "$(stat -c%s "${_src}")" -lt 100000 ]]; then if [[ "$(stat -c%s "${_src}")" -lt 100000 ]]; then
echo "error: ${_m} is an LFS pointer, not a model — run: git lfs pull" >&2 echo "error: ${_m} is an LFS pointer, not a model — run: git lfs pull" >&2
@@ -157,7 +157,7 @@ modules:
# inside the graph loader on the user's machine. Refuse instead, with the # inside the graph loader on the user's machine. Refuse instead, with the
# command that fixes it. # command that fixes it.
- | - |
for m in scrfd_500m_640.onnx arcface_mbf_b1.onnx; do for m in scrfd_500m_640.onnx scrfd_2.5g_640.onnx scrfd_10g_640.onnx arcface_mbf_b1.onnx; do
if [ "$(stat -c%s "models/face/$m")" -lt 100000 ]; then if [ "$(stat -c%s "models/face/$m")" -lt 100000 ]; then
echo "error: $m is an LFS pointer, not a model — run: git lfs pull" >&2 echo "error: $m is an LFS pointer, not a model — run: git lfs pull" >&2
exit 1 exit 1
+2
View File
@@ -7,6 +7,8 @@
# The two the face pipeline needs, verified 2026-08-26 (docs/faces.md §12 M1): # The two the face pipeline needs, verified 2026-08-26 (docs/faces.md §12 M1):
# #
# ... det_500m.onnx scrfd_500m_640.onnx --input input.1=1,3,640,640 # ... det_500m.onnx scrfd_500m_640.onnx --input input.1=1,3,640,640
# ... det_2.5g.onnx scrfd_2.5g_640.onnx --input input.1=1,3,640,640
# ... det_10g.onnx scrfd_10g_640.onnx --input input.1=1,3,640,640
# ... w600k_mbf.onnx arcface_mbf_b1.onnx --dim None=1 # ... w600k_mbf.onnx arcface_mbf_b1.onnx --dim None=1
# #
# ## Why this exists # ## Why this exists
+1 -7
View File
@@ -98,13 +98,7 @@ if [[ "${COMMIT}" == "1" ]]; then
if git diff --cached --quiet; then if git diff --cached --quiet; then
echo "==> nothing changed; no commit made" echo "==> nothing changed; no commit made"
else else
git commit -q -m "Say which version this is: ${VERSION} git commit -q -m "Release ${VERSION}"
Set by tools/set-version.sh, which is the only thing that should. The
workspace, the pacman package and — through Cargo.toml at link time — the
APK all state ${VERSION}, so a bug report naming a version names one commit.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>"
echo "==> committed: $(git log --oneline -1)" echo "==> committed: $(git log --oneline -1)"
fi fi
else else
+427
View File
@@ -0,0 +1,427 @@
//! TRACES: FR-CULL-8
//! Compare face detectors over the same proxies.
//!
//! cargo run --release -p dr-ui --example face_detectors -- \
//! CATALOG.db THUMBS_DIR BASELINE.onnx CANDIDATE.onnx [CANDIDATE.onnx…] \
//! [--sample N] [--sheet DIR]
//!
//! Runs every detector over the same sample of stored proxies and reports, per
//! detector, how long it took and how many faces it found by size; then, per
//! candidate, which of those faces the baseline also found and which it did
//! not. `--sheet` writes a contact sheet of the disagreements in each direction,
//! because a count of "extra faces" says nothing until someone has looked at
//! whether they are faces.
//!
//! # What it measures, and what it cannot
//!
//! docs/faces.md §12 M4 is recall against hand-labelled faces. There are no
//! labels here, so this is the cheaper question that decides whether M4 is
//! worth the labelling: *do the detectors disagree, where, and does the
//! disagreement look like faces*. A candidate whose extras are all real faces
//! under 20 px has found the group shots the baseline lost; one whose extras
//! are ears and door handles has found nothing.
//!
//! Both size gates are off, so what is counted is what the detector *emits*
//! above its confidence, not what indexing would keep. The confidence floor
//! is the production one, because a detector that only wins below it has not
//! won anything the pipeline would see.
//!
//! The models must have had their input dims frozen first; see
//! `tools/fix-face-model-shapes.sh`.
use std::path::{Path, PathBuf};
use std::time::Instant;
use dr_catalog::Catalog;
use dr_face::{DetectOptions, Detection, Detector};
use dr_thumbs::ThumbStore;
use dr_ui::faces;
const DEFAULT_SAMPLE: usize = 400;
/// Two boxes are the same face above this overlap.
const MATCH_IOU: f32 = 0.5;
/// Size buckets, on the box's shorter edge in **proxy pixels**. The proxy is
/// 1024 on its long edge and the detector sees it letterboxed to 640, so the
/// model's own view is 0.625 of these — the smallest bucket is a face under
/// 10 px to the network.
const BUCKETS: [(f32, &str); 4] = [
(16.0, "<16"),
(32.0, "16-32"),
(64.0, "32-64"),
(f32::INFINITY, ">=64"),
];
/// Tiles on a contact sheet: this many per row, this wide.
const SHEET_COLS: usize = 10;
const SHEET_TILE: usize = 112;
const SHEET_MAX: usize = 100;
fn main() {
env_logger::init();
let args: Vec<String> = std::env::args().skip(1).collect();
let mut sample = DEFAULT_SAMPLE;
let mut sheet: Option<PathBuf> = None;
let mut positional: Vec<String> = Vec::new();
let mut i = 0;
while i < args.len() {
match args[i].as_str() {
"--sample" => {
sample = args
.get(i + 1)
.and_then(|s| s.parse().ok())
.unwrap_or_else(|| usage());
i += 2;
}
"--sheet" => {
sheet = Some(PathBuf::from(args.get(i + 1).unwrap_or_else(|| usage())));
i += 2;
}
other => {
positional.push(other.to_string());
i += 1;
}
}
}
if positional.len() < 4 {
usage();
}
let catalog = Catalog::open(Path::new(&positional[0])).unwrap_or_else(|e| {
eprintln!("cannot open catalog {}: {e}", positional[0]);
std::process::exit(1);
});
let store = ThumbStore::open(Path::new(&positional[1])).unwrap_or_else(|e| {
eprintln!("cannot open thumbnail store {}: {e}", positional[1]);
std::process::exit(1);
});
let mut detectors: Vec<(String, Detector)> = positional[2..]
.iter()
.map(|p| {
let label = Path::new(p)
.file_stem()
.map(|s| s.to_string_lossy().into_owned())
.unwrap_or_else(|| p.clone());
let t = Instant::now();
let det = Detector::from_path(p).unwrap_or_else(|e| {
eprintln!("cannot load {p}: {e}");
std::process::exit(1);
});
println!(
"{label:<20} loaded in {:>5.0} ms strides {:?}",
t.elapsed().as_secs_f64() * 1e3,
det.strides()
);
(label, det)
})
.collect();
let file_ids = sample_ids(&catalog, &store, sample);
if file_ids.is_empty() {
println!("no proxies on disk to measure — browse the library first.");
return;
}
println!(
"\n{} image(s), evenly spaced through the library\n",
file_ids.len()
);
// Gates off, confidence as shipped: see the module note.
let options = DetectOptions {
min_face_px: 0.0,
min_source_px: 0.0,
min_sharpness: 0.0,
..Default::default()
};
// Per detector: per-image timings, and per-image detections.
let n = detectors.len();
let mut times: Vec<Vec<f64>> = vec![Vec::new(); n];
let mut found: Vec<Vec<Vec<Detection>>> = vec![Vec::new(); n];
// The decoded proxies the sheet will cut from, kept only when asked for.
let mut images: Vec<(u32, u32, Vec<u8>)> = Vec::new();
let mut done = 0usize;
for id in &file_ids {
let Ok(Some(thumb)) = store.get(*id, faces::FACE_TIER) else {
continue;
};
let Ok((w, h, rgba)) = dr_thumbs::codec::decode_rgba(&thumb.bytes) else {
continue;
};
let rgb: Vec<f32> = rgba
.chunks_exact(4)
.flat_map(|p| [p[0], p[1], p[2]].map(|c| c as f32 / 255.0))
.collect();
for (k, (_, det)) in detectors.iter_mut().enumerate() {
let t = Instant::now();
let dets = det
.detect(&rgb, w as usize, h as usize, &options)
.unwrap_or_default();
times[k].push(t.elapsed().as_secs_f64() * 1e3);
found[k].push(dets);
}
if sheet.is_some() {
images.push((w, h, rgba));
}
done += 1;
if done.is_multiple_of(50) {
println!(" {done}/{} images", file_ids.len());
}
}
println!("\n{done} image(s) measured\n");
// ---- per detector: speed and what it emits -------------------------
println!(
"{:<20} {:>8} {:>8} {:>7} {}",
"detector",
"mean ms",
"p50 ms",
"faces",
BUCKETS
.iter()
.map(|(_, l)| format!("{l:>7}"))
.collect::<String>()
);
for k in 0..n {
let mut t = times[k].clone();
t.sort_by(|a, b| a.total_cmp(b));
let mean = t.iter().sum::<f64>() / t.len().max(1) as f64;
let p50 = t.get(t.len() / 2).copied().unwrap_or(0.0);
let all: Vec<&Detection> = found[k].iter().flatten().collect();
let counts = bucket_counts(all.iter().copied());
println!(
"{:<20} {mean:>8.1} {p50:>8.1} {:>7} {}",
detectors[k].0,
all.len(),
counts.iter().map(|c| format!("{c:>7}")).collect::<String>()
);
}
// ---- per candidate: agreement with the baseline ----------------------
let (base_label, _) = &detectors[0];
for k in 1..n {
let label = &detectors[k].0;
// (image index, detection) for each side of the disagreement.
let mut matched: Vec<&Detection> = Vec::new();
let mut only_candidate: Vec<(usize, &Detection)> = Vec::new();
let mut only_baseline: Vec<(usize, &Detection)> = Vec::new();
for (img, (base, cand)) in found[0].iter().zip(&found[k]).enumerate() {
let mut base_used = vec![false; base.len()];
for c in cand {
let best = base
.iter()
.enumerate()
.filter(|(bi, _)| !base_used[*bi])
.map(|(bi, b)| (bi, iou(b, c)))
.filter(|(_, v)| *v >= MATCH_IOU)
.max_by(|a, b| a.1.total_cmp(&b.1));
match best {
Some((bi, _)) => {
base_used[bi] = true;
matched.push(c);
}
None => only_candidate.push((img, c)),
}
}
for (bi, b) in base.iter().enumerate() {
if !base_used[bi] {
only_baseline.push((img, b));
}
}
}
println!("\n{label} against {base_label}:");
println!(
"{:<28} {:>7} {}",
"",
"faces",
BUCKETS
.iter()
.map(|(_, l)| format!("{l:>7}"))
.collect::<String>()
);
for (name, set) in [
("both found", matched.clone()),
(
"candidate only",
only_candidate.iter().map(|(_, d)| *d).collect(),
),
(
"baseline only",
only_baseline.iter().map(|(_, d)| *d).collect(),
),
] {
let counts = bucket_counts(set.iter().copied());
println!(
" {name:<26} {:>7} {} median conf {:.2}",
set.len(),
counts.iter().map(|c| format!("{c:>7}")).collect::<String>(),
median_confidence(&set)
);
}
if let Some(dir) = &sheet {
std::fs::create_dir_all(dir).expect("create sheet dir");
for (suffix, set) in [("extra", &only_candidate), ("missed", &only_baseline)] {
let path = dir.join(format!("{label}-{suffix}.jpg"));
match write_sheet(&path, &images, set) {
Ok(n) => println!(" {suffix:<26} {n} tile(s) -> {}", path.display()),
Err(e) => eprintln!(" {suffix}: {e}"),
}
}
}
}
}
fn usage() -> ! {
eprintln!(
"usage: face_detectors CATALOG.db THUMBS_DIR BASELINE.onnx CANDIDATE.onnx [CANDIDATE.onnx…] \
[--sample N] [--sheet DIR]"
);
std::process::exit(2);
}
/// Up to `n` file ids with a proxy on disk, evenly spaced through the library
/// rather than its first `n` — the first `n` are one trip.
fn sample_ids(catalog: &Catalog, store: &ThumbStore, n: usize) -> Vec<u64> {
let mut stmt = catalog
.connection()
.prepare(
"SELECT r.file_id FROM remote r
JOIN images i ON i.id = r.image_id
WHERE r.file_id IS NOT NULL AND i.trashed_at IS NULL
ORDER BY i.id",
)
.expect("list images");
let with_proxy: Vec<u64> = stmt
.query_map([], |r| r.get::<_, i64>(0))
.into_iter()
.flatten()
.filter_map(Result::ok)
.map(|v| v as u64)
.filter(|id| store.contains(*id, faces::FACE_TIER))
.collect();
if with_proxy.len() <= n {
return with_proxy;
}
let step = with_proxy.len() as f64 / n as f64;
(0..n)
.map(|i| with_proxy[(i as f64 * step) as usize])
.collect()
}
fn iou(a: &Detection, b: &Detection) -> f32 {
let x0 = a.bbox.0.max(b.bbox.0);
let y0 = a.bbox.1.max(b.bbox.1);
let x1 = a.bbox.2.min(b.bbox.2);
let y1 = a.bbox.3.min(b.bbox.3);
let inter = (x1 - x0).max(0.0) * (y1 - y0).max(0.0);
let union = a.width() * a.height() + b.width() * b.height() - inter;
if union <= 0.0 {
0.0
} else {
inter / union
}
}
fn bucket_counts<'a>(dets: impl Iterator<Item = &'a Detection>) -> [usize; BUCKETS.len()] {
let mut counts = [0usize; BUCKETS.len()];
for d in dets {
let edge = d.width().min(d.height());
let b = BUCKETS
.iter()
.position(|(limit, _)| edge < *limit)
.unwrap_or(BUCKETS.len() - 1);
counts[b] += 1;
}
counts
}
fn median_confidence(dets: &[&Detection]) -> f32 {
if dets.is_empty() {
return 0.0;
}
let mut c: Vec<f32> = dets.iter().map(|d| d.confidence).collect();
c.sort_by(|a, b| a.total_cmp(b));
c[c.len() / 2]
}
/// A grid of face tiles, each the box enlarged by half again so there is
/// context to judge by, resampled to a fixed tile whatever its source size.
/// Smallest faces first: those are the ones the question is about.
fn write_sheet(
path: &Path,
images: &[(u32, u32, Vec<u8>)],
set: &[(usize, &Detection)],
) -> Result<usize, String> {
if set.is_empty() {
return Ok(0);
}
let mut ordered: Vec<&(usize, &Detection)> = set.iter().collect();
ordered.sort_by(|a, b| {
let ea = a.1.width().min(a.1.height());
let eb = b.1.width().min(b.1.height());
ea.total_cmp(&eb)
});
ordered.truncate(SHEET_MAX);
let rows = ordered.len().div_ceil(SHEET_COLS);
let (sw, sh) = (SHEET_COLS * SHEET_TILE, rows * SHEET_TILE);
let mut sheet = vec![0u8; sw * sh * 4];
for (i, (img, d)) in ordered.iter().enumerate() {
let (w, h, rgba) = &images[*img];
let (w, h) = (*w as usize, *h as usize);
let cx = (d.bbox.0 + d.bbox.2) * 0.5;
let cy = (d.bbox.1 + d.bbox.3) * 0.5;
let half = d.width().max(d.height()) * 0.75;
let (tx0, ty0) = ((i % SHEET_COLS) * SHEET_TILE, (i / SHEET_COLS) * SHEET_TILE);
for ty in 0..SHEET_TILE {
for tx in 0..SHEET_TILE {
let sx = cx - half + (tx as f32 + 0.5) / SHEET_TILE as f32 * half * 2.0;
let sy = cy - half + (ty as f32 + 0.5) / SHEET_TILE as f32 * half * 2.0;
let px = bilinear(rgba, w, h, sx, sy);
let o = ((ty0 + ty) * sw + tx0 + tx) * 4;
sheet[o..o + 4].copy_from_slice(&px);
}
}
}
let bytes = dr_thumbs::codec::encode_rgba(sw as u32, sh as u32, &sheet)
.map_err(|e| format!("encode: {e}"))?;
std::fs::write(path, bytes).map_err(|e| format!("write {}: {e}", path.display()))?;
Ok(ordered.len())
}
/// RGBA sample at a continuous position; black outside the image.
fn bilinear(rgba: &[u8], w: usize, h: usize, x: f32, y: f32) -> [u8; 4] {
if x < 0.0 || y < 0.0 || x >= (w - 1) as f32 || y >= (h - 1) as f32 {
return [0, 0, 0, 255];
}
let (x0, y0) = (x.floor() as usize, y.floor() as usize);
let (fx, fy) = (x - x0 as f32, y - y0 as f32);
let at = |xx: usize, yy: usize| &rgba[(yy * w + xx) * 4..(yy * w + xx) * 4 + 4];
let (p00, p10, p01, p11) = (
at(x0, y0),
at(x0 + 1, y0),
at(x0, y0 + 1),
at(x0 + 1, y0 + 1),
);
let mut out = [0u8; 4];
for c in 0..3 {
let top = p00[c] as f32 * (1.0 - fx) + p10[c] as f32 * fx;
let bot = p01[c] as f32 * (1.0 - fx) + p11[c] as f32 * fx;
out[c] = (top * (1.0 - fy) + bot * fy).round() as u8;
}
out[3] = 255;
out
}
+6 -5
View File
@@ -286,15 +286,16 @@ fn tune_thresholds(catalog: &Catalog) {
let model = dr_face::ModelId::new(MODEL_ID.to_string()); let model = dr_face::ModelId::new(MODEL_ID.to_string());
let mut candidates = Vec::with_capacity(stored.len()); let mut candidates = Vec::with_capacity(stored.len());
for (face_id, image_id, blob, crop_px) in stored { for f in stored {
let Some(emb) = dr_face::Embedding::from_f16_bytes(model.clone(), &blob) else { let Some(emb) = dr_face::Embedding::from_f16_bytes(model.clone(), &f.embedding) else {
continue; continue;
}; };
candidates.push(dr_face::Candidate { candidates.push(dr_face::Candidate {
face: face_id.0, face: f.face.0,
image: image_id.0, image: f.image.0,
embedding: emb.v.to_vec(), embedding: emb.v.to_vec(),
crop_px, crop_px: f.crop_px,
quality: f.quality,
confirmed_person: None, confirmed_person: None,
}); });
} }
+23 -2
View File
@@ -113,6 +113,11 @@ pub fn spawn_sync(
catalog_path: PathBuf, catalog_path: PathBuf,
place_path: PathBuf, place_path: PathBuf,
scratch: PathBuf, scratch: PathBuf,
// TRACES: FR-CULL-8
// Which face pipeline's shards to export and adopt. From the settings
// page by way of the library controller, because a shard written under
// one detector's faces must not be read back as another's.
face_model_id: String,
) -> std::sync::mpsc::Receiver<SyncMessage> { ) -> std::sync::mpsc::Receiver<SyncMessage> {
let (tx, rx) = std::sync::mpsc::channel(); let (tx, rx) = std::sync::mpsc::channel();
@@ -143,6 +148,7 @@ pub fn spawn_sync(
&catalog_path, &catalog_path,
&place_path, &place_path,
&scratch, &scratch,
&face_model_id,
&tx, &tx,
) )
.await .await
@@ -160,6 +166,10 @@ pub fn spawn_sync(
rx rx
} }
// Eight, because a sync touches eight distinct things — the same reason
// `spawn_face_sweep` carries the allow: bundling them into a struct would name
// nothing that exists.
#[allow(clippy::too_many_arguments)]
async fn run( async fn run(
backend: &dyn RemoteBackend, backend: &dyn RemoteBackend,
root: &str, root: &str,
@@ -167,6 +177,7 @@ async fn run(
catalog_path: &Path, catalog_path: &Path,
place_path: &Path, place_path: &Path,
scratch: &Path, scratch: &Path,
face_model_id: &str,
tx: &std::sync::mpsc::Sender<SyncMessage>, tx: &std::sync::mpsc::Sender<SyncMessage>,
) -> Result<SyncReport, String> { ) -> Result<SyncReport, String> {
let mut report = SyncReport::default(); let mut report = SyncReport::default();
@@ -180,7 +191,16 @@ async fn run(
sync_shards(backend, &base, thumbs_dir, scratch, &mut report).await?; sync_shards(backend, &base, thumbs_dir, scratch, &mut report).await?;
let _ = tx.send(SyncMessage::Status("checking faces…".into())); let _ = tx.send(SyncMessage::Status("checking faces…".into()));
sync_face_shards(backend, &base, catalog_path, scratch, &mut report, tx).await?; sync_face_shards(
backend,
&base,
catalog_path,
scratch,
face_model_id,
&mut report,
tx,
)
.await?;
let _ = tx.send(SyncMessage::Status("checking collections…".into())); let _ = tx.send(SyncMessage::Status("checking collections…".into()));
sync_catalog(backend, &base, catalog_path, scratch, &mut report).await?; sync_catalog(backend, &base, catalog_path, scratch, &mut report).await?;
@@ -361,6 +381,7 @@ async fn sync_face_shards(
base: &RemotePath, base: &RemotePath,
catalog_path: &Path, catalog_path: &Path,
scratch: &Path, scratch: &Path,
face_model_id: &str,
report: &mut SyncReport, report: &mut SyncReport,
tx: &std::sync::mpsc::Sender<SyncMessage>, tx: &std::sync::mpsc::Sender<SyncMessage>,
) -> Result<(), String> { ) -> Result<(), String> {
@@ -382,7 +403,7 @@ async fn sync_face_shards(
}; };
let client = store.client_id().to_string(); let client = store.client_id().to_string();
let model = crate::identity_ui::MODEL_ID; let model = face_model_id;
// ---- everything this device has detected, into the shards ------------ // ---- everything this device has detected, into the shards ------------
if let Ok(catalog) = dr_catalog::Catalog::open(catalog_path) { if let Ok(catalog) = dr_catalog::Catalog::open(catalog_path) {
+321 -13
View File
@@ -683,6 +683,32 @@ impl CropAspect {
} }
} }
/// TRACES: FR-DEV-19c
/// How one layer's mask is shown on the canvas.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
struct MaskView {
shown: bool,
/// Index into [`MASK_COLOURS`].
colour: usize,
}
/// TRACES: FR-DEV-19c
/// The colours a mask may be shown in, in linear sRGB.
///
/// Six, chosen to be told apart at half strength over a photograph rather
/// than to be pretty: red and green and blue at the corners, and the three
/// between them. Exposed so the panel draws its swatches from the same table
/// the shader is handed, and a seventh colour is one line here and nowhere
/// else.
pub const MASK_COLOURS: [[f32; 3]; 6] = [
[0.85, 0.10, 0.15],
[0.15, 0.80, 0.25],
[0.20, 0.45, 1.00],
[0.95, 0.80, 0.10],
[0.90, 0.20, 0.85],
[0.15, 0.85, 0.90],
];
pub struct DevelopSession { pub struct DevelopSession {
/// This session's name, for work that outlives the frame it started on. /// This session's name, for work that outlives the frame it started on.
id: SessionId, id: SessionId,
@@ -865,6 +891,28 @@ pub struct DevelopSession {
selected_spot: Option<String>, selected_spot: Option<String>,
/// Whether to draw the false-coloured region overlay. /// Whether to draw the false-coloured region overlay.
show_overlay: bool, show_overlay: bool,
/// TRACES: FR-DEV-19c
/// How shown masks are drawn — one style for all of them.
///
/// Interface state, like `show_overlay` beside it and `active_masks` above
/// — it changes no pixel of the photograph, it is not in the sidecar and
/// it is not on the undo stack. It reaches the pipeline as an argument to
/// the one composition that draws the canvas, which is what makes an
/// export structurally unable to carry it (`EditGraph::compose_revealing`).
reveal_style: dr_pipeline::mask::RevealStyle,
/// TRACES: FR-DEV-19c
/// Per layer: whether its mask is shown, and in what colour.
///
/// Per layer rather than "the selected one", because the question a
/// photographer asks of two masks is how they meet — where the sky's edge
/// sits against the building's — and that needs both on screen at once,
/// in colours that can be told apart. Keyed by id, and an id that is no
/// longer in the stack is simply never asked for; `reveal` walks the
/// stack, not this map.
///
/// Viewing state and not edit state, for the reason the style is: it does
/// not travel in a sidecar, so a photograph reopened has every eye closed.
mask_views: std::collections::HashMap<String, MaskView>,
/// Which attribute the panel is filtered to, or all of them. /// Which attribute the panel is filtered to, or all of them.
/// ///
/// `None` is "show everything" and is what a frontend that ignores /// `None` is "show everything" and is what a frontend that ignores
@@ -1020,6 +1068,8 @@ impl DevelopSession {
), ),
selected_spot: None, selected_spot: None,
show_overlay: false, show_overlay: false,
reveal_style: dr_pipeline::mask::RevealStyle::Tint,
mask_views: std::collections::HashMap::new(),
active_tab: None, active_tab: None,
curve_channel: 0, curve_channel: 0,
display_space: dr_types::ColourSpace::Srgb, display_space: dr_types::ColourSpace::Srgb,
@@ -1958,7 +2008,29 @@ impl DevelopSession {
} }
}; };
for layer in self.graph.masks().active() { // TRACES: FR-DEV-19c
// The reveal is in the key because it is in the *sequence*: revealing
// a layer with no adjustment on it gives that layer a slot, which
// renumbers every field after it. Leaving it out is the bug where
// clicking a subject shows the mask of whichever layer happened to be
// beneath it.
let reveal = self.reveal();
mix(reveal.as_ref().map_or(0, |r| {
// Every shown id, not only which are shown: a layer joining the
// shown set renumbers every slot after it, exactly as one joining
// the active set does. Colours are left out — they change what
// the shader draws, not which field it draws through.
let mut h: u64 = 1;
for l in &r.layers {
for b in l.layer.as_bytes() {
h = h.wrapping_mul(31).wrapping_add(*b as u64);
}
h = h.wrapping_mul(31).wrapping_add(0x1f);
}
h
}));
for layer in self.graph.masks().rendered(reveal.as_ref()) {
match &layer.base().source { match &layer.base().source {
MaskSource::Subject { index, .. } => { MaskSource::Subject { index, .. } => {
mix(1); mix(1);
@@ -2147,10 +2219,13 @@ impl DevelopSession {
} }
}; };
// In `active()` order, because that is the order the rasteriser walks // In `rendered()` order, because that is the order the rasteriser
// and the order it indexes these by. // walks and the order it indexes these by — the revealed layer
// included, which is why the reveal is asked for here and folded into
// the key above.
let reveal = self.reveal();
let mut fields: Vec<Vec<f32>> = Vec::new(); let mut fields: Vec<Vec<f32>> = Vec::new();
for layer in self.graph.masks().active() { for layer in self.graph.masks().rendered(reveal.as_ref()) {
let field = match self.layer_coverage(layer, pw, ph) { let field = match self.layer_coverage(layer, pw, ph) {
Some(coverage) => { Some(coverage) => {
dr_segment::Shaped::build( dr_segment::Shaped::build(
@@ -2193,7 +2268,16 @@ impl DevelopSession {
} }
fn rasterise_masks(&mut self) -> bool { fn rasterise_masks(&mut self) -> bool {
if self.graph.masks().is_neutral() { // TRACES: FR-DEV-19c
// A stack that changes no pixel is normally not worth a pass — except
// when one of its layers is being looked at, which is exactly the
// state a fresh selection is in. Asking `rendered_count` rather than
// `is_neutral` is what makes "click a category, see its mask" work at
// all: before it, the array was never rasterised, the slice the reveal
// samples held whatever was last in it, and the answer was a blank
// photograph.
let reveal = self.reveal();
if self.graph.masks().rendered_count(reveal.as_ref()) == 0 {
return false; return false;
} }
// **Source space, at the segmentation's proxy size** — not the // **Source space, at the segmentation's proxy size** — not the
@@ -2239,13 +2323,14 @@ impl DevelopSession {
// No label field: region masks were the watershed's, and nothing // No label field: region masks were the watershed's, and nothing
// produces one any more. A stored layer that still names regions is // produces one any more. A stored layer that still names regions is
// skipped by the rasteriser rather than drawn wrong. // skipped by the rasteriser rather than drawn wrong.
pass.render( pass.render_revealing(
self.graph.masks(), self.graph.masks(),
None, None,
subjects, subjects,
Some(source.as_ref()), Some(source.as_ref()),
pw, pw,
ph, ph,
reveal.as_ref(),
) )
.inspect_err(|e| log::warn!("mask rasterisation failed: {e}")) .inspect_err(|e| log::warn!("mask rasterisation failed: {e}"))
.is_ok() .is_ok()
@@ -2426,6 +2511,160 @@ impl DevelopSession {
.unwrap_or_default() .unwrap_or_default()
} }
// ----------------------------------------------------------------------
// Seeing the mask (FR-DEV-19c)
// ----------------------------------------------------------------------
/// TRACES: FR-DEV-19c
/// What the canvas should draw over the photograph, if anything.
///
/// Every layer whose eye is open, in stack order, each in its colour —
/// and `None` when no eye is, so the rasteriser and the composer can
/// take the path they always took.
///
/// Rebuilt per call rather than kept in step with the stack, because it
/// is a walk over at most eight layers — cheaper than the invalidation a
/// cached copy would need every time a layer is added, removed, renamed
/// or reordered.
pub(crate) fn reveal(&self) -> Option<dr_pipeline::mask::Reveal> {
use dr_pipeline::mask::{Reveal, RevealedLayer};
let layers: Vec<RevealedLayer> = self
.graph
.masks()
.layers()
.iter()
.filter_map(|l| {
let view = self.mask_views.get(&l.id).filter(|v| v.shown)?;
Some(RevealedLayer {
layer: l.id.clone(),
colour: MASK_COLOURS[view.colour % MASK_COLOURS.len()],
})
})
.collect();
if layers.is_empty() {
return None;
}
Some(Reveal {
layers,
style: self.reveal_style,
})
}
/// How shown masks are drawn, as an index into
/// [`dr_pipeline::mask::RevealStyle::ALL`].
///
/// An index because the panel offers it as a strip of chips and an index
/// is what a strip of chips reports. The enum stays the thing that is
/// stored, so a fourth style is a variant and a label rather than a number
/// two files have to agree on.
pub fn mask_view_style(&self) -> usize {
dr_pipeline::mask::RevealStyle::ALL
.iter()
.position(|&a| a == self.reveal_style)
.unwrap_or(0)
}
/// Choose how shown masks are drawn.
///
/// Takes no history step and marks nothing dirty: this is how the
/// photograph is being *looked at*, not an edit to it.
pub fn set_mask_view_style(&mut self, style: usize) {
if let Some(&s) = dr_pipeline::mask::RevealStyle::ALL.get(style) {
self.reveal_style = s;
}
}
/// Whether this layer's mask is drawn over the photograph.
pub fn mask_shown(&self, id: &str) -> bool {
self.mask_views.get(id).is_some_and(|v| v.shown)
}
/// Open or close one layer's eye.
pub fn set_mask_shown(&mut self, id: &str, shown: bool) {
let colour = self.next_mask_colour();
self.mask_views
.entry(id.to_string())
.or_insert(MaskView {
shown: false,
colour,
})
.shown = shown;
}
/// Whether any mask at all is being shown.
///
/// What the region overlay asks before drawing: two overlays that mean
/// different things, on top of each other, is neither.
pub fn any_mask_shown(&self) -> bool {
self.reveal().is_some()
}
/// Which of [`MASK_COLOURS`] this layer is shown in.
pub fn mask_colour(&self, id: &str) -> usize {
self.mask_views
.get(id)
.map_or(0, |v| v.colour % MASK_COLOURS.len())
}
/// Give this layer a colour from [`MASK_COLOURS`].
///
/// Choosing a colour is asking to see it: a swatch pressed on a layer
/// whose eye was closed opens the eye, because nothing else the press
/// could mean would change a pixel.
pub fn set_mask_colour(&mut self, id: &str, colour: usize) {
let colour = colour % MASK_COLOURS.len();
self.mask_views
.entry(id.to_string())
.and_modify(|v| {
v.colour = colour;
v.shown = true;
})
.or_insert(MaskView {
shown: true,
colour,
});
}
/// The colour the next layer to be shown should take: the first not
/// already in use, or round the palette again once all are.
///
/// So that two masks made one after the other come up in two colours
/// without anyone having to choose — which is the case that matters,
/// since "how do these two meet" is the question two masks are shown to
/// answer.
fn next_mask_colour(&self) -> usize {
let used: Vec<usize> = self.mask_views.values().map(|v| v.colour).collect();
(0..MASK_COLOURS.len())
.find(|c| !used.contains(c))
.unwrap_or(self.mask_views.len() % MASK_COLOURS.len())
}
/// TRACES: FR-DEV-19c
/// Show the mask of a layer that has just been made.
///
/// **Making a mask is asking what it selected**, and for a subject or a
/// category that question has no other answer: the model's outline is not
/// derivable from anything on screen, the layer carries no adjustment yet,
/// and the list it was chosen from says "architecture 23%" and nothing
/// about *which* 23%. So the mask appears with the layer rather than
/// waiting to be asked for a second time — its eye open, in the next
/// colour nothing else is using.
///
/// Only this layer's eye. Every other layer keeps whatever the
/// photographer set it to, which is the trap `Masking.overlay-hidden`
/// documents: an automatic reveal that undoes a switch somebody turned
/// off is worse than none.
fn show_new_mask(&mut self, id: &str) {
let colour = self.next_mask_colour();
self.mask_views.insert(
id.to_string(),
MaskView {
shown: true,
colour,
},
);
}
// ---------------------------------------------------------------------- // ----------------------------------------------------------------------
// The region overlay // The region overlay
// ---------------------------------------------------------------------- // ----------------------------------------------------------------------
@@ -3228,7 +3467,7 @@ impl DevelopSession {
}, },
); );
layer.name = name.to_string(); layer.name = name.to_string();
// Refined from the start, where there is anything to refine with. // Refined from the start, by as much as this photograph will bear.
// //
// A category's edges are twenty proxy pixels wide before this runs, so // A category's edges are twenty proxy pixels wide before this runs, so
// the unrefined mask is the wrong default for the common case — a // the unrefined mask is the wrong default for the common case — a
@@ -3236,20 +3475,30 @@ impl DevelopSession {
// chimney in it. Zero is still one drag away, and it is exactly the // chimney in it. Zero is still one drag away, and it is exactly the
// model's own weighting when they get there. // model's own weighting when they get there.
// //
// **Asked of the frame rather than taken from a constant**, and that
// is the correction rather than a refinement of the idea. The constant
// was `STRICTNESS_DEFAULT`, fitted on a synthetic sky; on real
// photographs it removes three quarters to all of `architecture`,
// `ground` and `vegetation`, so clicking a category produced an empty
// mask — the failure that reads as the feature not working at all,
// because the adjustment moves and no pixel changes. The measurements
// are on `dr_segment::Refinement::gentle`, which is what picks the
// number now.
//
// Set here rather than in `MaskLayer::new` because the number belongs // Set here rather than in `MaskLayer::new` because the number belongs
// to `dr_segment` and `dr-pipeline` does not depend on it — see // to `dr_segment` and `dr-pipeline` does not depend on it — see
// `dr_pipeline::mask::MAX_REFINE`. // `dr_pipeline::mask::MAX_REFINE`.
if self layer.base_mut().refine = self
.segmentation .segmentation
.as_ref() .as_ref()
.is_some_and(|seg| seg.category_is_refinable(name)) .map_or(dr_segment::STRICTNESS_OFF, |seg| {
{ seg.category_default_refine(name)
layer.base_mut().refine = dr_segment::STRICTNESS_DEFAULT; });
}
if !self.graph.masks_mut().push(layer) { if !self.graph.masks_mut().push(layer) {
return None; return None;
} }
self.active_masks = vec![id.clone()]; self.active_masks = vec![id.clone()];
self.show_new_mask(&id);
self.history self.history
.record(&self.graph, Edit::Action(labels::step::MASK_ADDED)); .record(&self.graph, Edit::Action(labels::step::MASK_ADDED));
Some(id) Some(id)
@@ -3293,6 +3542,7 @@ impl DevelopSession {
return None; return None;
} }
self.active_masks = vec![id.clone()]; self.active_masks = vec![id.clone()];
self.show_new_mask(&id);
self.history self.history
.record(&self.graph, Edit::Action(labels::step::MASK_ADDED)); .record(&self.graph, Edit::Action(labels::step::MASK_ADDED));
Some(id) Some(id)
@@ -3323,6 +3573,39 @@ impl DevelopSession {
return None; return None;
} }
self.active_masks = vec![id.clone()]; self.active_masks = vec![id.clone()];
self.show_new_mask(&id);
self.history
.record(&self.graph, Edit::Action(labels::step::MASK_ADDED));
Some(id)
}
/// TRACES: FR-DEV-19b
/// Add a mask that is nothing but hand-painted, and select it.
///
/// **The one route to a brush that starts from nothing.** Everything else
/// in this file makes a layer out of a selection — a gradient, a band, a
/// subject, a category — and painting was reachable only by making one of
/// those first and then joining a painted part to it. So the answer to
/// "brush a correction onto this corner of the sky" was "add a radial
/// gradient you do not want, then paint into it", which is not an answer.
///
/// The layer covers nothing until a stroke lands in it, which is exactly
/// what [`dr_pipeline::mask::MaskPart::covers`] is about: it is not active,
/// it costs no slice, and an invert on it would not take the adjustment
/// global. What the panel shows meanwhile is a row with the tools armed
/// over it — see `masks_ui`, which arms them.
pub fn add_brush_mask(&mut self) -> Option<String> {
let id = self.graph.masks().next_id();
if !self
.graph
.masks_mut()
.push(MaskLayer::new(id.clone(), MaskSource::brush()))
{
return None;
}
self.active_masks = vec![id.clone()];
self.active_part = 0;
self.show_new_mask(&id);
self.history self.history
.record(&self.graph, Edit::Action(labels::step::MASK_ADDED)); .record(&self.graph, Edit::Action(labels::step::MASK_ADDED));
Some(id) Some(id)
@@ -3349,6 +3632,7 @@ impl DevelopSession {
return None; return None;
} }
self.active_masks = vec![id.clone()]; self.active_masks = vec![id.clone()];
self.show_new_mask(&id);
self.history self.history
.record(&self.graph, Edit::Action(labels::step::MASK_ADDED)); .record(&self.graph, Edit::Action(labels::step::MASK_ADDED));
Some(id) Some(id)
@@ -3729,7 +4013,12 @@ impl DevelopSession {
// always entered the structure hash. Moving the window to a P3 panel // always entered the structure hash. Moving the window to a P3 panel
// therefore costs one shader compile and no pipeline change at all. // therefore costs one shader compile and no pipeline change at all.
let space = self.display_space; let space = self.display_space;
let shader = self.graph.compose_for(space); // TRACES: FR-DEV-19c
// **The one composition that may show a mask.** Every other caller of
// the graph — `render_the_file`, the thumbnail, `sample_as_shot` —
// goes through `compose_for`, which cannot ask for a reveal, so no
// exported file can carry one.
let shader = self.graph.compose_revealing(space, self.reveal().as_ref());
// Rasterise the masks first: the shader addresses array slices by // Rasterise the masks first: the shader addresses array slices by
// index, so the array has to describe *this* stack before it is bound. // index, so the array has to describe *this* stack before it is bound.
@@ -6401,6 +6690,16 @@ mod tests {
let mut live = session_with_a_left_half_subject(&ctx); let mut live = session_with_a_left_half_subject(&ctx);
let id = live.add_subject_mask(0).expect("a subject layer"); let id = live.add_subject_mask(0).expect("a subject layer");
// TRACES: FR-DEV-19c
// Making a mask opens its eye (`show_new_mask`), and this test is
// about the pixels the *edit* produces. Closed here rather than left
// open, and the asymmetry is the point rather than an inconvenience:
// the reveal is how somebody is looking at a photograph, so a session
// that has just made a layer legitimately draws a frame that a session
// which read the same layer out of a file does not. Both of those are
// correct, and only one of them is what a stored raster has to
// reproduce.
live.set_mask_shown(&id, false);
live.graph live.graph
.masks_mut() .masks_mut()
.get_mut(&id) .get_mut(&id)
@@ -6427,6 +6726,15 @@ mod tests {
"the point: nothing has run a model in this session" "the point: nothing has run a model in this session"
); );
reopened.apply_version(parsed.default_version().expect("a version")); reopened.apply_version(parsed.default_version().expect("a version"));
// TRACES: FR-DEV-19c
// And a sidecar carries no viewing state: a photograph reopened is not
// reopened with its masks tinted red. Checked here because this is the
// one test that puts an edit through a file and renders both ends, so
// it is where the property would first go wrong.
assert!(
!reopened.any_mask_shown(),
"restoring an edit must not open an eye"
);
let from_the_file = read_back(&ctx, &reopened.render(64, 64).expect("render")); let from_the_file = read_back(&ctx, &reopened.render(64, 64).expect("render"));
assert_eq!( assert_eq!(
+157 -12
View File
@@ -306,7 +306,7 @@ pub fn index_proxy(
continue; continue;
} }
let embedding = embedder.embed(&aligned)?; let embedded = embedder.embed(&aligned)?;
out.push(DetectedFace { out.push(DetectedFace {
x: d.bbox.0 / long_edge, x: d.bbox.0 / long_edge,
@@ -315,8 +315,9 @@ pub fn index_proxy(
h: d.height() / long_edge, h: d.height() / long_edge,
landmarks: normalise_landmarks(&d.landmarks, long_edge), landmarks: normalise_landmarks(&d.landmarks, long_edge),
confidence: d.confidence, confidence: d.confidence,
embedding: embedding.to_f16_bytes(), embedding: embedded.to_f16_bytes(),
crop_px: aligned.source_px(), crop_px: aligned.source_px(),
quality: Some(embedded.quality),
model_id: embedder.model().as_str().to_string(), model_id: embedder.model().as_str().to_string(),
// Cut here, while the buffer is still in hand. This is the only // Cut here, while the buffer is still in hand. This is the only
// moment in the whole pipeline where the pixels are free. // moment in the whole pipeline where the pixels are free.
@@ -419,7 +420,7 @@ pub fn index_native(
continue; continue;
} }
let embedding = embedder.embed(&aligned)?; let embedded = embedder.embed(&aligned)?;
let (bx, by) = (d.bbox.0 * sx, d.bbox.1 * sy); let (bx, by) = (d.bbox.0 * sx, d.bbox.1 * sy);
let (bw, bh) = (d.width() * sx, d.height() * sy); let (bw, bh) = (d.width() * sx, d.height() * sy);
@@ -430,8 +431,9 @@ pub fn index_native(
h: bh / long_edge, h: bh / long_edge,
landmarks: normalise_landmarks(&landmarks, long_edge), landmarks: normalise_landmarks(&landmarks, long_edge),
confidence: d.confidence, confidence: d.confidence,
embedding: embedding.to_f16_bytes(), embedding: embedded.to_f16_bytes(),
crop_px: aligned.source_px(), crop_px: aligned.source_px(),
quality: Some(embedded.quality),
model_id: embedder.model().as_str().to_string(), model_id: embedder.model().as_str().to_string(),
crop: cut_crop_native(native, width, height, (bx, by, bw, bh)).unwrap_or_default(), crop: cut_crop_native(native, width, height, (bx, by, bw, bh)).unwrap_or_default(),
}); });
@@ -440,6 +442,64 @@ pub fn index_native(
Ok(out) Ok(out)
} }
/// What re-embedding the faces already on one image produced.
#[derive(Debug, Default, PartialEq)]
pub struct Measured {
pub measured: Vec<faces::Measurement>,
/// Faces whose stored landmarks no longer make a warp. See
/// `dr_catalog::faces::record_measurements` for what becomes of them.
pub dropped: Vec<faces::FaceId>,
}
/// TRACES: FR-CULL-8 | FR-CULL-9
/// Embed the faces already found on one image again, from its native render.
///
/// The measuring half of the sweep, for faces stored before their quality was
/// kept (schema V14). No detector: the boxes and landmarks in the catalog are
/// taken as read, scaled back from the long edge they were normalised to, and
/// each face is warped out of the native frame and embedded exactly as
/// [`index_native`] would have done on the day. What comes back is the raw
/// vector and its length, to be written over the old unit one.
///
/// The size and sharpness gates are deliberately not re-applied. They decide
/// whether a face is worth *storing*, and these are stored; what is being
/// established now is how much the model can make of each, which is the
/// quality itself, and a face that would have failed a gate is precisely one
/// that should come out short and stop vouching for anyone.
pub fn measure_native(
embedder: &mut Embedder,
rgba: &[u8],
width: usize,
height: usize,
faces: &[faces::Face],
) -> Result<Measured, dr_face::FaceError> {
let mut out = Measured::default();
if !index_native_shape_ok(rgba, width, height) {
return Ok(out);
}
let long_edge = width.max(height) as f32;
let native = dr_face::Pixels::Rgba8(rgba);
for f in faces {
let mut landmarks = [(0.0_f32, 0.0_f32); 5];
for (o, &(x, y)) in landmarks.iter_mut().zip(f.landmarks.iter()) {
*o = (x * long_edge, y * long_edge);
}
let Some(aligned) = dr_face::warp_pixels(native, width, height, &landmarks) else {
log::debug!("face {:?} has degenerate landmarks, dropped", f.id);
out.dropped.push(f.id);
continue;
};
let embedded = embedder.embed(&aligned)?;
out.measured.push(faces::Measurement {
face: f.id,
embedding: embedded.to_f16_bytes(),
quality: embedded.quality,
});
}
Ok(out)
}
/// Whether a buffer is the native frame it claims to be. /// Whether a buffer is the native frame it claims to be.
/// ///
/// Checked before anything expensive, and before the detector above all: a /// Checked before anything expensive, and before the detector above all: a
@@ -837,19 +897,20 @@ impl Population {
let model = ModelId::new(model_id.to_string()); let model = ModelId::new(model_id.to_string());
let mut candidates = Vec::with_capacity(stored.len()); let mut candidates = Vec::with_capacity(stored.len());
let mut ids = Vec::with_capacity(stored.len()); let mut ids = Vec::with_capacity(stored.len());
for (face_id, image_id, blob, crop_px) in stored { for f in stored {
let Some(emb) = dr_face::Embedding::from_f16_bytes(model.clone(), &blob) else { let Some(emb) = dr_face::Embedding::from_f16_bytes(model.clone(), &f.embedding) else {
log::warn!("face {face_id:?} has a malformed embedding, skipped"); log::warn!("face {:?} has a malformed embedding, skipped", f.face);
continue; continue;
}; };
candidates.push(dr_face::Candidate { candidates.push(dr_face::Candidate {
face: face_id.0, face: f.face.0,
image: image_id.0, image: f.image.0,
embedding: emb.v.to_vec(), embedding: emb.v.to_vec(),
crop_px, crop_px: f.crop_px,
confirmed_person: anchors.get(&face_id).map(|p| p.0), quality: f.quality,
confirmed_person: anchors.get(&f.face).map(|p| p.0),
}); });
ids.push(face_id); ids.push(f.face);
} }
Ok(Self { Ok(Self {
@@ -1385,6 +1446,88 @@ mod tests {
assert_eq!(out[4], (360.0, 200.0)); assert_eq!(out[4], (360.0, 200.0));
} }
/// The measuring path end to end, with the real embedder where one is on
/// this machine: landmarks come back from the long edge they were
/// normalised to, the warp is built from them, and what is stored is the
/// raw vector, whose length is the quality reported beside it.
#[test]
fn measuring_stores_the_raw_vector_at_the_quality_it_reports() {
let dir = crate::library::shared_face_models_dir();
let path = dir.join("arcface_mbf_b1.onnx");
if !path.is_file() {
eprintln!("no embedder at {}, skipping", path.display());
return;
}
let model = ModelId::new("w600k_mbf");
let mut embedder = Embedder::from_path(&path, model.clone()).expect("load embedder");
// A 600×400 frame with a gradient in it, and one face whose landmarks
// are the ArcFace template scaled up and placed in the middle -- not
// a face, but an unambiguous warp.
let (w, h) = (600usize, 400usize);
let mut rgba = vec![0u8; w * h * 4];
for y in 0..h {
for x in 0..w {
let i = (y * w + x) * 4;
rgba[i] = (x * 255 / w) as u8;
rgba[i + 1] = (y * 255 / h) as u8;
rgba[i + 2] = ((x + y) % 256) as u8;
rgba[i + 3] = 255;
}
}
let long_edge = w.max(h) as f32;
let mut landmarks = [(0.0_f32, 0.0_f32); 5];
for (o, &(tx, ty)) in landmarks.iter_mut().zip(dr_face::ARCFACE_TEMPLATE.iter()) {
// Two and a half times the template, offset into the frame, then
// normalised to the long edge as the catalog stores it.
*o = (
(tx * 2.5 + 150.0) / long_edge,
(ty * 2.5 + 60.0) / long_edge,
);
}
let stored = faces::Face {
id: faces::FaceId(7),
image_id: ImageId(1),
x: 0.25,
y: 0.15,
w: 0.5,
h: 0.7,
landmarks,
confidence: 0.9,
crop_px: 280.0,
quality: None,
model_id: "w600k_mbf".into(),
person: None,
probability: 0.0,
confirmed: false,
};
let stored_copy = stored.clone();
let out = measure_native(&mut embedder, &rgba, w, h, &[stored]).expect("measure");
assert!(out.dropped.is_empty());
assert_eq!(out.measured.len(), 1);
let m = &out.measured[0];
assert_eq!(m.face, faces::FaceId(7));
assert!(m.quality > 0.0);
let (_, length) = dr_face::read_f16_bytes(model, &m.embedding).expect("decode");
assert!(
(length - m.quality).abs() < 0.05 * m.quality,
"stored length {length} against reported quality {}",
m.quality
);
// Degenerate landmarks -- five points on one spot, which no
// similarity can be fitted to -- are dropped, not embedded.
let junk = faces::Face {
id: faces::FaceId(8),
landmarks: [(0.3, 0.3); 5],
..stored_copy
};
let out = measure_native(&mut embedder, &rgba, w, h, &[junk]).expect("measure");
assert!(out.measured.is_empty());
assert_eq!(out.dropped, vec![faces::FaceId(8)]);
}
#[test] #[test]
fn a_buffer_that_is_not_the_stated_size_indexes_nothing() { fn a_buffer_that_is_not_the_stated_size_indexes_nothing() {
// No model is loaded here, so reaching the detector would panic. The // No model is loaded here, so reaching the detector would panic. The
@@ -1516,6 +1659,7 @@ mod tests {
landmarks: [(0.0, 0.0); 5], landmarks: [(0.0, 0.0); 5],
confidence: 0.9, confidence: 0.9,
crop_px: 120.0, crop_px: 120.0,
quality: None,
model_id: "w600k_mbf".into(), model_id: "w600k_mbf".into(),
person: None, person: None,
probability: 0.0, probability: 0.0,
@@ -1633,6 +1777,7 @@ mod tests {
confidence: 0.9, confidence: 0.9,
embedding: embedding(identity, cosine), embedding: embedding(identity, cosine),
crop_px: 150.0, crop_px: 150.0,
quality: None,
model_id: TEST_MODEL.to_string(), model_id: TEST_MODEL.to_string(),
crop: Vec::new(), crop: Vec::new(),
}; };
+14 -7
View File
@@ -36,13 +36,6 @@ pub const GESTURES: &[Gesture] = &[
pointer: "The scroll wheel over it", pointer: "The scroll wheel over it",
keys: "", keys: "",
}, },
Gesture {
title: "Paint a mask by hand",
section: "Develop",
touch: "Choose Paint or Erase, then drag on the photograph",
pointer: "Choose Paint or Erase, then drag",
keys: "",
},
Gesture { Gesture {
title: "Move a magnified photograph about", title: "Move a magnified photograph about",
section: "Develop", section: "Develop",
@@ -50,6 +43,13 @@ pub const GESTURES: &[Gesture] = &[
pointer: "Drag it", pointer: "Drag it",
keys: "", keys: "",
}, },
Gesture {
title: "Paint a mask by hand",
section: "Develop",
touch: "Choose Paint or Erase, then drag on the photograph",
pointer: "Choose Paint or Erase, then drag",
keys: "",
},
Gesture { Gesture {
title: "Take back the last change", title: "Take back the last change",
section: "Develop", section: "Develop",
@@ -120,6 +120,13 @@ pub const GESTURES: &[Gesture] = &[
pointer: "Click the ring at the head of its row", pointer: "Click the ring at the head of its row",
keys: "", keys: "",
}, },
Gesture {
title: "Show or hide one mask on the photograph",
section: "Develop",
touch: "Tap the eye on its row",
pointer: "Click the eye on its row",
keys: "",
},
Gesture { Gesture {
title: "Pick a collection up to rearrange the tree", title: "Pick a collection up to rearrange the tree",
section: "Collections sidebar", section: "Collections sidebar",
+73 -5
View File
@@ -97,6 +97,10 @@ pub struct FaceCell {
/// Source pixels across the aligned crop. Small faces embed worse, and the /// Source pixels across the aligned crop. Small faces embed worse, and the
/// user deserves to know which of a bad suggestion's causes is in play. /// user deserves to know which of a bad suggestion's causes is in play.
pub crop_px: f32, pub crop_px: f32,
/// The model's own reading of how recognisable the crop was — the length
/// of its raw embedding (`dr_face::MIN_GALLERY_QUALITY`). `None` for a
/// face indexed before it was kept.
pub quality: Option<f32>,
} }
impl FaceCell { impl FaceCell {
@@ -115,6 +119,31 @@ impl FaceCell {
format!("{:.0}% likely", self.probability * 100.0) format!("{:.0}% likely", self.probability * 100.0)
} }
} }
/// The quality as text — "Quality 17.3", or "Quality —" where it was never
/// measured.
///
/// Called *quality* and not *norm* on the screen, because that is what the
/// number is used as and what the user can act on: a low one is the model
/// saying it could not make the face out, and the fix is a better
/// photograph. One decimal, because the gate sits at a whole number and
/// the faces worth a second look are the ones just either side of it.
pub fn quality_label(&self) -> String {
match self.quality {
Some(q) => format!("Quality {q:.1}"),
None => "Quality —".into(),
}
}
/// Whether this face is good enough to be compared *against*.
///
/// What the screen dims the quality for: a face below the floor is still
/// somebody and still placed, but it vouches for no one else, and a user
/// wondering why a person's group did not gather the rest of them should
/// be able to see that none of its members can.
pub fn in_gallery(&self) -> bool {
dr_face::in_gallery(self.quality)
}
} }
/// Put a grouping preview into words. /// Put a grouping preview into words.
@@ -254,6 +283,7 @@ pub fn load_faces(
confirmed: f.confirmed, confirmed: f.confirmed,
probability: f.probability, probability: f.probability,
crop_px: f.crop_px, crop_px: f.crop_px,
quality: f.quality,
}); });
} }
@@ -615,23 +645,24 @@ pub fn preview_split(
let model = dr_face::ModelId::new(model_id.to_string()); let model = dr_face::ModelId::new(model_id.to_string());
let stored: std::collections::HashMap<_, _> = faces::embeddings(conn, model_id)? let stored: std::collections::HashMap<_, _> = faces::embeddings(conn, model_id)?
.into_iter() .into_iter()
.map(|(id, image, blob, crop_px)| (id, (image, blob, crop_px))) .map(|e| (e.face, e))
.collect(); .collect();
let mut candidates = Vec::with_capacity(cells.len()); let mut candidates = Vec::with_capacity(cells.len());
let mut order = Vec::with_capacity(cells.len()); let mut order = Vec::with_capacity(cells.len());
for c in &cells { for c in &cells {
let Some((image, blob, crop_px)) = stored.get(&c.face) else { let Some(e) = stored.get(&c.face) else {
continue; continue;
}; };
let Some(emb) = dr_face::Embedding::from_f16_bytes(model.clone(), blob) else { let Some(emb) = dr_face::Embedding::from_f16_bytes(model.clone(), &e.embedding) else {
continue; continue;
}; };
candidates.push(dr_face::Candidate { candidates.push(dr_face::Candidate {
face: c.face.0, face: c.face.0,
image: image.0, image: e.image.0,
embedding: emb.v.to_vec(), embedding: emb.v.to_vec(),
crop_px: *crop_px, crop_px: e.crop_px,
quality: e.quality,
confirmed_person: None, confirmed_person: None,
}); });
order.push(c.clone()); order.push(c.clone());
@@ -755,6 +786,7 @@ mod tests {
confidence: 0.9, confidence: 0.9,
embedding: vec![seed; 1024], embedding: vec![seed; 1024],
crop_px: 180.0, crop_px: 180.0,
quality: Some(f32::from(seed) + 10.0),
model_id: "w600k_mbf".into(), model_id: "w600k_mbf".into(),
crop: Vec::new(), crop: Vec::new(),
} }
@@ -802,6 +834,7 @@ mod tests {
confirmed: false, confirmed: false,
probability: 0.87, probability: 0.87,
crop_px: 120.0, crop_px: 120.0,
quality: Some(17.26),
}; };
assert_eq!(cell.confidence_label(), "87% likely"); assert_eq!(cell.confidence_label(), "87% likely");
@@ -812,6 +845,41 @@ mod tests {
assert_eq!(confirmed.confidence_label(), "Confirmed"); assert_eq!(confirmed.confidence_label(), "Confirmed");
} }
/// The number is the embedding's length, and the screen calls it what it
/// is used as. A face from before it was kept says so rather than showing
/// a zero that would read as the worst face in the library.
#[test]
fn the_quality_is_labelled_as_such_and_dimmed_below_the_floor() {
let cell = FaceCell {
face: FaceId(1),
image: ImageId(1),
crop: None,
confirmed: false,
probability: 0.5,
crop_px: 120.0,
quality: Some(17.26),
};
assert_eq!(cell.quality_label(), "Quality 17.3");
assert!(cell.in_gallery());
let poor = FaceCell {
quality: Some(9.4),
..cell.clone()
};
assert_eq!(poor.quality_label(), "Quality 9.4");
assert!(!poor.in_gallery());
let unmeasured = FaceCell {
quality: None,
..cell
};
assert_eq!(unmeasured.quality_label(), "Quality —");
assert!(
unmeasured.in_gallery(),
"an unmeasured face is not a poor one"
);
}
#[test] #[test]
fn the_people_rail_reports_the_unassigned_pool() { fn the_people_rail_reports_the_unassigned_pool() {
let c = catalog(); let c = catalog();
+63 -27
View File
@@ -21,10 +21,14 @@ use crate::{AppWindow, IdentityFace, IdentityPerson};
/// Which model's faces the screen is looking at. /// Which model's faces the screen is looking at.
/// ///
/// A constant for now because exactly one model is supported at a time; it is /// Every query in this file is keyed on it, and a library indexed across a
/// named rather than inlined because every query in this file is keyed on it, /// detector change holds faces from both pipelines: the screen shows the one
/// and a library indexed across a model change holds faces from both. /// the settings page currently names (`FaceDetector::model_id`), read at each
pub const MODEL_ID: &str = "w600k_mbf"; /// use rather than captured once, because the page can change it while the
/// screen is open.
pub fn model_id(settings: &crate::settings_ui::SettingsController) -> String {
settings.snapshot().faces.detector.model_id().to_string()
}
/// Screen state that outlives a single callback. /// Screen state that outlives a single callback.
#[derive(Default)] #[derive(Default)]
@@ -148,6 +152,7 @@ pub fn refresh(
ctl: &Rc<IdentityController>, ctl: &Rc<IdentityController>,
catalog: &Rc<RefCell<Option<Catalog>>>, catalog: &Rc<RefCell<Option<Catalog>>>,
store: Option<Rc<ThumbStore>>, store: Option<Rc<ThumbStore>>,
model_id: &str,
) { ) {
let borrow = catalog.borrow(); let borrow = catalog.borrow();
let Some(cat) = borrow.as_ref() else { let Some(cat) = borrow.as_ref() else {
@@ -156,7 +161,7 @@ pub fn refresh(
return; return;
}; };
let view = match identity::load_people(cat, MODEL_ID) { let view = match identity::load_people(cat, model_id) {
Ok(v) => v, Ok(v) => v,
Err(e) => { Err(e) => {
log::warn!("identity: reading people: {e}"); log::warn!("identity: reading people: {e}");
@@ -268,7 +273,7 @@ pub fn refresh(
window.set_identity_picked(ctl.picked.borrow().len() as i32); window.set_identity_picked(ctl.picked.borrow().len() as i32);
drop(borrow); drop(borrow);
refresh_coverage(window, catalog, store.as_deref()); refresh_coverage(window, catalog, store.as_deref(), model_id);
// Last, so a portrait cannot delay anything above it. // Last, so a portrait cannot delay anything above it.
if let Some(store) = store { if let Some(store) = store {
@@ -379,13 +384,14 @@ pub fn refresh_coverage(
window: &AppWindow, window: &AppWindow,
catalog: &Rc<RefCell<Option<Catalog>>>, catalog: &Rc<RefCell<Option<Catalog>>>,
store: Option<&ThumbStore>, store: Option<&ThumbStore>,
model_id: &str,
) { ) {
let borrow = catalog.borrow(); let borrow = catalog.borrow();
let (Some(cat), Some(store)) = (borrow.as_ref(), store) else { let (Some(cat), Some(store)) = (borrow.as_ref(), store) else {
window.set_identity_coverage(Default::default()); window.set_identity_coverage(Default::default());
return; return;
}; };
match crate::faces::audit(cat, store, MODEL_ID) { match crate::faces::audit(cat, store, model_id) {
Ok(a) => { Ok(a) => {
window.set_identity_coverage(a.summary().into()); window.set_identity_coverage(a.summary().into());
// Complete means nothing left to index, not "every image has a // Complete means nothing left to index, not "every image has a
@@ -478,6 +484,8 @@ fn push_faces(window: &AppWindow, ctl: &IdentityController, cells: &[FaceCell])
confirmed: c.confirmed, confirmed: c.confirmed,
confidence: c.confidence_label().into(), confidence: c.confidence_label().into(),
crop_px: c.crop_px as i32, crop_px: c.crop_px as i32,
quality: c.quality_label().into(),
in_gallery: c.in_gallery(),
picked: picked.contains(&c.face), picked: picked.contains(&c.face),
}) })
.collect(); .collect();
@@ -589,8 +597,8 @@ pub fn wire<S, M, P>(
// a rejection changes two counts — and a model patched by hand would // a rejection changes two counts — and a model patched by hand would
// drift from the catalog in exactly the cases that matter. // drift from the catalog in exactly the cases that matter.
macro_rules! reload { macro_rules! reload {
($w:expr, $ctl:expr, $catalog:expr, $store:expr) => { ($w:expr, $ctl:expr, $catalog:expr, $store:expr, $settings:expr) => {
refresh(&$w, &$ctl, &$catalog, $store()) refresh(&$w, &$ctl, &$catalog, $store(), &model_id(&$settings))
}; };
} }
@@ -599,6 +607,7 @@ pub fn wire<S, M, P>(
let ctl = ctl.clone(); let ctl = ctl.clone();
let catalog = catalog.clone(); let catalog = catalog.clone();
let store = store.clone(); let store = store.clone();
let settings = settings.clone();
let models_present = models.clone(); let models_present = models.clone();
window.on_identity_open(move || { window.on_identity_open(move || {
let Some(w) = weak.upgrade() else { return }; let Some(w) = weak.upgrade() else { return };
@@ -613,7 +622,7 @@ pub fn wire<S, M, P>(
// A fact about the filesystem, so it is re-checked on every open // A fact about the filesystem, so it is re-checked on every open
// rather than cached: the user may have just put the models there. // rather than cached: the user may have just put the models there.
w.set_identity_model_missing(models_present().is_none()); w.set_identity_model_missing(models_present().is_none());
reload!(w, ctl, catalog, store); reload!(w, ctl, catalog, store, settings);
}); });
} }
@@ -640,6 +649,7 @@ pub fn wire<S, M, P>(
let ctl = ctl.clone(); let ctl = ctl.clone();
let catalog = catalog.clone(); let catalog = catalog.clone();
let store = store.clone(); let store = store.clone();
let settings = settings.clone();
window.on_identity_person_picked(move |id| { window.on_identity_person_picked(move |id| {
let Some(w) = weak.upgrade() else { return }; let Some(w) = weak.upgrade() else { return };
// Before the selection moves: a name typed into the field and never // Before the selection moves: a name typed into the field and never
@@ -651,7 +661,7 @@ pub fn wire<S, M, P>(
// The offer was about the person being navigated away from. Left // The offer was about the person being navigated away from. Left
// up, its "Merge" would fold whoever is selected *now*. // up, its "Merge" would fold whoever is selected *now*.
clear_merge_offer(&w, &ctl); clear_merge_offer(&w, &ctl);
reload!(w, ctl, catalog, store); reload!(w, ctl, catalog, store, settings);
reset_name_field(&w); reset_name_field(&w);
}); });
} }
@@ -661,6 +671,7 @@ pub fn wire<S, M, P>(
let ctl = ctl.clone(); let ctl = ctl.clone();
let catalog = catalog.clone(); let catalog = catalog.clone();
let store = store.clone(); let store = store.clone();
let settings = settings.clone();
window.on_identity_rename(move |name| { window.on_identity_rename(move |name| {
let Some(w) = weak.upgrade() else { return }; let Some(w) = weak.upgrade() else { return };
let Some(person) = ctl.selected.get() else { let Some(person) = ctl.selected.get() else {
@@ -695,7 +706,7 @@ pub fn wire<S, M, P>(
Err(e) => log::warn!("identity: looking for a namesake: {e}"), Err(e) => log::warn!("identity: looking for a namesake: {e}"),
} }
} }
reload!(w, ctl, catalog, store); reload!(w, ctl, catalog, store, settings);
// `rename` trims; the field should show what was actually stored // `rename` trims; the field should show what was actually stored
// rather than the spacing the user happened to type. // rather than the spacing the user happened to type.
reset_name_field(&w); reset_name_field(&w);
@@ -712,6 +723,7 @@ pub fn wire<S, M, P>(
let ctl = ctl.clone(); let ctl = ctl.clone();
let catalog = catalog.clone(); let catalog = catalog.clone();
let store = store.clone(); let store = store.clone();
let settings = settings.clone();
window.on_identity_merge_accept(move || { window.on_identity_merge_accept(move || {
let Some(w) = weak.upgrade() else { return }; let Some(w) = weak.upgrade() else { return };
let (Some(target), Some(source)) = (ctl.merge_offer.get(), ctl.selected.get()) else { let (Some(target), Some(source)) = (ctl.merge_offer.get(), ctl.selected.get()) else {
@@ -731,7 +743,7 @@ pub fn wire<S, M, P>(
} }
} }
clear_merge_offer(&w, &ctl); clear_merge_offer(&w, &ctl);
reload!(w, ctl, catalog, store); reload!(w, ctl, catalog, store, settings);
reset_name_field(&w); reset_name_field(&w);
}); });
} }
@@ -765,6 +777,7 @@ pub fn wire<S, M, P>(
let ctl = ctl.clone(); let ctl = ctl.clone();
let catalog = catalog.clone(); let catalog = catalog.clone();
let store = store.clone(); let store = store.clone();
let settings = settings.clone();
window.on_identity_confirm_face(move |id| { window.on_identity_confirm_face(move |id| {
let Some(w) = weak.upgrade() else { return }; let Some(w) = weak.upgrade() else { return };
let Some(person) = ctl.selected.get() else { let Some(person) = ctl.selected.get() else {
@@ -775,7 +788,7 @@ pub fn wire<S, M, P>(
log::warn!("identity: confirm: {e}"); log::warn!("identity: confirm: {e}");
} }
} }
reload!(w, ctl, catalog, store); reload!(w, ctl, catalog, store, settings);
}); });
} }
@@ -784,6 +797,7 @@ pub fn wire<S, M, P>(
let ctl = ctl.clone(); let ctl = ctl.clone();
let catalog = catalog.clone(); let catalog = catalog.clone();
let store = store.clone(); let store = store.clone();
let settings = settings.clone();
window.on_identity_reject_face(move |id| { window.on_identity_reject_face(move |id| {
let Some(w) = weak.upgrade() else { return }; let Some(w) = weak.upgrade() else { return };
let Some(person) = ctl.selected.get() else { let Some(person) = ctl.selected.get() else {
@@ -794,7 +808,7 @@ pub fn wire<S, M, P>(
log::warn!("identity: reject: {e}"); log::warn!("identity: reject: {e}");
} }
} }
reload!(w, ctl, catalog, store); reload!(w, ctl, catalog, store, settings);
}); });
} }
@@ -825,6 +839,7 @@ pub fn wire<S, M, P>(
let ctl = ctl.clone(); let ctl = ctl.clone();
let catalog = catalog.clone(); let catalog = catalog.clone();
let store = store.clone(); let store = store.clone();
let settings = settings.clone();
window.on_identity_confirm_all(move || { window.on_identity_confirm_all(move || {
let Some(w) = weak.upgrade() else { return }; let Some(w) = weak.upgrade() else { return };
let Some(person) = ctl.selected.get() else { let Some(person) = ctl.selected.get() else {
@@ -836,7 +851,7 @@ pub fn wire<S, M, P>(
Err(e) => log::warn!("identity: confirm all: {e}"), Err(e) => log::warn!("identity: confirm all: {e}"),
} }
} }
reload!(w, ctl, catalog, store); reload!(w, ctl, catalog, store, settings);
}); });
} }
@@ -845,6 +860,7 @@ pub fn wire<S, M, P>(
let ctl = ctl.clone(); let ctl = ctl.clone();
let catalog = catalog.clone(); let catalog = catalog.clone();
let store = store.clone(); let store = store.clone();
let settings = settings.clone();
window.on_identity_split_picked(move || { window.on_identity_split_picked(move || {
let Some(w) = weak.upgrade() else { return }; let Some(w) = weak.upgrade() else { return };
let Some(person) = ctl.selected.get() else { let Some(person) = ctl.selected.get() else {
@@ -868,7 +884,7 @@ pub fn wire<S, M, P>(
} }
} }
ctl.clear_picks(); ctl.clear_picks();
reload!(w, ctl, catalog, store); reload!(w, ctl, catalog, store, settings);
}); });
} }
@@ -891,7 +907,7 @@ pub fn wire<S, M, P>(
w.set_identity_previewing(true); w.set_identity_previewing(true);
*ctl.preview.borrow_mut() = Some(crate::faces::spawn_grouping_preview( *ctl.preview.borrow_mut() = Some(crate::faces::spawn_grouping_preview(
catalog_path, catalog_path,
MODEL_ID.to_string(), model_id(&settings),
settings.snapshot().faces, settings.snapshot().faces,
)); ));
@@ -958,7 +974,7 @@ pub fn wire<S, M, P>(
w.set_identity_regroup_status("regrouping…".into()); w.set_identity_regroup_status("regrouping…".into());
*ctl.regroup.borrow_mut() = Some(crate::faces::spawn_recluster( *ctl.regroup.borrow_mut() = Some(crate::faces::spawn_recluster(
catalog_path, catalog_path,
MODEL_ID.to_string(), model_id(&settings_for_regroup),
settings_for_regroup.snapshot().faces, settings_for_regroup.snapshot().faces,
)); ));
@@ -970,6 +986,7 @@ pub fn wire<S, M, P>(
let ctl_tick = ctl.clone(); let ctl_tick = ctl.clone();
let catalog_tick = catalog.clone(); let catalog_tick = catalog.clone();
let store_tick = store.clone(); let store_tick = store.clone();
let settings_tick = settings_for_regroup.clone();
timer.start( timer.start(
slint::TimerMode::Repeated, slint::TimerMode::Repeated,
Duration::from_millis(100), Duration::from_millis(100),
@@ -1018,7 +1035,13 @@ pub fn wire<S, M, P>(
// makes new people; a stale cache would draw the // makes new people; a stale cache would draw the
// previous pass's faces beside the new groups. // previous pass's faces beside the new groups.
ctl_tick.covers.borrow_mut().clear(); ctl_tick.covers.borrow_mut().clear();
refresh(&w, &ctl_tick, &catalog_tick, store_tick()); refresh(
&w,
&ctl_tick,
&catalog_tick,
store_tick(),
&model_id(&settings_tick),
);
} }
}, },
); );
@@ -1034,6 +1057,7 @@ pub fn wire<S, M, P>(
let models = models.clone(); let models = models.clone();
let paths = paths.clone(); let paths = paths.clone();
let gpu = gpu.clone(); let gpu = gpu.clone();
let settings_for_sweep = settings.clone();
window.on_identity_index(move || { window.on_identity_index(move || {
let Some(w) = weak.upgrade() else { return }; let Some(w) = weak.upgrade() else { return };
if ctl.sweep.borrow().is_some() { if ctl.sweep.borrow().is_some() {
@@ -1064,7 +1088,7 @@ pub fn wire<S, M, P>(
store_dir, store_dir,
detector, detector,
embedder, embedder,
MODEL_ID.to_string(), model_id(&settings_for_sweep),
dr_face::DetectOptions::default(), dr_face::DetectOptions::default(),
gpu, gpu,
)); ));
@@ -1081,6 +1105,7 @@ pub fn wire<S, M, P>(
let ctl_tick = ctl.clone(); let ctl_tick = ctl.clone();
let catalog_tick = catalog.clone(); let catalog_tick = catalog.clone();
let store_tick = store.clone(); let store_tick = store.clone();
let settings_tick = settings_for_sweep.clone();
timer.start( timer.start(
slint::TimerMode::Repeated, slint::TimerMode::Repeated,
Duration::from_millis(250), Duration::from_millis(250),
@@ -1163,7 +1188,13 @@ pub fn wire<S, M, P>(
// grouped, so a sweep ending with an unchanged people // grouped, so a sweep ending with an unchanged people
// rail is expected. The coverage line is what shows it // rail is expected. The coverage line is what shows it
// worked, and Regroup is the next step. // worked, and Regroup is the next step.
refresh(&w, &ctl_tick, &catalog_tick, store_tick()); refresh(
&w,
&ctl_tick,
&catalog_tick,
store_tick(),
&model_id(&settings_tick),
);
} }
}, },
); );
@@ -1178,6 +1209,7 @@ pub fn wire<S, M, P>(
let ctl = ctl.clone(); let ctl = ctl.clone();
let catalog = catalog.clone(); let catalog = catalog.clone();
let store = store.clone(); let store = store.clone();
let settings = settings.clone();
window.on_identity_stop_indexing(move || { window.on_identity_stop_indexing(move || {
let Some(w) = weak.upgrade() else { return }; let Some(w) = weak.upgrade() else { return };
// Dropping the receiver *is* the cancellation: the worker's next // Dropping the receiver *is* the cancellation: the worker's next
@@ -1188,7 +1220,7 @@ pub fn wire<S, M, P>(
a.finish("stopped"); a.finish("stopped");
} }
w.set_identity_indexing(false); w.set_identity_indexing(false);
reload!(w, ctl, catalog, store); reload!(w, ctl, catalog, store, settings);
}); });
} }
@@ -1196,9 +1228,10 @@ pub fn wire<S, M, P>(
let weak = window.as_weak(); let weak = window.as_weak();
let catalog = catalog.clone(); let catalog = catalog.clone();
let store = store.clone(); let store = store.clone();
let settings = settings.clone();
window.on_identity_check_coverage(move || { window.on_identity_check_coverage(move || {
let Some(w) = weak.upgrade() else { return }; let Some(w) = weak.upgrade() else { return };
refresh_coverage(&w, &catalog, store().as_deref()); refresh_coverage(&w, &catalog, store().as_deref(), &model_id(&settings));
}); });
} }
@@ -1207,6 +1240,7 @@ pub fn wire<S, M, P>(
let ctl = ctl.clone(); let ctl = ctl.clone();
let catalog = catalog.clone(); let catalog = catalog.clone();
let store = store.clone(); let store = store.clone();
let settings = settings.clone();
window.on_identity_ignore_person(move |id, on| { window.on_identity_ignore_person(move |id, on| {
let Some(w) = weak.upgrade() else { return }; let Some(w) = weak.upgrade() else { return };
let person = PersonId(id.max(0) as u64); let person = PersonId(id.max(0) as u64);
@@ -1223,7 +1257,7 @@ pub fn wire<S, M, P>(
ctl.selected.set(None); ctl.selected.set(None);
ctl.clear_picks(); ctl.clear_picks();
} }
reload!(w, ctl, catalog, store); reload!(w, ctl, catalog, store, settings);
}); });
} }
@@ -1232,10 +1266,11 @@ pub fn wire<S, M, P>(
let ctl = ctl.clone(); let ctl = ctl.clone();
let catalog = catalog.clone(); let catalog = catalog.clone();
let store = store.clone(); let store = store.clone();
let settings = settings.clone();
window.on_identity_toggle_show_ignored(move || { window.on_identity_toggle_show_ignored(move || {
let Some(w) = weak.upgrade() else { return }; let Some(w) = weak.upgrade() else { return };
ctl.show_ignored.set(!ctl.show_ignored.get()); ctl.show_ignored.set(!ctl.show_ignored.get());
reload!(w, ctl, catalog, store); reload!(w, ctl, catalog, store, settings);
}); });
} }
@@ -1244,6 +1279,7 @@ pub fn wire<S, M, P>(
let ctl = ctl.clone(); let ctl = ctl.clone();
let catalog = catalog.clone(); let catalog = catalog.clone();
let store = store.clone(); let store = store.clone();
let settings = settings.clone();
window.on_identity_delete_all(move || { window.on_identity_delete_all(move || {
let Some(w) = weak.upgrade() else { return }; let Some(w) = weak.upgrade() else { return };
if let Some(cat) = catalog.borrow().as_ref() { if let Some(cat) = catalog.borrow().as_ref() {
@@ -1255,7 +1291,7 @@ pub fn wire<S, M, P>(
ctl.selected.set(None); ctl.selected.set(None);
ctl.clear_picks(); ctl.clear_picks();
ctl.covers.borrow_mut().clear(); ctl.covers.borrow_mut().clear();
reload!(w, ctl, catalog, store); reload!(w, ctl, catalog, store, settings);
}); });
} }
} }
+44 -11
View File
@@ -1358,9 +1358,10 @@ pub fn run(paths: Vec<PathBuf>) -> Result<()> {
// and their absence is the ordinary state of a fresh install. // and their absence is the ordinary state of a fresh install.
{ {
let lib = library.clone(); let lib = library.clone();
let settings = settings.clone();
move || { move || {
let conn = lib.session()?; let conn = lib.session()?;
library::face_models(&conn.account) library::face_models(&conn.account, settings.snapshot().faces.detector)
} }
}, },
// The sweep opens its own connection on its own thread, so it // The sweep opens its own connection on its own thread, so it
@@ -1504,6 +1505,7 @@ pub fn run(paths: Vec<PathBuf>) -> Result<()> {
library.set_cache_budget(stored.cache.original_budget_bytes); library.set_cache_budget(stored.cache.original_budget_bytes);
library.set_keep_opened_originals(stored.cache.keep_opened_originals); library.set_keep_opened_originals(stored.cache.keep_opened_originals);
library.set_timeline_bars(stored.library.timeline_bars); library.set_timeline_bars(stored.library.timeline_bars);
library.set_face_model_id(stored.faces.detector.model_id());
} }
// --- the export folder picker ---------------------------------------- // --- the export folder picker ----------------------------------------
@@ -1648,6 +1650,16 @@ pub fn run(paths: Vec<PathBuf>) -> Result<()> {
} }
} }
// The pipeline the next sync and the next sweep run under, and
// the two lines on the page that describe it: which detector
// file is installed, and how much of the library that
// pipeline has covered — which for a freshly chosen one is
// nothing, and saying so is the point.
lib.set_face_model_id(s.faces.detector.model_id());
if let Some(w) = weak.upgrade() {
refresh_face_status(&w, &lib, s.faces.detector);
}
// Lowering the ceiling evicts, so the figure beside it has just // Lowering the ceiling evicts, so the figure beside it has just
// changed — leaving the old one would show the cache still over a // changed — leaving the old one would show the cache still over a
// limit that was enforced a moment ago. // limit that was enforced a moment ago.
@@ -1662,18 +1674,9 @@ pub fn run(paths: Vec<PathBuf>) -> Result<()> {
// through here rather than being kept up to date continuously — // through here rather than being kept up to date continuously —
// they are only ever looked at while this page is on screen. // they are only ever looked at while this page is on screen.
let lib = library.clone(); let lib = library.clone();
let catalog = library.catalog();
let ctl = settings.clone(); let ctl = settings.clone();
move |w: &AppWindow| { move |w: &AppWindow| {
let store = lib.session().and_then(|c| { refresh_face_status(w, &lib, ctl.snapshot().faces.detector);
dr_thumbs::ThumbStore::open(&library::thumbs_dir(&c.account)).ok()
});
identity_ui::refresh_coverage(w, &catalog, store.as_ref());
w.set_identity_model_missing(
lib.session()
.and_then(|c| library::face_models(&c.account))
.is_none(),
);
// Here rather than at startup, on exactly the reasoning // Here rather than at startup, on exactly the reasoning
// above: the catalog this reads is opened on a worker now, // above: the catalog this reads is opened on a worker now,
// so a launch has nothing to describe, and the figure is // so a launch has nothing to describe, and the figure is
@@ -3588,6 +3591,36 @@ pub fn run(paths: Vec<PathBuf>) -> Result<()> {
/// the pinned figure is disk the user asked for and no ceiling will reclaim. /// the pinned figure is disk the user asked for and no ceiling will reclaim.
/// One combined number would make the budget look wrong whenever a large /// One combined number would make the budget look wrong whenever a large
/// collection was pinned. /// collection was pinned.
/// TRACES: FR-CULL-8
/// The two face lines on the settings page: whether the chosen detector's file
/// is installed, and how far the chosen pipeline has got through the library.
///
/// One function because they move together. A detector picked from the list
/// changes which file has to exist *and* which `model_id` the coverage is
/// counted under, and a page that updated one without the other would report
/// "no model installed" over a coverage figure for a different model.
fn refresh_face_status(
window: &AppWindow,
library: &Rc<library_ui::LibraryController>,
detector: dr_types::FaceDetector,
) {
let store = library
.session()
.and_then(|c| dr_thumbs::ThumbStore::open(&library::thumbs_dir(&c.account)).ok());
identity_ui::refresh_coverage(
window,
&library.catalog(),
store.as_ref(),
detector.model_id(),
);
window.set_identity_model_missing(
library
.session()
.and_then(|c| library::face_models(&c.account, detector))
.is_none(),
);
}
fn describe_cache_usage(library: &Rc<library_ui::LibraryController>) -> String { fn describe_cache_usage(library: &Rc<library_ui::LibraryController>) -> String {
let Some(cache) = library.cache() else { let Some(cache) = library.cache() else {
return String::new(); return String::new();
+243 -23
View File
@@ -3297,6 +3297,14 @@ fn flush_sweep(catalog: &Catalog, found: &mut Vec<MetadataFound>) {
/// so an image with no proxy is work to be done rather than work to be skipped /// so an image with no proxy is work to be done rather than work to be skipped
/// — which is the whole difference between indexing a library and indexing the /// — which is the whole difference between indexing a library and indexing the
/// fraction of it that has been browsed. /// fraction of it that has been browsed.
///
/// **An image whose faces are merely unmeasured is not here.** Schema V14
/// forgot the run marker of every such image so that *something* would look
/// at it again; [`faces_unmeasured`] is that something, and it does the cheap
/// thing — embed the faces already found — where this list would have the
/// whole detection run again. Both lists are drawn from the same catalog in
/// the same sweep, so the exclusion is in the query rather than left to the
/// caller to remember.
fn faces_unindexed( fn faces_unindexed(
catalog: &Catalog, catalog: &Catalog,
model_id: &str, model_id: &str,
@@ -3310,6 +3318,10 @@ fn faces_unindexed(
SELECT 1 FROM face_index fi SELECT 1 FROM face_index fi
WHERE fi.image_id = i.id AND fi.model_id = ?1 WHERE fi.image_id = i.id AND fi.model_id = ?1
) )
AND NOT EXISTS (
SELECT 1 FROM faces f
WHERE f.image_id = i.id AND f.model_id = ?1 AND f.quality IS NULL
)
ORDER BY i.id" ORDER BY i.id"
))?; ))?;
let rows = stmt let rows = stmt
@@ -3382,6 +3394,60 @@ fn faces_without_proxy(
Ok(rows) Ok(rows)
} }
/// TRACES: FR-CULL-9
/// Images holding a face this model found before its quality was kept.
///
/// The work list of the sweep's measuring pass: every stored face whose vector
/// is a unit one (schema V14) is embedded again from the native render, with
/// the landmarks it already has, and the raw vector and its length written
/// over it. Nothing is re-detected and no face changes identity — see
/// `dr_catalog::faces::record_measurements`.
///
/// Costs what the indexing pass costs per image, an original fetched and
/// rendered, because the length exists only at the moment of embedding and
/// there is no embedding without the pixels. What it saves is the detector,
/// and — the part that matters — every suggestion and confirmation on those
/// faces, which a re-detection would rebuild from box overlap.
fn faces_unmeasured(
catalog: &Catalog,
model_id: &str,
) -> Result<Vec<ThumbnailRequest>, dr_catalog::CatalogError> {
let mut stmt = catalog.connection().prepare(&format!(
"SELECT DISTINCT i.id, i.source_ref, r.file_id, i.file_size
FROM images i
JOIN remote r ON r.image_id = i.id
JOIN faces f ON f.image_id = i.id
WHERE r.file_id IS NOT NULL AND {VISIBLE}
AND f.model_id = ?1 AND f.quality IS NULL
ORDER BY i.id"
))?;
let rows = stmt
.query_map([model_id], |r| {
Ok(ThumbnailRequest {
full_resolution: true,
thumb_size: dr_thumbs::ThumbSize::Large,
row: 0,
image_id: r.get(0)?,
path: r.get(1)?,
file_id: r.get::<_, Option<i64>>(2)?.map(|v| v as u64),
size: r.get::<_, Option<i64>>(3)?.unwrap_or(0) as u64,
needs_metadata: false,
})
})?
.collect::<Result<Vec<_>, _>>()?;
Ok(rows)
}
/// What the sweep does with one fetched original.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum SweepWork {
/// Detect and embed from scratch: an image never indexed, or one whose
/// faces have nothing left to be cut from.
Detect,
/// Embed the faces already found, again — see [`faces_unmeasured`].
Measure,
}
/// TRACES: FR-CULL-8 | FR-EXP-9 | NFR-ARCH-2 | NFR-RES-2 /// TRACES: FR-CULL-8 | FR-EXP-9 | NFR-ARCH-2 | NFR-RES-2
/// Index faces across the **whole** library, at native resolution. /// Index faces across the **whole** library, at native resolution.
/// ///
@@ -3502,26 +3568,52 @@ pub fn spawn_face_sweep(
// un-indexed ones. Appended instead, they sit two hours of fetching // un-indexed ones. Appended instead, they sit two hours of fetching
// down the queue and the screen stays empty for the whole session — // down the queue and the screen stays empty for the whole session —
// which is indistinguishable from the repair not existing. // which is indistinguishable from the repair not existing.
let mut wanted = match faces_without_proxy(&catalog, &store, &model_id) { let mut wanted: Vec<(ThumbnailRequest, SweepWork)> =
Ok(repair) => { match faces_without_proxy(&catalog, &store, &model_id) {
if !repair.is_empty() { Ok(repair) => {
log::info!( if !repair.is_empty() {
log::info!(
"face sweep: repairing {} image(s) whose faces have no proxy to crop from", "face sweep: repairing {} image(s) whose faces have no proxy to crop from",
repair.len() repair.len()
); );
}
repair.into_iter().map(|r| (r, SweepWork::Detect)).collect()
} }
repair Err(e) => {
} log::warn!("face sweep: looking for orphaned faces: {e}");
Err(e) => { Vec::new()
log::warn!("face sweep: looking for orphaned faces: {e}"); }
Vec::new() };
}
};
// Disjoint from the above by construction: an image with faces recorded // Then the faces to measure again. Behind the proxy repair and ahead
// is not an image with no `face_index` row. // of the rest for the same reason that one leads: these are faces the
// People screen is showing and grouping *now*, on vectors the gallery
// rule cannot act on. An image already queued for a full re-detection
// gets its quality from that, so it is not queued twice.
let mut queued: std::collections::HashSet<i64> =
wanted.iter().map(|(r, _)| r.image_id).collect();
match faces_unmeasured(&catalog, &model_id) {
Ok(measure) => {
let fresh: Vec<_> = measure
.into_iter()
.filter(|r| queued.insert(r.image_id))
.collect();
if !fresh.is_empty() {
log::info!(
"face sweep: measuring the faces on {} image(s) stored before their quality was kept",
fresh.len()
);
}
wanted.extend(fresh.into_iter().map(|r| (r, SweepWork::Measure)));
}
Err(e) => log::warn!("face sweep: looking for unmeasured faces: {e}"),
}
// Disjoint from both of the above by construction: an image with faces
// recorded is not an image with no `face_index` row, and one whose
// faces are unmeasured is excluded by the query.
match faces_unindexed(&catalog, &model_id) { match faces_unindexed(&catalog, &model_id) {
Ok(fresh) => wanted.extend(fresh), Ok(fresh) => wanted.extend(fresh.into_iter().map(|r| (r, SweepWork::Detect))),
Err(e) => { Err(e) => {
log::warn!("face sweep: {e}"); log::warn!("face sweep: {e}");
if wanted.is_empty() { if wanted.is_empty() {
@@ -3587,7 +3679,7 @@ pub fn spawn_face_sweep(
// multiply the one allocation that actually threatens the budget // multiply the one allocation that actually threatens the budget
// while buying no parallelism that exists. // while buying no parallelism that exists.
for chunk in wanted.chunks(SWEEP_LANES) { for chunk in wanted.chunks(SWEEP_LANES) {
let fetched = futures_join_all(chunk.iter().map(|req| { let fetched = futures_join_all(chunk.iter().map(|(req, work)| {
let backend = &*backend; let backend = &*backend;
let pool = &pool; let pool = &pool;
async move { async move {
@@ -3595,11 +3687,11 @@ pub fn spawn_face_sweep(
Ok(h) => h, Ok(h) => h,
Err(e) if e.indicates_offline() => { Err(e) if e.indicates_offline() => {
log::info!("face sweep: {e}"); log::info!("face sweep: {e}");
return (req, Err(FetchOutcome::Offline)); return (req, *work, Err(FetchOutcome::Offline));
} }
Err(e) => { Err(e) => {
log::debug!("face sweep: {}: {e}", req.path); log::debug!("face sweep: {}: {e}", req.path);
return (req, Err(FetchOutcome::Failed)); return (req, *work, Err(FetchOutcome::Failed));
} }
}; };
@@ -3622,13 +3714,13 @@ pub fn spawn_face_sweep(
} }
}; };
drop(held); drop(held);
(req, got) (req, *work, got)
} }
})) }))
.await; .await;
let mut lane_failed = 0usize; let mut lane_failed = 0usize;
for (req, got) in fetched { for (req, work, got) in fetched {
let bytes = match got { let bytes = match got {
Ok(b) => b, Ok(b) => b,
Err(FetchOutcome::Offline) => { Err(FetchOutcome::Offline) => {
@@ -3641,6 +3733,36 @@ pub fn spawn_face_sweep(
} }
}; };
if work == SweepWork::Measure {
let image = dr_types::ImageId(req.image_id as u64);
match measure_one_native(
&gpu,
&mut embedder,
&catalog,
image,
&model_id,
&bytes,
) {
Ok(n) => {
images += 1;
found += n;
if tx
.send(FaceSweepMessage::Indexed { image, faces: n })
.is_err()
{
log::info!("face sweep: cancelled after {images} image(s)");
pool.release_all(&*backend).await;
return;
}
}
Err(e) => {
log::debug!("face sweep: measuring {}: {e}", req.path);
lane_failed += 1;
}
}
continue;
}
match index_one_native(&gpu, &mut detector, &mut embedder, &bytes, &options) { match index_one_native(&gpu, &mut detector, &mut embedder, &bytes, &options) {
Ok((faces, edge, proxy)) => { Ok((faces, edge, proxy)) => {
// Before the detections, so a kill between the two // Before the detections, so a kill between the two
@@ -3855,6 +3977,51 @@ fn index_one_native(
Ok((faces, edge, proxy)) Ok((faces, edge, proxy))
} }
/// TRACES: FR-CULL-8 | FR-CULL-9
/// Render one original at native resolution and embed its stored faces again.
///
/// [`index_one_native`]'s sibling for the measuring pass: the same render, no
/// detection, and the result written *over* the faces rather than in place of
/// them. Returns how many faces were measured.
///
/// The proxy is left alone. The faces here were found on a pass that stored
/// one, or [`faces_without_proxy`] would have claimed the image first.
fn measure_one_native(
gpu: &dr_gpu::GpuContext,
embedder: &mut dr_face::Embedder,
catalog: &Catalog,
image: dr_types::ImageId,
model_id: &str,
bytes: &[u8],
) -> Result<usize, String> {
let faces = dr_catalog::faces::unmeasured_on_image(catalog.connection(), image, model_id)
.map_err(|e| e.to_string())?;
if faces.is_empty() {
return Ok(0);
}
let frame = render_native(gpu, bytes)?;
let edge = frame.width.max(frame.height);
let measured = crate::faces::measure_native(
embedder,
&frame.rgba,
frame.width as usize,
frame.height as usize,
&faces,
)
.map_err(|e| e.to_string())?;
let n = measured.measured.len();
dr_catalog::faces::record_measurements(
catalog.connection(),
image,
model_id,
edge,
&measured.measured,
&measured.dropped,
)
.map_err(|e| e.to_string())?;
Ok(n)
}
/// The class the whole-library pass fills. /// The class the whole-library pass fills.
/// ///
/// Grid only, deliberately. The large class is four times the transfer for a /// Grid only, deliberately. The large class is four times the transfer for a
@@ -4252,12 +4419,15 @@ pub fn shared_face_models_dir() -> PathBuf {
/// 3. **The system directories.** Where a package installs them — the Arch /// 3. **The system directories.** Where a package installs them — the Arch
/// package puts the pair in `/usr/share/darkroom/models`. Last, so anything /// package puts the pair in `/usr/share/darkroom/models`. Last, so anything
/// the user placed themselves outranks what the package shipped. /// the user placed themselves outranks what the package shipped.
pub fn face_models(account: &Account) -> Option<(PathBuf, PathBuf)> { pub fn face_models(
fn pair(dir: PathBuf) -> Option<(PathBuf, PathBuf)> { account: &Account,
let detector = dir.join("scrfd_500m_640.onnx"); detector: dr_types::FaceDetector,
) -> Option<(PathBuf, PathBuf)> {
let pair = |dir: PathBuf| {
let detector = dir.join(detector.file_name());
let embedder = dir.join("arcface_mbf_b1.onnx"); let embedder = dir.join("arcface_mbf_b1.onnx");
(detector.is_file() && embedder.is_file()).then_some((detector, embedder)) (detector.is_file() && embedder.is_file()).then_some((detector, embedder))
} };
pair(face_models_dir(account)) pair(face_models_dir(account))
.or_else(|| pair(shared_face_models_dir())) .or_else(|| pair(shared_face_models_dir()))
.or_else(|| system_face_models_dirs().into_iter().find_map(pair)) .or_else(|| system_face_models_dirs().into_iter().find_map(pair))
@@ -5937,6 +6107,7 @@ mod tests {
confidence: 0.9, confidence: 0.9,
embedding: vec![0u8; 1024], embedding: vec![0u8; 1024],
crop_px: 120.0, crop_px: 120.0,
quality: None,
crop: Vec::new(), crop: Vec::new(),
model_id: "w600k_mbf".into(), model_id: "w600k_mbf".into(),
}; };
@@ -5986,6 +6157,7 @@ mod tests {
confidence: 0.9, confidence: 0.9,
embedding: vec![0u8; 1024], embedding: vec![0u8; 1024],
crop_px: 120.0, crop_px: 120.0,
quality: None,
crop: Vec::new(), crop: Vec::new(),
model_id: "w600k_mbf".into(), model_id: "w600k_mbf".into(),
}; };
@@ -6053,6 +6225,53 @@ mod tests {
assert_eq!(faces_unindexed(&catalog, "other").unwrap().len(), 3); assert_eq!(faces_unindexed(&catalog, "other").unwrap().len(), 3);
} }
/// The state schema V14 leaves: a face with no quality and an image with
/// no marker. It is the measuring pass's work, and *only* that pass's — a
/// full re-detection of the same image would throw away every suggestion
/// on it for nothing.
#[test]
fn an_unmeasured_face_is_measured_rather_than_re_detected() {
let catalog = with_images(3);
let ids = image_ids(&catalog);
let stored = |quality: Option<f32>| dr_catalog::faces::DetectedFace {
x: 0.1,
y: 0.1,
w: 0.2,
h: 0.2,
landmarks: [(0.0, 0.0); 5],
confidence: 0.9,
embedding: vec![0u8; 1024],
crop_px: 120.0,
quality,
crop: Vec::new(),
model_id: "w600k_mbf".into(),
};
let conn = catalog.connection();
dr_catalog::faces::record_detections(conn, ids[0], "w600k_mbf", 4000, &[stored(None)])
.unwrap();
dr_catalog::faces::record_detections(
conn,
ids[1],
"w600k_mbf",
4000,
&[stored(Some(18.0))],
)
.unwrap();
// What V14 does to the first: the marker goes, the face stays.
dr_catalog::faces::clear_index_marker(conn, ids[0], "w600k_mbf").unwrap();
let measure = faces_unmeasured(&catalog, "w600k_mbf").unwrap();
assert_eq!(measure.len(), 1);
assert_eq!(measure[0].image_id, ids[0].0 as i64);
assert!(measure[0].full_resolution);
// Not re-detected, marker or no marker; the third image, never seen,
// still is.
let detect = faces_unindexed(&catalog, "w600k_mbf").unwrap();
assert_eq!(detect.len(), 1);
assert_eq!(detect[0].image_id, ids[2].0 as i64);
}
#[test] #[test]
fn a_scoped_grid_shows_only_that_collections_images() { fn a_scoped_grid_shows_only_that_collections_images() {
use dr_catalog::collections::{self as coll, CollectionKind}; use dr_catalog::collections::{self as coll, CollectionKind};
@@ -7012,6 +7231,7 @@ mod tests {
confidence: 0.9, confidence: 0.9,
embedding: vec![0u8; 1024], embedding: vec![0u8; 1024],
crop_px: 120.0, crop_px: 120.0,
quality: None,
crop: Vec::new(), crop: Vec::new(),
model_id: "w600k_mbf".into(), model_id: "w600k_mbf".into(),
}; };
+23
View File
@@ -391,6 +391,15 @@ pub struct LibraryController {
/// wrong shape of question. A `Cell` because the page can change it while /// wrong shape of question. A `Cell` because the page can change it while
/// a library is open. /// a library is open.
timeline_bars: std::cell::Cell<u32>, timeline_bars: std::cell::Cell<u32>,
/// TRACES: FR-CULL-8
/// Which face pipeline this device indexes with, from the settings page.
///
/// Held here for the same reason as the two above: the derived sync runs
/// from a sweep's completion and from the Sync button, neither of which
/// has the settings in reach, and the shards it exports and adopts are
/// keyed on this id. A `RefCell` of the id rather than the enum so that
/// nothing here has to know how a detector becomes a model id.
face_model_id: RefCell<String>,
/// Where every worker this module starts reports what it is doing. /// Where every worker this module starts reports what it is doing.
/// ///
/// Held on the controller rather than passed to each function because the /// Held on the controller rather than passed to each function because the
@@ -458,9 +467,22 @@ impl LibraryController {
dr_types::CacheSettings::default().keep_opened_originals, dr_types::CacheSettings::default().keep_opened_originals,
), ),
timeline_bars: std::cell::Cell::new(dr_types::LibrarySettings::default().timeline_bars), timeline_bars: std::cell::Cell::new(dr_types::LibrarySettings::default().timeline_bars),
face_model_id: RefCell::new(dr_types::FaceDetector::default().model_id().to_string()),
}) })
} }
/// TRACES: FR-CULL-8
/// Which face pipeline the next sync exports and adopts under.
///
/// Set on every settings edit like the bars above. A sync already running
/// keeps the id it started with, which is right: its shards are half
/// written under that one.
pub fn set_face_model_id(&self, id: &str) {
if *self.face_model_id.borrow() != id {
*self.face_model_id.borrow_mut() = id.to_string();
}
}
/// TRACES: FR-CAT-6 /// TRACES: FR-CAT-6
/// How many bars the capture-time axis is cut into. /// How many bars the capture-time axis is cut into.
/// ///
@@ -4016,6 +4038,7 @@ fn start_derived_sync(window: &AppWindow, ctl: &Rc<LibraryController>) {
catalog_path, catalog_path,
library::place_path(&conn.account), library::place_path(&conn.account),
scratch, scratch,
ctl.face_model_id.borrow().clone(),
); );
let timer = slint::Timer::default(); let timer = slint::Timer::default();
+232 -6
View File
@@ -149,6 +149,26 @@ impl Running {
} }
} }
/// TRACES: FR-DEV-19c
/// One of [`crate::develop::MASK_COLOURS`] as the compositor draws it.
///
/// The table is linear sRGB because that is what the shader mixes in; the
/// swatch on the panel is display-encoded, so the conversion is here and not
/// in the table — a swatch that showed the linear values would be a darker,
/// duller colour than the one it promised.
fn mask_colour(index: usize) -> slint::Color {
let [r, g, b] = crate::develop::MASK_COLOURS[index % crate::develop::MASK_COLOURS.len()];
let encode = |v: f32| {
let v = v.clamp(0.0, 1.0);
if v <= 0.003_130_8 {
v * 12.92
} else {
1.055 * v.powf(1.0 / 2.4) - 0.055
}
};
slint::Color::from_rgb_f32(encode(r), encode(g), encode(b))
}
/// A descriptor name as a list label. /// A descriptor name as a list label.
/// ///
/// Only the first letter, because the names in `models/scene/categories.txt` /// Only the first letter, because the names in `models/scene/categories.txt`
@@ -210,6 +230,9 @@ pub(crate) fn sync(window: &AppWindow, session: &Rc<RefCell<Option<DevelopSessio
band_softness, band_softness,
hue, hue,
hue_width, hue_width,
shown: s.mask_shown(&id),
colour: mask_colour(s.mask_colour(&id)),
colour_index: s.mask_colour(&id) as i32,
id: id.into(), id: id.into(),
label: label.into(), label: label.into(),
enabled, enabled,
@@ -264,6 +287,16 @@ pub(crate) fn sync(window: &AppWindow, session: &Rc<RefCell<Option<DevelopSessio
}; };
masking.set_parts(ModelRc::new(VecModel::from(parts))); masking.set_parts(ModelRc::new(VecModel::from(parts)));
// TRACES: FR-DEV-19c
// Pushed rather than assumed, because the session is where it lives. The
// palette goes with it so the swatches and the shader read one table.
masking.set_mask_view_style(s.mask_view_style() as i32);
masking.set_mask_colours(ModelRc::new(VecModel::from(
(0..crate::develop::MASK_COLOURS.len())
.map(mask_colour)
.collect::<Vec<_>>(),
)));
let (radius, hardness, flow) = s.brush(); let (radius, hardness, flow) = s.brush();
masking.set_brush_radius(radius); masking.set_brush_radius(radius);
masking.set_brush_hardness(hardness); masking.set_brush_hardness(hardness);
@@ -279,7 +312,16 @@ pub(crate) fn sync(window: &AppWindow, session: &Rc<RefCell<Option<DevelopSessio
// proxy-sized RGBA buffer — a megabyte or so — and rebuilding it on every // proxy-sized RGBA buffer — a megabyte or so — and rebuilding it on every
// slider event would be a memcpy per frame for a picture that changes only // slider event would be a memcpy per frame for a picture that changes only
// when the level does. // when the level does.
match s.overlay_image() { //
// TRACES: FR-DEV-19c
// **Stood down while a mask is being shown.** The two overlays answer
// different questions — this one is what the model *detected*, the reveal
// is what a layer resolves to — and both at once is a false-coloured
// picture over a tinted one, through which neither can be read. The one
// describing the layer being worked on wins, because by the time a mask
// has been chosen the detections are what the photographer is choosing
// *between* rather than what they are looking at.
match s.overlay_image().filter(|_| !s.any_mask_shown()) {
Some(image) => { Some(image) => {
window.set_region_overlay(image); window.set_region_overlay(image);
window.set_overlay_on(true); window.set_overlay_on(true);
@@ -558,6 +600,7 @@ pub(crate) fn wire(
{ {
let weak = window.as_weak(); let weak = window.as_weak();
let session = session.clone(); let session = session.clone();
let redraw = redraw.clone();
let rows = rows.clone(); let rows = rows.clone();
window window
.global::<Masking>() .global::<Masking>()
@@ -587,6 +630,11 @@ pub(crate) fn wire(
// The scope changed, so the adjust panel below is now describing a // The scope changed, so the adjust panel below is now describing a
// different chain. // different chain.
sync_rows(&w, &rows, &session); sync_rows(&w, &rows, &session);
// TRACES: FR-DEV-19c
// And so has the picture, when a mask is being shown: the
// reveal follows the selection, so choosing another layer
// draws another mask.
redraw(&w);
}); });
} }
{ {
@@ -841,15 +889,102 @@ pub(crate) fn wire(
{ {
let weak = window.as_weak(); let weak = window.as_weak();
let session = session.clone(); let session = session.clone();
window.global::<Masking>().on_tool_picked(move |_tool| { let redraw = redraw.clone();
// The tool itself lives in the interface — it arms a gesture and window.global::<Masking>().on_tool_picked(move |tool| {
// changes no pixel — so nothing is set here. What this does is // **The tool is written back here, and that is the whole of this
// re-sync, because arming the brush is what makes the parts of a // handler.** `Masking.tool` is an `in` property: the panel reads
// mask worth showing. // it to light the right chip and `app.slint` reads it to decide
// whether a drag on the photograph paints, but only Rust may write
// it. So a version of this that recorded nothing left the strip
// reporting "Select" however many times "Paint" was pressed, and
// the paint area was never armed — the brush, the parts, the whole
// of FR-DEV-19b, reachable from no control in the application.
//
// It still changes no pixel, which is what the previous note was
// getting at: the tool arms a gesture and belongs to the
// interface, not to the edit, so it takes no history step and is
// not stored.
let Some(w) = weak.upgrade() else { return }; let Some(w) = weak.upgrade() else { return };
let tool = tool.clamp(0, 2);
w.global::<Masking>().set_tool(tool);
// TRACES: FR-DEV-19c
// Arming a brush shows the mask, if nothing was showing it. The
// same nudge `on_part_added` makes and on the same argument: a
// photographer about to correct an edge by hand needs to see the
// edge, and "the tool did nothing" is what a stroke into an
// invisible mask looks like.
//
// A nudge on an explicit action, never a standing rule. Turning
// the view off and then picking the eraser leaves it off — the
// trap `overlay-hidden` documents is that an automatic reveal
// which re-arms a switch somebody turned off is worse than no
// automatic reveal at all.
if tool > 0 {
if let Some(s) = session.borrow_mut().as_mut() {
if let Some(id) = s.active_mask().map(str::to_owned) {
if !s.mask_shown(&id) {
s.set_mask_shown(&id, true);
}
}
}
}
// Re-synced because arming the brush is what makes the parts of a
// mask worth showing.
sync(&w, &session); sync(&w, &session);
redraw(&w);
}); });
} }
// TRACES: FR-DEV-19c
// Three handlers, one shape: change how the masks are looked at, then a
// redraw and not only a sync, because the reveal is in the composed
// shader and what changed is the picture rather than the panel.
{
let weak = window.as_weak();
let session = session.clone();
let redraw = redraw.clone();
window
.global::<Masking>()
.on_mask_view_style_picked(move |style| {
let Some(w) = weak.upgrade() else { return };
if let Some(s) = session.borrow_mut().as_mut() {
s.set_mask_view_style(style.max(0) as usize);
}
sync(&w, &session);
redraw(&w);
});
}
{
let weak = window.as_weak();
let session = session.clone();
let redraw = redraw.clone();
window
.global::<Masking>()
.on_mask_shown_toggled(move |id, shown| {
let Some(w) = weak.upgrade() else { return };
if let Some(s) = session.borrow_mut().as_mut() {
s.set_mask_shown(&id, shown);
}
sync(&w, &session);
redraw(&w);
});
}
{
let weak = window.as_weak();
let session = session.clone();
let redraw = redraw.clone();
window
.global::<Masking>()
.on_mask_colour_picked(move |id, colour| {
let Some(w) = weak.upgrade() else { return };
if let Some(s) = session.borrow_mut().as_mut() {
s.set_mask_colour(&id, colour.max(0) as usize);
}
sync(&w, &session);
redraw(&w);
});
}
// TRACES: FR-DEV-19a // TRACES: FR-DEV-19a
{ {
let weak = window.as_weak(); let weak = window.as_weak();
@@ -936,6 +1071,34 @@ pub(crate) fn wire(
redraw(&w); redraw(&w);
}); });
} }
// TRACES: FR-DEV-19b
{
let weak = window.as_weak();
let session = session.clone();
let redraw = redraw.clone();
let rows = rows.clone();
window.global::<Masking>().on_add_brush(move || {
let Some(w) = weak.upgrade() else { return };
let made = session
.borrow_mut()
.as_mut()
.and_then(|s| s.add_brush_mask())
.is_some();
// Armed, and this is not a convenience: the layer covers no pixel
// until something is painted into it, so a press that made a row
// and left the pointer in "select" would be indistinguishable from
// a button that did nothing. The same reasoning as `on_part_added`
// below. Showing the mask is `DevelopSession::show_new_mask`'s
// job and every route to a new layer goes through it, so only the
// tool is set here.
if made {
w.global::<Masking>().set_tool(1);
}
sync(&w, &session);
sync_rows(&w, &rows, &session);
redraw(&w);
});
}
// TRACES: FR-DEV-10 // TRACES: FR-DEV-10
{ {
let weak = window.as_weak(); let weak = window.as_weak();
@@ -1127,6 +1290,12 @@ pub(crate) fn reset(window: &AppWindow) {
masking.set_segmenting(false); masking.set_segmenting(false);
masking.set_refining(false); masking.set_refining(false);
masking.set_segmented(false); masking.set_segmented(false);
// TRACES: FR-DEV-19b | FR-DEV-19c
// The tool and the mask view are both about one selected layer, and the
// next photograph has none. Left standing, they would arm a brush over a
// photograph with nothing to paint into and claim a mask was being shown.
masking.set_tool(0);
masking.set_mask_view_style(0);
masking.set_masks(ModelRc::new(VecModel::<MaskRow>::default())); masking.set_masks(ModelRc::new(VecModel::<MaskRow>::default()));
masking.set_subjects(ModelRc::new(VecModel::<SubjectRow>::default())); masking.set_subjects(ModelRc::new(VecModel::<SubjectRow>::default()));
masking.set_categories(ModelRc::new(VecModel::<CategoryRow>::default())); masking.set_categories(ModelRc::new(VecModel::<CategoryRow>::default()));
@@ -1491,3 +1660,60 @@ mod tests {
); );
} }
} }
/// TRACES: FR-DEV-19b | FR-DEV-19c
/// The order the canvas handlers are declared in, which decides which of them
/// receives a press.
///
/// Not a unit test of anything this file does, and here rather than nowhere
/// because there is nowhere better: it guards a fault that compiles, passes
/// every other test, and takes four separate tools out of the application at
/// once.
///
/// Slint hit-tests siblings front-to-back and a `TouchArea` grabs the first
/// press it is offered, so **the last handler declared is the one that wins**.
/// The full-canvas pan/zoom area must therefore come *first*, and `app.slint`
/// had it last — with each of the four handlers behind it carrying a comment
/// claiming it sat "above the pan/zoom area" because it was written earlier in
/// the file.
#[cfg(test)]
mod canvas_order {
/// Where each canvas handler is declared, by byte offset.
fn at(needle: &str) -> usize {
let source = include_str!("../ui/app.slint");
source
.find(needle)
.unwrap_or_else(|| panic!("app.slint no longer contains `{needle}`"))
}
#[test]
fn the_pan_backstop_is_declared_before_every_tool_it_would_swallow() {
let pan = at("--- the pan/zoom backstop ---");
for tool in [
"pick := TouchArea",
"paint := TouchArea",
"Develop.repairing && root.total > 0",
"root.sampling && root.total > 0",
] {
assert!(
pan < at(tool),
"`{tool}` is declared before the pan/zoom area, so the pan area \
is in front of it and will take every press it was meant to get"
);
}
}
/// The other end of the same rule: a handle drawn on the photograph has to
/// beat the tool armed over it, or a gradient cannot be moved while one is.
#[test]
fn the_gradient_handles_are_declared_last() {
let handles = at("GradientHandles {");
for behind in ["pick := TouchArea", "paint := TouchArea"] {
assert!(
handles > at(behind),
"`{behind}` is declared after GradientHandles and would swallow \
a press meant for a handle"
);
}
}
}
+16
View File
@@ -1608,6 +1608,22 @@ mod tests {
perms.set_mode(0o500); perms.set_mode(0o500);
std::fs::set_permissions(&dir, perms.clone()).unwrap(); std::fs::set_permissions(&dir, perms.clone()).unwrap();
// Root is not refused by a mode, and CI's desktop job runs as root
// inside its container: there the directory is as writable as it ever
// was and the "failed" write succeeds. Probed rather than assumed from
// the uid, because what the test needs is the refusal itself, and a
// filesystem mounted without permission checks would pass the uid
// test and fail this one all the same.
let probe = dir.join("probe");
let enforced = std::fs::write(&probe, b"").is_err();
if !enforced {
let _ = std::fs::remove_file(&probe);
perms.set_mode(0o700);
std::fs::set_permissions(&dir, perms).unwrap();
eprintln!("directory modes are not enforced here (root?); skipping");
return;
}
let failed = presets.insert("Warm", Preset::default()).is_err(); let failed = presets.insert("Warm", Preset::default()).is_err();
// Restore the permissions before asserting, so a failure here does not // Restore the permissions before asserting, so a failure here does not
+22
View File
@@ -201,6 +201,28 @@ impl Segmentation {
.is_some_and(|c| c.refinement.is_some()) .is_some_and(|c| c.refinement.is_some())
} }
/// Where this category's refine control should start on *this*
/// photograph.
///
/// [`dr_segment::STRICTNESS_OFF`] where nothing was fitted, and otherwise
/// whatever [`dr_segment::Refinement::gentle`] finds the frame will bear —
/// see there for why a constant could not do it and what a constant cost.
///
/// Costs one to four `apply` passes, tens of milliseconds each. That is
/// why it is asked here, when a layer is made, rather than for every
/// category during the precompute: eight categories nobody masked would be
/// seconds added to a wait, and the answer is only wanted for the one that
/// was clicked.
pub fn category_default_refine(&self, name: &str) -> f32 {
self.categories
.iter()
.find(|c| &*c.name == name)
.and_then(|c| Some((c.refinement.as_ref()?, &c.mask)))
.map_or(dr_segment::STRICTNESS_OFF, |(refinement, mask)| {
refinement.gentle(mask)
})
}
/// Replace one instance in place, keeping every other index and the /// Replace one instance in place, keeping every other index and the
/// signature unchanged. /// signature unchanged.
/// ///
+42 -2
View File
@@ -26,8 +26,8 @@ use std::rc::Rc;
use dr_types::settings::budget; use dr_types::settings::budget;
use dr_types::{ use dr_types::{
CollisionPolicy, ColourSpace, ExportFormat, ExportTarget, GroupNavigation, LibrarySettings, CollisionPolicy, ColourSpace, ExportFormat, ExportTarget, FaceDetector, GroupNavigation,
OutputSharpening, ScreenSize, Settings, SizingMode, LibrarySettings, OutputSharpening, ScreenSize, Settings, SizingMode,
}; };
use slint::ComponentHandle; use slint::ComponentHandle;
@@ -186,6 +186,24 @@ pub fn render(window: &AppWindow, controller: &SettingsController) {
); );
window.set_settings_cache_usage(controller.usage_label.borrow().clone().into()); window.set_settings_cache_usage(controller.usage_label.borrow().clone().into());
// --- faces ---------------------------------------------------------
//
// TRACES: FR-CULL-8
// Labelled from the enum's own order, so the chip index and
// `FaceDetector::ALL` cannot disagree about which detector was picked.
window.set_settings_face_detector_labels(slint::ModelRc::new(slint::VecModel::from(
FaceDetector::ALL
.iter()
.map(|d| slint::SharedString::from(d.label()))
.collect::<Vec<_>>(),
)));
window.set_settings_face_detector_selected(
FaceDetector::ALL
.iter()
.position(|d| *d == s.faces.detector)
.unwrap_or(0) as i32,
);
// --- library ------------------------------------------------------- // --- library -------------------------------------------------------
// //
// Labelled here rather than in the page, so the chips cannot offer a count // Labelled here rather than in the page, so the chips cannot offer a count
@@ -484,6 +502,28 @@ pub fn wire<F, G>(
}); });
} }
// --- faces ---------------------------------------------------------
//
// TRACES: FR-CULL-8
// Notified like a budget change, because the effect is outside this
// page: the library controller carries the pipeline id into the next
// sync, and the coverage line under the picker has to be re-counted
// under the new id — which the caller's `on_budget_changed` does.
{
let weak = window.as_weak();
let ctl = controller.clone();
let notify = on_budget_changed.clone();
window.on_settings_face_detector_picked(move |i| {
let Some(w) = weak.upgrade() else { return };
let Some(choice) = FaceDetector::ALL.get(i as usize).copied() else {
return;
};
ctl.edit(|s| s.faces.detector = choice);
notify(&ctl.snapshot());
render(&w, &ctl);
});
}
// --- export -------------------------------------------------------- // --- export --------------------------------------------------------
{ {
let weak = window.as_weak(); let weak = window.as_weak();
+119 -84
View File
@@ -879,6 +879,12 @@ export component AppWindow inherits Window {
in property <int> settings-target-selected: 0; in property <int> settings-target-selected: 0;
in property <string> settings-error: ""; in property <string> settings-error: "";
/// TRACES: FR-CULL-8
/// Which face detector the indexing pass runs.
in property <[string]> settings-face-detector-labels;
in property <int> settings-face-detector-selected: 0;
callback settings-face-detector-picked(int);
callback settings-group-nav-picked(int); callback settings-group-nav-picked(int);
callback settings-format-picked(int); callback settings-format-picked(int);
callback settings-quality-changed(int); callback settings-quality-changed(int);
@@ -1350,6 +1356,9 @@ in property <bool> panel-visible: true;
face-indexing: root.identity-indexing; face-indexing: root.identity-indexing;
face-coverage: root.identity-coverage; face-coverage: root.identity-coverage;
face-model-missing: root.identity-model-missing; face-model-missing: root.identity-model-missing;
face-detector-labels: root.settings-face-detector-labels;
face-detector-selected: root.settings-face-detector-selected;
face-detector-picked(i) => { root.settings-face-detector-picked(i); }
library-open: root.library-open; library-open: root.library-open;
thumbnail-library() => { root.library-thumbnail-all(); } thumbnail-library() => { root.library-thumbnail-all(); }
index-faces() => { root.identity-index(); } index-faces() => { root.identity-index(); }
@@ -1986,11 +1995,116 @@ in property <bool> panel-visible: true;
cancelled => { self.last-scale = 1.0; } cancelled => { self.last-scale = 1.0; }
} }
// Region picking, above the pan/zoom area so a click // --- the pan/zoom backstop ---------------------------------
// reaches it first. A separate area rather than a branch //
// inside the one below: panning wants press-drag-release // **Declared first among the canvas handlers, because in
// and picking wants a click, and interleaving the two in // Slint that puts it last in line for a press.**
// one handler is how a drag ends up selecting a region the //
// Slint hit-tests siblings `FrontToBack`
// (`i-slint-core`'s `send_mouse_event_to_item`), a
// `TouchArea` answers `GrabMouse` on any press it is
// enabled for, and the first grab aborts the traversal.
// Front means *last declared*. So this — full-canvas, and
// enabled for everything but a crop — took every press in
// the viewport, and the four handlers below it never
// received one: painting a mask, clicking a subject on the
// photograph, placing a repair and sampling a neutral were
// all dead, each of them carrying a comment claiming it sat
// "above the pan/zoom area" because it was written first.
//
// Nothing about the geometry says which wins, so nothing
// about the geometry can be adjusted to fix it. The order
// is the fix, and the rule to keep is: **the general case
// goes at the top of the file and the specific ones after
// it.** `GradientHandles` further down is the other end of
// the same rule, and is why dragging a handle has always
// worked while everything between it and here did not.
if root.total > 0 && root.load-error == "": TouchArea {
x: 0; y: 0;
width: 100%;
height: 100%;
// A pan is only meaningful once there is something outside
// the viewport to reach.
mouse-cursor: root.zoomed ? MouseCursor.grab : MouseCursor.default;
enabled: !Develop.cropping;
property <length> last-x;
property <length> last-y;
pointer-event(ev) => {
if (ev.kind == PointerEventKind.down) {
self.last-x = self.mouse-x;
self.last-y = self.mouse-y;
}
}
// GESTURE: Move a magnified photograph about
// where: Develop
// touch: Drag it
// pointer: Drag it
// why: Only once there is something outside the
// viewport to reach, which is why the
// cursor becomes a hand exactly then. The
// view is clamped to the frame: panning
// past the edge would show undefined area
// beside the photograph, and that reads as
// a rendering fault rather than as the end
// of the picture.
moved => {
if (self.pressed && root.zoomed) {
// Fractions of the *visible* area, which is what
// the session's pan expects. Negated: dragging
// right moves the image right, so the window onto
// it moves left.
root.pan-by(
-(self.mouse-x - self.last-x) / max(parent.shown-w, 1px),
-(self.mouse-y - self.last-y) / max(parent.shown-h, 1px),
);
self.last-x = self.mouse-x;
self.last-y = self.mouse-y;
}
}
scroll-event(ev) => {
if (ev.delta-y == 0) {
return reject;
}
// Anchored on the pointer, in fractions of the shown
// image, so whatever is under the cursor stays there.
root.zoom-at(
ev.delta-y > 0 ? 1.15 : 1.0 / 1.15,
(self.mouse-x - parent.shown-x) / max(parent.shown-w, 1px),
(self.mouse-y - parent.shown-y) / max(parent.shown-h, 1px),
);
return accept;
}
// TRACES: FR-UI-4
// Fit and 1:1, which is what FR-UI-4 asks a double
// tap for. It used to drop straight to fit, so the
// gesture only ever did half its job — and the half
// it did not do is the one that matters, since
// nothing else in the interface reached 100% at all.
//
// Anchored on the pointer, in fractions of the shown
// image, exactly as the wheel above is: the detail
// being inspected is the one under the finger, and a
// toggle that jumped to the centre would ask for a
// pan afterwards every single time.
double-clicked => {
root.inspect-toggled(
(self.mouse-x - parent.shown-x) / max(parent.shown-w, 1px),
(self.mouse-y - parent.shown-y) / max(parent.shown-h, 1px),
);
}
}
// Region picking, declared after the pan/zoom area so a
// click reaches it first — see the note there for why that
// is the way round it is. A separate area rather than a
// branch inside it: panning wants press-drag-release and
// picking wants a click, and interleaving the two in one
// handler is how a drag ends up selecting a region the
// user was only scrolling past. // user was only scrolling past.
if root.region-picking && root.total > 0 && root.load-error == "": pick := TouchArea { if root.region-picking && root.total > 0 && root.load-error == "": pick := TouchArea {
x: parent.shown-x; x: parent.shown-x;
@@ -2125,85 +2239,6 @@ in property <bool> panel-visible: true;
} }
} }
if root.total > 0 && root.load-error == "": TouchArea {
x: 0; y: 0;
width: 100%;
height: 100%;
// A pan is only meaningful once there is something outside
// the viewport to reach.
mouse-cursor: root.zoomed ? MouseCursor.grab : MouseCursor.default;
enabled: !Develop.cropping;
property <length> last-x;
property <length> last-y;
pointer-event(ev) => {
if (ev.kind == PointerEventKind.down) {
self.last-x = self.mouse-x;
self.last-y = self.mouse-y;
}
}
// GESTURE: Move a magnified photograph about
// where: Develop
// touch: Drag it
// pointer: Drag it
// why: Only once there is something outside the
// viewport to reach, which is why the
// cursor becomes a hand exactly then. The
// view is clamped to the frame: panning
// past the edge would show undefined area
// beside the photograph, and that reads as
// a rendering fault rather than as the end
// of the picture.
moved => {
if (self.pressed && root.zoomed) {
// Fractions of the *visible* area, which is what
// the session's pan expects. Negated: dragging
// right moves the image right, so the window onto
// it moves left.
root.pan-by(
-(self.mouse-x - self.last-x) / max(parent.shown-w, 1px),
-(self.mouse-y - self.last-y) / max(parent.shown-h, 1px),
);
self.last-x = self.mouse-x;
self.last-y = self.mouse-y;
}
}
scroll-event(ev) => {
if (ev.delta-y == 0) {
return reject;
}
// Anchored on the pointer, in fractions of the shown
// image, so whatever is under the cursor stays there.
root.zoom-at(
ev.delta-y > 0 ? 1.15 : 1.0 / 1.15,
(self.mouse-x - parent.shown-x) / max(parent.shown-w, 1px),
(self.mouse-y - parent.shown-y) / max(parent.shown-h, 1px),
);
return accept;
}
// TRACES: FR-UI-4
// Fit and 1:1, which is what FR-UI-4 asks a double
// tap for. It used to drop straight to fit, so the
// gesture only ever did half its job — and the half
// it did not do is the one that matters, since
// nothing else in the interface reached 100% at all.
//
// Anchored on the pointer, in fractions of the shown
// image, exactly as the wheel above is: the detail
// being inspected is the one under the finger, and a
// toggle that jumped to the centre would ask for a
// pan afterwards every single time.
double-clicked => {
root.inspect-toggled(
(self.mouse-x - parent.shown-x) / max(parent.shown-w, 1px),
(self.mouse-y - parent.shown-y) / max(parent.shown-h, 1px),
);
}
}
// --- crop overlay ------------------------------------------ // --- crop overlay ------------------------------------------
// //
+28
View File
@@ -60,6 +60,7 @@ component Ink inherits Path {
// collection collection-smart // collection collection-smart
// offline offline-held // offline offline-held
// photo crop mask repair // photo crop mask repair
// eye eye-closed
export component Icon inherits Rectangle { export component Icon inherits Rectangle {
in property <string> name; in property <string> name;
/// The single colour the whole drawing takes. Named `ink` rather than /// The single colour the whole drawing takes. Named `ink` rather than
@@ -335,6 +336,33 @@ export component Icon inherits Rectangle {
commands: "M 12 3.6 A 8.4 8.4 0 0 1 12 20.4 Z"; commands: "M 12 3.6 A 8.4 8.4 0 0 1 12 20.4 Z";
} }
// TRACES: FR-DEV-19c
// An eye, open: the almond and the pupil. The pupil is filled so that at
// 16px the open eye reads as a dot with a lid rather than as two arcs
// that could be anything, and it takes the same ink as the rest so the
// whole glyph can be drawn in the colour of the mask it shows.
if root.name == "eye": Ink {
stroke: root.ink;
stroke-width: root.weight;
commands: "M 2.4 12 C 5.2 6.8 8.4 4.8 12 4.8 C 15.6 4.8 18.8 6.8 21.6 12 "
+ "C 18.8 17.2 15.6 19.2 12 19.2 C 8.4 19.2 5.2 17.2 2.4 12 Z";
}
if root.name == "eye": Ink {
fill: root.ink;
stroke-width: root.weight;
commands: "M 12 8.6 A 3.4 3.4 0 1 0 12 15.4 A 3.4 3.4 0 1 0 12 8.6 Z";
}
// An eye, closed: the lower lid alone, with three lashes. The lid is the
// same lower curve as the open eye's, so the two glyphs sit at the same
// height and toggling between them does not make the row jump.
if root.name == "eye-closed": Ink {
stroke: root.ink;
stroke-width: root.weight;
commands: "M 2.4 12 C 5.2 17.2 8.4 19.2 12 19.2 C 15.6 19.2 18.8 17.2 21.6 12 "
+ "M 12 19.2 L 12 22 M 6.6 18 L 5 20.4 M 17.4 18 L 19 20.4";
}
// TRACES: FR-DEV-8 // TRACES: FR-DEV-8
// A repair: two circles, one sampling into the other. That is literally // A repair: two circles, one sampling into the other. That is literally
// what a spot is here — a source and a destination, which is why its // what a spot is here — a source and a destination, which is why its
+32 -1
View File
@@ -51,6 +51,15 @@ export struct IdentityFace {
// with has causes, and "the face was 41 pixels across" is one the user can // with has causes, and "the face was 41 pixels across" is one the user can
// act on by finding a better photograph. // act on by finding a better photograph.
crop-px: int, crop-px: int,
// "Quality 17.3" — the model's own reading of how recognisable the crop
// was, composed in Rust like `confidence` and for the same reason. The
// other cause a bad suggestion can have, and the one the grouping pass
// acts on: a face below the floor is placed but never compared against.
quality: string,
// Whether the face clears that floor. Drawn, not just known, because a
// group that has gathered none of a person's other photographs has an
// explanation the user can only see if every member's label says it.
in-gallery: bool,
// Part of the current multi-select — what a split would carry. // Part of the current multi-select — what a split would carry.
picked: bool, picked: bool,
} }
@@ -110,6 +119,28 @@ component FaceCell inherits Rectangle {
vertical-alignment: center; vertical-alignment: center;
} }
// The quality, over the foot of the crop. In the picture rather
// than the caption because the caption is the verdict controls'
// and at the compact width there is no room beside them; and
// dimmed below the floor, which is the one state of it that
// changes what the grouping does.
Rectangle {
x: 0;
y: parent.height - self.height;
width: parent.width;
height: 16px;
background: Theme.ground.with-alpha(0.55);
Text {
text: face.quality;
color: face.in-gallery ? Theme.ink-dim : Theme.warn-ink;
font-size: Theme.text-sm;
horizontal-alignment: center;
vertical-alignment: center;
width: parent.width;
height: parent.height;
}
}
// A confirmed face carries a quiet marker rather than a badge: the // A confirmed face carries a quiet marker rather than a badge: the
// grid is mostly confirmed once the user has worked through it, and // grid is mostly confirmed once the user has worked through it, and
// a loud mark on the common case is just noise. // a loud mark on the common case is just noise.
@@ -754,7 +785,7 @@ export component IdentityScreen inherits Rectangle {
border-radius: Theme.radius; border-radius: Theme.radius;
background: Theme.surface-raised; background: Theme.surface-raised;
Text { Text {
text: "No face model installed — indexing is off."; text: "The chosen face detector is not installed — indexing is off.";
color: Theme.warn-ink; color: Theme.warn-ink;
font-size: Theme.text-sm; font-size: Theme.text-sm;
} }
+234 -50
View File
@@ -9,6 +9,7 @@
import { Theme } from "theme.slint"; import { Theme } from "theme.slint";
import { Develop } from "session.slint"; import { Develop } from "session.slint";
import { Button, Caption, IconButton, Label, PanelHeading, Value } from "widgets.slint"; import { Button, Caption, IconButton, Label, PanelHeading, Value } from "widgets.slint";
import { Icon } from "icons.slint";
import { Segmented, SliderRow } from "controls.slint"; import { Segmented, SliderRow } from "controls.slint";
/// One layer in the stack. /// One layer in the stack.
@@ -66,6 +67,15 @@ export struct MaskRow {
/// Whether it currently changes any pixel. A bare selection does not, and /// Whether it currently changes any pixel. A bare selection does not, and
/// saying so is what stops it reading as broken. /// saying so is what stops it reading as broken.
adjusted: bool, adjusted: bool,
/// TRACES: FR-DEV-19c
/// Whether its mask is drawn over the photograph — the eye on its row.
shown: bool,
/// The colour it is drawn in, and which swatch that is. Both, because the
/// row draws the colour and the swatch strip lights the index, and
/// deriving one from the other here would mean a second copy of the
/// palette in this file.
colour: color,
colour-index: int,
} }
/// One selection inside a layer's mask (FR-DEV-19a). /// One selection inside a layer's mask (FR-DEV-19a).
@@ -167,6 +177,9 @@ component MaskEntry inherits Rectangle {
callback part-removed(int); callback part-removed(int);
/// Join a fresh painted correction: 0 adds, 1 subtracts. /// Join a fresh painted correction: 0 adds, 1 subtracts.
callback part-added(int); callback part-added(int);
/// TRACES: FR-DEV-19c
callback shown-toggled(bool);
callback colour-picked(int);
height: layout.preferred-height; height: layout.preferred-height;
background: root.data.selected ? Theme.surface-raised : transparent; background: root.data.selected ? Theme.surface-raised : transparent;
@@ -258,6 +271,41 @@ component MaskEntry inherits Rectangle {
} }
} }
// GESTURE: Show or hide one mask on the photograph
// where: Develop
// touch: Tap the eye on its row
// pointer: Click the eye on its row
// why: A mask is judged by seeing where it falls, and two
// are judged by seeing where they meet — so each row
// has its own eye rather than the panel having one,
// and the eye is drawn in the colour the mask shows
// in, so the row says which shape on the picture is
// its. Nothing about the edit changes: this is how
// the photograph is looked at, and takes no history
// step.
//
// TRACES: FR-DEV-19c
eye := TouchArea {
width: Theme.touch-target;
height: Theme.touch-target;
mouse-cursor: pointer;
enabled: root.enabled;
clicked => { root.shown-toggled(!root.data.shown); }
Icon {
x: (parent.width - self.width) / 2;
y: (parent.height - self.height) / 2;
size: 18px;
name: root.data.shown ? "eye" : "eye-closed";
// Open, in the mask's own colour; closed, in the dim ink
// every idle control wears — so a glance down the stack
// says which masks are on the picture and in what.
ink: root.data.shown
? root.data.colour
: (eye.has-hover ? Theme.ink : Theme.ink-dim);
}
}
remove := TouchArea { remove := TouchArea {
width: Theme.touch-target; width: Theme.touch-target;
height: Theme.touch-target; height: Theme.touch-target;
@@ -370,6 +418,109 @@ component MaskEntry inherits Rectangle {
} }
} }
// TRACES: FR-DEV-19b
// **The brush lives with the mask it paints into.** It used to sit
// at the top of the panel, a screen's height from the row that
// had just been made, and appeared only once a row was selected —
// so the whole of the answer to "how do I paint" was a strip
// somebody had to scroll up to find and had no reason to look
// for. Here it is under the parts it adds to, which is what a
// stroke does.
//
// In the panel and not in the tool rail, still: the rail's
// entries arm a gesture for the whole photograph, and a brush has
// no meaning without a mask to paint into.
Segmented {
label: "Brush";
// Said on the model's own masks especially: a subject or a
// category arrives approximately right, and the correction
// the photographer reaches for is a stroke where it stopped
// short and an erase where it leaked — which becomes a part
// of this mask, joined to the model's, and can be taken out
// again from the list above.
hint: (root.data.kind == "subject" || root.data.kind == "category")
? "Correct the model's outline by hand: Paint adds to this "
+ "mask under the pointer, Erase takes away. Each becomes "
+ "a part above, and a stroke is one step in the history."
: "Paint adds to this mask under the pointer; Erase takes "
+ "away. A stroke is one step in the history.";
options: ["Select", "Paint", "Erase"];
selected: Masking.tool;
picked(i) => { Masking.tool-picked(i); }
}
if Masking.tool > 0: SliderRow {
label: "Size";
value: Masking.brush-radius;
default-value: 0.05;
minimum: 0.005;
maximum: 0.4;
precision: 3;
changed(v) => { Masking.brush-changed(v, Masking.brush-hardness, Masking.brush-flow); }
reset => { Masking.brush-changed(0.05, Masking.brush-hardness, Masking.brush-flow); }
}
if Masking.tool > 0: SliderRow {
label: "Hardness";
value: Masking.brush-hardness;
default-value: 0.5;
minimum: 0.0;
maximum: 1.0;
precision: 2;
changed(v) => { Masking.brush-changed(Masking.brush-radius, v, Masking.brush-flow); }
reset => { Masking.brush-changed(Masking.brush-radius, 0.5, Masking.brush-flow); }
}
if Masking.tool > 0: SliderRow {
label: "Flow";
value: Masking.brush-flow;
default-value: 1.0;
minimum: 0.05;
maximum: 1.0;
precision: 2;
changed(v) => { Masking.brush-changed(Masking.brush-radius, Masking.brush-hardness, v); }
reset => { Masking.brush-changed(Masking.brush-radius, Masking.brush-hardness, 1.0); }
}
// TRACES: FR-DEV-19c
// The colour this mask is shown in. Swatches rather than a picker:
// six colours that can be told apart at half strength over a
// photograph is the whole of the requirement, and a wheel would
// offer a thousand that cannot. Pressing one opens the eye, since
// choosing a colour for a mask nobody can see would be a control
// that changes no pixel.
VerticalLayout {
spacing: 4px;
Caption { text: "Shown in"; }
HorizontalLayout {
spacing: Theme.gap-sm;
alignment: start;
for c[i] in Masking.mask-colours: swatch := TouchArea {
width: Theme.control-height;
height: Theme.control-height;
mouse-cursor: pointer;
enabled: root.enabled;
clicked => { root.colour-picked(i); }
Rectangle {
width: 18px;
height: 18px;
x: (parent.width - self.width) / 2;
y: (parent.height - self.height) / 2;
border-radius: 9px;
background: c;
// The chosen one is ringed rather than enlarged,
// so the strip does not shimmer as the choice
// moves along it.
border-width: (root.data.shown && root.data.colour-index == i) ? 2px : 0px;
border-color: Theme.ink;
opacity: swatch.has-hover || (root.data.shown && root.data.colour-index == i) ? 1.0 : 0.7;
}
}
}
}
SliderRow { SliderRow {
label: "Opacity"; label: "Opacity";
value: root.data.opacity; value: root.data.opacity;
@@ -412,7 +563,15 @@ component MaskEntry inherits Rectangle {
+ "outline onto the edge the photograph actually has. " + "outline onto the edge the photograph actually has. "
+ "Zero is the model's own outline."; + "Zero is the model's own outline.";
value: root.data.refine; value: root.data.refine;
default-value: 4; // Zero, which is the model's own outline — and *not* the
// position a layer starts at, which is chosen per photograph
// because a nat of evidence means different things over a
// smooth sky and over a stone facade (see
// `dr_segment::Refinement::gentle`). A fixed default here used
// to be 4, which is half travel and looks like the natural
// resting place; on an ordinary frame it takes three quarters
// of the category away, so "reset" emptied the mask.
default-value: 0;
minimum: 0; minimum: 0;
maximum: 8; maximum: 8;
// Tenths, not whole nats. The interval between a flag going // Tenths, not whole nats. The interval between a flag going
@@ -421,7 +580,7 @@ component MaskEntry inherits Rectangle {
// the range it exists to explore. // the range it exists to explore.
precision: 1; precision: 1;
changed(v) => { root.refine-changed(v); } changed(v) => { root.refine-changed(v); }
reset => { root.refine-changed(4); } reset => { root.refine-changed(0); }
} }
// The band (FR-DEV-10). Above the edge controls because for a // The band (FR-DEV-10). Above the edge controls because for a
@@ -574,7 +733,27 @@ export global Masking {
in property <[PartRow]> parts; in property <[PartRow]> parts;
/// TRACES: FR-DEV-19b /// TRACES: FR-DEV-19b
/// What a drag on the photograph does: 0 selects, 1 paints, 2 erases. /// What a drag on the photograph does: 0 selects, 1 paints, 2 erases.
///
/// Written by Rust and read here, like every other `in` property on this
/// global: `tool-picked` is what the strip reports, and the answer comes
/// back through this. A version of the handler that recorded nothing left
/// the strip stuck on "Select" and the brush reachable from no control.
in property <int> tool: 0; in property <int> tool: 0;
/// TRACES: FR-DEV-19c
/// How shown masks are drawn over the photograph — tint, alpha or edge,
/// as an index into `dr_pipeline::mask::RevealStyle::ALL`. One style for
/// every shown mask; *which* masks are shown is each row's own eye.
///
/// **Not `overlay-hidden`.** That switch belongs to the region overlay,
/// which is a picture of what the *model detected*; this is the mask a
/// layer actually resolves to, feather, morphology, invert and all. Two
/// overlays that look alike and mean different things is worse than
/// either, so they are named apart and switched apart.
in property <int> mask-view-style: 0;
/// The colours a mask may be shown in — `MASK_COLOURS` in the session,
/// pushed rather than declared here so the swatch and the shader are
/// reading one table.
in property <[color]> mask-colours;
/// The brush: radius as a fraction of the frame's shorter edge, hardness, /// The brush: radius as a fraction of the frame's shorter edge, hardness,
/// and flow. /// and flow.
in property <float> brush-radius: 0.05; in property <float> brush-radius: 0.05;
@@ -622,6 +801,10 @@ export global Masking {
/// TRACES: FR-DEV-19b /// TRACES: FR-DEV-19b
callback tool-picked(int); callback tool-picked(int);
/// TRACES: FR-DEV-19c
callback mask-view-style-picked(int);
callback mask-shown-toggled(string, bool);
callback mask-colour-picked(string, int);
callback brush-changed(float, float, float); callback brush-changed(float, float, float);
/// TRACES: FR-DEV-19a /// TRACES: FR-DEV-19a
callback part-selected(string, int); callback part-selected(string, int);
@@ -630,6 +813,8 @@ export global Masking {
callback part-added(string, int); callback part-added(string, int);
callback add-gradient(bool); callback add-gradient(bool);
/// TRACES: FR-DEV-19b
callback add-brush();
/// `true` asks for the colour range rather than the brightness one. /// `true` asks for the colour range rather than the brightness one.
callback add-range(bool); callback add-range(bool);
callback add-subject(int); callback add-subject(int);
@@ -660,51 +845,6 @@ export component MaskPanel inherits Rectangle {
if !Develop.enabled: Caption { text: "No image"; } if !Develop.enabled: Caption { text: "No image"; }
// TRACES: FR-DEV-19b
// **In the panel and not in the tool rail.** The rail's entries arm a
// gesture for the whole photograph; a brush has no meaning without a
// mask to paint into, and a rail entry that lit up with nothing
// selected would either do nothing or silently make a layer. Here the
// strip can simply be absent until there is something to paint on.
if Develop.enabled && Masking.parts.length > 0: Segmented {
options: ["Select", "Paint", "Erase"];
selected: Masking.tool;
picked(i) => { Masking.tool-picked(i); }
}
if Develop.enabled && Masking.tool > 0: SliderRow {
label: "Size";
value: Masking.brush-radius;
default-value: 0.05;
minimum: 0.005;
maximum: 0.4;
precision: 3;
changed(v) => { Masking.brush-changed(v, Masking.brush-hardness, Masking.brush-flow); }
reset => { Masking.brush-changed(0.05, Masking.brush-hardness, Masking.brush-flow); }
}
if Develop.enabled && Masking.tool > 0: SliderRow {
label: "Hardness";
value: Masking.brush-hardness;
default-value: 0.5;
minimum: 0.0;
maximum: 1.0;
precision: 2;
changed(v) => { Masking.brush-changed(Masking.brush-radius, v, Masking.brush-flow); }
reset => { Masking.brush-changed(Masking.brush-radius, 0.5, Masking.brush-flow); }
}
if Develop.enabled && Masking.tool > 0: SliderRow {
label: "Flow";
value: Masking.brush-flow;
default-value: 1.0;
minimum: 0.05;
maximum: 1.0;
precision: 2;
changed(v) => { Masking.brush-changed(Masking.brush-radius, Masking.brush-hardness, v); }
reset => { Masking.brush-changed(Masking.brush-radius, Masking.brush-hardness, 1.0); }
}
// --- the region map --------------------------------------------- // --- the region map ---------------------------------------------
// //
// Segmentation is a thing the user starts. Half a second of watershed // Segmentation is a thing the user starts. Half a second of watershed
@@ -744,10 +884,18 @@ export component MaskPanel inherits Rectangle {
// a mask is judged against the photograph under it, and you cannot see // a mask is judged against the photograph under it, and you cannot see
// that photograph through the thing describing it. // that photograph through the thing describing it.
// //
// Reads as its own action rather than its state: "Show the mask" is // Reads as its own action rather than its state: "Show what was found"
// what pressing it will do, not what is currently true. // is what pressing it will do, not what is currently true.
//
// TRACES: FR-DEV-19c
// **Named for what it actually hides**, which is not the mask. This
// said "Show the mask" / "Hide the mask" while switching the
// false-coloured picture of what the *model detected* — and now that
// there is a control which really does show a mask (the eye on each
// row of the stack), two things called the same thing and meaning
// different ones would be worse than either.
if Develop.enabled && Masking.segmented: Button { if Develop.enabled && Masking.segmented: Button {
text: Masking.overlay-hidden ? "Show the mask" : "Hide the mask"; text: Masking.overlay-hidden ? "Show what was found" : "Hide what was found";
clicked => { Masking.overlay-hidden = !Masking.overlay-hidden; } clicked => { Masking.overlay-hidden = !Masking.overlay-hidden; }
} }
@@ -892,6 +1040,24 @@ export component MaskPanel inherits Rectangle {
// --- the stack ----------------------------------------------------- // --- the stack -----------------------------------------------------
Caption { text: "Masks"; } Caption { text: "Masks"; }
// TRACES: FR-DEV-19c
// How every shown mask is drawn. One strip for all of them, above
// the rows whose eyes decide which: a tint beside an outline beside an
// alpha would be three pictures that cannot be read against each
// other, where three tints in three colours are one.
//
// Three chips rather than a toggle because the three answer three
// different questions and no one of them answers all three: a tint
// says whether the right thing is selected, an alpha says where the
// edge is, an outline says whether that edge is registered against
// detail the other two hide.
if Develop.enabled && Masking.masks.length > 0: Segmented {
label: "Show masks as";
options: ["Tint", "Alpha", "Edge"];
selected: Masking.mask-view-style;
picked(i) => { Masking.mask-view-style-picked(i); }
}
// Gradients need no segmentation, so they are offered whether or not // Gradients need no segmentation, so they are offered whether or not
// one has been computed — a graduated filter over a sky is a local // one has been computed — a graduated filter over a sky is a local
// adjustment that never needed to know what a sky is. // adjustment that never needed to know what a sky is.
@@ -908,6 +1074,22 @@ export component MaskPanel inherits Rectangle {
enabled: Develop.enabled; enabled: Develop.enabled;
clicked => { Masking.add-gradient(true); } clicked => { Masking.add-gradient(true); }
} }
// TRACES: FR-DEV-19b
// **A mask that is nothing but a hand.** Beside the gradients
// because it belongs to the same answer to "where" — and because
// without it the only way to reach the brush was to make a
// gradient nobody wanted and paint into that.
//
// It covers nothing until a stroke lands, so pressing this arms
// the brush as well as making the layer: a row that appeared and
// changed no pixel, with the pointer still in "select", would look
// exactly like a button that did nothing.
Button {
text: "Paint";
enabled: Develop.enabled;
clicked => { Masking.add-brush(); }
}
} }
// Ranges, on their own row rather than beside the gradients. A // Ranges, on their own row rather than beside the gradients. A
@@ -968,6 +1150,8 @@ export component MaskPanel inherits Rectangle {
part-join-picked(i, j) => { Masking.part-join-picked(mask.id, i, j); } part-join-picked(i, j) => { Masking.part-join-picked(mask.id, i, j); }
part-removed(i) => { Masking.part-removed(mask.id, i); } part-removed(i) => { Masking.part-removed(mask.id, i); }
part-added(j) => { Masking.part-added(mask.id, j); } part-added(j) => { Masking.part-added(mask.id, j); }
shown-toggled(on) => { Masking.mask-shown-toggled(mask.id, on); }
colour-picked(i) => { Masking.mask-colour-picked(mask.id, i); }
} }
// No caption saying which chain the sliders below are pointed at. // No caption saying which chain the sliders below are pointed at.
+27 -1
View File
@@ -91,6 +91,13 @@ export component SettingsPage inherits Rectangle {
/// No model on disk, so the pass cannot run at all. /// No model on disk, so the pass cannot run at all.
in property <bool> face-model-missing: false; in property <bool> face-model-missing: false;
callback index-faces(); callback index-faces();
/// TRACES: FR-CULL-8
/// Which detector the pass finds faces with — docs/faces.md §12.3 for
/// what each costs and finds. The choice is a model change: coverage is
/// counted per pipeline, so picking another one starts from nothing.
in property <[string]> face-detector-labels;
in property <int> face-detector-selected: 0;
callback face-detector-picked(int);
// --- cache --------------------------------------------------------- // --- cache ---------------------------------------------------------
in-out property <string> original-budget; in-out property <string> original-budget;
@@ -444,13 +451,32 @@ export component SettingsPage inherits Rectangle {
wrap: word-wrap; wrap: word-wrap;
} }
if root.library-open: Segmented {
label: "Face detector";
options: root.face-detector-labels;
selected: root.face-detector-selected;
enabled: !root.face-indexing;
picked(i) => { root.face-detector-picked(i); }
}
if root.library-open: Caption {
text: "Fast misses the small faces in a group and "
+ "mistakes the odd dog for one. Balanced "
+ "finds a seventh more for almost the same "
+ "time. Thorough finds the most and takes "
+ "three times as long. Changing it indexes "
+ "the library again; names you have "
+ "confirmed are kept.";
wrap: word-wrap;
}
if root.library-open && root.face-coverage != "": Caption { if root.library-open && root.face-coverage != "": Caption {
text: root.face-coverage; text: root.face-coverage;
wrap: word-wrap; wrap: word-wrap;
} }
if root.library-open && root.face-model-missing: Caption { if root.library-open && root.face-model-missing: Caption {
text: "No face model is installed, so this cannot run."; text: "The chosen detector is not installed, so this cannot run.";
wrap: word-wrap; wrap: word-wrap;
} }