dtourolle/DarkRoom
1
0
Fork 0
Code Issues 39 Pull Requests Actions Packages Projects Releases 20 Wiki Activity

Compare commits

base: dtourolle/DarkRoom:v0.20.0
Branches Tags
dtourolle/DarkRoom:master dtourolle/DarkRoom:release/0.21 dtourolle/DarkRoom:feat/macos-inference
dtourolle/DarkRoom:v0.24.0 dtourolle/DarkRoom:v0.23.0 dtourolle/DarkRoom:v0.22.1 dtourolle/DarkRoom:v0.22.0 dtourolle/DarkRoom:v0.21.0 dtourolle/DarkRoom:v0.20.0 dtourolle/DarkRoom:v0.19.4 dtourolle/DarkRoom:v0.19.3 dtourolle/DarkRoom:v0.19.2 dtourolle/DarkRoom:v0.19.1 dtourolle/DarkRoom:v0.19.0 dtourolle/DarkRoom:v0.18.2 dtourolle/DarkRoom:v0.18.1 dtourolle/DarkRoom:v0.18.0 dtourolle/DarkRoom:v0.17.0 dtourolle/DarkRoom:v0.16.0 dtourolle/DarkRoom:v0.15.0 dtourolle/DarkRoom:v0.14.1 dtourolle/DarkRoom:v0.14.0 dtourolle/DarkRoom:v0.13.6 dtourolle/DarkRoom:v0.13.5 dtourolle/DarkRoom:v0.13.4 dtourolle/DarkRoom:v0.13.3 dtourolle/DarkRoom:v0.13.2 dtourolle/DarkRoom:v0.13.1 dtourolle/DarkRoom:v0.13.0 dtourolle/DarkRoom:v0.12.1 dtourolle/DarkRoom:v0.12.0 dtourolle/DarkRoom:v0.11.0 dtourolle/DarkRoom:v0.10.0 dtourolle/DarkRoom:v0.8.0 dtourolle/DarkRoom:v0.7.0 dtourolle/DarkRoom:v0.6.0 dtourolle/DarkRoom:v0.5.0 dtourolle/DarkRoom:v0.3.0
..
compare: dtourolle/DarkRoom:v0.21.0
Branches Tags
dtourolle/DarkRoom:master dtourolle/DarkRoom:release/0.21 dtourolle/DarkRoom:feat/macos-inference
dtourolle/DarkRoom:v0.24.0 dtourolle/DarkRoom:v0.23.0 dtourolle/DarkRoom:v0.22.1 dtourolle/DarkRoom:v0.22.0 dtourolle/DarkRoom:v0.21.0 dtourolle/DarkRoom:v0.20.0 dtourolle/DarkRoom:v0.19.4 dtourolle/DarkRoom:v0.19.3 dtourolle/DarkRoom:v0.19.2 dtourolle/DarkRoom:v0.19.1 dtourolle/DarkRoom:v0.19.0 dtourolle/DarkRoom:v0.18.2 dtourolle/DarkRoom:v0.18.1 dtourolle/DarkRoom:v0.18.0 dtourolle/DarkRoom:v0.17.0 dtourolle/DarkRoom:v0.16.0 dtourolle/DarkRoom:v0.15.0 dtourolle/DarkRoom:v0.14.1 dtourolle/DarkRoom:v0.14.0 dtourolle/DarkRoom:v0.13.6 dtourolle/DarkRoom:v0.13.5 dtourolle/DarkRoom:v0.13.4 dtourolle/DarkRoom:v0.13.3 dtourolle/DarkRoom:v0.13.2 dtourolle/DarkRoom:v0.13.1 dtourolle/DarkRoom:v0.13.0 dtourolle/DarkRoom:v0.12.1 dtourolle/DarkRoom:v0.12.0 dtourolle/DarkRoom:v0.11.0 dtourolle/DarkRoom:v0.10.0 dtourolle/DarkRoom:v0.8.0 dtourolle/DarkRoom:v0.7.0 dtourolle/DarkRoom:v0.6.0 dtourolle/DarkRoom:v0.5.0 dtourolle/DarkRoom:v0.3.0
36 Commits
v0.20.0 .. v0.21.0
Author SHA1 Message Date
dtourolle a9dfe66a70 Count the denoise model in the Windows installer's smoke test
🐳 Android image / Build and push (push) Successful in 4s
Details
Build and test / android-image (push) Successful in 5s
Details
Build and test / Desktop (Linux) (push) Successful in 1h27m54s
Details
🐳 Windows image / Build and push (push) Successful in 5s
Details
Build and test / windows-image (push) Successful in 5s
Details
Build and test / Layer separation (push) Successful in 33s
Details
Build and test / Android (aarch64) (push) Successful in 24m50s
Details
Build and test / Windows (x86_64, cross) (push) Successful in 55m19s
Details
Build and test / Publish the release (push) Successful in 1m20s
Details 
package.sh stages models/denoise beside face, scene and inpaint, and
the smoke test counted only the other three, so 0.21.0's Windows job
failed with "expected 14 model files, installed 15". The count reads
the same directories package.sh copies, as its comment intends.
 2026-10-04 02:49:48 -04:00
dtourolle ff0effbfe1 Count the denoise model in the APK's bundled-model list
Benchmarks / CPU and I/O (per commit) (push) Successful in 3m17s
Details
Benchmarks / Frame budget (on demand) (push) Skipped
Details
Traceability / Requirement traces (push) Successful in 56s
Details
Build and test / Android (aarch64) (push) Successful in 47m9s
Details
Build and test / android-image (push) Successful in 3s
Details
🐳 Android image / Build and push (push) Successful in 2s
Details
Build and test / Desktop (Linux) (push) Successful in 1h2m3s
Details
Build and test / windows-image (push) Successful in 2s
Details
🐳 Windows image / Build and push (push) Successful in 1s
Details
Build and test / Layer separation (push) Successful in 30s
Details
Build and test / Windows (x86_64, cross) (push) Failing after 49m14s
Details
Build and test / Publish the release (push) Skipped
Details 
96001480 added mosaic-1408.onnx to BUNDLED as a fifteenth entry and
left the array's declared length at 14, so the Android build failed
and 0.21.0 got no release. The workspace gates never compile the
Android crate, which is why nothing before CI saw it.
 2026-10-03 22:14:26 -04:00
dtourolle 1a03cb52b4 Release 0.21.0
Benchmarks / CPU and I/O (per commit) (push) Successful in 8m38s
Details
Benchmarks / Frame budget (on demand) (push) Skipped
Details
Traceability / Requirement traces (push) Successful in 1m27s
Details
Build and test / Android (aarch64) (push) Failing after 29m52s
Details
Build and test / android-image (push) Successful in 2s
Details
🐳 Android image / Build and push (push) Successful in 2s
Details
Build and test / Desktop (Linux) (push) Successful in 1h2m28s
Details
Build and test / windows-image (push) Successful in 2s
Details
🐳 Windows image / Build and push (push) Successful in 2s
Details
Build and test / Layer separation (push) Successful in 30s
Details
Build and test / Windows (x86_64, cross) (push) Failing after 48m48s
Details
Build and test / Publish the release (push) Skipped
Details
2026-10-03 17:06:49 -04:00
dtourolle ff4b30fbaa Link the inference engine for macOS in a zig container 
`docker/macos` builds for aarch64-apple-darwin from Linux with
cargo-zigbuild. Zig carries libSystem and the C headers, so tract's SIMD
kernels compile and the engine's test binaries and examples link as Mach-O
arm64 — the check `cargo check --target` could not do, because tract's
build script needs a macOS C compiler. Crates that link an Apple framework
(dr-plat's keyring, and so the app) still need the Xcode SDK and fail at
the link; macos.md says so.
 2026-10-03 16:50:37 -04:00
dtourolle c73743394f Add a CoreML rung on macOS 
The macOS ladder was the CPU provider alone, with CoreML listed as a gap.
It is now CoreML, then the CPU, then tract — unmeasured, since nobody here
has a Mac, and safe to ship unmeasured because the probe's clock rejects a
CoreML slower than the CPU and `attempt` refuses one that crashes.

- `Rung::CoreMl`, a compiling rung like TensorRT: an ML Program with every
  compute unit allowed, falling back to the CPU until each model's program
  is built. The embedder stays on the CPU, as on the Hexagon (§7).
- The cache is one directory per model and runtime version. CoreML keys a
  model committed from memory on its input and node names, not its
  weights (ONNX Runtime 1.29, coreml_execution_provider.cc), so two
  exports of one architecture would otherwise share a program.
- The fingerprint on macOS is the chip and the OS release, which ships
  CoreML.
- The desktop looks for the runtime in the bundle's Contents/Frameworks
  and Homebrew's prefixes; fetch-desktop-runtime.sh on a Mac downloads
  ONNX Runtime 1.29.0 for Apple silicon, which carries CoreML.

docs/dev/macos.md says what exists, how to build it, and which log lines
to ask a Mac user for.
 2026-10-03 16:50:37 -04:00
dtourolle 872e35670c Log like a debug build on macOS, where a Mac user can find it 
Nobody working on DarkRoom has a Mac, so every macOS build is in the hands
of someone who can send a log and cannot attach a debugger. Three changes
make that log worth sending:

- The desktop's default filter on macOS is `debug` for every `dr_*` crate,
  the desktop crate and `onnxruntime` (the runtime's own session log).
- The state directory — the log and crash records — is `~/Library/Logs`
  on macOS rather than the `~/.local/state` Finder hides; Console.app
  lists it. Config and data keep the Unix rules.
- A `diagnostic` cargo profile: release plus line tables, so a crash
  record's backtrace reads file:line. On macOS the tables are in the
  `.dSYM` beside the executable, which the bundle must keep.
 2026-10-03 16:49:58 -04:00
dtourolle b562d7b1af Stop retrying a provider that took the app down 
The probe runs in the app's process, and a provider can fail by aborting
rather than by returning an error — XNNPACK did on SCRFD. A rung that does
that once would do it on every launch, before the first photograph is on
screen.

Every session build above the CPU, the probe's and each background
compile's, now writes what it is attempting to `attempt` in the cache
directory first and removes it after. After two launches in a row that
died inside the same attempt it is refused and recorded — a rung in
`failed`, an engine in the new `refused` — until the fingerprint changes.
Two, not one, because quitting during a TensorRT compile leaves the same
file.
 2026-10-03 16:49:58 -04:00
dtourolle 3689b06c35 Send ONNX Runtime's session log to the app's log 
A native session's messages went to ONNX Runtime's stdio logger, which is
nowhere once the app is launched from a menu — and what a provider says
while partitioning a graph (nodes taken, operators declined, a library that
failed to load) is most of what a failed rung tells you. Each session now
forwards them to `log` under the target `onnxruntime`: warnings always,
the runtime's info lines at `debug`, its verbose lines at `trace`.
 2026-10-03 16:49:58 -04:00
dtourolle 64ea44aefe Stop reading dates at the first sign the server is unreachable 
A window of cells whose thumbnails were cached but whose dates were not
sent a header read per cell, and offline each one was three attempts at
a 15 s connect timeout: `is_transient` counts a network error as worth
retrying, and `read_metadata_only` returned a bare bool that could not
say why a read failed. So the grid sat on "reading N dates" for minutes
against a server that was not there, and no banner went up, because
nothing in that loop ever reported the connection.

`read_metadata_only` now returns a `DateRead`: reached, failed, or
offline. An offline error is returned on the first attempt rather than
retried — a dead server answers the second exactly as the first — while
a 423 lock is still retried, which is what the retry was for. The grid's
worker stops on it and sends `Offline`, as its fetch loop already did,
so the banner goes up and the bar stops. The sweep's lanes stop on it
too, one timeout each rather than one per image.
 2026-10-03 16:49:49 -04:00
dtourolle 32a4da0e94 Import DEFAULT_CONTRAST only where the tests use it 
The calibration commit imported it at module level, where only the
test module reads it; the build warned and clippy's -D warnings
refuses that.
 2026-10-03 16:45:02 -04:00
dtourolle 33779a70bd Answer a thumbnail miss with the other stored class before the network 
Offline, a grid zoomed past 256px was blank wherever it had not been
zoomed over before. The store was asked only for the exact class the cell
wanted, and the sweep stores only the grid class, so every zoomed cell
missed and went to a server that was not there — with its 256px thumbnail
sitting in the store the whole time. Online it cost the same round trip,
just without the blank cell at the end of it.

The split now tries the other class on a miss. A smaller one stands in
and the fetch for the real class still goes out; a larger one answers
the request outright, since there is nothing a fetch would improve on.
`ThumbnailReady` carries the class its pixels are, and the drain records
that rather than the batch's class, so a stand-in is replaced on the next
reload instead of being counted as served. `already_served` counts a
held large thumbnail as serving the grid class too, so zooming back out
does not re-read the store for pixels already on screen.

The split moves into `split_by_store` so it can be tested without a
worker thread or a network.
 2026-10-03 16:44:58 -04:00
dtourolle 185e134ead Describe the measured tone and vibrance in DarkRoom's own terms 
The calibration commits named DarkRoom's default curve after another
product and described vibrance as doing what another editor's does at
the same value. The curve is the DNG SDK's reference, so it is called
that; vibrance is scaled to deliver the strength its value names, as
measured against the photographer's earlier exports. Two test names
follow. The measurements and where they came from are unchanged.
 2026-10-03 16:41:11 -04:00
dtourolle 7ae1e27810 Make vibrance deliver the strength its value names 
Vibrance delivered about a third of its nominal effect. Its falloff measured
saturation on scene-linear values, where an ordinary tan reads as 0.78 and
keeps a twentieth of the effect; and its skin guard halved it wherever red led
green led blue — 38 % of the pixels of the gallery's exports, every warm colour
rather than skin. The Vivid presets lean on vibrance, which is why they added
less colour than their values promised.

Saturation is now judged on display-encoded values, the guard covers skin hues
(about 10-50 degrees, not strongly saturated), and the gain is fitted: on 45
Lightroom exports whose only colour setting was Vibrance (about +24), the
measured-to-nominal scale was 1.9 before and 1.08 at a gain of 1.2, so 1.3.
 2026-10-03 15:54:41 -04:00
dtourolle a4ff7ec2b9 Render raws through the DNG reference curve by default, at contrast 1.5 
Decides D21 by measurement. The photo gallery holds Lightroom 6 exports of
raws in the library, each carrying its Camera Raw settings; clustered by
those settings, 663 had no look applied. On 60 of them with their raws,
a third held out, the held-out MSE against Lightroom's JPEG was about 1200
for 0.20.0's sigmoid (0.7 EV darker and flatter), 224 for the DNG reference curve
after baseline exposure, and about 140 once its input is bent by 1.5/1.4
about grey.

So the curve choice defaults to the DNG reference, keeping its index (sidecars
record it), and the default contrast is 1.5. Contrast under the DNG reference curve is
now a power relative to REFERENCE_CONTRAST (1.4), where the table is
untouched; the sigmoid at that contrast still matches the retired base
curve. JPEGs are unaffected: the view transform skips a rendered source.
 2026-10-03 15:54:21 -04:00
dtourolle b69fb3e191 Give the denoise work's test images a baseline exposure 
The learned-denoise branch merged while this one was open, and two of
its test fixtures build a RawImage without the baseline_exposure field
this branch added; zero is the no-op value.
 2026-10-03 14:44:48 -04:00
dtourolle 7a09b640d7 Satisfy clippy on the reference tone curve's data 
One sample of the ACR3 table is 0.70711, which clippy reads as an
approximation of 1/sqrt(2). It is the curve's published value, so the
lint is allowed on the table with that reason rather than the number
replaced. And the pair-count check uses is_multiple_of.
 2026-10-03 14:22:49 -04:00
dtourolle 8294e6b59f Call the reference curve what it is, and drop wording that reads as copying 
The view transform's second curve is the DNG SDK's published reference
rendering — the ACR3 default curve applied by RefBaselineRGBTone — so
it is the "DNG Reference" curve in the panel, D21 and the code, not a
name borrowed from another product. Comments and docs that justified a
choice by another editor doing it ("as their Amount", "so a
photographer arriving from it finds the name") now give the actual
reason. The Vivid presets no longer describe themselves as reaching
for another editor's look; they are DarkRoom's own.

Factual mentions stay: which program wrote the library's DNGs, what
was measured against, and preset import. camera-profiles.md gains §15,
on starting a photograph from the edit it already carries.
 2026-10-03 14:22:49 -04:00
dtourolle e12783da9f Open a photograph with the edit it already carries, when DarkRoom has none 
The library's DNGs carry the photographer's earlier develop settings
in their embedded XMP — the house style their photographs were made
with. A photograph opened with no edit of DarkRoom's now starts from
that earlier edit, translated (HSL bands, highlights, blacks and the
rest), as one undoable step named "Earlier Edit"; from there it is an
ordinary edit, saved with the photograph. Export does the same, so a
photograph never opened exports as opening it would show.

Only on positive evidence that there is no DarkRoom edit: a local file
with no sidecar beside it, or a server that answered "no such file"
with nothing in the cache. The stored-edit fetch now says which
(FetchedSidecar::absent). Offline, unreachable or unreadable never
counts — the earlier edit would otherwise be saved over a real edit
that merely failed to arrive.
 2026-10-03 14:22:49 -04:00
dtourolle 013596e1bd Translate Lightroom's HSL panel, and read the edit inside a DNG 
The library's DNGs carry a Lightroom house look in their embedded XMP
— Blue +58, Aqua +50, Yellow and Purple +23, Highlights -40, Blacks
-20 on most — and that, not the camera profile, is why the same files
look richer in Lightroom. The importer skipped exactly that part: the
HSL panel was on its list of structures it did not translate.

Lightroom's eight HSL bands now map onto the colour mixer, hue,
saturation and luminance each one for one: Aqua to our cyan and Purple
to our violet, the nearest of our twelve bands by hue; chartreuse,
spring, azure and rose are left alone. The figures are a first
translation that lr-fit's measurement against Lightroom's output may
yet scale.

read_embedded finds the XMP packet in a photograph's bytes by its
delimiters and translates it, or answers None for a file whose XMP has
no Camera Raw settings — darktable's sidecars, a camera's own packet.
A test reads the library's _MG_9080.dng when it is present.
 2026-10-03 14:22:48 -04:00
dtourolle 1e8594724e Keep the sigmoid as the default curve; Camera Raw's tone is a choice 
The Camera Raw default rested on comparing against Lightroom previews
of photographs that carry the user's Lightroom edits — HSL saturation
Blue +58, Aqua +50 and more, Highlights -40, Blacks -20, in every
DNG's XMP — so it measured the house look, not Camera Raw's base
rendering. Under the ACR3 curve _MG_9080 renders brighter than its
Lightroom preview (mean 0.39 against 0.31).

So the curve choice's first variant, the default, is the sigmoid again
and every raw renders as in 0.20.0 apart from baseline exposure. D21
and camera-profiles.md §12 now say the default is open, to be decided
by measuring against Lightroom exports of unedited photographs. Tests
that are about Camera Raw's tone choose it explicitly.
 2026-10-03 14:22:48 -04:00
dtourolle 0730ef1016 Sync camera profiles through the library, and name the tone curves 
camera-profiles.md §13: the derived sync pass gains a profiles step,
after the catalog and before the place, that exchanges the profiles
directory with <library>/.darkroom-derived/profiles. Profiles are
immutable and named for what they hold, so name and size decide: it
uploads what the server lacks or holds at another size and downloads
what this device lacks — parsed before it is kept, written beside its
name and renamed — then reloads the set, so a profile copied out of a
DNG on the desktop renders the body's CR2s on the tablet after its
next sync. Like the place it never fails the pass. Not a catalog
table: a schema change would stop an older peer merging at all.

Labels for the view transform's new curve choice: Curve, Camera Raw,
Sigmoid.
 2026-10-03 14:22:47 -04:00
dtourolle db7593f3dc Render raws through Camera Raw's tone by default, after baseline exposure 
The rendering half of camera-profiles.md §11-§12 (D21). The view
transform gains a curve choice — Camera Raw (the default) or D19's
sigmoid. Camera Raw converts to linear ProPhoto, clips to [0, 1], runs
the curve on the largest and smallest channel and places the middle
one at its old fraction between them (RefBaselineRGBTone), and
converts back: hue kept, saturation raised where the curve is steep,
which is where Adobe Standard's look desaturated. White sets the input
scale (1 at its default, so sensor white is display white) and
contrast bends the input about grey (1 at its default).

The curve rides in the profile buffer after the tables: the profile's
own, else the ACR3 default, which the placeholder every profile-less
source binds also carries — so a CR2 with no .dcp still gets Camera
Raw's tone. Baseline exposure is a gain folded into the rendering
matrix at upload; RawImage::color_matrix stays the file's for the
merge's linear DNG.

camera_raw::apply_reference is the CPU statement; GPU tests hold the
shader to it on 256 colours and on greys against the ACR3 table. The
sigmoid's own tests now choose it explicitly.
 2026-10-03 14:22:47 -04:00
dtourolle 38d414912c Read baseline exposure and profile tone curves; carry the ACR3 curve 
The decoding half of camera-profiles.md §11-§12. RawImage gains
baseline_exposure: the file's BaselineExposure plus the chosen
profile's BaselineExposureOffset, as the DNG SDK sums them (+0.25 for
the library's 6D DNGs). A profile copied out of a DNG carries that
DNG's baseline as its offset, so the body's CR2s, which have none,
land at the same total.

ProfileTables gains the profile's ProfileToneCurve, resampled at
decode onto 1025 points with a natural cubic spline; an identity curve
counts as none. dr-types now holds Camera Raw's ACR3 default curve,
RawTherapee's adobe_camera_raw_default_curve copied value for value,
for every raw whose profile has no curve. Nothing renders through
either yet.
 2026-10-03 14:22:46 -04:00
dtourolle b58873ef57 Spec baseline exposure, Camera Raw tone and profile sync (D21) 
camera-profiles.md §11-§14 close what 0.20.0 left open. Baseline
exposure is the file's plus the profile's offset, applied as a gain on
the camera matrix, and a copied profile carries the DNG's baseline so a
CR2 lands at the same brightness. The view transform gains a Camera Raw
curve — the profile's ProfileToneCurve, else the ACR3 default — applied
Camera Raw's way, on the outer channels in linear ProPhoto, and it is
the default for every raw (D21, the user's choice). Profiles sync
through .darkroom-derived/profiles on the server as a step of the
derived sync pass, not as a catalog table.
 2026-10-03 14:22:45 -04:00
dtourolle ababd628ed Show AI denoise in the manual, on a night frame with no faces 
A section after Looking closer: what it is for, the switch and its wait,
Keep grain, which cameras it takes and where its noise figures come from.
The scene opens the Brooklyn Bridge at ISO 8000 from the face-free demo
set at 1:1, switches it on, waits for the result to land and keeps some
grain, with a still before and after. It waits on the app's own log line
rather than a fixed time: the network takes seconds on a GPU and more on
the CPU, which is where the recording X server leaves it (13 s).
 2026-10-03 12:05:26 -04:00
dtourolle 4eb7cf77f5 Record what the learned denoise shipped as, and what was measured 
The grain blend replaces the Amount of §7.2, and why its objection to a
blend does not hold for brightness alone; the Hexagon is out (int8 -6 to
-9 dB); §11 holds the data, the noise model taken from the library, the
model, the validation table, the blind estimate's reach and the speed.
 2026-10-03 11:51:02 -04:00
dtourolle 960014803a Ship the denoise model in the Arch package, the APK and the Windows installer 
Same LFS-pointer guard as the other models; the APK copies it out of its
assets with the rest.
 2026-10-03 11:51:01 -04:00
dtourolle dd43f498fb Run the learned denoise in develop, and export with it 
A Bayer photograph keeps its mosaic in the session and is offered the AI
Denoise switch. Asked for, the network runs on the decode executor from a
hot-pixel-repaired copy — the app's own pass — with the frame's noise from
its best source, and its progress in the activity bar; the classical
demosaic shows until the result lands, and the finished job says where the
noise figures came from. Keep grain is a GrainBlend of the two, made once
per value; the render draws it as its source and the adjust pass never
knows. demosaiced stays the classical result, so the raw histogram, the
white balance picker, masks and segmentation still read the sensor.

The develop view reconciles on a 250 ms poll rather than on each way an
edit can change (slider, undo, preset, version, a sidecar from another
device): two comparisons when nothing changed, and no path that can forget.
A failure is not retried until the switch is toggled. An export of a
photograph that asks for it waits for a running job or computes it.
 2026-10-03 11:51:00 -04:00
dtourolle ad6bb892f3 Carry the learned denoise's switch and grain as edit settings 
Whether to use the learned denoise, and how much grain to keep, are what a
photographer sets, so they travel the one road every setting does: published
as a capability, captured by Preset, stored in the sidecar, replayed by the
undo stack (FR-DEV-3c). Published only on a photograph that can take it, for
the lens switch's reason; the availability is derived from the file and is
not in the state. Off by default, grain 0; a reset returns both.
 2026-10-03 11:22:29 -04:00
dtourolle 8ea3c3181a Upload the learned demosaic's result, and blend grain back into it 
DemosaicedImage::from_rgb_f32 takes the network's linear camera RGB and
stands it beside the classical source of the same photograph: the matrix,
profile tables and as-shot balance are that source's, the id is new, so
nothing downstream can tell which demosaic ran and every cache keyed on the
source sees a new one.

GrainBlend is the denoise's live control. It returns only the brightness of
the noise the network removed, taken after the as-shot balance and handed
back divided by it, so the grain is neutral in the finished picture; colour
speckle and demosaic false colour stay out. It writes a new source rather
than adding a term to the adjust shader: the blend depends on two images and
one number, a 20 MP pass is milliseconds, and a fresh source id is all the
adjust pass's caches need. The test reads it back: at 0 the network's
result, at 1 the same white-balanced step in every channel.
 2026-10-03 11:20:48 -04:00
dtourolle d8304d7c82 Add dr-denoise: the learned demosaic and denoise, without the UI 
The noise model takes the best source the frame has: the body's measured
table (the Canon EOS 6D's, from the library), the DNG's NoiseProfile, or
the frame itself — read, row and column noise from its masked border, and
only the shot gain estimated, from the quietest flat patches. Checked on
130 6D frames, the estimate is within 10 % from ISO 1000 up; the network
loses under 0.3 dB for a sigma off by 15-20 %, so every Bayer body is
eligible.

Tiles of 1408 keep their central 1024 behind a 192-photosite halo, past the
185-photosite receptive field, and the frame is extended by reflection,
which keeps every photosite's colour; a pattern that starts on another
colour is read from one photosite up or left so the network sees RGGB, and
nothing is cropped. The tests run every Bayer phase, tiled against whole,
with a stand-in network of known reach.

The model ships as models/denoise/mosaic-1408.onnx (LFS), trained in
darkroom-denoise on the maintainer's own photographs, GPL like the code.
denoise_raw runs a file end to end: on a 6D frame at ISO 8000 the result
matches the training repository's own path to 2.5e-4 at worst, and takes
3.1 s on TensorRT fp16 (75 dB from f32) or 14.4 s on the CPU.
 2026-10-03 11:15:50 -04:00
dtourolle 20b7bd7663 Feed every input a model declares when probing a rung 
The probe built one zero tensor from the first input and ran the session
with it. Every model so far had one input; the denoiser has two (mosaic and
sigma), so every rung failed with "Missing Input: sigma" and the role was
left on the CPU: 14.4 s for a 20 MP frame where TensorRT fp16 takes 3.1 s.
Zeros now go to each input by name.
 2026-10-03 11:15:49 -04:00
dtourolle 6b0d29cc15 Read a DNG's NoiseProfile 
The converter's measured noise for the body at that ISO, (S, O) per CFA
plane: the learned denoise's best source for a body with no table of its
own (denoise.md §3.3). Read from the header beside the colour tags, and
printed by rawinfo. Checked against tifffile on a 6D DNG at ISO 5000: all
six values agree.
 2026-10-03 10:39:15 -04:00
dtourolle eb91fa02c2 Give the inference engine a denoiser role, kept off the Hexagon 
The learned demosaic-and-denoise (denoise.md) runs through the engine like
every other model. fp16 cost it nothing measurable (0.00 dB at every ISO on
validation tiles), so it takes TensorRT's and MIGraphX's fp16 like the
detectors. int8 cost it 6 to 9 dB, far past a 0.5 dB gate, so the Hexagon
refuses the role outright rather than relying on no int8 sibling existing,
and the tablet runs it on the CPU.
 2026-10-03 10:39:14 -04:00
dtourolle d4248bc0dd Run the app's hot-pixel pass alone, and dump through it 
The learned demosaic replaces the classical one and takes its input, the
mosaic hot_pixels.wgsl leaves (denoise.md §2), so its training data and its
input in the app must come through that pass and not a lookalike. The pass
was recorded inline in Demosaicer::run; it is now built by hot_pass and
recorded by record_hot_pass, which run still uses unchanged, and
Demosaicer::repair_hot_pixels runs it on its own and reads the mosaic back.

mosaic_dump moves to dr-gpu to call it, records how many photosites changed,
and keeps --unrepaired for a raw readout.
 2026-10-03 10:25:28 -04:00
dtourolle 1f266a4478 Dump RAW mosaics for training the learned denoise 
denoise.md §4.4 requires the training repo to read photosites through
dr-decode, not LibRaw, so black and white levels, the active area and the
CFA phase match what the app will feed the network. mosaic_dump reads
`input<TAB>prefix` lines and writes the whole readout as .npy plus a JSON
of what decode and metadata report. The masked border is kept: its
optically black photosites are a free dark frame for the noise profile.
 2026-10-03 10:25:27 -04:00
91 changed files with 5670 additions and 405 deletions
Expand all files Collapse all files
.gitea/workflows/build-and-test.yml
+1 -1
View File
@@ -490,7 +490,7 @@ jobs:
# As many files as package.sh stages: everything but the READMEs in
# the directories it copies. A literal here went stale the first
# time a model was added.
WANT=$(find models/face models/scene models/inpaint -maxdepth 1 -type f ! -name README.md | wc -l)
WANT=$(find models/face models/scene models/inpaint models/denoise -maxdepth 1 -type f ! -name README.md | wc -l)
GOT=$(ls "$INST/models" | wc -l)
[ "$GOT" = "$WANT" ] || { echo "FAIL: expected $WANT model files, installed $GOT"; exit 1; }
# The manual, and every picture it shows, counted the same way.
Cargo.lock
Generated
+43 -25
View File
@@ -1265,7 +1265,7 @@ checksum = "f27ae1dd37df86211c42e150270f82743308803d90a6f6e6651cd730d5e1732f"
[[package]]
name = "darkroom-android"
version = "0.20.0"
version = "0.21.0"
dependencies = [
"android_logger",
"dr-plat",
@@ -1278,7 +1278,7 @@ dependencies = [
[[package]]
name = "darkroom-desktop"
version = "0.20.0"
version = "0.21.0"
dependencies = [
"anyhow",
"dr-plat",
@@ -1454,7 +1454,7 @@ checksum = "d8b14ccef22fc6f5a8f4d7d768562a182c04ce9a3b3157b91390b52ddfdf1a76"
[[package]]
name = "dr-bench"
version = "0.20.0"
version = "0.21.0"
dependencies = [
"anyhow",
"dr-catalog",
@@ -1471,7 +1471,7 @@ dependencies = [
[[package]]
name = "dr-catalog"
version = "0.20.0"
version = "0.21.0"
dependencies = [
"dr-face",
"dr-plat",
@@ -1486,7 +1486,7 @@ dependencies = [
[[package]]
name = "dr-decode"
version = "0.20.0"
version = "0.21.0"
dependencies = [
"dr-types",
"env_logger",
@@ -1498,9 +1498,26 @@ dependencies = [
"zune-jpeg 0.4.21",
]
[[package]]
name = "dr-denoise"
version = "0.21.0"
dependencies = [
"dr-decode",
"dr-gpu",
"dr-inference-engine",
"env_logger",
"log",
"ndarray",
"ort",
"pollster",
"serde",
"serde_norway",
"thiserror 2.0.20",
]
[[package]]
name = "dr-export"
version = "0.20.0"
version = "0.21.0"
dependencies = [
"dr-decode",
"dr-gpu",
@@ -1519,7 +1536,7 @@ dependencies = [
[[package]]
name = "dr-face"
version = "0.20.0"
version = "0.21.0"
dependencies = [
"dr-inference-engine",
"env_logger",
@@ -1532,7 +1549,7 @@ dependencies = [
[[package]]
name = "dr-film"
version = "0.20.0"
version = "0.21.0"
dependencies = [
"log",
"serde",
@@ -1541,7 +1558,7 @@ dependencies = [
[[package]]
name = "dr-gpu"
version = "0.20.0"
version = "0.21.0"
dependencies = [
"bytemuck",
"dr-decode",
@@ -1559,7 +1576,7 @@ dependencies = [
[[package]]
name = "dr-inference-engine"
version = "0.20.0"
version = "0.21.0"
dependencies = [
"env_logger",
"libloading",
@@ -1574,7 +1591,7 @@ dependencies = [
[[package]]
name = "dr-ingest"
version = "0.20.0"
version = "0.21.0"
dependencies = [
"dr-plat",
"dr-types",
@@ -1586,7 +1603,7 @@ dependencies = [
[[package]]
name = "dr-lens"
version = "0.20.0"
version = "0.21.0"
dependencies = [
"lensfun",
"log",
@@ -1594,7 +1611,7 @@ dependencies = [
[[package]]
name = "dr-pano"
version = "0.20.0"
version = "0.21.0"
dependencies = [
"dr-decode",
"dr-inference-engine",
@@ -1608,7 +1625,7 @@ dependencies = [
[[package]]
name = "dr-pipeline"
version = "0.20.0"
version = "0.21.0"
dependencies = [
"dr-types",
"log",
@@ -1617,7 +1634,7 @@ dependencies = [
[[package]]
name = "dr-plat"
version = "0.20.0"
version = "0.21.0"
dependencies = [
"android-native-keyring-store",
"dr-types",
@@ -1633,7 +1650,7 @@ dependencies = [
[[package]]
name = "dr-preset-xmp"
version = "0.20.0"
version = "0.21.0"
dependencies = [
"dr-pipeline",
"log",
@@ -1643,7 +1660,7 @@ dependencies = [
[[package]]
name = "dr-segment"
version = "0.20.0"
version = "0.21.0"
dependencies = [
"dr-inference-engine",
"env_logger",
@@ -1656,7 +1673,7 @@ dependencies = [
[[package]]
name = "dr-sync"
version = "0.20.0"
version = "0.21.0"
dependencies = [
"async-trait",
"dr-plat",
@@ -1670,7 +1687,7 @@ dependencies = [
[[package]]
name = "dr-sync-folder"
version = "0.20.0"
version = "0.21.0"
dependencies = [
"async-trait",
"dr-sync",
@@ -1682,7 +1699,7 @@ dependencies = [
[[package]]
name = "dr-sync-nextcloud"
version = "0.20.0"
version = "0.21.0"
dependencies = [
"async-trait",
"dr-decode",
@@ -1704,7 +1721,7 @@ dependencies = [
[[package]]
name = "dr-thumbs"
version = "0.20.0"
version = "0.21.0"
dependencies = [
"dr-types",
"jpeg-encoder",
@@ -1716,7 +1733,7 @@ dependencies = [
[[package]]
name = "dr-types"
version = "0.20.0"
version = "0.21.0"
dependencies = [
"serde",
"serde_json",
@@ -1725,12 +1742,13 @@ dependencies = [
[[package]]
name = "dr-ui"
version = "0.20.0"
version = "0.21.0"
dependencies = [
"anyhow",
"async-trait",
"dr-catalog",
"dr-decode",
"dr-denoise",
"dr-export",
"dr-face",
"dr-film",
@@ -1773,7 +1791,7 @@ dependencies = [
[[package]]
name = "dr-xmp"
version = "0.20.0"
version = "0.21.0"
dependencies = [
"dr-types",
"log",
@@ -7107,7 +7125,7 @@ checksum = "8df9b6e13f2d32c91b9bd719c00d1958837bc7dec474d94952798cc8e69eeec3"
[[package]]
name = "traceability"
version = "0.20.0"
version = "0.21.0"
dependencies = [
"anyhow",
"proc-macro2",
Cargo.toml
+15 -1
View File
@@ -5,6 +5,7 @@ members = [
"core/dr-catalog",
"core/dr-thumbs",
"core/dr-decode",
"core/dr-denoise",
"core/dr-export",
"core/dr-face",
"core/dr-film",
@@ -32,7 +33,7 @@ members = [
exclude = ["third_party"]
[workspace.package]
version = "0.20.0"
version = "0.21.0"
edition = "2021"
rust-version = "1.92"
license = "GPL-3.0-or-later"
@@ -44,6 +45,7 @@ dr-types = { path = "core/dr-types" }
dr-catalog = { path = "core/dr-catalog" }
dr-thumbs = { path = "core/dr-thumbs" }
dr-decode = { path = "core/dr-decode" }
dr-denoise = { path = "core/dr-denoise" }
dr-export = { path = "core/dr-export" }
# Stated explicitly for the same reason as `dr-segment` below: no dependant
# should drag in an ONNX runtime by accident. Members opt in with
@@ -276,6 +278,18 @@ opt-level = 0
lto = "thin"
codegen-units = 1
# A release build that can say where it panicked: line tables, so a crash
# record's backtrace (`dr_plat::crash`) reads `file.rs:123` rather than bare
# addresses. The macOS build uses it (docs/dev/macos.md) — no one here can
# reproduce a Mac bug, so its reports carry what a debugger would have — at
# the price of a larger binary and no slower code. On macOS the tables land
# in a `.dSYM` beside the executable (rustc's default `packed`), and the
# bundle must carry that directory next to the binary for the backtrace to
# find it.
[profile.diagnostic]
inherits = "release"
debug = "line-tables-only"
# Three upstream crates carry a local patch: wgpu-hal and Slint's Skia
# renderer so that the Android build can draw with wgpu on a rotated display
# (technical-debt.md TD-1), and rawler so that a linear DNG wider than 16 700
README.md
+1 -1
View File
@@ -201,7 +201,7 @@ controls, its place in the chain and its tests.
## Where it stands
**0.20.0**, thirty-four tagged releases in. 193 numbered requirements in
**0.21.0**, thirty-five tagged releases in. 193 numbered requirements in
scope, 85% of them claimed by code and [traced to it](docs/dev/traceability.md);
the rest are written down rather than merely absent.
apps/darkroom-android/src/lib.rs
+2 -1
View File
@@ -335,7 +335,7 @@ fn unpack_bundled_models(app: &slint::android::AndroidApp) {
// The int8 forms beside the three detectors are what the Hexagon runs
// (docs/dev/inference.md §5); the engine loads the sibling when the probe
// chose that rung and ignores it otherwise.
const BUNDLED: [(&std::ffi::CStr, &str); 14] = [
const BUNDLED: [(&std::ffi::CStr, &str); 15] = [
(c"models/scrfd_500m_640.onnx", "scrfd_500m_640.onnx"),
(
c"models/scrfd_500m_640.int8.onnx",
@@ -360,6 +360,7 @@ fn unpack_bundled_models(app: &slint::android::AndroidApp) {
(c"models/categories.txt", "categories.txt"),
// The panorama border filler (FR-MRG-4); MIT, 28 MB.
(c"models/migan-512.onnx", "migan-512.onnx"),
(c"models/mosaic-1408.onnx", "mosaic-1408.onnx"),
];
let dir = dr_ui::shared_face_models_dir();
apps/darkroom-desktop/src/main.rs
+35 -4
View File
@@ -15,6 +15,22 @@ use std::path::PathBuf;
use dr_plat::diagnostics::Installed;
/// What the log keeps when `RUST_LOG` does not say.
#[cfg(not(target_os = "macos"))]
const DEFAULT_LOG: &str =
"info,wgpu_core=warn,wgpu_hal=warn,zbus=warn,tracing=warn,calloop=warn,rawler=warn";
/// The same, and `debug` from this application's own crates and from ONNX
/// Runtime, whose `debug` is how many nodes each provider took
/// (docs/dev/macos.md). Nobody here runs a Mac: every macOS build is in
/// the hands of someone who can send us a log and cannot attach a debugger,
/// so the log is written as if for a debug build. `dr_` is a prefix, and
/// `env_logger` matches directives by prefix, so it names every `dr-*`
/// crate — present and future — without naming a dependency.
#[cfg(target_os = "macos")]
const DEFAULT_LOG: &str = "info,dr_=debug,darkroom_desktop=debug,onnxruntime=debug,\
wgpu_core=warn,wgpu_hal=warn,zbus=warn,tracing=warn,calloop=warn,rawler=warn";
fn main() -> anyhow::Result<()> {
// TRACES: FR-PLAT-WIN-3
// Before the logger, the crash hook and everything else: this exists so a
@@ -33,10 +49,9 @@ fn main() -> anyhow::Result<()> {
// (NFR-OPS-1). `filter()` is asked afterwards because the environment may
// have overridden the default below, and the file must not be quieter than
// the terminal.
let console = env_logger::Builder::from_env(env_logger::Env::default().default_filter_or(
"info,wgpu_core=warn,wgpu_hal=warn,zbus=warn,tracing=warn,calloop=warn,rawler=warn",
))
.build();
let console =
env_logger::Builder::from_env(env_logger::Env::default().default_filter_or(DEFAULT_LOG))
.build();
let level = console.filter();
let logging = dr_plat::diagnostics::install(Box::new(console), level);
@@ -109,5 +124,21 @@ fn runtime_dirs() -> Vec<PathBuf> {
PathBuf::from("/usr/lib/darkroom"),
PathBuf::from("/usr/lib"),
]);
// An app bundle keeps its libraries in `Contents/Frameworks`, beside
// the `Contents/MacOS` the executable is in; then Homebrew's
// `onnxruntime`, Apple silicon's prefix before Intel's. Homebrew's build
// may lack CoreML, which the probe finds out for itself.
#[cfg(target_os = "macos")]
{
if let Ok(exe) = std::env::current_exe() {
if let Some(bin) = exe.parent() {
dirs.push(bin.join("../Frameworks"));
}
}
dirs.extend([
PathBuf::from("/opt/homebrew/lib"),
PathBuf::from("/usr/local/lib"),
]);
}
dirs
}
core/dr-decode/examples/rawinfo.rs
+4
View File
@@ -32,6 +32,10 @@ fn main() {
println!("black {:?}", raw.black_level);
println!("white {}", raw.white_level);
println!("wb_coeffs {:?}", raw.wb_coeffs);
match dr_decode::noise_profile(&bytes) {
Some(p) => println!("noise profile {p:?} ((S, O) per plane)"),
None => println!("noise profile none"),
}
match raw.color_matrix {
Some(m) => {
core/dr-decode/src/dcp.rs
+48 -1
View File
@@ -68,6 +68,11 @@ pub struct Dcp {
/// `ProfileToneCurve`, as stored: input/output pairs. Carried so a copy
/// keeps it, never applied — tone is the view transform's (D19, D20).
pub tone_curve: Option<Vec<f32>>,
/// `BaselineExposureOffset`, in stops: the profile's correction to the
/// file's `BaselineExposure` (camera-profiles.md §11). A profile copied
/// out of a DNG carries that DNG's baseline here, so a raw with no
/// baseline of its own lands at the same brightness.
pub baseline_exposure_offset: f32,
}
impl Dcp {
@@ -120,6 +125,7 @@ impl Dcp {
.get_entry(DngTag::ProfileToneCurve)
.and_then(|e| floats(&e.value))
.filter(|v| v.len() >= 4 && v.len() % 2 == 0),
baseline_exposure_offset: stops(ifd, DngTag::BaselineExposureOffset),
})
}
@@ -218,6 +224,10 @@ impl Dcp {
origin,
hue_sat: hue_sat.filter(|t| !t.is_identity()),
look: self.look.clone().filter(|t| !t.is_identity()),
tone_curve: self
.tone_curve
.as_deref()
.and_then(dr_types::tone::resample_tone_curve),
}
}
@@ -305,6 +315,15 @@ impl Dcp {
if let Some(curve) = &self.tone_curve {
dir.add_value(DngTag::ProfileToneCurve, Value::Float(curve.clone()));
}
if self.baseline_exposure_offset != 0.0 {
dir.add_value(
DngTag::BaselineExposureOffset,
Value::SRational(vec![SRational::new(
(self.baseline_exposure_offset * 100.0).round() as i32,
100,
)]),
);
}
writer.build(dir).map_err(|e| e.to_string())?;
let mut bytes = cursor.into_inner();
// The writer stamps TIFF's 42 in its own byte order; a profile is the
@@ -340,6 +359,15 @@ fn string(ifd: &IFD, tag: DngTag) -> Option<String> {
.filter(|s| !s.is_empty())
}
/// A single rational tag in stops, zero where absent or unreadable — the
/// DNG specification's default for both exposure tags.
fn stops(ifd: &IFD, tag: DngTag) -> f32 {
ifd.get_entry(tag)
.and_then(|e| e.value.get_f32(0).ok().flatten())
.filter(|v| v.is_finite())
.unwrap_or(0.0)
}
fn floats(value: &Value) -> Option<Vec<f32>> {
(0..value.count())
.map(|i| value.get_f32(i).ok().flatten())
@@ -507,7 +535,12 @@ pub fn embedded_in(bytes: &[u8]) -> Option<Dcp> {
.ifd(rawler::decoders::WellKnownIFD::Root)
.ok()
.flatten()?;
Dcp::from_ifd(&root)
// The copy carries the file's baseline as its offset, so a raw from the
// same body that has no baseline of its own — a CR2 — gets the total the
// DNG renders at (camera-profiles.md §11).
let mut profile = Dcp::from_ifd(&root)?;
profile.baseline_exposure_offset += stops(&root, DngTag::BaselineExposure);
Some(profile)
}
/// TRACES: FR-DEV-3e
@@ -518,6 +551,9 @@ pub struct Resolved {
pub profile: Option<CameraProfile>,
pub tables: Option<Arc<ProfileTables>>,
pub embedded: Option<Arc<Dcp>>,
/// Stops to add at render: the file's `BaselineExposure` plus the
/// chosen profile's `BaselineExposureOffset` (camera-profiles.md §11).
pub baseline_exposure: f32,
}
/// TRACES: FR-DEV-3e
@@ -532,6 +568,7 @@ pub fn resolve(
model: &str,
) -> Resolved {
let embedded = root.and_then(Dcp::from_ifd).map(Arc::new);
let file_baseline = root.map_or(0.0, |r| stops(r, DngTag::BaselineExposure));
if let Some(dcp) = &embedded {
let tables = matrices
.as_ref()
@@ -541,6 +578,7 @@ pub fn resolve(
return Resolved {
profile: matrices,
tables,
baseline_exposure: file_baseline + dcp.baseline_exposure_offset,
embedded,
};
}
@@ -553,6 +591,7 @@ pub fn resolve(
tables: (!tables.is_empty()).then(|| Arc::new(tables)),
profile: Some(own),
embedded: None,
baseline_exposure: file_baseline + dcp.baseline_exposure_offset,
};
}
}
@@ -560,6 +599,7 @@ pub fn resolve(
profile: matrices,
tables: None,
embedded: None,
baseline_exposure: file_baseline,
}
}
@@ -609,6 +649,7 @@ mod tests {
],
look: Some(table(4, 2, 3, [0.0, 1.2, 0.95])),
tone_curve: Some(vec![0.0, 0.0, 0.5, 0.6, 1.0, 1.0]),
baseline_exposure_offset: 0.25,
}
}
@@ -627,6 +668,7 @@ mod tests {
assert_eq!(back.unique_camera_model, original.unique_camera_model);
assert_eq!(back.illuminants, original.illuminants);
assert_eq!(back.tone_curve, original.tone_curve);
assert_eq!(back.baseline_exposure_offset, 0.25);
assert_eq!(
back.forward_matrix, original.forward_matrix,
"four decimals, as the file has"
@@ -746,6 +788,11 @@ mod tests {
assert_eq!(tables.origin, ProfileOrigin::Embedded);
assert_eq!(tables.name, "Adobe Standard");
assert!(tables.hue_sat.is_some() && tables.look.is_some());
assert!(
tables.tone_curve.is_none(),
"Adobe Standard has no curve of its own"
);
assert_eq!(raw.baseline_exposure, 0.25);
}
#[test]
core/dr-decode/src/lib.rs
+19
View File
@@ -147,6 +147,12 @@ pub struct RawImage {
/// renders a decoded file renders it through the same profile without
/// having to be told — see camera-profiles.md §3.
pub profile_tables: Option<std::sync::Arc<dr_types::ProfileTables>>,
/// TRACES: FR-DEV-3e
/// Stops the render adds before anything else: the file's
/// `BaselineExposure` plus the profile's `BaselineExposureOffset`
/// (camera-profiles.md §11). Applied by the GPU side as a gain on the
/// camera matrix; `color_matrix` itself stays the file's.
pub baseline_exposure: f32,
/// The body, as rawler cleans the names: what `Make`/`Model` say, and
/// what a `.dcp`'s `UniqueCameraModel` is matched against.
pub make: String,
@@ -544,6 +550,17 @@ pub(crate) fn parse_exif_offset(s: &str) -> Option<i32> {
Some(sign * (h * 60 + m))
}
/// TRACES: FR-DEV-3g
/// The DNG `NoiseProfile` of a file, if it carries one: `(S, O)` per CFA
/// colour plane, variance `S·x + O` in black-to-white normalised units. See
/// [`profile::read_noise_profile`]. Reads the header, not the image.
pub fn noise_profile(bytes: &[u8]) -> Option<Vec<(f32, f32)>> {
use rawler::rawsource::RawSource;
let source = RawSource::new_from_slice(bytes);
let decoder = rawler::get_decoder(&source).ok()?;
profile::read_noise_profile(decoder.as_ref())
}
/// TRACES: FR-RAW-3 | FR-EXP-9
/// Fully decode sensor data.
///
@@ -591,6 +608,7 @@ fn decode_unguarded(bytes: &[u8]) -> Result<RawImage, DecodeError> {
let dcp::Resolved {
profile,
tables: profile_tables,
baseline_exposure,
..
} = dcp::resolve(
profile,
@@ -689,6 +707,7 @@ fn decode_unguarded(bytes: &[u8]) -> Result<RawImage, DecodeError> {
samples_per_pixel,
profile,
profile_tables,
baseline_exposure,
make: image.camera.clean_make.clone(),
model: image.camera.clean_model.clone(),
})
core/dr-decode/src/profile.rs
+29
View File
@@ -740,6 +740,35 @@ pub fn read_dng_matrices(decoder: &dyn rawler::decoders::Decoder) -> DngMatrices
}
}
/// TRACES: FR-DEV-3g
/// The DNG `NoiseProfile` tag (51041): the converter's measured noise for
/// this body at this ISO, as `(S, O)` per CFA colour plane, so that a
/// photosite's variance is `S·x + O` with `x` normalised black-to-white.
///
/// One pair means all planes share it. `None` where the file has no such
/// tag — every proprietary raw, and DNGs from converters that do not measure
/// — or where a value is not a finite non-negative number. The learned
/// denoise's second-best noise source (denoise.md §3.3), after a measured
/// table for the body.
pub fn read_noise_profile(decoder: &dyn rawler::decoders::Decoder) -> Option<Vec<(f32, f32)>> {
use rawler::decoders::WellKnownIFD;
use rawler::tags::DngTag;
let ifd = decoder.ifd(WellKnownIFD::Root).ok()??;
let entry = ifd.get_entry_recursive(DngTag::NoiseProfile)?;
let n = entry.count() as usize;
if n < 2 || !n.is_multiple_of(2) {
return None;
}
let pairs: Vec<(f32, f32)> = (0..n / 2)
.map(|i| (entry.force_f32(2 * i), entry.force_f32(2 * i + 1)))
.collect();
pairs
.iter()
.all(|(s, o)| s.is_finite() && o.is_finite() && *s >= 0.0 && *o >= 0.0)
.then_some(pairs)
}
#[cfg(test)]
mod tests {
use super::*;
core/dr-denoise/Cargo.toml
+32
View File
@@ -0,0 +1,32 @@
[package]
name = "dr-denoise"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
[dependencies]
dr-decode.workspace = true
serde = { workspace = true }
serde_norway.workspace = true
thiserror.workspace = true
log.workspace = true
# The network runs under the inference engine like every other model
# (docs/dev/inference.md): `ort` is the API, the engine picks the rung.
# Optional so the noise model and the tiling test without a runtime.
ort = { workspace = true, optional = true }
dr-inference-engine = { workspace = true, optional = true }
ndarray = { workspace = true, optional = true }
[features]
default = ["onnx"]
onnx = ["dep:ort", "dep:dr-inference-engine", "dep:ndarray"]
# A real ONNX Runtime from disk rather than tract alone, as the app links it.
native = ["onnx", "dr-inference-engine/native"]
[dev-dependencies]
# The example repairs hot photosites with the app's own pass, as develop will.
dr-gpu.workspace = true
pollster.workspace = true
env_logger.workspace = true
core/dr-denoise/examples/denoise_raw.rs
+132
View File
@@ -0,0 +1,132 @@
//! Denoise one RAW file end to end, as develop will, and time it.
//!
//! ```sh
//! DARKROOM_ORT_DIR=~/.local/share/darkroom/runtime \
//! cargo run --release -p dr-denoise --features native --example denoise_raw -- IMG.CR2 out
//! ```
//!
//! Decode, the app's hot-pixel pass, the frame's noise from its best source,
//! then the shipped network under the inference engine on whatever rung this
//! machine probes to. Writes `out.npy` — the active area, `h×w×3` f32 linear
//! camera RGB — for comparison with the training repo's own path
//! (`tools/compare_rust.py` in darkroom-denoise). `DARKROOM_ORT_DIR` points
//! at an ONNX Runtime build; the engine's cache goes to `DR_ENGINE_CACHE` or
//! a temporary directory.
use std::path::PathBuf;
use std::time::{Duration, Instant};
use dr_denoise::onnx::OnnxNet;
use dr_inference_engine::{Config, Role};
fn main() {
env_logger::Builder::from_env(env_logger::Env::default().default_filter_or("warn")).init();
let mut args = std::env::args().skip(1);
let (Some(input), Some(out)) = (args.next(), args.next()) else {
eprintln!("usage: denoise_raw RAW OUT_PREFIX");
std::process::exit(2);
};
let model =
PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../models/denoise/mosaic-1408.onnx");
let cache = std::env::var_os("DR_ENGINE_CACHE")
.map(PathBuf::from)
.unwrap_or_else(|| std::env::temp_dir().join("dr-denoise-engines"));
let started = Instant::now();
dr_inference_engine::init(Config {
runtime_dirs: std::env::var_os("DARKROOM_ORT_DIR")
.map(PathBuf::from)
.into_iter()
.collect(),
cache_dir: cache,
models: vec![(Role::Denoiser, model.clone())],
embedded: Vec::new(),
ceiling: None,
threads: 0,
decay: Duration::ZERO,
});
// Wait for the probe and the engine build, so the timing below is the
// rung this machine settles on, not the fallback used while it compiles.
// The probe starts on its own thread; give it a moment to say so.
std::thread::sleep(Duration::from_secs(1));
loop {
let s = dr_inference_engine::status();
if !s.probing && s.engines.0 >= s.engines.1 {
println!(
"engine {} ({:.1} s to settle)",
s.line(),
started.elapsed().as_secs_f64()
);
break;
}
std::thread::sleep(Duration::from_millis(200));
}
let bytes = std::fs::read(&input).expect("read raw");
let t = Instant::now();
let mut raw = dr_decode::decode(&bytes).expect("decode");
let meta = dr_decode::metadata(&bytes).expect("metadata");
let decode = t.elapsed();
let t = Instant::now();
let ctx =
pollster::block_on(dr_gpu::GpuContext::new_headless()).expect("GPU for the hot-pixel pass");
let repaired = dr_gpu::Demosaicer::new(&ctx)
.expect("demosaicer")
.repair_hot_pixels(&mut raw)
.expect("repair");
let repair = t.elapsed();
let noise = dr_denoise::noise::for_frame(&raw, &bytes, meta.iso)
.expect("no noise source for this frame");
println!(
"frame {} {} ISO {:?}, {}×{}, {:?}, {repaired} hot photosites repaired",
raw.make, raw.model, meta.iso, raw.crop.width, raw.crop.height, raw.cfa_pattern
);
println!(
"noise {} — σ at 10 % grey (G) {:.5}, read {:.5}, row {:.5}, col {:.5}",
noise.source.label(),
noise.sigma(1, 0.1),
noise.o[1].sqrt(),
noise.row,
noise.col
);
let mut net = OnnxNet::from_path(&model).expect("model");
println!(
"rung {}",
net.rung().map(|r| r.label()).unwrap_or("?")
);
let t = Instant::now();
let rgb = dr_denoise::denoise(&raw, &noise, &mut net, &mut |done, total| {
eprint!("\rtile {done}/{total}");
true
})
.expect("denoise")
.expect("not cancelled");
let run = t.elapsed();
eprintln!();
println!(
"time decode {:.2} s · hot pixels {:.2} s · network {:.2} s ({:.1} MP)",
decode.as_secs_f64(),
repair.as_secs_f64(),
run.as_secs_f64(),
(raw.crop.width * raw.crop.height) as f64 / 1e6
);
let (h, w) = (raw.crop.height as usize, raw.crop.width as usize);
let mut npy = Vec::with_capacity(rgb.len() * 4 + 128);
let mut header =
format!("{{'descr': '<f4', 'fortran_order': False, 'shape': ({h}, {w}, 3), }}");
while (10 + header.len() + 1) % 64 != 0 {
header.push(' ');
}
header.push('\n');
npy.extend_from_slice(b"\x93NUMPY\x01\x00");
npy.extend_from_slice(&(header.len() as u16).to_le_bytes());
npy.extend_from_slice(header.as_bytes());
for v in &rgb {
npy.extend_from_slice(&v.to_le_bytes());
}
std::fs::write(format!("{out}.npy"), npy).expect("write");
println!("wrote {out}.npy");
}
core/dr-denoise/src/lib.rs
+79
View File
@@ -0,0 +1,79 @@
//! TRACES: FR-DEV-3g
//! Learned demosaic and denoise on the raw mosaic (docs/dev/denoise.md).
//!
//! A network trained on the library's own base-ISO raws with the 6D's
//! measured noise added takes the repaired, normalised mosaic and a σ for
//! every photosite, and returns linear camera RGB at full resolution — the
//! texture the classical demosaic would have produced, with the noise gone.
//! It replaces the demosaic box; nothing downstream changes (§2).
//!
//! - [`noise`] says how noisy each photosite is, from the best source the
//! frame has.
//! - [`tile`] runs a fixed-shape network over a whole frame, exactly.
//! - [`onnx`] is that network under the inference engine.
//!
//! The input must already have been through the app's hot-pixel pass
//! (`dr_gpu::Demosaicer::repair_hot_pixels`): the noise model was fitted
//! with what that pass removes left out.
pub mod noise;
#[cfg(feature = "onnx")]
pub mod onnx;
pub mod tile;
use dr_decode::RawImage;
pub use noise::{NoiseModel, Source};
pub use tile::{TileNet, HALO};
#[derive(Debug, thiserror::Error)]
pub enum DenoiseError {
#[error("the network cannot take this photograph: {0}")]
Unsupported(String),
#[error("the denoise model misbehaved: {0}")]
Model(String),
#[error("could not read the denoise model: {0}")]
ModelRead(#[from] std::io::Error),
#[cfg(feature = "onnx")]
#[error(transparent)]
Engine(#[from] dr_inference_engine::Error),
#[cfg(feature = "onnx")]
#[error(transparent)]
Ort(#[from] ort::Error),
}
/// Whether the learned stage can take this frame at all: a Bayer mosaic.
/// X-Trans needs its own model (§9); a linear DNG has no photosites.
pub fn eligible(raw: &RawImage) -> bool {
raw.samples_per_pixel == 1 && tile::rggb_offset(raw.cfa_pattern).is_some()
}
/// The active area of `raw`, denoised and demosaiced: `crop.height ×
/// crop.width` interleaved RGB, linear camera space, normalised black 0 and
/// white 1 per photosite as the classical demosaic normalises.
///
/// `raw` must be hot-pixel repaired. `None` when `progress` stopped it.
pub fn denoise(
raw: &RawImage,
noise: &NoiseModel,
net: &mut dyn TileNet,
progress: &mut dyn FnMut(usize, usize) -> bool,
) -> Result<Option<Vec<f32>>, DenoiseError> {
if !eligible(raw) {
return Err(DenoiseError::Unsupported(format!(
"{:?} with {} samples per photosite",
raw.cfa_pattern, raw.samples_per_pixel
)));
}
let active = noise::active(raw);
let (h, w) = (active.h, active.w);
tile::run_tiled(
net,
h,
w,
raw.cfa_pattern,
&|y, x| active.at(y, x),
&|c, v| noise.sigma(c, v),
progress,
)
}
core/dr-denoise/src/noise.rs
+407
View File
@@ -0,0 +1,407 @@
//! TRACES: FR-DEV-3g
//! How noisy each photosite is: the network is told, not left to guess
//! (denoise.md §3.3).
//!
//! The model is `σ² = S·x + O + row² + col²` per photosite, `x` the signal
//! normalised black-to-white the way the demosaic normalises it. Three
//! sources, best first:
//!
//! 1. **A measured table** for the body ([`Source::Table`]) — the Canon EOS 6D
//! today, from the library's own frames.
//! 2. **The DNG's `NoiseProfile`** ([`Source::DngProfile`]) — what Adobe's
//! converter measured for the body at that ISO.
//! 3. **The frame itself** ([`Source::Measured`]) — read, row and column
//! noise from its masked border, which is a dark frame taken in the same
//! instant, and only the shot gain estimated, from the quietest flat
//! patches. Checked against the 6D's table on 130 frames: within ±10 % at
//! ISO 1000 and above, scattered below; the network loses under 0.3 dB for
//! a σ off by 15–20 %, and over-estimating costs half what
//! under-estimating does, so the estimate leans high.
//!
//! Row and column noise come from the masked border whenever the frame has
//! one, whatever the source of the rest.
use dr_decode::{CfaPattern, RawImage};
use serde::Deserialize;
/// Where a frame's noise figures came from, for develop to say.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Source {
Table,
DngProfile,
Measured,
}
impl Source {
pub fn label(self) -> &'static str {
match self {
Source::Table => "measured for this camera",
Source::DngProfile => "from the DNG's noise profile",
Source::Measured => "estimated from this photograph",
}
}
}
/// Per-photosite noise in the frame's own normalisation (black 0, white 1).
#[derive(Clone, Debug, PartialEq)]
pub struct NoiseModel {
/// Shot gain per colour, R G B.
pub s: [f32; 3],
/// Read variance per colour, R G B.
pub o: [f32; 3],
/// Standard deviation shared by a whole row, and by a whole column.
pub row: f32,
pub col: f32,
pub source: Source,
}
impl NoiseModel {
/// σ for a photosite of colour `c` (0 R, 1 G, 2 B) reading `x`.
#[inline]
pub fn sigma(&self, c: usize, x: f32) -> f32 {
(self.s[c] * x.max(0.0) + self.o[c] + self.row * self.row + self.col * self.col).sqrt()
}
/// The same figures scaled for the Amount the spec describes (§3.3):
/// above 1 tells the network there is more noise than there is.
pub fn scaled(&self, amount: f32) -> NoiseModel {
let a2 = amount * amount;
NoiseModel {
s: self.s.map(|v| v * a2),
o: self.o.map(|v| v * a2),
row: self.row * amount,
col: self.col * amount,
source: self.source,
}
}
}
/// The frame's noise, from the best source it has.
///
/// `bytes` is the file (for a DNG's `NoiseProfile`), `iso` its EXIF ISO.
/// `None` only for a frame with no masked border, no profile and no table
/// that is also too dark or too busy to measure.
pub fn for_frame(raw: &RawImage, bytes: &[u8], iso: Option<u32>) -> Option<NoiseModel> {
for_frame_with(raw, dr_decode::noise_profile(bytes).as_deref(), iso)
}
/// [`for_frame`], given the file's `NoiseProfile` already read
/// ([`dr_decode::noise_profile`]) rather than the file, for a caller that
/// keeps the header's answer and not the bytes.
pub fn for_frame_with(
raw: &RawImage,
profile: Option<&[(f32, f32)]>,
iso: Option<u32>,
) -> Option<NoiseModel> {
let dark = dark_border(raw);
let mut model = iso
.and_then(|iso| from_table(raw, iso))
.or_else(|| profile.and_then(|p| from_dng_profile(raw, p)))
.or_else(|| measured(raw, dark.as_ref()))?;
if let Some(d) = dark {
// The border saw this exposure's row and column noise directly.
if model.source != Source::Table {
model.row = d.row;
model.col = d.col;
}
}
Some(model)
}
#[derive(Deserialize)]
struct Table {
make: String,
model: String,
rows: Vec<TableRow>,
}
#[derive(Deserialize)]
struct TableRow {
iso: u32,
s_dn: [f32; 4],
o_dn: [f32; 4],
row_dn: f32,
col_dn: f32,
}
const TABLES: &[&str] = &[include_str!("../tables/canon-eos-6d.yaml")];
/// The body's measured table at the nearest ISO it holds, converted from DN
/// to this frame's normalisation.
pub fn from_table(raw: &RawImage, iso: u32) -> Option<NoiseModel> {
let table = TABLES.iter().find_map(|t| {
let t: Table = serde_norway::from_str(t).ok()?;
(t.make.eq_ignore_ascii_case(&raw.make) && t.model.eq_ignore_ascii_case(&raw.model))
.then_some(t)
})?;
let row = table.rows.iter().min_by(|a, b| {
let d = |r: &TableRow| ((r.iso as f32).ln() - (iso as f32).ln()).abs();
d(a).total_cmp(&d(b))
})?;
let span = span(raw);
// RGGB positions → colours: the greens share.
let s = [row.s_dn[0], 0.5 * (row.s_dn[1] + row.s_dn[2]), row.s_dn[3]].map(|v| v / span);
let o =
[row.o_dn[0], 0.5 * (row.o_dn[1] + row.o_dn[2]), row.o_dn[3]].map(|v| v / (span * span));
Some(NoiseModel {
s,
o,
row: row.row_dn / span,
col: row.col_dn / span,
source: Source::Table,
})
}
/// A DNG's `NoiseProfile`: one pair for every plane, or one per colour plane
/// (R, G, B for a Bayer DNG), already in the file's black-to-white units —
/// which are the units `dr-decode` normalises by.
pub fn from_dng_profile(raw: &RawImage, pairs: &[(f32, f32)]) -> Option<NoiseModel> {
if raw.cfa_pattern.is_xtrans() || raw.samples_per_pixel != 1 {
return None;
}
let (s, o) = match pairs {
[(s, o)] => ([*s; 3], [*o; 3]),
[r, g, b, ..] => ([r.0, g.0, b.0], [r.1, g.1, b.1]),
_ => return None,
};
Some(NoiseModel {
s,
o,
row: 0.0,
col: 0.0,
source: Source::DngProfile,
})
}
/// Read, row and column noise measured on the masked border, normalised.
#[derive(Clone, Copy, Debug)]
pub struct Dark {
pub read: f32,
pub row: f32,
pub col: f32,
}
/// The optically black photosites beside and above the active area.
///
/// Keeps well clear of the active area: on the 6D the dozen columns nearest
/// it see light. Photosites over 8σ are the strip's own hot photosites — the
/// same ones in every frame — and are left out, as the app's hot-pixel pass
/// removes their kin before the network sees them.
pub fn dark_border(raw: &RawImage) -> Option<Dark> {
let (x0, y0, w, h) = (
raw.crop.x as usize,
raw.crop.y as usize,
raw.crop.width as usize,
raw.crop.height as usize,
);
let stride = raw.width as usize;
let span = span(raw);
if x0 < 40 || raw.samples_per_pixel != 1 {
return None;
}
let cols = 4..x0 - 16;
let nc = cols.len() as f32;
// Residual after removing each row's mean and each column's mean.
let mut row_means = Vec::with_capacity(h);
let mut col_sum = vec![0.0f64; cols.len()];
for y in y0..y0 + h {
let line = &raw.data[y * stride..y * stride + x0];
let m = cols.clone().map(|x| line[x] as f32).sum::<f32>() / nc;
row_means.push(m);
for (k, x) in cols.clone().enumerate() {
col_sum[k] += (line[x] as f32 - m) as f64;
}
}
let col_mean: Vec<f32> = col_sum.iter().map(|s| (*s / h as f64) as f32).collect();
let resid = |y: usize, k: usize, x: usize| {
raw.data[y * stride + x] as f32 - row_means[y - y0] - col_mean[k]
};
let (mut s1, mut n) = (0.0f64, 0usize);
for y in y0..y0 + h {
for (k, x) in cols.clone().enumerate() {
s1 += (resid(y, k, x) as f64).powi(2);
n += 1;
}
}
let rough = (s1 / n as f64).sqrt() as f32;
let (mut s2, mut n2) = (0.0f64, 0usize);
for y in y0..y0 + h {
for (k, x) in cols.clone().enumerate() {
let r = resid(y, k, x);
if r.abs() < 8.0 * rough {
s2 += (r as f64).powi(2);
n2 += 1;
}
}
}
let read = (s2 / n2.max(1) as f64).sqrt() as f32;
let rm = row_means.iter().sum::<f32>() / h as f32;
let row_var = row_means.iter().map(|m| (m - rm).powi(2)).sum::<f32>() / h as f32;
let row = (row_var - read * read / nc).max(0.0).sqrt();
// Columns: the masked rows above the image span every column.
let col = if y0 >= 24 {
let rows = 4..y0 - 12;
let nr = rows.len() as f32;
let means: Vec<f32> = (x0..x0 + w)
.map(|x| {
rows.clone()
.map(|y| raw.data[y * stride + x] as f32)
.sum::<f32>()
/ nr
})
.collect();
let mm = means.iter().sum::<f32>() / means.len() as f32;
let var = means.iter().map(|m| (m - mm).powi(2)).sum::<f32>() / means.len() as f32;
(var - read * read / nr).max(0.0).sqrt()
} else {
0.0
};
Some(Dark {
read: read / span,
row: row / span,
col: col / span,
})
}
/// The quietest-third bias of the patch variance, and the residual bias the
/// estimate showed against the 6D's table (0.91 at the median), in one: the
/// estimate is divided by this.
const QUIET_FACTOR: f32 = 0.85 * 0.91;
/// The frame's own noise: read noise from the border (or, lacking one, the
/// floor of the quietest patches), shot gain from flat patches of one green
/// plane, the same for every colour, as a sensor's gain is.
pub fn measured(raw: &RawImage, dark: Option<&Dark>) -> Option<NoiseModel> {
if raw.cfa_pattern.is_xtrans() || raw.samples_per_pixel != 1 {
return None;
}
let m = active(raw);
let (h, w) = (m.h, m.w);
// One green plane at a two-photosite pitch.
let (gy, gx) = green_offset(raw.cfa_pattern)?;
let ph = (h - gy) / 2;
let pw = (w - gx) / 2;
let g = |y: usize, x: usize| m.at(gy + 2 * y, gx + 2 * x);
const B: usize = 8;
let mut patches: Vec<(f32, f32)> = Vec::new(); // (level, variance)
for by in 0..ph / B {
for bx in 0..(pw - 2) / B {
let (mut s, mut s2, mut lv) = (0.0f32, 0.0f32, 0.0f32);
for y in by * B..by * B + B {
for x in bx * B..bx * B + B {
// Second difference: cancels any gradient; var = 6σ².
let d = g(y, x + 2) - 2.0 * g(y, x + 1) + g(y, x);
s += d;
s2 += d * d;
lv += g(y, x + 1);
}
}
let n = (B * B) as f32;
let var = (s2 / n - (s / n).powi(2)) / 6.0;
patches.push((lv / n, var));
}
}
let floor = dark.map(|d| d.read);
let lo = 4.0 * floor.unwrap_or(0.002);
patches.retain(|(l, _)| *l > lo && *l < 0.7);
if patches.len() < 500 {
return None;
}
patches.sort_by(|a, b| a.0.total_cmp(&b.0));
let bins = 12;
let per = patches.len() / bins;
let mut ests = Vec::new();
let mut floors = Vec::new();
for b in 0..bins {
let mut bin: Vec<(f32, f32)> = patches[b * per..(b + 1) * per].to_vec();
if bin.len() < 60 {
continue;
}
bin.sort_by(|a, b| a.1.total_cmp(&b.1));
let quiet = &bin[..bin.len() / 3];
let read2 = floor.map(|r| r * r);
let mut e: Vec<f32> = quiet
.iter()
.map(|(l, v)| (v / QUIET_FACTOR - read2.unwrap_or(0.0)) / l)
.collect();
e.sort_by(f32::total_cmp);
ests.push(e[e.len() / 2]);
floors.push(quiet[quiet.len() / 2]);
}
ests.sort_by(f32::total_cmp);
let s = *ests.get(ests.len() / 2)?;
if !(s.is_finite() && s > 0.0) {
return None;
}
// No border: the read variance is what the darkest bin leaves unexplained.
let read2 = match floor {
Some(r) => r * r,
None => {
let (l, v) = floors.first().copied()?;
(v / QUIET_FACTOR - s * l).max(1e-9)
}
};
Some(NoiseModel {
s: [s; 3],
o: [read2; 3],
row: dark.map_or(0.0, |d| d.row),
col: dark.map_or(0.0, |d| d.col),
source: Source::Measured,
})
}
/// Black-to-white range of the frame, as the demosaic normalises it.
pub(crate) fn span(raw: &RawImage) -> f32 {
let black = raw.black_level.iter().map(|&b| b as f32).sum::<f32>() / 4.0;
(raw.white_level as f32 - black).max(1.0)
}
/// Where a green photosite sits in the pattern's 2×2 cell, (dy, dx).
fn green_offset(p: CfaPattern) -> Option<(usize, usize)> {
match p {
CfaPattern::Rggb | CfaPattern::Bggr => Some((0, 1)),
CfaPattern::Grbg | CfaPattern::Gbrg => Some((0, 0)),
_ => None,
}
}
/// The active area, normalised, read lazily.
pub(crate) struct Active<'a> {
raw: &'a RawImage,
black: [f32; 4],
inv: [f32; 4],
pub h: usize,
pub w: usize,
}
impl Active<'_> {
/// Photosite (y, x) of the active area, black 0, white 1.
#[inline]
pub fn at(&self, y: usize, x: usize) -> f32 {
let c = (y & 1) * 2 + (x & 1);
let v = self.raw.data[(self.raw.crop.y as usize + y) * self.raw.width as usize
+ self.raw.crop.x as usize
+ x];
(v as f32 - self.black[c]) * self.inv[c]
}
}
/// Black levels per position of the crop's 2×2 cell, as the demosaic reads
/// them: one reported level is broadcast.
pub(crate) fn active(raw: &RawImage) -> Active<'_> {
let b = raw.black_level;
let black = if b[1] == 0 && b[2] == 0 && b[3] == 0 {
[b[0] as f32; 4]
} else {
b.map(|v| v as f32)
};
let inv = black.map(|bl| 1.0 / (raw.white_level as f32 - bl).max(1.0));
Active {
raw,
black,
inv,
h: raw.crop.height as usize,
w: raw.crop.width as usize,
}
}
core/dr-denoise/src/onnx.rs
+66
View File
@@ -0,0 +1,66 @@
//! TRACES: FR-DEV-3g
//! The denoise network under the inference engine.
//!
//! The shipped export takes `mosaic` and `sigma`, `1×1×1408×1408`, and
//! returns `rgb`, `1×3×1408×1408` (darkroom-denoise `denoise/export.py`,
//! fixed shape because every model the engine runs is). The engine picks the
//! rung: fp16 on TensorRT and MIGraphX, which measured 0.00 dB from f32; f32
//! on CUDA and the CPU; never the Hexagon, where int8 lost 6–9 dB.
use crate::tile::TileNet;
use crate::DenoiseError;
use dr_inference_engine::{Model, Role};
/// The edge of the tile the shipped export takes.
pub const TILE: usize = 1408;
pub struct OnnxNet {
model: Model,
tile: usize,
}
impl OnnxNet {
pub fn from_path(path: &std::path::Path) -> Result<Self, DenoiseError> {
let (path, form) = dr_inference_engine::resolve_model(Role::Denoiser, path);
let bytes = std::fs::read(&path)?;
Ok(OnnxNet {
model: dr_inference_engine::open(Role::Denoiser, form, &bytes)?,
tile: TILE,
})
}
/// Where it runs, for a status line.
pub fn rung(&self) -> Result<dr_inference_engine::Rung, DenoiseError> {
Ok(self.model.acquire()?.rung())
}
}
impl TileNet for OnnxNet {
fn tile(&self) -> usize {
self.tile
}
fn run(&mut self, mosaic: &[f32], sigma: &[f32]) -> Result<Vec<f32>, DenoiseError> {
let n = self.tile;
let shape = ndarray::IxDyn(&[1, 1, n, n]);
let m = ort::value::Tensor::from_array(
ndarray::Array::from_shape_vec(shape.clone(), mosaic.to_vec())
.map_err(|e| DenoiseError::Model(e.to_string()))?,
)?;
let s = ort::value::Tensor::from_array(
ndarray::Array::from_shape_vec(shape, sigma.to_vec())
.map_err(|e| DenoiseError::Model(e.to_string()))?,
)?;
let acquired = self.model.acquire()?;
let mut session = acquired.lock();
let outputs = session.run(ort::inputs!["mosaic" => m, "sigma" => s])?;
let (shape, data) = outputs[0].try_extract_tensor::<f32>()?;
let dims: Vec<i64> = shape.iter().copied().collect();
if dims != [1, 3, n as i64, n as i64] {
return Err(DenoiseError::Model(format!(
"output is {dims:?}, expected [1, 3, {n}, {n}]"
)));
}
Ok(data.to_vec())
}
}
core/dr-denoise/src/tile.rs
+295
View File
@@ -0,0 +1,295 @@
//! TRACES: FR-DEV-3g
//! A whole frame through a fixed-shape network, exactly (denoise.md §3.4).
//!
//! The network sees `TILE_IN`² photosites and its output is exact in the
//! central `TILE_IN − 2·HALO`: the halo is wider than its receptive field
//! (185 photosites, counted from the layers), so a tile's centre equals the
//! whole frame's at the same place. The frame is extended by reflection
//! about its edge photosites, which keeps every photosite's CFA colour, so
//! edge tiles see real context too.
//!
//! **Phase.** The network was trained on RGGB. A frame whose pattern starts
//! on another colour is read from one photosite up and/or left — the
//! reflection supplies that row or column — so its top-left is red, and the
//! output is read back from the same offset. Nothing is cropped.
use dr_decode::CfaPattern;
/// Photosites of context beyond a tile's kept centre, on every side.
pub const HALO: usize = 192;
/// A fixed-shape network: `mosaic` and `sigma`, `n×n` RGGB, in; `3×n×n`
/// planar linear camera RGB out.
pub trait TileNet {
/// The edge `n` of the square tile the network takes.
fn tile(&self) -> usize;
fn run(&mut self, mosaic: &[f32], sigma: &[f32]) -> Result<Vec<f32>, crate::DenoiseError>;
}
/// Index into `0..n` by reflection about the end photosites, any distance
/// out: …2 1 [0 1 2 … n−1] n−2 n−3…, period `2(n−1)`. Parity is kept, which
/// is what keeps a CFA colour.
#[inline]
pub fn reflect(i: isize, n: usize) -> usize {
if n == 1 {
return 0;
}
let p = 2 * (n as isize - 1);
let m = i.rem_euclid(p);
(if m < n as isize { m } else { p - m }) as usize
}
/// How far up and left to start reading so the first photosite is red.
pub fn rggb_offset(p: CfaPattern) -> Option<(usize, usize)> {
match p {
CfaPattern::Rggb => Some((0, 0)),
CfaPattern::Grbg => Some((0, 1)),
CfaPattern::Gbrg => Some((1, 0)),
CfaPattern::Bggr => Some((1, 1)),
_ => None,
}
}
/// Run `net` over an `h×w` mosaic given by `at(y, x)`, with σ from
/// `sigma(colour, value)`, and return `h×w` interleaved RGB.
///
/// `progress(done, total)` is called after each tile and stops the run by
/// returning `false`, in which case the result is `Ok(None)`.
#[allow(clippy::too_many_arguments)]
pub fn run_tiled(
net: &mut dyn TileNet,
h: usize,
w: usize,
pattern: CfaPattern,
at: &dyn Fn(usize, usize) -> f32,
sigma: &dyn Fn(usize, f32) -> f32,
progress: &mut dyn FnMut(usize, usize) -> bool,
) -> Result<Option<Vec<f32>>, crate::DenoiseError> {
let (dy, dx) = rggb_offset(pattern).ok_or_else(|| {
crate::DenoiseError::Unsupported(format!("{pattern:?} is not a Bayer pattern"))
})?;
let n = net.tile();
if n <= 2 * HALO || !(n - 2 * HALO).is_multiple_of(2) {
return Err(crate::DenoiseError::Model(format!(
"tile {n} leaves no even centre past a {HALO} halo"
)));
}
let core = n - 2 * HALO;
// In unified coordinates the frame spans u ∈ [dy, dy + h), v ∈ [dx, dx + w).
let (uh, uw) = (h + dy, w + dx);
let (ty, tx) = (uh.div_ceil(core), uw.div_ceil(core));
let total = ty * tx;
let mut out = vec![0.0f32; h * w * 3];
let mut mos = vec![0.0f32; n * n];
let mut sig = vec![0.0f32; n * n];
// RGGB colour of unified position (u, v).
let colour = |u: usize, v: usize| [[0, 1], [1, 2]][u & 1][v & 1];
for (k, (i, j)) in (0..ty)
.flat_map(|i| (0..tx).map(move |j| (i, j)))
.enumerate()
{
let (u0, v0) = (i * core, j * core);
for r in 0..n {
// Unified row u = u0 + r − HALO; frame row y = u − dy, reflected.
let u = u0 as isize + r as isize - HALO as isize;
let y = reflect(u - dy as isize, h);
for c in 0..n {
let v = v0 as isize + c as isize - HALO as isize;
let x = reflect(v - dx as isize, w);
let val = at(y, x);
mos[r * n + c] = val;
sig[r * n + c] = sigma(colour(r, c), val);
}
}
let rgb = net.run(&mos, &sig)?;
if rgb.len() != 3 * n * n {
return Err(crate::DenoiseError::Model(format!(
"network returned {} values for a {n}² tile",
rgb.len()
)));
}
for r in HALO..HALO + core {
let u = u0 + r - HALO;
if u < dy || u >= uh {
continue;
}
let y = u - dy;
for c in HALO..HALO + core {
let v = v0 + c - HALO;
if v < dx || v >= uw {
continue;
}
let x = v - dx;
let o = (y * w + x) * 3;
for ch in 0..3 {
out[o + ch] = rgb[ch * n * n + r * n + c];
}
}
}
if !progress(k + 1, total) {
return Ok(None);
}
}
Ok(Some(out))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn reflection_keeps_parity_any_distance_out() {
let n = 7;
for i in -40isize..40 {
let r = reflect(i, n);
assert!(r < n);
assert_eq!(
r % 2,
i.rem_euclid(2) as usize,
"index {i} reflected to {r}"
);
}
assert_eq!(reflect(-1, n), 1);
assert_eq!(reflect(7, n), 5);
}
/// A stand-in network with a known, finite reach: each output photosite
/// is its 2×2 quad's (R, mean G, B), averaged over the quads within
/// `reach` quads. Purely a function of the tile, like the real one.
struct BoxNet {
n: usize,
reach: usize,
}
impl TileNet for BoxNet {
fn tile(&self) -> usize {
self.n
}
fn run(&mut self, m: &[f32], _s: &[f32]) -> Result<Vec<f32>, crate::DenoiseError> {
let n = self.n;
let q = n / 2;
let quad = |qy: usize, qx: usize| {
let (y, x) = (2 * qy, 2 * qx);
[
m[y * n + x],
0.5 * (m[y * n + x + 1] + m[(y + 1) * n + x]),
m[(y + 1) * n + x + 1],
]
};
let mut out = vec![0.0; 3 * n * n];
for qy in 0..q {
for qx in 0..q {
let mut acc = [0.0f32; 3];
let mut cnt = 0.0;
for a in qy.saturating_sub(self.reach)..(qy + self.reach + 1).min(q) {
for b in qx.saturating_sub(self.reach)..(qx + self.reach + 1).min(q) {
let v = quad(a, b);
for c in 0..3 {
acc[c] += v[c];
}
cnt += 1.0;
}
}
for (dy, dx) in [(0, 0), (0, 1), (1, 0), (1, 1)] {
for c in 0..3 {
out[c * n * n + (2 * qy + dy) * n + 2 * qx + dx] = acc[c] / cnt;
}
}
}
}
Ok(out)
}
}
/// The mosaic of a smooth colour field in `pattern`, read at (y, x).
fn field(pattern: CfaPattern) -> impl Fn(usize, usize) -> f32 {
move |y, x| {
let rgb = [0.2 + 0.0004 * x as f32, 0.5, 0.1 + 0.0003 * y as f32];
rgb[pattern.colour_at(x as u32, y as u32) as usize]
}
}
#[test]
fn every_bayer_phase_comes_back_as_its_own_colours() {
// A frame of each pattern, its colours known: the network must see
// red where the frame's red photosites are, whatever the phase.
for p in [
CfaPattern::Rggb,
CfaPattern::Grbg,
CfaPattern::Gbrg,
CfaPattern::Bggr,
] {
let (h, w) = (300, 410);
let at = field(p);
let mut net = BoxNet {
n: 2 * HALO + 64,
reach: 0,
};
let out = run_tiled(&mut net, h, w, p, &at, &|_, _| 0.01, &mut |_, _| true)
.unwrap()
.unwrap();
for (y, x) in [(10, 10), (150, 201), (299, 409), (0, 0), (77, 333)] {
let o = &out[(y * w + x) * 3..(y * w + x) * 3 + 3];
let want = [0.2 + 0.0004 * x as f32, 0.5, 0.1 + 0.0003 * y as f32];
for c in 0..3 {
// Within the quad the binned value is at most a photosite away.
assert!(
(o[c] - want[c]).abs() < 0.0012,
"{p:?} at ({y},{x}) channel {c}: {} vs {}",
o[c],
want[c]
);
}
}
}
}
#[test]
fn tiles_reproduce_one_pass_over_the_reflected_frame() {
// A network whose reach is inside the halo gives the same answer
// tiled small as in one tile covering everything.
let (h, w) = (230, 170);
for p in [CfaPattern::Rggb, CfaPattern::Bggr] {
let at = |y: usize, x: usize| ((y * 7919 + x * 104729) % 1000) as f32 / 1000.0;
let mut small = BoxNet {
n: 2 * HALO + 32,
reach: 20,
};
let mut big = BoxNet {
n: 2 * HALO + 256,
reach: 20,
};
let a = run_tiled(&mut small, h, w, p, &at, &|_, _| 0.0, &mut |_, _| true)
.unwrap()
.unwrap();
let b = run_tiled(&mut big, h, w, p, &at, &|_, _| 0.0, &mut |_, _| true)
.unwrap()
.unwrap();
let worst = a
.iter()
.zip(&b)
.map(|(x, y)| (x - y).abs())
.fold(0.0f32, f32::max);
assert!(worst < 1e-5, "{p:?}: tiled and whole differ by {worst}");
}
}
#[test]
fn a_cancelled_run_returns_nothing() {
let mut net = BoxNet {
n: 2 * HALO + 32,
reach: 0,
};
let r = run_tiled(
&mut net,
100,
100,
CfaPattern::Rggb,
&|_, _| 0.5,
&|_, _| 0.0,
&mut |done, _| done < 2,
)
.unwrap();
assert!(r.is_none());
}
}
core/dr-denoise/tables/canon-eos-6d.yaml
+36
View File
@@ -0,0 +1,36 @@
# Canon EOS 6D noise, measured from the library's own frames (denoise.md §5).
# Shot gain S and read variance O per RGGB position from Adobe's NoiseProfile in
# converted DNGs, in DN at the ISO's own white level; read noise checked against
# the masked border (within 2-3 %); row and column noise from the masked border.
# ISO 50 and 100 are extrapolated (S proportional to ISO). Generated by
# darkroom-denoise tools/profile.py; regenerate there, never edit by hand.
make: Canon
model: EOS 6D
black: 2048
rows:
- {iso: 50, white: 15000, s_dn: [0.0854021, 0.085467, 0.085467, 0.0839724], o_dn: [38.2741, 38.6675, 38.6675, 38.9649], row_dn: 0.3423, col_dn: 0.505}
- {iso: 100, white: 15000, s_dn: [0.170804, 0.170934, 0.170934, 0.167945], o_dn: [38.339, 38.7332, 38.7332, 39.031], row_dn: 0.3423, col_dn: 0.505}
- {iso: 125, white: 15035, s_dn: [0.228108, 0.230361, 0.230361, 0.228345], o_dn: [36.9455, 37.8995, 37.8995, 38.2358], row_dn: 0.3423, col_dn: 0.505}
- {iso: 160, white: 12373, s_dn: [0.289653, 0.294915, 0.294915, 0.286887], o_dn: [15.3717, 16.1346, 16.1346, 16.0413], row_dn: 0.212, col_dn: 0.07151}
- {iso: 200, white: 15035, s_dn: [0.370969, 0.369922, 0.369922, 0.361443], o_dn: [24.2761, 24.0847, 24.0847, 24.2414], row_dn: 0.2692, col_dn: 0}
- {iso: 250, white: 15035, s_dn: [0.461889, 0.457975, 0.457975, 0.449318], o_dn: [38.0975, 37.5041, 37.5041, 37.7424], row_dn: 0.3345, col_dn: 0.4786}
- {iso: 320, white: 12323, s_dn: [0.590765, 0.59755, 0.59755, 0.576843], o_dn: [18.5426, 18.9163, 18.9163, 19.1202], row_dn: 0.3158, col_dn: 0.5174}
- {iso: 400, white: 15035, s_dn: [0.753591, 0.740586, 0.740586, 0.729874], o_dn: [29.3028, 29.8496, 29.8496, 29.7961], row_dn: 0.4378, col_dn: 0.2691}
- {iso: 500, white: 15035, s_dn: [0.937458, 0.920836, 0.920836, 0.899293], o_dn: [45.2196, 46.3954, 46.3954, 46.1012], row_dn: 0.5473, col_dn: 0.4328}
- {iso: 640, white: 12323, s_dn: [1.12726, 1.13527, 1.13527, 1.10159], o_dn: [24.9951, 25.1029, 25.1029, 25.7183], row_dn: 0.316, col_dn: 0.4544}
- {iso: 800, white: 15035, s_dn: [1.44048, 1.42299, 1.42299, 1.40795], o_dn: [38.7891, 39.302, 39.302, 40.079], row_dn: 0.3877, col_dn: 0.2132}
- {iso: 1000, white: 15000, s_dn: [1.77595, 1.75662, 1.75662, 1.74584], o_dn: [63.9499, 64.4203, 64.4203, 65.0739], row_dn: 0.4593, col_dn: 0.3307}
- {iso: 1250, white: 12346, s_dn: [2.18211, 2.18313, 2.18313, 2.11979], o_dn: [41.6124, 42.9483, 42.9483, 43.2075], row_dn: 0.3979, col_dn: 0.4496}
- {iso: 1600, white: 15035, s_dn: [2.75544, 2.74633, 2.74633, 2.69951], o_dn: [66.3905, 66.4104, 66.4104, 67.253], row_dn: 0.4944, col_dn: 0.4593}
- {iso: 2000, white: 15035, s_dn: [3.42754, 3.40445, 3.40445, 3.36808], o_dn: [104.349, 103.648, 103.648, 106.404], row_dn: 0.6094, col_dn: 0.3602}
- {iso: 2500, white: 12330, s_dn: [4.17112, 4.17551, 4.17551, 4.17175], o_dn: [94.4289, 91.8508, 91.8508, 96.3598], row_dn: 0.5671, col_dn: 0}
- {iso: 3200, white: 15035, s_dn: [5.30088, 5.25742, 5.25742, 5.21782], o_dn: [147.421, 147.302, 147.302, 147.01], row_dn: 0.748, col_dn: 0.8611}
- {iso: 4000, white: 15035, s_dn: [6.62037, 6.59922, 6.59922, 6.60871], o_dn: [224.765, 232.408, 232.408, 231.419], row_dn: 0.9335, col_dn: 1.125}
- {iso: 5000, white: 12323, s_dn: [8.49542, 8.48265, 8.48265, 8.41176], o_dn: [232.672, 233.922, 233.922, 256.059], row_dn: 1.085, col_dn: 1.852}
- {iso: 6400, white: 15035, s_dn: [10.6956, 10.7417, 10.7417, 10.6503], o_dn: [360.311, 368.198, 368.198, 362.848], row_dn: 1.326, col_dn: 2.277}
- {iso: 8000, white: 15035, s_dn: [13.1307, 13.3864, 13.3864, 13.147], o_dn: [615.02, 566.666, 566.666, 611.738], row_dn: 1.768, col_dn: 3.141}
- {iso: 10000, white: 12365, s_dn: [16.5338, 16.7603, 16.7603, 16.4739], o_dn: [914.064, 904.583, 904.583, 938.024], row_dn: 2.214, col_dn: 3.605}
- {iso: 12800, white: 15000, s_dn: [18.4717, 20.9315, 20.9315, 19.3821], o_dn: [1431.85, 1432.33, 1432.33, 1477.82], row_dn: 2.568, col_dn: 4.661}
- {iso: 16000, white: 15000, s_dn: [20.527, 26.0841, 26.0841, 21.8866], o_dn: [2203.77, 2357.78, 2357.78, 2193.1], row_dn: 3.521, col_dn: 5.805}
- {iso: 20000, white: 13000, s_dn: [25.1517, 32.5307, 32.5307, 26.2303], o_dn: [3490.34, 3647.93, 3647.93, 3423.33], row_dn: 4.336, col_dn: 7.143}
- {iso: 25600, white: 15000, s_dn: [22.8743, 40.4641, 40.4641, 23.5537], o_dn: [5184.57, 5690.43, 5690.43, 5286.07], row_dn: 5.682, col_dn: 9.193}
core/dr-gpu/examples/mosaic_dump.rs
+115
View File
@@ -0,0 +1,115 @@
//! Dump RAW files' mosaics for training the learned denoise (FR-DEV-3g).
//!
//! The training repo must read photosites the way the app reads them —
//! same black and white levels, same active area, same CFA phase — or a
//! network trained on one phase runs on another and paints moiré everywhere
//! (denoise.md §4.4). So it reads this, not LibRaw.
//!
//! The photosites are those the demosaic reads: hot and dead ones repaired by
//! the app's own pass ([`Demosaicer::repair_hot_pixels`], the same shader
//! `run` dispatches), because the learned stage replaces the demosaic and
//! takes its input (denoise.md §2). `--unrepaired` skips it.
//!
//! Reads `input<TAB>output-prefix` lines on stdin and writes, per line,
//! `prefix.npy` (the whole readout, masked border included, `u16`, row-major)
//! and `prefix.json` (what `decode` and `metadata` say about it). The border
//! is kept, and the repair never touches it, because its optically black
//! photosites are a dark frame for free: read noise and row noise at that ISO.
//!
//! ```sh
//! printf 'IMG_0001.CR2\tout/IMG_0001\n' |
//! cargo run --release -p dr-gpu --example mosaic_dump
//! ```
use std::io::{BufRead, Write};
use dr_gpu::{Demosaicer, GpuContext};
fn main() {
let repair = !std::env::args().any(|a| a == "--unrepaired");
let ctx = pollster::block_on(GpuContext::new_headless()).expect("a GPU for the hot-pixel pass");
let demosaicer = Demosaicer::new(&ctx).expect("demosaicer");
let mut failed = 0;
for line in std::io::stdin().lock().lines() {
let line = line.expect("stdin");
let Some((input, prefix)) = line.split_once('\t') else {
continue;
};
match dump(input, prefix, repair.then_some(&demosaicer)) {
Ok(()) => println!("ok\t{input}"),
Err(e) => {
failed += 1;
println!("fail\t{input}\t{e}");
}
}
std::io::stdout().flush().ok();
}
std::process::exit(if failed > 0 { 1 } else { 0 });
}
fn dump(input: &str, prefix: &str, repair: Option<&Demosaicer>) -> Result<(), String> {
let bytes = std::fs::read(input).map_err(|e| e.to_string())?;
let mut raw = dr_decode::decode(&bytes).map_err(|e| e.to_string())?;
if raw.samples_per_pixel != 1 {
return Err("linear DNG: no photosites".into());
}
let repaired = match repair {
Some(d) => d.repair_hot_pixels(&mut raw).map_err(|e| e.to_string())? as i64,
None => -1,
};
let meta = dr_decode::metadata(&bytes).map_err(|e| e.to_string())?;
let mut npy = Vec::with_capacity(raw.data.len() * 2 + 128);
let mut header = format!(
"{{'descr': '<u2', 'fortran_order': False, 'shape': ({}, {}), }}",
raw.height, raw.width
);
// The header, its magic and length are padded to a multiple of 64.
while (10 + header.len() + 1) % 64 != 0 {
header.push(' ');
}
header.push('\n');
npy.extend_from_slice(b"\x93NUMPY\x01\x00");
npy.extend_from_slice(&(header.len() as u16).to_le_bytes());
npy.extend_from_slice(header.as_bytes());
for v in &raw.data {
npy.extend_from_slice(&v.to_le_bytes());
}
std::fs::write(format!("{prefix}.npy"), npy).map_err(|e| e.to_string())?;
let opt = |v: Option<f32>| v.map_or("null".to_string(), |v| v.to_string());
let matrix = raw
.color_matrix
.map_or("null".to_string(), |m| format!("{m:?}"));
let json = format!(
concat!(
"{{\"source\": {:?}, \"make\": {:?}, \"model\": {:?}, ",
"\"width\": {}, \"height\": {}, ",
"\"crop\": [{}, {}, {}, {}], \"cfa\": {:?}, ",
"\"black\": {:?}, \"white\": {}, \"wb\": {:?}, \"cam_to_srgb\": {}, ",
"\"iso\": {}, \"shutter\": {}, \"aperture\": {}, \"captured_at\": {}, ",
"\"hot_repaired\": {}}}\n"
),
input,
raw.make,
raw.model,
raw.width,
raw.height,
raw.crop.x,
raw.crop.y,
raw.crop.width,
raw.crop.height,
format!("{:?}", raw.cfa_pattern),
raw.black_level,
raw.white_level,
raw.wb_coeffs,
matrix,
meta.iso.map_or("null".to_string(), |v| v.to_string()),
opt(meta.shutter),
opt(meta.aperture),
meta.captured_at
.map_or("null".to_string(), |v| v.to_string()),
repaired,
);
std::fs::write(format!("{prefix}.json"), json).map_err(|e| e.to_string())
}
core/dr-gpu/src/adjust.rs
+12 -1
View File
@@ -1889,6 +1889,7 @@ mod tests {
samples_per_pixel: 1,
profile: None,
profile_tables: None,
baseline_exposure: 0.0,
make: String::new(),
model: String::new(),
crop: CropRect {
@@ -2093,6 +2094,7 @@ mod tests {
samples_per_pixel: 1,
profile: None,
profile_tables: None,
baseline_exposure: 0.0,
make: String::new(),
model: String::new(),
crop: CropRect {
@@ -2702,6 +2704,7 @@ mod tests {
samples_per_pixel: 1,
profile: None,
profile_tables: None,
baseline_exposure: 0.0,
make: String::new(),
model: String::new(),
crop: CropRect {
@@ -2806,6 +2809,7 @@ mod tests {
samples_per_pixel: 1,
profile: None,
profile_tables: None,
baseline_exposure: 0.0,
make: String::new(),
model: String::new(),
crop: CropRect {
@@ -3416,8 +3420,15 @@ mod tests {
read_centre(&ctx, t)
};
// The default rendering is the DNG reference curve (D21); for a grey its
// ProPhoto round trip is the identity, so the reference applies as is.
let scene = 3537.0 / 16383.0;
let viewed = dr_pipeline::view::Sigmoid::default_curve().channel(scene);
let viewed = dr_pipeline::camera_raw::apply_reference(
&dr_types::tone::ACR3_DEFAULT,
[scene; 3],
dr_pipeline::view::DEFAULT_CONTRAST,
dr_pipeline::view::DEFAULT_WHITE,
)[0];
let expected = (dr_types::Transfer::Srgb.encode(viewed) * 255.0).round() as i32;
let delta = (i32::from(from_sensor[0]) - expected).abs();
assert!(
core/dr-gpu/src/demosaic.rs
+281 -64
View File
@@ -346,6 +346,95 @@ impl DemosaicedImage {
}
impl DemosaicedImage {
/// TRACES: FR-DEV-3g
/// The learned demosaic's output for the photograph `like` was
/// demosaiced from: `width × height` interleaved RGB, linear camera
/// space, normalised as the demosaic normalises — the same texture the
/// classical path made, with the noise gone (denoise.md §2).
///
/// Everything that describes the photograph rather than its pixels —
/// matrix, profile tables, as-shot balance — is `like`'s, so nothing
/// downstream can tell which demosaic ran. A new [`Self::id`], so every
/// cache keyed on the source sees a new source.
pub fn from_rgb_f32(
ctx: &GpuContext,
like: &DemosaicedImage,
width: u32,
height: u32,
rgb: &[f32],
) -> Result<Self, GpuError> {
let n = width as usize * height as usize;
if rgb.len() != n * 3 {
return Err(GpuError::TooLarge(format!(
"{} values for a {width}×{height} RGB image",
rgb.len()
)));
}
let limits = ctx.device.limits();
if width > limits.max_texture_dimension_2d || height > limits.max_texture_dimension_2d {
return Err(GpuError::TooLarge(format!(
"{width}×{height} exceeds the device limit of {}",
limits.max_texture_dimension_2d
)));
}
let mut half = vec![0u16; n * 4];
let one = f32_to_f16_bits(1.0);
let threads = std::thread::available_parallelism().map_or(1, |n| n.get());
let per = n.div_ceil(threads).max(1);
std::thread::scope(|scope| {
for (k, out) in half.chunks_mut(per * 4).enumerate() {
scope.spawn(move || {
for (i, texel) in out.chunks_mut(4).enumerate() {
let src = &rgb[(k * per + i) * 3..(k * per + i) * 3 + 3];
for c in 0..3 {
texel[c] = f32_to_f16_bits_unclamped(src[c]);
}
texel[3] = one;
}
});
}
});
let texture = ctx.device.create_texture_with_data(
&ctx.queue,
&wgpu::TextureDescriptor {
label: Some("learned-demosaic-source"),
size: wgpu::Extent3d {
width,
height,
depth_or_array_layers: 1,
},
mip_level_count: 1,
sample_count: 1,
dimension: wgpu::TextureDimension::D2,
format: Self::FORMAT,
usage: wgpu::TextureUsages::TEXTURE_BINDING | wgpu::TextureUsages::COPY_SRC,
view_formats: &[],
},
wgpu::util::TextureDataOrder::LayerMajor,
bytemuck::cast_slice(&half),
);
Ok(like.sibling(texture, width, height))
}
/// A new source standing for the same photograph as `self`: its
/// description kept, its pixels `texture`, a fresh id.
pub(crate) fn sibling(&self, texture: wgpu::Texture, width: u32, height: u32) -> Self {
let view = texture.create_view(&Default::default());
Self {
texture,
view,
width,
height,
color_matrix: self.color_matrix,
profile_tables: self.profile_tables.clone(),
as_shot_wb: self.as_shot_wb,
non_linear: self.non_linear,
id: next_image_id(),
frame: self.frame,
window: self.window,
}
}
/// TRACES: FR-MRG-3
/// A source that is already RGB in camera space: a linear DNG, which is
/// what a merge writes. No demosaic; the samples are normalised by the
@@ -489,7 +578,7 @@ impl DemosaicedImage {
view,
width,
height,
color_matrix: raw.color_matrix.unwrap_or(IDENTITY_3X3),
color_matrix: rendering_matrix(raw),
profile_tables: raw.profile_tables.clone(),
as_shot_wb: [raw.wb_coeffs[0], raw.wb_coeffs[1], raw.wb_coeffs[2]],
non_linear: false,
@@ -500,6 +589,23 @@ impl DemosaicedImage {
}
}
/// TRACES: FR-DEV-3e
/// The camera matrix a raw renders through: the file's — identity where the
/// body is uncalibrated, so the image renders with no colour transform
/// rather than not at all — times the baseline exposure as a gain
/// (camera-profiles.md §11).
///
/// A uniform gain commutes with every scene operation before the view
/// transform, so folding it in here is the same as an exposure step at the
/// head of the chain, at no cost. The camera-space tap and the white-balance
/// probe read camera RGB before this matrix and are unaffected.
/// `RawImage::color_matrix` stays the file's: a merge writes a linear DNG
/// from it and must not bake a gain into its pixels.
fn rendering_matrix(raw: &RawImage) -> [f32; 9] {
let gain = raw.baseline_exposure.exp2();
raw.color_matrix.unwrap_or(IDENTITY_3X3).map(|v| v * gain)
}
/// Convert an f32 to half-precision bits, the general case: sign,
/// subnormals, round-to-nearest-even, saturation at the largest finite.
///
@@ -761,62 +867,14 @@ impl Demosaicer {
// TRACES: FR-RAW-3
// The mosaic the demosaic actually reads: the readout with its hot and
// dead photosites repaired. A second buffer rather than in place,
// because every photosite's verdict reads its neighbours' originals.
let repaired = self.ctx.device.create_buffer(&wgpu::BufferDescriptor {
label: Some("raw-repaired"),
size: raw_buf.size(),
usage: wgpu::BufferUsages::STORAGE,
mapped_at_creation: false,
});
let words = packed.len() as u32;
let groups = words.div_ceil(HOT_PIXEL_GROUP).max(1);
// A 24 MP readout is 190,000 workgroups, past the 65,535 one
// dispatch dimension may hold, so the grid folds into rows.
let groups_x = groups.min(
self.ctx
.device
.limits()
.max_compute_workgroups_per_dimension,
);
let groups_y = groups.div_ceil(groups_x);
let hot_params = hot_pixel_params(
// dead photosites repaired.
let hot = self.hot_pass(
raw,
(width, height),
words,
groups_x * HOT_PIXEL_GROUP,
&raw_buf,
packed.len() as u32,
xtrans_tile,
);
let hot_params_buf =
self.ctx
.device
.create_buffer_init(&wgpu::util::BufferInitDescriptor {
label: Some("hot-pixel-params"),
contents: bytemuck::bytes_of(&hot_params),
usage: wgpu::BufferUsages::UNIFORM,
});
let hot_bind_group = self
.ctx
.device
.create_bind_group(&wgpu::BindGroupDescriptor {
label: Some("hot-pixel-bg"),
layout: &self.hot_pixel_layout,
entries: &[
wgpu::BindGroupEntry {
binding: 0,
resource: raw_buf.as_entire_binding(),
},
wgpu::BindGroupEntry {
binding: 1,
resource: hot_params_buf.as_entire_binding(),
},
wgpu::BindGroupEntry {
binding: 2,
resource: repaired.as_entire_binding(),
},
],
});
let params_buf = self
.ctx
.device
@@ -855,7 +913,7 @@ impl Demosaicer {
entries: &[
wgpu::BindGroupEntry {
binding: 0,
resource: repaired.as_entire_binding(),
resource: hot.repaired.as_entire_binding(),
},
wgpu::BindGroupEntry {
binding: 1,
@@ -877,15 +935,7 @@ impl Demosaicer {
// Two passes in one submission. wgpu orders a storage write in one
// pass before a read of the same buffer in the next, so the demosaic
// sees every repair.
{
let mut pass = enc.begin_compute_pass(&wgpu::ComputePassDescriptor {
label: Some("hot-pixel-pass"),
timestamp_writes: None,
});
pass.set_pipeline(&self.hot_pixel_pipeline);
pass.set_bind_group(0, &hot_bind_group, &[]);
pass.dispatch_workgroups(groups_x, groups_y, 1);
}
self.record_hot_pass(&mut enc, &hot);
{
let mut pass = enc.begin_compute_pass(&wgpu::ComputePassDescriptor {
label: Some("demosaic-pass"),
@@ -904,7 +954,7 @@ impl Demosaicer {
height,
// Identity where the body is uncalibrated: the image renders with
// no colour transform rather than not at all.
color_matrix: raw.color_matrix.unwrap_or(IDENTITY_3X3),
color_matrix: rendering_matrix(raw),
profile_tables: raw.profile_tables.clone(),
as_shot_wb: [raw.wb_coeffs[0], raw.wb_coeffs[1], raw.wb_coeffs[2]],
// Whatever the profile database had for this body (FR-DEV-3e),
@@ -921,6 +971,169 @@ impl Demosaicer {
}
}
/// The hot-pixel pass's resources for one frame, ready to record.
struct HotPass {
repaired: wgpu::Buffer,
bind_group: wgpu::BindGroup,
groups: (u32, u32),
}
impl Demosaicer {
/// Buffers and bindings for the hot and dead photosite repair of `raw`,
/// whose packed samples are in `raw_buf`.
fn hot_pass(
&self,
raw: &RawImage,
(width, height): (u32, u32),
raw_buf: &wgpu::Buffer,
words: u32,
xtrans_tile: Option<[u32; 4]>,
) -> HotPass {
// A second buffer rather than in place, because every photosite's
// verdict reads its neighbours' originals.
let repaired = self.ctx.device.create_buffer(&wgpu::BufferDescriptor {
label: Some("raw-repaired"),
size: raw_buf.size(),
usage: wgpu::BufferUsages::STORAGE | wgpu::BufferUsages::COPY_SRC,
mapped_at_creation: false,
});
let groups = words.div_ceil(HOT_PIXEL_GROUP).max(1);
// A 24 MP readout is 190,000 workgroups, past the 65,535 one
// dispatch dimension may hold, so the grid folds into rows.
let groups_x = groups.min(
self.ctx
.device
.limits()
.max_compute_workgroups_per_dimension,
);
let groups_y = groups.div_ceil(groups_x);
let hot_params = hot_pixel_params(
raw,
(width, height),
words,
groups_x * HOT_PIXEL_GROUP,
xtrans_tile,
);
let hot_params_buf =
self.ctx
.device
.create_buffer_init(&wgpu::util::BufferInitDescriptor {
label: Some("hot-pixel-params"),
contents: bytemuck::bytes_of(&hot_params),
usage: wgpu::BufferUsages::UNIFORM,
});
let bind_group = self
.ctx
.device
.create_bind_group(&wgpu::BindGroupDescriptor {
label: Some("hot-pixel-bg"),
layout: &self.hot_pixel_layout,
entries: &[
wgpu::BindGroupEntry {
binding: 0,
resource: raw_buf.as_entire_binding(),
},
wgpu::BindGroupEntry {
binding: 1,
resource: hot_params_buf.as_entire_binding(),
},
wgpu::BindGroupEntry {
binding: 2,
resource: repaired.as_entire_binding(),
},
],
});
HotPass {
repaired,
bind_group,
groups: (groups_x, groups_y),
}
}
fn record_hot_pass(&self, enc: &mut wgpu::CommandEncoder, hot: &HotPass) {
let mut pass = enc.begin_compute_pass(&wgpu::ComputePassDescriptor {
label: Some("hot-pixel-pass"),
timestamp_writes: None,
});
pass.set_pipeline(&self.hot_pixel_pipeline);
pass.set_bind_group(0, &hot.bind_group, &[]);
pass.dispatch_workgroups(hot.groups.0, hot.groups.1, 1);
}
/// TRACES: FR-RAW-3 | FR-DEV-3g
/// Repair `raw`'s hot and dead photosites in place, exactly as [`Self::run`]
/// does before it demosaics, and return how many changed.
///
/// For the learned demosaic (denoise.md §2), which reads the same repaired
/// mosaic the classical one does: its training data and its input in the
/// app must have been through this one pass, not a lookalike.
pub fn repair_hot_pixels(&self, raw: &mut RawImage) -> Result<usize, GpuError> {
if raw.samples_per_pixel != 1 {
return Ok(0);
}
let (width, height) = (raw.crop.width.max(1), raw.crop.height.max(1));
let xtrans_tile = raw
.cfa_pattern
.is_xtrans()
.then(|| xtrans_params_for(raw, width, height).tile);
let packed = pack_samples(&raw.data);
let raw_buf = self
.ctx
.device
.create_buffer_init(&wgpu::util::BufferInitDescriptor {
label: Some("raw-samples"),
contents: bytemuck::cast_slice(&packed),
usage: wgpu::BufferUsages::STORAGE,
});
let hot = self.hot_pass(
raw,
(width, height),
&raw_buf,
packed.len() as u32,
xtrans_tile,
);
let readback = self.ctx.device.create_buffer(&wgpu::BufferDescriptor {
label: Some("raw-repaired-readback"),
size: hot.repaired.size(),
usage: wgpu::BufferUsages::COPY_DST | wgpu::BufferUsages::MAP_READ,
mapped_at_creation: false,
});
let mut enc = self
.ctx
.device
.create_command_encoder(&wgpu::CommandEncoderDescriptor {
label: Some("hot-pixel-encoder"),
});
self.record_hot_pass(&mut enc, &hot);
enc.copy_buffer_to_buffer(&hot.repaired, 0, &readback, 0, hot.repaired.size());
self.ctx.queue.submit(Some(enc.finish()));
let slice = readback.slice(..);
let (tx, rx) = std::sync::mpsc::channel();
slice.map_async(wgpu::MapMode::Read, move |r| {
let _ = tx.send(r);
});
self.ctx
.device
.poll(wgpu::PollType::wait_indefinitely())
.map_err(|e| GpuError::Readback(e.to_string()))?;
rx.recv()
.map_err(|e| GpuError::Readback(e.to_string()))?
.map_err(|e| GpuError::Readback(e.to_string()))?;
let words: Vec<u32> = bytemuck::cast_slice(&slice.get_mapped_range()).to_vec();
readback.unmap();
let mut changed = 0;
for (i, v) in raw.data.iter_mut().enumerate() {
let w = words[i / 2];
let new = if i % 2 == 0 { w & 0xFFFF } else { w >> 16 } as u16;
changed += usize::from(new != *v);
*v = new;
}
Ok(changed)
}
}
const IDENTITY_3X3: [f32; 9] = [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0];
/// Pack u16 samples two per u32, little-endian within the word.
@@ -1414,6 +1627,7 @@ mod tests {
samples_per_pixel: 1,
profile: None,
profile_tables: None,
baseline_exposure: 0.0,
make: String::new(),
model: String::new(),
crop: CropRect {
@@ -1530,6 +1744,7 @@ mod tests {
samples_per_pixel: 1,
profile: None,
profile_tables: None,
baseline_exposure: 0.0,
make: String::new(),
model: String::new(),
crop: CropRect {
@@ -1827,6 +2042,7 @@ mod tests {
samples_per_pixel: 1,
profile: None,
profile_tables: None,
baseline_exposure: 0.0,
make: String::new(),
model: String::new(),
crop: CropRect {
@@ -1912,6 +2128,7 @@ mod tests {
samples_per_pixel: 1,
profile: None,
profile_tables: None,
baseline_exposure: 0.0,
make: String::new(),
model: String::new(),
crop: CropRect {
core/dr-gpu/src/grain.rs
+219
View File
@@ -0,0 +1,219 @@
//! TRACES: FR-DEV-3g
//! Grain back into a denoised photograph, as brightness only.
//!
//! The learned denoise's one live control. The network's result and the
//! classical demosaic of the same mosaic differ by the noise the network
//! removed — plus the classical path's colour speckle and demosaic false
//! colour, which nobody wants back. So only the brightness of the difference
//! is returned, in proportion to `grain`:
//!
//! `out = denoised + grain · ΔY / wb`, with `ΔY = Y(wb · (classical − denoised))`
//!
//! `Y` is taken after the as-shot balance and handed back divided by it, so
//! the grain is neutral in the finished picture rather than tinted the
//! colour of the sensor's raw response. At 0 the result is the network's
//! exactly; at 1 the brightness noise is all back, the colour noise none.
//!
//! A pass of its own producing a new source rather than a term in the
//! adjust shader: the blend depends only on the two images and one number,
//! a 20 MP pass is a few milliseconds, and a new source id is all the
//! adjust pass's caches need to know it changed.
use std::sync::Arc;
use crate::demosaic::DemosaicedImage;
use crate::{GpuContext, GpuError};
const SHADER: &str = r#"
struct Params {
grain: f32,
_pad0: f32,
_pad1: f32,
_pad2: f32,
wb: vec4<f32>,
}
@group(0) @binding(0) var denoised: texture_2d<f32>;
@group(0) @binding(1) var classical: texture_2d<f32>;
@group(0) @binding(2) var<uniform> p: Params;
@group(0) @binding(3) var out: texture_storage_2d<rgba16float, write>;
@compute @workgroup_size(8, 8)
fn main(@builtin(global_invocation_id) gid: vec3<u32>) {
let dims = textureDimensions(denoised);
if (gid.x >= dims.x || gid.y >= dims.y) {
return;
}
let xy = vec2<i32>(gid.xy);
let d = textureLoad(denoised, xy, 0).rgb;
let c = textureLoad(classical, xy, 0).rgb;
let wb = p.wb.rgb;
let dy = p.grain * dot(vec3<f32>(0.2126, 0.7152, 0.0722), wb * (c - d));
textureStore(out, xy, vec4<f32>(d + dy / wb, 1.0));
}
"#;
#[repr(C)]
#[derive(Copy, Clone, bytemuck::Pod, bytemuck::Zeroable)]
struct Params {
grain: f32,
_pad: [f32; 3],
wb: [f32; 4],
}
pub struct GrainBlend {
ctx: GpuContext,
pipeline: wgpu::ComputePipeline,
layout: wgpu::BindGroupLayout,
}
impl GrainBlend {
pub fn new(ctx: &GpuContext) -> Self {
let device = &ctx.device;
let module = device.create_shader_module(wgpu::ShaderModuleDescriptor {
label: Some("grain-blend"),
source: wgpu::ShaderSource::Wgsl(SHADER.into()),
});
let texture = |binding| wgpu::BindGroupLayoutEntry {
binding,
visibility: wgpu::ShaderStages::COMPUTE,
ty: wgpu::BindingType::Texture {
sample_type: wgpu::TextureSampleType::Float { filterable: false },
view_dimension: wgpu::TextureViewDimension::D2,
multisampled: false,
},
count: None,
};
let layout = device.create_bind_group_layout(&wgpu::BindGroupLayoutDescriptor {
label: Some("grain-blend-layout"),
entries: &[
texture(0),
texture(1),
wgpu::BindGroupLayoutEntry {
binding: 2,
visibility: wgpu::ShaderStages::COMPUTE,
ty: wgpu::BindingType::Buffer {
ty: wgpu::BufferBindingType::Uniform,
has_dynamic_offset: false,
min_binding_size: None,
},
count: None,
},
wgpu::BindGroupLayoutEntry {
binding: 3,
visibility: wgpu::ShaderStages::COMPUTE,
ty: wgpu::BindingType::StorageTexture {
access: wgpu::StorageTextureAccess::WriteOnly,
format: DemosaicedImage::FORMAT,
view_dimension: wgpu::TextureViewDimension::D2,
},
count: None,
},
],
});
let pipeline_layout = device.create_pipeline_layout(&wgpu::PipelineLayoutDescriptor {
label: Some("grain-blend-pipeline-layout"),
bind_group_layouts: &[Some(&layout)],
immediate_size: 0,
});
let pipeline = device.create_compute_pipeline(&wgpu::ComputePipelineDescriptor {
label: Some("grain-blend"),
layout: Some(&pipeline_layout),
module: &module,
entry_point: Some("main"),
compilation_options: Default::default(),
cache: None,
});
Self {
ctx: ctx.clone(),
pipeline,
layout,
}
}
/// `denoised` with `grain` (0–1) of `classical`'s brightness noise back.
/// Both must be the same photograph at the same size.
pub fn blend(
&self,
denoised: &DemosaicedImage,
classical: &DemosaicedImage,
grain: f32,
) -> Result<Arc<DemosaicedImage>, GpuError> {
let (w, h) = (denoised.texture().width(), denoised.texture().height());
if (classical.texture().width(), classical.texture().height()) != (w, h) {
return Err(GpuError::TooLarge(format!(
"grain from a {}×{} source into a {w}×{h} one",
classical.texture().width(),
classical.texture().height()
)));
}
let device = &self.ctx.device;
let texture = device.create_texture(&wgpu::TextureDescriptor {
label: Some("grain-blended-source"),
size: wgpu::Extent3d {
width: w,
height: h,
depth_or_array_layers: 1,
},
mip_level_count: 1,
sample_count: 1,
dimension: wgpu::TextureDimension::D2,
format: DemosaicedImage::FORMAT,
usage: wgpu::TextureUsages::STORAGE_BINDING
| wgpu::TextureUsages::TEXTURE_BINDING
| wgpu::TextureUsages::COPY_SRC,
view_formats: &[],
});
let out_view = texture.create_view(&Default::default());
let wb = denoised.as_shot_wb();
let g = wb[1].max(1e-6);
let params = Params {
grain: grain.clamp(0.0, 1.0),
_pad: [0.0; 3],
// Green-normalised, and never zero: the shader divides by it.
wb: [(wb[0] / g).max(1e-3), 1.0, (wb[2] / g).max(1e-3), 1.0],
};
use wgpu::util::DeviceExt;
let buffer = device.create_buffer_init(&wgpu::util::BufferInitDescriptor {
label: Some("grain-blend-params"),
contents: bytemuck::bytes_of(&params),
usage: wgpu::BufferUsages::UNIFORM,
});
let bind = device.create_bind_group(&wgpu::BindGroupDescriptor {
label: Some("grain-blend-bg"),
layout: &self.layout,
entries: &[
wgpu::BindGroupEntry {
binding: 0,
resource: wgpu::BindingResource::TextureView(denoised.view()),
},
wgpu::BindGroupEntry {
binding: 1,
resource: wgpu::BindingResource::TextureView(classical.view()),
},
wgpu::BindGroupEntry {
binding: 2,
resource: buffer.as_entire_binding(),
},
wgpu::BindGroupEntry {
binding: 3,
resource: wgpu::BindingResource::TextureView(&out_view),
},
],
});
let mut enc = device.create_command_encoder(&wgpu::CommandEncoderDescriptor {
label: Some("grain-blend"),
});
{
let mut pass = enc.begin_compute_pass(&wgpu::ComputePassDescriptor {
label: Some("grain-blend"),
timestamp_writes: None,
});
pass.set_pipeline(&self.pipeline);
pass.set_bind_group(0, &bind, &[]);
pass.dispatch_workgroups(w.div_ceil(8), h.div_ceil(8), 1);
}
self.ctx.queue.submit(Some(enc.finish()));
Ok(Arc::new(denoised.sibling(texture, w, h)))
}
}
core/dr-gpu/src/lib.rs
+2
View File
@@ -26,6 +26,7 @@ mod demosaic;
mod detail;
mod error;
mod focus;
mod grain;
mod histogram;
mod mask;
mod merge;
@@ -41,6 +42,7 @@ pub use demosaic::{DemosaicedImage, Demosaicer};
pub use detail::INTERMEDIATE_FORMAT as DETAIL_INTERMEDIATE_FORMAT;
pub use error::GpuError;
pub use focus::{FocusPeakPass, FocusPeaking, PeakColour, PeakSensitivity};
pub use grain::GrainBlend;
pub use merge::{Band, MergeFrame, MergeOutput, MergePass};
// Renamed on the way out: `BINS` says enough inside `histogram`, and nothing
// at all at a crate root shared with demosaic and segmentation.
core/dr-gpu/tests/camera_profile.rs
+42
View File
@@ -91,6 +91,7 @@ fn frame(tables: Option<ProfileTables>) -> RawImage {
samples_per_pixel: 3,
profile: None,
profile_tables: tables.map(Arc::new),
baseline_exposure: 0.0,
make: String::new(),
model: String::new(),
crop: CropRect {
@@ -133,6 +134,7 @@ fn strong_tables() -> ProfileTables {
origin: ProfileOrigin::Embedded,
hue_sat: Some(HueSatTable::new(hd, sd, 1, false, hue_sat).unwrap()),
look: Some(look),
tone_curve: None,
}
}
@@ -190,6 +192,46 @@ fn the_shader_agrees_with_the_cpu_reference() {
assert_agrees(&got, |c| apply_reference(&tables, c, 2.0), "look at 200%");
}
#[test]
fn camera_raw_tone_agrees_with_its_cpu_reference() {
// TRACES: FR-DEV-3j
// D21's rendering on 256 colours: the ProPhoto round trip, the clip, the
// curve from the profile buffer's placeholder, and RGBTone's placement
// of the middle channel, against `camera_raw::apply_reference`.
let Some(ctx) = ctx() else {
eprintln!("skipping: no GPU adapter");
return;
};
let source = Demosaicer::new(&ctx)
.expect("demosaicer")
.run(&frame(None))
.expect("upload");
let mut view = dr_pipeline::ops::ViewTransform::new();
view.set_param(
dr_pipeline::ops::view_transform::CURVE,
dr_pipeline::ops::view_transform::CAMERA_RAW,
);
let ops: Vec<Box<dyn Operation>> = vec![Box::new(view)];
let shader = dr_pipeline::compose(&ops);
let mut adjust = AdjustPass::new(&ctx);
adjust.render(&source, &shader, SIZE, SIZE).expect("render");
let (pixels, _, _) = adjust.export_pixels().expect("readback");
let got: Vec<[u8; 3]> = pixels.chunks_exact(4).map(|p| [p[0], p[1], p[2]]).collect();
let curve = &dr_types::tone::ACR3_DEFAULT;
assert_agrees(
&got,
|c| {
dr_pipeline::camera_raw::apply_reference(
curve,
c,
dr_pipeline::view::DEFAULT_CONTRAST,
dr_pipeline::view::DEFAULT_WHITE,
)
},
"DNG reference tone",
);
}
#[test]
fn switched_off_or_absent_the_render_is_unchanged() {
let Some(ctx) = ctx() else {
core/dr-gpu/tests/film_sim.rs
+1
View File
@@ -51,6 +51,7 @@ fn flat_raw(level: u16) -> RawImage {
samples_per_pixel: 1,
profile: None,
profile_tables: None,
baseline_exposure: 0.0,
make: String::new(),
model: String::new(),
crop: CropRect {
core/dr-gpu/tests/grain.rs
+146
View File
@@ -0,0 +1,146 @@
//! TRACES: FR-DEV-3g
//! The grain blend, read back off the device.
use dr_decode::{CfaPattern, CropRect, RawImage};
use dr_gpu::{DemosaicedImage, Demosaicer, GpuContext, GrainBlend};
const W: u32 = 16;
const H: u32 = 8;
fn ctx() -> Option<GpuContext> {
pollster::block_on(GpuContext::new_headless()).ok()
}
/// A photograph to stand the uploads beside: its as-shot balance is what
/// the grain is made neutral under.
fn like(ctx: &GpuContext) -> DemosaicedImage {
let raw = RawImage {
width: W,
height: H,
data: vec![400; (W * H) as usize],
cfa_pattern: CfaPattern::Rggb,
black_level: [0; 4],
white_level: 4095,
wb_coeffs: [2.0, 1.0, 1.5, 1.0],
color_matrix: Some([1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]),
samples_per_pixel: 1,
profile: None,
profile_tables: None,
baseline_exposure: 0.0,
make: String::new(),
model: String::new(),
crop: CropRect {
x: 0,
y: 0,
width: W,
height: H,
},
};
Demosaicer::new(ctx).unwrap().run(&raw).unwrap()
}
fn read(ctx: &GpuContext, img: &DemosaicedImage) -> Vec<[f32; 4]> {
let (w, h) = (img.texture().width(), img.texture().height());
let padded =
(w * 8).div_ceil(wgpu::COPY_BYTES_PER_ROW_ALIGNMENT) * wgpu::COPY_BYTES_PER_ROW_ALIGNMENT;
let buf = ctx.device.create_buffer(&wgpu::BufferDescriptor {
label: None,
size: (padded * h) as u64,
usage: wgpu::BufferUsages::COPY_DST | wgpu::BufferUsages::MAP_READ,
mapped_at_creation: false,
});
let mut enc = ctx.device.create_command_encoder(&Default::default());
enc.copy_texture_to_buffer(
img.texture().as_image_copy(),
wgpu::TexelCopyBufferInfo {
buffer: &buf,
layout: wgpu::TexelCopyBufferLayout {
offset: 0,
bytes_per_row: Some(padded),
rows_per_image: Some(h),
},
},
wgpu::Extent3d {
width: w,
height: h,
depth_or_array_layers: 1,
},
);
ctx.queue.submit(Some(enc.finish()));
let slice = buf.slice(..);
slice.map_async(wgpu::MapMode::Read, |_| {});
ctx.device
.poll(wgpu::PollType::wait_indefinitely())
.unwrap();
let bytes = slice.get_mapped_range();
let mut out = Vec::new();
for y in 0..h as usize {
let row: &[u16] =
bytemuck::cast_slice(&bytes[y * padded as usize..y * padded as usize + w as usize * 8]);
for t in row.chunks(4) {
out.push([0, 1, 2, 3].map(|c| half_to_f32(t[c])));
}
}
out
}
fn half_to_f32(h: u16) -> f32 {
let s = if h & 0x8000 != 0 { -1.0 } else { 1.0 };
let e = ((h >> 10) & 0x1f) as i32;
let m = (h & 0x3ff) as f32;
if e == 0 {
s * m * 2f32.powi(-24)
} else {
s * (1.0 + m / 1024.0) * 2f32.powi(e - 15)
}
}
#[test]
fn grain_returns_only_neutral_brightness() {
let Some(ctx) = ctx() else {
eprintln!("no GPU adapter; skipping");
return;
};
let base = like(&ctx);
let n = (W * H) as usize;
let d: Vec<f32> = (0..n).flat_map(|_| [0.20, 0.30, 0.10]).collect();
// The classical result: the same colour plus noise, coloured noise too.
let c: Vec<f32> = (0..n)
.flat_map(|i| {
let a = ((i * 37) % 11) as f32 / 110.0 - 0.05;
let b = ((i * 53) % 7) as f32 / 140.0 - 0.025;
[0.20 + a, 0.30 + b, 0.10 - a]
})
.collect();
let denoised = DemosaicedImage::from_rgb_f32(&ctx, &base, W, H, &d).unwrap();
let classical = DemosaicedImage::from_rgb_f32(&ctx, &base, W, H, &c).unwrap();
let blend = GrainBlend::new(&ctx);
let wb = [2.0f32, 1.0, 1.5];
let none = read(&ctx, &blend.blend(&denoised, &classical, 0.0).unwrap());
for p in &none {
for ch in 0..3 {
assert!(
(p[ch] - d[ch]).abs() < 1e-3,
"grain 0 must be the network's result: {p:?}"
);
}
}
let all = read(&ctx, &blend.blend(&denoised, &classical, 1.0).unwrap());
for (i, p) in all.iter().enumerate() {
let want_dy: f32 = [0.2126f32, 0.7152, 0.0722]
.iter()
.enumerate()
.map(|(ch, k)| k * wb[ch] * (c[i * 3 + ch] - d[ch]))
.sum();
// After white balance every channel moved by the same amount.
for ch in 0..3 {
let moved = wb[ch] * (p[ch] - d[ch]);
assert!(
(moved - want_dy).abs() < 2e-3,
"pixel {i} channel {ch}: moved {moved}, want {want_dy}"
);
}
}
}
core/dr-gpu/tests/hot_pixels.rs
+31
View File
@@ -35,6 +35,7 @@ fn frame(pattern: CfaPattern, level: u16, set: &[(u32, u32, u16)]) -> RawImage {
samples_per_pixel: 1,
profile: None,
profile_tables: None,
baseline_exposure: 0.0,
make: String::new(),
model: String::new(),
crop: CropRect {
@@ -140,3 +141,33 @@ fn a_hot_photosite_on_x_trans_is_invisible() {
let diff = worst(&clean, &hot);
assert!(diff <= 1, "a hot X-Trans photosite still shows, by {diff}");
}
/// The repair alone, read back (FR-DEV-3g): the learned demosaic takes the
/// mosaic this pass leaves, so it must be the same pass and nothing more —
/// the hot photosite replaced, a real highlight and every other photosite
/// untouched.
#[test]
fn the_repaired_mosaic_reads_back_with_only_the_defect_changed() {
let Some(ctx) = ctx() else {
eprintln!("no GPU adapter; skipping");
return;
};
let d = Demosaicer::new(&ctx).expect("demosaicer");
let mut star = vec![(MIDDLE, MIDDLE, WHITE)];
for dy in 0..3 {
for dx in 0..3 {
star.push((4 + dx, 4 + dy, WHITE));
}
}
let before = frame(CfaPattern::Rggb, 40, &star);
let mut raw = before.clone();
let changed = d.repair_hot_pixels(&mut raw).expect("repair");
assert_eq!(changed, 1, "only the lone hot photosite should change");
let at = (MIDDLE * SIZE + MIDDLE) as usize;
assert_eq!(
raw.data[at], 40,
"repaired to its brightest same-colour neighbour"
);
let others = (0..raw.data.len()).filter(|&i| i != at);
assert!(others.into_iter().all(|i| raw.data[i] == before.data[i]));
}
core/dr-gpu/tests/scene_referred.rs
+1
View File
@@ -95,6 +95,7 @@ fn flat(ctx: &GpuContext, level: f32) -> dr_gpu::DemosaicedImage {
samples_per_pixel: 1,
profile: None,
profile_tables: None,
baseline_exposure: 0.0,
make: String::new(),
model: String::new(),
crop: CropRect {
core/dr-gpu/tests/source_window.rs
+1
View File
@@ -59,6 +59,7 @@ fn linear_frame(w: u32, h: u32, noise: bool) -> RawImage {
samples_per_pixel: 3,
profile: None,
profile_tables: None,
baseline_exposure: 0.0,
make: String::new(),
model: String::new(),
crop: CropRect {
core/dr-gpu/tests/view_transform.rs
+50 -2
View File
@@ -34,6 +34,7 @@ fn flat_raw(level: u16) -> RawImage {
samples_per_pixel: 1,
profile: None,
profile_tables: None,
baseline_exposure: 0.0,
make: String::new(),
model: String::new(),
crop: CropRect {
@@ -45,6 +46,53 @@ fn flat_raw(level: u16) -> RawImage {
}
}
/// The default chain with D19's sigmoid chosen explicitly, so these tests
/// stay about the sigmoid whichever curve is the default (D21).
fn sigmoid_chain() -> EditGraph {
let mut g = EditGraph::default_chain();
g.set_param(
dr_pipeline::ops::view_transform::ID,
dr_pipeline::ops::view_transform::CURVE,
dr_pipeline::ops::view_transform::SIGMOID,
);
g
}
#[test]
fn camera_raw_tone_agrees_with_the_acr3_curve() {
// TRACES: FR-DEV-3j
// D21: a raw with no profile, the DNG reference curve chosen, renders a grey
// through the ACR3 default curve, which the profile buffer's placeholder
// carries.
let Some(ctx) = ctx() else {
eprintln!("skipping: no GPU adapter");
return;
};
let mut graph = EditGraph::default_chain();
graph.set_param(
dr_pipeline::ops::view_transform::ID,
dr_pipeline::ops::view_transform::CURVE,
dr_pipeline::ops::view_transform::CAMERA_RAW,
);
// The table itself, so its own contrast: the default bends the input a
// little past it (D21).
graph.set_param(
dr_pipeline::ops::view_transform::ID,
dr_pipeline::ops::view_transform::CONTRAST,
dr_pipeline::view::REFERENCE_CONTRAST,
);
for level in [500u16, 4_000, 8_520, 20_000, 40_000] {
let scene = f32::from(level) / f32::from(u16::MAX);
let display = dr_types::tone::evaluate(&dr_types::tone::ACR3_DEFAULT, scene);
let expected = (dr_types::Transfer::Srgb.encode(display) * 255.0).round() as i32;
let got = i32::from(rendered(&ctx, level, &graph));
assert!(
(got - expected).abs() <= 2,
"raw {level} rendered as {got}, the ACR3 curve says {expected}"
);
}
}
/// Render `graph` over a flat frame and return the centre pixel's red.
///
/// The centre rather than a corner: a demosaic has to invent its edges.
@@ -69,7 +117,7 @@ fn the_shader_agrees_with_the_cpu_reference() {
return;
};
let curve = Sigmoid::default_curve();
let graph = EditGraph::default_chain();
let graph = sigmoid_chain();
for level in [0u16, 500, 4_000, 8_520, 32_768, 60_000, u16::MAX] {
let scene = f32::from(level) / f32::from(u16::MAX);
let display = curve.channel(scene).min(1.0);
@@ -95,7 +143,7 @@ fn highlights_above_one_stay_distinct() {
eprintln!("skipping: no GPU adapter");
return;
};
let mut graph = EditGraph::default_chain();
let mut graph = sigmoid_chain();
graph.set_param(
dr_pipeline::ops::exposure::ID,
dr_pipeline::ops::exposure::EXPOSURE,
core/dr-inference-engine/Cargo.toml
+5
View File
@@ -38,6 +38,11 @@ ort = { workspace = true, features = ["cuda", "tensorrt"] }
[target.'cfg(target_os = "android")'.dependencies]
ort = { workspace = true, features = ["qnn"] }
# The Apple rung: CoreML's option builder, which fills the runtime's generic
# key/value map. `ort-sys`'s `coreml` feature is empty; nothing links.
[target.'cfg(target_os = "macos")'.dependencies]
ort = { workspace = true, features = ["coreml"] }
[features]
# The floor: `tract` supplies the API table when no runtime file is found, or
# always, in a build without `native`. Tests want this and nothing else.
core/dr-inference-engine/src/engines.rs
+32 -3
View File
@@ -44,6 +44,20 @@ pub fn context_path(cfg: &Config, bytes: &[u8]) -> PathBuf {
.join(format!("{:016x}_ctx.onnx", hash(bytes)))
}
/// Where CoreML compiles `bytes` to: one directory per model, because
/// CoreML's own cache key leaves out the weights of a model loaded from
/// memory (`session::coreml`), and one per runtime version, which wrote it.
pub fn coreml_dir(cfg: &Config, bytes: &[u8]) -> PathBuf {
let runtime = match crate::api::runtime() {
crate::Runtime::OnnxRuntime { version, .. } => version,
crate::Runtime::Tract => "tract".into(),
};
cfg.cache_dir
.join("coreml")
.join(runtime)
.join(format!("{:016x}", hash(bytes)))
}
/// After the probe: compile every configured model the selected rung can
/// take, smallest first, recording each as it lands.
pub fn run() {
@@ -90,12 +104,27 @@ pub fn run() {
Source::Bytes(b) => (b.to_vec(), format!("embedded {role:?}")),
};
let key = key(rung, &bytes);
if state().lock().unwrap().cache.compiled.contains(&key) {
continue;
{
let s = state().lock().unwrap();
if s.cache.compiled.contains(&key) || s.cache.refused.contains(&key) {
continue;
}
}
log::info!("inference: compiling {name} for {}", rung.label());
let started = std::time::Instant::now();
match crate::session::build(rung, role, &bytes, &cfg) {
let built = match crate::probe::attempt(&cfg, &key, || {
crate::session::build(rung, role, &bytes, &cfg)
}) {
Ok(built) => built,
Err(_) => {
// Refused: the process died inside this compile before.
let mut s = state().lock().unwrap();
s.cache.refused.insert(key);
crate::probe::write_cache(&s.config, &s.cache);
continue;
}
};
match built {
Ok(session) => {
drop(session);
let mut s = state().lock().unwrap();
core/dr-inference-engine/src/lib.rs
+51 -3
View File
@@ -45,6 +45,11 @@ pub enum Role {
/// convolutions, so any rung serves it; fp16 on TensorRT and int8 on
/// the Hexagon are the point of it.
Inpainter,
/// The learned demosaic and denoise on the raw mosaic (docs/dev/denoise.md).
/// fp16 costs it nothing measurable; int8 costs 6–9 dB, because 256
/// levels cannot hold the shadow steps it exists to recover — so the
/// Hexagon does not take it.
Denoiser,
}
/// Which numeric form of a model a session was built from.
@@ -78,6 +83,12 @@ pub enum Rung {
MiGraphX,
/// Qualcomm's Hexagon NPU through QNN, int8 models only. Android only.
Hexagon,
/// Apple, through CoreML: the Neural Engine, the GPU or the CPU, as
/// CoreML schedules it. macOS only. Compiles an ML Program per model on
/// first use, so it is a compiling rung with the CPU below it. The
/// embedder stays on the CPU, as on the Hexagon: the Neural Engine
/// computes in fp16 (§7).
CoreMl,
}
impl Rung {
@@ -88,6 +99,7 @@ impl Rung {
Rung::TensorRt => "TensorRT",
Rung::MiGraphX => "MIGraphX",
Rung::Hexagon => "Hexagon NPU",
Rung::CoreMl => "CoreML",
}
}
@@ -96,13 +108,16 @@ impl Rung {
fn fallback(self) -> Rung {
match self {
Rung::TensorRt => Rung::Cuda,
Rung::MiGraphX | Rung::Hexagon | Rung::Cuda | Rung::Cpu => Rung::Cpu,
Rung::MiGraphX | Rung::Hexagon | Rung::CoreMl | Rung::Cuda | Rung::Cpu => Rung::Cpu,
}
}
/// Whether a session on this rung needs an engine built first.
fn compiles(self) -> bool {
matches!(self, Rung::TensorRt | Rung::MiGraphX | Rung::Hexagon)
matches!(
self,
Rung::TensorRt | Rung::MiGraphX | Rung::Hexagon | Rung::CoreMl
)
}
/// The model form this rung wants for a role.
@@ -116,9 +131,14 @@ impl Rung {
/// Whether this rung runs `role` at all. The Hexagon takes int8 graphs
/// only, and the embedder is never int8 (§7) — it runs on the CPU
/// beside a detector on the NPU, so its vectors compare across devices.
/// Nor is the denoiser: its int8 form failed the 0.5 dB gate by 6–9 dB
/// (denoise.md §8), so it runs on the CPU there too. CoreML is kept off
/// the embedder for the same reason as the Hexagon: the Neural Engine is
/// fp16, and which unit runs a graph is CoreML's choice.
fn serves(self, role: Role) -> bool {
match self {
Rung::Hexagon => role != Role::Embedder,
Rung::Hexagon => !matches!(role, Role::Embedder | Role::Denoiser),
Rung::CoreMl => role != Role::Embedder,
_ => true,
}
}
@@ -348,6 +368,12 @@ struct Cache {
/// the fingerprint changes: a wedged driver must not cost every launch
/// thirty seconds.
failed: Vec<(Rung, String)>,
/// Engine keys whose compile the process died inside, launch after
/// launch (`probe::attempt`). Left on the fallback until the
/// fingerprint changes. Defaulted, so a cache from before this field
/// still reads.
#[serde(default)]
refused: BTreeSet<String>,
}
struct State {
@@ -586,6 +612,7 @@ mod tests {
#[test]
fn the_hexagon_never_takes_the_embedder() {
assert!(!Rung::Hexagon.serves(Role::Embedder));
assert!(!Rung::Hexagon.serves(Role::Denoiser));
assert!(Rung::Hexagon.serves(Role::Detector));
assert_eq!(Rung::Hexagon.form(Role::Detector), Form::Int8);
// A detector offered in f32 on a Hexagon device lands on the CPU.
@@ -631,6 +658,27 @@ mod tests {
);
}
#[test]
fn coreml_takes_a_compiled_detector_and_never_the_embedder() {
let hash = engines::hash(b"detector");
let mut s = State {
config: Config::default(),
cache: Cache {
rung: Some(Rung::CoreMl),
..Cache::default()
},
probing: false,
wanted: 0,
};
let on = |s: &State, role| effective_rung(s, Rung::CoreMl, role, Form::F32, hash);
// Before its program is compiled the detector waits on the CPU.
assert_eq!(on(&s, Role::Detector), Rung::Cpu);
s.cache.compiled.insert(engines::key_of(Rung::CoreMl, hash));
assert_eq!(on(&s, Role::Detector), Rung::CoreMl);
// The embedder does not move, compiled or not (§7).
assert_eq!(on(&s, Role::Embedder), Rung::Cpu);
}
#[test]
fn the_status_reports_only_the_rungs_above_the_selection() {
let _serial = serial();
core/dr-inference-engine/src/probe.rs
+171 -15
View File
@@ -16,10 +16,15 @@ use crate::{api::Runtime, state, Cache, Config, Form, Role, Rung};
fn ladder(ceiling: Option<Rung>) -> Vec<Rung> {
#[cfg(target_os = "android")]
let all = [Rung::Hexagon];
// Unmeasured (§2 ⁵): it is on the ladder because the probe's clock and
// `attempt` make a wrong guess cost one slow or failed probe, not a
// slow or crashing app.
#[cfg(target_os = "macos")]
let all = [Rung::CoreMl];
// A desktop has one vendor's GPU; the other vendor's providers are
// "not enabled in this build" or a library that fails to load, and
// either answer arrives in milliseconds.
#[cfg(not(target_os = "android"))]
#[cfg(not(any(target_os = "android", target_os = "macos")))]
let all = [Rung::TensorRt, Rung::Cuda, Rung::MiGraphX];
all.into_iter()
.filter(|r| ceiling.is_none_or(|c| *r <= c))
@@ -81,7 +86,11 @@ pub fn run(runtime: Runtime) {
log::info!("inference: floor {floor:.1} ms on the CPU provider");
for rung in ladder(cfg.ceiling) {
match time_rung(rung, role, &canonical, &cfg) {
let timed = attempt(&cfg, &format!("probe {}", rung.label()), || {
time_rung(rung, role, &canonical, &cfg)
})
.and_then(|timed| timed);
match timed {
Ok((ms, key)) if ms < floor => {
cache.rung = Some(rung);
cache.reason = format!("{ms:.1} ms against {floor:.1} ms on the CPU");
@@ -118,6 +127,52 @@ fn finish(cache: Cache) {
s.probing = false;
}
/// How many launches in a row may die inside one attempt before it is
/// refused. Two, not one: quitting the app while TensorRT spends forty
/// seconds on an engine leaves the same trace as a provider that aborted.
const STRIKES: u32 = 2;
/// Run `f` — a session build on a provider — with `what` written down
/// first, so that if the provider takes the process with it the next launch
/// knows what to stop trying.
///
/// A provider can fail by aborting rather than by returning an error:
/// XNNPACK did on SCRFD (§2), and a C++ exception or a panic across the C
/// API is an abort. The probe runs in the app's own process, so a rung that
/// does this once would do it on every launch, before the first photograph
/// is on screen. The file (`attempt` in the cache directory) holds the
/// attempt and how many launches have started it without finishing;
/// finishing, by success or by error, removes it. After [`STRIKES`] the
/// attempt is refused, and the caller records the refusal in the cache,
/// where it lasts until the fingerprint changes like any other failure.
pub fn attempt<T>(cfg: &Config, what: &str, f: impl FnOnce() -> T) -> Result<T, String> {
if cfg.cache_dir.as_os_str().is_empty() {
return Ok(f());
}
let path = cfg.cache_dir.join("attempt");
let died = std::fs::read_to_string(&path)
.ok()
.and_then(|s| {
let (w, n) = s.split_once('\t')?;
(w == what).then(|| n.trim().parse::<u32>().ok())?
})
.unwrap_or(0);
if died >= STRIKES {
log::error!("inference: the app died during `{what}` on the last {died} launches; not trying it again");
return Err(format!(
"the app died while trying this on {died} launches in a row"
));
}
if died > 0 {
log::warn!("inference: the last launch died during `{what}`; trying it once more");
}
let _ = std::fs::create_dir_all(&cfg.cache_dir);
let _ = std::fs::write(&path, format!("{what}\t{}", died + 1));
let out = f();
let _ = std::fs::remove_file(&path);
Ok(out)
}
/// The smallest detector, or the smallest model of any role if there is
/// none. A ~2 MB detector is the cheapest real test of a provider, and the
/// detector is the role the int8 forms exist for — the eye classifiers are
@@ -168,21 +223,33 @@ fn time_rung(
started.elapsed().as_secs_f64()
);
let shape: Vec<usize> = session.inputs()[0]
.dtype()
.tensor_shape()
.ok_or("model input is not a tensor")?
// Zeros for every input the model declares, by name — the denoiser
// takes two (mosaic and σ), and a probe that fed only the first failed
// every rung and left it on the CPU.
let feeds: Vec<(String, Vec<usize>)> = session
.inputs()
.iter()
.map(|&d| if d > 0 { d as usize } else { 1 })
.collect();
let zeros = vec![0f32; shape.iter().product()];
.map(|i| {
let shape = i
.dtype()
.tensor_shape()
.ok_or("model input is not a tensor")?
.iter()
.map(|&d| if d > 0 { d as usize } else { 1 })
.collect();
Ok((i.name().to_string(), shape))
})
.collect::<Result<_, &str>>()?;
let run = |session: &mut ort::session::Session| -> Result<f64, String> {
let input = ort::value::Tensor::from_array((shape.clone(), zeros.clone()))
.map_err(|e| e.to_string())?;
let mut inputs: Vec<(String, ort::session::SessionInputValue)> = Vec::new();
for (name, shape) in &feeds {
let zeros = vec![0f32; shape.iter().product()];
let t = ort::value::Tensor::from_array((shape.clone(), zeros))
.map_err(|e| e.to_string())?;
inputs.push((name.clone(), t.into()));
}
let t = Instant::now();
let out = session
.run(ort::inputs![input])
.map_err(|e| e.to_string())?;
let out = session.run(inputs).map_err(|e| e.to_string())?;
let _ = out[0]
.try_extract_tensor::<f32>()
.map_err(|e| e.to_string())?;
@@ -310,7 +377,50 @@ fn system_property(name: &str) -> String {
String::from_utf8_lossy(&buf[..n.max(0) as usize]).into_owned()
}
#[cfg(not(any(target_os = "linux", target_os = "android")))]
#[cfg(target_os = "macos")]
fn device_identity() -> String {
// The chip, and the OS release: CoreML ships with the OS, so a macOS
// update is a new provider as surely as a new driver is on Linux.
format!(
"{} macOS {}",
sysctl("machdep.cpu.brand_string"),
sysctl("kern.osproductversion")
)
}
#[cfg(target_os = "macos")]
fn sysctl(name: &str) -> String {
extern "C" {
fn sysctlbyname(
name: *const std::ffi::c_char,
oldp: *mut std::ffi::c_void,
oldlenp: *mut usize,
newp: *mut std::ffi::c_void,
newlen: usize,
) -> i32;
}
let name = std::ffi::CString::new(name).unwrap();
let mut buf = [0u8; 256];
let mut len = buf.len();
// SAFETY: libSystem's documented call; `len` is the buffer's size in and
// the string's length, with its terminator, out.
let rc = unsafe {
sysctlbyname(
name.as_ptr(),
buf.as_mut_ptr().cast(),
&mut len,
std::ptr::null_mut(),
0,
)
};
if rc != 0 {
return String::new();
}
let s = &buf[..len.min(buf.len())];
String::from_utf8_lossy(s.strip_suffix(&[0]).unwrap_or(s)).into_owned()
}
#[cfg(not(any(target_os = "linux", target_os = "android", target_os = "macos")))]
fn device_identity() -> String {
String::new()
}
@@ -338,3 +448,49 @@ pub fn write_cache(cfg: &Config, cache: &Cache) {
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn a_cache_dir(name: &str) -> Config {
let dir = std::env::temp_dir().join(format!("dr-attempt-{}-{name}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
Config {
cache_dir: dir,
..Config::default()
}
}
/// What a launch that died inside `what` leaves behind.
fn died_inside(cfg: &Config, what: &str, launches: u32) {
std::fs::create_dir_all(&cfg.cache_dir).unwrap();
std::fs::write(cfg.cache_dir.join("attempt"), format!("{what}\t{launches}")).unwrap();
}
#[test]
fn a_finished_attempt_leaves_no_trace() {
let cfg = a_cache_dir("finished");
assert_eq!(attempt(&cfg, "probe CoreML", || 7), Ok(7));
assert!(!cfg.cache_dir.join("attempt").exists());
}
#[test]
fn one_death_is_forgiven_and_two_are_not() {
let cfg = a_cache_dir("strikes");
died_inside(&cfg, "probe CoreML", 1);
assert_eq!(attempt(&cfg, "probe CoreML", || 7), Ok(7));
died_inside(&cfg, "probe CoreML", 2);
let mut ran = false;
assert!(attempt(&cfg, "probe CoreML", || ran = true).is_err());
assert!(!ran, "a refused attempt must not run");
}
#[test]
fn another_attempts_deaths_do_not_count() {
let cfg = a_cache_dir("other");
died_inside(&cfg, "probe TensorRT", 2);
assert_eq!(attempt(&cfg, "probe CUDA", || 7), Ok(7));
}
}
core/dr-inference-engine/src/session.rs
+85 -10
View File
@@ -19,24 +19,61 @@ pub fn build(rung: Rung, role: Role, bytes: &[u8], cfg: &Config) -> ort::Result<
// `stack_tensors`) — a panic across the C API, which is an abort. The
// app never asked tract for that and does not start now.
let mut b = Session::builder()?.with_intra_threads(threads(cfg))?;
if crate::api::runtime().is_native() {
b = with_runtime_log(b)?;
}
// A Hexagon session loads the compiled context when there is one and
// compiles it from the model when there is not; the engine thread is
// what makes the second case rare (§6).
let context = (rung == Rung::Hexagon).then(|| crate::engines::context_path(cfg, bytes));
let ready = context.as_ref().is_some_and(|p| p.is_file());
b = providers(
b,
rung,
role,
cfg,
if ready { None } else { context.as_deref() },
)?;
// What the rung keeps for this model: the context the Hexagon is to
// write, or the directory CoreML compiles into.
let per_model = match rung {
Rung::CoreMl => Some(crate::engines::coreml_dir(cfg, bytes)),
_ if ready => None,
_ => context.clone(),
};
b = providers(b, rung, role, cfg, per_model.as_deref())?;
match (ready, context) {
(true, Some(path)) => b.commit_from_file(path),
_ => b.commit_from_memory(bytes),
}
}
/// Send the runtime's own messages for this session to `log`, under the
/// target `onnxruntime`, instead of to ONNX Runtime's stdio logger.
///
/// Its stderr is nowhere once the app is launched from a menu, and what a
/// provider says while it partitions a graph — how many nodes it took, which
/// operator it declined, the library it failed to load — is most of what a
/// failed rung tells you (docs/dev/inference.md §4). The level follows the
/// filter: warnings always, `debug` adds the runtime's info lines (the
/// partition counts), `trace` its verbose ones (every node placement).
fn with_runtime_log(
b: ort::session::builder::SessionBuilder,
) -> ort::Result<ort::session::builder::SessionBuilder> {
use ort::logging::LogLevel;
let level = if log::log_enabled!(target: "onnxruntime", log::Level::Trace) {
LogLevel::Verbose
} else if log::log_enabled!(target: "onnxruntime", log::Level::Debug) {
LogLevel::Info
} else {
LogLevel::Warning
};
let forward = |level: LogLevel, _category: &str, _id: &str, location: &str, message: &str| {
let level = match level {
LogLevel::Verbose => log::Level::Trace,
LogLevel::Info => log::Level::Debug,
LogLevel::Warning => log::Level::Warn,
LogLevel::Error | LogLevel::Fatal => log::Level::Error,
};
log::log!(target: "onnxruntime", level, "{message} ({location})");
};
Ok(b.with_logger(std::sync::Arc::new(forward))?
.with_log_level(level)?)
}
/// The intra-op pool: what the config says, else the cores less two for
/// the compositor and the decoder (§9). tract ignores it.
fn threads(cfg: &Config) -> usize {
@@ -54,11 +91,12 @@ fn providers(
rung: Rung,
role: Role,
cfg: &Config,
_generate_context: Option<&std::path::Path>,
per_model: Option<&std::path::Path>,
) -> ort::Result<ort::session::builder::SessionBuilder> {
use ort::ep;
match rung {
Rung::Cpu => Ok(b),
Rung::CoreMl => coreml(b, per_model),
Rung::Cuda => {
Ok(b.with_execution_providers([ep::CUDA::default().build().error_on_failure()])?)
}
@@ -103,6 +141,43 @@ fn providers(
}
}
/// CoreML, compiling an ML Program — the format with the operators these
/// graphs use and the one that reaches the Neural Engine — into `cache`.
///
/// The option names are those ONNX Runtime 1.29 reads from the generic
/// key/value map (`coreml_options.cc`), which is what `ort`'s builder
/// fills. The cache is per model because of how CoreML keys it: a model
/// committed from memory, as every session here is, has no path, and the
/// key falls back to a hash of the graph's input and node names — not its
/// weights. Two exports of one architecture would share a program. The
/// directory `engines::coreml_dir` names is the hash of the bytes.
///
/// Every compute unit is allowed, so CoreML may place a graph on the
/// Neural Engine, the GPU or the CPU; the probe's clock judges the result.
#[cfg(target_os = "macos")]
fn coreml(
b: ort::session::builder::SessionBuilder,
cache: Option<&std::path::Path>,
) -> ort::Result<ort::session::builder::SessionBuilder> {
use ort::ep::{self, coreml};
let mut ep = ep::CoreML::default()
.with_model_format(coreml::ModelFormat::MLProgram)
.with_compute_units(coreml::ComputeUnits::All);
if let Some(dir) = cache {
let _ = std::fs::create_dir_all(dir);
ep = ep.with_model_cache_dir(dir.to_string_lossy());
}
Ok(b.with_execution_providers([ep.build().error_on_failure()])?)
}
#[cfg(not(any(target_os = "android", target_os = "macos")))]
fn coreml(
_b: ort::session::builder::SessionBuilder,
_cache: Option<&std::path::Path>,
) -> ort::Result<ort::session::builder::SessionBuilder> {
unreachable!("the CoreML rung is on the macOS ladder only")
}
/// Register MIGraphX through ONNX Runtime's generic key/value entry point.
///
/// `ort`'s own builder (`ep::MIGraphX`) fills the legacy
@@ -175,8 +250,8 @@ fn providers(
.build()
.error_on_failure()])?)
}
Rung::Cuda | Rung::TensorRt | Rung::MiGraphX => {
unreachable!("no desktop GPU rung on Android")
Rung::Cuda | Rung::TensorRt | Rung::MiGraphX | Rung::CoreMl => {
unreachable!("no desktop rung on Android")
}
}
}
core/dr-pipeline/ops/vibrance.yaml
+36 -10
View File
@@ -21,24 +21,41 @@ params:
kind: amount
uniforms:
amount: vibrance / 100
amount:
value: vibrance / 100 * 1.3
doc: |
Scaled so that a value delivers the strength it names. Measured, not
chosen: fitted on 45 of the photographer's earlier exports whose only
colour setting was a vibrance of about +24, against their raws.
helpers: [luminance, tone_position, colour_saturation]
helpers: [luminance]
wgsl: |
let luma = luminance(c);
let sat = colour_saturation(c);
// How saturated a colour *looks*, so measured on display-encoded values.
// In scene-linear light an ordinary tan reads as 0.78 saturated and the
// falloff below would leave it a twentieth of the effect; encoded, it reads
// as 0.5, which is what the eye sees.
let e = pow(max(c, vec3<f32>(0.0)), vec3<f32>(1.0 / 2.2));
let e_hi = max(e.r, max(e.g, e.b));
let e_lo = min(e.r, min(e.g, e.b));
let sat = select(0.0, (e_hi - e_lo) / e_hi, e_hi > 0.00001);
// The vibrance curve: full effect on grey, tapering to nothing on colours
// that are already saturated. Squaring the falloff keeps the mid-range
// responsive while still protecting the extremes.
let falloff = (1.0 - sat) * (1.0 - sat);
// Skin protection. Skin sits in a narrow band of hue where red leads green
// leads blue; pushing it is what makes vibrance look wrong on portraits.
// Detected by channel ordering rather than a hue angle, which costs a
// conversion and buys nothing here.
let is_skin = f32(c.r > c.g && c.g > c.b);
// Skin protection, for skin: hues between about 10 and 50 degrees (red
// leading, green between red and blue) that are not strongly saturated.
// Red-over-green-over-blue alone is every warm colour in a photograph —
// wood, sand, brick, sunlit grass — and halving all of them is most of why
// vibrance used to do so little.
let span = max(e_hi - e_lo, 0.00001);
let skin_hue = select(0.0, 60.0 * (e.g - e.b) / span, e.r >= e.g && e.g >= e.b);
let in_band = smoothstep(4.0, 12.0, skin_hue) * (1.0 - smoothstep(42.0, 52.0, skin_hue));
let is_skin = in_band * (1.0 - smoothstep(0.45, 0.7, sat)) * f32(e.r >= e.g && e.g >= e.b);
let skin_guard = 1.0 - is_skin * 0.5;
let strength = amount * falloff * skin_guard;
@@ -49,9 +66,18 @@ tests:
- name: it_starts_neutral
expect_active: false
- name: the_amount_is_normalised_to_unit_range
- name: the_amount_is_the_measured_scale
why: |
Fitted against the photographer's earlier exports, so a value delivers
the strength it names.
set: { vibrance: 100 }
expect: { amount: 1.0 }
expect: { amount: 1.3 }
- name: saturation_is_judged_as_displayed
why: |
Judged in scene-linear light, ordinary warm colours read as nearly
saturated and get almost none of the effect.
expect_wgsl: ["let e = pow(max(c, vec3<f32>(0.0)), vec3<f32>(1.0 / 2.2));"]
- name: muted_colours_get_more_than_saturated_ones
why: |
core/dr-pipeline/presets/vivid.drpl
+2 -6
View File
@@ -1,12 +1,8 @@
drpl 1
# Vivid: more colour than the default rendering, for the photographer used
# to Lightroom's richer starting point (camera-profiles.md §9).
# Vivid: more colour than the default rendering (camera-profiles.md §9).
#
# That difference is mostly Camera Raw's tone curve, not the camera
# profile: Adobe Standard's look table, which D20 applies, desaturates dark
# tones (camera-profiles.md §1). So these do the work themselves, and work
# on every photograph — a JPEG, a body with no profile. They lean on
# These do the work themselves, and work on every photograph — a JPEG, a body with no profile. They lean on
# vibrance before saturation: vibrance lifts muted colours most and holds
# skin back, so a frame gets richer before anything in it looks painted.
# Saturation, which moves every colour alike, is used sparingly on top.
core/dr-pipeline/src/camera_raw.rs
+168
View File
@@ -0,0 +1,168 @@
//! TRACES: FR-DEV-3j | FR-DEV-3e
//! The DNG SDK's reference tone, as a rendering the view transform can
//! choose (D21).
//!
//! The DNG specification's reference rendering runs a raw through the
//! profile's `ProfileToneCurve`, or the ACR3 default for a profile with none.
//! Half of what the curve does is *how* it is applied. The SDK's
//! `RefBaselineRGBTone` runs it on the largest and the smallest channel, and
//! places the middle channel at the fraction between them it had before. Hue
//! is kept; saturation rises wherever the curve is steeper than the
//! diagonal, which for the ACR3 curve is the shadows and the midtones.
//!
//! It runs in linear ProPhoto, as the SDK does, on values clipped to
//! `[0, 1]`; its output is linear and goes to the output transform as the
//! sigmoid's does. The curve is read from the profile buffer
//! (`ops::camera_profile::profile_buffer`), which always carries one.
//!
//! [`apply_reference`] is the arithmetic on the CPU; the GPU test holds the
//! shader to it.
use crate::ops::camera_profile::{mul, working_prophoto};
use crate::view::{DEFAULT_WHITE, REFERENCE_CONTRAST, SCENE_GREY};
/// The input scale for a white point: 1 at the default, so sensor white is
/// display white as in the SDK's reference; each stop of `white` above it halves the
/// input.
pub fn input_scale(white: f32) -> f32 {
(DEFAULT_WHITE - white).exp2()
}
/// The power the input is bent by about middle grey: 1 at
/// [`REFERENCE_CONTRAST`], where the curve is the reference's untouched.
///
/// The default contrast sits above it, so a photograph out of the camera is
/// bent by `DEFAULT_CONTRAST / REFERENCE_CONTRAST` — the extra contrast
/// Lightroom's exports showed over the bare reference curve (D21 addendum).
pub fn contrast_power(contrast: f32) -> f32 {
contrast / REFERENCE_CONTRAST
}
/// The curve, its scale and its contrast applied to one ProPhoto colour.
fn rgb_tone(curve: &[f32], p: [f32; 3]) -> [f32; 3] {
let p = p.map(|v| v.clamp(0.0, 1.0));
let hi = p[0].max(p[1]).max(p[2]);
let lo = p[0].min(p[1]).min(p[2]);
let (c_hi, c_lo) = (
dr_types::tone::evaluate(curve, hi),
dr_types::tone::evaluate(curve, lo),
);
if hi - lo <= 1e-7 {
return [c_hi; 3];
}
p.map(|v| c_lo + (c_hi - c_lo) * (v - lo) / (hi - lo))
}
/// TRACES: FR-DEV-3j
/// The view transform's DNG reference rendering of one working-space colour.
pub fn apply_reference(curve: &[f32], c: [f32; 3], contrast: f32, white: f32) -> [f32; 3] {
let (to, back) = working_prophoto();
let scale = input_scale(white);
let power = contrast_power(contrast);
let mut p = mul(to, c).map(|v| v * scale);
if power != 1.0 {
p = p.map(|v| SCENE_GREY * (v.max(0.0) / SCENE_GREY).powf(power));
}
mul(back, rgb_tone(curve, p))
}
/// The WGSL, a helper the view transform asks for after
/// `ops::camera_profile`'s ProPhoto constants. Mirrors [`apply_reference`].
pub const CAMERA_RAW_WGSL: &str = "
fn camera_raw_curve(x: f32) -> f32 {
let base = profile_curve_base();
let n = u32(profile_table[2].x);
let s = clamp(x, 0.0, 1.0) * f32(n - 1u);
let i = min(u32(s), n - 2u);
return mix(profile_table[base + i].x, profile_table[base + i + 1u].x, s - f32(i));
}
// The SDK's RGBTone: the curve on the largest and smallest channel, the
// middle one kept at its fraction between them, so hue survives.
fn camera_raw_tone(c: vec3<f32>, scale: f32, power: f32, grey: f32) -> vec3<f32> {
var p = PROFILE_FROM_WORKING * c * scale;
if (power != 1.0) {
p = grey * pow(max(p, vec3<f32>(0.0)) / grey, vec3<f32>(power));
}
p = clamp(p, vec3<f32>(0.0), vec3<f32>(1.0));
let hi = max(p.r, max(p.g, p.b));
let lo = min(p.r, min(p.g, p.b));
let c_hi = camera_raw_curve(hi);
let c_lo = camera_raw_curve(lo);
var out = vec3<f32>(c_hi);
if (hi - lo > 1e-7) {
out = vec3<f32>(c_lo) + (c_hi - c_lo) * (p - vec3<f32>(lo)) / (hi - lo);
}
return PROFILE_TO_WORKING * out;
}
";
#[cfg(test)]
mod tests {
use super::*;
use crate::view::DEFAULT_CONTRAST;
use dr_types::tone::{evaluate, ACR3_DEFAULT};
fn identity() -> Vec<f32> {
(0..1025).map(|i| i as f32 / 1024.0).collect()
}
#[test]
fn at_the_reference_the_input_is_untouched() {
assert_eq!(input_scale(DEFAULT_WHITE), 1.0);
assert_eq!(contrast_power(REFERENCE_CONTRAST), 1.0);
}
#[test]
fn the_default_adds_the_measured_contrast() {
// Fitted on Lightroom exports with neutral settings (D21 addendum):
// the bare reference curve is a little flat against them.
let p = contrast_power(DEFAULT_CONTRAST);
assert!((1.05..1.12).contains(&p), "{p}");
}
#[test]
fn grey_goes_through_the_curve_and_stays_grey() {
for v in [0.02, 0.13, 0.5] {
let out = apply_reference(&ACR3_DEFAULT, [v; 3], REFERENCE_CONTRAST, DEFAULT_WHITE);
let want = evaluate(&ACR3_DEFAULT, v);
assert!(
out.iter().all(|o| (o - want).abs() < 1e-4),
"{v}: {out:?} vs {want}"
);
}
}
#[test]
fn an_identity_curve_changes_nothing_inside_the_range() {
let c = [0.4, 0.2, 0.1];
let out = apply_reference(&identity(), c, REFERENCE_CONTRAST, DEFAULT_WHITE);
assert!(
out.iter().zip(c).all(|(o, c)| (o - c).abs() < 1e-4),
"{out:?}"
);
}
#[test]
fn the_middle_channel_keeps_its_place_between_the_other_two() {
let p = [0.3, 0.12, 0.05];
let out = rgb_tone(&ACR3_DEFAULT, p);
let before = (p[1] - p[2]) / (p[0] - p[2]);
let after = (out[1] - out[2]) / (out[0] - out[2]);
assert!((before - after).abs() < 1e-5, "{before} {after}");
}
#[test]
fn the_acr_curve_raises_saturation_in_the_midtones() {
let p = [0.15, 0.08, 0.05];
let out = rgb_tone(&ACR3_DEFAULT, p);
let sat = |c: [f32; 3]| (c[0] - c[2]) / c[0];
assert!(sat(out) > sat(p), "{p:?} -> {out:?}");
}
#[test]
fn white_halves_the_input_per_stop() {
assert_eq!(input_scale(DEFAULT_WHITE + 1.0), 0.5);
assert!(contrast_power(2.8) > 1.0);
}
}
core/dr-pipeline/src/descriptor.rs
+9
View File
@@ -464,6 +464,15 @@ impl ParamDescriptor {
}
}
/// The same choice with another variant as its default.
///
/// For a choice whose variants were numbered before its default was
/// settled: a sidecar records the index, so reordering the variants to
/// put the default first would change what saved edits mean.
pub fn with_default(self, default: f32) -> Self {
Self { default, ..self }
}
/// A 0…1 fraction — a proportion of something, rather than an amount.
///
/// Its own constructor because the crop rect needs four of them and the
core/dr-pipeline/src/graph.rs
+127
View File
@@ -170,6 +170,18 @@ pub struct EditGraph {
/// correction the photograph asked for — see
/// [`crate::descriptor::ParamDescriptor::switch_on`].
lens_profile_applied: bool,
/// TRACES: FR-DEV-3g
/// Whether this photograph can take the learned denoise — a Bayer
/// mosaic — set by whoever opened it. Derived from the file like the
/// lens profile, so not in the state; it only decides whether the
/// switch below is offered.
denoise_available: bool,
/// Whether the learned denoise replaces the demosaic. An edit: published
/// as [`crate::learned_denoise`], captured, stored and undone with the
/// rest (FR-DEV-3c).
denoise_applied: bool,
/// How much of the removed noise's brightness to put back, 0–100.
denoise_grain: f32,
}
/// TRACES: FR-DEV-3f
@@ -226,6 +238,9 @@ impl EditGraph {
],
lens_profile: None,
lens_profile_applied: true,
denoise_available: false,
denoise_applied: false,
denoise_grain: 0.0,
}
}
@@ -399,6 +414,26 @@ impl EditGraph {
self.lens_profile.as_ref()
}
/// TRACES: FR-DEV-3g
/// Offer the learned denoise, or not: true for a Bayer mosaic.
pub fn set_denoise_available(&mut self, available: bool) {
self.denoise_available = available;
}
/// TRACES: FR-DEV-3g
/// Whether the learned denoise is asked for. A setting kept on a
/// photograph that cannot take it is harmless and does nothing, as a
/// lens switch with no profile does.
pub fn denoise_applied(&self) -> bool {
self.denoise_applied
}
/// TRACES: FR-DEV-3g
/// The grain to keep, 0–1.
pub fn denoise_grain(&self) -> f32 {
self.denoise_grain / 100.0
}
/// TRACES: FR-DEV-3
/// Whether the matched profile is being applied.
pub fn lens_profile_applied(&self) -> bool {
@@ -581,9 +616,37 @@ impl EditGraph {
}
});
// TRACES: FR-DEV-3g
// Offered only where the photograph can take it, for the lens
// switch's reason: a control that can do nothing must not look as if
// it could.
let denoise = self.denoise_available.then(|| {
let desc = crate::learned_denoise::descriptor();
OpCapability {
id: desc.id,
label: desc.label,
active: self.denoise_applied,
params: desc
.params
.iter()
.map(|p| ParamCapability {
id: p.id,
label: p.label,
kind: p.kind.clone(),
default: p.default,
value: self.param(desc.id, p.id).unwrap_or(p.default),
facet: p.facet,
})
.collect(),
presentation: None,
attributes: desc.attributes.clone(),
}
});
switch
.into_iter()
.chain(warps)
.chain(denoise)
.chain(ops)
.chain(std::iter::once(framing))
.collect()
@@ -675,6 +738,11 @@ impl EditGraph {
// `capabilities`, with the operations and the warps and for the
// same reason (FR-DEV-3c).
lens_profile_applied: _,
// Derived from the file, like the profile above.
denoise_available: _,
// Edits, in the state through `capabilities` like the lens switch.
denoise_applied: _,
denoise_grain: _,
masks,
film,
spots,
@@ -746,6 +814,16 @@ impl EditGraph {
}
pub fn set_param(&mut self, op: OpId, param: ParamId, value: f32) {
if op == crate::learned_denoise::ID {
match param {
p if p == crate::learned_denoise::APPLY => self.denoise_applied = value != 0.0,
p if p == crate::learned_denoise::GRAIN => {
self.denoise_grain = value.clamp(0.0, 100.0)
}
_ => log::warn!("unknown parameter {param} on {op}; ignoring"),
}
return;
}
if op == crate::lens::profile_switch::ID {
if param != crate::lens::profile_switch::APPLY {
log::warn!("unknown parameter {param} on {op}; ignoring");
@@ -803,6 +881,15 @@ impl EditGraph {
/// Read a parameter back.
pub fn param(&self, op: OpId, param: ParamId) -> Option<f32> {
if op == crate::learned_denoise::ID {
return match param {
p if p == crate::learned_denoise::APPLY => {
Some(if self.denoise_applied { 1.0 } else { 0.0 })
}
p if p == crate::learned_denoise::GRAIN => Some(self.denoise_grain),
_ => None,
};
}
if op == crate::lens::profile_switch::ID {
return (param == crate::lens::profile_switch::APPLY)
.then_some(if self.lens_profile_applied { 1.0 } else { 0.0 });
@@ -849,6 +936,10 @@ impl EditGraph {
// a reset does not change which lens took the photograph. What returns
// to default is the answer to whether to use it, which is on.
self.set_lens_profile_applied(true);
// The learned denoise returns to off; whether it is available is the
// file's and stays.
self.denoise_applied = false;
self.denoise_grain = 0.0;
}
/// Set the crop rectangle. Clamped to keep it inside the frame.
@@ -1995,4 +2086,40 @@ mod tests {
let after = cropped.render_scale(source, (1500, 1000));
assert!(after.ratio() > fit.ratio());
}
#[test]
fn the_learned_denoise_is_offered_only_where_it_can_run() {
use crate::learned_denoise;
let mut g = EditGraph::default_chain();
assert!(!g.capabilities().iter().any(|c| c.id == learned_denoise::ID));
g.set_denoise_available(true);
let cap = g
.capabilities()
.into_iter()
.find(|c| c.id == learned_denoise::ID)
.expect("offered");
assert!(!cap.active, "off until asked for");
g.set_param(learned_denoise::ID, learned_denoise::APPLY, 1.0);
g.set_param(learned_denoise::ID, learned_denoise::GRAIN, 30.0);
assert!(g.denoise_applied());
assert!((g.denoise_grain() - 0.3).abs() < 1e-6);
g.reset();
assert!(!g.denoise_applied());
assert_eq!(g.denoise_grain(), 0.0);
}
#[test]
fn the_learned_denoise_travels_in_the_state() {
use crate::learned_denoise;
let mut g = EditGraph::default_chain();
g.set_denoise_available(true);
g.set_param(learned_denoise::ID, learned_denoise::APPLY, 1.0);
g.set_param(learned_denoise::ID, learned_denoise::GRAIN, 40.0);
let state = g.state();
let mut h = EditGraph::default_chain();
h.set_denoise_available(true);
let _ = h.set_state(&state);
assert!(h.denoise_applied());
assert!((h.denoise_grain() - 0.4).abs() < 1e-6);
}
}
core/dr-pipeline/src/learned_denoise.rs
+51
View File
@@ -0,0 +1,51 @@
//! TRACES: FR-DEV-3g
//! The learned denoise's settings: whether to use it, and how much grain to
//! keep.
//!
//! Not an [`crate::operation::Operation`]: the learned stage replaces the
//! demosaic and runs once per photograph, off the render path
//! (docs/dev/denoise.md §2, §7), and the grain is a blend of its result with
//! the classical one, done where the source is chosen. But what a
//! photographer sets travels the one road every setting travels — the
//! capability list feeds the panel, [`crate::Preset`] captures it, the
//! sidecar stores it, the undo stack replays it (FR-DEV-3c) — so it is
//! published as a capability, like the lens profile switch.
use std::sync::{Arc, LazyLock};
use crate::descriptor::{Attribute, LocalizedKey, OpDescriptor, ParamDescriptor, Scale, Unit};
use crate::{OpId, ParamId};
pub const ID: OpId = OpId("learned_denoise");
pub const APPLY: ParamId = ParamId("apply");
pub const GRAIN: ParamId = ParamId("grain");
/// Off by default: it costs seconds per photograph and replaces the
/// demosaic, which is the photographer's call. Grain 0 is the network's
/// result as it is.
pub(crate) static DESCRIPTOR: LazyLock<Arc<OpDescriptor>> = LazyLock::new(|| {
Arc::new(OpDescriptor {
id: ID,
label: LocalizedKey("op.learned_denoise"),
params: vec![
ParamDescriptor::switch("apply", "param.learned_denoise.apply"),
ParamDescriptor::scalar(
"grain",
"param.learned_denoise.grain",
0.0,
100.0,
0.0,
Unit::Percent,
Scale::Linear,
0,
),
],
// With the classical noise reduction, which is what a photographer
// looks for it beside.
attributes: vec![Attribute::Detail],
})
});
pub fn descriptor() -> Arc<OpDescriptor> {
DESCRIPTOR.clone()
}
core/dr-pipeline/src/lib.rs
+2
View File
@@ -33,6 +33,7 @@
//! data neither would be physically meaningful (ARCH §5.2).
pub mod bundled;
pub mod camera_raw;
pub mod coverage;
pub mod declared;
pub mod descriptor;
@@ -40,6 +41,7 @@ pub mod detail;
pub mod framing;
pub mod graph;
pub mod history;
pub mod learned_denoise;
pub mod lens;
pub mod mask;
pub mod neutral;
core/dr-pipeline/src/ops/camera_profile.rs
+70 -25
View File
@@ -48,13 +48,14 @@ pub const LOOK: ParamId = ParamId("look");
/// The look's strength at which the LookTable is applied as the profile
/// states it, in percent.
pub const DEFAULT_LOOK: f32 = 100.0;
/// Twice the profile's look: Lightroom's *Amount* reaches the same.
/// Twice the profile's look.
pub const MAX_LOOK: f32 = 200.0;
/// Entries of the buffer's header: one `vec4` describing each table —
/// `(hue divisions, saturation divisions, value divisions, sRGB-encoded)`,
/// zero hue divisions meaning absent — before the entries themselves.
pub const HEADER_ENTRIES: usize = 2;
/// Entries of the buffer's header, before the entries themselves: one
/// `vec4` describing each table — `(hue divisions, saturation divisions,
/// value divisions, sRGB-encoded)`, zero hue divisions meaning absent — and
/// a third whose `.x` is the tone curve's length (camera-profiles.md §12).
pub const HEADER_ENTRIES: usize = 3;
static DESCRIPTOR: LazyLock<Arc<OpDescriptor>> = LazyLock::new(|| {
Arc::new(OpDescriptor {
@@ -80,7 +81,7 @@ static DESCRIPTOR: LazyLock<Arc<OpDescriptor>> = LazyLock::new(|| {
/// Linear sRGB (the working space) to linear ProPhoto, and back, row-major,
/// each row scaled to sum to one so that working white is ProPhoto white
/// exactly and a neutral reaches the tables with zero saturation.
fn working_prophoto() -> &'static ([f32; 9], [f32; 9]) {
pub(crate) fn working_prophoto() -> &'static ([f32; 9], [f32; 9]) {
static M: LazyLock<([f32; 9], [f32; 9])> = LazyLock::new(|| {
let to = normalise_rows(dr_types::ColourSpace::ProPhoto.from_linear_srgb());
let back = normalise_rows(invert(&to).expect("ProPhoto's matrix is invertible"));
@@ -117,7 +118,7 @@ fn invert(m: &[f32; 9]) -> Option<[f32; 9]> {
Some(inv.map(|v| v as f32))
}
fn mul(m: &[f32; 9], c: [f32; 3]) -> [f32; 3] {
pub(crate) fn mul(m: &[f32; 9], c: [f32; 3]) -> [f32; 3] {
std::array::from_fn(|r| m[r * 3] * c[0] + m[r * 3 + 1] * c[1] + m[r * 3 + 2] * c[2])
}
@@ -127,19 +128,48 @@ fn wgsl_mat(m: &[f32; 9]) -> String {
format!("mat3x3<f32>({}, {}, {})", col(0), col(1), col(2))
}
static HELPERS: LazyLock<[Helper; 1]> = LazyLock::new(|| {
/// The working space to ProPhoto and back, as WGSL constants, and where
/// the profile buffer's sections begin. A helper of its own because the
/// view transform's DNG reference curve needs it too, and helpers are emitted
/// once each, in the order first asked for.
pub(crate) static PROPHOTO_HELPER: LazyLock<Helper> = LazyLock::new(|| {
let (to, back) = working_prophoto();
let source = format!(
"const PROFILE_FROM_WORKING = {};\nconst PROFILE_TO_WORKING = {};\n{LOOKUP_WGSL}",
"const PROFILE_FROM_WORKING = {};\nconst PROFILE_TO_WORKING = {};\n{SECTIONS_WGSL}",
wgsl_mat(to),
wgsl_mat(back)
);
[Helper {
name: "profile_apply",
Helper {
name: "profile_curve_base",
source: Box::leak(source.into_boxed_str()),
}]
}
});
static HELPERS: LazyLock<[Helper; 2]> = LazyLock::new(|| {
[
*PROPHOTO_HELPER,
Helper {
name: "profile_apply",
source: LOOKUP_WGSL,
},
]
});
/// Where each section of the profile buffer starts, from its header.
const SECTIONS_WGSL: &str = "
fn profile_entries(dims: vec4<f32>) -> u32 {
return u32(dims.x * dims.y * dims.z);
}
fn profile_look_base() -> u32 {
return 3u + profile_entries(profile_table[0]);
}
fn profile_curve_base() -> u32 {
return profile_look_base() + profile_entries(profile_table[1]);
}
";
/// The lookup, in WGSL. Mirrors [`apply_reference`] line for line.
const LOOKUP_WGSL: &str = r#"
fn profile_srgb_encode(v: f32) -> f32 {
@@ -327,12 +357,11 @@ if (hue_sat_dims.x > 0.0 || look_dims.x > 0.0) {
// A colour outside ProPhoto has no HSV the tables were made for; it
// passes through rather than being floored, which would clip it (D19).
if (min(p.r, min(p.g, p.b)) >= 0.0) {
let look_base = 2u + u32(hue_sat_dims.x * hue_sat_dims.y * hue_sat_dims.z);
if (hue_sat_dims.x > 0.0) {
p = profile_apply(hue_sat_dims, 2u, p, 1.0);
p = profile_apply(hue_sat_dims, 3u, p, 1.0);
}
if (look_dims.x > 0.0 && look > 0.0) {
p = profile_apply(look_dims, look_base, p, look);
p = profile_apply(look_dims, profile_look_base(), p, look);
}
c = PROFILE_TO_WORKING * p;
}
@@ -352,13 +381,16 @@ if (hue_sat_dims.x > 0.0 || look_dims.x > 0.0) {
}
}
/// TRACES: FR-DEV-3e
/// The storage buffer a source's tables are uploaded as: the two header
/// `vec4`s, the HueSatMap's entries, then the LookTable's, each entry
/// `(hue shift, saturation scale, value scale, 0)`.
/// TRACES: FR-DEV-3e | FR-DEV-3j
/// The storage buffer a source's profile is uploaded as: the three header
/// `vec4`s, the HueSatMap's entries, the LookTable's, each entry
/// `(hue shift, saturation scale, value scale, 0)`, then the tone curve's
/// samples in `.x`.
///
/// Two zero `vec4`s where there are no tables — the placeholder every source
/// without a profile binds, and what makes the fragment pass through.
/// The curve is always there: the profile's own where it has one, Camera
/// Raw's ACR3 default otherwise — including in the placeholder every source
/// without a profile binds, whose tables are absent, so a raw with no
/// profile still has the reference tone curve when it is chosen (D21).
pub fn profile_buffer(tables: Option<&ProfileTables>) -> Vec<[f32; 4]> {
let header = |t: Option<&HueSatTable>| match t {
Some(t) => [
@@ -371,10 +403,18 @@ pub fn profile_buffer(tables: Option<&ProfileTables>) -> Vec<[f32; 4]> {
};
let hue_sat = tables.and_then(|t| t.hue_sat.as_ref());
let look = tables.and_then(|t| t.look.as_ref());
let mut out = vec![header(hue_sat), header(look)];
let curve: &[f32] = tables
.and_then(|t| t.tone_curve.as_deref())
.unwrap_or(&dr_types::tone::ACR3_DEFAULT);
let mut out = vec![
header(hue_sat),
header(look),
[curve.len() as f32, 0.0, 0.0, 0.0],
];
for t in [hue_sat, look].into_iter().flatten() {
out.extend(t.entries.iter().map(|e| [e[0], e[1], e[2], 0.0]));
}
out.extend(curve.iter().map(|&v| [v, 0.0, 0.0, 0.0]));
out
}
@@ -536,6 +576,7 @@ mod tests {
origin: ProfileOrigin::Embedded,
hue_sat,
look,
tone_curve: None,
}
}
@@ -663,7 +704,10 @@ mod tests {
#[test]
fn the_buffer_puts_the_header_first_and_the_look_after_the_hue_sat_map() {
assert_eq!(profile_buffer(None), vec![[0.0; 4]; 2]);
let bare = profile_buffer(None);
assert_eq!(bare[..2], [[0.0; 4]; 2], "no tables");
assert_eq!(bare[2][0], 1025.0, "and the reference default curve");
assert_eq!(bare.len(), HEADER_ENTRIES + 1025);
let t = tables(
Some(uniform(2, 2, 1, [1.0, 2.0, 3.0])),
Some(uniform(3, 2, 2, [4.0, 5.0, 6.0])),
@@ -671,8 +715,9 @@ mod tests {
let b = profile_buffer(Some(&t));
assert_eq!(b[0], [2.0, 2.0, 1.0, 0.0]);
assert_eq!(b[1], [3.0, 2.0, 2.0, 0.0]);
assert_eq!(b.len(), HEADER_ENTRIES + 4 + 12);
assert_eq!(b[2], [1.0, 2.0, 3.0, 0.0]);
assert_eq!(b.len(), HEADER_ENTRIES + 4 + 12 + 1025);
assert_eq!(b[HEADER_ENTRIES], [1.0, 2.0, 3.0, 0.0]);
assert_eq!(b[HEADER_ENTRIES + 4], [4.0, 5.0, 6.0, 0.0]);
assert_eq!(b[HEADER_ENTRIES + 16][0], dr_types::tone::ACR3_DEFAULT[0]);
}
}
core/dr-pipeline/src/ops/view_transform.rs
+64 -8
View File
@@ -33,11 +33,33 @@ use crate::view::{Sigmoid, CONTRAST_RANGE, DEFAULT_CONTRAST, DEFAULT_WHITE, WHIT
pub const ID: OpId = OpId("view_transform");
pub const CONTRAST: ParamId = ParamId("contrast");
pub const WHITE: ParamId = ParamId("white");
/// TRACES: FR-DEV-3j
/// Which curve renders: the DNG reference (D21, the default) or D19's sigmoid.
///
/// The reference because it is the one that matches what the photographs were
/// first developed with: on 60 Lightroom exports with neutral settings it
/// renders their raws within MSE ~150 of Lightroom's own JPEGs at the default
/// contrast, where 0.20.0's sigmoid was ~1200 (darker by about 0.7 EV and
/// flatter). The sigmoid keeps index 0 because sidecars record the index.
pub const CURVE: ParamId = ParamId("curve");
static HELPERS: [Helper; 1] = [Helper {
name: "view_sigmoid",
source: crate::view::VIEW_SIGMOID_WGSL,
}];
/// [`CURVE`]'s values, in the order of its variants.
pub const SIGMOID: f32 = 0.0;
pub const CAMERA_RAW: f32 = 1.0;
static HELPERS: LazyLock<[Helper; 3]> = LazyLock::new(|| {
[
Helper {
name: "view_sigmoid",
source: crate::view::VIEW_SIGMOID_WGSL,
},
*crate::ops::camera_profile::PROPHOTO_HELPER,
Helper {
name: "camera_raw_tone",
source: crate::camera_raw::CAMERA_RAW_WGSL,
},
]
});
static DESCRIPTOR: LazyLock<Arc<OpDescriptor>> = LazyLock::new(|| {
Arc::new(OpDescriptor {
@@ -67,6 +89,15 @@ static DESCRIPTOR: LazyLock<Arc<OpDescriptor>> = LazyLock::new(|| {
Scale::Linear,
1,
),
ParamDescriptor::choice(
"curve",
"param.view_transform.curve",
vec![
LocalizedKey("param.view_transform.curve.sigmoid"),
LocalizedKey("param.view_transform.curve.camera_raw"),
],
)
.with_default(CAMERA_RAW),
],
})
});
@@ -75,6 +106,7 @@ static DESCRIPTOR: LazyLock<Arc<OpDescriptor>> = LazyLock::new(|| {
pub struct ViewTransform {
contrast: f32,
white: f32,
curve: f32,
}
impl Default for ViewTransform {
@@ -82,6 +114,7 @@ impl Default for ViewTransform {
Self {
contrast: DEFAULT_CONTRAST,
white: DEFAULT_WHITE,
curve: CAMERA_RAW,
}
}
}
@@ -106,6 +139,7 @@ impl Operation for ViewTransform {
match id {
CONTRAST => self.contrast = value,
WHITE => self.white = value,
CURVE => self.curve = value.round(),
_ => log::warn!("view_transform: unknown parameter {id}"),
}
}
@@ -114,12 +148,13 @@ impl Operation for ViewTransform {
match id {
CONTRAST => self.contrast,
WHITE => self.white,
CURVE => self.curve,
_ => 0.0,
}
}
fn is_active(&self) -> bool {
self.contrast != DEFAULT_CONTRAST || self.white != DEFAULT_WHITE
self.contrast != DEFAULT_CONTRAST || self.white != DEFAULT_WHITE || self.curve != CAMERA_RAW
}
fn stage(&self) -> Stage {
@@ -130,8 +165,13 @@ impl Operation for ViewTransform {
"\
// Skipped for an already-rendered source: a JPEG is a display rendering
// already, and rendering it again would compress it twice.
// D19's sigmoid by default, the DNG reference tone by choice (D21).
if (!non_linear) {
c = view_sigmoid(c, slope, inv_k, peak);
if (mode > 0.5) {
c = camera_raw_tone(c, cr_scale, cr_power, cr_grey);
} else {
c = view_sigmoid(c, slope, inv_k, peak);
}
}"
.into()
}
@@ -151,11 +191,27 @@ if (!non_linear) {
name: "peak",
value: s.w,
},
Uniform {
name: "mode",
value: self.curve,
},
Uniform {
name: "cr_scale",
value: crate::camera_raw::input_scale(self.white),
},
Uniform {
name: "cr_power",
value: crate::camera_raw::contrast_power(self.contrast),
},
Uniform {
name: "cr_grey",
value: crate::view::SCENE_GREY,
},
]
}
fn helpers(&self) -> &[Helper] {
&HELPERS
HELPERS.as_slice()
}
}
@@ -191,7 +247,7 @@ mod tests {
let s = Sigmoid::new(2.0, 6.0);
let u = op.uniforms();
assert_eq!(
u.iter().map(|u| u.value).collect::<Vec<_>>(),
u.iter().take(3).map(|u| u.value).collect::<Vec<_>>(),
vec![s.n, s.inv_k, s.w]
);
}
core/dr-pipeline/src/view.rs
+19 -6
View File
@@ -51,8 +51,19 @@ pub const SCENE_GREY: f32 = 0.13;
/// Display-linear middle grey — what a camera JPEG shows a grey card as.
pub const DISPLAY_GREY: f32 = 0.18;
/// The default contrast, the sigmoid's log-log slope parameter `n`.
pub const DEFAULT_CONTRAST: f32 = 1.4;
/// The default contrast: the sigmoid's log-log slope `n`, and for the DNG
/// reference curve a power of `DEFAULT_CONTRAST / REFERENCE_CONTRAST` about grey.
///
/// Fitted, not chosen: on Lightroom 6 exports whose look settings were neutral,
/// the DNG reference curve matched the exports best with the input bent by about
/// 1.08 (held-out MSE 224 at 1.4, ~150 at 1.5). The sigmoid, which is no longer
/// the default, fitted best near 1.7 and is better at 1.5 than at 1.4.
pub const DEFAULT_CONTRAST: f32 = 1.5;
/// The contrast at which each curve is its own reference: the sigmoid's match
/// to the retired base curve (see the module note), and the reference table
/// untouched.
pub const REFERENCE_CONTRAST: f32 = 1.4;
/// The default white point, in stops above [`SCENE_GREY`].
pub const DEFAULT_WHITE: f32 = 4.0;
@@ -223,11 +234,13 @@ mod tests {
}
#[test]
fn the_default_stays_close_to_the_retired_curve() {
fn the_reference_stays_close_to_the_retired_curve() {
// TRACES: FR-DEV-3j | FR-DEV-3e
// D19's promise to every existing photograph: the midtones do not
// move by more than a third of a stop.
let s = Sigmoid::default_curve();
// D19's promise, now kept by the sigmoid at its reference contrast
// rather than by the default (D21 moved the default to the DNG reference
// curve): the midtones do not move by more than a third of a stop
// from the retired base curve.
let s = Sigmoid::new(REFERENCE_CONTRAST, DEFAULT_WHITE);
let mut x = 0.03_f32;
while x <= 1.0 {
let ev = (s.channel(x) / retired_default(x)).log2();
core/dr-preset-xmp/src/lib.rs
+224 -1
View File
@@ -35,7 +35,7 @@
//! wrong on most images and invisibly so, which is worse than an honest gap,
//! so these keys are counted as skipped and reported.
//!
//! **Tone curves, colour mixing, masks, lens profiles and grain.** Each is a
//! **Tone curves, masks, lens profiles and grain.** Each is a
//! structure rather than a number, and each would need its own argument about
//! whether the two applications mean the same thing. They are skipped by
//! omission — a key not in the table is simply not understood — and the
@@ -137,6 +137,158 @@ const MAPPINGS: &[Mapping] = &[
param: "saturation",
convert: Convert::Direct,
},
// TRACES: FR-DEV-6
// Lightroom's HSL panel: eight bands, each ±100 for hue, saturation and
// luminance, onto the colour mixer's twelve. Lightroom's bands sit where
// ours do except two, matched to the nearest of ours by hue: Aqua (180°)
// is our cyan, Purple (270°) our violet; chartreuse, spring, azure and
// rose have no Lightroom counterpart and are left alone. One for one, as
// a first translation — the band widths differ, and `lr-fit`'s
// measurement against Lightroom's own output may yet scale these.
Mapping {
crs: "HueAdjustmentRed",
op: "colour_mixer",
param: "red_hue",
convert: Convert::Direct,
},
Mapping {
crs: "SaturationAdjustmentRed",
op: "colour_mixer",
param: "red_sat",
convert: Convert::Direct,
},
Mapping {
crs: "LuminanceAdjustmentRed",
op: "colour_mixer",
param: "red_lum",
convert: Convert::Direct,
},
Mapping {
crs: "HueAdjustmentOrange",
op: "colour_mixer",
param: "orange_hue",
convert: Convert::Direct,
},
Mapping {
crs: "SaturationAdjustmentOrange",
op: "colour_mixer",
param: "orange_sat",
convert: Convert::Direct,
},
Mapping {
crs: "LuminanceAdjustmentOrange",
op: "colour_mixer",
param: "orange_lum",
convert: Convert::Direct,
},
Mapping {
crs: "HueAdjustmentYellow",
op: "colour_mixer",
param: "yellow_hue",
convert: Convert::Direct,
},
Mapping {
crs: "SaturationAdjustmentYellow",
op: "colour_mixer",
param: "yellow_sat",
convert: Convert::Direct,
},
Mapping {
crs: "LuminanceAdjustmentYellow",
op: "colour_mixer",
param: "yellow_lum",
convert: Convert::Direct,
},
Mapping {
crs: "HueAdjustmentGreen",
op: "colour_mixer",
param: "green_hue",
convert: Convert::Direct,
},
Mapping {
crs: "SaturationAdjustmentGreen",
op: "colour_mixer",
param: "green_sat",
convert: Convert::Direct,
},
Mapping {
crs: "LuminanceAdjustmentGreen",
op: "colour_mixer",
param: "green_lum",
convert: Convert::Direct,
},
Mapping {
crs: "HueAdjustmentAqua",
op: "colour_mixer",
param: "cyan_hue",
convert: Convert::Direct,
},
Mapping {
crs: "SaturationAdjustmentAqua",
op: "colour_mixer",
param: "cyan_sat",
convert: Convert::Direct,
},
Mapping {
crs: "LuminanceAdjustmentAqua",
op: "colour_mixer",
param: "cyan_lum",
convert: Convert::Direct,
},
Mapping {
crs: "HueAdjustmentBlue",
op: "colour_mixer",
param: "blue_hue",
convert: Convert::Direct,
},
Mapping {
crs: "SaturationAdjustmentBlue",
op: "colour_mixer",
param: "blue_sat",
convert: Convert::Direct,
},
Mapping {
crs: "LuminanceAdjustmentBlue",
op: "colour_mixer",
param: "blue_lum",
convert: Convert::Direct,
},
Mapping {
crs: "HueAdjustmentPurple",
op: "colour_mixer",
param: "violet_hue",
convert: Convert::Direct,
},
Mapping {
crs: "SaturationAdjustmentPurple",
op: "colour_mixer",
param: "violet_sat",
convert: Convert::Direct,
},
Mapping {
crs: "LuminanceAdjustmentPurple",
op: "colour_mixer",
param: "violet_lum",
convert: Convert::Direct,
},
Mapping {
crs: "HueAdjustmentMagenta",
op: "colour_mixer",
param: "magenta_hue",
convert: Convert::Direct,
},
Mapping {
crs: "SaturationAdjustmentMagenta",
op: "colour_mixer",
param: "magenta_sat",
convert: Convert::Direct,
},
Mapping {
crs: "LuminanceAdjustmentMagenta",
op: "colour_mixer",
param: "magenta_lum",
convert: Convert::Direct,
},
// Adobe's sharpening runs 0…150 where ours runs 0…100, so a preset asking
// for its maximum gets ours rather than being clamped there silently.
Mapping {
@@ -192,6 +344,27 @@ pub enum ImportError {
NoSettings,
}
/// TRACES: FR-DEV-6
/// The Lightroom edit stored inside a photograph — the XMP packet Lightroom
/// writes into a DNG — translated as a preset is.
///
/// `None` where the file carries no packet, or one with no Camera Raw
/// settings in it (darktable's sidecars, a camera's own XMP). The packet is
/// found by its delimiters rather than by walking the TIFF structure: it is
/// plain text by specification, and the same search serves any container.
pub fn read_embedded(bytes: &[u8]) -> Option<Import> {
const OPEN: &[u8] = b"<x:xmpmeta";
const CLOSE: &[u8] = b"</x:xmpmeta>";
let start = find(bytes, OPEN)?;
let end = start + find(&bytes[start..], CLOSE)? + CLOSE.len();
let text = std::str::from_utf8(&bytes[start..end]).ok()?;
read_xmp(text).ok().filter(|i| !i.preset.is_empty())
}
fn find(haystack: &[u8], needle: &[u8]) -> Option<usize> {
haystack.windows(needle.len()).position(|w| w == needle)
}
/// Read one Lightroom `.xmp` preset.
///
/// Tolerant in the same direction the sidecar parser is: a value that will not
@@ -375,6 +548,56 @@ mod tests {
.copied()
}
#[test]
fn a_dngs_embedded_lightroom_edit_comes_across_with_its_hsl() {
// TRACES: FR-DEV-6
// The shape Lightroom 6 writes into a DNG, trimmed: the library's
// house look, as camera-profiles.md's D21 note records it.
let mut file = b"II*\0 binary header bytes ".to_vec();
file.extend_from_slice(
br#"<?xpacket begin="" id="W5M0MpCehiHzreSzNTczkc9d"?><x:xmpmeta xmlns:x="adobe:ns:meta/"><rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"><rdf:Description rdf:about="" xmlns:crs="http://ns.adobe.com/camera-raw-settings/1.0/" crs:ProcessVersion="6.7" crs:Exposure2012="0.00" crs:Highlights2012="-40" crs:Blacks2012="-20" crs:SaturationAdjustmentBlue="+58" crs:SaturationAdjustmentAqua="+50" crs:SaturationAdjustmentPurple="+23" crs:HueAdjustmentRed="-5" crs:LuminanceAdjustmentGreen="+7"/></rdf:RDF></x:xmpmeta><?xpacket end="w"?>"#,
);
file.extend_from_slice(b"\0 more binary");
let import = read_embedded(&file).expect("an edit");
let p = import.preset.params();
let get = |op: &str, param: &str| p.get(&(op.to_string(), param.to_string())).copied();
assert_eq!(get("colour_mixer", "blue_sat"), Some(58.0));
assert_eq!(get("colour_mixer", "cyan_sat"), Some(50.0));
assert_eq!(get("colour_mixer", "violet_sat"), Some(23.0));
assert_eq!(get("colour_mixer", "red_hue"), Some(-5.0));
assert_eq!(get("colour_mixer", "green_lum"), Some(7.0));
assert_eq!(get("highlights_shadows", "highlights"), Some(-40.0));
assert_eq!(get("blacks_whites", "blacks"), Some(-20.0));
}
#[test]
fn the_libraries_dngs_carry_the_house_look() {
// TRACES: FR-DEV-6
// The library's own Lightroom 6 DNG, where it is on this machine.
let Some(home) = std::env::var_os("HOME") else {
return;
};
let path =
std::path::Path::new(&home).join("Nextcloud/PhotosRaw/2017/2017-08-12/_MG_9080.dng");
let Ok(bytes) = std::fs::read(&path) else {
eprintln!("skipped: no sample DNG at {}", path.display());
return;
};
let import = read_embedded(&bytes).expect("Lightroom's edit");
let p = import.preset.params();
let get = |op: &str, param: &str| p.get(&(op.to_string(), param.to_string())).copied();
assert_eq!(get("colour_mixer", "blue_sat"), Some(58.0));
assert_eq!(get("colour_mixer", "cyan_sat"), Some(50.0));
assert_eq!(get("highlights_shadows", "highlights"), Some(-40.0));
}
#[test]
fn a_file_with_no_camera_raw_settings_has_no_edit() {
assert!(read_embedded(b"no packet at all").is_none());
let darktable = br#"<x:xmpmeta xmlns:x="adobe:ns:meta/"><rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"><rdf:Description rdf:about="" xmlns:xmp="http://ns.adobe.com/xap/1.0/" xmp:Rating="3"/></rdf:RDF></x:xmpmeta>"#;
assert!(read_embedded(darktable).is_none());
}
#[test]
fn every_mapping_names_a_parameter_this_build_actually_has() {
// The test that keeps the table honest. Adobe's half cannot be checked
core/dr-types/src/hue_sat.rs
+6 -1
View File
@@ -115,6 +115,11 @@ pub struct ProfileTables {
pub origin: ProfileOrigin,
pub hue_sat: Option<HueSatTable>,
pub look: Option<HueSatTable>,
/// The profile's `ProfileToneCurve`, resampled onto
/// [`crate::tone::TONE_SAMPLES`] points; `None` where it has none, and
/// the view transform's DNG reference curve then uses
/// [`crate::tone::ACR3_DEFAULT`] (D21).
pub tone_curve: Option<Vec<f32>>,
}
/// Where a profile was found (camera-profiles.md §4).
@@ -129,7 +134,7 @@ pub enum ProfileOrigin {
impl ProfileTables {
/// Whether there is anything to apply.
pub fn is_empty(&self) -> bool {
self.hue_sat.is_none() && self.look.is_none()
self.hue_sat.is_none() && self.look.is_none() && self.tone_curve.is_none()
}
}
core/dr-types/src/lib.rs
+1
View File
@@ -14,6 +14,7 @@ pub mod place;
pub mod selector;
pub mod settings;
pub mod time;
pub mod tone;
pub use colour::{Chromaticities, Transfer};
pub use hue_sat::{HueSatTable, ProfileOrigin, ProfileTables};
core/dr-types/src/tone.rs
+284
View File
@@ -0,0 +1,284 @@
//! TRACES: FR-DEV-3j | FR-DEV-3e
//! The DNG SDK's reference tone curve, as data (D21).
//!
//! # Where the numbers come from
//!
//! [`ACR3_DEFAULT`] is the "ACR3 default" tone curve of Adobe's DNG SDK
//! (`dng_tone_curve_acr3_default`): what the SDK's reference renders a raw through
//! when its camera profile carries no `ProfileToneCurve` of its own, which is
//! true of Adobe Standard. The values are RawTherapee's
//! `adobe_camera_raw_default_curve` (`rtengine/dcp.cc`, GPLv3), copied
//! digit for digit; `the_default_curve_is_rawtherapees` checks a sample of
//! them.
//!
//! It maps linear values to linear values, 1025 samples evenly over
//! `[0, 1]`, interpolated linearly between them. See
//! `docs/dev/camera-profiles.md` §12 for how it is applied — on the largest
//! and smallest channel, not on each — which is half of what it does.
/// Samples in a resolved tone curve: the ACR3 table's own resolution, and
/// what a profile's curve is resampled onto.
pub const TONE_SAMPLES: usize = 1025;
/// The ACR3 default tone curve, linear in, linear out.
///
/// `approx_constant` is allowed because one sample is 0.70711, which clippy
/// takes for an approximation of 1/√2. It is a measured value of the curve,
/// copied as published; replacing it with the constant would change it.
#[rustfmt::skip]
#[allow(clippy::approx_constant)]
pub const ACR3_DEFAULT: [f32; TONE_SAMPLES] = [
0.00000, 0.00078, 0.00160, 0.00242, 0.00314, 0.00385, 0.00460, 0.00539,
0.00623, 0.00712, 0.00806, 0.00906, 0.01012, 0.01122, 0.01238, 0.01359,
0.01485, 0.01616, 0.01751, 0.01890, 0.02033, 0.02180, 0.02331, 0.02485,
0.02643, 0.02804, 0.02967, 0.03134, 0.03303, 0.03475, 0.03648, 0.03824,
0.04002, 0.04181, 0.04362, 0.04545, 0.04730, 0.04916, 0.05103, 0.05292,
0.05483, 0.05675, 0.05868, 0.06063, 0.06259, 0.06457, 0.06655, 0.06856,
0.07057, 0.07259, 0.07463, 0.07668, 0.07874, 0.08081, 0.08290, 0.08499,
0.08710, 0.08921, 0.09134, 0.09348, 0.09563, 0.09779, 0.09996, 0.10214,
0.10433, 0.10652, 0.10873, 0.11095, 0.11318, 0.11541, 0.11766, 0.11991,
0.12218, 0.12445, 0.12673, 0.12902, 0.13132, 0.13363, 0.13595, 0.13827,
0.14061, 0.14295, 0.14530, 0.14765, 0.15002, 0.15239, 0.15477, 0.15716,
0.15956, 0.16197, 0.16438, 0.16680, 0.16923, 0.17166, 0.17410, 0.17655,
0.17901, 0.18148, 0.18395, 0.18643, 0.18891, 0.19141, 0.19391, 0.19641,
0.19893, 0.20145, 0.20398, 0.20651, 0.20905, 0.21160, 0.21416, 0.21672,
0.21929, 0.22185, 0.22440, 0.22696, 0.22950, 0.23204, 0.23458, 0.23711,
0.23963, 0.24215, 0.24466, 0.24717, 0.24967, 0.25216, 0.25465, 0.25713,
0.25961, 0.26208, 0.26454, 0.26700, 0.26945, 0.27189, 0.27433, 0.27676,
0.27918, 0.28160, 0.28401, 0.28641, 0.28881, 0.29120, 0.29358, 0.29596,
0.29833, 0.30069, 0.30305, 0.30540, 0.30774, 0.31008, 0.31241, 0.31473,
0.31704, 0.31935, 0.32165, 0.32395, 0.32623, 0.32851, 0.33079, 0.33305,
0.33531, 0.33756, 0.33981, 0.34205, 0.34428, 0.34650, 0.34872, 0.35093,
0.35313, 0.35532, 0.35751, 0.35969, 0.36187, 0.36404, 0.36620, 0.36835,
0.37050, 0.37264, 0.37477, 0.37689, 0.37901, 0.38112, 0.38323, 0.38533,
0.38742, 0.38950, 0.39158, 0.39365, 0.39571, 0.39777, 0.39982, 0.40186,
0.40389, 0.40592, 0.40794, 0.40996, 0.41197, 0.41397, 0.41596, 0.41795,
0.41993, 0.42191, 0.42388, 0.42584, 0.42779, 0.42974, 0.43168, 0.43362,
0.43554, 0.43747, 0.43938, 0.44129, 0.44319, 0.44509, 0.44698, 0.44886,
0.45073, 0.45260, 0.45447, 0.45632, 0.45817, 0.46002, 0.46186, 0.46369,
0.46551, 0.46733, 0.46914, 0.47095, 0.47275, 0.47454, 0.47633, 0.47811,
0.47989, 0.48166, 0.48342, 0.48518, 0.48693, 0.48867, 0.49041, 0.49214,
0.49387, 0.49559, 0.49730, 0.49901, 0.50072, 0.50241, 0.50410, 0.50579,
0.50747, 0.50914, 0.51081, 0.51247, 0.51413, 0.51578, 0.51742, 0.51906,
0.52069, 0.52232, 0.52394, 0.52556, 0.52717, 0.52878, 0.53038, 0.53197,
0.53356, 0.53514, 0.53672, 0.53829, 0.53986, 0.54142, 0.54297, 0.54452,
0.54607, 0.54761, 0.54914, 0.55067, 0.55220, 0.55371, 0.55523, 0.55673,
0.55824, 0.55973, 0.56123, 0.56271, 0.56420, 0.56567, 0.56715, 0.56861,
0.57007, 0.57153, 0.57298, 0.57443, 0.57587, 0.57731, 0.57874, 0.58017,
0.58159, 0.58301, 0.58443, 0.58583, 0.58724, 0.58864, 0.59003, 0.59142,
0.59281, 0.59419, 0.59556, 0.59694, 0.59830, 0.59966, 0.60102, 0.60238,
0.60373, 0.60507, 0.60641, 0.60775, 0.60908, 0.61040, 0.61173, 0.61305,
0.61436, 0.61567, 0.61698, 0.61828, 0.61957, 0.62087, 0.62216, 0.62344,
0.62472, 0.62600, 0.62727, 0.62854, 0.62980, 0.63106, 0.63232, 0.63357,
0.63482, 0.63606, 0.63730, 0.63854, 0.63977, 0.64100, 0.64222, 0.64344,
0.64466, 0.64587, 0.64708, 0.64829, 0.64949, 0.65069, 0.65188, 0.65307,
0.65426, 0.65544, 0.65662, 0.65779, 0.65897, 0.66013, 0.66130, 0.66246,
0.66362, 0.66477, 0.66592, 0.66707, 0.66821, 0.66935, 0.67048, 0.67162,
0.67275, 0.67387, 0.67499, 0.67611, 0.67723, 0.67834, 0.67945, 0.68055,
0.68165, 0.68275, 0.68385, 0.68494, 0.68603, 0.68711, 0.68819, 0.68927,
0.69035, 0.69142, 0.69249, 0.69355, 0.69461, 0.69567, 0.69673, 0.69778,
0.69883, 0.69988, 0.70092, 0.70196, 0.70300, 0.70403, 0.70506, 0.70609,
0.70711, 0.70813, 0.70915, 0.71017, 0.71118, 0.71219, 0.71319, 0.71420,
0.71520, 0.71620, 0.71719, 0.71818, 0.71917, 0.72016, 0.72114, 0.72212,
0.72309, 0.72407, 0.72504, 0.72601, 0.72697, 0.72794, 0.72890, 0.72985,
0.73081, 0.73176, 0.73271, 0.73365, 0.73460, 0.73554, 0.73647, 0.73741,
0.73834, 0.73927, 0.74020, 0.74112, 0.74204, 0.74296, 0.74388, 0.74479,
0.74570, 0.74661, 0.74751, 0.74842, 0.74932, 0.75021, 0.75111, 0.75200,
0.75289, 0.75378, 0.75466, 0.75555, 0.75643, 0.75730, 0.75818, 0.75905,
0.75992, 0.76079, 0.76165, 0.76251, 0.76337, 0.76423, 0.76508, 0.76594,
0.76679, 0.76763, 0.76848, 0.76932, 0.77016, 0.77100, 0.77183, 0.77267,
0.77350, 0.77432, 0.77515, 0.77597, 0.77680, 0.77761, 0.77843, 0.77924,
0.78006, 0.78087, 0.78167, 0.78248, 0.78328, 0.78408, 0.78488, 0.78568,
0.78647, 0.78726, 0.78805, 0.78884, 0.78962, 0.79040, 0.79118, 0.79196,
0.79274, 0.79351, 0.79428, 0.79505, 0.79582, 0.79658, 0.79735, 0.79811,
0.79887, 0.79962, 0.80038, 0.80113, 0.80188, 0.80263, 0.80337, 0.80412,
0.80486, 0.80560, 0.80634, 0.80707, 0.80780, 0.80854, 0.80926, 0.80999,
0.81072, 0.81144, 0.81216, 0.81288, 0.81360, 0.81431, 0.81503, 0.81574,
0.81645, 0.81715, 0.81786, 0.81856, 0.81926, 0.81996, 0.82066, 0.82135,
0.82205, 0.82274, 0.82343, 0.82412, 0.82480, 0.82549, 0.82617, 0.82685,
0.82753, 0.82820, 0.82888, 0.82955, 0.83022, 0.83089, 0.83155, 0.83222,
0.83288, 0.83354, 0.83420, 0.83486, 0.83552, 0.83617, 0.83682, 0.83747,
0.83812, 0.83877, 0.83941, 0.84005, 0.84069, 0.84133, 0.84197, 0.84261,
0.84324, 0.84387, 0.84450, 0.84513, 0.84576, 0.84639, 0.84701, 0.84763,
0.84825, 0.84887, 0.84949, 0.85010, 0.85071, 0.85132, 0.85193, 0.85254,
0.85315, 0.85375, 0.85436, 0.85496, 0.85556, 0.85615, 0.85675, 0.85735,
0.85794, 0.85853, 0.85912, 0.85971, 0.86029, 0.86088, 0.86146, 0.86204,
0.86262, 0.86320, 0.86378, 0.86435, 0.86493, 0.86550, 0.86607, 0.86664,
0.86720, 0.86777, 0.86833, 0.86889, 0.86945, 0.87001, 0.87057, 0.87113,
0.87168, 0.87223, 0.87278, 0.87333, 0.87388, 0.87443, 0.87497, 0.87552,
0.87606, 0.87660, 0.87714, 0.87768, 0.87821, 0.87875, 0.87928, 0.87981,
0.88034, 0.88087, 0.88140, 0.88192, 0.88244, 0.88297, 0.88349, 0.88401,
0.88453, 0.88504, 0.88556, 0.88607, 0.88658, 0.88709, 0.88760, 0.88811,
0.88862, 0.88912, 0.88963, 0.89013, 0.89063, 0.89113, 0.89163, 0.89212,
0.89262, 0.89311, 0.89360, 0.89409, 0.89458, 0.89507, 0.89556, 0.89604,
0.89653, 0.89701, 0.89749, 0.89797, 0.89845, 0.89892, 0.89940, 0.89987,
0.90035, 0.90082, 0.90129, 0.90176, 0.90222, 0.90269, 0.90316, 0.90362,
0.90408, 0.90454, 0.90500, 0.90546, 0.90592, 0.90637, 0.90683, 0.90728,
0.90773, 0.90818, 0.90863, 0.90908, 0.90952, 0.90997, 0.91041, 0.91085,
0.91130, 0.91173, 0.91217, 0.91261, 0.91305, 0.91348, 0.91392, 0.91435,
0.91478, 0.91521, 0.91564, 0.91606, 0.91649, 0.91691, 0.91734, 0.91776,
0.91818, 0.91860, 0.91902, 0.91944, 0.91985, 0.92027, 0.92068, 0.92109,
0.92150, 0.92191, 0.92232, 0.92273, 0.92314, 0.92354, 0.92395, 0.92435,
0.92475, 0.92515, 0.92555, 0.92595, 0.92634, 0.92674, 0.92713, 0.92753,
0.92792, 0.92831, 0.92870, 0.92909, 0.92947, 0.92986, 0.93025, 0.93063,
0.93101, 0.93139, 0.93177, 0.93215, 0.93253, 0.93291, 0.93328, 0.93366,
0.93403, 0.93440, 0.93478, 0.93515, 0.93551, 0.93588, 0.93625, 0.93661,
0.93698, 0.93734, 0.93770, 0.93807, 0.93843, 0.93878, 0.93914, 0.93950,
0.93986, 0.94021, 0.94056, 0.94092, 0.94127, 0.94162, 0.94197, 0.94231,
0.94266, 0.94301, 0.94335, 0.94369, 0.94404, 0.94438, 0.94472, 0.94506,
0.94540, 0.94573, 0.94607, 0.94641, 0.94674, 0.94707, 0.94740, 0.94774,
0.94807, 0.94839, 0.94872, 0.94905, 0.94937, 0.94970, 0.95002, 0.95035,
0.95067, 0.95099, 0.95131, 0.95163, 0.95194, 0.95226, 0.95257, 0.95289,
0.95320, 0.95351, 0.95383, 0.95414, 0.95445, 0.95475, 0.95506, 0.95537,
0.95567, 0.95598, 0.95628, 0.95658, 0.95688, 0.95718, 0.95748, 0.95778,
0.95808, 0.95838, 0.95867, 0.95897, 0.95926, 0.95955, 0.95984, 0.96013,
0.96042, 0.96071, 0.96100, 0.96129, 0.96157, 0.96186, 0.96214, 0.96242,
0.96271, 0.96299, 0.96327, 0.96355, 0.96382, 0.96410, 0.96438, 0.96465,
0.96493, 0.96520, 0.96547, 0.96574, 0.96602, 0.96629, 0.96655, 0.96682,
0.96709, 0.96735, 0.96762, 0.96788, 0.96815, 0.96841, 0.96867, 0.96893,
0.96919, 0.96945, 0.96971, 0.96996, 0.97022, 0.97047, 0.97073, 0.97098,
0.97123, 0.97149, 0.97174, 0.97199, 0.97223, 0.97248, 0.97273, 0.97297,
0.97322, 0.97346, 0.97371, 0.97395, 0.97419, 0.97443, 0.97467, 0.97491,
0.97515, 0.97539, 0.97562, 0.97586, 0.97609, 0.97633, 0.97656, 0.97679,
0.97702, 0.97725, 0.97748, 0.97771, 0.97794, 0.97817, 0.97839, 0.97862,
0.97884, 0.97907, 0.97929, 0.97951, 0.97973, 0.97995, 0.98017, 0.98039,
0.98061, 0.98082, 0.98104, 0.98125, 0.98147, 0.98168, 0.98189, 0.98211,
0.98232, 0.98253, 0.98274, 0.98295, 0.98315, 0.98336, 0.98357, 0.98377,
0.98398, 0.98418, 0.98438, 0.98458, 0.98478, 0.98498, 0.98518, 0.98538,
0.98558, 0.98578, 0.98597, 0.98617, 0.98636, 0.98656, 0.98675, 0.98694,
0.98714, 0.98733, 0.98752, 0.98771, 0.98789, 0.98808, 0.98827, 0.98845,
0.98864, 0.98882, 0.98901, 0.98919, 0.98937, 0.98955, 0.98973, 0.98991,
0.99009, 0.99027, 0.99045, 0.99063, 0.99080, 0.99098, 0.99115, 0.99133,
0.99150, 0.99167, 0.99184, 0.99201, 0.99218, 0.99235, 0.99252, 0.99269,
0.99285, 0.99302, 0.99319, 0.99335, 0.99351, 0.99368, 0.99384, 0.99400,
0.99416, 0.99432, 0.99448, 0.99464, 0.99480, 0.99495, 0.99511, 0.99527,
0.99542, 0.99558, 0.99573, 0.99588, 0.99603, 0.99619, 0.99634, 0.99649,
0.99664, 0.99678, 0.99693, 0.99708, 0.99722, 0.99737, 0.99751, 0.99766,
0.99780, 0.99794, 0.99809, 0.99823, 0.99837, 0.99851, 0.99865, 0.99879,
0.99892, 0.99906, 0.99920, 0.99933, 0.99947, 0.99960, 0.99974, 0.99987,
1.00000,
];
/// TRACES: FR-DEV-3e
/// A profile's `ProfileToneCurve` — `(x, y)` pairs in `[0, 1]` — resampled
/// onto [`TONE_SAMPLES`] even points with a natural cubic spline, the DNG
/// SDK's `dng_spline_solver`.
///
/// `None` where the pairs do not describe a curve (fewer than two points,
/// x not increasing, values outside `[0, 1]`) or describe the identity,
/// which RawTherapee also treats as no curve.
pub fn resample_tone_curve(pairs: &[f32]) -> Option<Vec<f32>> {
if pairs.len() < 4 || !pairs.len().is_multiple_of(2) {
return None;
}
let xs: Vec<f64> = pairs.iter().step_by(2).map(|&v| f64::from(v)).collect();
let ys: Vec<f64> = pairs
.iter()
.skip(1)
.step_by(2)
.map(|&v| f64::from(v))
.collect();
let sane = xs.windows(2).all(|w| w[1] > w[0])
&& xs
.iter()
.chain(&ys)
.all(|v| v.is_finite() && (-1e-6..=1.0 + 1e-6).contains(v));
if !sane {
return None;
}
if xs.iter().zip(&ys).all(|(x, y)| (x - y).abs() < 1e-6) {
return None;
}
let n = xs.len();
// Natural cubic spline: second derivatives zero at both ends, solved by
// the tridiagonal (Thomas) algorithm.
let mut m = vec![0.0f64; n];
if n > 2 {
let h: Vec<f64> = xs.windows(2).map(|w| w[1] - w[0]).collect();
let mut a = vec![0.0; n];
let mut b = vec![1.0; n];
let mut c = vec![0.0; n];
let mut d = vec![0.0; n];
for i in 1..n - 1 {
a[i] = h[i - 1];
b[i] = 2.0 * (h[i - 1] + h[i]);
c[i] = h[i];
d[i] = 6.0 * ((ys[i + 1] - ys[i]) / h[i] - (ys[i] - ys[i - 1]) / h[i - 1]);
}
for i in 1..n {
let w = a[i] / b[i - 1];
b[i] -= w * c[i - 1];
d[i] -= w * d[i - 1];
}
m[n - 1] = d[n - 1] / b[n - 1];
for i in (0..n - 1).rev() {
m[i] = (d[i] - c[i] * m[i + 1]) / b[i];
}
}
let eval = |x: f64| -> f64 {
if x <= xs[0] {
return ys[0];
}
if x >= xs[n - 1] {
return ys[n - 1];
}
let i = xs.windows(2).position(|w| x <= w[1]).unwrap_or(n - 2);
let h = xs[i + 1] - xs[i];
let t = (x - xs[i]) / h;
let u = 1.0 - t;
u * ys[i]
+ t * ys[i + 1]
+ ((u * u * u - u) * m[i] + (t * t * t - t) * m[i + 1]) * h * h / 6.0
};
Some(
(0..TONE_SAMPLES)
.map(|k| eval(k as f64 / (TONE_SAMPLES - 1) as f64).clamp(0.0, 1.0) as f32)
.collect(),
)
}
/// A resolved curve at `x`, linear interpolation between samples, `x`
/// clamped to `[0, 1]` — the shader's lookup, for the CPU reference.
pub fn evaluate(curve: &[f32], x: f32) -> f32 {
let last = curve.len() - 1;
let s = x.clamp(0.0, 1.0) * last as f32;
let i = (s as usize).min(last - 1);
let f = s - i as f32;
curve[i] + (curve[i + 1] - curve[i]) * f
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn the_default_curve_is_rawtherapees() {
assert_eq!(ACR3_DEFAULT.len(), 1025);
assert_eq!(ACR3_DEFAULT[0], 0.0);
assert_eq!(ACR3_DEFAULT[1024], 1.0);
assert_eq!(ACR3_DEFAULT[1], 0.00078);
assert_eq!(ACR3_DEFAULT[256], 0.52069);
assert_eq!(ACR3_DEFAULT[512], 0.80486);
assert_eq!(ACR3_DEFAULT[768], 0.93986);
assert!(ACR3_DEFAULT.windows(2).all(|w| w[1] >= w[0]), "monotone");
}
#[test]
fn a_resampled_curve_passes_through_its_points() {
let c = resample_tone_curve(&[0.0, 0.0, 0.5, 0.6, 1.0, 1.0]).unwrap();
assert_eq!(c.len(), TONE_SAMPLES);
assert!((evaluate(&c, 0.5) - 0.6).abs() < 1e-4);
assert!(evaluate(&c, 0.0).abs() < 1e-6 && (evaluate(&c, 1.0) - 1.0).abs() < 1e-6);
assert!(
evaluate(&c, 0.25) > 0.25,
"a lifted curve lifts between its points"
);
}
#[test]
fn an_identity_or_broken_curve_is_no_curve() {
assert!(resample_tone_curve(&[0.0, 0.0, 1.0, 1.0]).is_none());
assert!(resample_tone_curve(&[0.0, 0.0, 0.5]).is_none());
assert!(resample_tone_curve(&[0.0, 0.0, 0.6, 0.5, 0.4, 0.9, 1.0, 1.0]).is_none());
}
}
docker/android/assemble-apk.sh
+1 -1
View File
@@ -299,7 +299,7 @@ fi
rm -rf "${OUT}/staging/assets/models"
mkdir -p "${OUT}/staging/assets/models"
_bundled=""
for _dir in face scene inpaint; do
for _dir in face scene inpaint denoise; do
ASSETS="${REPO}/models/${_dir}"
compgen -G "${ASSETS}/*.onnx" >/dev/null || continue
# An LFS pointer is ~130 bytes and looks exactly like a model to `cp`. Left
docker/macos/Dockerfile
+48
View File
@@ -0,0 +1,48 @@
# DarkRoom — macOS link check
#
# Compiles and links for macOS from Linux, with zig as the linker
# (cargo-zigbuild). Zig carries macOS's libSystem stubs and C headers, so the
# crates that need only libSystem — the inference engine, dr-plat — build,
# link and produce Mach-O test binaries here. Nothing runs: there is no macOS
# to run them on (docs/dev/macos.md §2). The desktop app needs Apple's
# framework headers (AppKit, Metal, Security), which only the Xcode SDK
# carries, so it does not link here.
#
# Build: docker build -t darkroom-macos:latest docker/macos
# Use: ./docker/macos/build.sh cargo zigbuild --target aarch64-apple-darwin -p dr-inference-engine --all-targets
FROM docker.io/library/debian:trixie-slim
# Pinned, like the Windows and Android images. Rust matches rust-toolchain.toml.
ARG RUST_VERSION=1.92.0
ARG ZIG_VERSION=0.15.2
ARG ZIG_SHA256=02aa270f183da276e5b5920b1dac44a63f1a49e55050ebde3aecc9eb82f93239
ARG CARGO_ZIGBUILD_VERSION=0.23.4
ENV DEBIAN_FRONTEND=noninteractive \
CARGO_HOME=/opt/cargo \
RUSTUP_HOME=/opt/rustup \
PATH=/opt/zig:/opt/cargo/bin:$PATH
RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates curl git xz-utils \
# A host C compiler: build scripts and proc-macros are Linux binaries.
gcc libc6-dev \
# `file` says Mach-O; the smoke check in build.sh reads it.
file \
&& rm -rf /var/lib/apt/lists/*
RUN curl -fsSL "https://ziglang.org/download/${ZIG_VERSION}/zig-x86_64-linux-${ZIG_VERSION}.tar.xz" -o /tmp/zig.tar.xz \
&& echo "${ZIG_SHA256} /tmp/zig.tar.xz" | sha256sum -c - \
&& mkdir /opt/zig && tar xJf /tmp/zig.tar.xz -C /opt/zig --strip-components=1 \
&& rm /tmp/zig.tar.xz && zig version
# The components rust-toolchain.toml lists, baked in so rustup does not fetch
# them inside every run.
RUN curl -fsSL https://sh.rustup.rs | sh -s -- -y --profile minimal \
--default-toolchain "${RUST_VERSION}" \
--component rustfmt,clippy,rust-analyzer \
--target aarch64-apple-darwin,x86_64-apple-darwin \
&& cargo install --locked "cargo-zigbuild@${CARGO_ZIGBUILD_VERSION}" \
&& rm -rf /opt/cargo/registry \
&& chmod -R a+rwX /opt/cargo /opt/rustup
docker/macos/build.sh
Executable
+62
View File
@@ -0,0 +1,62 @@
#!/usr/bin/env bash
# Run a command inside the DarkRoom macOS link-check container.
#
# ./docker/macos/build.sh cargo zigbuild --target aarch64-apple-darwin -p dr-inference-engine --all-targets
# ./docker/macos/build.sh # interactive shell
#
# Builds the image on first use; `--rebuild` after editing the Dockerfile.
set -euo pipefail
IMAGE="darkroom-macos:latest"
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO="$(cd "${HERE}/../.." && pwd)"
if command -v podman >/dev/null 2>&1; then
ENGINE=podman
elif command -v docker >/dev/null 2>&1; then
ENGINE=docker
else
echo "error: neither podman nor docker found" >&2
exit 1
fi
if [[ "${1:-}" == "--rebuild" ]]; then
shift
"${ENGINE}" build -t "${IMAGE}" "${HERE}"
elif ! "${ENGINE}" image inspect "${IMAGE}" >/dev/null 2>&1; then
echo "==> building ${IMAGE} (first run; a few minutes)"
"${ENGINE}" build -t "${IMAGE}" "${HERE}"
fi
# Registry, target and zig's own cache persist across runs.
CACHE="${XDG_CACHE_HOME:-${HOME}/.cache}/darkroom-macos"
mkdir -p "${CACHE}/registry" "${CACHE}/target" "${CACHE}/home"
ARGS=(
--rm
-v "${REPO}:/work:z"
-v "${CACHE}/registry:/opt/cargo/registry:z"
-v "${CACHE}/target:/work/target-macos:z"
-v "${CACHE}/home:/tmp/home:z"
-e HOME=/tmp/home
-e CARGO_TARGET_DIR=/work/target-macos
-w /work
)
# Capped for the same reason as the Windows image: a cross build otherwise
# takes every thread on the host.
JOBS="${DARKROOM_BUILD_JOBS:-8}"
if [[ "${JOBS}" != "0" ]]; then
ARGS+=(--cpus "${JOBS}" -e "CARGO_BUILD_JOBS=${JOBS}")
fi
if [[ "${ENGINE}" == "docker" ]]; then
ARGS+=(--user "$(id -u):$(id -g)")
fi
if [[ $# -eq 0 ]]; then
ARGS+=(-it)
set -- /bin/bash
fi
exec "${ENGINE}" run "${ARGS[@]}" "${IMAGE}" "$@"
docker/windows/package.sh
+2 -2
View File
@@ -51,10 +51,10 @@ sed 's/$/\r/' "${REPO}/LICENSE" > "${STAGE}/LICENSE"
#
# The directories are the ones the APK stages (assemble-apk.sh) and the Arch
# package installs: the face pair and its eye-state models, the scene model
# with its two descriptors, and the panorama border filler. The installer
# with its two descriptors, the panorama border filler and the denoiser. The installer
# smoke test counts the same directories, so a model added here is expected
# there without a number to update.
for dir in face scene inpaint; do
for dir in face scene inpaint denoise; do
for f in "${REPO}/models/${dir}"/*; do
case "$(basename "${f}")" in
README.md) continue ;;
docs/dev/camera-profiles.md
+145 -12
View File
@@ -95,7 +95,7 @@ Two departures, both forced by D19's unbounded scene-linear values (the SDK runs
both tables, which is where the second reason matters:
- **The look sees exposure.** The SDK applies `LookTable` after its exposure ramp, so a look that
desaturates highlights finds the highlights the photographer chose. At 25 it does too. Contrast,
tone and the colour controls come after it, as they do in Camera Raw.
tone and the colour controls come after it, as they do in the DNG SDK's reference rendering.
**Tables are per source, like the matrix.** They are decoded with the raw, interpolated once at
decode (the HueSatMap blend uses the as-shot neutral, as the matrix does) and carried on
@@ -151,7 +151,7 @@ reads a buffer no declaration can name:
| Parameter | Kind | Default | Meaning |
|---|---|---|---|
| `apply` | Bool | on | Use the profile's tables, or the matrix alone |
| `look` | Scalar 0–200 | 100 | Strength of the `LookTable`, as Lightroom's *Amount* |
| `look` | Scalar 0–200 | 100 | Strength of the `LookTable` |
`look` scales the look's deltas: `hueShift · a`, `1 + (satScale − 1)·a`, `1 + (valScale − 1)·a`,
with `a = look/100`, scales floored at 0. At 200 the look is twice as strong, which is the
@@ -172,22 +172,17 @@ action sits on that line.
## 6. Not done, and why
- **`ProfileToneCurve` is read and ignored.** D19 gives tone to the view transform, one for every
body, and rejected per-body curves as its defaults. A profile's curve is a per-body curve. If it
comes back, it comes back as an option of the view transform, not as a stage here. The 6D's
Adobe Standard has none, so the case that matters today loses nothing.
- **`BaselineExposure` is not applied** (it is not today either). Adobe Standard was tuned with it,
and the 6D's is +0.25 EV. Separate change; it moves every photograph's brightness.
- ~~**`ProfileToneCurve` is read and ignored.**~~ *Done after 0.20.0: §12, D21.* It came back as
an option of the view transform, as this bullet said it would, and that option is the default
for raws.
- ~~**`BaselineExposure` is not applied.**~~ *Done after 0.20.0: §11.*
- **The interpolation follows the as-shot neutral, not the white-balance slider**, as the matrix
does. Camera Raw re-blends on every temperature change; doing so here means the matrix moves too,
which is its own change.
- **Masks select on the matrix's colour.** A colour-range mask sees colour before the profile, as
it sees colour before every other operation. Deterministic, and a mask is drawn on the picture
the user sees only approximately anyway.
- **The profiles directory does not sync.** A CR2 rendered on a desktop with a copied 6D profile
and on a tablet without one will differ. The panel line says which profile each device used, so
the difference is visible rather than silent. Syncing the directory with the library is the
follow-up.
- ~~**The profiles directory does not sync.**~~ *Done after 0.20.0: §13.*
- **Rec.2020 working primaries** stay deferred (D19); nothing here depends on them.
## 7. What it costs
@@ -256,3 +251,141 @@ is inside its control's range, and every preset changes something.
4. `dr-ui`: profile line, copy-out action, labels; the profiles directory set at start-up on
desktop and Android.
5. The *Vivid* presets.
---
# After 0.20.0: tone, exposure and sync
0.20.0 shipped the tables and §1's measurement showed they were not the gap to Lightroom's colour.
The three items §6 left open are closed here. Draft of 2026-10-03.
## 11. Baseline exposure
`BaselineExposure` (DNG tag 50730) is the stops a converter adds so a camera's middle grey lands
where its maker meant it; the 6D's DNGs say +0.25. `BaselineExposureOffset` (51109) is a profile's
correction to it. The DNG SDK's total is their sum, and so is this one's.
- **Applied as a gain on the camera matrix** in `dr-gpu` when a raw is uploaded:
`cam_to_srgb · 2^total`. A uniform gain commutes with every scene operation before the view
transform, and the camera-space tap and the white-balance probe read camera RGB before the
matrix, so neither changes. `RawImage::color_matrix` itself stays the file's: a merge writes a
linear DNG from it and must not bake a gain into the pixels it also declares in a tag.
- **A copied profile carries the DNG's baseline.** When an embedded profile is saved as a `.dcp`
(§4) its `BaselineExposureOffset` is written as the DNG's `BaselineExposure` plus the profile's
own offset. A CR2 has no baseline of its own, so its total is then the DNG's: the two files of
one body render at one brightness. The cost: a DNG that embeds no profile, carries its own
baseline, and matches a copied `.dcp` counts the baseline twice. Every Adobe-written DNG embeds
its profile, so that DNG is a hand-made one.
## 12. DNG reference tone (D21)
The DNG SDK's reference rendering runs a raw through the profile's
`ProfileToneCurve`, or, for a profile that has none — Adobe Standard among them — through the
*ACR3 default curve*, a 1025-point table published in the DNG SDK and carried by RawTherapee
(GPLv3) as `adobe_camera_raw_default_curve`.
**How it is applied** is half of what it does. The reference does not run the curve on each channel:
`RefBaselineRGBTone` runs it on the largest and smallest channel and places the middle one at the
same fraction between them as before. Hue is kept; saturation rises where the curve is steep —
the shadows and midtones — which is exactly where Adobe Standard's look table desaturated to
compensate. It runs in linear ProPhoto, on values clipped to `[0, 1]`, and its output is linear.
**As the view transform**, not as a stage. D19 has one rendering, last; this is a second kind of
that rendering, chosen by a new parameter on `view_transform`:
| `curve` | What it is | Default for |
|---|---|---|
| Sigmoid | D19's log-logistic curve | — (a choice) |
| DNG reference | the profile's curve, else ACR3, via RGBTone in ProPhoto | every raw (D21) |
*Decided 2026-10-03:* the DNG reference is the default, at contrast 1.5 — measured against
Lightroom exports of photographs with neutral look settings (D21 has the table).
*Amended earlier on 2026-10-03:* the DNG reference was the default in the first draft. The measurement it rested on
compared against Lightroom renders of *edited* photographs; see D21. The default is decided by
measuring against Lightroom exports of unedited ones.
A JPEG is still not rendered again (FR-DEV-3j). Film simulation still replaces the view transform
when a stock is chosen.
**The two sliders keep meaning something** under the DNG reference curve:
- `white` (stops above grey at which the scene reaches display white) sets the input scale:
`2^(4 − white)`. At its default of 4 the scale is 1 — sensor white is display white, as in
the SDK's reference.
- `contrast` bends the input about middle grey before the curve, as a power of
`contrast / 1.4`: 1 at its default, so the curve is the reference's untouched.
**What the ACR curve gives up** is D19's shoulder. Values above display white clip, as they do
in the SDK's reference; highlight recovery is the highlights slider's job before it. Sigmoid stays one
click away for a photograph that wants the shoulder.
**A profile's own curve** is a list of `(x, y)` pairs. It is resampled at decode onto the same
1025 points with a natural cubic spline, the DNG SDK's `dng_spline_solver`. A curve that is the
identity is treated as absent, as RawTherapee does.
**On the GPU** the curve rides in the profile buffer (`@binding(8)`) after the tables: a third
header entry gives its length, then the samples. The placeholder bound for a source without a
profile carries the ACR3 curve, so a CR2 with no `.dcp` renders through the reference tone when that curve is chosen.
## 13. Profiles sync
The profiles directory travels with the library, in the server folder that already carries what
every device must agree on: `<library root>/.darkroom-derived/profiles/`. The scanner excludes
its parent, as it excludes the trash.
- **A step of the derived sync pass** (`derived_sync::run`), after the catalog and before the
place file, and like the place file it never fails the pass. It lists the server folder and the
local one; uploads every local `.dcp` the server lacks, or holds at a different size; downloads
every one the device lacks, reading through a placeholder where the library is a synced folder,
and writing `.tmp` then renaming so a half-written file is never parsed. If it fetched
anything, it reloads the profile set.
- **Files are immutable and named for what they hold** (`<camera> <profile>.dcp`), so a name and a
size say whether two copies are the same. Two devices that copy the same profile write the same
name; neither wins over anything.
- **Not a catalog table.** A schema change stops an older peer merging the catalog at all
(see the memory of 0.13.3), and a blob of ~120 KB would ride in every catalog upload.
- **One directory per install**, the union of every library's profiles. A profile describes a
camera, not a library, so a profile one library brought is right for the same camera in
another.
- **Not handled: deleting.** There is no way to remove a profile from the app; a file removed by
hand on one device comes back from the server on the next pass. A tombstone list is the
follow-up if deleting is added.
## 14. Acceptance for §11–§13
- The ACR3 table is 1025 points from 0 to 1, monotone, and matches RawTherapee's values.
- RGBTone: grey goes through the curve unchanged in hue; a colour keeps its hue (the middle
channel's fraction between the outer two is unchanged); a curve that is the identity changes
nothing; the shader agrees with the CPU reference on a device.
- Sigmoid is the default curve; at its defaults it renders to the bit what 0.20.0 rendered,
apart from baseline exposure.
- A DNG with `BaselineExposure` +0.25 renders a flat grey 0.25 EV brighter than the same pixels
with none; a `.dcp` copied from it carries `BaselineExposureOffset` 0.25 and gives a CR2 the same
total.
- A profile resampled from `(0,0) (0.5,0.6) (1,1)` passes through its points.
- Sync: a `.dcp` present only locally is uploaded; one present only on the server is downloaded,
parsed and matched on the next decode; a file of the same name and size is left alone.
- Measured again on `_MG_9080`: mean saturation at defaults closer to Lightroom's 0.49 than 0.20.0's
0.35.
## 15. The photographer's earlier edit
What §1 and D21 first took for a difference in rendering is an edit. Every DNG in the library
carries, in its embedded XMP, the develop settings it was given before it came to DarkRoom — a
consistent house style: per-colour saturation (blue +58, aqua +50, yellow and purple +23, orange
+13, green +10), highlights −40, blacks −20, with a second variant (vibrance −10, blue +31). Those
settings, not the profile and not the tone curve, are why the same photographs looked richer
before.
- **Translated on open** (`dr_preset_xmp::read_embedded`), the HSL bands onto the colour mixer —
aqua to cyan and purple to violet, the nearest of its twelve by hue — and the rest as the preset
importer already did.
- **Applied only to a photograph DarkRoom has no edit of**, and only on positive evidence: no
sidecar beside a local file, or a server that answered "no such file" with nothing cached
(`FetchedSidecar::absent`). An edit that failed to arrive is not an absent one, and this would
otherwise be saved over it.
- **One undoable step, "Earlier Edit"**, then an ordinary edit, saved with the photograph. Export
applies it the same way, so a photograph never opened exports as opening it would show.
- **The translation is one for one for now.** Measurement against the earlier exports (the
`lr-fit` work) may scale individual bands.
docs/dev/denoise.md
+74 -9
View File
@@ -1,8 +1,9 @@
# Learned denoise — joint demosaic and denoise on the mosaic
Design for **FR-DEV-3g** ([requirements.md](requirements.md)), the learned stage
[outstanding.md §3](outstanding.md) says is missing. Draft of 2026-09-27: nothing here is built,
and every figure marked *estimate* is waiting for the measurement that replaces it.
[outstanding.md §3](outstanding.md) says is missing. Drafted 2026-09-27; a first version shipped
in 0.21.0, and §11 records what was built and measured. Figures still marked *estimate* are
waiting for the measurement that replaces them.
---
@@ -288,17 +289,27 @@ is ~120 MB, and a derived file inside a synced tree is exactly what
with progress over the canvas — the same pattern as a photograph that is only on the server.
- Export needs the result and computes it if the cache has lost it.
### 7.2 The Amount control
### 7.2 The grain control
A Denoise toggle and one Amount slider in develop. Moving the slider runs inference on the
**visible viewport only** (~1 MP, a fraction of a second — *estimate*) so the photographer judges
on the real result; releasing it queues the whole frame. There is no per-frame blend between the
two paths: blending the classical output back in re-adds the noise the network removed.
What shipped is a switch and a **Keep grain** slider, not the Amount described first. The slider
blends the two demosaics per pixel — but only the *brightness* of their difference: `out =
denoised + grain · ΔY / wb`, with `ΔY` the luminance of `wb · (classical − denoised)`. Taken after
the as-shot balance and handed back divided by it, the grain is neutral in the finished picture.
The objection that stood here — that blending the classical output back in re-adds the noise —
holds for a plain mix, which also brings back the classical path's colour speckle and false
colour. A luminance-only blend returns film-like grain and nothing else, and it needs no
inference: one elementwise GPU pass (`dr_gpu::GrainBlend`) per slider value, producing a new
source the adjust pass draws. Comparing the two on real 6D frames, the user chose this one.
The σ-map Amount (§3.3) still works — `NoiseModel::scaled` — and stays available for a later
"strength" control; its cost is a re-run of the network.
### 7.3 Runtime
Through `dr-inference-engine`, as the other models run ([inference.md](inference.md)): TensorRT or
CUDA fp16 on the laptop, MIGraphX on the desktop, ORT CPU everywhere, QNN on the tablet. Work is
Through `dr-inference-engine`, as the other models run ([inference.md](inference.md)), as
`Role::Denoiser`: TensorRT or CUDA fp16 on the laptop, MIGraphX on the desktop, ORT CPU everywhere.
**Not the Hexagon** — see §11 — so the tablet runs it on its CPU. Work is
scheduled in the `Background` class so a slider never waits on it (architecture §5.3).
## 8. Speed and the tablet
@@ -356,3 +367,57 @@ MIT architecture, so this model adds no third-party licence to D13.
4. Whether a Lightroom or DxO comparison is available for §6.2.
5. A borrowed X-Trans body, or X-Trans experimental in v1.
## 11. What shipped in 0.21.0, and what was measured
**Data.** 500 distinct ISO 50–100 6D frames from the library, over 121 shooting days (bursts and
near-duplicate perceptual hashes dropped; 55 frames from held-out days for validation). Read through
`dr-gpu`'s `mosaic_dump` example — `dr-decode` and the app's own hot-pixel pass — so the network's
input is the mosaic the classical demosaic reads. Truth by 2×2 binning with a Catmull-Rom quarter-pixel
shift of red and blue (§4.2). Training lives in `darkroom-denoise`, beside `darkroom-infill`.
**Noise model (§5), from the library instead of a capture.** Shot gain and read variance per ISO from
Adobe's `NoiseProfile` in the converted DNGs; read noise checked against each frame's masked border
(agreement within 2–3 % from ISO 125 to 25600); read-noise *shape* taken from the border as quantiles
on a tail-dense grid (excess kurtosis up to ~10 at high ISO), with only the photosites the app's
hot-pixel rule would remove left out; row noise from the border's row means; **column noise** from
the masked rows above the image — about a third of its variance is this sensor's fixed pattern.
Third stops are their own rows: ISO 160, 320 and 640 are quieter than their neighbours, as §5.1
expected. Training without the column noise left the 6D's vertical stripes in (0.90 DN of 1.01);
with it, 0.17 DN.
**Model.** Not NAFNet: its channel attention averages over the whole input, which breaks exact
tiling. A U-Net of 3×3 convolutions, ReLU, strided and transposed convolutions and additive skips —
3.2 M parameters, 48 GMAC per raw megapixel, receptive field 185 photosites (counted from the
layers; a perturbation probe under-read it as 157 because a switched-off ReLU hides a path).
Tiles of 1408 keep their central 1024 behind a 192 halo, exactly.
**Results.** PSNR after the display transform, held-out days, step 60 000:
| ISO | Network | Bilinear | Bilinear on a clean mosaic |
|---|---|---|---|
| 400 | 41.6 | 36.8 | 40.0 |
| 1600 | 40.8 | 33.8 | 40.0 |
| 6400 | 39.5 | 29.3 | 40.0 |
| 25600 | 37.8 | 24.6 | 40.0 |
Unbiased in linear light on real frames (shadow level within 1 % of a heavily averaged bilinear).
Checked against the app's own render for channel and axis order (`tools/check_against_app.py`).
**Precision (§8).** fp16: 0.00 dB at every ISO. int8 QDQ, calibrated on training tiles: −6 to −9 dB
— the shadow steps §8 feared losing are lost. So the Hexagon refuses the role and the tablet runs f32
on its CPU; the residual head of §8 is the route back.
**Noise for any Bayer body (§3.3).** Table, then `NoiseProfile`, then the frame itself: read, row and
column noise from its masked border, the shot gain alone estimated from the quietest flat patches.
On 130 6D frames the estimate is within ±10 % of the table from ISO 1000 up and scattered below. The
network loses under 0.3 dB for σ off by 15–20 % and twice as much for under- as for over-estimating;
the estimate leans high. Every Bayer body is offered the switch; develop says which source was used.
**Speed, a whole 6D frame (20 MP).** TensorRT fp16 3.1 s, ONNX Runtime CPU 14.4 s, on the laptop —
measured while the GPU sat power-capped at an 810 MHz memory clock; uncapped is expected to be
about four times faster. The Rust path reproduces the training repository's output to 2.5e-4 at
worst; TensorRT fp16 is 75 dB from f32.
**Not yet:** the result is not cached across sessions (§7.1) — reopening recomputes; the tripod real
pairs of §6.1; X-Trans (§9); the hand-written WGSL path, for which `export.py` already writes the
weights blob and a manifest a shader can follow.
docs/dev/inference.md
+15 -3
View File
@@ -151,10 +151,14 @@ winning:
| Linux / Windows, NVIDIA GPU | TensorRT, f32 model, fp16 engine | CUDA provider, f32 | ORT CPU, f32 | tract |
| Linux, AMD GPU with ROCm | MIGraphX, f32 model, fp16 program | ORT CPU, f32 | — | tract |
| Linux / Windows, no GPU stack | ORT CPU, f32 | — | — | tract |
| macOS ⁵ | ORT CPU, f32 | — | — | tract |
| macOS ⁵ | CoreML, f32 model, ML Program | ORT CPU, f32 | — | tract |
⁵ CoreML is the obvious rung and is unmeasured; it is listed so its absence is a gap and not an
oversight.
⁵ **Unmeasured**, and the one exception to the rule below: nobody here has a Mac. The rung is on
the ladder because the probe makes a wrong guess cheap — a CoreML that is slower than the CPU is
rejected by §4's clock, one that errors is recorded as failed, and one that takes the process
down is refused on the third launch (§4, `attempt`). The embedder stays on the CPU (§7). The first
macOS log that shows a probe line is this row's measurement; [macos.md](macos.md) says what to
ask for.
Deliberately **not** on any ladder, with the measurement that excluded each: NNAPI (no driver),
XNNPACK (slower than CPU, aborts on SCRFD), WebGPU (slower than CPU), the Adreno through QNN (works,
@@ -270,6 +274,14 @@ What the probe may not do:
- **Retry a rung that failed within a session.** A failed probe is cached as a failure with the
same inputs; the rung is tried again when an input changes. Otherwise a wedged driver means a
thirty-second stall on every launch.
- **Crash the app twice for the same reason.** The probe runs in the app's process, and a provider
can fail by aborting rather than by returning an error (XNNPACK on SCRFD, §2). Every session build
on a rung above the CPU — the probe's, and each background compile of §6 — writes what it is
attempting to `attempt` in the cache directory first and removes it after. A launch that finds the
file knows the last one died inside that attempt; after two such launches in a row the attempt is
refused and recorded like any other failure (a rung in `failed`, an engine in `refused`), until
the fingerprint changes. Two, not one, because quitting during a forty-second TensorRT compile
leaves the same file.
- **Choose for the user without saying so.** Settings gains one row, *Inference backend*, showing
what was chosen and why in one line ("Hexagon NPU · int8 · QNN 2.42"; "CPU · ONNX Runtime 1.30 ·
TensorRT probe failed: cuDNN 8 required"), with an override to force any lower rung. The about
docs/dev/macos.md
+86
View File
@@ -0,0 +1,86 @@
# macOS
macOS is out of scope for v1 ([requirements.md](requirements.md)), and nobody working on
DarkRoom has a Mac. This page records what exists anyway, and how a macOS build is set up so
that someone who does have one can send back enough to fix what they hit.
## 1. What exists
- **Inference** ([inference.md §2](inference.md)). The macOS ladder is CoreML, then ONNX
Runtime's CPU provider, then tract. CoreML is unmeasured. The probe decides whether it is used,
and the crash guard (§4, `attempt`) covers the case where the provider takes the process down.
The device fingerprint is the chip (`machdep.cpu.brand_string`) and the OS release, because
CoreML ships with the OS.
- **Where files go** ([`dr_plat::dirs`](../../platform/dr-plat/src/dirs.rs)). The Unix rules,
except the state directory (the log and crash records), which is `~/Library/Logs/darkroom`.
- **A diagnostic build**, described in §3.
The rest is not built, packaged or run on macOS by anyone here. This covers the window,
Metal through wgpu, the display profile (FR-DSP-8 asks X11 and Wayland), the keyring, the
bundle, and signing. `dr-plat` sends every non-Android Unix to the X11/Wayland dependencies.
## 2. Building
`docker/macos` compiles and links for macOS from Linux, using zig as the linker
(`cargo-zigbuild`). Zig carries libSystem's stubs and the C headers, so tract's SIMD kernels
compile and anything that needs only libSystem links:
./docker/macos/build.sh cargo zigbuild --target aarch64-apple-darwin -p dr-inference-engine --features native --all-targets
./docker/macos/build.sh cargo-zigbuild clippy --target aarch64-apple-darwin -p dr-inference-engine --features native --all-targets -- -D warnings
That produces Mach-O arm64 test binaries and the `ladder` and `ep_probe` examples. Nothing runs
them. Anything that links an Apple framework needs the Xcode SDK, which zig does not carry. That
includes `dr-plat` (through the keyring's Security and CoreFoundation) and so the desktop app, and
its link fails with `unable to find framework`. `cargo check` for those still works in the
container.
Linking the app needs Apple's SDK, which means a Mac. On one:
cargo build --profile diagnostic -p darkroom-desktop
./tools/fetch-desktop-runtime.sh # ONNX Runtime 1.29.0 with CoreML, Apple silicon only
The fetch script puts `libonnxruntime.dylib` in the user's `runtime/` directory, next to the
models. The app also looks in `Contents/Frameworks` of its own bundle, and in Homebrew's
`/opt/homebrew/lib` and `/usr/local/lib`. Homebrew's build may not include CoreML; the probe
reports that as a failed rung and uses the CPU.
**For whoever packages it.** A notarised app runs with the hardened runtime, whose library
validation refuses to `dlopen` a library signed by another team. A bundled
`Contents/Frameworks/libonnxruntime.dylib` must be signed with the app. A runtime the user
fetched needs the `com.apple.security.cs.disable-library-validation` entitlement, or it will not
load, and the app will be the tract build without saying why beyond one log line.
## 3. The diagnostic build
Every macOS build is in the hands of someone who can send a log but cannot attach a debugger,
so it is set up to log like a debug build while running at release speed.
- **The log says more.** With no `RUST_LOG`, the desktop's default filter is `debug` for every
`dr_*` crate, for `darkroom_desktop`, and for `onnxruntime`. That last one is ONNX Runtime's
own session log, which the engine forwards into `log` on every platform (`session.rs`,
`with_runtime_log`). At `debug` it includes how many nodes each provider took. At `trace`
(`RUST_LOG=onnxruntime=trace`) it lists every node's placement, which is long. The log cap is
the same as everywhere (two files of 4 MiB).
- **Backtraces have line numbers.** `--profile diagnostic` is release plus line tables. On
macOS the tables go into a `.dSYM` beside the executable, and the backtrace in a crash record
finds them only if the `.dSYM` stays next to the binary. Keep it in the bundle.
## 4. What to ask a Mac user for
`~/Library/Logs/darkroom/darkroom.log`, plus `darkroom.log.1` if present, after the first launch
and after the first scan with faces. Console.app lists it under *Log Reports*. The Settings
diagnostics bundle collects the same files. The lines that answer the open questions are:
| Line | What it tells us |
|---|---|
| `inference: ONNX Runtime … from …` / `inference: runtime tract` | Whether a runtime was found, and which one |
| `inference: floor … ms on the CPU provider` | The CPU number for §2's table |
| `inference: CoreML session built in … s` | CoreML's first compile of the probe model |
| `inference: CoreML rejected: …` / `failed: …` | Why the CPU was kept |
| `onnxruntime` lines naming `CoreMLExecutionProvider::GetCapability` | How much of the graph CoreML took |
| `inference: the app died during …` | The crash guard fired, and on what |
| `inference: compiling … for CoreML` / `ready on CoreML in … s` | Each model's compile, and any that CoreML refused |
Also ask for the settings row (*Settings › About › Inference*), which is one line and says the
same in short. When one of these logs comes back with CoreML numbers, they go into
[inference.md §1–2](inference.md), and footnote ⁵ becomes a measurement.
docs/dev/requirements.md
+52
View File
@@ -2437,6 +2437,7 @@ Rationale, evidence, and the eliminated alternatives are recorded in
| D12 | Scope versus pace | **DECIDED 2026-09-19** — settled by events; full scope stands, no v1 date |
| D18 | Derived images | **DECIDED 2026-09-19** — a merge writes a new source file; no multi-source Version |
| D19 | Scene-referred pipeline | **DECIDED 2026-09-27** — edits on unbounded scene-linear colour; one view transform, last; per-body base curves retired |
| D21 | DNG reference tone for raws | **DECIDED 2026-10-03** — the view transform's DNG reference curve (profile's, else ACR3 default, via RGBTone in ProPhoto) is the default for every raw, at contrast 1.5 (a ×1.07 power about grey); measured against Lightroom exports of photographs with neutral look settings; the sigmoid stays a choice |
| D20 | DCP camera profiles | **DECIDED 2026-10-02** — HueSatMap and LookTable as a scene operation after exposure; embedded profile first, then a matched `.dcp`; tone curve not applied; none shipped |
### D11 — product positioning
@@ -2772,6 +2773,57 @@ matrix does.
*What it costs.* Every DNG with an embedded profile renders differently; previews refresh only when
rendered again; tablet and desktop release together. The profiles directory does not sync yet.
### D21 — DNG reference tone for raws · **DECIDED 2026-10-03**
*Decided by measurement, later the same day.* The library's photo gallery holds Lightroom 6
exports of raws that are in the library, each carrying its Camera Raw settings. Clustered by
those settings, 663 exports had none of the house look (Linear curve, no HSL, no parametric
curve, no split toning); 60 of them with their raws, two thirds fitted and one third held out,
rendered by DarkRoom against Lightroom's JPEG (MSE, sRGB 8-bit):
| Rendering | Held-out MSE |
|---|---|
| 0.20.0's sigmoid at its defaults | ~1200 — about 0.7 EV darker, and flatter |
| Sigmoid, exposure, contrast and white fitted | ~150 (contrast 1.73, +0.73 EV) |
| DNG reference curve after baseline exposure, at contrast 1.4 | 224 |
| DNG reference curve, contrast 1.5 | ~150 |
| DNG reference curve, exposure, contrast and white fitted | 143 |
So the DNG reference curve is the default for every raw, and the default contrast is 1.5 — under that
curve a power of 1.5/1.4 about grey (`REFERENCE_CONTRAST` is where the curve is untouched). The
brightness needs nothing: baseline exposure and the curve together land where the earlier exports do. The
profile's look strength, vibrance and saturation bought nothing measurable on those exports. The
user chose to change every photograph rather than keep edited ones on the old rendering. The
fitting tools live outside the repository (`darkroom-lrfit`).
*Amended earlier the same day:* the default was **not** decided. The measurement below was against
Lightroom previews of photographs carrying the user's Lightroom edits — a house look of HSL
saturation (Blue +58, Aqua +50, …), Highlights −40 and Blacks −20 in every DNG's XMP — so it said
nothing about Camera Raw's base rendering. Under the DNG reference curve `_MG_9080` renders brighter
than its Lightroom preview (mean 0.39 against 0.31). The sigmoid stays the default; the curve
below is a choice; the default is decided by measurement against Lightroom exports of unedited
photographs (the `lr-fit` work). What follows is the original text.
**The view transform has two curves, and Camera Raw's is the default for every raw.** It is the
profile's `ProfileToneCurve`, or the ACR3 default curve where the profile has none or there is no
profile, applied Camera Raw's way — on the largest and smallest channel in linear ProPhoto, the
middle placed proportionally — after `BaselineExposure`. D19's sigmoid stays as the other choice.
Design: [camera-profiles.md](camera-profiles.md) §11–§13.
*Why.* Measured on the library's 6D DNGs after D20 (camera-profiles.md §1): the profile tables
lowered saturation, because Adobe's look tables were tuned to sit under this curve. The user's
complaint was that Lightroom's rendering is more colourful, and this curve is most of the reason.
Chosen by the user over limiting it to raws with a profile, or making it opt-in.
*What it reverses in D19.* D19 rejected per-body curves as defaults because their provenance was
unknown. A DCP's curve and the ACR3 table have known provenance — Adobe's, published — and so the
objection that retired the base curves does not apply. D19's other half stands: nothing before
the view transform clamps, and the curve is the view transform, last.
*What it costs.* Every raw renders differently again, and highlights above display white clip
where the sigmoid rolled them off; Sigmoid is one click away. Previews refresh only when rendered
again; tablet and desktop release together.
### D16 — plugin licensing · **OPEN, post-v1**
> Deferred with §3.10 on 2026-09-19. Still to be answered before the format is published as
docs/dev/traceability.md
+96 -96
View File
File diff suppressed because one or more lines are too long
docs/manual/README.md
+29
View File
@@ -205,6 +205,35 @@ the sensor recorded.
![Zooming to 1:1 with a double-click, panning, then further in with the wheel](media/develop-zoom.gif)
### AI denoise
For a photograph taken in poor light at a high ISO. `AI Denoise`, in the
Detail group, replaces how the camera's raw data is turned into colour: a
network trained on this library's own photographs removes the noise and
the blotches of colour that come with it, while keeping the fine detail.
Look at it at 1:1, where noise lives.
Switch it on with `Apply`. The photograph keeps showing as it was while
the network works, with its progress in the bar at the top, and changes
when it is done — a few seconds on a computer with a graphics card, about
fifteen on its processor alone, longer on the tablet. `Keep grain` puts
back some of what was removed, as grain without colour, for a picture that
does not look too smooth.
![An ISO 8000 night frame at 1:1, AI Denoise switched on, then some grain kept](media/develop-denoise.gif)
| Before | After |
|---|---|
| ![The railing and the lamp at ISO 8000, as the camera recorded them](media/develop-denoise-before.png) | ![The same, with AI Denoise](media/develop-denoise-after.png) |
It works on raw files from any camera with the usual colour pattern of
red, green and blue squares — not on JPEGs, and not yet on Fujifilm's
X-Trans. How noisy the camera is at each ISO was measured for the Canon
EOS 6D; for other cameras it is read from a DNG's own figures or
estimated from the photograph, and the finished job in the activity list
says which. An export uses
it whenever the photograph has it switched on.
### Moving between photographs
The roll along the foot of the canvas holds the photographs the grid was
docs/manual/index.html
+24
View File
@@ -117,6 +117,7 @@ th { color: var(--ink-dim); font-weight: 600; }
<ul>
<li><a href="#light">Light</a></li>
<li><a href="#looking-closer">Looking closer</a></li>
<li><a href="#ai-denoise">AI denoise</a></li>
<li><a href="#moving-between-photographs">Moving between photographs</a></li>
<li><a href="#white-balance-from-the-photograph">White balance from the photograph</a></li>
<li><a href="#composing">Composing</a></li>
@@ -287,6 +288,29 @@ wheel zooms to any amount in between. Past 1:1 the file's own pixels are
drawn as hard-edged blocks rather than smoothed, so what you see is what
the sensor recorded.</p>
<figure><img loading="lazy" src="media/develop-zoom.gif" alt="Zooming to 1:1 with a double-click, panning, then further in with the wheel"><figcaption>Zooming to 1:1 with a double-click, panning, then further in with the wheel</figcaption></figure>
<h3 id="ai-denoise">AI denoise</h3>
<p>For a photograph taken in poor light at a high ISO. <code>AI Denoise</code>, in the
Detail group, replaces how the camera's raw data is turned into colour: a
network trained on this library's own photographs removes the noise and
the blotches of colour that come with it, while keeping the fine detail.
Look at it at 1:1, where noise lives.</p>
<p>Switch it on with <code>Apply</code>. The photograph keeps showing as it was while
the network works, with its progress in the bar at the top, and changes
when it is done — a few seconds on a computer with a graphics card, about
fifteen on its processor alone, longer on the tablet. <code>Keep grain</code> puts
back some of what was removed, as grain without colour, for a picture that
does not look too smooth.</p>
<figure><img loading="lazy" src="media/develop-denoise.gif" alt="An ISO 8000 night frame at 1:1, AI Denoise switched on, then some grain kept"><figcaption>An ISO 8000 night frame at 1:1, AI Denoise switched on, then some grain kept</figcaption></figure>
<table><thead><tr><th>Before</th><th>After</th></tr></thead><tbody>
<tr><td><img src="media/develop-denoise-before.png" alt="The railing and the lamp at ISO 8000, as the camera recorded them" /></td><td><img src="media/develop-denoise-after.png" alt="The same, with AI Denoise" /></td></tr>
</tbody></table>
<p>It works on raw files from any camera with the usual colour pattern of
red, green and blue squares — not on JPEGs, and not yet on Fujifilm's
X-Trans. How noisy the camera is at each ISO was measured for the Canon
EOS 6D; for other cameras it is read from a DNG's own figures or
estimated from the photograph, and the finished job in the activity list
says which. An export uses
it whenever the photograph has it switched on.</p>
<h3 id="moving-between-photographs">Moving between photographs</h3>
<p>The roll along the foot of the canvas holds the photographs the grid was
showing; click one to open it. The right arrow, <code>D</code> or space opens the next,
docs/manual/media/develop-denoise-after.png
LFS
BIN
View File
Binary file not shown.
docs/manual/media/develop-denoise-before.png
LFS
BIN
View File
Binary file not shown.
docs/manual/media/develop-denoise.gif
LFS
BIN
View File
Binary file not shown.
models/LICENCE.md
+13
View File
@@ -122,3 +122,16 @@ position of any model here. The training set is Places2, a research dataset,
but the weights are released under the repository's licence without a
data-derived restriction (contrast the gaze models §7 of the requirements
declined, and the InsightFace grant of D13).
## `denoise/` — the mosaic denoiser, the project's own
| File | Source | Trained on | Used by |
|---|---|---|---|
| `denoise/mosaic-1408.onnx` | trained from scratch in the `darkroom-denoise` repository (2026-10-03, run `m2`, 60 000 steps) | 427 of the maintainer's own base-ISO Canon EOS 6D raws, with the 6D's measured noise added | the learned demosaic and denoise (FR-DEV-3g) |
A U-Net of plain 3×3 convolutions, ReLU, strided and transposed
convolutions and additive skips — no third-party architecture code or
weights — at a fixed `1×1×1408×1408` for `mosaic` and `sigma`, exported by
`python -m denoise.export` in `darkroom-denoise`. Trained only on
photographs the maintainer owns, so the weights carry no grant but the
project's own: GPL-3.0-or-later, like the code (denoise.md §10).
models/denoise/mosaic-1408.onnx
LFS
BIN
View File
Binary file not shown.
packaging/PKGBUILD
+10 -1
View File
@@ -4,7 +4,7 @@
# makes `makepkg -si` in this directory install what you are actually working
# on. Swap `source` for a tagged tarball when there is something to release.
pkgname=darkroom
pkgver=0.20.0
pkgver=0.21.0
# Back to 1 with the version: a new pkgver is a new archive name, so there is
# nothing for makepkg to reuse and nothing for a release number to disambiguate.
pkgrel=1
@@ -123,4 +123,13 @@ package() {
return 1
fi
install -Dm644 "${_src}" "${pkgdir}/usr/share/darkroom/models/migan-512.onnx"
# The learned demosaic and denoise (the project's own weights, GPL —
# models/LICENCE.md). Same pointer check, same directory.
_src="models/denoise/mosaic-1408.onnx"
if [[ "$(stat -c%s "${_src}")" -lt 100000 ]]; then
echo "error: the denoise model is an LFS pointer — run: git lfs pull" >&2
return 1
fi
install -Dm644 "${_src}" "${pkgdir}/usr/share/darkroom/models/mosaic-1408.onnx"
}
platform/dr-plat/src/crash.rs
+2 -1
View File
@@ -98,7 +98,8 @@ pub fn set_state_dir(dir: PathBuf) {
/// Where this application keeps state that is neither configuration nor cache.
///
/// `$XDG_STATE_HOME/darkroom`, falling back to `~/.local/state/darkroom`.
/// `$XDG_STATE_HOME/darkroom`, falling back to `~/.local/state/darkroom` —
/// `~/Library/Logs/darkroom` on macOS (`dirs`).
/// State rather than cache because a crash record must survive the sweep that
/// a cache directory exists to permit, and rather than config because it is
/// not something the user edits.
platform/dr-plat/src/diagnostics.rs
+2
View File
@@ -46,6 +46,8 @@
//!
//! * Linux: `$XDG_STATE_HOME/darkroom/darkroom.log`, else
//! `~/.local/state/darkroom/darkroom.log`.
//! * macOS: `$XDG_STATE_HOME/darkroom/darkroom.log`, else
//! `~/Library/Logs/darkroom/darkroom.log`, where Console.app lists it.
//! * Android: `/sdcard/Android/data/paris.tourolle.darkroom/files/darkroom.log`,
//! which `adb pull` reads from an ordinary release build. See
//! [`crate::state`] for why not the internal directory, and
platform/dr-plat/src/dirs.rs
+15 -3
View File
@@ -17,6 +17,11 @@
//! | Unix default | `~/.config` | `~/.local/share` | `~/.local/state` |
//! | Windows | `%APPDATA%` | `%LOCALAPPDATA%` | `%LOCALAPPDATA%`, then `state` |
//! | Windows default | `%USERPROFILE%\AppData\Roaming` | `…\AppData\Local` | `…\AppData\Local` |
//! | macOS default | `~/.config` | `~/.local/share` | `~/Library/Logs` |
//!
//! macOS follows the Unix rules except for the one directory a user is asked
//! to find by hand: the log. Finder hides `~/.local`, and `~/Library/Logs`
//! is where Console.app and a Mac user already look (docs/dev/macos.md).
//!
//! then `darkroom` under each. Config roams on Windows and the rest does not,
//! which is the same split XDG makes between config and everything else, and
@@ -100,11 +105,18 @@ fn resolve(kind: Base, env: impl Fn(&str) -> Option<OsString>) -> PathBuf {
}
}
/// Where state goes under `$HOME` when `XDG_STATE_HOME` does not say.
const STATE_UNDER_HOME: &str = if cfg!(target_os = "macos") {
"Library/Logs"
} else {
".local/state"
};
fn xdg_base(kind: Base, env: &impl Fn(&str) -> Option<OsString>) -> Option<PathBuf> {
let (var, under_home) = match kind {
Base::Config => ("XDG_CONFIG_HOME", ".config"),
Base::Data => ("XDG_DATA_HOME", ".local/share"),
Base::State => ("XDG_STATE_HOME", ".local/state"),
Base::State => ("XDG_STATE_HOME", STATE_UNDER_HOME),
};
absolute(env(var)).or_else(|| absolute(env("HOME")).map(|h| h.join(under_home)))
}
@@ -150,7 +162,7 @@ mod tests {
);
assert_eq!(
xdg_base(Base::State, &e),
Some(PathBuf::from("/home/someone/.local/state"))
Some(PathBuf::from("/home/someone").join(STATE_UNDER_HOME))
);
}
@@ -162,7 +174,7 @@ mod tests {
let e = env(&[("XDG_STATE_HOME", "state"), ("HOME", "/home/someone")]);
assert_eq!(
xdg_base(Base::State, &e),
Some(PathBuf::from("/home/someone/.local/state"))
Some(PathBuf::from("/home/someone").join(STATE_UNDER_HOME))
);
assert_eq!(xdg_base(Base::State, &env(&[("HOME", "")])), None);
}
tools/fetch-desktop-runtime.sh
+26
View File
@@ -20,6 +20,32 @@
# directory (docs/inference.md §1.3).
set -euo pipefail
DEST="${1:-${XDG_DATA_HOME:-${HOME}/.local/share}/darkroom/runtime}"
# macOS: Microsoft's release archive, which carries the CoreML provider in the
# one library. Pinned, because the CoreML options the engine sets were read
# from this version's source (docs/dev/macos.md, CLAUDE.md "Providers").
# Apple silicon only: no Intel archive is published since 1.29; an Intel Mac
# takes Homebrew's `onnxruntime` or stays on tract.
if [[ "$(uname -s)" == Darwin ]]; then
ORT_VERSION=1.29.0
[[ "$(uname -m)" == arm64 ]] || {
echo "error: no ONNX Runtime ${ORT_VERSION} archive for $(uname -m); try: brew install onnxruntime" >&2
exit 1
}
NAME="onnxruntime-osx-arm64-${ORT_VERSION}"
WORK="$(mktemp -d)"
trap 'rm -rf "${WORK}"' EXIT
echo "==> downloading ${NAME}"
curl -fsSL "https://github.com/microsoft/onnxruntime/releases/download/v${ORT_VERSION}/${NAME}.tgz" \
| tar xz -C "${WORK}"
mkdir -p "${DEST}"
cp "${WORK}/${NAME}/lib/libonnxruntime.dylib" "${WORK}/${NAME}/LICENSE" "${DEST}/"
echo "==> runtime in ${DEST}:"
ls -1 "${DEST}" | sed 's/^/ /'
echo " (the app finds it on its next launch; Settings › About › Inference says what it chose)"
exit 0
fi
WORK="$(mktemp -d -p /var/tmp fetch-desktop-runtime.XXXXXX)"
trap 'rm -rf "${WORK}"' EXIT
tools/manual/scenes.py
+63
View File
@@ -63,6 +63,7 @@ ALPS_SKY = '_MG_8330' # an upright alpine frame, its top third sky and clo
TOWER = '_MG_8693' # towers shot looking up: verticals that converge
ROAD = '_MG_8672' # a road under a sky, with marks on it: masks and repair
ROAD_MARK = (0.305, 0.794) # a dark mark on its tarmac
DENOISE = '_MG_8862' # Brooklyn Bridge at ISO 8000: a lit railing, a lamp and a camera against the night
# --- the registry -----------------------------------------------------------
SCENES = {}
@@ -841,6 +842,68 @@ def develop_zoom():
pause(1.2)
@scene(media=['develop-denoise.gif', 'develop-denoise-before.png', 'develop-denoise-after.png'],
sources=DEVELOP_SRC + ['ui/dr-ui/src/develop/denoise.rs', 'core/dr-denoise/**',
'core/dr-gpu/src/grain.rs', 'models/denoise/**'])
def develop_denoise():
"""A night frame at ISO 8000 at 1:1, AI Denoise switched on and landed,
then some grain kept. Waits for the network rather than for a fixed
time: on the CPU it takes several times what it does on a GPU."""
at_develop(DENOISE)
a = dr.photo(0.45, 0.55) # the lit lamp, the railing and the skyline over the water
dr.move(*a)
pause(0.3)
dr.x('click', '--repeat', 2, '--delay', 80, 1)
pause(1.5)
group('Detail')
in_column('AI Denoise@Text')
shot('develop-denoise-before')
rec('develop-denoise')
pause(0.8)
mark = log_size()
dr.click(*denoise_switch())
t0 = time.time()
while time.time() - t0 < 300 and not denoise_landed(mark):
pause(0.5)
pause(1.5)
shot('develop-denoise-after')
slide('Keep grain', 60)
pause(2.0)
cut()
undo_all()
dr.move(*a)
dr.x('click', '--repeat', 2, '--delay', 80, 1)
pause(1.2)
def denoise_switch():
"""The `Apply` box under the AI Denoise heading — the lens profile's
switch is also called Apply, so it is found by where it sits."""
head = dr.matches('AI Denoise@Text', within=column())[0]
below = [e for e in dr.matches('Apply@CheckBox', within=column()) if e['y'] > head['y']]
e = min(below, key=lambda e: e['y'])
return int(e['x'] + e['w'] / 2), int(e['y'] + e['h'] / 2)
def log_size():
try:
return os.path.getsize(f'{dr.HOME}/app.log')
except OSError:
return 0
def denoise_landed(since):
"""The app has logged the result landing since `since` bytes into its
log. The activity bar's progress has no name to wait on, and the time
it takes is the device's — seconds on a GPU, more on the CPU."""
try:
with open(f'{dr.HOME}/app.log', 'rb') as f:
f.seek(since)
return b'learned denoise:' in f.read()
except OSError:
return False
@scene(media=['develop-wb.gif'], sources=DEVELOP_SRC + ['ui/dr-ui/src/develop/white_balance.rs'])
def develop_wb():
at_develop()
ui/dr-ui/Cargo.toml
+2
View File
@@ -36,6 +36,8 @@ dr-export.workspace = true
# The panorama's geometry and its keypoint detector (§3.11). The detector's
# runtime is the same tract the faces and masks already carry.
dr-pano = { workspace = true, features = ["xfeat", "embedded-model"] }
# The learned demosaic and denoise (FR-DEV-3g), under the same engine.
dr-denoise = { workspace = true, features = ["native"] }
dr-ingest.workspace = true
# The sameness probe of a catalog duplicate (FR-CAT-11a): SHA-256 over the
# ends of each copy, the digest the import already uses for whole files.
ui/dr-ui/src/activity.rs
+5
View File
@@ -83,6 +83,9 @@ pub enum Kind {
/// TRACES: FR-EXP-7
/// Rendering and writing finished files.
Export,
/// TRACES: FR-DEV-3g
/// The learned demosaic and denoise working through a photograph.
Denoise,
}
impl Kind {
@@ -111,6 +114,8 @@ impl Kind {
// background sweep in the same sentence as a download they are
// waiting on.
Kind::Index => false,
// Local work on a file already open.
Kind::Denoise => false,
}
}
}
ui/dr-ui/src/decoder_seam.rs
+2 -1
View File
@@ -130,6 +130,7 @@ impl Decoder for Stub {
samples_per_pixel: 1,
profile: None,
profile_tables: None,
baseline_exposure: 0.0,
make: "Stubco".into(),
model: "Stubco One".into(),
})
@@ -198,7 +199,7 @@ fn the_catalog_scan_reads_headers_through_the_trait() {
&mut found,
));
assert!(reached);
assert!(matches!(reached, crate::library::DateRead::Reached));
assert_eq!(found.len(), 1, "the stub's header was read");
assert_eq!(found[0].image_id, 7);
assert_eq!(found[0].camera.as_deref(), Some("Stubco One"));
ui/dr-ui/src/derived_sync.rs
+200
View File
@@ -91,6 +91,11 @@ pub struct SyncReport {
/// is a fact only the other device knew.
pub place_adopted: bool,
pub place_uploaded: bool,
/// TRACES: FR-DEV-3e
/// Camera profiles exchanged with the library's `profiles` folder
/// (camera-profiles.md §13).
pub profiles_uploaded: usize,
pub profiles_downloaded: usize,
}
impl SyncReport {
@@ -103,6 +108,7 @@ impl SyncReport {
|| self.face_shards_uploaded > 0
|| self.face_shards_downloaded > 0
|| self.place_adopted
|| self.profiles_downloaded > 0
}
}
@@ -224,6 +230,16 @@ async fn run(
let _ = tx.send(SyncMessage::Status("checking thumbnails…".into()));
sync_shards(backend, &base, thumbs_dir, scratch, &mut report).await?;
// TRACES: FR-DEV-3e
// The camera profiles, so a profile copied out of a DNG on one device
// renders that body's raws on every device (camera-profiles.md §13).
// After the catalog and before the place; like the place it never fails
// the pass.
if let Some(dir) = dr_decode::dcp::profiles_directory() {
let _ = tx.send(SyncMessage::Status("checking camera profiles…".into()));
sync_profiles(backend, &base, &dir, &mut report).await;
}
// TRACES: FR-UI-8
// Last, and it costs one small GET plus at most one small PUT. Last because
// it is the only thing here that is not derived state and so the only thing
@@ -995,6 +1011,117 @@ async fn sync_place(
}
}
/// TRACES: FR-DEV-3e
/// Exchange camera profiles with `<derived>/profiles` (camera-profiles.md
/// §13).
///
/// Profiles are immutable and named for what they hold (`<camera>
/// <profile>.dcp`, `dcp::save`), so a name and a size say whether two copies
/// are the same: upload what the server lacks or holds at another size,
/// download what this device lacks. A download is parsed before it is kept,
/// and written beside its name then renamed, because the profile loader
/// reads every `.dcp` in the folder and a half-written one would be skipped
/// with a warning rather than retried. Never fails the pass: a profile that
/// did not travel this time travels next time.
async fn sync_profiles(
backend: &dyn RemoteBackend,
base: &RemotePath,
local_dir: &Path,
report: &mut SyncReport,
) {
let dir = RemotePath::new(format!("{}/profiles", base.as_str()));
let is_profile = |name: &str| {
Path::new(name)
.extension()
.is_some_and(|x| x.eq_ignore_ascii_case("dcp"))
};
let remote: std::collections::HashMap<String, u64> = backend
.list(&dir, None)
.await
.map(|entries| {
entries
.into_iter()
.filter(|e| e.kind == dr_sync::EntryKind::File && is_profile(e.path.name()))
.map(|e| (e.path.name().to_string(), e.size))
.collect()
})
// Absent until the first device uploads one; an empty answer.
.unwrap_or_default();
let local: std::collections::HashMap<String, PathBuf> = std::fs::read_dir(local_dir)
.map(|entries| {
entries
.flatten()
.filter_map(|e| {
let name = e.file_name().to_string_lossy().into_owned();
is_profile(&name).then(|| (name, e.path()))
})
.collect()
})
.unwrap_or_default();
// ---- upload ----------------------------------------------------------
let to_upload: Vec<(&String, &PathBuf)> = local
.iter()
.filter(|(name, path)| {
let size = std::fs::metadata(path).map(|m| m.len()).unwrap_or(0);
remote.get(*name) != Some(&size)
})
.collect();
if !to_upload.is_empty() {
let _ = backend.create_dir(&dir).await;
}
for (name, path) in to_upload {
let Ok(bytes) = std::fs::read(path) else {
continue;
};
let target = RemotePath::new(format!("{}/{name}", dir.as_str()));
match backend.put(&target, bytes, None).await {
Ok(_) => report.profiles_uploaded += 1,
Err(e) => log::debug!("uploading camera profile {name}: {e}"),
}
}
// ---- download --------------------------------------------------------
for name in remote.keys().filter(|n| !local.contains_key(*n)) {
let source = RemotePath::new(format!("{}/{name}", dir.as_str()));
let bytes = match read_derived(backend, &source).await {
Ok(b) => b,
Err(e) => {
log::debug!("fetching camera profile {name}: {e}");
continue;
}
};
if let Err(why) = dr_decode::dcp::Dcp::parse(&bytes) {
log::warn!("camera profile {name} on the server is not one: {why}");
continue;
}
let path = local_dir.join(name);
let partial = local_dir.join(format!("{name}.part"));
let written = std::fs::create_dir_all(local_dir)
.and_then(|()| std::fs::write(&partial, &bytes))
.and_then(|()| std::fs::rename(&partial, &path));
match written {
Ok(()) => report.profiles_downloaded += 1,
Err(e) => {
let _ = std::fs::remove_file(&partial);
log::warn!("saving camera profile {name}: {e}");
}
}
}
if report.profiles_downloaded > 0 {
log::info!(
"camera profiles: {} fetched from the library",
report.profiles_downloaded
);
// Decodes from now on see them; one already in flight keeps the set
// it started with.
dr_decode::dcp::set_profiles_directory(local_dir.to_path_buf());
}
}
/// TRACES: FR-UI-8
/// Fetch just the place, for the handover at launch.
///
@@ -1734,6 +1861,79 @@ mod derived_guard_tests {
(puts.load(Ordering::SeqCst), report, after, sent)
}
/// A small valid profile, as bytes, named for `model`.
fn profile_bytes(model: &str) -> Vec<u8> {
dr_decode::dcp::Dcp {
name: "Test Standard".into(),
unique_camera_model: Some(model.into()),
copyright: None,
calibration_signature: None,
embed_policy: 0,
illuminants: [Some(21), None],
color_matrix: [
Some([[1.0, 0.0, 0.0], [0.0, 1.0, 0.0], [0.0, 0.0, 1.0]]),
None,
],
forward_matrix: [None, None],
hue_sat: [None, None],
look: dr_types::HueSatTable::new(2, 2, 1, false, vec![[5.0, 1.1, 1.0]; 4]),
tone_curve: None,
baseline_exposure_offset: 0.0,
}
.to_bytes()
.unwrap()
}
#[tokio::test]
async fn camera_profiles_travel_both_ways_and_once() {
// TRACES: FR-DEV-3e
// camera-profiles.md §13: ours goes up, theirs comes down, and a
// second pass with nothing new moves nothing.
let root = std::env::temp_dir().join(format!("dr-profile-sync-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&root);
let server = root.join("server");
let device = root.join("device");
std::fs::create_dir_all(server.join(".darkroom-derived/profiles")).unwrap();
std::fs::create_dir_all(&device).unwrap();
std::fs::write(device.join("Ours A.dcp"), profile_bytes("Ours A")).unwrap();
std::fs::write(
server.join(".darkroom-derived/profiles/Theirs B.dcp"),
profile_bytes("Theirs B"),
)
.unwrap();
// Not a profile, and must be left alone in both directions.
std::fs::write(server.join(".darkroom-derived/profiles/notes.txt"), b"x").unwrap();
std::fs::write(
server.join(".darkroom-derived/profiles/Broken C.dcp"),
b"not a profile",
)
.unwrap();
let backend = dr_sync_folder::FolderBackend::new(&server).unwrap();
let base = RemotePath::new(".darkroom-derived");
let mut report = SyncReport::default();
sync_profiles(&backend, &base, &device, &mut report).await;
assert_eq!(report.profiles_uploaded, 1);
assert_eq!(report.profiles_downloaded, 1, "the broken one is refused");
assert!(server
.join(".darkroom-derived/profiles/Ours A.dcp")
.exists());
assert!(device.join("Theirs B.dcp").exists());
assert!(!device.join("Broken C.dcp").exists());
assert!(!device.join("notes.txt").exists());
assert!(
dr_decode::dcp::find(Some("Theirs B"), "", "").is_some(),
"a fetched profile is matched on the next decode"
);
let mut again = SyncReport::default();
sync_profiles(&backend, &base, &device, &mut again).await;
assert_eq!((again.profiles_uploaded, again.profiles_downloaded), (0, 0));
let _ = std::fs::remove_dir_all(&root);
}
#[tokio::test]
async fn a_place_that_could_not_be_read_is_never_written_over() {
// A dehydrated placeholder, and the record on the server may well be
ui/dr-ui/src/develop/denoise.rs
+431
View File
@@ -0,0 +1,431 @@
//! TRACES: FR-DEV-3g
//! The learned denoise in a develop session (docs/dev/denoise.md §7).
//!
//! The classical demosaic shows at once; when the photograph asks for the
//! learned one, it is computed off the UI thread from the mosaic the
//! session kept, and swapped in when it lands. Grain is a blend of the two
//! results, made once per slider value by [`dr_gpu::GrainBlend`] and handed
//! to the render as its source — the adjust pass never knows.
//!
//! `demosaiced` stays the classical result for the session's life: the raw
//! histogram, the white balance picker, masks and segmentation measure the
//! sensor data, and only [`DevelopSession::developed_source`] — what the
//! render draws — changes.
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::mpsc;
use std::sync::Arc;
use dr_decode::RawImage;
use dr_gpu::{DemosaicedImage, GrainBlend};
use super::session::DevelopSession;
/// What the session holds for the learned denoise.
#[derive(Default)]
pub(crate) struct DenoiseState {
/// The mosaic as decoded, kept only for a photograph that can take the
/// learned stage. The job repairs a copy.
mosaic: Option<Arc<RawImage>>,
/// The file's `NoiseProfile` and ISO, read from the header at open.
profile: Option<Vec<(f32, f32)>>,
iso: Option<u32>,
/// The network's result, once it has landed.
result: Option<Arc<DemosaicedImage>>,
/// The last grain blend made, and the grain it was made at.
blended: Option<(f32, Arc<DemosaicedImage>)>,
blend: Option<GrainBlend>,
job: Option<Job>,
/// Why the last attempt failed; not retried until the switch is
/// toggled, so a photograph that cannot be denoised does not loop.
failed: Option<String>,
/// Where the noise figures came from, for the panel.
source: Option<dr_denoise::Source>,
}
struct Job {
rx: mpsc::Receiver<Msg>,
cancel: Arc<AtomicBool>,
}
enum Msg {
Progress(usize, usize),
Done(Result<Finished, String>),
}
struct Finished {
rgb: Vec<f32>,
width: u32,
height: u32,
source: dr_denoise::Source,
rung: String,
seconds: f64,
}
/// What a poll found, for the develop view's status line and redraw.
#[derive(Debug, PartialEq)]
pub enum DenoiseStatus {
/// Nothing running and nothing new.
Idle,
/// Tiles done of tiles.
Running(usize, usize),
/// The result landed this poll: redraw.
Landed,
Failed(String),
}
impl DevelopSession {
/// Keep the mosaic for the learned denoise, if it can take this frame.
pub(super) fn keep_mosaic(&mut self, raw: RawImage) {
if dr_denoise::eligible(&raw) {
self.denoise.mosaic = Some(Arc::new(raw));
}
self.graph
.set_denoise_available(self.denoise.mosaic.is_some());
}
/// The header's part: the DNG's measured noise and the ISO.
pub fn prepare_denoise(&mut self, bytes: &[u8], meta: &dr_decode::Metadata) {
if self.denoise.mosaic.is_some() {
self.denoise.profile = dr_decode::noise_profile(bytes);
self.denoise.iso = meta.iso;
}
}
/// Bring what is computed in line with what the edit asks for: start the
/// network when it is wanted and has not run, stop it when it is not.
/// Cheap when nothing changed; the develop view calls it after every
/// change to the edit, whatever made it — a slider, undo, a version.
pub fn reconcile_denoise(&mut self) {
let wanted = self.graph.denoise_applied() && self.denoise.mosaic.is_some();
if !wanted {
if let Some(job) = self.denoise.job.take() {
job.cancel.store(true, Ordering::Relaxed);
}
// Toggling off is how a failure is retried.
self.denoise.failed = None;
return;
}
if self.denoise.result.is_some()
|| self.denoise.job.is_some()
|| self.denoise.failed.is_some()
{
return;
}
let Some(model) = crate::library::denoise_model() else {
self.denoise.failed = Some("the denoise model is not installed".into());
return;
};
let (tx, rx) = mpsc::channel();
let cancel = Arc::new(AtomicBool::new(false));
let work = Work {
ctx: self.ctx.clone(),
raw: self.denoise.mosaic.clone().expect("checked above"),
profile: self.denoise.profile.clone(),
iso: self.denoise.iso,
model,
cancel: cancel.clone(),
};
crate::executors::spawn(crate::executors::Executor::Decode, "denoise", move || {
let result = work.run(&mut |done, total| {
let _ = tx.send(Msg::Progress(done, total));
});
let _ = tx.send(Msg::Done(result));
});
self.denoise.job = Some(Job { rx, cancel });
}
/// Collect what the job sent since the last poll.
pub fn poll_denoise(&mut self) -> DenoiseStatus {
let Some(job) = &self.denoise.job else {
return DenoiseStatus::Idle;
};
let mut last = None;
let mut done = None;
while let Ok(msg) = job.rx.try_recv() {
match msg {
Msg::Progress(d, t) => last = Some((d, t)),
Msg::Done(r) => done = Some(r),
}
}
match done {
Some(Ok(f)) => {
self.denoise.job = None;
match self.land(f) {
Ok(()) => DenoiseStatus::Landed,
Err(e) => self.fail(e),
}
}
Some(Err(e)) => {
self.denoise.job = None;
self.fail(e)
}
None => last.map_or(DenoiseStatus::Running(0, 1), |(d, t)| {
DenoiseStatus::Running(d, t)
}),
}
}
/// Compute the result now, on this thread, if the edit wants it and it
/// is not here: for an export, which must not write the classical
/// picture of a photograph that asks for the learned one (§7.1).
pub fn denoise_blocking(&mut self) -> Result<(), String> {
if !self.graph.denoise_applied() || self.denoise.result.is_some() {
return Ok(());
}
let Some(raw) = self.denoise.mosaic.clone() else {
return Ok(());
};
// Already under way: wait for it rather than start again.
if let Some(job) = self.denoise.job.take() {
for msg in job.rx.iter() {
if let Msg::Done(result) = msg {
return result.and_then(|f| self.land(f));
}
}
}
let model = crate::library::denoise_model().ok_or("the denoise model is not installed")?;
let work = Work {
ctx: self.ctx.clone(),
raw,
profile: self.denoise.profile.clone(),
iso: self.denoise.iso,
model,
cancel: Arc::new(AtomicBool::new(false)),
};
let finished = work.run(&mut |_, _| {})?;
self.land(finished)
}
/// What the render draws: the learned result with the asked-for grain
/// where there is one, else the classical demosaic.
pub(super) fn developed_source(&mut self) -> Arc<DemosaicedImage> {
if !self.graph.denoise_applied() {
return self.demosaiced.clone();
}
let Some(result) = self.denoise.result.clone() else {
return self.demosaiced.clone();
};
let grain = self.graph.denoise_grain();
if grain <= 0.0 {
return result;
}
if let Some((g, image)) = &self.denoise.blended {
if (*g - grain).abs() < 1e-4 {
return image.clone();
}
}
let blend = self
.denoise
.blend
.get_or_insert_with(|| GrainBlend::new(&self.ctx));
match blend.blend(&result, &self.demosaiced, grain) {
Ok(image) => {
self.denoise.blended = Some((grain, image.clone()));
image
}
Err(e) => {
log::warn!("grain blend failed, showing the denoised result without grain: {e}");
result
}
}
}
/// For the panel: where the noise figures came from, once computed.
pub fn denoise_source(&self) -> Option<dr_denoise::Source> {
self.denoise.source
}
pub fn denoise_failure(&self) -> Option<&str> {
self.denoise.failed.as_deref()
}
fn land(&mut self, f: Finished) -> Result<(), String> {
let image =
DemosaicedImage::from_rgb_f32(&self.ctx, &self.demosaiced, f.width, f.height, &f.rgb)
.map_err(|e| e.to_string())?;
log::info!(
"learned denoise: {}×{} on {} in {:.1} s, noise {}",
f.width,
f.height,
f.rung,
f.seconds,
f.source.label()
);
self.denoise.result = Some(Arc::new(image));
self.denoise.blended = None;
self.denoise.source = Some(f.source);
Ok(())
}
fn fail(&mut self, e: String) -> DenoiseStatus {
log::warn!("learned denoise failed: {e}");
self.denoise.failed = Some(e.clone());
DenoiseStatus::Failed(e)
}
}
/// Everything the job needs, owned, so it can leave the UI thread.
struct Work {
ctx: dr_gpu::GpuContext,
raw: Arc<RawImage>,
profile: Option<Vec<(f32, f32)>>,
iso: Option<u32>,
model: std::path::PathBuf,
cancel: Arc<AtomicBool>,
}
impl Work {
fn run(self, progress: &mut dyn FnMut(usize, usize)) -> Result<Finished, String> {
let started = std::time::Instant::now();
// The app's own hot-pixel pass, on a copy: the classical source was
// repaired by the same pass inside `Demosaicer::run`.
let mut raw = (*self.raw).clone();
dr_gpu::Demosaicer::new(&self.ctx)
.and_then(|d| d.repair_hot_pixels(&mut raw))
.map_err(|e| e.to_string())?;
let noise = dr_denoise::noise::for_frame_with(&raw, self.profile.as_deref(), self.iso)
.ok_or("this photograph gives no way to measure its noise")?;
let mut net =
dr_denoise::onnx::OnnxNet::from_path(&self.model).map_err(|e| e.to_string())?;
let rung = net
.rung()
.map(|r| r.label().to_string())
.unwrap_or_default();
let cancel = self.cancel;
let rgb = dr_denoise::denoise(&raw, &noise, &mut net, &mut |done, total| {
progress(done, total);
!cancel.load(Ordering::Relaxed)
})
.map_err(|e| e.to_string())?
.ok_or("stopped")?;
Ok(Finished {
rgb,
width: raw.crop.width,
height: raw.crop.height,
source: noise.source,
rung,
seconds: started.elapsed().as_secs_f64(),
})
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::develop::test_support::headless;
use dr_pipeline::learned_denoise;
fn bayer(w: u32, h: u32) -> RawImage {
RawImage {
width: w,
height: h,
data: vec![800; (w * h) as usize],
cfa_pattern: dr_decode::CfaPattern::Rggb,
black_level: [0; 4],
white_level: 4095,
wb_coeffs: [2.0, 1.0, 1.5, 1.0],
color_matrix: Some([1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]),
samples_per_pixel: 1,
profile: None,
profile_tables: None,
baseline_exposure: 0.0,
make: String::new(),
model: String::new(),
crop: dr_decode::CropRect {
x: 0,
y: 0,
width: w,
height: h,
},
}
}
fn landed(session: &mut DevelopSession) {
let (w, h) = (64, 64);
session
.land(Finished {
rgb: vec![0.2; (w * h * 3) as usize],
width: w,
height: h,
source: dr_denoise::Source::Measured,
rung: "test".into(),
seconds: 0.0,
})
.expect("upload");
}
#[test]
fn a_bayer_raw_offers_the_switch_and_an_rgb_image_does_not() {
let Some(ctx) = headless() else { return };
let raw =
DevelopSession::open_owned(&ctx, bayer(64, 64), dr_types::Orientation::NORMAL).unwrap();
assert!(raw
.graph
.capabilities()
.iter()
.any(|c| c.id == learned_denoise::ID));
let rgba: Vec<u8> = (0..64 * 64).flat_map(|_| [128u8, 128, 128, 255]).collect();
let jpeg =
DevelopSession::open_rgb(&ctx, &rgba, 64, 64, dr_types::Orientation::NORMAL).unwrap();
assert!(!jpeg
.graph
.capabilities()
.iter()
.any(|c| c.id == learned_denoise::ID));
}
#[test]
fn the_render_draws_what_the_edit_asks_for() {
let Some(ctx) = headless() else { return };
let mut s =
DevelopSession::open_owned(&ctx, bayer(64, 64), dr_types::Orientation::NORMAL).unwrap();
let classical = s.demosaiced.clone();
landed(&mut s);
let result = s.denoise.result.clone().unwrap();
let same = |a: &Arc<DemosaicedImage>, b: &Arc<DemosaicedImage>| Arc::ptr_eq(a, b);
// Off: the classical demosaic, result or no result.
assert!(same(&s.developed_source(), &classical));
// On, no grain: the network's result as it is.
s.graph
.set_param(learned_denoise::ID, learned_denoise::APPLY, 1.0);
assert!(same(&s.developed_source(), &result));
// Grain: a blend, made once per value and reused until it moves.
s.graph
.set_param(learned_denoise::ID, learned_denoise::GRAIN, 40.0);
let blended = s.developed_source();
assert!(!same(&blended, &result) && !same(&blended, &classical));
assert!(
same(&s.developed_source(), &blended),
"the same grain must not blend again"
);
s.graph
.set_param(learned_denoise::ID, learned_denoise::GRAIN, 60.0);
assert!(!same(&s.developed_source(), &blended));
// The sensor's own reading stays the classical one throughout.
assert!(same(&s.demosaiced, &classical));
}
#[test]
fn switching_off_stops_and_forgets_a_failure() {
let Some(ctx) = headless() else { return };
let mut s =
DevelopSession::open_owned(&ctx, bayer(64, 64), dr_types::Orientation::NORMAL).unwrap();
s.denoise.failed = Some("no model".into());
s.graph
.set_param(learned_denoise::ID, learned_denoise::APPLY, 1.0);
s.reconcile_denoise();
assert!(
s.denoise.job.is_none(),
"a failure is not retried while the switch stays on"
);
s.graph
.set_param(learned_denoise::ID, learned_denoise::APPLY, 0.0);
s.reconcile_denoise();
assert!(
s.denoise.failed.is_none(),
"toggling off is how a failure is retried"
);
}
}
ui/dr-ui/src/develop/mod.rs
+2
View File
@@ -17,6 +17,7 @@
//! only methods were ever exported.
mod curves;
mod denoise;
mod framing;
mod history;
mod mask_ops;
@@ -29,6 +30,7 @@ mod session;
mod tabs;
mod white_balance;
pub use denoise::DenoiseStatus;
pub use framing::CropAspect;
pub use masks::MASK_COLOURS;
pub use segmentation::{Abandon, RefinedInstance, Segmented, SessionId};
ui/dr-ui/src/develop/render.rs
+10 -1
View File
@@ -235,7 +235,10 @@ impl DevelopSession {
reach: u32,
) -> Result<std::sync::Arc<dr_gpu::DemosaicedImage>, String> {
let Some(full) = self.full.clone() else {
return Ok(self.demosaiced.clone());
// TRACES: FR-DEV-3g
// The learned demosaic, with its grain, where the edit asks for
// it and it has landed; the classical one otherwise.
return Ok(self.developed_source());
};
let frame = self.demosaiced.size();
let ratio = self.graph.render_scale(frame, (w, h)).ratio();
@@ -706,6 +709,10 @@ impl DevelopSession {
&mut self,
space: dr_types::ColourSpace,
) -> Result<dr_export::Frame, String> {
// TRACES: FR-DEV-3g
// A photograph that asks for the learned demosaic is exported with
// it, computed now if it is not here (denoise.md §7.1).
self.denoise_blocking()?;
let (sw, sh) = self.demosaiced.size();
let (w, h) = self.graph.output_size(sw, sh);
@@ -1617,6 +1624,7 @@ mod tests {
samples_per_pixel: 1,
profile: None,
profile_tables: None,
baseline_exposure: 0.0,
make: String::new(),
model: String::new(),
crop: dr_decode::CropRect {
@@ -1871,6 +1879,7 @@ mod tests {
samples_per_pixel: 3,
profile: None,
profile_tables: None,
baseline_exposure: 0.0,
make: String::new(),
model: String::new(),
crop: dr_decode::CropRect {
ui/dr-ui/src/develop/session.rs
+75 -1
View File
@@ -76,6 +76,11 @@ pub struct DevelopSession {
/// the info panel offers to save for every photograph from that camera
/// (D20). `None` once taken up.
pub(super) profile_offer: Option<Arc<dr_decode::dcp::Dcp>>,
/// TRACES: FR-DEV-6
/// The edit the file already carries from the photographer's earlier work, translated, waiting for the
/// word that this photograph has no edit of DarkRoom's own
/// ([`Self::adopt_earlier_edit`]).
pub(super) earlier_edit: Option<Preset>,
/// Kept so the session can build GPU resources after construction.
///
/// The distance fields behind a subject mask are made when a layer is
@@ -118,6 +123,10 @@ pub struct DevelopSession {
/// history still stands on that crop. See [`super::framing::CropNotice`].
pub(super) crop_notice: Option<super::framing::CropNotice>,
pub(super) demosaiced: Arc<DemosaicedImage>,
/// TRACES: FR-DEV-3g
/// The learned demosaic: the kept mosaic, the job, the result. See
/// [`super::denoise`].
pub(super) denoise: super::denoise::DenoiseState,
/// TRACES: FR-DSP-2 | NFR-RES-2
/// The photograph at full resolution, when it is too large to hold in one
/// texture. `demosaiced` is then a reduced copy of it, which is all the
@@ -376,7 +385,12 @@ impl DevelopSession {
let (w, h) = (raw.crop.width.max(1), raw.crop.height.max(1));
let edge = PROXY_EDGE.min(DemosaicedImage::max_dimension(ctx));
if raw.samples_per_pixel != 3 || w.max(h) <= edge {
return Self::open(ctx, &raw, orientation);
let mut session = Self::open(ctx, &raw, orientation)?;
// TRACES: FR-DEV-3g
// The mosaic stays with the session when the learned demosaic
// could take it, so asking for it later needs no second decode.
session.keep_mosaic(raw);
return Ok(session);
}
let reduce = w.max(h).div_ceil(edge);
log::info!("{w}×{h} is larger than one texture; developing from a 1/{reduce} copy");
@@ -430,6 +444,7 @@ impl DevelopSession {
// and not found" — `lens_summary` distinguishes them.
lens_profile_found: false,
profile_offer: None,
earlier_edit: None,
ctx: ctx.clone(),
graph,
history,
@@ -438,6 +453,7 @@ impl DevelopSession {
compared_snapshot: None,
crop_notice: None,
demosaiced: Arc::new(demosaiced),
denoise: Default::default(),
full: None,
window: None,
adjust: AdjustPass::new(ctx),
@@ -600,6 +616,33 @@ impl DevelopSession {
.map(Arc::new);
}
/// TRACES: FR-DEV-6
/// Remember the earlier edit stored in the file this session opened.
pub fn set_earlier_edit(&mut self, edit: Option<Preset>) {
self.earlier_edit = edit;
}
/// TRACES: FR-DEV-6
/// Apply the file's earlier edit, as the photograph's starting point, so it
/// keeps the style it was given before it came to DarkRoom.
///
/// Only for a photograph DarkRoom has no edit of: the caller says so, and
/// must know it rather than guess — a stored edit that could not be
/// fetched is not an absent one, and the earlier edit would then be
/// saved over it on the way out. One undoable step, so undo shows the
/// photograph without it; from there it is an ordinary edit, saved with
/// the photograph. Taken, so a second call does nothing.
pub fn adopt_earlier_edit(&mut self) -> bool {
let Some(edit) = self.earlier_edit.take() else {
return false;
};
let rebake = edit.apply(&mut self.graph, Scope::adjustments());
self.pay_film_debt(&rebake);
self.history
.record(&self.graph, Edit::Action(labels::step::EARLIER_EDIT));
true
}
/// TRACES: FR-DEV-3e
/// What to tell the photographer about the camera profile (D20).
///
@@ -822,6 +865,7 @@ mod tests {
samples_per_pixel: 1,
profile: None,
profile_tables: tables.map(Arc::new),
baseline_exposure: 0.0,
make: "Canon".into(),
model: "EOS 6D".into(),
crop: dr_decode::CropRect {
@@ -842,6 +886,7 @@ mod tests {
origin,
hue_sat: None,
look: dr_types::HueSatTable::new(2, 2, 1, false, vec![[5.0, 1.1, 1.0]; 4]),
tone_curve: None,
};
let open = |t| {
DevelopSession::open(&ctx, &raw_with(t), dr_types::Orientation::NORMAL)
@@ -876,6 +921,34 @@ mod tests {
assert_eq!(jpeg.profile_summary(), "");
}
#[test]
fn a_earlier_edit_is_adopted_once_and_can_be_undone() {
// TRACES: FR-DEV-6
let Some(ctx) = headless() else { return };
let mut s = DevelopSession::open(&ctx, &raw_with(None), dr_types::Orientation::NORMAL)
.expect("session");
assert!(!s.adopt_earlier_edit(), "nothing to adopt yet");
let earlier = dr_preset_xmp::read_xmp(
r#"<x:xmpmeta xmlns:x="adobe:ns:meta/"><rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"><rdf:Description rdf:about="" xmlns:crs="http://ns.adobe.com/camera-raw-settings/1.0/" crs:SaturationAdjustmentBlue="+58" crs:Highlights2012="-40"/></rdf:RDF></x:xmpmeta>"#,
)
.expect("an edit")
.preset;
s.set_earlier_edit(Some(earlier));
assert!(s.adopt_earlier_edit());
let blue = || {
s.graph.param(
dr_pipeline::ops::colour_mixer::ID,
dr_pipeline::ParamId("blue_sat"),
)
};
assert_eq!(blue(), Some(58.0));
assert!(!s.adopt_earlier_edit(), "taken: a second call does nothing");
assert!(!s.graph.is_neutral());
s.undo();
assert!(s.graph.is_neutral(), "undo shows the photograph without it");
}
#[test]
fn only_a_copyable_profile_is_offered() {
// TRACES: FR-DEV-3e
@@ -897,6 +970,7 @@ mod tests {
hue_sat: [None, None],
look: dr_types::HueSatTable::new(2, 2, 1, false, vec![[5.0, 1.1, 1.0]; 4]),
tone_curve: None,
baseline_exposure_offset: 0.0,
};
s.set_embedded_profile(Some(profile(0)));
assert_eq!(
ui/dr-ui/src/develop/white_balance.rs
+3
View File
@@ -233,6 +233,7 @@ mod tests {
samples_per_pixel: 3,
profile: None,
profile_tables: None,
baseline_exposure: 0.0,
make: String::new(),
model: String::new(),
crop: dr_decode::CropRect {
@@ -307,6 +308,7 @@ mod tests {
samples_per_pixel: 3,
profile: None,
profile_tables: None,
baseline_exposure: 0.0,
make: String::new(),
model: String::new(),
crop: dr_decode::CropRect {
@@ -360,6 +362,7 @@ mod tests {
samples_per_pixel: 3,
profile: None,
profile_tables: None,
baseline_exposure: 0.0,
make: String::new(),
model: String::new(),
crop: dr_decode::CropRect {
ui/dr-ui/src/develop_ui.rs
+80
View File
@@ -22,6 +22,7 @@ use slint::ComponentHandle as _;
use crate::activity;
use crate::collections_ui::CollectionsController;
use crate::develop;
use crate::develop::DenoiseStatus;
use crate::library_ui::LibraryController;
use crate::masks_ui;
use crate::peaking;
@@ -132,6 +133,85 @@ pub(crate) fn wire(
wire_rotation_flips_straighten(window, &w, &crop_aspect, &crop_portrait);
wire_navigation(window, &w);
wire_peaking(window, &w);
wire_learned_denoise(window, &w);
}
/// TRACES: FR-DEV-3g
/// Keep the learned denoise in step with the edit, and show it working.
///
/// A poll rather than a hook on each way an edit can change — a slider,
/// undo, a preset, a version opened, a sidecar merged from another device:
/// [`DevelopSession::reconcile_denoise`] costs two comparisons when nothing
/// changed, and a hook that one of those paths forgot would leave a
/// photograph that asks for the learned demosaic showing the classical one.
fn wire_learned_denoise(window: &AppWindow, w: &DevelopWiring) {
let weak = window.as_weak();
let session = w.session.clone();
let redraw = w.redraw.clone();
let rows = w.rows.clone();
let activity = w.activity.clone();
let row: Rc<RefCell<Option<activity::Activity>>> = Rc::new(RefCell::new(None));
let timer = slint::Timer::default();
timer.start(
slint::TimerMode::Repeated,
std::time::Duration::from_millis(250),
move || {
let Some(win) = weak.upgrade() else { return };
let status = {
let Ok(mut guard) = session.try_borrow_mut() else {
return;
};
let Some(s) = guard.as_mut() else {
// The photograph closed: whatever was running went with it.
if let Some(r) = row.borrow_mut().take() {
r.finish_quietly();
}
return;
};
s.reconcile_denoise();
s.poll_denoise()
};
match status {
DenoiseStatus::Idle => {
// Stopped without landing: switched off, or a new photograph.
if let Some(r) = row.borrow_mut().take() {
r.finish_quietly();
}
}
DenoiseStatus::Running(done, total) => {
let mut r = row.borrow_mut();
let r = r.get_or_insert_with(|| {
activity.begin(activity::Kind::Denoise, "AI denoise")
});
r.progress(done, total);
}
DenoiseStatus::Landed => {
// Kept in the list with where its noise figures came
// from: measured for the body, the DNG's, or estimated.
let source = session
.borrow()
.as_ref()
.and_then(|s| s.denoise_source())
.map(|s| format!("noise {}", s.label()))
.unwrap_or_default();
let r = row.borrow_mut().take();
let r =
r.unwrap_or_else(|| activity.begin(activity::Kind::Denoise, "AI denoise"));
r.finish(source);
sync_rows(&win, &rows, &session);
redraw(&win);
}
DenoiseStatus::Failed(e) => {
let r = row.borrow_mut().take();
let r =
r.unwrap_or_else(|| activity.begin(activity::Kind::Denoise, "AI denoise"));
r.fail(e);
}
}
},
);
// For the life of the window, like the screen it serves.
std::mem::forget(timer);
}
/// ---- copying settings between photographs (FR-DEV-6) ----------------
ui/dr-ui/src/export.rs
+10 -5
View File
@@ -1293,15 +1293,15 @@ fn render_from_library(
}
};
let sidecar = match wait_for(&sidecar_rx, cancel) {
Waited::Got(sidecar) => sidecar,
let fetched = match wait_for(&sidecar_rx, cancel) {
Waited::Got(fetched) => fetched,
Waited::Cancelled => return None,
// An unedited photograph has no sidecar and a fetch that died looks
// identical from here. Exporting at defaults is what opening it would
// A fetch that died. Exporting at defaults is what opening it would
// do, and refusing the image over a missing edit it may never have had
// would fail the common case.
Waited::Silent => None,
Waited::Silent => crate::library::FetchedSidecar::default(),
};
let sidecar = fetched.sidecar;
let (meta, mut session) = match open_for_export(gpu, request.decoder, &bytes) {
Ok(opened) => opened,
@@ -1318,6 +1318,11 @@ fn render_from_library(
.and_then(dr_pipeline::Sidecar::default_version)
{
session.apply_version(version);
} else if fetched.absent {
// TRACES: FR-DEV-6
// No edit of DarkRoom's, by the server's word: export what opening it
// would show — the photograph's earlier edit, where it carries one.
session.adopt_earlier_edit();
}
// The last cheap place to stop. Everything past here is a full-resolution
ui/dr-ui/src/inference.rs
+1
View File
@@ -34,6 +34,7 @@ pub fn init(runtime_dirs: Vec<PathBuf>) {
(Role::EyeClassifier, crate::library::EYE_MODEL),
(Role::EyeClassifier, crate::library::SUNGLASSES_MODEL),
(Role::Inpainter, crate::library::INPAINT_MODEL),
(Role::Denoiser, crate::library::DENOISE_MODEL),
]);
let models: Vec<(Role, PathBuf)> = wanted
.into_iter()
ui/dr-ui/src/labels.rs
+16 -4
View File
@@ -27,6 +27,9 @@ pub mod step {
use dr_pipeline::LocalizedKey;
pub const PASTE: LocalizedKey = LocalizedKey("history.paste");
/// TRACES: FR-DEV-6
/// The edit a photograph already carried, brought across on first opening.
pub const EARLIER_EDIT: LocalizedKey = LocalizedKey("history.earlier_edit");
pub const FILM: LocalizedKey = LocalizedKey("history.film");
/// TRACES: FR-DEV-5
/// The photograph put back to a named snapshot, as one step.
@@ -100,6 +103,7 @@ pub mod step {
dr_pipeline::history::OPENED,
dr_pipeline::history::UNNAMED,
PASTE,
EARLIER_EDIT,
SNAPSHOT_RESTORED,
FILM,
SAMPLED_NEUTRAL,
@@ -203,14 +207,14 @@ fn catalogued(key: &str) -> Option<&'static str> {
// it is a tick box, and a checkbox labelled with a verb reads as a
// button that does something once rather than a state that is on.
"op.lens_profile" => "Lens Profile",
"op.learned_denoise" => "AI Denoise",
// The view transform (FR-DEV-3j). "Tone Mapping" rather than the
// "View Transform" `derive` would give: the id names where it sits in
// the pipeline, and the photographer is choosing how the scene's range
// is fitted onto the screen.
"op.view_transform" => "Tone Mapping",
// The DNG camera profile's tables (D20). "Camera Profile", as Camera
// Raw calls the same thing, so a photographer arriving from Lightroom
// finds it under the name they know.
// The DNG camera profile's tables (D20). "Camera Profile", the DNG
// specification's own name for what the file carries.
"op.camera_profile" => "Camera Profile",
// Parameters
@@ -218,12 +222,19 @@ fn catalogued(key: &str) -> Option<&'static str> {
// parameter is titled by its operation and this one never reaches the
// panel under its own name — see `rows_filtered`.
"param.lens_profile.apply" => "Apply",
"param.learned_denoise.apply" => "Apply",
"param.learned_denoise.grain" => "Keep grain",
"param.camera_profile.apply" => "Use Profile",
// The LookTable's strength, as Lightroom's profile "Amount".
// The LookTable's strength.
"param.camera_profile.look" => "Look Amount",
"param.view_transform.contrast" => "Contrast",
// In stops above middle grey: where the scene reaches display white.
"param.view_transform.white" => "White Point",
// The view transform's curve (D21): the DNG SDK's reference
// rendering, or D19's sigmoid with its highlight shoulder.
"param.view_transform.curve" => "Curve",
"param.view_transform.curve.camera_raw" => "DNG Reference",
"param.view_transform.curve.sigmoid" => "Sigmoid",
"param.temperature" => "Temperature",
"param.tint" => "Tint",
"param.exposure" => "Exposure",
@@ -314,6 +325,7 @@ fn catalogued(key: &str) -> Option<&'static str> {
"history.opened" => "Opened",
"history.edit" => "Edit",
"history.paste" => "Paste Settings",
"history.earlier_edit" => "Earlier Edit",
"history.snapshot_restored" => "Restore A Snapshot",
"history.film" => "Film Stock",
"history.sampled_neutral" => "Sample Neutral",
ui/dr-ui/src/lib.rs
+51 -8
View File
@@ -324,6 +324,9 @@ pub(crate) fn open_session(
// the capture date because of this line; before it, the same photograph
// exported from the grid kept them and exported from develop did not.
session.set_source_metadata(meta.clone());
// TRACES: FR-DEV-3g
// The DNG's measured noise and the ISO, for the learned denoise.
session.prepare_denoise(bytes, meta);
// TRACES: FR-DEV-3e
// The embedded camera profile, read again from the header, for the info
// panel's offer to copy it. Only a DNG carries one; the read is the
@@ -331,6 +334,10 @@ pub(crate) fn open_session(
if dr_decode::probe(bytes) != Some(dr_types::Format::Jpeg) {
session.set_embedded_profile(dr_decode::dcp::embedded_in(bytes));
}
// TRACES: FR-DEV-6
// And the earlier edit stored in it, applied by the caller only once it
// knows DarkRoom has no edit of this photograph.
session.set_earlier_edit(dr_preset_xmp::read_embedded(bytes).map(|i| i.preset));
Ok(session)
}
@@ -817,7 +824,7 @@ pub(crate) fn refresh_export_label(window: &AppWindow) {
/// whose session this sidecar must not be applied to.
fn apply_when_ready(
window: &AppWindow,
rx: Rc<std::sync::mpsc::Receiver<Option<dr_pipeline::Sidecar>>>,
rx: Rc<std::sync::mpsc::Receiver<library::FetchedSidecar>>,
session: &Rc<RefCell<Option<DevelopSession>>>,
rows: &Rc<slint::VecModel<ParamRow>>,
redraw: &Rc<dyn Fn(&AppWindow)>,
@@ -827,9 +834,7 @@ fn apply_when_ready(
// Already here — the common case.
if let Ok(got) = rx.try_recv() {
if let Some(sidecar) = got {
presets::apply_stored_edit(window, &sidecar, session, rows);
}
apply_fetched(window, got, session, rows);
redraw(window);
return;
}
@@ -864,10 +869,7 @@ fn apply_when_ready(
return;
};
held.stop();
let applied = match got {
Some(sidecar) => presets::apply_stored_edit(&w, &sidecar, &session, &rows),
None => false,
};
let applied = apply_fetched(&w, got, &session, &rows);
if applied || !drawn.get() {
redraw(&w);
}
@@ -875,6 +877,42 @@ fn apply_when_ready(
);
}
/// TRACES: FR-CAT-8 | FR-DEV-6
/// Apply what the stored-edit fetch answered: the edit, or — only where the
/// server said there is none — the photograph's earlier edit. An answer
/// that could not be had (offline, unreachable, unreadable) applies nothing,
/// so a real edit that failed to arrive is never saved over.
fn apply_fetched(
window: &AppWindow,
got: library::FetchedSidecar,
session: &Rc<RefCell<Option<DevelopSession>>>,
rows: &Rc<slint::VecModel<ParamRow>>,
) -> bool {
match got.sidecar {
Some(sidecar) => presets::apply_stored_edit(window, &sidecar, session, rows),
None if got.absent => adopt_earlier_edit(window, session, rows),
None => false,
}
}
/// TRACES: FR-DEV-6
/// Start the open photograph from its earlier edit, if it has one.
fn adopt_earlier_edit(
window: &AppWindow,
session: &Rc<RefCell<Option<DevelopSession>>>,
rows: &Rc<slint::VecModel<ParamRow>>,
) -> bool {
let adopted = session
.borrow_mut()
.as_mut()
.is_some_and(DevelopSession::adopt_earlier_edit);
if adopted {
log::info!("started from the photograph's earlier edit");
sync_rows(window, rows, session);
}
adopted
}
/// TRACES: FR-DEV-3f
/// Push the film choice out to the panel.
///
@@ -2620,6 +2658,11 @@ fn build_show(
*open_image.borrow_mut() = presets::Stored::Local(path.to_path_buf());
if let Some(sidecar) = presets::load_local(path) {
presets::apply_stored_edit(window, &sidecar, &session, &rows);
} else {
// TRACES: FR-DEV-6
// No edit of DarkRoom's beside the file: start
// from the earlier edit stored in it.
adopt_earlier_edit(window, &session, &rows);
}
// TRACES: FR-UI-4
ui/dr-ui/src/library/cells.rs
+1
View File
@@ -1015,6 +1015,7 @@ mod tests {
width,
height,
rgba,
class: dr_thumbs::ThumbSize::Grid,
from_cache: true,
};
ui/dr-ui/src/library/paths.rs
+10
View File
@@ -375,6 +375,16 @@ pub fn inpaint_model() -> Option<PathBuf> {
shared_model(INPAINT_MODEL)
}
/// TRACES: FR-DEV-3g
/// The learned demosaic and denoise, as shipped in `models/denoise/`.
pub const DENOISE_MODEL: &str = "mosaic-1408.onnx";
/// TRACES: FR-DEV-3g
/// Where the denoise model is, by the border filler's search.
pub fn denoise_model() -> Option<PathBuf> {
shared_model(DENOISE_MODEL)
}
#[cfg(test)]
mod tests {
use super::*;
ui/dr-ui/src/library/sweep.rs
+156 -7
View File
@@ -83,7 +83,7 @@ pub(super) fn flush_metadata(
/// shards synced from another device, which carry pixels but no metadata.
/// Read a header for its date.
///
/// Returns whether the file was **reached**, which the caller needs and cannot
/// Reports whether the file was **reached**, which the caller needs and cannot
/// otherwise tell: a header that carried no EXIF and a fetch that never
/// happened both leave `found` untouched, and recording the second as "this
/// image has no date" would let one lock mark it dateless for good.
@@ -92,7 +92,7 @@ pub(crate) async fn read_metadata_only(
decoder: &dyn dr_decode::Decoder,
req: &ThumbnailRequest,
found: &mut Vec<MetadataFound>,
) -> bool {
) -> DateRead {
let id = RemoteId::Path(RemotePath::new(&req.path));
// Retried, because one failure here is usually a lock rather than a
@@ -108,7 +108,16 @@ pub(crate) async fn read_metadata_only(
match backend.get(&id, Some(0..decoder.header_bytes())).await {
Ok(header) => {
collect_metadata(backend, decoder, &id, &header, req, found).await;
return true;
return DateRead::Reached;
}
// Not retried, unlike a lock. A dead server answers the second
// attempt exactly as it answered the first, after the same 15 s
// connect timeout — three of those per image turned a window of
// cached-but-undated cells into minutes of "reading dates"
// against nothing, with no banner, because nothing said why.
Err(e) if e.indicates_offline() => {
log::debug!("reading date for {}: {e}", req.path);
return DateRead::Offline(e.to_string());
}
Err(e) if e.is_transient() && attempt < ATTEMPTS => {
// Backing off at all matters more than the exact interval: the
@@ -121,11 +130,23 @@ pub(crate) async fn read_metadata_only(
// timeline rather than breaking anything, and the next sweep
// retries it regardless.
log::debug!("reading date for {} ({attempt} attempts): {e}", req.path);
return false;
return DateRead::Failed;
}
}
}
false
DateRead::Failed
}
/// What one header read for a date came to.
#[derive(Debug, PartialEq, Eq)]
pub(crate) enum DateRead {
/// The header arrived. Whatever EXIF it held is in `found`.
Reached,
/// This file could not be read; the next pass tries it again.
Failed,
/// The server could not be reached, so no read after this one will be
/// either. The caller stops rather than paying a timeout per image.
Offline(String),
}
/// Write capture metadata read during the thumbnail pass.
@@ -381,8 +402,12 @@ pub fn spawn_sweep(conn: Connection, catalog_path: PathBuf) -> Receiver<SweepMes
let mut found = Vec::new();
let mut reached = Vec::new();
for req in lane {
if read_metadata_only(backend, decoder, req, &mut found).await {
reached.push(req.image_id);
match read_metadata_only(backend, decoder, req, &mut found).await {
DateRead::Reached => reached.push(req.image_id),
DateRead::Failed => {}
// The other lanes find the same, each after
// one timeout rather than one per image.
DateRead::Offline(_) => break,
}
}
(found, reached)
@@ -901,6 +926,130 @@ pub(super) fn thumbnails_outstanding(
mod tests {
use super::*;
/// A backend whose every read fails the same way, counting the attempts.
struct Refusing {
error: fn() -> dr_sync::RemoteError,
gets: std::sync::atomic::AtomicUsize,
caps: dr_sync::Capabilities,
}
#[async_trait::async_trait]
impl RemoteBackend for Refusing {
fn capabilities(&self) -> &dr_sync::Capabilities {
&self.caps
}
fn name(&self) -> &str {
"refusing"
}
async fn list(
&self,
_dir: &RemotePath,
_since: Option<&dr_sync::Validator>,
) -> Result<Vec<dr_sync::RemoteEntry>, dr_sync::RemoteError> {
Err((self.error)())
}
async fn dir_validator(
&self,
_dir: &RemotePath,
) -> Result<dr_sync::Validator, dr_sync::RemoteError> {
Err((self.error)())
}
async fn delta(
&self,
_c: &dr_sync::Cursor,
) -> Result<(Vec<dr_sync::RemoteChange>, dr_sync::Cursor), dr_sync::RemoteError> {
Err((self.error)())
}
async fn get(
&self,
_id: &RemoteId,
_r: Option<std::ops::Range<u64>>,
) -> Result<Vec<u8>, dr_sync::RemoteError> {
self.gets.fetch_add(1, std::sync::atomic::Ordering::SeqCst);
Err((self.error)())
}
async fn put(
&self,
_p: &RemotePath,
_b: Vec<u8>,
_c: Option<dr_sync::Precondition>,
) -> Result<dr_sync::Validator, dr_sync::RemoteError> {
Err((self.error)())
}
async fn delete(
&self,
_id: &RemoteId,
_c: Option<dr_sync::Precondition>,
) -> Result<(), dr_sync::RemoteError> {
Err((self.error)())
}
async fn move_to(
&self,
_f: &RemoteId,
_t: &RemotePath,
) -> Result<(), dr_sync::RemoteError> {
Err((self.error)())
}
async fn create_dir(&self, _p: &RemotePath) -> Result<(), dr_sync::RemoteError> {
Err((self.error)())
}
}
fn read_date_against(error: fn() -> dr_sync::RemoteError) -> (DateRead, usize) {
let backend = Refusing {
error,
gets: Default::default(),
caps: dr_sync::Capabilities::minimal(),
};
let req = ThumbnailRequest {
row: 0,
path: "a.CR2".into(),
file_id: Some(1),
size: 0,
image_id: 1,
thumb_size: dr_thumbs::ThumbSize::Grid,
needs_metadata: true,
full_resolution: false,
};
let rt = crate::net_runtime::build().unwrap();
let outcome = rt.block_on(read_metadata_only(
&backend,
dr_decode::default(),
&req,
&mut Vec::new(),
));
(
outcome,
backend.gets.load(std::sync::atomic::Ordering::SeqCst),
)
}
#[test]
fn a_date_read_against_a_dead_server_asks_once_and_says_so() {
// Each attempt waited out a 15 s connect timeout, three per image, for
// every cached-but-undated cell in the window — minutes of "reading
// dates" with no banner, because the caller could not tell a dead
// server from a missing file.
let (outcome, gets) =
read_date_against(|| dr_sync::RemoteError::Network("connection refused".into()));
assert!(matches!(outcome, DateRead::Offline(_)), "{outcome:?}");
assert_eq!(gets, 1, "retrying a dead server buys another timeout");
}
#[test]
fn a_date_read_that_hits_a_lock_still_retries() {
// The reason the retry exists: Nextcloud answers a read with 423 under
// our own concurrency, and the same range succeeds moments later.
let (outcome, gets) = read_date_against(|| dr_sync::RemoteError::Server {
status: 423,
detail: "locked".into(),
});
assert_eq!(outcome, DateRead::Failed);
assert_eq!(gets, 3);
}
#[test]
fn a_header_gives_the_size_the_photograph_is_seen_at() {
let mut md = dr_decode::Metadata {
ui/dr-ui/src/library/thumbnails_fetch.rs
+43 -11
View File
@@ -440,7 +440,7 @@ pub fn spawn_sidecar_fetch(
image_path: String,
cache_dir: PathBuf,
offline: bool,
) -> Receiver<Option<dr_pipeline::Sidecar>> {
) -> Receiver<FetchedSidecar> {
let (tx, rx) = std::sync::mpsc::channel();
executors::spawn(Executor::Network, "sidecars", move || {
@@ -455,7 +455,7 @@ pub fn spawn_sidecar_fetch(
// which the next save would then write back over the top of.
if cache.is_pending(&path_str) {
log::debug!("{path_str} has queued local edits; opening from the cache");
let _ = tx.send(cache.load(&path_str));
let _ = tx.send(FetchedSidecar::unknown(cache.load(&path_str)));
return;
}
@@ -472,7 +472,7 @@ pub fn spawn_sidecar_fetch(
}
};
let Some(rt) = rt else {
let _ = tx.send(cache.load(&path_str));
let _ = tx.send(FetchedSidecar::unknown(cache.load(&path_str)));
return;
};
@@ -481,7 +481,7 @@ pub fn spawn_sidecar_fetch(
Ok(b) => b,
Err(e) => {
log::debug!("sidecar fetch backend: {e}");
let _ = tx.send(cache.load(&path_str));
let _ = tx.send(FetchedSidecar::unknown(cache.load(&path_str)));
return;
}
};
@@ -491,12 +491,21 @@ pub fn spawn_sidecar_fetch(
// A 404 is the normal case on a library that has never been
// edited, so this is `ok()` rather than an error path.
let Ok(bytes) = backend.get(&id, None).await else {
// Unreachable, or no such file. The cache cannot tell those
// apart and does not need to: either way it holds the best
// answer this device has.
let _ = tx.send(cache.load(&path_str));
return;
let bytes = match backend.get(&id, None).await {
Ok(bytes) => bytes,
Err(e) => {
// Unreachable, or no such file. The cache holds the best
// answer this device has either way — but only a server
// that said "no such file", with nothing in the cache,
// is the word that there is no edit (FR-DEV-6).
let cached = cache.load(&path_str);
let absent = cached.is_none() && matches!(e, dr_sync::RemoteError::NotFound(_));
let _ = tx.send(FetchedSidecar {
sidecar: cached,
absent,
});
return;
}
};
let text = String::from_utf8_lossy(&bytes).into_owned();
@@ -530,13 +539,36 @@ pub fn spawn_sidecar_fetch(
}
}
let _ = tx.send(parsed);
let _ = tx.send(FetchedSidecar::unknown(parsed));
});
});
rx
}
/// TRACES: FR-CAT-8 | FR-DEV-6
/// What a stored-edit fetch answered.
///
/// `absent` is true only when the server said there is no such file and no
/// cached copy stood in — the one answer that means "DarkRoom has no edit of
/// this photograph", and so the only one that lets its earlier edit in.
/// Every other way of arriving at no sidecar — offline, unreachable, a file
/// that would not parse — leaves it false.
#[derive(Debug, Default)]
pub struct FetchedSidecar {
pub sidecar: Option<dr_pipeline::Sidecar>,
pub absent: bool,
}
impl FetchedSidecar {
fn unknown(sidecar: Option<dr_pipeline::Sidecar>) -> Self {
Self {
sidecar,
absent: false,
}
}
}
/// Fetch one file in full, for opening it in develop.
///
/// Deliberately *not* the preview path. Browsing fetches a range and decodes
ui/dr-ui/src/library/thumbnails_gen.rs
+204 -42
View File
@@ -9,7 +9,7 @@ use dr_thumbs::ThumbStore;
use std::path::PathBuf;
use std::sync::mpsc::Receiver;
use super::sweep::{flush_metadata, read_metadata_only, MetadataFound};
use super::sweep::{flush_metadata, read_metadata_only, DateRead, MetadataFound};
#[cfg(test)]
use super::sweep::{thumbnails_outstanding, SWEEP_THUMB_SIZE};
use super::thumbnails_fetch::{ThumbnailRequest, MAX_PREVIEW_BYTES};
@@ -33,6 +33,11 @@ pub struct ThumbnailReady {
pub width: u32,
pub height: u32,
pub rgba: Vec<u8>,
/// Which class these pixels are, which is not always the class asked for:
/// the store answers a miss with the other class where it holds one. The
/// grid records this, so a stand-in is replaced once the real class can be
/// fetched rather than counted as served.
pub class: dr_thumbs::ThumbSize,
/// Whether these pixels came off local disk rather than the server.
///
/// The grid paints both identically, so this exists solely for
@@ -61,6 +66,9 @@ pub enum ThumbnailMessage {
cached: usize,
fetching: usize,
dating: usize,
/// Of `cached`, how many are the other class standing in for one
/// still being fetched. Each of those rows is delivered twice.
standing_in: usize,
},
/// One header-only date read is starting.
///
@@ -115,46 +123,12 @@ pub fn spawn_thumbnails(
// Split the batch before delivering anything, so the plan can be
// reported first and the UI knows the shape of the work up front.
// Decoding happens here rather than in the split, because a corrupt
// blob turns a hit into a miss.
let mut hits = Vec::new();
let mut to_fetch = Vec::new();
// Images whose thumbnail is cached but whose date is still unknown.
//
// These need a header read even though no pixels are wanted. Without
// this pass an image is dated *only* on the one visit that produced
// its thumbnail — so a library browsed once before the EXIF code
// existed, or synced from another device's shards, stays permanently
// undated and never appears on the timeline.
let mut metadata_only = Vec::new();
for req in wanted {
let stored = req
.file_id
.zip(store.as_ref())
.and_then(|(id, s)| s.get(id, req.thumb_size).ok().flatten());
match stored.map(|t| dr_thumbs::decode_rgba(&t.bytes)) {
Some(Ok((width, height, rgba))) => {
if req.needs_metadata {
metadata_only.push(req.clone());
}
hits.push(ThumbnailReady {
row: req.row,
width,
height,
rgba,
from_cache: true,
});
}
// A corrupt stored blob is a miss, not a failure.
Some(Err(e)) => {
log::debug!("stored thumbnail unreadable, refetching: {e}");
to_fetch.push(req);
}
None => to_fetch.push(req),
}
}
let Split {
hits,
to_fetch,
metadata_only,
standing_in,
} = split_by_store(store.as_ref(), wanted);
log::info!(
"thumbnails: {} from store, {} to fetch{}",
@@ -171,6 +145,7 @@ pub fn spawn_thumbnails(
cached: hits.len(),
fetching: to_fetch.len(),
dating: metadata_only.len(),
standing_in,
})
.is_err()
{
@@ -251,7 +226,14 @@ pub fn spawn_thumbnails(
if tx.send(ThumbnailMessage::DateProgress).is_err() {
break;
}
read_metadata_only(&*backend, decoder, &req, &mut found).await;
if let DateRead::Offline(reason) =
read_metadata_only(&*backend, decoder, &req, &mut found).await
{
// Said once, as the fetch loop does, so the banner goes
// up and the bar stops rather than sweeping for ever.
let _ = tx.send(ThumbnailMessage::Offline { reason });
break;
}
if found.len() >= FLUSH_EVERY {
flush_metadata(&catalog_path, &mut found, &tx);
@@ -269,6 +251,105 @@ pub fn spawn_thumbnails(
rx
}
/// A batch divided by what the store can answer.
#[derive(Default)]
struct Split {
/// Decoded off local disk, ready to deliver.
hits: Vec<ThumbnailReady>,
/// Needing the network: a miss, or a smaller class standing in.
to_fetch: Vec<ThumbnailRequest>,
/// Images whose thumbnail is cached but whose date is still unknown.
///
/// These need a header read even though no pixels are wanted. Without
/// this pass an image is dated *only* on the one visit that produced
/// its thumbnail — so a library browsed once before the EXIF code
/// existed, or synced from another device's shards, stays permanently
/// undated and never appears on the timeline.
metadata_only: Vec<ThumbnailRequest>,
/// Rows in both `hits` and `to_fetch`: the grid class shown while the
/// large one is fetched.
standing_in: usize,
}
/// Divide a batch into what the store holds and what must be fetched.
///
/// Decoding happens here rather than after, because a corrupt blob turns a
/// hit into a miss.
fn split_by_store(store: Option<&ThumbStore>, wanted: Vec<ThumbnailRequest>) -> Split {
let mut split = Split::default();
for req in wanted {
if let Some(hit) = read_stored(store, &req, req.thumb_size) {
if req.needs_metadata {
split.metadata_only.push(req.clone());
}
split.hits.push(hit);
continue;
}
// The other class, before the network. The sweep stores only the
// grid class, so a zoomed grid asked the server for every cell it had
// not zoomed over before — and offline, left each one blank with its
// 256px thumbnail sitting in the store. A softer cell is better than
// an empty one, and a sharper one is simply better.
// A wide panorama cell falls back to the grid class too: the sweep
// always stores it, and smaller, it only stands in while the wide
// class is fetched.
let other = match req.thumb_size {
dr_thumbs::ThumbSize::Grid => dr_thumbs::ThumbSize::Large,
dr_thumbs::ThumbSize::Large
| dr_thumbs::ThumbSize::Wide2
| dr_thumbs::ThumbSize::Wide3
| dr_thumbs::ThumbSize::Wide4 => dr_thumbs::ThumbSize::Grid,
};
match read_stored(store, &req, other) {
// Larger than asked for: nothing a fetch would improve on.
Some(hit) if other > req.thumb_size => {
if req.needs_metadata {
split.metadata_only.push(req.clone());
}
split.hits.push(hit);
}
// Smaller: shown now, and the fetch still goes out to replace it.
// The fetch reads the header, so it dates the image too.
Some(stand_in) => {
split.standing_in += 1;
split.hits.push(stand_in);
split.to_fetch.push(req);
}
None => split.to_fetch.push(req),
}
}
split
}
/// A stored thumbnail of one class, decoded for the grid.
///
/// `None` for a miss, for a request with no file id to key on, and for a
/// corrupt blob — which is a miss, not a failure, and gets refetched.
fn read_stored(
store: Option<&ThumbStore>,
req: &ThumbnailRequest,
class: dr_thumbs::ThumbSize,
) -> Option<ThumbnailReady> {
let stored = store?.get(req.file_id?, class).ok().flatten()?;
match dr_thumbs::decode_rgba(&stored.bytes) {
Ok((width, height, rgba)) => Some(ThumbnailReady {
row: req.row,
width,
height,
rgba,
class,
from_cache: true,
}),
Err(e) => {
log::debug!("stored thumbnail unreadable, refetching: {e}");
None
}
}
}
pub(super) async fn fetch_one(
backend: &dyn RemoteBackend,
decoder: &dyn dr_decode::Decoder,
@@ -302,6 +383,7 @@ pub(super) async fn fetch_one(
width: preview.width,
height: preview.height,
rgba: preview.rgba,
class: req.thumb_size,
from_cache: false,
}))
}
@@ -546,6 +628,86 @@ pub fn camera_label(make: Option<&str>, model: Option<&str>) -> Option<String> {
mod tests {
use super::*;
/// A store holding one image at one class, and a request for it.
fn store_with(tag: &str, file_id: u64, edge: u32, class: dr_thumbs::ThumbSize) -> ThumbStore {
let dir = std::env::temp_dir().join(format!("dr-ui-split-{tag}-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
let rgba: Vec<u8> = std::iter::repeat_n([10u8, 20, 30, 255], (edge * edge) as usize)
.flatten()
.collect();
let mut store = ThumbStore::open(&dir).unwrap();
let bytes = dr_thumbs::encode_rgba(edge, edge, &rgba).unwrap();
store
.put(
file_id,
class,
&dr_thumbs::Thumbnail {
width: edge,
height: edge,
bytes,
},
)
.unwrap();
store
}
fn asking(file_id: u64, thumb_size: dr_thumbs::ThumbSize) -> ThumbnailRequest {
ThumbnailRequest {
row: 0,
path: "a.CR2".into(),
file_id: Some(file_id),
size: 0,
image_id: 1,
thumb_size,
needs_metadata: false,
full_resolution: false,
}
}
#[test]
fn a_zoomed_cell_is_drawn_from_the_grid_class_while_the_large_one_is_fetched() {
// The offline symptom: the sweep stores only the grid class, so a grid
// zoomed past 256px found nothing at its class and went to a server
// that was not there — every cell blank, its thumbnail in the store.
use dr_thumbs::ThumbSize::{Grid, Large};
let store = store_with("standin", 7, 8, Grid);
let split = split_by_store(Some(&store), vec![asking(7, Large)]);
assert_eq!(split.hits.len(), 1, "drawn from the store now");
assert_eq!(split.hits[0].class, Grid, "and recorded as what it is");
assert_eq!(split.to_fetch.len(), 1, "the large class is still wanted");
assert_eq!(split.standing_in, 1);
}
#[test]
fn a_large_thumbnail_answers_a_grid_request_without_a_fetch() {
use dr_thumbs::ThumbSize::{Grid, Large};
let store = store_with("larger", 7, 16, Large);
let split = split_by_store(Some(&store), vec![asking(7, Grid)]);
assert_eq!(split.hits.len(), 1);
assert_eq!(split.hits[0].class, Large);
assert!(
split.to_fetch.is_empty(),
"nothing a fetch would improve on"
);
assert_eq!(split.standing_in, 0);
}
#[test]
fn an_exact_hit_is_not_a_stand_in() {
use dr_thumbs::ThumbSize::Large;
let store = store_with("exact", 7, 16, Large);
let split = split_by_store(Some(&store), vec![asking(7, Large)]);
assert_eq!(split.hits.len(), 1);
assert!(split.to_fetch.is_empty());
assert_eq!(split.standing_in, 0);
}
#[test]
fn the_thumbnail_pass_asks_only_for_what_is_missing() {
// The work list is the whole point of the pass being resumable and of
ui/dr-ui/src/library_ui/window.rs
+39 -8
View File
@@ -96,8 +96,20 @@ fn already_served(
held: &std::collections::HashMap<i64, Held>,
ids: impl Iterator<Item = i64>,
) -> std::collections::HashSet<(i64, dr_thumbs::ThumbSize)> {
ids.filter_map(|id| Some((id, held.get(&id)?.class?)))
.collect()
use dr_thumbs::ThumbSize;
let mut served = std::collections::HashSet::new();
for id in ids {
let Some(class) = held.get(&id).and_then(|h| h.class) else {
continue;
};
served.insert((id, class));
// The large class is everything the grid class would be, and more: a
// cell the store answered with it has nothing left to ask for.
if class == ThumbSize::Large {
served.insert((id, ThumbSize::Grid));
}
}
served
}
/// Where the loaded window starts for a view whose first visible cell is
@@ -1014,11 +1026,13 @@ fn drain_thumbnails(
cached,
fetching,
dating,
standing_in,
} => {
// Date reads produce no cell, so they are counted into
// the bar's denominator or it finishes while work is
// still running.
job.add_total(dating);
// still running. A stand-in is delivered once from the
// store and again from the fetch, so it counts twice.
job.add_total(dating + standing_in);
let mut parts = Vec::new();
if cached > 0 {
@@ -1084,10 +1098,11 @@ fn drain_thumbnails(
row.has_thumb = true;
model.set_row_data(t.row, row);
// What this cell is now showing, so the next reload
// can carry it over and know not to ask again.
if let Some(class) = classes.get(t.row) {
record_class(&ctl_cb, t.row, *class);
}
// can carry it over and know not to ask again. The
// pixels' own class, not the one the row asked for:
// a stand-in recorded as the class it stands in for
// would never be replaced.
record_class(&ctl_cb, t.row, t.class);
}
}
ThumbnailMessage::Unavailable { row, reason } => {
@@ -1542,6 +1557,22 @@ mod tests {
);
}
#[test]
fn a_large_thumbnail_satisfies_the_grid_class() {
// Zooming out past 256px reloads the window at the grid class. A cell
// already showing the large class has every pixel the small one would
// give, so asking the store for it again is a read for nothing.
let held = hold_thumbnails(
&model_of(&[5], &[5]),
&[5],
&[Some(dr_thumbs::ThumbSize::Large)],
);
let served = already_served(&held, [5i64].into_iter());
assert!(served.contains(&(5, dr_thumbs::ThumbSize::Large)));
assert!(served.contains(&(5, dr_thumbs::ThumbSize::Grid)));
}
#[test]
fn a_cell_whose_class_was_never_recorded_is_fetched_again() {
// Pixels with no class are pixels from before this bookkeeping existed