dtourolle/DarkRoom
1
0
Fork 0
Code Issues 39 Pull Requests Actions Packages Projects Releases 20 Wiki Activity

Replace the per-body base curve with a scene-referred view transform

Browse Source 
The base curve was a five-point spline on the unit square, flat past its
last point: every value above 1.0 left it as the same number, per
channel. Exposure and highlight recovery put values up there, and the
curve threw them away, then handed the result on as though it were
still scene-linear. The six per-body curves were also, by their own
file's account, hand-tuned shapes rather than measurements, and not
enough is known about where they came from to keep them (D19).

In their place, one view transform for every body (FR-DEV-3j): a
log-logistic sigmoid per channel, with the middle channel put back
between the other two so a hue survives the shoulder. Its two free
constants are solved from two conditions rather than set: scene grey
0.13, where the retired default curve put it, lands on display 0.18,
and the scene white four stops above grey lands on 1.0. So a highlight
a stop past sensor saturation still rolls into white, and the midtones
stay within 0.26 EV of the retired default between scene 0.03 and 1.0.
`dr_pipeline::view` holds the CPU reference and the WGSL, and the tests
there are FR-DEV-3j's acceptance criteria.

It is still fixed and still in the fused pass's tail, so a detail stage
still sees rendered values; the next commits make it an operation and
move it after the detail stage. It is skipped for a JPEG, as the base
curve was, and absent from the camera-space tap.

The base curve's database, its lookup and its twelve uniform slots go.
`RawImage` and `DemosaicedImage` lose the field, and the GPU test that
proved a curve reached the shader is replaced by one that renders the
view transform against the CPU reference and shows two highlights above
1.0 still render apart. The JPEG-and-sensor test now asserts the two
differ by exactly the view transform, where before an identity fixture
curve had made them match.
This commit is contained in:
dtourolle
2026-09-27 16:52:53 -04:00
parent db7b84795c
commit 37a6d99dc4
23 changed files with 564 additions and 1442 deletions
Download Patch File Download Diff File Expand all files Collapse all files
core/dr-decode/profiles/base_curves.yaml
-160
View File
@@ -1,160 +0,0 @@
# DarkRoom camera base curves (FR-DEV-3e).
#
# ---------------------------------------------------------------------------
# Adding a body is editing this file. It is not a code change.
# ---------------------------------------------------------------------------
#
# The copy you are reading is compiled into the binary as a floor. At startup
# `dr_decode::base_curve::load` also looks for `base_curves.yaml` in:
#
# 1. $DARKROOM_PROFILES/ (set it while you are tuning)
# 2. $XDG_DATA_HOME/darkroom/profiles/
# or $HOME/.local/share/darkroom/profiles/
#
# and uses the first one it finds *whose `version:` is higher than this one's*.
# So: bump `version`, drop the file in that directory, restart. A body added
# this afternoon renders correctly this afternoon, with no release and no
# rebuild — which is what the requirement asks for, and what makes these
# contributable under the GPL.
#
# The version check runs both ways on purpose. A file older than the built-in
# copy is ignored with a log line, so upgrading DarkRoom cannot silently lose
# curves to a pack somebody downloaded a year ago.
#
# ---------------------------------------------------------------------------
# What the numbers mean
# ---------------------------------------------------------------------------
#
# Five `[x, y]` control points on a monotone spline (Fritsch-Carlson, the same
# one the tone curve widget draws). Both axes are **linear**:
#
# x scene-referred camera RGB after white balance, 1.0 = sensor saturation
# y display-referred linear; the sRGB transfer function is applied later,
# at the end of the shader, so do not pre-apply a gamma here
#
# The identity is y = x, and it is what an unrecognised body gets if `default:`
# is removed. It is also the wrong answer for almost every photograph: linear
# scene data has middle grey at about 13% and a camera JPEG puts it near 18%,
# so an uncurved render is roughly half a stop dark through the midtones and
# has no highlight rolloff at all.
#
# A curve that works has three parts, and it is worth naming them because they
# are what you are actually tuning:
#
# the toe the first span, slope near or below 1. Deep shadows stay
# deep. Lift it and blacks go milky; crush it and shadow
# detail the sensor recorded disappears.
# the midtones the middle spans, slope well above 1. This is the contrast
# and the brightness people read as "the camera's look".
# the shoulder the last span, slope well below 1. Highlights compress
# toward white instead of arriving there and clipping. It is
# the difference between a rolled-off sky and a white hole.
#
# Two invariants are enforced in code and tested, so a mistake here fails the
# build rather than the photograph: x must strictly increase, y must not
# decrease, and everything must lie inside the unit square.
#
# ---------------------------------------------------------------------------
# Honesty about these values
# ---------------------------------------------------------------------------
#
# These are hand-tuned shapes, not measurements. They encode what every camera
# JPEG rendering has in common — the toe/midtone/shoulder structure above —
# plus each maker's well-known house differences: Canon's gentler shoulder and
# warmer-reading midtones, Nikon's slightly higher midtone contrast, Sony's
# flatter and more conservative default, Fujifilm's markedly contrastier
# Provia-derived rendering.
#
# FR-DEV-3e's acceptance criterion is subjective comparison against each body's
# own JPEG, and meeting it properly needs a frame from that body in front of
# you. Where that has not been done, the entry is still much closer to right
# than the identity — which is the bar these have to clear, and do.
version: 1
# The rendering for a body with no entry of its own.
#
# **Deliberately not the identity.** The failure this requirement exists to fix
# is the flat render, and a conservative curve is far closer to right for every
# body than no curve is for any of them. It is gentler than the per-body
# entries below — a shallower midtone and an earlier, softer shoulder — because
# it has to be safe on a sensor nobody has looked at, and the cost of being too
# tame is a photograph that wants a little contrast rather than one that has
# lost its highlights.
default:
points:
- [0.00, 0.000]
- [0.04, 0.043]
- [0.13, 0.175]
- [0.45, 0.690]
- [1.00, 1.000]
bodies:
# Canon. A soft toe and a long, gradual shoulder — the reason Canon files
# are described as forgiving in highlights and a little low in contrast
# straight out of camera.
- make: Canon
model: EOS 6D
points:
- [0.00, 0.000]
- [0.04, 0.045]
- [0.13, 0.190]
- [0.45, 0.720]
- [1.00, 1.000]
- make: Canon
model: EOS R6
points:
- [0.00, 0.000]
- [0.04, 0.044]
- [0.13, 0.195]
- [0.45, 0.730]
- [1.00, 1.000]
# Nikon. A slightly deeper toe and more midtone slope than Canon, which is
# the "punchier out of camera" difference people describe between the two.
- make: Nikon
model: Z 6
points:
- [0.00, 0.000]
- [0.04, 0.038]
- [0.13, 0.200]
- [0.46, 0.750]
- [1.00, 1.000]
- make: Nikon
model: D750
points:
- [0.00, 0.000]
- [0.04, 0.039]
- [0.13, 0.198]
- [0.46, 0.745]
- [1.00, 1.000]
# Sony. The flattest default of the four, and intentionally so — Sony's own
# rendering leaves more headroom than it uses, which is why Sony files are
# the ones people describe as needing the most work.
- make: Sony
model: ILCE-7M3
points:
- [0.00, 0.000]
- [0.04, 0.048]
- [0.13, 0.185]
- [0.44, 0.700]
- [1.00, 1.000]
# Fujifilm. Provia, the default film simulation: a firm toe, the steepest
# midtones here, and a hard shoulder. It is the most distinctive rendering of
# the four and the one where a flat render looks most obviously wrong.
#
# This entry does *not* read the in-RAF film simulation tag — that is
# FR-DEV-3f, and until it lands every Fujifilm file gets the Provia shape
# whatever the camera was set to.
- make: Fujifilm
model: X-T3
points:
- [0.00, 0.000]
- [0.045, 0.040]
- [0.14, 0.215]
- [0.47, 0.775]
- [1.00, 1.000]
core/dr-decode/src/base_curve.rs
-752
View File
@@ -1,752 +0,0 @@
//! TRACES: FR-DEV-3e
//! Base curves — the per-body rendering that turns a correct exposure into a
//! photograph.
//!
//! # What this is for
//!
//! A camera matrix gets the *colours* right and leaves the picture flat. Sensor
//! data is scene-referred and very nearly linear; a print, a screen and a
//! camera's own JPEG are none of those things. Rendering linear data straight
//! out is the dcraw default, and FR-DEV-3e names it precisely: "the flat,
//! poor-skin-tone rendering characteristic of dcraw defaults, which is the
//! documented reason people abandon darktable in the first hour."
//!
//! The fix is a tone curve applied as part of *reading* the file rather than as
//! an edit — a toe, a steep midtone, and a shoulder that rolls highlights off
//! instead of clipping them. Every raw converter has one. Adobe calls it the
//! camera profile's tone curve, darktable calls it the base curve, and the name
//! here follows darktable's because the placement does too: it runs in camera
//! RGB, after white balance and the user's adjustments, immediately before the
//! conversion out to a working space.
//!
//! # Why it is not an edit
//!
//! It never reaches the sidecar and there is no slider for it, for the same
//! reason the EXIF orientation is not an edit (FR-DEV-3h): it is a property of
//! the body that took the frame, not of what anyone decided about the frame.
//! Sidecars are shared between devices and bodies (FR-NC-9), and one camera's
//! rendering must not follow an edit onto another camera's file.
//!
//! # Why it is data
//!
//! FR-DEV-3e requires the profile database to be "versioned independently of
//! the app binary so bodies and curves can be added without a release — and,
//! under D8's GPLv3, contributed by users". So the curves live in
//! `profiles/base_curves.yaml`, a file that is compiled in as a floor and
//! *overridden* by a copy on disk carrying a higher `version:`. Adding a body
//! is adding ten numbers to a YAML file; shipping that body to users is
//! publishing the file. Neither is a code change and neither needs a release.
//!
//! See [`load`] for the search path and [`Curves::body`] for the matching.
use std::path::{Path, PathBuf};
use std::sync::OnceLock;
/// How many control points a base curve has.
///
/// Five, which is not a coincidence: it is what the tone curve widget uses
/// (`dr_pipeline::ops::curve::POINTS`), so the shader evaluates a profile's
/// curve and a photographer's curve through exactly the same spline. A profile
/// author and a photographer dragging a point mean the same thing by it, and
/// the generated shader carries one implementation rather than two that could
/// disagree.
pub const POINTS: usize = 5;
/// TRACES: FR-DEV-3e
/// A base curve: five points on a monotone spline through the unit square.
///
/// `xs` is scene-linear camera RGB, normalised so that 1.0 is the sensor's
/// saturation point. `ys` is display-referred linear — *not* gamma-encoded,
/// because the sRGB transfer function is applied at the very end of the
/// generated shader and applying it twice would wash the image out.
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct BaseCurve {
pub xs: [f32; POINTS],
pub ys: [f32; POINTS],
}
impl BaseCurve {
/// The curve that does nothing — the identity diagonal.
///
/// What an unrecognised body gets if the database carries no default, and
/// what a JPEG gets always: an already-rendered image must not be rendered
/// a second time.
pub const IDENTITY: Self = Self {
xs: [0.0, 0.25, 0.5, 0.75, 1.0],
ys: [0.0, 0.25, 0.5, 0.75, 1.0],
};
/// Whether this curve would leave the image alone.
///
/// The shader is told to skip the stage entirely when it would, so an
/// unprofiled body costs a branch that is uniform across the dispatch
/// rather than a spline evaluation per channel per pixel.
pub fn is_identity(&self) -> bool {
self.xs
.iter()
.zip(self.ys.iter())
.all(|(x, y)| (x - y).abs() < 1e-6)
}
/// Build from raw pairs, rejecting anything that is not a curve.
///
/// A profile file is data a user may have edited, so this is the boundary
/// where "ten numbers" becomes "a curve": the x coordinates must increase,
/// the y coordinates must not decrease, and both must lie in the unit
/// square. A non-monotone x sends the spline's span search backwards and
/// divides by a negative width; a decreasing y inverts tones locally,
/// which reads as a dark halo through smooth gradients rather than as a
/// bad profile.
///
/// Endpoints are not forced to (0,0) and (1,1). A curve that lifts black
/// slightly, or that places the shoulder below white, is a legitimate
/// rendering choice and several bodies make it.
pub fn from_points(points: &[[f32; 2]]) -> Option<Self> {
if points.len() != POINTS {
return None;
}
let mut xs = [0.0f32; POINTS];
let mut ys = [0.0f32; POINTS];
for (i, p) in points.iter().enumerate() {
if !p[0].is_finite() || !p[1].is_finite() {
return None;
}
if !(0.0..=1.0).contains(&p[0]) || !(0.0..=1.0).contains(&p[1]) {
return None;
}
xs[i] = p[0];
ys[i] = p[1];
}
for i in 1..POINTS {
// Strictly increasing in x — the spline divides by the span width.
if xs[i] <= xs[i - 1] {
return None;
}
// Non-decreasing in y. Flat is allowed: a curve that holds a
// highlight range at white is clipping deliberately.
if ys[i] < ys[i - 1] {
return None;
}
}
Some(Self { xs, ys })
}
}
/// One body's entry in the database.
#[derive(Debug, Clone, PartialEq)]
pub struct BodyCurve {
/// The manufacturer, as the file writes it — "Canon", "NIKON CORPORATION".
pub make: String,
/// The model, as the file writes it — "EOS 6D", "ILCE-7M3".
pub model: String,
pub curve: BaseCurve,
}
/// TRACES: FR-DEV-3e
/// The base curve database.
///
/// Versioned as a whole rather than per body, because that is the unit a user
/// downloads and the unit that has to beat the built-in copy. See [`load`].
#[derive(Debug, Clone, PartialEq)]
pub struct Curves {
version: u32,
default: Option<BaseCurve>,
bodies: Vec<BodyCurve>,
}
impl Curves {
/// TRACES: FR-DEV-3e
/// The curve to render a frame from this body with.
///
/// Falls back, in order, to the database's `default:` and then to the
/// identity. **The default is deliberately not the identity**: an
/// unrecognised body rendered flat is the failure this requirement exists
/// to prevent, and a gentle, conservative curve is much closer to right for
/// every body than no curve is for any of them. A body with its own entry
/// gets that instead.
///
/// # What "this body" has to survive
///
/// The same camera names itself three ways depending on which program last
/// touched the file. A native NEF says make "NIKON CORPORATION", model
/// "NIKON Z 6"; rawler's own database cleans that to "Nikon" and "Z 6"; an
/// Adobe-converted DNG keeps the uncleaned pair. A database that had to
/// spell every variant would go stale the first time a maker changed its
/// mind about its own name, so the matching does the folding instead:
///
/// - Case, punctuation and runs of whitespace are flattened, so
/// "ILCE-7M3", "ILCE 7M3" and "ilce-7m3" are one body.
/// - The make is compared on its **first word only**. Every maker's
/// trailing corporate boilerplate — "CORPORATION", "IMAGING CORP" — is
/// noise, and no two camera manufacturers share a first word.
/// - The model is tried both as written and with a leading copy of the
/// make removed, which is what lets one "Canon"/"EOS 6D" entry cover
/// "Canon EOS 6D" as well.
pub fn body(&self, make: &str, model: &str) -> BaseCurve {
let (make, model) = (make_key(make), normalise(model));
// The model with a leading copy of the maker's name removed.
let bare = model.strip_prefix(&format!("{make} ")).unwrap_or(&model);
self.bodies
.iter()
.find(|b| {
let entry_model = normalise(&b.model);
make_key(&b.make) == make && (entry_model == model || entry_model == bare)
})
.map(|b| b.curve)
.or(self.default)
.unwrap_or(BaseCurve::IDENTITY)
}
/// The database version. Higher wins; see [`load`].
pub fn version(&self) -> u32 {
self.version
}
/// How many bodies have their own curve, excluding the default.
pub fn len(&self) -> usize {
self.bodies.len()
}
pub fn is_empty(&self) -> bool {
self.bodies.is_empty()
}
/// Parse a database from YAML.
///
/// Entries that are not curves are dropped with a warning rather than
/// failing the parse. A user-contributed file with one bad body should
/// cost that body's rendering, not every body's — and the alternative is an
/// application that will not open a photograph because somebody typed a
/// comma.
pub fn parse(yaml: &str) -> Result<Self, String> {
let file: File = serde_norway::from_str(yaml).map_err(|e| e.to_string())?;
let default = file.default.and_then(|d| {
BaseCurve::from_points(&d.points).or_else(|| {
log::warn!("base curves: the default entry is not a monotone curve; ignoring it");
None
})
});
let bodies = file
.bodies
.into_iter()
.filter_map(|b| match BaseCurve::from_points(&b.points) {
Some(curve) => Some(BodyCurve {
make: b.make,
model: b.model,
curve,
}),
None => {
log::warn!(
"base curves: {} {} is not a monotone curve; ignoring it",
b.make,
b.model
);
None
}
})
.collect();
Ok(Self {
version: file.version,
default,
bodies,
})
}
}
/// The copy that ships inside the binary.
///
/// A floor, not the answer: [`load`] prefers a newer file on disk. Compiled in
/// so that a fresh install with no profile directory — and every Android build,
/// where there is no such directory to speak of — still renders properly.
const BUILT_IN: &str = include_str!("../profiles/base_curves.yaml");
/// TRACES: FR-DEV-3e
/// The base curve database, loaded once.
///
/// # The search path, and why it is a version comparison
///
/// 1. `$DARKROOM_PROFILES`, a directory, when set. The escape hatch: a profile
/// author iterating on a curve points this at their working copy and does
/// not have to install anything.
/// 2. `$XDG_DATA_HOME/darkroom/profiles/`, else `$HOME/.local/share/darkroom/profiles/`.
/// The same base directory the catalog uses, chosen there for the same
/// reason — it is data, not cache, and must survive a storage sweep.
/// 3. The copy compiled into the binary.
///
/// The first file that parses *and carries a higher `version:` than the
/// built-in copy* wins. The version check is the whole mechanism the
/// requirement asks for, and it runs in both directions:
///
/// - A downloaded pack at version 7 supersedes a binary shipping version 3, so
/// a body added after the release renders correctly with no release.
/// - A stale pack at version 2 does **not** supersede a binary shipping version
/// 3, so upgrading the application cannot silently lose curves to a file
/// somebody downloaded a year ago and forgot.
///
/// Failures are warnings, never errors. A malformed profile file must cost the
/// user their curves, not their photographs.
pub fn load() -> &'static Curves {
static LOADED: OnceLock<Curves> = OnceLock::new();
LOADED.get_or_init(|| {
let built_in = Curves::parse(BUILT_IN).unwrap_or_else(|e| {
// Unreachable in a build that ran its tests — `the_shipped_database_parses`
// asserts exactly this — but a panic here would mean an
// application that cannot open a photograph because of a typo in a
// data file, which is never the right trade.
log::error!("base curves: the built-in database does not parse: {e}");
Curves {
version: 0,
default: None,
bodies: Vec::new(),
}
});
choose(built_in, &search_path())
})
}
/// The version comparison, separated from where the directories come from.
///
/// Split out so it can be tested against real files in a real directory
/// without the process-wide `OnceLock` and the environment `load` reads. The
/// rule this implements is the whole of what FR-DEV-3e asks for, so it is
/// worth being able to state it as a test rather than as a comment.
fn choose(built_in: Curves, dirs: &[PathBuf]) -> Curves {
for dir in dirs {
let path = dir.join("base_curves.yaml");
let Ok(text) = std::fs::read_to_string(&path) else {
continue;
};
match Curves::parse(&text) {
Ok(external) if external.version > built_in.version => {
log::info!(
"base curves: using {} (version {}, {} bodies) over the built-in version {}",
path.display(),
external.version,
external.len(),
built_in.version
);
return external;
}
Ok(external) => log::info!(
"base curves: ignoring {} at version {}; the built-in database is version {}",
path.display(),
external.version,
built_in.version
),
Err(e) => log::warn!("base curves: {} does not parse: {e}", path.display()),
}
}
built_in
}
/// TRACES: FR-DEV-3e
/// The curve for a body, from the loaded database.
///
/// The one call site the decoder needs; everything above is reachable for
/// tests and for a future profile editor.
pub fn for_body(make: &str, model: &str) -> BaseCurve {
load().body(make, model)
}
/// Directories that may hold a `base_curves.yaml`, most specific first.
fn search_path() -> Vec<PathBuf> {
let mut dirs = Vec::new();
if let Some(explicit) = std::env::var_os("DARKROOM_PROFILES") {
dirs.push(PathBuf::from(explicit));
}
// The same resolution `dr_ui::library::catalog_path` uses, and for the
// same reason: this is data a user may have installed, not a cache. It is
// duplicated rather than shared because `dr-decode` sits far below the UI
// and must not acquire a dependency on it to find a directory.
let base = std::env::var_os("XDG_DATA_HOME")
.map(PathBuf::from)
.or_else(|| std::env::var_os("HOME").map(|h| Path::new(&h).join(".local/share")));
if let Some(base) = base {
dirs.push(base.join("darkroom").join("profiles"));
}
dirs
}
/// A manufacturer's first word, folded.
///
/// "NIKON CORPORATION", "Nikon" and "nikon" all become `NIKON`. The corporate
/// suffixes are not information — they appear or not depending on whether the
/// file went through a DNG converter — and no two camera manufacturers share a
/// first word, so nothing is lost by dropping them.
fn make_key(s: &str) -> String {
normalise(s)
.split(' ')
.next()
.unwrap_or_default()
.to_string()
}
/// Fold a make or model into something two files can agree on.
///
/// Upper-cased, with every run of non-alphanumeric characters collapsed to one
/// space and the ends trimmed, so that "ILCE-7M3", "ILCE 7M3" and "ilce-7m3"
/// become one.
fn normalise(s: &str) -> String {
let mut out = String::with_capacity(s.len());
let mut pending_space = false;
for c in s.chars() {
if c.is_ascii_alphanumeric() {
if pending_space && !out.is_empty() {
out.push(' ');
}
pending_space = false;
out.push(c.to_ascii_uppercase());
} else {
pending_space = true;
}
}
out
}
// ---- The on-disk shape, kept apart from the in-memory one ----------------
//
// Deliberately separate types. The file is data a user edits and is allowed to
// be wrong; `Curves` is a parsed database whose every entry is known to be a
// monotone curve. Deriving `Deserialize` on `BaseCurve` directly would delete
// that boundary and let an unchecked five-point array reach the shader.
//
// Unknown fields are **accepted**, which is not laziness. The database is
// versioned independently of the binary and moves in both directions: a pack
// published after this release may carry keys this build has never heard of —
// a hue twist, a look table (FR-DEV-3f) — and it must still deliver its curves
// to an older DarkRoom rather than failing to parse and leaving every body
// flat. `deny_unknown_fields` would trade that for a diagnostic nobody needs.
#[derive(serde::Deserialize)]
struct File {
version: u32,
#[serde(default)]
default: Option<Entry>,
#[serde(default)]
bodies: Vec<BodyEntry>,
}
#[derive(serde::Deserialize)]
struct Entry {
points: Vec<[f32; 2]>,
}
#[derive(serde::Deserialize)]
struct BodyEntry {
make: String,
model: String,
points: Vec<[f32; 2]>,
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn the_shipped_database_parses_and_carries_a_default() {
// The one test that must never be allowed to fail quietly: `load`
// degrades to an empty database rather than panicking, so without this
// a typo in the YAML would ship as "every photograph renders flat"
// rather than as a build failure.
let curves = Curves::parse(BUILT_IN).expect("the shipped database parses");
assert!(curves.version() >= 1);
assert!(!curves.is_empty(), "the database ships bodies");
assert!(
!curves.body("Nobody", "Nothing").is_identity(),
"an unknown body must still get the default rendering"
);
}
#[test]
fn every_shipped_curve_lifts_the_midtones_and_rolls_the_highlights() {
// What makes a base curve a base curve rather than a decoration. If a
// shipped curve failed either half it would be a worse rendering than
// the flat one it replaced, which is the one outcome forbidden.
let curves = Curves::parse(BUILT_IN).expect("parses");
let all = curves
.bodies
.iter()
.map(|b| (format!("{} {}", b.make, b.model), b.curve))
.chain(curves.default.map(|c| ("default".to_string(), c)));
for (name, curve) in all {
// The midtone point sits above the diagonal: a linear midtone is
// roughly a stop and a half darker than any camera renders it.
let mid = 2;
assert!(
curve.ys[mid] > curve.xs[mid],
"{name} does not lift its midtones ({} -> {})",
curve.xs[mid],
curve.ys[mid]
);
// And the last span is shallower than the one before it, which is
// what a shoulder *is*. Without one the curve clips highlights
// harder than the linear rendering did.
let slope =
|i: usize| (curve.ys[i + 1] - curve.ys[i]) / (curve.xs[i + 1] - curve.xs[i]);
assert!(
slope(POINTS - 2) < slope(POINTS - 3),
"{name} has no highlight shoulder"
);
}
}
#[test]
fn a_curve_that_is_not_monotone_is_refused() {
// The profile file is user-editable, so this is a real boundary and
// not a formality. A decreasing y inverts tones locally and shows up
// as a dark halo in a gradient, which reads as a rendering fault
// rather than as a bad profile.
assert_eq!(
BaseCurve::from_points(&[[0.0, 0.0], [0.25, 0.4], [0.5, 0.3], [0.75, 0.8], [1.0, 1.0]]),
None
);
}
#[test]
fn a_curve_whose_x_does_not_advance_is_refused() {
// The spline divides by the span width; a repeated x is a division by
// zero in the shader, which is a NaN pixel rather than an error.
assert_eq!(
BaseCurve::from_points(&[
[0.0, 0.0],
[0.25, 0.3],
[0.25, 0.5],
[0.75, 0.8],
[1.0, 1.0]
]),
None
);
}
#[test]
fn a_curve_of_the_wrong_length_is_refused() {
assert_eq!(BaseCurve::from_points(&[[0.0, 0.0], [1.0, 1.0]]), None);
}
#[test]
fn values_outside_the_unit_square_are_refused() {
// The shader clamps its output at the very end anyway, but a control
// point above 1.0 would put the shoulder outside the range the curve
// is defined over and silently flatten everything below it.
assert_eq!(
BaseCurve::from_points(&[[0.0, 0.0], [0.25, 0.3], [0.5, 1.4], [0.75, 1.5], [1.0, 1.6]]),
None
);
}
#[test]
fn a_body_with_its_own_entry_beats_the_default() {
let curves = Curves::parse(
"version: 2
default:
points: [[0.0, 0.0], [0.25, 0.3], [0.5, 0.6], [0.75, 0.85], [1.0, 1.0]]
bodies:
- make: Canon
model: EOS 6D
points: [[0.0, 0.0], [0.25, 0.35], [0.5, 0.7], [0.75, 0.9], [1.0, 1.0]]
",
)
.expect("parses");
assert_eq!(curves.body("Canon", "EOS 6D").ys[1], 0.35);
assert_eq!(curves.body("Canon", "EOS 5D").ys[1], 0.30);
}
#[test]
fn the_make_may_be_repeated_in_the_model() {
// Canon writes "Canon" as the make and "Canon EOS 6D" as the model;
// rawler's cleaned strings drop the repetition and both reach here.
// One entry has to cover both or half the files on a card miss.
let curves = Curves::parse(
"version: 1
bodies:
- make: Canon
model: EOS 6D
points: [[0.0, 0.0], [0.25, 0.35], [0.5, 0.7], [0.75, 0.9], [1.0, 1.0]]
",
)
.expect("parses");
assert_eq!(curves.body("Canon", "Canon EOS 6D").ys[1], 0.35);
assert_eq!(curves.body("Canon", "EOS 6D").ys[1], 0.35);
assert_eq!(curves.body("CANON", "eos 6d").ys[1], 0.35);
}
#[test]
fn a_corporate_suffix_does_not_hide_a_body() {
// The same Z 6 arrives as "Nikon"/"Z 6" from rawler's camera database
// and as "NIKON CORPORATION"/"NIKON Z 6" from a DNG converted out of
// the same file. Both must find the entry, or converting a file to
// DNG would silently change how it renders.
let curves = Curves::parse(
"version: 1
bodies:
- make: Nikon
model: Z 6
points: [[0.0, 0.0], [0.25, 0.35], [0.5, 0.7], [0.75, 0.9], [1.0, 1.0]]
",
)
.expect("parses");
assert_eq!(curves.body("Nikon", "Z 6").ys[1], 0.35);
assert_eq!(curves.body("NIKON CORPORATION", "NIKON Z 6").ys[1], 0.35);
}
#[test]
fn punctuation_and_spacing_do_not_decide_whether_a_body_is_known() {
let curves = Curves::parse(
"version: 1
bodies:
- make: Sony
model: ILCE-7M3
points: [[0.0, 0.0], [0.25, 0.35], [0.5, 0.7], [0.75, 0.9], [1.0, 1.0]]
",
)
.expect("parses");
assert_eq!(curves.body("SONY", "ILCE 7M3").ys[1], 0.35);
assert_eq!(curves.body("sony", "ilce-7m3").ys[1], 0.35);
}
#[test]
fn one_bad_entry_does_not_cost_the_rest() {
// A user-contributed file with one typo should cost that body's
// rendering, not every body's.
let curves = Curves::parse(
"version: 1
bodies:
- make: Broken
model: Body
points: [[0.0, 0.0], [0.25, 0.9], [0.5, 0.1], [0.75, 0.9], [1.0, 1.0]]
- make: Canon
model: EOS 6D
points: [[0.0, 0.0], [0.25, 0.35], [0.5, 0.7], [0.75, 0.9], [1.0, 1.0]]
",
)
.expect("parses");
assert_eq!(curves.len(), 1);
assert_eq!(curves.body("Canon", "EOS 6D").ys[1], 0.35);
assert!(curves.body("Broken", "Body").is_identity());
}
#[test]
fn a_pack_from_the_future_still_delivers_its_curves() {
// The database is versioned independently of the binary, so a pack
// published after this build may carry keys this build has never heard
// of. It must still hand over the curves it does understand — failing
// the parse would leave every body flat, which is the exact failure
// FR-DEV-3e exists to prevent, delivered by the mechanism meant to
// prevent it.
let curves = Curves::parse(
"version: 9
look_table: ambitious
bodies:
- make: Canon
model: EOS 6D
hue_twist: [1, 2, 3]
points: [[0.0, 0.0], [0.25, 0.35], [0.5, 0.7], [0.75, 0.9], [1.0, 1.0]]
",
)
.expect("an unfamiliar key must not fail the parse");
assert_eq!(curves.version(), 9);
assert_eq!(curves.body("Canon", "EOS 6D").ys[1], 0.35);
}
#[test]
fn an_unknown_body_with_no_default_gets_the_identity() {
// Graceful fallback, stated as a property: never worse than a flat
// render, and never a curve tuned for somebody else's sensor when the
// database declines to offer one.
let curves = Curves::parse("version: 1\nbodies: []\n").expect("parses");
assert!(curves.body("Nobody", "Nothing").is_identity());
}
/// A directory holding one `base_curves.yaml`, unique to the caller.
fn a_pack_dir(name: &str, yaml: &str) -> PathBuf {
let dir = std::env::temp_dir().join(format!("darkroom-base-curves-{name}"));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).expect("a writable temp directory");
std::fs::write(dir.join("base_curves.yaml"), yaml).expect("write");
dir
}
const A_CANON_ENTRY: &str = "bodies:
- make: Canon
model: EOS 6D
points: [[0.0, 0.0], [0.25, 0.42], [0.5, 0.7], [0.75, 0.9], [1.0, 1.0]]
";
#[test]
fn a_newer_pack_on_disk_supersedes_the_built_in_database() {
// **This is the requirement.** FR-DEV-3e asks for a profile database
// versioned independently of the app binary "so bodies and curves can
// be added without a release". A file with a higher version, dropped
// in the profile directory, is what that means in practice.
let built_in = Curves::parse(BUILT_IN).expect("parses");
let newer = format!("version: {}\n{A_CANON_ENTRY}", built_in.version() + 1);
let dir = a_pack_dir("newer", &newer);
let chosen = choose(built_in.clone(), &[dir]);
assert_eq!(chosen.version(), built_in.version() + 1);
assert_eq!(chosen.body("Canon", "EOS 6D").ys[1], 0.42);
}
#[test]
fn a_stale_pack_does_not_survive_an_upgrade() {
// The other direction, and the one that protects the user. Somebody
// downloads a pack, a release later ships better curves for the same
// bodies, and the forgotten file must not quietly hold the application
// back at last year's rendering.
let built_in = Curves::parse(BUILT_IN).expect("parses");
let stale = format!("version: {}\n{A_CANON_ENTRY}", built_in.version());
let dir = a_pack_dir("stale", &stale);
let chosen = choose(built_in.clone(), &[dir]);
assert_eq!(chosen.version(), built_in.version());
assert_ne!(
chosen.body("Canon", "EOS 6D").ys[1],
0.42,
"an equal version must not displace the built-in database"
);
}
#[test]
fn a_broken_pack_costs_the_curves_and_not_the_photographs() {
// A malformed profile file must degrade to the built-in database, not
// to an error. The user came here to look at a photograph.
let built_in = Curves::parse(BUILT_IN).expect("parses");
let dir = a_pack_dir("broken", "version: [this is not a number\n");
let chosen = choose(built_in.clone(), &[dir]);
assert_eq!(chosen.version(), built_in.version());
assert_eq!(chosen.len(), built_in.len());
}
#[test]
fn a_directory_with_no_pack_in_it_is_simply_skipped() {
// The ordinary case on every machine: the search path exists, the file
// does not. It must not be a warning, an error, or a slow path.
let built_in = Curves::parse(BUILT_IN).expect("parses");
let missing = std::env::temp_dir().join("darkroom-base-curves-nothing-here");
let _ = std::fs::remove_dir_all(&missing);
assert_eq!(choose(built_in.clone(), &[missing]), built_in);
}
#[test]
fn the_identity_is_recognised_as_doing_nothing() {
assert!(BaseCurve::IDENTITY.is_identity());
assert!(!Curves::parse(BUILT_IN)
.expect("parses")
.body("Canon", "EOS 6D")
.is_identity());
}
}
core/dr-decode/src/lib.rs
-25
View File
@@ -16,14 +16,12 @@
//! second decoder can be put behind them without changing any of them
//! (FR-RAW-2). [`Rawler`] is the one that ships; [`default`] hands it out.
pub mod base_curve;
mod decoder;
mod error;
mod locate;
mod preview;
pub mod profile;
pub use base_curve::BaseCurve;
pub use decoder::{default, Decoder, Rawler};
pub use error::DecodeError;
pub use locate::{
@@ -125,19 +123,6 @@ pub struct RawImage {
/// for the light the frame was shot under; see [`profile::CameraProfile`].
pub color_matrix: Option<[f32; 9]>,
/// TRACES: FR-DEV-3e
/// The per-body rendering curve, the other half of the camera profile.
///
/// The matrix above decides what the colours *are*; this decides what the
/// picture looks like. Carried on the decoded image rather than looked up
/// downstream because this is the only point in the system that knows
/// which body took the frame, and because it is not an edit: it belongs to
/// the file in the same way the masked-photosite crop does, and must never
/// reach a sidecar (FR-NC-9).
///
/// [`BaseCurve::IDENTITY`] for an unknown body with no default in the
/// database, which renders exactly as this decoder did before profiles
/// existed.
pub base_curve: BaseCurve,
/// The usable region of `data`, excluding masked and border photosites.
pub crop: CropRect,
/// TRACES: FR-MRG-3
@@ -592,15 +577,6 @@ fn decode_unguarded(bytes: &[u8]) -> Result<RawImage, DecodeError> {
profile.as_ref().map(|p| p.xyz_to_cam()).as_ref(),
);
// The rendering half of the profile (FR-DEV-3e). rawler's cleaned strings
// are preferred where it has them — they are what the shipped database is
// written against — and the matching folds the variants either way, so a
// DNG naming the same body differently still finds its curve.
let base_curve = base_curve::for_body(
image.camera.clean_make.as_str(),
image.camera.clean_model.as_str(),
);
// TRACES: FR-MRG-3
// A linear DNG — three samples per pixel, no colour filter array — is a
// composite this application wrote (or any other demosaiced DNG). It
@@ -683,7 +659,6 @@ fn decode_unguarded(bytes: &[u8]) -> Result<RawImage, DecodeError> {
.unwrap_or(u16::MAX),
wb_coeffs,
color_matrix,
base_curve,
samples_per_pixel,
profile,
make: image.camera.clean_make.clone(),
core/dr-decode/src/profile.rs
+4 -2
View File
@@ -6,8 +6,10 @@
//! colour needs two things the file cannot supply on its own: a **matrix**
//! saying how this sensor's three responses relate to the CIE observer, and a
//! **rendering** saying what to do with the resulting scene-referred values so
//! that a photograph looks like a photograph. This module supplies the first
//! and looks up the second ([`crate::base_curve`]).
//! that a photograph looks like a photograph. This module supplies the first.
//! The second is not the body's: since D19 it is the pipeline's view
//! transform (FR-DEV-3j), one for every camera, and the per-body base curves
//! that used to be looked up here are retired.
//!
//! # What is extracted, and from where
//!
core/dr-film/src/profile.rs
+3 -3
View File
@@ -2,9 +2,9 @@
//!
//! # Why it is data
//!
//! The same argument `dr_decode::base_curve` makes for camera bodies, and for
//! the same requirement: under the GPLv3 a stock should be contributable
//! without a release. A profile is three tables and a handful of facts, all of
//! Under the GPLv3 a stock should be contributable without a release (the
//! argument the retired per-body base curves made for camera bodies, before
//! D19). A profile is three tables and a handful of facts, all of
//! them published in the manufacturer's datasheet, so adding a stock is adding
//! a file — not a code change, not a shader, and not a new operation.
//!
core/dr-gpu/src/adjust.rs
+21 -41
View File
@@ -34,16 +34,6 @@ use crate::{DemosaicedImage, GpuContext, GpuError};
/// reads them.
const RESERVED_FIELDS: usize = dr_pipeline::RESERVED_UNIFORM_FIELDS;
/// TRACES: FR-DEV-3e
/// The two crates must agree on how many points a base curve has.
///
/// `dr-decode` reads them from the profile database and `dr-pipeline` declares
/// the uniform slots; this file is the only place the two meet, and it packs
/// them by index. A disagreement would not fail to compile — it would upload a
/// curve with a point missing or a stale float in it, which renders as a
/// plausible-looking wrong tone response. Cheaper to catch here, at build time.
const _: () = assert!(dr_decode::base_curve::POINTS == dr_pipeline::BASE_CURVE_POINTS);
/// Runs composed operation chains against demosaiced images.
pub struct AdjustPass {
ctx: GpuContext,
@@ -1275,22 +1265,6 @@ impl AdjustPass {
// runs. See `DemosaicedImage::is_non_linear`.
let non_linear = if source.is_non_linear() { 1.0 } else { 0.0 };
uniforms[12..16].copy_from_slice(&[wb[0], wb[1], wb[2], non_linear]);
// TRACES: FR-DEV-3e
// The camera profile's base curve, packed the way the generated block
// declares it: four x, four y, then the fifth point and the flag. The
// flag is what lets one compiled shader serve a profiled body and an
// unprofiled one, so the pipeline cache is not split in two by which
// camera took the frame.
//
// Written here rather than at the call site so that *both* callers —
// the plain render and the masked one — carry the profile. Filling it
// at one of them was how the two halves of this merge each had it.
let curve = source.base_curve();
let on = if curve.is_identity() { 0.0 } else { 1.0 };
let b = dr_pipeline::BASE_CURVE_UNIFORM_OFFSET;
uniforms[b..b + 4].copy_from_slice(&curve.xs[0..4]);
uniforms[b + 4..b + 8].copy_from_slice(&curve.ys[0..4]);
uniforms[b + 8..b + 12].copy_from_slice(&[curve.xs[4], curve.ys[4], on, 0.0]);
uniforms
}
@@ -1478,8 +1452,6 @@ impl AdjustPass {
uniforms[4..8].copy_from_slice(&[0.0, 1.0, 0.0, 0.0]);
uniforms[8..12].copy_from_slice(&[0.0, 0.0, 1.0, 0.0]);
uniforms[12..16].copy_from_slice(&[1.0, 1.0, 1.0, non_linear]);
let b = dr_pipeline::BASE_CURVE_UNIFORM_OFFSET;
uniforms[b + 10] = 0.0;
let params_buf = self
.ctx
@@ -1707,7 +1679,7 @@ pub(crate) fn numbered(src: &str) -> String {
#[cfg(test)]
mod tests {
use super::*;
use dr_decode::{BaseCurve, CfaPattern, CropRect, RawImage};
use dr_decode::{CfaPattern, CropRect, RawImage};
use dr_pipeline::ops::{colour_mixer, exposure, saturation};
use dr_pipeline::EditGraph;
// For `Operation::detail`, which is how `the_whole_chain_at_once_compiles`
@@ -1745,7 +1717,6 @@ mod tests {
// Identity, so the test reasons about the operations alone
// rather than about a camera's colour response.
color_matrix: Some([1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]),
base_curve: BaseCurve::IDENTITY,
samples_per_pixel: 1,
profile: None,
make: String::new(),
@@ -1949,7 +1920,6 @@ mod tests {
white_level: 16383,
wb_coeffs: [1.0, 1.0, 1.0, 1.0],
color_matrix: Some([1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]),
base_curve: BaseCurve::IDENTITY,
samples_per_pixel: 1,
profile: None,
make: String::new(),
@@ -2538,7 +2508,6 @@ mod tests {
white_level: 16383,
wb_coeffs: [1.0, 1.0, 1.0, 1.0],
color_matrix: Some([1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]),
base_curve: BaseCurve::IDENTITY,
samples_per_pixel: 1,
profile: None,
make: String::new(),
@@ -2642,7 +2611,6 @@ mod tests {
white_level: 16383,
wb_coeffs: [1.0, 1.0, 1.0, 1.0],
color_matrix: Some([1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]),
base_curve: BaseCurve::IDENTITY,
samples_per_pixel: 1,
profile: None,
make: String::new(),
@@ -3227,11 +3195,16 @@ mod tests {
}
#[test]
fn a_jpeg_and_sensor_data_agree_on_the_same_scene_value() {
// The two producers must be interchangeable. A mid-grey that is
// linearly 0.216 (sRGB 128) arriving as sensor data and as a JPEG
// must render the same, or an edit would mean different things
// depending on which decoder opened the file.
fn a_jpeg_and_sensor_data_differ_by_exactly_the_view_transform() {
// TRACES: FR-DEV-3j
// The two producers must be interchangeable up to the rendering. A
// mid-grey that is linearly 0.216 (sRGB 128) arriving as sensor data
// is scene-referred and goes through the view transform; arriving as
// a JPEG it is already a rendering and must come out as it went in.
// Before D19 the fixture's identity base curve made both unrendered
// and this asserted they matched; what it guards is unchanged — the
// linearisation of each agrees — but the rendering between them is
// now always there for sensor data.
let Some(ctx) = ctx() else { return };
let mut pass = AdjustPass::new(&ctx);
let shader = EditGraph::default_chain().compose();
@@ -3250,11 +3223,18 @@ mod tests {
read_centre(&ctx, t)
};
let delta = (i32::from(from_sensor[0]) - i32::from(from_jpeg[0])).abs();
let scene = 3537.0 / 16383.0;
let viewed = dr_pipeline::view::Sigmoid::default_curve().channel(scene);
let expected = (dr_types::Transfer::Srgb.encode(viewed) * 255.0).round() as i32;
let delta = (i32::from(from_sensor[0]) - expected).abs();
assert!(
delta <= 3,
"the same scene value rendered {from_sensor:?} from sensor data \
and {from_jpeg:?} from a JPEG"
"sensor data rendered {from_sensor:?}, expected about {expected}"
);
let delta = (i32::from(from_jpeg[0]) - 128).abs();
assert!(
delta <= 3,
"a JPEG was rendered again: {from_jpeg:?} from sRGB 128"
);
}
}
core/dr-gpu/src/demosaic.rs
+1 -27
View File
@@ -10,7 +10,7 @@
//! pass over this texture; it does not re-demosaic, which is what keeps the
//! interaction budget (NFR-P9) reachable on a 24 MP file.
use dr_decode::{BaseCurve, CfaPattern, RawImage};
use dr_decode::{CfaPattern, RawImage};
use wgpu::util::DeviceExt;
use crate::{GpuContext, GpuError};
@@ -109,16 +109,6 @@ pub struct DemosaicedImage {
color_matrix: [f32; 9],
/// As-shot white balance, the neutral starting point for the WB control.
as_shot_wb: [f32; 3],
/// TRACES: FR-DEV-3e
/// The camera profile's rendering curve, carried through for the adjust
/// pass exactly as `color_matrix` is.
///
/// It rides on the image rather than on the edit graph because it is not
/// an edit: it belongs to the body that took the frame, the way the
/// masked-photosite crop and the EXIF orientation do, and a sidecar shared
/// between two bodies must never carry one body's rendering onto the
/// other's file (FR-NC-9).
base_curve: BaseCurve,
/// Whether the texture holds gamma-encoded rather than linear values.
non_linear: bool,
/// Which upload this is, unique for the life of the process. See
@@ -165,15 +155,6 @@ impl DemosaicedImage {
self.color_matrix
}
/// TRACES: FR-DEV-3e
/// The camera profile's base curve, as five `(x, y)` points.
///
/// [`BaseCurve::IDENTITY`] where the body is unprofiled or the source was
/// never raw, in which case the adjust pass skips the stage entirely.
pub fn base_curve(&self) -> BaseCurve {
self.base_curve
}
/// As-shot white balance multipliers, green-normalised.
///
/// The white balance control is expressed *relative* to these, so its
@@ -291,7 +272,6 @@ impl DemosaicedImage {
// camera's base curve baked in by the camera; applying one again
// would render the rendering, crushing the shadows and flattening
// the highlights of an image that was already finished.
base_curve: BaseCurve::IDENTITY,
non_linear: true,
id: next_image_id(),
})
@@ -368,7 +348,6 @@ impl DemosaicedImage {
height,
color_matrix: raw.color_matrix.unwrap_or(IDENTITY_3X3),
as_shot_wb: [raw.wb_coeffs[0], raw.wb_coeffs[1], raw.wb_coeffs[2]],
base_curve: raw.base_curve,
non_linear: false,
id: next_image_id(),
})
@@ -784,7 +763,6 @@ impl Demosaicer {
// Whatever the profile database had for this body (FR-DEV-3e),
// resolved at decode because that is the only place the make and
// model are known.
base_curve: raw.base_curve,
// Sensor data is linear by construction — the demosaic shader
// normalises against black and white levels and applies no
// transfer function.
@@ -1284,7 +1262,6 @@ mod tests {
white_level: white,
wb_coeffs: [1.0, 1.0, 1.0, 1.0],
color_matrix: None,
base_curve: BaseCurve::IDENTITY,
samples_per_pixel: 1,
profile: None,
make: String::new(),
@@ -1400,7 +1377,6 @@ mod tests {
white_level: white,
wb_coeffs: [1.0, 1.0, 1.0, 1.0],
color_matrix: None,
base_curve: BaseCurve::IDENTITY,
samples_per_pixel: 1,
profile: None,
make: String::new(),
@@ -1697,7 +1673,6 @@ mod tests {
white_level: 16383,
wb_coeffs: [1.0, 1.0, 1.0, 1.0],
color_matrix: None,
base_curve: BaseCurve::IDENTITY,
samples_per_pixel: 1,
profile: None,
make: String::new(),
@@ -1782,7 +1757,6 @@ mod tests {
1.0,
],
color_matrix: None,
base_curve: BaseCurve::IDENTITY,
samples_per_pixel: 1,
profile: None,
make: String::new(),
core/dr-gpu/tests/base_curve.rs
-181
View File
@@ -1,181 +0,0 @@
//! TRACES: FR-DEV-3e
//! The camera profile's base curve, end to end on a device.
//!
//! The unit tests either side of this one check halves. `dr-decode` asserts
//! that the shipped database parses and that every curve in it lifts its
//! midtones; `dr-pipeline` asserts that the generated WGSL evaluates a curve
//! in the right place. Neither would notice if the two agreed with each other
//! and both were wrong — a curve packed into the wrong uniform slots, or a
//! flag read from the wrong component, satisfies both and renders nothing.
//!
//! So this renders real pixels twice, once with a profiled body's curve and
//! once with the identity, and asserts the difference is the one a base curve
//! is for: midtones lifted, black still black, white still white.
use dr_decode::{BaseCurve, CfaPattern, CropRect, RawImage};
use dr_gpu::{AdjustPass, Demosaicer, GpuContext};
use dr_pipeline::EditGraph;
const SIZE: u32 = 16;
fn ctx() -> Option<GpuContext> {
pollster::block_on(GpuContext::new_headless()).ok()
}
/// A flat RGGB frame at `level` out of 65535, carrying `curve`.
///
/// Every photosite the same value, so the demosaic result is a uniform grey
/// and the only thing that can move a pixel is the curve. The colour matrix is
/// the identity and the balance is neutral for the same reason: this test is
/// about one stage, and a real body's matrix would make every assertion below
/// a statement about that body instead.
fn flat_raw(level: u16, curve: BaseCurve) -> RawImage {
RawImage {
width: SIZE,
height: SIZE,
data: vec![level; (SIZE * SIZE) as usize],
cfa_pattern: CfaPattern::Rggb,
black_level: [0; 4],
white_level: u16::MAX,
wb_coeffs: [1.0, 1.0, 1.0, 1.0],
color_matrix: Some([1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]),
base_curve: curve,
samples_per_pixel: 1,
profile: None,
make: String::new(),
model: String::new(),
crop: CropRect {
x: 0,
y: 0,
width: SIZE,
height: SIZE,
},
}
}
/// Render a neutral edit over a flat frame and return the centre pixel's red.
///
/// The centre rather than a corner: a demosaic has to invent its edges, and
/// the interpolated border of a 16×16 frame is not where anyone should be
/// reading a tone off.
fn rendered_level(ctx: &GpuContext, level: u16, curve: BaseCurve) -> u8 {
let raw = flat_raw(level, curve);
let source = Demosaicer::new(ctx)
.expect("demosaicer")
.run(&raw)
.expect("demosaic");
let shader = EditGraph::default_chain().compose();
let mut adjust = AdjustPass::new(ctx);
adjust.render(&source, &shader, SIZE, SIZE).expect("render");
let (pixels, _, _) = adjust.export_pixels().expect("readback");
let centre = ((SIZE / 2) * SIZE + SIZE / 2) * 4;
pixels[centre as usize]
}
/// The Canon EOS 6D's curve, from the shipped profile database.
///
/// Looked up by name rather than written out, so this also asserts the thing
/// no other test can: that a curve travels from the YAML, through the body
/// match, onto the decoded image and into the uniform block that the shader
/// actually reads.
fn six_d() -> BaseCurve {
let curve = dr_decode::base_curve::for_body("Canon", "EOS 6D");
assert!(
!curve.is_identity(),
"the shipped database must have a curve for the EOS 6D"
);
curve
}
#[test]
fn a_profiled_body_renders_brighter_midtones_than_a_flat_one() {
// **The whole requirement, in one assertion.** A linear midtone renders
// roughly half a stop dark, which is the flat, lifeless look FR-DEV-3e
// exists to get away from. If the curve did not reach the shader — wrong
// slot, wrong flag, wrong stage — this is the only test that would fail.
let Some(ctx) = ctx() else {
eprintln!("skipping: no GPU adapter");
return;
};
// 13% of full scale: roughly where a camera places middle grey, leaving
// about two and a half stops of highlight headroom above it.
let level = (0.13 * 65535.0) as u16;
let flat = rendered_level(&ctx, level, BaseCurve::IDENTITY);
let profiled = rendered_level(&ctx, level, six_d());
assert!(
profiled > flat + 8,
"the profile lifted middle grey from {flat} only to {profiled}"
);
}
#[test]
fn the_curve_leaves_black_black_and_white_white() {
// A base curve renders the range between the endpoints; it must not move
// the endpoints themselves. A curve that lifted black would put a grey
// veil over every night photograph, and one that pulled white down would
// make a correctly exposed frame look underexposed.
let Some(ctx) = ctx() else {
eprintln!("skipping: no GPU adapter");
return;
};
let curve = six_d();
assert_eq!(rendered_level(&ctx, 0, curve), 0, "black moved");
assert_eq!(rendered_level(&ctx, u16::MAX, curve), 255, "white moved");
}
#[test]
fn an_unprofiled_body_renders_exactly_as_it_did_before_profiles_existed() {
// The graceful fallback, asserted as a number rather than as a promise.
// With no curve the pipeline must still be a pass-through: black level
// out, white level in, sRGB encoding on the way to the screen and nothing
// else. "Never worse than today" is the one property this change was not
// allowed to trade away, and the way it would break is silently — a flag
// read from the wrong component would apply a curve nobody asked for.
let Some(ctx) = ctx() else {
eprintln!("skipping: no GPU adapter");
return;
};
for level in [0u16, 4_000, 8_520, 32_768, 60_000, u16::MAX] {
let scene = f32::from(level) / f32::from(u16::MAX);
let expected = (dr_types::Transfer::Srgb.encode(scene) * 255.0).round() as i32;
let got = i32::from(rendered_level(&ctx, level, BaseCurve::IDENTITY));
// Two 8-bit steps: the texture holding the demosaiced frame is
// `Rgba16Float`, so a value round-trips through eleven mantissa bits
// before it is encoded. That is well under one step at any level, and
// the tolerance is for the rounding either side of it rather than for
// the transform being approximate.
assert!(
(got - expected).abs() <= 2,
"raw {level} rendered as {got}, expected about {expected}"
);
}
}
#[test]
fn the_curve_is_monotone_through_the_whole_range() {
// The property the spline's tangent limiting exists to guarantee, checked
// where it actually matters: on the device, through the real uniform
// packing. A curve that dipped anywhere would put a dark band across a
// smooth gradient — a sky, most visibly — and it would read as a
// rendering fault rather than as a bad profile.
let Some(ctx) = ctx() else {
eprintln!("skipping: no GPU adapter");
return;
};
let curve = six_d();
let mut previous = 0u8;
for step in 0..=16u32 {
let level = (step * 65535 / 16) as u16;
let value = rendered_level(&ctx, level, curve);
assert!(
value >= previous,
"the curve fell from {previous} to {value} at raw level {level}"
);
previous = value;
}
}
core/dr-gpu/tests/film_sim.rs
+1 -2
View File
@@ -15,7 +15,7 @@
//! model is checked against the reference, and the shader is checked against
//! the CPU model.
use dr_decode::{BaseCurve, CfaPattern, CropRect, RawImage};
use dr_decode::{CfaPattern, CropRect, RawImage};
use dr_film::bake::{bake, Recipe, Settings};
use dr_gpu::{AdjustPass, Demosaicer, GpuContext, LabelField, MaskPass};
use dr_pipeline::mask::{MaskLayer, MaskSource};
@@ -48,7 +48,6 @@ fn flat_raw(level: u16) -> RawImage {
// Off deliberately: a film replaces the camera's rendering, and
// leaving a curve here would test the suppression rather than the
// film. `dr-pipeline` asserts the suppression on the generated source.
base_curve: BaseCurve::IDENTITY,
samples_per_pixel: 1,
profile: None,
make: String::new(),
core/dr-gpu/tests/hot_pixels.rs
+1 -2
View File
@@ -6,7 +6,7 @@
//! anything: the repair happens on the mosaic, and what a photographer would
//! see of a defect it missed is the coloured cross the demosaic makes of it.
use dr_decode::{BaseCurve, CfaPattern, CropRect, RawImage};
use dr_decode::{CfaPattern, CropRect, RawImage};
use dr_gpu::{AdjustPass, Demosaicer, GpuContext};
use dr_pipeline::EditGraph;
@@ -32,7 +32,6 @@ fn frame(pattern: CfaPattern, level: u16, set: &[(u32, u32, u16)]) -> RawImage {
white_level: WHITE,
wb_coeffs: [1.0, 1.0, 1.0, 1.0],
color_matrix: Some([1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]),
base_curve: BaseCurve::IDENTITY,
samples_per_pixel: 1,
profile: None,
make: String::new(),
core/dr-gpu/tests/view_transform.rs
+129
View File
@@ -0,0 +1,129 @@
//! TRACES: FR-DEV-3j | FR-DEV-2
//! The view transform, end to end on a device.
//!
//! `dr-pipeline` checks the curve on the CPU and that the composer emits it in
//! the right place. Neither would notice a shader that disagreed with the CPU
//! reference, or a clamp somewhere upstream that made two highlights the same
//! number before the curve ever saw them — which is exactly what the retired
//! base curve did, and why D19 exists. So this renders real pixels.
use dr_decode::{CfaPattern, CropRect, RawImage};
use dr_gpu::{AdjustPass, Demosaicer, GpuContext};
use dr_pipeline::view::Sigmoid;
use dr_pipeline::EditGraph;
const SIZE: u32 = 16;
fn ctx() -> Option<GpuContext> {
pollster::block_on(GpuContext::new_headless()).ok()
}
/// A flat RGGB frame at `level` out of 65535, with an identity matrix and a
/// neutral balance, so the only things that move a pixel are the edit and the
/// view transform.
fn flat_raw(level: u16) -> RawImage {
RawImage {
width: SIZE,
height: SIZE,
data: vec![level; (SIZE * SIZE) as usize],
cfa_pattern: CfaPattern::Rggb,
black_level: [0; 4],
white_level: u16::MAX,
wb_coeffs: [1.0, 1.0, 1.0, 1.0],
color_matrix: Some([1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]),
samples_per_pixel: 1,
profile: None,
make: String::new(),
model: String::new(),
crop: CropRect {
x: 0,
y: 0,
width: SIZE,
height: SIZE,
},
}
}
/// Render `graph` over a flat frame and return the centre pixel's red.
///
/// The centre rather than a corner: a demosaic has to invent its edges.
fn rendered(ctx: &GpuContext, level: u16, graph: &EditGraph) -> u8 {
let source = Demosaicer::new(ctx)
.expect("demosaicer")
.run(&flat_raw(level))
.expect("demosaic");
let shader = graph.compose();
let mut adjust = AdjustPass::new(ctx);
adjust.render(&source, &shader, SIZE, SIZE).expect("render");
let (pixels, _, _) = adjust.export_pixels().expect("readback");
let centre = ((SIZE / 2) * SIZE + SIZE / 2) * 4;
pixels[centre as usize]
}
#[test]
fn the_shader_agrees_with_the_cpu_reference() {
// TRACES: FR-DEV-3j
let Some(ctx) = ctx() else {
eprintln!("skipping: no GPU adapter");
return;
};
let curve = Sigmoid::default_curve();
let graph = EditGraph::default_chain();
for level in [0u16, 500, 4_000, 8_520, 32_768, 60_000, u16::MAX] {
let scene = f32::from(level) / f32::from(u16::MAX);
let display = curve.channel(scene).min(1.0);
let expected = (dr_types::Transfer::Srgb.encode(display) * 255.0).round() as i32;
let got = i32::from(rendered(&ctx, level, &graph));
// Two 8-bit steps, for the `Rgba16Float` intermediate and the
// rounding either side of the encode.
assert!(
(got - expected).abs() <= 2,
"raw {level} rendered as {got}, expected about {expected}"
);
}
}
#[test]
fn highlights_above_one_stay_distinct() {
// TRACES: FR-DEV-2 | FR-DEV-3j
// The failure D19 names first. Two stops of exposure put these two
// frames at 1.0 and 1.5 of sensor saturation. The base curve was flat
// past 1.0, so both rendered as the same white; the view transform's
// shoulder still separates them.
let Some(ctx) = ctx() else {
eprintln!("skipping: no GPU adapter");
return;
};
let mut graph = EditGraph::default_chain();
graph.set_param(
dr_pipeline::ops::exposure::ID,
dr_pipeline::ops::exposure::EXPOSURE,
2.0,
);
let lower = rendered(&ctx, u16::MAX / 4, &graph);
let upper = rendered(&ctx, (u16::MAX / 8) * 3, &graph);
assert!(
upper > lower,
"scene 1.0 rendered {lower} and scene 1.5 rendered {upper}"
);
assert!(upper < 255, "scene 1.5 is below the default white point");
}
#[test]
fn the_rendering_is_monotone_through_the_whole_range() {
// TRACES: FR-DEV-3j
// A dip anywhere puts a dark band across a smooth gradient — a sky, most
// visibly.
let Some(ctx) = ctx() else {
eprintln!("skipping: no GPU adapter");
return;
};
let graph = EditGraph::default_chain();
let mut last = 0u8;
for step in 0..=32u32 {
let level = (step * u32::from(u16::MAX) / 32) as u16;
let got = rendered(&ctx, level, &graph);
assert!(got >= last, "raw {level} rendered {got}, below {last}");
last = got;
}
}
core/dr-pipeline/ops/README.md
+11 -33
View File
@@ -294,40 +294,18 @@ in raw pixels is a different photograph on screen and in the exported file.
## What is not a node, and why
Three things act on every pixel and are deliberately not in this directory:
the as-shot white balance, the camera matrix, and the **base curve**
(FR-DEV-3e). They are emitted by [`../src/operation.rs`](../src/operation.rs)
into the composed shader's fixed preamble, around the block of nodes.
the as-shot white balance, the camera matrix, and the **view transform**
(FR-DEV-3j). They are emitted by [`../src/operation.rs`](../src/operation.rs)
into the composed shader around the block of nodes.
The test is not "does it transform a colour" — all three do. It is **whose
decision is it**. A node is something a photographer chose: it has parameters,
it moves off a neutral, it lands in the sidecar, it can be undone. These three
are properties of the *file*, at the same standing as the masked-photosite crop
(FR-RAW-3) and the stored orientation (FR-DEV-3h). Nobody chose the sensor's
green sensitivity or the body's rendering; they are what reading the file
correctly means.
Making the base curve a node would have said the opposite in four places at
once. It would have appeared in the develop panel as a control, so an
unprofiled body would show a slider that does nothing. Its values would have
gone into the sidecar, and sidecars are shared between devices and bodies
(FR-NC-9) — one camera's rendering would follow an edit onto another camera's
file. Its neutral would have had to be "the identity", so a profiled body would
open reporting itself modified. And there is no seam through which a node could
learn which camera took the frame: the profile arrives on the decoded image,
travels through `DemosaicedImage` beside the matrix it belongs with, and is
written into the uniform block by the same three lines in `dr-gpu` — which is
exactly the path the matrix already took, because it is exactly the same kind
of thing.
What it *does* share with the tone curve node is the spline. The composer asks
`ToneCurve` for its `curve_span`/`curve_eval` helpers rather than emitting a
second copy, so a profile author placing a control point and a photographer
dragging one mean the same thing by it.
The order still reads correctly from this directory: the base curve runs after
every node in the chain. That is the same reasoning `exposure` records under
`placement:` — corrections to capture are only meaningful on linear values, so
the rendering goes last.
The first two are properties of the *file*, at the same standing as the
masked-photosite crop (FR-RAW-3) and the stored orientation (FR-DEV-3h): nobody
chose the sensor's green sensitivity, and reading the file correctly means
undoing it. The view transform is different in kind — it is the one stage that
maps scene-linear colour to a display range (D19, ARCH §6.14) — and it runs
after every node, because corrections to capture are only meaningful on linear
values. It replaced the per-body **base curve**, which was looked up by camera
model and flat past 1.0, so it clipped every recovered highlight.
## Stages
core/dr-pipeline/src/lib.rs
+2 -2
View File
@@ -50,6 +50,7 @@ pub mod preset;
pub mod sidecar;
pub mod spot;
pub mod state;
pub mod view;
pub use coverage::Coverage;
pub use declared::{Declaration, DeclaredOp};
@@ -66,8 +67,7 @@ pub use history::{Edit, Entry as HistoryEntry, History, Step};
pub use lens::{compose_warps, ComposedWarp, LensProfile, Tca, Warp};
pub use operation::{
compose, compose_with_framing, Affects, ComposedShader, Helper, Invalidation, Operation,
OutputMode, Uniform, BASE_CURVE_POINTS, BASE_CURVE_UNIFORM_OFFSET, CLIP_ONSET,
RESERVED_UNIFORM_FIELDS, SAMPLE_CACHE_UNIFORM_OFFSET,
OutputMode, Stage, Uniform, CLIP_ONSET, RESERVED_UNIFORM_FIELDS, SAMPLE_CACHE_UNIFORM_OFFSET,
};
pub use preset::{LibraryParseError, NameError, Preset, PresetLibrary, Reach, Scope};
pub use sidecar::{Sidecar, Version};
core/dr-pipeline/src/operation.rs
+67 -163
View File
@@ -557,12 +557,12 @@ pub struct ComposedShader {
/// Fields the generated uniform struct always carries, before op uniforms.
///
/// WGSL requires a uniform struct to be non-empty and 16-byte aligned; these
/// are needed by every generated shader in any case.
/// are needed by every generated shader in any case: the matrix, the as-shot
/// balance and the sample cache's flags. Framing's block follows.
///
/// Twelve of the twenty-eight are the camera profile's base curve
/// ([`BASE_CURVE_UNIFORM_FIELDS`]); the rest are the matrix, the as-shot
/// balance and framing's own block.
const BASE_UNIFORM_FIELDS: usize = 16 + SAMPLE_CACHE_UNIFORM_FIELDS + BASE_CURVE_UNIFORM_FIELDS;
/// Twelve fewer than before D19, which retired the base curve that sat at the
/// end of this block.
const BASE_UNIFORM_FIELDS: usize = 16 + SAMPLE_CACHE_UNIFORM_FIELDS;
/// Slots the sample cache's two flags occupy: read, write, and two spare to
/// keep the block a whole `vec4`. See [`ComposedShader::sample_key`].
@@ -571,36 +571,11 @@ const SAMPLE_CACHE_UNIFORM_FIELDS: usize = 4;
/// Where the sample cache's flags sit in the generated uniform block: `x` says
/// read the source colour from the cache, `y` says write it there.
///
/// Exported for the reason [`BASE_CURVE_UNIFORM_OFFSET`] is — `dr-gpu` writes
/// Exported for the reason [`RESERVED_UNIFORM_FIELDS`] is — `dr-gpu` writes
/// these by index — and zero in every block the composer hands out, so a
/// caller that never heard of the cache gets the direct read it always had.
pub const SAMPLE_CACHE_UNIFORM_OFFSET: usize = 16;
/// TRACES: FR-DEV-3e
/// Slots the base curve occupies: five `(x, y)` points and an active flag.
///
/// Twelve rather than eleven so the block stays a whole number of `vec4`s,
/// which is what std140 requires of a uniform struct's members. The spare
/// float is left zero rather than repurposed — a uniform slot that means one
/// thing today and two things next year is how a shader comes to read a
/// highlight rolloff out of a crop rectangle.
const BASE_CURVE_UNIFORM_FIELDS: usize = 12;
/// TRACES: FR-DEV-3e
/// Where the base curve's slots begin in the generated uniform block.
///
/// Exported for the same reason [`RESERVED_UNIFORM_FIELDS`] is: `dr-gpu`
/// writes these by index, and an offset computed independently at both ends is
/// an offset that will eventually disagree with itself.
pub const BASE_CURVE_UNIFORM_OFFSET: usize =
SAMPLE_CACHE_UNIFORM_OFFSET + SAMPLE_CACHE_UNIFORM_FIELDS;
/// How many control points a base curve carries.
///
/// The same five the tone curve widget has, deliberately — see the helper
/// selection in [`compose_full`].
pub const BASE_CURVE_POINTS: usize = 5;
/// Where the highlight desaturation begins: the fraction of the white level
/// above which a photosite is treated as clipped.
///
@@ -869,42 +844,20 @@ fn compose_inner(
\x20 // The sample cache (see `ComposedShader::sample_key`): `.x` reads\n\
\x20 // the source colour from `sampled`, `.y` writes it to\n\
\x20 // `sample_out`. Zero for both is the direct read.\n\
\x20 sample_cache: vec4<f32>,\n\
\x20 // The camera profile's base curve (FR-DEV-3e): five points on a\n\
\x20 // monotone spline, packed as x0..x3, y0..y3, then (x4, y4, on).\n\
\x20 // `.z` of the last is the flag, not padding — it is 0 for a\n\
\x20 // body with no profile and for an already-rendered source.\n\
\x20 base_curve_x: vec4<f32>,\n\
\x20 base_curve_y: vec4<f32>,\n\
\x20 base_curve_last: vec4<f32>,\n",
\x20 sample_cache: vec4<f32>,\n",
);
uniform_values.resize(BASE_UNIFORM_FIELDS, 0.0);
// TRACES: FR-DEV-3e
// The spline the base curve is evaluated on is the *tone curve's* spline,
// reached through the trait rather than reimplemented here.
//
// Two reasons, and the second is the one that matters. The obvious one is
// that a shader carrying two `curve_eval`s would not compile, and the
// composer's helper de-duplication is what makes both stages able to ask
// for it. The real one is that a profile author placing a control point
// and a photographer dragging one must mean the same thing by it — down to
// the Fritsch-Carlson tangent limiting, which is what decides how a
// shoulder actually rolls off. Two implementations that agreed today would
// be two that could disagree later, and the disagreement would show up as
// a body whose profile renders subtly differently from the curve someone
// drew to match it.
//
// Emitted unconditionally, unlike an operation's helpers. The base curve
// is active for every RAW frame — an unprofiled body still gets the
// database's default rendering — so making the shader's shape depend on it
// would split the pipeline cache in two for no benefit. The uniform flag
// above turns it off for the cases that are genuinely already rendered,
// and a branch on a uniform is coherent across the whole dispatch.
for h in crate::ops::ToneCurve::new().helpers() {
if matches!(h.name, "curve_span" | "curve_eval") {
helpers.push(*h);
}
// TRACES: FR-DEV-3j
// The view transform's function, whenever the composer emits the view
// transform — which is every render but the camera-space tap and one a
// rendering operation has taken over. See `rendering_tail` below.
let views = !op_renders && output_mode != OutputMode::CameraLinear;
if views {
helpers.push(Helper {
name: "view_sigmoid",
source: crate::view::VIEW_SIGMOID_WGSL,
});
}
// Framing's block follows the base one at a fixed offset, for the same
@@ -1161,47 +1114,32 @@ fn compose_inner(
),
};
// The base curve, which an operation may have taken over.
let rendering_tail = if op_renders {
" // The base curve is absent: an operation declaring\n // `Operation::renders` has done its job, and doing it again would render\n // the picture twice.\n"
// The view transform, which an operation may have taken over.
let rendering_tail = if !views {
" // No view transform: an operation declaring `Operation::renders` has\n // mapped the scene to a display range itself, or this is the\n // camera-space tap, which stores the sensor's own numbers.\n"
.to_string()
} else {
" // ==== camera profile: the base curve (FR-DEV-3e) ====
let curve = crate::view::Sigmoid::default_curve();
format!(
" // ==== the view transform (FR-DEV-3j) ====
//
// Marked with `====` and not the `----` an operation block carries: this
// is not one, and the difference is what several tests count on to tell
// an edit apart from the reading of a file.
// an edit apart from the rendering of one.
//
// After every operation, and — since D19 moved the matrix to the front of
// the chain — on working-space colour rather than the camera RGB it was
// tuned against. An interim placement: the per-body curve is being
// replaced by a view transform after the detail stage (FR-DEV-3j).
// The one stage allowed to map scene-linear colour to a display range
// (D19, ARCH §6.14), after every operation. Everything above it is
// unbounded; everything the shoulder has not brought under 1.0 is
// clipped by the output transform, at the last moment.
//
// The branch is on a uniform, so the whole dispatch takes the same path.
// It is off for a JPEG and any other already-rendered source, which must
// not be rendered twice, and for a body the profile database declines to
// offer any curve for at all.
if (u.base_curve_last.z > 0.5) {
c = vec3<f32>(
curve_eval(
u.base_curve_x.x, u.base_curve_y.x, u.base_curve_x.y, u.base_curve_y.y,
u.base_curve_x.z, u.base_curve_y.z, u.base_curve_x.w, u.base_curve_y.w,
u.base_curve_last.x, u.base_curve_last.y, c.r,
),
curve_eval(
u.base_curve_x.x, u.base_curve_y.x, u.base_curve_x.y, u.base_curve_y.y,
u.base_curve_x.z, u.base_curve_y.z, u.base_curve_x.w, u.base_curve_y.w,
u.base_curve_last.x, u.base_curve_last.y, c.g,
),
curve_eval(
u.base_curve_x.x, u.base_curve_y.x, u.base_curve_x.y, u.base_curve_y.y,
u.base_curve_x.z, u.base_curve_y.z, u.base_curve_x.w, u.base_curve_y.w,
u.base_curve_last.x, u.base_curve_last.y, c.b,
),
);
}
"
.to_string()
// Skipped for an already-rendered source — a JPEG is a display rendering
// already, and rendering it again would compress it twice.
if (!non_linear) {{
c = view_sigmoid(c, {:?}, {:?}, {:?});
}}
",
curve.n, curve.inv_k, curve.w
)
};
// Formatted with Rust's `Display` so the shader reads the same threshold
@@ -2270,10 +2208,10 @@ mod tests {
}
#[test]
fn a_rendering_operation_takes_over_the_base_curve() {
// TRACES: FR-DEV-3e | FR-DEV-3f
// A film stock's characteristic curve does the base curve's job.
// Emitting the profile's rendering as well would render the scene
fn a_rendering_operation_takes_over_the_view_transform() {
// TRACES: FR-DEV-3j | FR-DEV-3f
// A film stock's characteristic curve does the view transform's job.
// Emitting the default rendering as well would render the scene
// twice — a picture that comes out looking like neither the camera's
// rendering nor the film's, with a colour-management bug's signature
// and no colour-management bug to find.
@@ -2299,8 +2237,8 @@ mod tests {
.find("---- film_sim ----")
.expect("the operation itself must still be emitted");
assert!(
!source.contains("base_curve_last.z > 0.5"),
"the base curve is still being applied on top of the film"
!source.contains("view_sigmoid"),
"the view transform is still being applied on top of the film"
);
// D19: the film no longer converts out of camera space itself. The
// composer does, once, ahead of it — the film is handed working-space
@@ -2323,7 +2261,7 @@ mod tests {
// suppressed the tail unconditionally renders every ordinary edit
// flat and uncorrected, which reads as a broken camera profile.
let source = compose(&[fake(DESC_A.clone(), 2.0, false)]).source;
assert!(source.contains("base_curve_last.z > 0.5"));
assert!(source.contains("c = view_sigmoid("));
assert!(source.contains("camera profile: the matrix"));
}
@@ -2335,7 +2273,7 @@ mod tests {
// catalogue, and it must not disturb the camera's own rendering.
let film: Box<dyn Operation> = Box::new(crate::ops::FilmSim::new());
let source = compose(&[film, fake(DESC_A.clone(), 2.0, false)]).source;
assert!(source.contains("base_curve_last.z > 0.5"));
assert!(source.contains("c = view_sigmoid("));
assert!(source.contains("camera profile: the matrix"));
}
@@ -2374,8 +2312,8 @@ mod tests {
}
#[test]
fn the_base_curve_runs_after_the_operations() {
// TRACES: FR-DEV-3e
fn the_view_transform_runs_after_the_operations() {
// TRACES: FR-DEV-3j
// Exposure and the tonal controls are corrections to capture, and
// they are only meaningful on linear values. A stop is a doubling; run
// exposure after a curve and it is not one any more, and every slider
@@ -2383,66 +2321,23 @@ mod tests {
let ops = vec![fake(DESC_A.clone(), 2.0, false)];
let source = compose(&ops).source;
let op = source.find("---- op_a ----").expect("op present");
let curve = source
.find("if (u.base_curve_last.z > 0.5)")
.expect("base curve applied");
assert!(op < curve, "the base curve must come after the operations");
}
#[test]
fn the_base_curve_reaches_a_shader_with_no_operations_at_all() {
// TRACES: FR-DEV-3e
// The same property as as-shot white balance, and for the same reason:
// it is part of interpreting the file, not part of the edit. An
// unedited RAW must open looking like a photograph rather than like a
// scan of one.
let shader = compose(&[]);
assert!(shader.source.contains("u.base_curve_x"));
let view = source.find("c = view_sigmoid(").expect("view applied");
assert!(
shader.source.contains("fn curve_eval("),
"the spline it is evaluated on must be emitted too"
op < view,
"the view transform must come after the operations"
);
}
#[test]
fn the_base_curve_and_the_tone_curve_share_one_spline() {
// TRACES: FR-DEV-3e
// Two `curve_eval`s in one shader would not compile — but the reason
// the helper is *shared* rather than merely renamed is that a profile
// author placing a control point and a photographer dragging one must
// mean the same thing by it, down to the tangent limiting that decides
// how a shoulder rolls off.
let mut curve = crate::ops::ToneCurve::new();
curve.set_param(crate::ops::curve::P2_Y, 0.7);
assert!(
curve.is_active(),
"the fixture must actually reach the shader"
);
let source = compose(&[Box::new(curve)]).source;
assert_eq!(
source.matches("fn curve_eval(").count(),
1,
"the spline must be declared exactly once"
);
assert_eq!(source.matches("fn curve_span(").count(), 1);
}
#[test]
fn the_base_curve_owns_the_slots_dr_gpu_writes() {
// TRACES: FR-DEV-3e
// `dr-gpu` fills these by index. The offset is exported rather than
// recomputed there, and this asserts the exported number still points
// at the block the shader declares — the failure otherwise is a
// highlight rolloff read out of a crop rectangle, which renders as
// nonsense rather than as an error.
assert_eq!(
BASE_CURVE_UNIFORM_OFFSET + BASE_CURVE_UNIFORM_FIELDS,
BASE_UNIFORM_FIELDS,
"the base curve must be the last thing in the base block"
);
assert_eq!(BASE_CURVE_POINTS * 2 + 1, BASE_CURVE_UNIFORM_FIELDS - 1);
assert!(compose(&[]).uniforms.len() >= BASE_UNIFORM_FIELDS);
fn the_view_transform_reaches_a_shader_with_no_operations_at_all() {
// TRACES: FR-DEV-3j
// An unedited RAW must open looking like a photograph rather than
// like a scan of one, and it is skipped only for a source that is
// already a rendering.
let source = compose(&[]).source;
assert!(source.contains("c = view_sigmoid("));
assert!(source.contains("fn view_sigmoid("));
assert!(source.contains("if (!non_linear)"));
}
/// Compose with neutral framing into a chosen output space.
@@ -2450,6 +2345,15 @@ mod tests {
compose_with_framing(ops, &Framing::new(), output)
}
#[test]
fn the_camera_space_tap_has_no_view_transform() {
// TRACES: FR-MRG-2
// A merge stitches the sensor's own numbers; a rendering in them
// would be developed a second time when the composite is opened.
let source = compose_camera_probe(&[], &crate::framing::Framing::default()).source;
assert!(!source.contains("view_sigmoid"));
}
#[test]
fn an_srgb_render_is_byte_for_byte_what_it_was_before_output_spaces_existed() {
// The display path is the shader compiled on nearly every frame, and
core/dr-pipeline/src/view.rs
+276
View File
@@ -0,0 +1,276 @@
//! TRACES: FR-DEV-3j | FR-DEV-2
//! The view transform — the one stage that maps scene-linear colour to a
//! display range (D19, ARCH §6.14).
//!
//! # What it is
//!
//! A log-logistic sigmoid, per channel:
//!
//! ```text
//! f(x) = w · r / (1 + r), r = (x / k)^n
//! ```
//!
//! `n` is the contrast — the slope in log-log terms, before the shoulder
//! bends it. `k` and `w` are solved from two conditions rather than set:
//! scene middle grey lands on display middle grey, and the scene white the
//! photographer chose lands on display white. So the curve has a toe, a
//! midtone slope and a shoulder that approaches `w` — a hair above 1.0 —
//! without ever reaching it. Everything the shoulder has not reached by the
//! white point is clipped by the output transform, which is the last moment
//! and the only place a clip belongs.
//!
//! # Why per channel, and why the middle channel is put back
//!
//! Per channel is what makes a bright saturated colour desaturate as it
//! approaches white — a blown sky rolls toward white rather than toward a
//! saturated corner of the gamut, which is what film and every camera JPEG
//! do. It also bends hue: the three channels sit at different places on the
//! curve, so their ratios change, and an orange flame drifts toward yellow.
//! So after the curve the middle channel is moved back to where it sat
//! *between the other two* before it — the same fraction of the way from the
//! smallest to the largest. The smallest and largest keep what the curve gave
//! them, which keeps the desaturation; the hue, which is decided by that
//! fraction, survives. It is the "preserve hue" step of darktable's sigmoid,
//! at full strength.
//!
//! # Why these defaults
//!
//! [`SCENE_GREY`] is where the retired default base curve put middle grey
//! (FR-DEV-3e): linear sensor data from a correctly exposed frame has it
//! near 13% of saturation, and a camera JPEG shows it at 18%. The contrast
//! and white defaults were chosen against that same retired curve: at 1.4 and
//! 4 stops the midtones stay within a quarter of a stop of it between scene
//! 0.03 and 1.0, while a highlight a stop past sensor saturation still rolls
//! into white rather than stopping dead at it. The upper midtones come out a
//! little darker than the curve had them, which is the price of that
//! headroom and what the white slider is for.
/// Scene-linear middle grey: where the retired default curve placed it.
pub const SCENE_GREY: f32 = 0.13;
/// Display-linear middle grey — what a camera JPEG shows a grey card as.
pub const DISPLAY_GREY: f32 = 0.18;
/// The default contrast, the sigmoid's log-log slope parameter `n`.
pub const DEFAULT_CONTRAST: f32 = 1.4;
/// The default white point, in stops above [`SCENE_GREY`].
pub const DEFAULT_WHITE: f32 = 4.0;
/// The contrast range a photographer is offered.
pub const CONTRAST_RANGE: (f32, f32) = (1.0, 3.0);
/// The white point range, in stops above middle grey.
///
/// The floor is not taste. The two conditions `k` and `w` are solved from
/// have a solution only while `2^(white · n)` exceeds `1 / DISPLAY_GREY`,
/// and at the lowest contrast that needs `white` above about 2.47 stops.
pub const WHITE_RANGE: (f32, f32) = (2.5, 10.0);
/// The curve's three numbers, solved from the photographer's two.
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct Sigmoid {
/// Contrast: the exponent.
pub n: f32,
/// `1 / k`, so the shader multiplies rather than divides.
pub inv_k: f32,
/// The asymptote the shoulder approaches, a little above 1.0.
pub w: f32,
}
impl Sigmoid {
/// Solve the curve for a contrast and a white point in stops.
///
/// Out-of-range inputs are clamped to [`CONTRAST_RANGE`] and
/// [`WHITE_RANGE`] rather than trusted: they arrive from a sidecar, which
/// may have been written by a build with other limits, and outside them
/// the solution below divides by something that is no longer positive.
///
/// With `r_g` the value of `r` at scene grey and `q = 2^(white · n)`, the
/// two conditions `f(grey) = display grey` and `f(grey · 2^white) = 1`
/// are `w·r_g/(1+r_g) = g` and `w·q·r_g/(1+q·r_g) = 1`. Dividing one by
/// the other eliminates `w` and leaves `r_g = (g·q − 1) / (q·(1 − g))`.
pub fn new(contrast: f32, white: f32) -> Self {
let n = contrast.clamp(CONTRAST_RANGE.0, CONTRAST_RANGE.1) as f64;
let white = white.clamp(WHITE_RANGE.0, WHITE_RANGE.1) as f64;
let g = f64::from(DISPLAY_GREY);
let q = (white * n).exp2();
let r_grey = (g * q - 1.0) / (q * (1.0 - g));
let w = g * (1.0 + r_grey) / r_grey;
// r = (x / k)^n, and r at scene grey is r_grey, so
// k = grey / r_grey^(1/n).
let k = f64::from(SCENE_GREY) / r_grey.powf(1.0 / n);
Self {
n: n as f32,
inv_k: (1.0 / k) as f32,
w: w as f32,
}
}
/// The default curve.
pub fn default_curve() -> Self {
Self::new(DEFAULT_CONTRAST, DEFAULT_WHITE)
}
/// One channel through the curve. The CPU reference the shader is
/// tested against.
pub fn channel(&self, x: f32) -> f32 {
let r = (x.max(0.0) * self.inv_k).powf(self.n);
self.w * r / (1.0 + r)
}
/// A colour through the curve, with the middle channel put back between
/// the other two. See the module documentation.
pub fn apply(&self, c: [f32; 3]) -> [f32; 3] {
let x = c.map(|v| v.max(0.0));
let y = x.map(|v| self.channel(v));
let lo = x[0].min(x[1]).min(x[2]);
let hi = x[0].max(x[1]).max(x[2]);
if hi - lo <= 1e-9 {
return y;
}
let (y_lo, y_hi) = (self.channel(lo), self.channel(hi));
x.map(|v| y_lo + (y_hi - y_lo) * (v - lo) / (hi - lo))
}
}
/// The WGSL twin of [`Sigmoid::apply`], as a helper function.
pub const VIEW_SIGMOID_WGSL: &str = "\
// The view transform (FR-DEV-3j): a log-logistic sigmoid per channel, then the
// middle channel put back between the other two so that the hue survives the
// shoulder. See `dr_pipeline::view` for the derivation and the defaults.
fn view_sigmoid(c: vec3<f32>, n: f32, inv_k: f32, w: f32) -> vec3<f32> {
// Negative components are colours outside the working primaries. They are
// floored here, at the last stage, which is the one place a gamut clip
// belongs.
let x = max(c, vec3<f32>(0.0));
let lo = min(x.r, min(x.g, x.b));
let hi = max(x.r, max(x.g, x.b));
let r_lo = pow(lo * inv_k, n);
let r_hi = pow(hi * inv_k, n);
let y_lo = w * r_lo / (1.0 + r_lo);
let y_hi = w * r_hi / (1.0 + r_hi);
// Each channel's place between the smallest and the largest. A neutral
// has no spread, and every channel then takes the one value there is.
let spread = hi - lo;
let t = select((x - vec3<f32>(lo)) / max(spread, 1e-9), vec3<f32>(0.0), spread <= 1e-9);
return vec3<f32>(y_lo) + (y_hi - y_lo) * t;
}
";
#[cfg(test)]
mod tests {
use super::*;
/// The retired default base curve, for the acceptance comparison: five
/// points through the unit square, sampled here by straight lines between
/// them in log-log terms — close enough to the monotone spline that drew
/// it for a tolerance measured in quarters of a stop. Below its second
/// point it was a straight line from the origin.
fn retired_default(x: f32) -> f32 {
const P: [(f32, f32); 4] = [(0.04, 0.043), (0.13, 0.175), (0.45, 0.690), (1.0, 1.0)];
if x < 0.04 {
return x * (0.043 / 0.04);
}
let x = x.min(1.0);
let i = P.windows(2).position(|w| x <= w[1].0).unwrap_or(2);
let ((x0, y0), (x1, y1)) = (P[i], P[i + 1]);
let t = (x.ln() - x0.ln()) / (x1.ln() - x0.ln());
(y0.ln() + t * (y1.ln() - y0.ln())).exp()
}
#[test]
fn middle_grey_lands_on_display_grey() {
// TRACES: FR-DEV-3j
for (contrast, white) in [(1.0, 3.0), (1.4, 4.0), (2.5, 8.0), (3.0, 10.0)] {
let s = Sigmoid::new(contrast, white);
let got = s.channel(SCENE_GREY);
assert!(
(got - DISPLAY_GREY).abs() < 0.01,
"contrast {contrast}, white {white}: grey went to {got}"
);
}
}
#[test]
fn the_white_point_reaches_display_white() {
// TRACES: FR-DEV-3j
// The whole meaning of the slider: the scene value it names is where
// the picture reaches white, and not before.
for (contrast, white) in [(1.0, 3.0), (1.4, 4.0), (2.5, 8.0)] {
let s = Sigmoid::new(contrast, white);
let at = SCENE_GREY * white.exp2();
assert!((s.channel(at) - 1.0).abs() < 1e-4, "{}", s.channel(at));
assert!(s.channel(at * 0.9) < 1.0);
assert!(s.w > 1.0, "the shoulder must approach a value above white");
}
}
#[test]
fn the_curve_is_monotone_and_keeps_going_past_one() {
// TRACES: FR-DEV-3j | FR-DEV-2
// What the base curve got wrong: it was flat past 1.0, so every
// recovered highlight left it as the same number.
let s = Sigmoid::default_curve();
let mut last = -1.0;
for i in 0..=2000 {
let x = i as f32 * 0.004;
let y = s.channel(x);
assert!(y > last || (x == 0.0 && y == 0.0), "not increasing at {x}");
last = y;
}
assert!(s.channel(2.0) > s.channel(1.0));
}
#[test]
fn the_default_stays_close_to_the_retired_curve() {
// TRACES: FR-DEV-3j | FR-DEV-3e
// D19's promise to every existing photograph: the midtones do not
// move by more than a third of a stop.
let s = Sigmoid::default_curve();
let mut x = 0.03_f32;
while x <= 1.0 {
let ev = (s.channel(x) / retired_default(x)).log2();
assert!(ev.abs() < 0.3, "at scene {x} the default moved {ev:+.2} EV");
x *= 1.1;
}
}
#[test]
fn a_neutral_stays_neutral() {
// TRACES: FR-DEV-3j
let s = Sigmoid::default_curve();
for v in [0.0, 0.01, 0.13, 1.0, 7.0] {
let [r, g, b] = s.apply([v, v, v]);
assert_eq!(r, g);
assert_eq!(g, b);
}
}
#[test]
fn the_hue_survives_the_shoulder() {
// TRACES: FR-DEV-3j
// The middle channel's place between the other two is what decides
// the hue. Without the correction an orange at the shoulder drifts
// toward yellow as the red channel saturates first.
let s = Sigmoid::default_curve();
let orange = [2.0, 0.8, 0.1];
let out = s.apply(orange);
let before = (orange[1] - orange[2]) / (orange[0] - orange[2]);
let after = (out[1] - out[2]) / (out[0] - out[2]);
assert!((before - after).abs() < 1e-5, "{before} became {after}");
// And the extremes keep what the curve gave them — the desaturation
// toward white is the point of working per channel.
assert!((out[0] - s.channel(2.0)).abs() < 1e-6);
assert!((out[2] - s.channel(0.1)).abs() < 1e-6);
}
#[test]
fn out_of_range_settings_are_clamped_not_trusted() {
// A sidecar from another build may carry anything, and outside the
// range the solution divides by a value that is no longer positive.
let s = Sigmoid::new(0.0, 0.0);
assert!(s.n.is_finite() && s.inv_k.is_finite() && s.w.is_finite());
assert_eq!(s, Sigmoid::new(CONTRAST_RANGE.0, WHITE_RANGE.0));
}
}
docs/dev/architecture.md
+1 -1
View File
@@ -452,7 +452,7 @@ exactly the stall §6.1 exists to prevent.
**Two reductions, not one.** The display histogram (FR-DSP-7) counts the frame the output transform
produced: its axis is the output code value, and a clipped bin means a highlight that is gone as the
image currently stands. The raw histogram for culling (FR-CULL-3) counts the **demosaiced
scene-linear texture** — before white balance, the camera matrix, the base curve and the tone chain
scene-linear texture** — before white balance, the camera matrix, the tone chain and the view transform
— on an axis of stops below sensor saturation, which is how it reports headroom the embedded JPEG's
histogram cannot. A culling decision needs the second, an export decision needs the first, and
neither answers for the other. Both are drawn by the same panel and chosen between.
docs/dev/mask-editing.md
+2 -2
View File
@@ -446,8 +446,8 @@ reading a flag.
After the output transform, immediately before the clip and the encode — not
among the layer blocks. Everything there runs on scene-referred colour in the
working space, where a flat tint would be pushed through the base curve and
the camera matrix and arrive as some other colour, and an alpha's white on
working space, where a flat tint would be pushed through the view transform
(the base curve and the camera matrix, before D19) and arrive as some other colour, and an alpha's white on
black would arrive as neither.
### 6.3 Not on the graph
docs/dev/panorama.md
+6 -2
View File
@@ -128,7 +128,11 @@ stays hot for it.
### 5.1 The tap — S15.3, answered by reading the composer
The fused shader's order, fixed by `operation.rs`'s own tests: warp → as-shot
white balance → operations → base curve → camera matrix → store. The store is
white balance → operations → base curve → camera matrix → store. *(Amended
2026-09-27, D19: warp → as-shot white balance → white balance → camera matrix
→ operations → view transform → store. The tap is unaffected: it has no
operations, its caller fills the matrix with the identity, and the composer
emits no view transform in `OutputMode::CameraLinear`.)* The store is
either the display encode or, in `OutputMode::LinearWorking`, an unclipped
`rgba16float` of linear sRGB. That mode exists for the detail stage and is
selected from the operations, never by a caller flag, so that a shader and
@@ -155,7 +159,7 @@ FR-MRG-11 already bounds.
**What the DNG carries as a consequence:** the first source's `Make`,
`Model` and `UniqueCameraModel` — so `base_curve::for_body` finds the 6D's
curve — its `ColorMatrix1`/`2` with illuminants, and its `AsShotNeutral`. The
curve (retired with the base curves, D19) — its `ColorMatrix1`/`2` with illuminants, and its `AsShotNeutral`. The
composite then develops through the same profile as its sources, applied
once. The spike's 64 × 48 file (§8) already carries the matrix and neutral;
the body name is a string.
docs/dev/traceability.md
+36 -36
View File
File diff suppressed because one or more lines are too long
platform/dr-plat/src/diagnostics.rs
+1 -1
View File
@@ -451,7 +451,7 @@ mod tests {
use super::*;
/// An empty directory of this test's own. The convention elsewhere in the
/// workspace (`dr_decode::base_curve`), and it matters more here: two
/// workspace, and it matters more here: two
/// tests sharing a log directory would rotate each other's files.
fn a_log_dir(name: &str) -> PathBuf {
let dir = std::env::temp_dir().join(format!("darkroom-diagnostics-{name}"));
ui/dr-ui/src/decoder_seam.rs
+2 -3
View File
@@ -16,8 +16,8 @@
use std::sync::Mutex;
use dr_decode::{
BaseCurve, CfaPattern, CropRect, DecodeError, Decoder, Metadata, Preview, PreviewLocation,
PreviewSize, RawImage,
CfaPattern, CropRect, DecodeError, Decoder, Metadata, Preview, PreviewLocation, PreviewSize,
RawImage,
};
use dr_types::Orientation;
@@ -121,7 +121,6 @@ impl Decoder for Stub {
white_level: 4095,
wb_coeffs: [1.0; 4],
color_matrix: None,
base_curve: BaseCurve::IDENTITY,
crop: CropRect {
x: 0,
y: 0,
ui/dr-ui/src/develop/render.rs
-1
View File
@@ -1475,7 +1475,6 @@ mod tests {
white_level: WHITE,
wb_coeffs: [1.0, 1.0, 1.0, 1.0],
color_matrix: None,
base_curve: dr_decode::BaseCurve::IDENTITY,
samples_per_pixel: 1,
profile: None,
make: String::new(),
ui/dr-ui/src/develop/white_balance.rs
-3
View File
@@ -230,7 +230,6 @@ mod tests {
white_level: 65535,
wb_coeffs: [as_shot[0], as_shot[1], as_shot[2], 0.0],
color_matrix: Some(cam_to_srgb),
base_curve: dr_decode::BaseCurve::IDENTITY,
samples_per_pixel: 3,
profile: None,
make: String::new(),
@@ -304,7 +303,6 @@ mod tests {
white_level: 65535,
wb_coeffs: [1.0, 1.0, 1.0, 0.0],
color_matrix: None,
base_curve: dr_decode::BaseCurve::IDENTITY,
samples_per_pixel: 3,
profile: None,
make: String::new(),
@@ -357,7 +355,6 @@ mod tests {
white_level: 65535,
wb_coeffs: [1.9, 1.0, 1.7, 0.0],
color_matrix: None,
base_curve: dr_decode::BaseCurve::IDENTITY,
samples_per_pixel: 3,
profile: None,
make: String::new(),