dtourolle/DarkRoom
1
0
Fork 0
Code Issues 39 Pull Requests Actions Packages Projects Releases 20 Wiki Activity

Merge branch 'worktree-agent-afd449f5e7a01e341' into integration

Browse Source 
# Conflicts:
#	core/dr-gpu/src/adjust.rs
#	core/dr-pipeline/ops/README.md
#	core/dr-pipeline/src/lib.rs
#	docs/traceability.md
This commit is contained in:
dtourolle
2026-08-22 15:35:29 +02:00
parent 7f60a2547c 60d5504fb4
commit c963dafd09
12 changed files with 2542 additions and 71 deletions
Download Patch File Download Diff File Expand all files Collapse all files
core/dr-decode/Cargo.toml
+7
View File
@@ -8,6 +8,13 @@ license.workspace = true
[dependencies]
dr-types.workspace = true
rawler.workspace = true
# The camera profile database is data, not code (FR-DEV-3e): a YAML file that
# ships with the binary and is superseded by a newer one on disk. serde_norway
# is the workspace's YAML crate — the fork still receiving releases — and it is
# already in the tree for `dr-pipeline`'s node declarations and `dr-ui`'s style
# tokens. Pure Rust, so it costs nothing under the Android NDK.
serde = { workspace = true }
serde_norway.workspace = true
zune-jpeg.workspace = true
thiserror.workspace = true
log.workspace = true
core/dr-decode/profiles/base_curves.yaml
+160
View File
@@ -0,0 +1,160 @@
# DarkRoom camera base curves (FR-DEV-3e).
#
# ---------------------------------------------------------------------------
# Adding a body is editing this file. It is not a code change.
# ---------------------------------------------------------------------------
#
# The copy you are reading is compiled into the binary as a floor. At startup
# `dr_decode::base_curve::load` also looks for `base_curves.yaml` in:
#
# 1. $DARKROOM_PROFILES/ (set it while you are tuning)
# 2. $XDG_DATA_HOME/darkroom/profiles/
# or $HOME/.local/share/darkroom/profiles/
#
# and uses the first one it finds *whose `version:` is higher than this one's*.
# So: bump `version`, drop the file in that directory, restart. A body added
# this afternoon renders correctly this afternoon, with no release and no
# rebuild — which is what the requirement asks for, and what makes these
# contributable under the GPL.
#
# The version check runs both ways on purpose. A file older than the built-in
# copy is ignored with a log line, so upgrading DarkRoom cannot silently lose
# curves to a pack somebody downloaded a year ago.
#
# ---------------------------------------------------------------------------
# What the numbers mean
# ---------------------------------------------------------------------------
#
# Five `[x, y]` control points on a monotone spline (Fritsch-Carlson, the same
# one the tone curve widget draws). Both axes are **linear**:
#
# x scene-referred camera RGB after white balance, 1.0 = sensor saturation
# y display-referred linear; the sRGB transfer function is applied later,
# at the end of the shader, so do not pre-apply a gamma here
#
# The identity is y = x, and it is what an unrecognised body gets if `default:`
# is removed. It is also the wrong answer for almost every photograph: linear
# scene data has middle grey at about 13% and a camera JPEG puts it near 18%,
# so an uncurved render is roughly half a stop dark through the midtones and
# has no highlight rolloff at all.
#
# A curve that works has three parts, and it is worth naming them because they
# are what you are actually tuning:
#
# the toe the first span, slope near or below 1. Deep shadows stay
# deep. Lift it and blacks go milky; crush it and shadow
# detail the sensor recorded disappears.
# the midtones the middle spans, slope well above 1. This is the contrast
# and the brightness people read as "the camera's look".
# the shoulder the last span, slope well below 1. Highlights compress
# toward white instead of arriving there and clipping. It is
# the difference between a rolled-off sky and a white hole.
#
# Two invariants are enforced in code and tested, so a mistake here fails the
# build rather than the photograph: x must strictly increase, y must not
# decrease, and everything must lie inside the unit square.
#
# ---------------------------------------------------------------------------
# Honesty about these values
# ---------------------------------------------------------------------------
#
# These are hand-tuned shapes, not measurements. They encode what every camera
# JPEG rendering has in common — the toe/midtone/shoulder structure above —
# plus each maker's well-known house differences: Canon's gentler shoulder and
# warmer-reading midtones, Nikon's slightly higher midtone contrast, Sony's
# flatter and more conservative default, Fujifilm's markedly contrastier
# Provia-derived rendering.
#
# FR-DEV-3e's acceptance criterion is subjective comparison against each body's
# own JPEG, and meeting it properly needs a frame from that body in front of
# you. Where that has not been done, the entry is still much closer to right
# than the identity — which is the bar these have to clear, and do.
version: 1
# The rendering for a body with no entry of its own.
#
# **Deliberately not the identity.** The failure this requirement exists to fix
# is the flat render, and a conservative curve is far closer to right for every
# body than no curve is for any of them. It is gentler than the per-body
# entries below — a shallower midtone and an earlier, softer shoulder — because
# it has to be safe on a sensor nobody has looked at, and the cost of being too
# tame is a photograph that wants a little contrast rather than one that has
# lost its highlights.
default:
points:
- [0.00, 0.000]
- [0.04, 0.043]
- [0.13, 0.175]
- [0.45, 0.690]
- [1.00, 1.000]
bodies:
# Canon. A soft toe and a long, gradual shoulder — the reason Canon files
# are described as forgiving in highlights and a little low in contrast
# straight out of camera.
- make: Canon
model: EOS 6D
points:
- [0.00, 0.000]
- [0.04, 0.045]
- [0.13, 0.190]
- [0.45, 0.720]
- [1.00, 1.000]
- make: Canon
model: EOS R6
points:
- [0.00, 0.000]
- [0.04, 0.044]
- [0.13, 0.195]
- [0.45, 0.730]
- [1.00, 1.000]
# Nikon. A slightly deeper toe and more midtone slope than Canon, which is
# the "punchier out of camera" difference people describe between the two.
- make: Nikon
model: Z 6
points:
- [0.00, 0.000]
- [0.04, 0.038]
- [0.13, 0.200]
- [0.46, 0.750]
- [1.00, 1.000]
- make: Nikon
model: D750
points:
- [0.00, 0.000]
- [0.04, 0.039]
- [0.13, 0.198]
- [0.46, 0.745]
- [1.00, 1.000]
# Sony. The flattest default of the four, and intentionally so — Sony's own
# rendering leaves more headroom than it uses, which is why Sony files are
# the ones people describe as needing the most work.
- make: Sony
model: ILCE-7M3
points:
- [0.00, 0.000]
- [0.04, 0.048]
- [0.13, 0.185]
- [0.44, 0.700]
- [1.00, 1.000]
# Fujifilm. Provia, the default film simulation: a firm toe, the steepest
# midtones here, and a hard shoulder. It is the most distinctive rendering of
# the four and the one where a flat render looks most obviously wrong.
#
# This entry does *not* read the in-RAF film simulation tag — that is
# FR-DEV-3f, and until it lands every Fujifilm file gets the Provia shape
# whatever the camera was set to.
- make: Fujifilm
model: X-T3
points:
- [0.00, 0.000]
- [0.045, 0.040]
- [0.14, 0.215]
- [0.47, 0.775]
- [1.00, 1.000]
core/dr-decode/src/base_curve.rs
+763
View File
@@ -0,0 +1,763 @@
//! TRACES: FR-DEV-3e
//! Base curves — the per-body rendering that turns a correct exposure into a
//! photograph.
//!
//! # What this is for
//!
//! A camera matrix gets the *colours* right and leaves the picture flat. Sensor
//! data is scene-referred and very nearly linear; a print, a screen and a
//! camera's own JPEG are none of those things. Rendering linear data straight
//! out is the dcraw default, and FR-DEV-3e names it precisely: "the flat,
//! poor-skin-tone rendering characteristic of dcraw defaults, which is the
//! documented reason people abandon darktable in the first hour."
//!
//! The fix is a tone curve applied as part of *reading* the file rather than as
//! an edit — a toe, a steep midtone, and a shoulder that rolls highlights off
//! instead of clipping them. Every raw converter has one. Adobe calls it the
//! camera profile's tone curve, darktable calls it the base curve, and the name
//! here follows darktable's because the placement does too: it runs in camera
//! RGB, after white balance and the user's adjustments, immediately before the
//! conversion out to a working space.
//!
//! # Why it is not an edit
//!
//! It never reaches the sidecar and there is no slider for it, for the same
//! reason the EXIF orientation is not an edit (FR-DEV-3h): it is a property of
//! the body that took the frame, not of what anyone decided about the frame.
//! Sidecars are shared between devices and bodies (FR-NC-9), and one camera's
//! rendering must not follow an edit onto another camera's file.
//!
//! # Why it is data
//!
//! FR-DEV-3e requires the profile database to be "versioned independently of
//! the app binary so bodies and curves can be added without a release — and,
//! under D8's GPLv3, contributed by users". So the curves live in
//! `profiles/base_curves.yaml`, a file that is compiled in as a floor and
//! *overridden* by a copy on disk carrying a higher `version:`. Adding a body
//! is adding ten numbers to a YAML file; shipping that body to users is
//! publishing the file. Neither is a code change and neither needs a release.
//!
//! See [`load`] for the search path and [`Curves::body`] for the matching.
use std::path::{Path, PathBuf};
use std::sync::OnceLock;
/// How many control points a base curve has.
///
/// Five, which is not a coincidence: it is what the tone curve widget uses
/// (`dr_pipeline::ops::curve::POINTS`), so the shader evaluates a profile's
/// curve and a photographer's curve through exactly the same spline. A profile
/// author and a photographer dragging a point mean the same thing by it, and
/// the generated shader carries one implementation rather than two that could
/// disagree.
pub const POINTS: usize = 5;
/// TRACES: FR-DEV-3e
/// A base curve: five points on a monotone spline through the unit square.
///
/// `xs` is scene-linear camera RGB, normalised so that 1.0 is the sensor's
/// saturation point. `ys` is display-referred linear — *not* gamma-encoded,
/// because the sRGB transfer function is applied at the very end of the
/// generated shader and applying it twice would wash the image out.
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct BaseCurve {
pub xs: [f32; POINTS],
pub ys: [f32; POINTS],
}
impl BaseCurve {
/// The curve that does nothing — the identity diagonal.
///
/// What an unrecognised body gets if the database carries no default, and
/// what a JPEG gets always: an already-rendered image must not be rendered
/// a second time.
pub const IDENTITY: Self = Self {
xs: [0.0, 0.25, 0.5, 0.75, 1.0],
ys: [0.0, 0.25, 0.5, 0.75, 1.0],
};
/// Whether this curve would leave the image alone.
///
/// The shader is told to skip the stage entirely when it would, so an
/// unprofiled body costs a branch that is uniform across the dispatch
/// rather than a spline evaluation per channel per pixel.
pub fn is_identity(&self) -> bool {
self.xs
.iter()
.zip(self.ys.iter())
.all(|(x, y)| (x - y).abs() < 1e-6)
}
/// Build from raw pairs, rejecting anything that is not a curve.
///
/// A profile file is data a user may have edited, so this is the boundary
/// where "ten numbers" becomes "a curve": the x coordinates must increase,
/// the y coordinates must not decrease, and both must lie in the unit
/// square. A non-monotone x sends the spline's span search backwards and
/// divides by a negative width; a decreasing y inverts tones locally,
/// which reads as a dark halo through smooth gradients rather than as a
/// bad profile.
///
/// Endpoints are not forced to (0,0) and (1,1). A curve that lifts black
/// slightly, or that places the shoulder below white, is a legitimate
/// rendering choice and several bodies make it.
pub fn from_points(points: &[[f32; 2]]) -> Option<Self> {
if points.len() != POINTS {
return None;
}
let mut xs = [0.0f32; POINTS];
let mut ys = [0.0f32; POINTS];
for (i, p) in points.iter().enumerate() {
if !p[0].is_finite() || !p[1].is_finite() {
return None;
}
if !(0.0..=1.0).contains(&p[0]) || !(0.0..=1.0).contains(&p[1]) {
return None;
}
xs[i] = p[0];
ys[i] = p[1];
}
for i in 1..POINTS {
// Strictly increasing in x — the spline divides by the span width.
if xs[i] <= xs[i - 1] {
return None;
}
// Non-decreasing in y. Flat is allowed: a curve that holds a
// highlight range at white is clipping deliberately.
if ys[i] < ys[i - 1] {
return None;
}
}
Some(Self { xs, ys })
}
}
/// One body's entry in the database.
#[derive(Debug, Clone, PartialEq)]
pub struct BodyCurve {
/// The manufacturer, as the file writes it — "Canon", "NIKON CORPORATION".
pub make: String,
/// The model, as the file writes it — "EOS 6D", "ILCE-7M3".
pub model: String,
pub curve: BaseCurve,
}
/// TRACES: FR-DEV-3e
/// The base curve database.
///
/// Versioned as a whole rather than per body, because that is the unit a user
/// downloads and the unit that has to beat the built-in copy. See [`load`].
#[derive(Debug, Clone, PartialEq)]
pub struct Curves {
version: u32,
default: Option<BaseCurve>,
bodies: Vec<BodyCurve>,
}
impl Curves {
/// TRACES: FR-DEV-3e
/// The curve to render a frame from this body with.
///
/// Falls back, in order, to the database's `default:` and then to the
/// identity. **The default is deliberately not the identity**: an
/// unrecognised body rendered flat is the failure this requirement exists
/// to prevent, and a gentle, conservative curve is much closer to right for
/// every body than no curve is for any of them. A body with its own entry
/// gets that instead.
///
/// # What "this body" has to survive
///
/// The same camera names itself three ways depending on which program last
/// touched the file. A native NEF says make "NIKON CORPORATION", model
/// "NIKON Z 6"; rawler's own database cleans that to "Nikon" and "Z 6"; an
/// Adobe-converted DNG keeps the uncleaned pair. A database that had to
/// spell every variant would go stale the first time a maker changed its
/// mind about its own name, so the matching does the folding instead:
///
/// - Case, punctuation and runs of whitespace are flattened, so
/// "ILCE-7M3", "ILCE 7M3" and "ilce-7m3" are one body.
/// - The make is compared on its **first word only**. Every maker's
/// trailing corporate boilerplate — "CORPORATION", "IMAGING CORP" — is
/// noise, and no two camera manufacturers share a first word.
/// - The model is tried both as written and with a leading copy of the
/// make removed, which is what lets one "Canon"/"EOS 6D" entry cover
/// "Canon EOS 6D" as well.
pub fn body(&self, make: &str, model: &str) -> BaseCurve {
let (make, model) = (make_key(make), normalise(model));
// The model with a leading copy of the maker's name removed.
let bare = model.strip_prefix(&format!("{make} ")).unwrap_or(&model);
self.bodies
.iter()
.find(|b| {
let entry_model = normalise(&b.model);
make_key(&b.make) == make && (entry_model == model || entry_model == bare)
})
.map(|b| b.curve)
.or(self.default)
.unwrap_or(BaseCurve::IDENTITY)
}
/// The database version. Higher wins; see [`load`].
pub fn version(&self) -> u32 {
self.version
}
/// How many bodies have their own curve, excluding the default.
pub fn len(&self) -> usize {
self.bodies.len()
}
pub fn is_empty(&self) -> bool {
self.bodies.is_empty()
}
/// Parse a database from YAML.
///
/// Entries that are not curves are dropped with a warning rather than
/// failing the parse. A user-contributed file with one bad body should
/// cost that body's rendering, not every body's — and the alternative is an
/// application that will not open a photograph because somebody typed a
/// comma.
pub fn parse(yaml: &str) -> Result<Self, String> {
let file: File = serde_norway::from_str(yaml).map_err(|e| e.to_string())?;
let default = file.default.and_then(|d| {
BaseCurve::from_points(&d.points).or_else(|| {
log::warn!("base curves: the default entry is not a monotone curve; ignoring it");
None
})
});
let bodies = file
.bodies
.into_iter()
.filter_map(|b| match BaseCurve::from_points(&b.points) {
Some(curve) => Some(BodyCurve {
make: b.make,
model: b.model,
curve,
}),
None => {
log::warn!(
"base curves: {} {} is not a monotone curve; ignoring it",
b.make,
b.model
);
None
}
})
.collect();
Ok(Self {
version: file.version,
default,
bodies,
})
}
}
/// The copy that ships inside the binary.
///
/// A floor, not the answer: [`load`] prefers a newer file on disk. Compiled in
/// so that a fresh install with no profile directory — and every Android build,
/// where there is no such directory to speak of — still renders properly.
const BUILT_IN: &str = include_str!("../profiles/base_curves.yaml");
/// TRACES: FR-DEV-3e
/// The base curve database, loaded once.
///
/// # The search path, and why it is a version comparison
///
/// 1. `$DARKROOM_PROFILES`, a directory, when set. The escape hatch: a profile
/// author iterating on a curve points this at their working copy and does
/// not have to install anything.
/// 2. `$XDG_DATA_HOME/darkroom/profiles/`, else `$HOME/.local/share/darkroom/profiles/`.
/// The same base directory the catalog uses, chosen there for the same
/// reason — it is data, not cache, and must survive a storage sweep.
/// 3. The copy compiled into the binary.
///
/// The first file that parses *and carries a higher `version:` than the
/// built-in copy* wins. The version check is the whole mechanism the
/// requirement asks for, and it runs in both directions:
///
/// - A downloaded pack at version 7 supersedes a binary shipping version 3, so
/// a body added after the release renders correctly with no release.
/// - A stale pack at version 2 does **not** supersede a binary shipping version
/// 3, so upgrading the application cannot silently lose curves to a file
/// somebody downloaded a year ago and forgot.
///
/// Failures are warnings, never errors. A malformed profile file must cost the
/// user their curves, not their photographs.
pub fn load() -> &'static Curves {
static LOADED: OnceLock<Curves> = OnceLock::new();
LOADED.get_or_init(|| {
let built_in = Curves::parse(BUILT_IN).unwrap_or_else(|e| {
// Unreachable in a build that ran its tests — `the_shipped_database_parses`
// asserts exactly this — but a panic here would mean an
// application that cannot open a photograph because of a typo in a
// data file, which is never the right trade.
log::error!("base curves: the built-in database does not parse: {e}");
Curves {
version: 0,
default: None,
bodies: Vec::new(),
}
});
choose(built_in, &search_path())
})
}
/// The version comparison, separated from where the directories come from.
///
/// Split out so it can be tested against real files in a real directory
/// without the process-wide `OnceLock` and the environment `load` reads. The
/// rule this implements is the whole of what FR-DEV-3e asks for, so it is
/// worth being able to state it as a test rather than as a comment.
fn choose(built_in: Curves, dirs: &[PathBuf]) -> Curves {
for dir in dirs {
let path = dir.join("base_curves.yaml");
let Ok(text) = std::fs::read_to_string(&path) else {
continue;
};
match Curves::parse(&text) {
Ok(external) if external.version > built_in.version => {
log::info!(
"base curves: using {} (version {}, {} bodies) over the built-in version {}",
path.display(),
external.version,
external.len(),
built_in.version
);
return external;
}
Ok(external) => log::info!(
"base curves: ignoring {} at version {}; the built-in database is version {}",
path.display(),
external.version,
built_in.version
),
Err(e) => log::warn!("base curves: {} does not parse: {e}", path.display()),
}
}
built_in
}
/// TRACES: FR-DEV-3e
/// The curve for a body, from the loaded database.
///
/// The one call site the decoder needs; everything above is reachable for
/// tests and for a future profile editor.
pub fn for_body(make: &str, model: &str) -> BaseCurve {
load().body(make, model)
}
/// Directories that may hold a `base_curves.yaml`, most specific first.
fn search_path() -> Vec<PathBuf> {
let mut dirs = Vec::new();
if let Some(explicit) = std::env::var_os("DARKROOM_PROFILES") {
dirs.push(PathBuf::from(explicit));
}
// The same resolution `dr_ui::library::catalog_path` uses, and for the
// same reason: this is data a user may have installed, not a cache. It is
// duplicated rather than shared because `dr-decode` sits far below the UI
// and must not acquire a dependency on it to find a directory.
let base = std::env::var_os("XDG_DATA_HOME")
.map(PathBuf::from)
.or_else(|| std::env::var_os("HOME").map(|h| Path::new(&h).join(".local/share")));
if let Some(base) = base {
dirs.push(base.join("darkroom").join("profiles"));
}
dirs
}
/// A manufacturer's first word, folded.
///
/// "NIKON CORPORATION", "Nikon" and "nikon" all become `NIKON`. The corporate
/// suffixes are not information — they appear or not depending on whether the
/// file went through a DNG converter — and no two camera manufacturers share a
/// first word, so nothing is lost by dropping them.
fn make_key(s: &str) -> String {
normalise(s)
.split(' ')
.next()
.unwrap_or_default()
.to_string()
}
/// Fold a make or model into something two files can agree on.
///
/// Upper-cased, with every run of non-alphanumeric characters collapsed to one
/// space and the ends trimmed, so that "ILCE-7M3", "ILCE 7M3" and "ilce-7m3"
/// become one.
fn normalise(s: &str) -> String {
let mut out = String::with_capacity(s.len());
let mut pending_space = false;
for c in s.chars() {
if c.is_ascii_alphanumeric() {
if pending_space && !out.is_empty() {
out.push(' ');
}
pending_space = false;
out.push(c.to_ascii_uppercase());
} else {
pending_space = true;
}
}
out
}
// ---- The on-disk shape, kept apart from the in-memory one ----------------
//
// Deliberately separate types. The file is data a user edits and is allowed to
// be wrong; `Curves` is a parsed database whose every entry is known to be a
// monotone curve. Deriving `Deserialize` on `BaseCurve` directly would delete
// that boundary and let an unchecked five-point array reach the shader.
//
// Unknown fields are **accepted**, which is not laziness. The database is
// versioned independently of the binary and moves in both directions: a pack
// published after this release may carry keys this build has never heard of —
// a hue twist, a look table (FR-DEV-3f) — and it must still deliver its curves
// to an older DarkRoom rather than failing to parse and leaving every body
// flat. `deny_unknown_fields` would trade that for a diagnostic nobody needs.
#[derive(serde::Deserialize)]
struct File {
version: u32,
#[serde(default)]
default: Option<Entry>,
#[serde(default)]
bodies: Vec<BodyEntry>,
}
#[derive(serde::Deserialize)]
struct Entry {
points: Vec<[f32; 2]>,
}
#[derive(serde::Deserialize)]
struct BodyEntry {
make: String,
model: String,
points: Vec<[f32; 2]>,
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn the_shipped_database_parses_and_carries_a_default() {
// The one test that must never be allowed to fail quietly: `load`
// degrades to an empty database rather than panicking, so without this
// a typo in the YAML would ship as "every photograph renders flat"
// rather than as a build failure.
let curves = Curves::parse(BUILT_IN).expect("the shipped database parses");
assert!(curves.version() >= 1);
assert!(!curves.is_empty(), "the database ships bodies");
assert!(
!curves.body("Nobody", "Nothing").is_identity(),
"an unknown body must still get the default rendering"
);
}
#[test]
fn every_shipped_curve_lifts_the_midtones_and_rolls_the_highlights() {
// What makes a base curve a base curve rather than a decoration. If a
// shipped curve failed either half it would be a worse rendering than
// the flat one it replaced, which is the one outcome forbidden.
let curves = Curves::parse(BUILT_IN).expect("parses");
let all = curves
.bodies
.iter()
.map(|b| (format!("{} {}", b.make, b.model), b.curve))
.chain(curves.default.map(|c| ("default".to_string(), c)));
for (name, curve) in all {
// The midtone point sits above the diagonal: a linear midtone is
// roughly a stop and a half darker than any camera renders it.
let mid = 2;
assert!(
curve.ys[mid] > curve.xs[mid],
"{name} does not lift its midtones ({} -> {})",
curve.xs[mid],
curve.ys[mid]
);
// And the last span is shallower than the one before it, which is
// what a shoulder *is*. Without one the curve clips highlights
// harder than the linear rendering did.
let slope = |i: usize| (curve.ys[i + 1] - curve.ys[i]) / (curve.xs[i + 1] - curve.xs[i]);
assert!(
slope(POINTS - 2) < slope(POINTS - 3),
"{name} has no highlight shoulder"
);
}
}
#[test]
fn a_curve_that_is_not_monotone_is_refused() {
// The profile file is user-editable, so this is a real boundary and
// not a formality. A decreasing y inverts tones locally and shows up
// as a dark halo in a gradient, which reads as a rendering fault
// rather than as a bad profile.
assert_eq!(
BaseCurve::from_points(&[
[0.0, 0.0],
[0.25, 0.4],
[0.5, 0.3],
[0.75, 0.8],
[1.0, 1.0]
]),
None
);
}
#[test]
fn a_curve_whose_x_does_not_advance_is_refused() {
// The spline divides by the span width; a repeated x is a division by
// zero in the shader, which is a NaN pixel rather than an error.
assert_eq!(
BaseCurve::from_points(&[
[0.0, 0.0],
[0.25, 0.3],
[0.25, 0.5],
[0.75, 0.8],
[1.0, 1.0]
]),
None
);
}
#[test]
fn a_curve_of_the_wrong_length_is_refused() {
assert_eq!(BaseCurve::from_points(&[[0.0, 0.0], [1.0, 1.0]]), None);
}
#[test]
fn values_outside_the_unit_square_are_refused() {
// The shader clamps its output at the very end anyway, but a control
// point above 1.0 would put the shoulder outside the range the curve
// is defined over and silently flatten everything below it.
assert_eq!(
BaseCurve::from_points(&[
[0.0, 0.0],
[0.25, 0.3],
[0.5, 1.4],
[0.75, 1.5],
[1.0, 1.6]
]),
None
);
}
#[test]
fn a_body_with_its_own_entry_beats_the_default() {
let curves = Curves::parse(
"version: 2
default:
points: [[0.0, 0.0], [0.25, 0.3], [0.5, 0.6], [0.75, 0.85], [1.0, 1.0]]
bodies:
- make: Canon
model: EOS 6D
points: [[0.0, 0.0], [0.25, 0.35], [0.5, 0.7], [0.75, 0.9], [1.0, 1.0]]
",
)
.expect("parses");
assert_eq!(curves.body("Canon", "EOS 6D").ys[1], 0.35);
assert_eq!(curves.body("Canon", "EOS 5D").ys[1], 0.30);
}
#[test]
fn the_make_may_be_repeated_in_the_model() {
// Canon writes "Canon" as the make and "Canon EOS 6D" as the model;
// rawler's cleaned strings drop the repetition and both reach here.
// One entry has to cover both or half the files on a card miss.
let curves = Curves::parse(
"version: 1
bodies:
- make: Canon
model: EOS 6D
points: [[0.0, 0.0], [0.25, 0.35], [0.5, 0.7], [0.75, 0.9], [1.0, 1.0]]
",
)
.expect("parses");
assert_eq!(curves.body("Canon", "Canon EOS 6D").ys[1], 0.35);
assert_eq!(curves.body("Canon", "EOS 6D").ys[1], 0.35);
assert_eq!(curves.body("CANON", "eos 6d").ys[1], 0.35);
}
#[test]
fn a_corporate_suffix_does_not_hide_a_body() {
// The same Z 6 arrives as "Nikon"/"Z 6" from rawler's camera database
// and as "NIKON CORPORATION"/"NIKON Z 6" from a DNG converted out of
// the same file. Both must find the entry, or converting a file to
// DNG would silently change how it renders.
let curves = Curves::parse(
"version: 1
bodies:
- make: Nikon
model: Z 6
points: [[0.0, 0.0], [0.25, 0.35], [0.5, 0.7], [0.75, 0.9], [1.0, 1.0]]
",
)
.expect("parses");
assert_eq!(curves.body("Nikon", "Z 6").ys[1], 0.35);
assert_eq!(curves.body("NIKON CORPORATION", "NIKON Z 6").ys[1], 0.35);
}
#[test]
fn punctuation_and_spacing_do_not_decide_whether_a_body_is_known() {
let curves = Curves::parse(
"version: 1
bodies:
- make: Sony
model: ILCE-7M3
points: [[0.0, 0.0], [0.25, 0.35], [0.5, 0.7], [0.75, 0.9], [1.0, 1.0]]
",
)
.expect("parses");
assert_eq!(curves.body("SONY", "ILCE 7M3").ys[1], 0.35);
assert_eq!(curves.body("sony", "ilce-7m3").ys[1], 0.35);
}
#[test]
fn one_bad_entry_does_not_cost_the_rest() {
// A user-contributed file with one typo should cost that body's
// rendering, not every body's.
let curves = Curves::parse(
"version: 1
bodies:
- make: Broken
model: Body
points: [[0.0, 0.0], [0.25, 0.9], [0.5, 0.1], [0.75, 0.9], [1.0, 1.0]]
- make: Canon
model: EOS 6D
points: [[0.0, 0.0], [0.25, 0.35], [0.5, 0.7], [0.75, 0.9], [1.0, 1.0]]
",
)
.expect("parses");
assert_eq!(curves.len(), 1);
assert_eq!(curves.body("Canon", "EOS 6D").ys[1], 0.35);
assert!(curves.body("Broken", "Body").is_identity());
}
#[test]
fn a_pack_from_the_future_still_delivers_its_curves() {
// The database is versioned independently of the binary, so a pack
// published after this build may carry keys this build has never heard
// of. It must still hand over the curves it does understand — failing
// the parse would leave every body flat, which is the exact failure
// FR-DEV-3e exists to prevent, delivered by the mechanism meant to
// prevent it.
let curves = Curves::parse(
"version: 9
look_table: ambitious
bodies:
- make: Canon
model: EOS 6D
hue_twist: [1, 2, 3]
points: [[0.0, 0.0], [0.25, 0.35], [0.5, 0.7], [0.75, 0.9], [1.0, 1.0]]
",
)
.expect("an unfamiliar key must not fail the parse");
assert_eq!(curves.version(), 9);
assert_eq!(curves.body("Canon", "EOS 6D").ys[1], 0.35);
}
#[test]
fn an_unknown_body_with_no_default_gets_the_identity() {
// Graceful fallback, stated as a property: never worse than a flat
// render, and never a curve tuned for somebody else's sensor when the
// database declines to offer one.
let curves = Curves::parse("version: 1\nbodies: []\n").expect("parses");
assert!(curves.body("Nobody", "Nothing").is_identity());
}
/// A directory holding one `base_curves.yaml`, unique to the caller.
fn a_pack_dir(name: &str, yaml: &str) -> PathBuf {
let dir = std::env::temp_dir().join(format!("darkroom-base-curves-{name}"));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).expect("a writable temp directory");
std::fs::write(dir.join("base_curves.yaml"), yaml).expect("write");
dir
}
const A_CANON_ENTRY: &str = "bodies:
- make: Canon
model: EOS 6D
points: [[0.0, 0.0], [0.25, 0.42], [0.5, 0.7], [0.75, 0.9], [1.0, 1.0]]
";
#[test]
fn a_newer_pack_on_disk_supersedes_the_built_in_database() {
// **This is the requirement.** FR-DEV-3e asks for a profile database
// versioned independently of the app binary "so bodies and curves can
// be added without a release". A file with a higher version, dropped
// in the profile directory, is what that means in practice.
let built_in = Curves::parse(BUILT_IN).expect("parses");
let newer = format!("version: {}\n{A_CANON_ENTRY}", built_in.version() + 1);
let dir = a_pack_dir("newer", &newer);
let chosen = choose(built_in.clone(), &[dir]);
assert_eq!(chosen.version(), built_in.version() + 1);
assert_eq!(chosen.body("Canon", "EOS 6D").ys[1], 0.42);
}
#[test]
fn a_stale_pack_does_not_survive_an_upgrade() {
// The other direction, and the one that protects the user. Somebody
// downloads a pack, a release later ships better curves for the same
// bodies, and the forgotten file must not quietly hold the application
// back at last year's rendering.
let built_in = Curves::parse(BUILT_IN).expect("parses");
let stale = format!("version: {}\n{A_CANON_ENTRY}", built_in.version());
let dir = a_pack_dir("stale", &stale);
let chosen = choose(built_in.clone(), &[dir]);
assert_eq!(chosen.version(), built_in.version());
assert_ne!(
chosen.body("Canon", "EOS 6D").ys[1],
0.42,
"an equal version must not displace the built-in database"
);
}
#[test]
fn a_broken_pack_costs_the_curves_and_not_the_photographs() {
// A malformed profile file must degrade to the built-in database, not
// to an error. The user came here to look at a photograph.
let built_in = Curves::parse(BUILT_IN).expect("parses");
let dir = a_pack_dir("broken", "version: [this is not a number\n");
let chosen = choose(built_in.clone(), &[dir]);
assert_eq!(chosen.version(), built_in.version());
assert_eq!(chosen.len(), built_in.len());
}
#[test]
fn a_directory_with_no_pack_in_it_is_simply_skipped() {
// The ordinary case on every machine: the search path exists, the file
// does not. It must not be a warning, an error, or a slow path.
let built_in = Curves::parse(BUILT_IN).expect("parses");
let missing = std::env::temp_dir().join("darkroom-base-curves-nothing-here");
let _ = std::fs::remove_dir_all(&missing);
assert_eq!(choose(built_in.clone(), &[missing]), built_in);
}
#[test]
fn the_identity_is_recognised_as_doing_nothing() {
assert!(BaseCurve::IDENTITY.is_identity());
assert!(!Curves::parse(BUILT_IN)
.expect("parses")
.body("Canon", "EOS 6D")
.is_identity());
}
}
core/dr-decode/src/lib.rs
+52 -66
View File
@@ -12,10 +12,13 @@
//! Fusing them would force a full decode where a header read suffices, which
//! is exactly why Lightroom stalls ~2 s per image during culling.
pub mod base_curve;
mod error;
mod locate;
mod preview;
pub mod profile;
pub use base_curve::BaseCurve;
pub use error::DecodeError;
pub use locate::{
defects, is_complete_jpeg, locate_preview, BadLine, BadPixel, Defects, PreviewLocation,
@@ -25,6 +28,7 @@ pub use preview::{
decode_jpeg, extract_embedded_preview, extract_preview, Preview, PreviewSize,
PREVIEW_PROBE_BYTES,
};
pub use profile::CameraProfile;
use dr_types::{Format, Orientation};
@@ -89,7 +93,24 @@ pub struct RawImage {
/// `None` where the body is unknown to the decoder, in which case the
/// pipeline falls back to identity and the result is uncalibrated rather
/// than wrong-by-a-guess.
///
/// Where the body carries two calibrations this is already *interpolated*
/// for the light the frame was shot under; see [`profile::CameraProfile`].
pub color_matrix: Option<[f32; 9]>,
/// TRACES: FR-DEV-3e
/// The per-body rendering curve, the other half of the camera profile.
///
/// The matrix above decides what the colours *are*; this decides what the
/// picture looks like. Carried on the decoded image rather than looked up
/// downstream because this is the only point in the system that knows
/// which body took the frame, and because it is not an edit: it belongs to
/// the file in the same way the masked-photosite crop does, and must never
/// reach a sidecar (FR-NC-9).
///
/// [`BaseCurve::IDENTITY`] for an unknown body with no default in the
/// database, which renders exactly as this decoder did before profiles
/// existed.
pub base_curve: BaseCurve,
/// The usable region of `data`, excluding masked and border photosites.
pub crop: CropRect,
}
@@ -421,15 +442,38 @@ pub fn decode(bytes: &[u8]) -> Result<RawImage, DecodeError> {
let source = RawSource::new_from_slice(bytes);
let decoder =
rawler::get_decoder(&source).map_err(|e| DecodeError::Unsupported(e.to_string()))?;
// Read before decoding, while the decoder is still the cheapest thing in
// the room. These are the DNG tags rawler parses into its IFD and then
// never surfaces — `ForwardMatrix1/2` above all — and they are empty for
// every non-DNG file, which is not a failure (FR-DEV-3e).
let dng = profile::read_dng_matrices(decoder.as_ref());
let image = decoder
.raw_image(&source, &Default::default(), false)
.map_err(|e| DecodeError::Decode(e.to_string()))?;
// Both derived before the match below moves `image.data`, and from the
// same matrix: the balance and the conversion must agree about which white
// is neutral or the frame carries a cast that looks like a decode fault.
let color_matrix = cam_to_srgb(&image);
let wb_coeffs = sane_wb(image.wb_coeffs, xyz_to_cam_of(&image).as_ref());
// The camera profile, and both of the things derived from it, are built
// before the match below moves `image.data`.
//
// They come from *one* profile deliberately: the balance and the
// conversion must agree about which white is neutral, or the frame carries
// a cast that looks like a decode fault. That agreement used to be
// maintained by hand — two functions reading the same illuminant key — and
// is now structural, because there is only one interpolated matrix and
// both callers ask the same object for it.
let profile = profile::CameraProfile::extract(&image, &dng);
let color_matrix = profile.as_ref().and_then(|p| p.cam_to_srgb());
let wb_coeffs = sane_wb(image.wb_coeffs, profile.as_ref().map(|p| p.xyz_to_cam()).as_ref());
// The rendering half of the profile (FR-DEV-3e). rawler's cleaned strings
// are preferred where it has them — they are what the shipped database is
// written against — and the matching folds the variants either way, so a
// DNG naming the same body differently still finds its curve.
let base_curve = base_curve::for_body(
image.camera.clean_make.as_str(),
image.camera.clean_model.as_str(),
);
let data = match image.data {
rawler::RawImageData::Integer(v) => v,
@@ -498,68 +542,10 @@ pub fn decode(bytes: &[u8]) -> Result<RawImage, DecodeError> {
.unwrap_or(u16::MAX),
wb_coeffs,
color_matrix,
base_curve,
})
}
/// TRACES: FR-DEV-3e
/// Compose the camera→sRGB-linear matrix from rawler's XYZ→camera.
///
/// **Two rawler traps this avoids**, both measured on a Canon 6D CR2
/// (2026-08-09):
///
/// 1. `RawImage::xyz_to_cam` is **all zeros** — it carries an upstream
/// deprecation note and 0.7.2 no longer fills it. The live data is
/// `color_matrix`, keyed by illuminant. Reading the old field silently
/// yields no colour transform at all.
/// 2. `cam_to_xyz_normalized()` divides each of four rows by its own sum, and
/// the fourth row (emerald/white, unused on any Bayer body) sums to zero.
/// Every element came back `NaN`. Inverting the 3×3 ourselves avoids the
/// fourth channel entirely.
fn cam_to_srgb(image: &rawler::RawImage) -> Option<[f32; 9]> {
use rawler::imgop::xyz::Illuminant;
// Prefer D65 — it matches sRGB's white point, so no chromatic adaptation
// is needed. Illuminant A (tungsten) is a distant fallback for bodies
// that ship only one matrix; adapting it properly is a v0.2 colour-
// management concern (ARCH §5.2), not something to fake here.
let flat = image
.color_matrix
.get(&Illuminant::D65)
.or_else(|| image.color_matrix.get(&Illuminant::A))?;
if flat.len() < 9 {
return None;
}
let xyz_to_cam: [[f32; 3]; 3] = [
[flat[0], flat[1], flat[2]],
[flat[3], flat[4], flat[5]],
[flat[6], flat[7], flat[8]],
];
cam_to_srgb_from(&xyz_to_cam)
}
/// The camera's XYZ→camera matrix, as rawler holds it.
///
/// Split out so the white-balance fallback and the colour matrix read the same
/// data through the same illuminant preference; two readers disagreeing about
/// which matrix a body uses would balance to one white and convert from
/// another.
fn xyz_to_cam_of(image: &rawler::RawImage) -> Option<[[f32; 3]; 3]> {
use rawler::imgop::xyz::Illuminant;
let flat = image
.color_matrix
.get(&Illuminant::D65)
.or_else(|| image.color_matrix.get(&Illuminant::A))?;
if flat.len() < 9 {
return None;
}
Some([
[flat[0], flat[1], flat[2]],
[flat[3], flat[4], flat[5]],
[flat[6], flat[7], flat[8]],
])
}
/// The matrix maths, split out so it can be tested without a RAW file.
//
// The constants below are quoted at their published precision rather than
@@ -567,7 +553,7 @@ fn xyz_to_cam_of(image: &rawler::RawImage) -> Option<[[f32; 3]; 3]> {
// a linter makes it harder to check against the specification, and the
// rounding happens identically either way.
#[allow(clippy::excessive_precision)]
fn cam_to_srgb_from(xyz_to_cam: &[[f32; 3]; 3]) -> Option<[f32; 9]> {
pub(crate) fn cam_to_srgb_from(xyz_to_cam: &[[f32; 3]; 3]) -> Option<[f32; 9]> {
// XYZ (D65) → linear sRGB, the standard primaries.
const XYZ_TO_SRGB: [[f32; 3]; 3] = [
[3.2404542, -1.5371385, -0.4985314],
@@ -722,7 +708,7 @@ pub fn daylight_wb(xyz_to_cam: &[[f32; 3]; 3]) -> Option<[f32; 3]> {
}
/// Invert a 3×3 matrix, or `None` if it is singular.
fn invert3(m: &[[f32; 3]; 3]) -> Option<[[f32; 3]; 3]> {
pub(crate) fn invert3(m: &[[f32; 3]; 3]) -> Option<[[f32; 3]; 3]> {
let det = m[0][0] * (m[1][1] * m[2][2] - m[1][2] * m[2][1])
- m[0][1] * (m[1][0] * m[2][2] - m[1][2] * m[2][0])
+ m[0][2] * (m[1][0] * m[2][1] - m[1][1] * m[2][0]);
core/dr-decode/src/profile.rs
+1081
View File
File diff suppressed because it is too large Load Diff
core/dr-gpu/src/adjust.rs
+31 -1
View File
@@ -34,6 +34,16 @@ use crate::{DemosaicedImage, GpuContext, GpuError};
/// reads them.
const RESERVED_FIELDS: usize = dr_pipeline::RESERVED_UNIFORM_FIELDS;
/// TRACES: FR-DEV-3e
/// The two crates must agree on how many points a base curve has.
///
/// `dr-decode` reads them from the profile database and `dr-pipeline` declares
/// the uniform slots; this file is the only place the two meet, and it packs
/// them by index. A disagreement would not fail to compile — it would upload a
/// curve with a point missing or a stale float in it, which renders as a
/// plausible-looking wrong tone response. Cheaper to catch here, at build time.
const _: () = assert!(dr_decode::base_curve::POINTS == dr_pipeline::BASE_CURVE_POINTS);
/// Runs composed operation chains against demosaiced images.
pub struct AdjustPass {
ctx: GpuContext,
@@ -677,6 +687,22 @@ impl AdjustPass {
// runs. See `DemosaicedImage::is_non_linear`.
let non_linear = if source.is_non_linear() { 1.0 } else { 0.0 };
uniforms[12..16].copy_from_slice(&[wb[0], wb[1], wb[2], non_linear]);
// TRACES: FR-DEV-3e
// The camera profile's base curve, packed the way the generated block
// declares it: four x, four y, then the fifth point and the flag. The
// flag is what lets one compiled shader serve a profiled body and an
// unprofiled one, so the pipeline cache is not split in two by which
// camera took the frame.
//
// Written here rather than at the call site so that *both* callers —
// the plain render and the masked one — carry the profile. Filling it
// at one of them was how the two halves of this merge each had it.
let curve = source.base_curve();
let on = if curve.is_identity() { 0.0 } else { 1.0 };
let b = dr_pipeline::BASE_CURVE_UNIFORM_OFFSET;
uniforms[b..b + 4].copy_from_slice(&curve.xs[0..4]);
uniforms[b + 4..b + 8].copy_from_slice(&curve.ys[0..4]);
uniforms[b + 8..b + 12].copy_from_slice(&[curve.xs[4], curve.ys[4], on, 0.0]);
uniforms
}
@@ -862,7 +888,7 @@ pub(crate) fn numbered(src: &str) -> String {
#[cfg(test)]
mod tests {
use super::*;
use dr_decode::{CfaPattern, CropRect, RawImage};
use dr_decode::{BaseCurve, CfaPattern, CropRect, RawImage};
use dr_pipeline::ops::{colour_mixer, exposure, saturation};
use dr_pipeline::EditGraph;
@@ -896,6 +922,7 @@ mod tests {
// Identity, so the test reasons about the operations alone
// rather than about a camera's colour response.
color_matrix: Some([1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]),
base_curve: BaseCurve::IDENTITY,
crop: CropRect {
x: 0,
y: 0,
@@ -1069,6 +1096,7 @@ mod tests {
white_level: 16383,
wb_coeffs: [1.0, 1.0, 1.0, 1.0],
color_matrix: Some([1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]),
base_curve: BaseCurve::IDENTITY,
crop: CropRect {
x: 0,
y: 0,
@@ -1491,6 +1519,7 @@ mod tests {
white_level: 16383,
wb_coeffs: [1.0, 1.0, 1.0, 1.0],
color_matrix: Some([1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]),
base_curve: BaseCurve::IDENTITY,
crop: CropRect {
x: 0,
y: 0,
@@ -1590,6 +1619,7 @@ mod tests {
white_level: 16383,
wb_coeffs: [1.0, 1.0, 1.0, 1.0],
color_matrix: Some([1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]),
base_curve: BaseCurve::IDENTITY,
crop: CropRect {
x: 0,
y: 0,
core/dr-gpu/src/demosaic.rs
+34 -1
View File
@@ -10,7 +10,7 @@
//! pass over this texture; it does not re-demosaic, which is what keeps the
//! interaction budget (NFR-P9) reachable on a 24 MP file.
use dr_decode::{CfaPattern, RawImage};
use dr_decode::{BaseCurve, CfaPattern, RawImage};
use wgpu::util::DeviceExt;
use crate::{GpuContext, GpuError};
@@ -83,6 +83,16 @@ pub struct DemosaicedImage {
color_matrix: [f32; 9],
/// As-shot white balance, the neutral starting point for the WB control.
as_shot_wb: [f32; 3],
/// TRACES: FR-DEV-3e
/// The camera profile's rendering curve, carried through for the adjust
/// pass exactly as `color_matrix` is.
///
/// It rides on the image rather than on the edit graph because it is not
/// an edit: it belongs to the body that took the frame, the way the
/// masked-photosite crop and the EXIF orientation do, and a sidecar shared
/// between two bodies must never carry one body's rendering onto the
/// other's file (FR-NC-9).
base_curve: BaseCurve,
/// Whether the texture holds gamma-encoded rather than linear values.
non_linear: bool,
}
@@ -108,6 +118,15 @@ impl DemosaicedImage {
self.color_matrix
}
/// TRACES: FR-DEV-3e
/// The camera profile's base curve, as five `(x, y)` points.
///
/// [`BaseCurve::IDENTITY`] where the body is unprofiled or the source was
/// never raw, in which case the adjust pass skips the stage entirely.
pub fn base_curve(&self) -> BaseCurve {
self.base_curve
}
/// As-shot white balance multipliers, green-normalised.
///
/// The white balance control is expressed *relative* to these, so its
@@ -220,6 +239,12 @@ impl DemosaicedImage {
height,
color_matrix: IDENTITY_3X3,
as_shot_wb: [1.0, 1.0, 1.0],
// **The identity, and this is the whole reason the field is here
// rather than resolved further down.** A JPEG has already had its
// camera's base curve baked in by the camera; applying one again
// would render the rendering, crushing the shadows and flattening
// the highlights of an image that was already finished.
base_curve: BaseCurve::IDENTITY,
non_linear: true,
})
}
@@ -495,6 +520,10 @@ impl Demosaicer {
// no colour transform rather than not at all.
color_matrix: raw.color_matrix.unwrap_or(IDENTITY_3X3),
as_shot_wb: [raw.wb_coeffs[0], raw.wb_coeffs[1], raw.wb_coeffs[2]],
// Whatever the profile database had for this body (FR-DEV-3e),
// resolved at decode because that is the only place the make and
// model are known.
base_curve: raw.base_curve,
// Sensor data is linear by construction — the demosaic shader
// normalises against black and white levels and applies no
// transfer function.
@@ -808,6 +837,7 @@ mod tests {
white_level: white,
wb_coeffs: [1.0, 1.0, 1.0, 1.0],
color_matrix: None,
base_curve: BaseCurve::IDENTITY,
crop: CropRect {
x: 0,
y: 0,
@@ -919,6 +949,7 @@ mod tests {
white_level: white,
wb_coeffs: [1.0, 1.0, 1.0, 1.0],
color_matrix: None,
base_curve: BaseCurve::IDENTITY,
crop: CropRect {
x: 0,
y: 0,
@@ -1164,6 +1195,7 @@ mod tests {
white_level: 16383,
wb_coeffs: [1.0, 1.0, 1.0, 1.0],
color_matrix: None,
base_curve: BaseCurve::IDENTITY,
crop: CropRect {
x: 0,
y: 0,
@@ -1244,6 +1276,7 @@ mod tests {
1.0,
],
color_matrix: None,
base_curve: BaseCurve::IDENTITY,
crop: CropRect {
x: 0,
y: 0,
core/dr-gpu/tests/base_curve.rs
+179
View File
@@ -0,0 +1,179 @@
//! TRACES: FR-DEV-3e
//! The camera profile's base curve, end to end on a device.
//!
//! The unit tests either side of this one check halves. `dr-decode` asserts
//! that the shipped database parses and that every curve in it lifts its
//! midtones; `dr-pipeline` asserts that the generated WGSL evaluates a curve
//! in the right place. Neither would notice if the two agreed with each other
//! and both were wrong — a curve packed into the wrong uniform slots, or a
//! flag read from the wrong component, satisfies both and renders nothing.
//!
//! So this renders real pixels twice, once with a profiled body's curve and
//! once with the identity, and asserts the difference is the one a base curve
//! is for: midtones lifted, black still black, white still white.
use dr_decode::{BaseCurve, CfaPattern, CropRect, RawImage};
use dr_gpu::{AdjustPass, Demosaicer, GpuContext};
use dr_pipeline::EditGraph;
const SIZE: u32 = 16;
fn ctx() -> Option<GpuContext> {
pollster::block_on(GpuContext::new_headless()).ok()
}
/// A flat RGGB frame at `level` out of 65535, carrying `curve`.
///
/// Every photosite the same value, so the demosaic result is a uniform grey
/// and the only thing that can move a pixel is the curve. The colour matrix is
/// the identity and the balance is neutral for the same reason: this test is
/// about one stage, and a real body's matrix would make every assertion below
/// a statement about that body instead.
fn flat_raw(level: u16, curve: BaseCurve) -> RawImage {
RawImage {
width: SIZE,
height: SIZE,
data: vec![level; (SIZE * SIZE) as usize],
cfa_pattern: CfaPattern::Rggb,
black_level: [0; 4],
white_level: u16::MAX,
wb_coeffs: [1.0, 1.0, 1.0, 1.0],
color_matrix: Some([1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]),
base_curve: curve,
crop: CropRect {
x: 0,
y: 0,
width: SIZE,
height: SIZE,
},
}
}
/// Render a neutral edit over a flat frame and return the centre pixel's red.
///
/// The centre rather than a corner: a demosaic has to invent its edges, and
/// the interpolated border of a 16×16 frame is not where anyone should be
/// reading a tone off.
fn rendered_level(ctx: &GpuContext, level: u16, curve: BaseCurve) -> u8 {
let raw = flat_raw(level, curve);
let source = Demosaicer::new(ctx)
.expect("demosaicer")
.run(&raw)
.expect("demosaic");
let shader = EditGraph::default_chain().compose();
let mut adjust = AdjustPass::new(ctx);
adjust
.render(&source, &shader, SIZE, SIZE)
.expect("render");
let (pixels, _, _) = adjust.export_pixels().expect("readback");
let centre = ((SIZE / 2) * SIZE + SIZE / 2) * 4;
pixels[centre as usize]
}
/// The Canon EOS 6D's curve, from the shipped profile database.
///
/// Looked up by name rather than written out, so this also asserts the thing
/// no other test can: that a curve travels from the YAML, through the body
/// match, onto the decoded image and into the uniform block that the shader
/// actually reads.
fn six_d() -> BaseCurve {
let curve = dr_decode::base_curve::for_body("Canon", "EOS 6D");
assert!(
!curve.is_identity(),
"the shipped database must have a curve for the EOS 6D"
);
curve
}
#[test]
fn a_profiled_body_renders_brighter_midtones_than_a_flat_one() {
// **The whole requirement, in one assertion.** A linear midtone renders
// roughly half a stop dark, which is the flat, lifeless look FR-DEV-3e
// exists to get away from. If the curve did not reach the shader — wrong
// slot, wrong flag, wrong stage — this is the only test that would fail.
let Some(ctx) = ctx() else {
eprintln!("skipping: no GPU adapter");
return;
};
// 13% of full scale: roughly where a camera places middle grey, leaving
// about two and a half stops of highlight headroom above it.
let level = (0.13 * 65535.0) as u16;
let flat = rendered_level(&ctx, level, BaseCurve::IDENTITY);
let profiled = rendered_level(&ctx, level, six_d());
assert!(
profiled > flat + 8,
"the profile lifted middle grey from {flat} only to {profiled}"
);
}
#[test]
fn the_curve_leaves_black_black_and_white_white() {
// A base curve renders the range between the endpoints; it must not move
// the endpoints themselves. A curve that lifted black would put a grey
// veil over every night photograph, and one that pulled white down would
// make a correctly exposed frame look underexposed.
let Some(ctx) = ctx() else {
eprintln!("skipping: no GPU adapter");
return;
};
let curve = six_d();
assert_eq!(rendered_level(&ctx, 0, curve), 0, "black moved");
assert_eq!(rendered_level(&ctx, u16::MAX, curve), 255, "white moved");
}
#[test]
fn an_unprofiled_body_renders_exactly_as_it_did_before_profiles_existed() {
// The graceful fallback, asserted as a number rather than as a promise.
// With no curve the pipeline must still be a pass-through: black level
// out, white level in, sRGB encoding on the way to the screen and nothing
// else. "Never worse than today" is the one property this change was not
// allowed to trade away, and the way it would break is silently — a flag
// read from the wrong component would apply a curve nobody asked for.
let Some(ctx) = ctx() else {
eprintln!("skipping: no GPU adapter");
return;
};
for level in [0u16, 4_000, 8_520, 32_768, 60_000, u16::MAX] {
let scene = f32::from(level) / f32::from(u16::MAX);
let expected = (dr_types::Transfer::Srgb.encode(scene) * 255.0).round() as i32;
let got = i32::from(rendered_level(&ctx, level, BaseCurve::IDENTITY));
// Two 8-bit steps: the texture holding the demosaiced frame is
// `Rgba16Float`, so a value round-trips through eleven mantissa bits
// before it is encoded. That is well under one step at any level, and
// the tolerance is for the rounding either side of it rather than for
// the transform being approximate.
assert!(
(got - expected).abs() <= 2,
"raw {level} rendered as {got}, expected about {expected}"
);
}
}
#[test]
fn the_curve_is_monotone_through_the_whole_range() {
// The property the spline's tangent limiting exists to guarantee, checked
// where it actually matters: on the device, through the real uniform
// packing. A curve that dipped anywhere would put a dark band across a
// smooth gradient — a sky, most visibly — and it would read as a
// rendering fault rather than as a bad profile.
let Some(ctx) = ctx() else {
eprintln!("skipping: no GPU adapter");
return;
};
let curve = six_d();
let mut previous = 0u8;
for step in 0..=16u32 {
let level = (step * 65535 / 16) as u16;
let value = rendered_level(&ctx, level, curve);
assert!(
value >= previous,
"the curve fell from {previous} to {value} at raw level {level}"
);
previous = value;
}
}
core/dr-pipeline/ops/README.md
+38
View File
@@ -250,6 +250,44 @@ count of source pixels (`RenderScale::source_pixels` — capture sharpening,
luminance NR). `passes()` is given the scale and converts on the CPU. A radius
in raw pixels is a different photograph on screen and in the exported file.
## What is not a node, and why
Three things act on every pixel and are deliberately not in this directory:
the as-shot white balance, the camera matrix, and the **base curve**
(FR-DEV-3e). They are emitted by [`../src/operation.rs`](../src/operation.rs)
into the composed shader's fixed preamble, around the block of nodes.
The test is not "does it transform a colour" — all three do. It is **whose
decision is it**. A node is something a photographer chose: it has parameters,
it moves off a neutral, it lands in the sidecar, it can be undone. These three
are properties of the *file*, at the same standing as the masked-photosite crop
(FR-RAW-3) and the stored orientation (FR-DEV-3h). Nobody chose the sensor's
green sensitivity or the body's rendering; they are what reading the file
correctly means.
Making the base curve a node would have said the opposite in four places at
once. It would have appeared in the develop panel as a control, so an
unprofiled body would show a slider that does nothing. Its values would have
gone into the sidecar, and sidecars are shared between devices and bodies
(FR-NC-9) — one camera's rendering would follow an edit onto another camera's
file. Its neutral would have had to be "the identity", so a profiled body would
open reporting itself modified. And there is no seam through which a node could
learn which camera took the frame: the profile arrives on the decoded image,
travels through `DemosaicedImage` beside the matrix it belongs with, and is
written into the uniform block by the same three lines in `dr-gpu` — which is
exactly the path the matrix already took, because it is exactly the same kind
of thing.
What it *does* share with the tone curve node is the spline. The composer asks
`ToneCurve` for its `curve_span`/`curve_eval` helpers rather than emitting a
second copy, so a profile author placing a control point and a photographer
dragging one mean the same thing by it.
The order still reads correctly from this directory: the base curve runs after
every node in the chain and before the conversion out of camera space. That is
the same reasoning `exposure` records under `placement:` — corrections to
capture are only meaningful on linear values, so the rendering goes last.
## Errors
The build script reports failures by naming the key you got wrong, and exits
core/dr-pipeline/src/lib.rs
+1 -1
View File
@@ -56,7 +56,7 @@ pub use history::{Edit, History};
pub use lens::{compose_warps, ComposedWarp, Warp};
pub use operation::{
compose, compose_with_framing, Affects, ComposedShader, Helper, Invalidation, Operation,
OutputMode, Uniform, RESERVED_UNIFORM_FIELDS,
OutputMode, Uniform, BASE_CURVE_POINTS, BASE_CURVE_UNIFORM_OFFSET, RESERVED_UNIFORM_FIELDS,
};
pub use preset::{Preset, Scope};
pub use sidecar::{Sidecar, Version};
core/dr-pipeline/src/operation.rs
+194 -2
View File
@@ -360,7 +360,35 @@ pub struct ComposedShader {
///
/// WGSL requires a uniform struct to be non-empty and 16-byte aligned; these
/// are needed by every generated shader in any case.
const BASE_UNIFORM_FIELDS: usize = 16;
///
/// Twelve of the twenty-eight are the camera profile's base curve
/// ([`BASE_CURVE_UNIFORM_FIELDS`]); the rest are the matrix, the as-shot
/// balance and framing's own block.
const BASE_UNIFORM_FIELDS: usize = 16 + BASE_CURVE_UNIFORM_FIELDS;
/// TRACES: FR-DEV-3e
/// Slots the base curve occupies: five `(x, y)` points and an active flag.
///
/// Twelve rather than eleven so the block stays a whole number of `vec4`s,
/// which is what std140 requires of a uniform struct's members. The spare
/// float is left zero rather than repurposed — a uniform slot that means one
/// thing today and two things next year is how a shader comes to read a
/// highlight rolloff out of a crop rectangle.
const BASE_CURVE_UNIFORM_FIELDS: usize = 12;
/// TRACES: FR-DEV-3e
/// Where the base curve's slots begin in the generated uniform block.
///
/// Exported for the same reason [`RESERVED_UNIFORM_FIELDS`] is: `dr-gpu`
/// writes these by index, and an offset computed independently at both ends is
/// an offset that will eventually disagree with itself.
pub const BASE_CURVE_UNIFORM_OFFSET: usize = 16;
/// How many control points a base curve carries.
///
/// The same five the tone curve widget has, deliberately — see the helper
/// selection in [`compose_full`].
pub const BASE_CURVE_POINTS: usize = 5;
/// Where an operation's own uniforms begin in the generated block.
///
@@ -471,10 +499,44 @@ pub fn compose_full(
\x20 // `.w` is not padding: it flags a non-linear source (1.0 for a\n\
\x20 // gamma-encoded JPEG, 0.0 for demosaiced sensor data), which the\n\
\x20 // prologue reads to decide whether to linearise.\n\
\x20 as_shot_wb: vec4<f32>,\n",
\x20 as_shot_wb: vec4<f32>,\n\
\x20 // The camera profile's base curve (FR-DEV-3e): five points on a\n\
\x20 // monotone spline, packed as x0..x3, y0..y3, then (x4, y4, on).\n\
\x20 // `.z` of the last is the flag, not padding — it is 0 for a\n\
\x20 // body with no profile and for an already-rendered source.\n\
\x20 base_curve_x: vec4<f32>,\n\
\x20 base_curve_y: vec4<f32>,\n\
\x20 base_curve_last: vec4<f32>,\n",
);
uniform_values.resize(BASE_UNIFORM_FIELDS, 0.0);
// TRACES: FR-DEV-3e
// The spline the base curve is evaluated on is the *tone curve's* spline,
// reached through the trait rather than reimplemented here.
//
// Two reasons, and the second is the one that matters. The obvious one is
// that a shader carrying two `curve_eval`s would not compile, and the
// composer's helper de-duplication is what makes both stages able to ask
// for it. The real one is that a profile author placing a control point
// and a photographer dragging one must mean the same thing by it — down to
// the Fritsch-Carlson tangent limiting, which is what decides how a
// shoulder actually rolls off. Two implementations that agreed today would
// be two that could disagree later, and the disagreement would show up as
// a body whose profile renders subtly differently from the curve someone
// drew to match it.
//
// Emitted unconditionally, unlike an operation's helpers. The base curve
// is active for every RAW frame — an unprofiled body still gets the
// database's default rendering — so making the shader's shape depend on it
// would split the pipeline cache in two for no benefit. The uniform flag
// above turns it off for the cases that are genuinely already rendered,
// and a branch on a uniform is coherent across the whole dispatch.
for h in crate::ops::ToneCurve::new().helpers() {
if matches!(h.name, "curve_span" | "curve_eval") {
helpers.push(*h);
}
}
// Framing's block follows the base one at a fixed offset, for the same
// reason: the prologue is emitted whether or not any operation is active,
// so these slots cannot be positioned by the op loop below.
@@ -687,6 +749,58 @@ fn main(@builtin(global_invocation_id) gid: vec3<u32>) {{
c = mix(c, neutral, clipped);
}}
{body}
// ==== camera profile: the base curve (FR-DEV-3e) ====
//
// Marked with `====` and not the `----` an operation block carries: this
// is not one, and the difference is what several tests count on to tell
// an edit apart from the reading of a file.
//
// The stage between demosaic and the working space that turns a correct
// exposure into a photograph. Sensor data is scene-referred and nearly
// linear; nothing anybody looks at is. Rendering it straight out is the
// dcraw default, and it is flat, dark through the midtones and clips its
// highlights instead of rolling them off.
//
// **In camera RGB, and after the adjustments**, which is a deliberate pair
// of choices:
//
// - Before the matrix, because that is where a base curve is defined and
// where every other converter applies one. The curve was tuned against
// this body's own primaries; moving it after the conversion would apply
// a Canon rendering to sRGB values and change what it does.
// - After exposure and the tonal operations, because those are corrections
// to *capture* and are only meaningful on linear values. A stop is a
// doubling; run exposure after a curve and it stops being one.
//
// Per channel rather than on luminance. It desaturates the extremes
// slightly, and that is the point — it is what makes a blown sky roll
// toward white rather than toward a saturated corner of the gamut, and it
// is what the camera's own JPEG does.
//
// The branch is on a uniform, so the whole dispatch takes the same path.
// It is off for a JPEG and any other already-rendered source, which must
// not be rendered twice, and for a body the profile database declines to
// offer any curve for at all.
if (u.base_curve_last.z > 0.5) {{
c = vec3<f32>(
curve_eval(
u.base_curve_x.x, u.base_curve_y.x, u.base_curve_x.y, u.base_curve_y.y,
u.base_curve_x.z, u.base_curve_y.z, u.base_curve_x.w, u.base_curve_y.w,
u.base_curve_last.x, u.base_curve_last.y, c.r,
),
curve_eval(
u.base_curve_x.x, u.base_curve_y.x, u.base_curve_x.y, u.base_curve_y.y,
u.base_curve_x.z, u.base_curve_y.z, u.base_curve_x.w, u.base_curve_y.w,
u.base_curve_last.x, u.base_curve_last.y, c.g,
),
curve_eval(
u.base_curve_x.x, u.base_curve_y.x, u.base_curve_x.y, u.base_curve_y.y,
u.base_curve_x.z, u.base_curve_y.z, u.base_curve_x.w, u.base_curve_y.w,
u.base_curve_last.x, u.base_curve_last.y, c.b,
),
);
}}
// Camera space -> linear sRGB. Applied after the adjustments so white
// balance and exposure act on sensor-native values, which is where they
// are physically meaningful.
@@ -1225,6 +1339,84 @@ mod tests {
assert!(op < matrix, "the camera matrix must come after operations");
}
#[test]
fn the_base_curve_runs_after_the_operations_and_before_the_camera_matrix() {
// TRACES: FR-DEV-3e
// Both halves matter and for different reasons.
//
// After the operations: exposure and the tonal controls are
// corrections to capture, and they are only meaningful on linear
// values. A stop is a doubling; run exposure after a curve and it is
// not one any more, and every slider in the panel starts lying about
// what it does.
//
// Before the matrix: the curve was tuned against this body's own
// primaries. Applied after the conversion it would be a Canon
// rendering acting on sRGB values, which is a different curve.
let ops = vec![fake(&DESC_A, 2.0, false)];
let source = compose(&ops).source;
let op = source.find("---- op_a ----").expect("op present");
let curve = source
.find("if (u.base_curve_last.z > 0.5)")
.expect("base curve applied");
let matrix = source.find("u.cam_to_srgb_0").expect("matrix applied");
assert!(op < curve, "the base curve must come after the operations");
assert!(curve < matrix, "and before the camera matrix");
}
#[test]
fn the_base_curve_reaches_a_shader_with_no_operations_at_all() {
// TRACES: FR-DEV-3e
// The same property as as-shot white balance, and for the same reason:
// it is part of interpreting the file, not part of the edit. An
// unedited RAW must open looking like a photograph rather than like a
// scan of one.
let shader = compose(&[]);
assert!(shader.source.contains("u.base_curve_x"));
assert!(
shader.source.contains("fn curve_eval("),
"the spline it is evaluated on must be emitted too"
);
}
#[test]
fn the_base_curve_and_the_tone_curve_share_one_spline() {
// TRACES: FR-DEV-3e
// Two `curve_eval`s in one shader would not compile — but the reason
// the helper is *shared* rather than merely renamed is that a profile
// author placing a control point and a photographer dragging one must
// mean the same thing by it, down to the tangent limiting that decides
// how a shoulder rolls off.
let mut curve = crate::ops::ToneCurve::new();
curve.set_param(crate::ops::curve::P2_Y, 0.7);
assert!(curve.is_active(), "the fixture must actually reach the shader");
let source = compose(&[Box::new(curve)]).source;
assert_eq!(
source.matches("fn curve_eval(").count(),
1,
"the spline must be declared exactly once"
);
assert_eq!(source.matches("fn curve_span(").count(), 1);
}
#[test]
fn the_base_curve_owns_the_slots_dr_gpu_writes() {
// TRACES: FR-DEV-3e
// `dr-gpu` fills these by index. The offset is exported rather than
// recomputed there, and this asserts the exported number still points
// at the block the shader declares — the failure otherwise is a
// highlight rolloff read out of a crop rectangle, which renders as
// nonsense rather than as an error.
assert_eq!(
BASE_CURVE_UNIFORM_OFFSET + BASE_CURVE_UNIFORM_FIELDS,
BASE_UNIFORM_FIELDS,
"the base curve must be the last thing in the base block"
);
assert_eq!(BASE_CURVE_POINTS * 2 + 1, BASE_CURVE_UNIFORM_FIELDS - 1);
assert!(compose(&[]).uniforms.len() >= BASE_UNIFORM_FIELDS);
}
/// Compose with neutral framing into a chosen output space.
fn compose_to(ops: &[Box<dyn Operation>], output: ColourSpace) -> ComposedShader {
compose_with_framing(ops, &Framing::new(), output)