dtourolle/DarkRoom
1
0
Fork 0
Code Issues 39 Pull Requests Actions Packages Projects Releases 20 Wiki Activity

Render each body through the profile its own files describe

Browse Source 
Colour came from whichever matrix rawler happened to key `D65`, the second
one was discarded, and the rendering was left linear. That is the dcraw
default, and FR-DEV-3e names it as the reason people abandon a converter in
the first hour: correct in the abstract, flat and poor on skin in practice.

The decoder now builds a camera profile.

- `ColorMatrix1/2` and `CalibrationIlluminant1/2`. rawler surfaces these as
  an illuminant-keyed map — for DNGs from the tags, and for native formats
  from its own camera database — so a Canon CR2 arrives with a tungsten
  matrix and a daylight matrix exactly as an Adobe DNG of the same frame
  would. Dual-illuminant support is therefore not a DNG feature here.

- `ForwardMatrix1/2`, read straight from the root IFD, because rawler parses
  them and never surfaces them. Where a file carries both, they replace the
  inverted colour matrix: the same relationship measured in the direction
  rendering actually wants, rather than an inversion that amplifies the
  measurement error exactly where skin lives.

- `AsShotNeutral`, used to estimate what the scene was lit by and to
  interpolate between the two calibrations in mireds. The estimate is
  circular — the temperature needs a matrix and the matrix needs the
  temperature — so it is a fixed point, three rounds, as Adobe's SDK does it.

Bodies calibrated at neither D65 nor A stopped rendering uncalibrated as a
side effect: a Phase One IQ3 carries D55 and D75 and used to get no matrix
at all.

And a base curve, applied per channel in camera RGB between the last
adjustment and the conversion out of camera space — a toe, a steep midtone
and a shoulder, which is the difference between a photograph and a scan of
one. It is not an edit: no slider, nothing in the sidecar, because it
belongs to the body rather than to anything anyone decided, and a sidecar is
shared between bodies. It is not a develop node either, and `ops/README.md`
now records why. It evaluates on the tone curve's own spline rather than a
second copy, so a profile author placing a control point and a photographer
dragging one mean the same thing by it.

The curves are data. `core/dr-decode/profiles/base_curves.yaml` ships inside
the binary as a floor and is superseded by any copy on disk carrying a
higher `version:`, so a body can be added and distributed without a release
— and, under the GPL, contributed. The comparison runs both ways: a stale
pack cannot hold an upgraded binary back at last year's rendering.

Canon EOS 6D and R6, Nikon Z 6 and D750, Sony A7 III and Fujifilm X-T3 ship
with their own curves. Every other body gets a conservative default, which
is much closer to right than the identity is for any of them. A JPEG gets
none — it has already been rendered once, by the camera.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
dtourolle
2026-08-22 14:37:33 +02:00
co-authored by Claude Opus 5
parent 586698db00
commit 743fefe7f1
12 changed files with 2396 additions and 108 deletions
Download Patch File Download Diff File Expand all files Collapse all files
core/dr-pipeline/src/lib.rs
+1 -1
View File
@@ -52,7 +52,7 @@ pub use history::{Edit, History};
pub use lens::{compose_warps, ComposedWarp, Warp};
pub use operation::{
compose, compose_with_framing, Affects, ComposedShader, Helper, Operation, Uniform,
RESERVED_UNIFORM_FIELDS,
BASE_CURVE_POINTS, BASE_CURVE_UNIFORM_OFFSET, RESERVED_UNIFORM_FIELDS,
};
pub use preset::{Preset, Scope};
pub use sidecar::{Sidecar, Version};
core/dr-pipeline/src/operation.rs
+194 -2
View File
@@ -145,7 +145,35 @@ pub struct ComposedShader {
///
/// WGSL requires a uniform struct to be non-empty and 16-byte aligned; these
/// are needed by every generated shader in any case.
const BASE_UNIFORM_FIELDS: usize = 16;
///
/// Twelve of the twenty-eight are the camera profile's base curve
/// ([`BASE_CURVE_UNIFORM_FIELDS`]); the rest are the matrix, the as-shot
/// balance and framing's own block.
const BASE_UNIFORM_FIELDS: usize = 16 + BASE_CURVE_UNIFORM_FIELDS;
/// TRACES: FR-DEV-3e
/// Slots the base curve occupies: five `(x, y)` points and an active flag.
///
/// Twelve rather than eleven so the block stays a whole number of `vec4`s,
/// which is what std140 requires of a uniform struct's members. The spare
/// float is left zero rather than repurposed — a uniform slot that means one
/// thing today and two things next year is how a shader comes to read a
/// highlight rolloff out of a crop rectangle.
const BASE_CURVE_UNIFORM_FIELDS: usize = 12;
/// TRACES: FR-DEV-3e
/// Where the base curve's slots begin in the generated uniform block.
///
/// Exported for the same reason [`RESERVED_UNIFORM_FIELDS`] is: `dr-gpu`
/// writes these by index, and an offset computed independently at both ends is
/// an offset that will eventually disagree with itself.
pub const BASE_CURVE_UNIFORM_OFFSET: usize = 16;
/// How many control points a base curve carries.
///
/// The same five the tone curve widget has, deliberately — see the helper
/// selection in [`compose_full`].
pub const BASE_CURVE_POINTS: usize = 5;
/// Where an operation's own uniforms begin in the generated block.
///
@@ -235,10 +263,44 @@ pub fn compose_full(
\x20 // `.w` is not padding: it flags a non-linear source (1.0 for a\n\
\x20 // gamma-encoded JPEG, 0.0 for demosaiced sensor data), which the\n\
\x20 // prologue reads to decide whether to linearise.\n\
\x20 as_shot_wb: vec4<f32>,\n",
\x20 as_shot_wb: vec4<f32>,\n\
\x20 // The camera profile's base curve (FR-DEV-3e): five points on a\n\
\x20 // monotone spline, packed as x0..x3, y0..y3, then (x4, y4, on).\n\
\x20 // `.z` of the last is the flag, not padding — it is 0 for a\n\
\x20 // body with no profile and for an already-rendered source.\n\
\x20 base_curve_x: vec4<f32>,\n\
\x20 base_curve_y: vec4<f32>,\n\
\x20 base_curve_last: vec4<f32>,\n",
);
uniform_values.resize(BASE_UNIFORM_FIELDS, 0.0);
// TRACES: FR-DEV-3e
// The spline the base curve is evaluated on is the *tone curve's* spline,
// reached through the trait rather than reimplemented here.
//
// Two reasons, and the second is the one that matters. The obvious one is
// that a shader carrying two `curve_eval`s would not compile, and the
// composer's helper de-duplication is what makes both stages able to ask
// for it. The real one is that a profile author placing a control point
// and a photographer dragging one must mean the same thing by it — down to
// the Fritsch-Carlson tangent limiting, which is what decides how a
// shoulder actually rolls off. Two implementations that agreed today would
// be two that could disagree later, and the disagreement would show up as
// a body whose profile renders subtly differently from the curve someone
// drew to match it.
//
// Emitted unconditionally, unlike an operation's helpers. The base curve
// is active for every RAW frame — an unprofiled body still gets the
// database's default rendering — so making the shader's shape depend on it
// would split the pipeline cache in two for no benefit. The uniform flag
// above turns it off for the cases that are genuinely already rendered,
// and a branch on a uniform is coherent across the whole dispatch.
for h in crate::ops::ToneCurve::new().helpers() {
if matches!(h.name, "curve_span" | "curve_eval") {
helpers.push(*h);
}
}
// Framing's block follows the base one at a fixed offset, for the same
// reason: the prologue is emitted whether or not any operation is active,
// so these slots cannot be positioned by the op loop below.
@@ -419,6 +481,58 @@ fn main(@builtin(global_invocation_id) gid: vec3<u32>) {{
c = mix(c, neutral, clipped);
}}
{body}
// ==== camera profile: the base curve (FR-DEV-3e) ====
//
// Marked with `====` and not the `----` an operation block carries: this
// is not one, and the difference is what several tests count on to tell
// an edit apart from the reading of a file.
//
// The stage between demosaic and the working space that turns a correct
// exposure into a photograph. Sensor data is scene-referred and nearly
// linear; nothing anybody looks at is. Rendering it straight out is the
// dcraw default, and it is flat, dark through the midtones and clips its
// highlights instead of rolling them off.
//
// **In camera RGB, and after the adjustments**, which is a deliberate pair
// of choices:
//
// - Before the matrix, because that is where a base curve is defined and
// where every other converter applies one. The curve was tuned against
// this body's own primaries; moving it after the conversion would apply
// a Canon rendering to sRGB values and change what it does.
// - After exposure and the tonal operations, because those are corrections
// to *capture* and are only meaningful on linear values. A stop is a
// doubling; run exposure after a curve and it stops being one.
//
// Per channel rather than on luminance. It desaturates the extremes
// slightly, and that is the point — it is what makes a blown sky roll
// toward white rather than toward a saturated corner of the gamut, and it
// is what the camera's own JPEG does.
//
// The branch is on a uniform, so the whole dispatch takes the same path.
// It is off for a JPEG and any other already-rendered source, which must
// not be rendered twice, and for a body the profile database declines to
// offer any curve for at all.
if (u.base_curve_last.z > 0.5) {{
c = vec3<f32>(
curve_eval(
u.base_curve_x.x, u.base_curve_y.x, u.base_curve_x.y, u.base_curve_y.y,
u.base_curve_x.z, u.base_curve_y.z, u.base_curve_x.w, u.base_curve_y.w,
u.base_curve_last.x, u.base_curve_last.y, c.r,
),
curve_eval(
u.base_curve_x.x, u.base_curve_y.x, u.base_curve_x.y, u.base_curve_y.y,
u.base_curve_x.z, u.base_curve_y.z, u.base_curve_x.w, u.base_curve_y.w,
u.base_curve_last.x, u.base_curve_last.y, c.g,
),
curve_eval(
u.base_curve_x.x, u.base_curve_y.x, u.base_curve_x.y, u.base_curve_y.y,
u.base_curve_x.z, u.base_curve_y.z, u.base_curve_x.w, u.base_curve_y.w,
u.base_curve_last.x, u.base_curve_last.y, c.b,
),
);
}}
// Camera space -> linear sRGB. Applied after the adjustments so white
// balance and exposure act on sensor-native values, which is where they
// are physically meaningful.
@@ -960,6 +1074,84 @@ mod tests {
assert!(op < matrix, "the camera matrix must come after operations");
}
#[test]
fn the_base_curve_runs_after_the_operations_and_before_the_camera_matrix() {
// TRACES: FR-DEV-3e
// Both halves matter and for different reasons.
//
// After the operations: exposure and the tonal controls are
// corrections to capture, and they are only meaningful on linear
// values. A stop is a doubling; run exposure after a curve and it is
// not one any more, and every slider in the panel starts lying about
// what it does.
//
// Before the matrix: the curve was tuned against this body's own
// primaries. Applied after the conversion it would be a Canon
// rendering acting on sRGB values, which is a different curve.
let ops = vec![fake(&DESC_A, 2.0, false)];
let source = compose(&ops).source;
let op = source.find("---- op_a ----").expect("op present");
let curve = source
.find("if (u.base_curve_last.z > 0.5)")
.expect("base curve applied");
let matrix = source.find("u.cam_to_srgb_0").expect("matrix applied");
assert!(op < curve, "the base curve must come after the operations");
assert!(curve < matrix, "and before the camera matrix");
}
#[test]
fn the_base_curve_reaches_a_shader_with_no_operations_at_all() {
// TRACES: FR-DEV-3e
// The same property as as-shot white balance, and for the same reason:
// it is part of interpreting the file, not part of the edit. An
// unedited RAW must open looking like a photograph rather than like a
// scan of one.
let shader = compose(&[]);
assert!(shader.source.contains("u.base_curve_x"));
assert!(
shader.source.contains("fn curve_eval("),
"the spline it is evaluated on must be emitted too"
);
}
#[test]
fn the_base_curve_and_the_tone_curve_share_one_spline() {
// TRACES: FR-DEV-3e
// Two `curve_eval`s in one shader would not compile — but the reason
// the helper is *shared* rather than merely renamed is that a profile
// author placing a control point and a photographer dragging one must
// mean the same thing by it, down to the tangent limiting that decides
// how a shoulder rolls off.
let mut curve = crate::ops::ToneCurve::new();
curve.set_param(crate::ops::curve::P2_Y, 0.7);
assert!(curve.is_active(), "the fixture must actually reach the shader");
let source = compose(&[Box::new(curve)]).source;
assert_eq!(
source.matches("fn curve_eval(").count(),
1,
"the spline must be declared exactly once"
);
assert_eq!(source.matches("fn curve_span(").count(), 1);
}
#[test]
fn the_base_curve_owns_the_slots_dr_gpu_writes() {
// TRACES: FR-DEV-3e
// `dr-gpu` fills these by index. The offset is exported rather than
// recomputed there, and this asserts the exported number still points
// at the block the shader declares — the failure otherwise is a
// highlight rolloff read out of a crop rectangle, which renders as
// nonsense rather than as an error.
assert_eq!(
BASE_CURVE_UNIFORM_OFFSET + BASE_CURVE_UNIFORM_FIELDS,
BASE_UNIFORM_FIELDS,
"the base curve must be the last thing in the base block"
);
assert_eq!(BASE_CURVE_POINTS * 2 + 1, BASE_CURVE_UNIFORM_FIELDS - 1);
assert!(compose(&[]).uniforms.len() >= BASE_UNIFORM_FIELDS);
}
/// Compose with neutral framing into a chosen output space.
fn compose_to(ops: &[Box<dyn Operation>], output: ColourSpace) -> ComposedShader {
compose_with_framing(ops, &Framing::new(), output)