Render a film stock on the GPU, and let it take over the rendering
The stock model landed in dr-film with no way to see it. This is the pipeline node, the two texture bindings it reads, and the end-to-end test that proves the shader agrees with the model. The design point is that a film simulation is not an adjustment. Every other node changes a picture; this one makes it. A stock's characteristic curve does the camera profile's base curve's job -- from measurements rather than from a curve somebody drew -- so running both renders the scene twice: the camera's rendering, and then a film's rendering of that. It looks like neither, and it reads as a colour-management bug with no colour-management bug to find. So `Operation::renders` is new. A node declaring it takes camera RGB and hands back linear sRGB, and the composer emits neither the base curve nor the conversion out of camera space. Both halves move together, and the composer keeps them as one string precisely so that getting half of it right is impossible. The tables are not parameters, for the reason vignetting's coefficients are not: they are measurements. dr-pipeline declares the layout as a plain struct and keeps its no-dependency property; the two crates share no types on purpose. `EditGraph::set_film_tables` offers them to every node rather than to the one that wants them, because knowing which concrete type is which is what the graph is organised not to know. Bindings 4 and 5 follow the masks precedent: declared unconditionally so one bind group layout serves every generated shader, bound to 1x1 placeholders when no stock is loaded. Both are interpolated by hand with textureLoad -- this pipeline binds no sampler, and adding one for two lookups would cost a binding in every shader. Uploads are keyed on content so an unchanged stock does not push half a megabyte across the bus per frame. The end-to-end test earned its place immediately: it found the density lookup being filled z-fastest while a 3D texture upload wants x-fastest, so the red and blue axes were transposed. Green matched exactly, which is what that bug looks like -- a plausible photograph of the wrong colour, and one that every unit test on either side of the seam passes. dr-film now pins the layout in a test that needs no device, and states it where the field is declared. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5
@@ -0,0 +1,410 @@
|
||||
//! TRACES: FR-DEV-3f
|
||||
//! Film simulation — the stock renders the picture.
|
||||
//!
|
||||
//! # Why this one replaces the base curve
|
||||
//!
|
||||
//! [`crate::ops`]' other nodes adjust a picture. This one *makes* it. The base
|
||||
//! curve exists because sensor data is scene-referred and nothing anybody looks
|
||||
//! at is (FR-DEV-3e); a film stock's characteristic curve does the same job,
|
||||
//! from measurements, with a toe and a shoulder that were coated onto acetate
|
||||
//! rather than drawn. Running both renders the image twice — the camera's
|
||||
//! JPEG-ish rendering, and then a film's rendering of that — which is not what
|
||||
//! either is for and looks like neither.
|
||||
//!
|
||||
//! So this node declares [`Operation::renders`], and the composer answers by
|
||||
//! emitting neither the base curve nor the camera matrix. Both jobs move here:
|
||||
//! the fragment takes camera RGB, converts it to linear sRGB itself with the
|
||||
//! matrix already in the uniform block, and returns linear sRGB. That is a
|
||||
//! contract worth stating plainly, because a node that got half of it wrong
|
||||
//! would produce a picture that renders perfectly and is wrong everywhere.
|
||||
//!
|
||||
//! # Why the tables are not parameters
|
||||
//!
|
||||
//! For the same reason [`crate::ops::vignetting`]'s coefficients are not: they
|
||||
//! are measurements of a physical thing, not something a slider moves. The
|
||||
//! sliders here are exposure and print exposure, which are what a photographer
|
||||
//! and a printer actually control. `dr-film` turns a stock plus those two
|
||||
//! numbers into [`FilmTables`]; this node knows only the layout.
|
||||
//!
|
||||
//! Declared as a plain struct here rather than imported, so that dr-pipeline
|
||||
//! keeps its no-dependency property (ARCH §6.5a) exactly as `vignetting` does
|
||||
//! with `Pa`.
|
||||
|
||||
use crate::descriptor::{Attribute, LocalizedKey, OpDescriptor, OpId, ParamDescriptor, ParamId};
|
||||
use crate::operation::{Operation, Uniform};
|
||||
|
||||
pub const ID: OpId = OpId("film_sim");
|
||||
pub const EXPOSURE: ParamId = ParamId("exposure");
|
||||
pub const PRINT_EXPOSURE: ParamId = ParamId("print_exposure");
|
||||
|
||||
/// How many samples a characteristic curve carries.
|
||||
///
|
||||
/// Must agree with `dr_film::profile::CURVE_SAMPLES`. Restated rather than
|
||||
/// imported because importing it is exactly the dependency this crate does not
|
||||
/// take; [`FilmTables::is_well_formed`] is what stops the two drifting.
|
||||
pub const CURVE_SAMPLES: usize = 256;
|
||||
|
||||
/// The uniform field names the fragment reads the exposure matrix from.
|
||||
///
|
||||
/// A table rather than a formatted string, because a `Uniform`'s name is
|
||||
/// `&'static str`: building one per composition would mean leaking a string
|
||||
/// every time a slider moved.
|
||||
static MATRIX_FIELDS: [[&str; 3]; 3] = [
|
||||
["m00", "m01", "m02"],
|
||||
["m10", "m11", "m12"],
|
||||
["m20", "m21", "m22"],
|
||||
];
|
||||
|
||||
static DESCRIPTOR: OpDescriptor = OpDescriptor {
|
||||
// Tone and colour both, and not `Effect`: a stock is not something applied
|
||||
// on top of a photograph, it is what the photograph was made on.
|
||||
attributes: &[Attribute::Tone, Attribute::Colour],
|
||||
id: ID,
|
||||
label: LocalizedKey("op.film_sim"),
|
||||
params: &[
|
||||
ParamDescriptor::stops("exposure", "param.film_sim.exposure", -3.0, 3.0),
|
||||
ParamDescriptor::stops("print_exposure", "param.film_sim.print_exposure", -3.0, 3.0),
|
||||
],
|
||||
};
|
||||
|
||||
/// A stock reduced to what a shader runs, as `dr-film` bakes it.
|
||||
///
|
||||
/// Layout is the contract between the two crates, so it is written down here
|
||||
/// and checked rather than assumed:
|
||||
///
|
||||
/// - `exposure_matrix[l][c]` — layer `l`'s response to linear sRGB channel `c`.
|
||||
/// - `curves` — `CURVE_SAMPLES` density triples, uniform over
|
||||
/// `[curve_log_min, curve_log_max]`.
|
||||
/// - `lut` — `lut_size³` linear sRGB triples in x-major order, uniform over
|
||||
/// `[0, density_max]` on each axis.
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct FilmTables {
|
||||
pub exposure_matrix: [[f32; 3]; 3],
|
||||
pub curves: Vec<[f32; 3]>,
|
||||
pub curve_log_min: f32,
|
||||
pub curve_log_max: f32,
|
||||
pub lut: Vec<[f32; 3]>,
|
||||
pub density_max: f32,
|
||||
pub lut_size: usize,
|
||||
}
|
||||
|
||||
impl FilmTables {
|
||||
/// Whether these tables are the shape the shader will index them at.
|
||||
///
|
||||
/// Checked on the way in, because the failure otherwise is a shader
|
||||
/// sampling past the end of a texture: undefined, silent, and different on
|
||||
/// every driver.
|
||||
pub fn is_well_formed(&self) -> bool {
|
||||
self.curves.len() == CURVE_SAMPLES
|
||||
&& self.lut_size >= 2
|
||||
&& self.lut.len() == self.lut_size.pow(3)
|
||||
&& self.density_max > 0.0
|
||||
&& self.curve_log_max > self.curve_log_min
|
||||
}
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-3f
|
||||
#[derive(Debug, Default, Clone)]
|
||||
pub struct FilmSim {
|
||||
exposure: f32,
|
||||
print_exposure: f32,
|
||||
tables: Option<FilmTables>,
|
||||
}
|
||||
|
||||
impl FilmSim {
|
||||
pub fn new() -> Self {
|
||||
Self::default()
|
||||
}
|
||||
|
||||
/// Load a baked stock, or clear it.
|
||||
///
|
||||
/// Malformed tables are refused rather than stored: an operation that is
|
||||
/// active but cannot be indexed is worse than one that is off, because the
|
||||
/// first renders garbage and the second renders the photograph.
|
||||
pub fn set_tables(&mut self, tables: Option<FilmTables>) {
|
||||
match tables {
|
||||
Some(t) if !t.is_well_formed() => {
|
||||
log::error!(
|
||||
"film_sim: refusing malformed tables ({} curve samples, {} lut entries at size {})",
|
||||
t.curves.len(),
|
||||
t.lut.len(),
|
||||
t.lut_size
|
||||
);
|
||||
self.tables = None;
|
||||
}
|
||||
other => self.tables = other,
|
||||
}
|
||||
}
|
||||
|
||||
/// The loaded stock's tables, for whoever has to upload them.
|
||||
pub fn tables(&self) -> Option<&FilmTables> {
|
||||
self.tables.as_ref()
|
||||
}
|
||||
}
|
||||
|
||||
impl Operation for FilmSim {
|
||||
fn descriptor(&self) -> &'static OpDescriptor {
|
||||
&DESCRIPTOR
|
||||
}
|
||||
|
||||
fn set_param(&mut self, id: ParamId, value: f32) {
|
||||
match id {
|
||||
EXPOSURE => self.exposure = value,
|
||||
PRINT_EXPOSURE => self.print_exposure = value,
|
||||
_ => log::warn!("film_sim: unknown parameter {id}"),
|
||||
}
|
||||
}
|
||||
|
||||
fn param(&self, id: ParamId) -> f32 {
|
||||
match id {
|
||||
EXPOSURE => self.exposure,
|
||||
PRINT_EXPOSURE => self.print_exposure,
|
||||
_ => 0.0,
|
||||
}
|
||||
}
|
||||
|
||||
/// Active exactly when a stock is loaded.
|
||||
///
|
||||
/// Not "when a slider has moved", which is the rule everywhere else and
|
||||
/// would be wrong here: a stock at zero exposure compensation is the whole
|
||||
/// point of choosing it, and a node that went quiet at its defaults would
|
||||
/// mean picking a film did nothing until you also nudged something.
|
||||
fn is_active(&self) -> bool {
|
||||
self.tables.is_some()
|
||||
}
|
||||
|
||||
/// This node renders; the camera's own rendering must not also run.
|
||||
fn renders(&self) -> bool {
|
||||
true
|
||||
}
|
||||
|
||||
fn set_film_tables(&mut self, tables: Option<&FilmTables>) {
|
||||
self.set_tables(tables.cloned());
|
||||
}
|
||||
|
||||
fn uniforms(&self) -> Vec<Uniform> {
|
||||
let Some(t) = &self.tables else {
|
||||
return Vec::new();
|
||||
};
|
||||
let m = t.exposure_matrix;
|
||||
// Exposure rides in the matrix on the CPU when the stock is baked, so
|
||||
// what is left here is the *shader's* copy of the same nine numbers.
|
||||
// Spelled out one at a time because a uniform is a named `f32` in this
|
||||
// pipeline and a matrix would be a second kind of thing for one caller.
|
||||
let mut out = Vec::with_capacity(MATRIX_FIELDS.len() + 5);
|
||||
for (l, row) in m.iter().enumerate() {
|
||||
for (c, v) in row.iter().enumerate() {
|
||||
out.push(Uniform { name: MATRIX_FIELDS[l][c], value: *v });
|
||||
}
|
||||
}
|
||||
out.push(Uniform { name: "log_min", value: t.curve_log_min });
|
||||
out.push(Uniform { name: "log_max", value: t.curve_log_max });
|
||||
out.push(Uniform { name: "density_max", value: t.density_max });
|
||||
out.push(Uniform { name: "lut_size", value: t.lut_size as f32 });
|
||||
out.push(Uniform { name: "print_exposure", value: self.print_exposure });
|
||||
out
|
||||
}
|
||||
|
||||
fn wgsl_body(&self) -> String {
|
||||
// Filtered by hand rather than through a sampler, which is what the
|
||||
// framing prologue already does for the source: this pipeline has no
|
||||
// sampler binding, and adding one to interpolate two lookups would
|
||||
// cost a binding in every shader whether or not a film is loaded.
|
||||
"\
|
||||
// Camera RGB to linear sRGB. The film's exposure matrix is defined against
|
||||
// sRGB primaries, and this node has taken over the conversion the composer
|
||||
// would otherwise have emitted at the end — see `Operation::renders`.
|
||||
let scene = vec3<f32>(
|
||||
dot(u.cam_to_srgb_0.rgb, c),
|
||||
dot(u.cam_to_srgb_1.rgb, c),
|
||||
dot(u.cam_to_srgb_2.rgb, c),
|
||||
);
|
||||
|
||||
// What each emulsion layer was exposed to. A matrix, exactly: the scene
|
||||
// spectrum reconstructed from an sRGB triple is linear in that triple, so the
|
||||
// integral over wavelength collapsed into these nine numbers when the stock
|
||||
// was baked.
|
||||
let exposure = vec3<f32>(
|
||||
dot(vec3<f32>(m00, m01, m02), scene),
|
||||
dot(vec3<f32>(m10, m11, m12), scene),
|
||||
dot(vec3<f32>(m20, m21, m22), scene),
|
||||
);
|
||||
// 1e-10 rather than a clamp to zero: a black pixel has to land somewhere on
|
||||
// the curve, and the toe is where it belongs.
|
||||
let log_exposure = log10(max(exposure, vec3<f32>(0.0)) + 1e-10);
|
||||
|
||||
// The characteristic curve: what density each layer develops to. Clamped, not
|
||||
// extrapolated — past the shoulder a real emulsion stops responding, and
|
||||
// extrapolating would turn a blown highlight into a colour cast that grows the
|
||||
// more it is overexposed.
|
||||
let density = film_curve(clamp((log_exposure - log_min) / (log_max - log_min),
|
||||
vec3<f32>(0.0), vec3<f32>(1.0)));
|
||||
|
||||
// Dye absorption, the print through the negative, the paper, the viewing
|
||||
// illuminant and the chromatic adaptation — all of which take exactly three
|
||||
// numbers in, which is why they fit in one lookup.
|
||||
c = film_lut(clamp(density / density_max, vec3<f32>(0.0), vec3<f32>(1.0)), lut_size);"
|
||||
.into()
|
||||
}
|
||||
|
||||
fn helpers(&self) -> &'static [crate::operation::Helper] {
|
||||
&HELPERS
|
||||
}
|
||||
}
|
||||
|
||||
static HELPERS: [crate::operation::Helper; 3] = [
|
||||
crate::operation::Helper {
|
||||
name: "log10",
|
||||
source: "\
|
||||
// WGSL has no log10, and `log2(x) * log10(2)` is the cheap identity for it.
|
||||
fn log10(v: vec3<f32>) -> vec3<f32> {
|
||||
return log2(v) * 0.30103;
|
||||
}",
|
||||
},
|
||||
crate::operation::Helper {
|
||||
name: "film_curve",
|
||||
source: "\
|
||||
// Three characteristic curves, sampled from a 256-wide texture and
|
||||
// interpolated by hand. `t` is already normalised to the curve's domain.
|
||||
fn film_curve(t: vec3<f32>) -> vec3<f32> {
|
||||
let samples = u32(textureDimensions(film_curves).x);
|
||||
let last = f32(samples - 1u);
|
||||
var out = vec3<f32>(0.0);
|
||||
for (var ch = 0u; ch < 3u; ch = ch + 1u) {
|
||||
let x = t[ch] * last;
|
||||
let i = min(u32(floor(x)), samples - 2u);
|
||||
let f = x - f32(i);
|
||||
let a = textureLoad(film_curves, vec2<i32>(i32(i), 0), 0);
|
||||
let b = textureLoad(film_curves, vec2<i32>(i32(i) + 1, 0), 0);
|
||||
out[ch] = mix(a[ch], b[ch], f);
|
||||
}
|
||||
return out;
|
||||
}",
|
||||
},
|
||||
crate::operation::Helper {
|
||||
name: "film_lut",
|
||||
source: "\
|
||||
// Trilinear interpolation of the density lookup, by hand for the same reason
|
||||
// the curve above is: there is no sampler bound, and the eight loads are
|
||||
// cache-neighbours.
|
||||
fn film_lut(t: vec3<f32>, size: f32) -> vec3<f32> {
|
||||
let n = i32(size);
|
||||
let x = t * (size - 1.0);
|
||||
let base = min(vec3<i32>(floor(x)), vec3<i32>(n - 2));
|
||||
let f = x - vec3<f32>(base);
|
||||
|
||||
var out = vec3<f32>(0.0);
|
||||
for (var dx = 0; dx < 2; dx = dx + 1) {
|
||||
let wx = select(1.0 - f.x, f.x, dx == 1);
|
||||
for (var dy = 0; dy < 2; dy = dy + 1) {
|
||||
let wy = select(1.0 - f.y, f.y, dy == 1);
|
||||
for (var dz = 0; dz < 2; dz = dz + 1) {
|
||||
let wz = select(1.0 - f.z, f.z, dz == 1);
|
||||
let p = base + vec3<i32>(dx, dy, dz);
|
||||
out = out + wx * wy * wz
|
||||
* textureLoad(film_lut_texture, p, 0).rgb;
|
||||
}
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}",
|
||||
},
|
||||
];
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn tables() -> FilmTables {
|
||||
FilmTables {
|
||||
exposure_matrix: [[5.0, 0.5, 0.2], [0.1, 5.0, 0.3], [0.2, 0.5, 4.0]],
|
||||
curves: vec![[0.0, 0.0, 0.0]; CURVE_SAMPLES],
|
||||
curve_log_min: -3.0,
|
||||
curve_log_max: 4.0,
|
||||
lut: vec![[0.5, 0.5, 0.5]; 32 * 32 * 32],
|
||||
density_max: 3.0,
|
||||
lut_size: 32,
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn it_starts_inactive() {
|
||||
assert!(!FilmSim::new().is_active());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn loading_a_stock_is_what_turns_it_on() {
|
||||
// Not a moved slider, which is the rule for every other node. Choosing
|
||||
// a film has to do something on its own, or picking one would appear
|
||||
// to be broken until you also nudged the exposure.
|
||||
let mut op = FilmSim::new();
|
||||
op.set_tables(Some(tables()));
|
||||
assert!(op.is_active());
|
||||
op.set_tables(None);
|
||||
assert!(!op.is_active());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn malformed_tables_are_refused_rather_than_stored() {
|
||||
// The alternative is a shader indexing past the end of a texture,
|
||||
// which is undefined, silent, and different on every driver.
|
||||
let mut op = FilmSim::new();
|
||||
let mut bad = tables();
|
||||
bad.lut.truncate(10);
|
||||
op.set_tables(Some(bad));
|
||||
assert!(!op.is_active(), "malformed tables were accepted");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_short_curve_is_refused_too() {
|
||||
let mut op = FilmSim::new();
|
||||
let mut bad = tables();
|
||||
bad.curves.truncate(CURVE_SAMPLES - 1);
|
||||
op.set_tables(Some(bad));
|
||||
assert!(!op.is_active());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn it_declares_itself_a_rendering_transform() {
|
||||
// The whole reason the composer skips the base curve and the camera
|
||||
// matrix. If this ever returned false the picture would be rendered
|
||||
// twice and converted twice, which looks like a colour management bug
|
||||
// a long way from here.
|
||||
assert!(FilmSim::new().renders());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_matrix_reaches_the_shader_in_the_order_the_fragment_reads_it() {
|
||||
// `m01` must be layer 0's response to sRGB green. A transposed matrix
|
||||
// compiles, runs, and swaps the picture's colours.
|
||||
let mut op = FilmSim::new();
|
||||
op.set_tables(Some(tables()));
|
||||
let uniforms = op.uniforms();
|
||||
let named = |n: &str| uniforms.iter().find(|u| u.name == n).unwrap().value;
|
||||
assert_eq!(named("m01"), 0.5);
|
||||
assert_eq!(named("m10"), 0.1);
|
||||
assert_eq!(named("m22"), 4.0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_inactive_node_publishes_no_uniforms() {
|
||||
assert!(FilmSim::new().uniforms().is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_fragment_converts_out_of_camera_space_itself() {
|
||||
// It has to: it has taken over the conversion the composer would
|
||||
// otherwise emit at the end.
|
||||
let mut op = FilmSim::new();
|
||||
op.set_tables(Some(tables()));
|
||||
let wgsl = op.wgsl_body();
|
||||
assert!(wgsl.contains("cam_to_srgb_0"), "{wgsl}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn every_helper_defines_the_function_it_names() {
|
||||
for h in HELPERS {
|
||||
assert!(h.source.contains(&format!("fn {}(", h.name)), "{}", h.name);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -62,6 +62,7 @@ pub mod capture_sharpen;
|
||||
pub mod colour_mixer;
|
||||
pub mod curve;
|
||||
pub mod distortion;
|
||||
pub mod film_sim;
|
||||
pub mod local_contrast;
|
||||
pub mod noise_reduction;
|
||||
pub mod vignetting;
|
||||
@@ -71,6 +72,7 @@ pub use capture_sharpen::CaptureSharpen;
|
||||
pub use colour_mixer::ColourMixer;
|
||||
pub use curve::ToneCurve;
|
||||
pub use distortion::Distortion;
|
||||
pub use film_sim::{FilmSim, FilmTables};
|
||||
// Clarity and texture are one implementation at two scales; see the module's
|
||||
// documentation for why that is two nodes and not one.
|
||||
pub use local_contrast::{Clarity, Texture};
|
||||
|
||||
Reference in New Issue
Block a user