A linear DNG in and out: the writer, and a three-sample RawImage
dr-export gains write_linear_dng — LinearRaw, DNG 1.4, u16 samples at the sensor's scale, the body's matrices with their illuminants, the as-shot neutral, the EXIF block an export writes — streamed strip by strip through a closure so the composite is never held (FR-MRG-11). The tiff crate's directory is a map, so PhotometricInterpretation is written over what new_image set, which is the trick the S15.1 spike thought it had to hand-roll around. The test reads the file back through rawler. dr-decode's RawImage carries samples_per_pixel (a linear DNG is 3), the body's profile with its calibrations mapped back to EXIF illuminant codes, and the cleaned make and model. The GPU uploads a three-sample image as it is, normalised by black and white like a photosite, through a full f16 conversion — subnormals kept, because a 14-bit LSB sits at f16's smallest normal and rounding it to zero would crush exactly the shadows the file was written to keep.
This commit is contained in:
@@ -0,0 +1,257 @@
|
||||
//! TRACES: FR-MRG-3
|
||||
//! A linear DNG: the container a merge writes its composite into.
|
||||
//!
|
||||
//! Decided by S15.1 (2026-09-19): rawler reads back a `LinearRaw` DNG the
|
||||
//! application writes, so a composite re-enters the library as
|
||||
//! `Format::Dng` through the decoder every camera DNG uses. What is written
|
||||
//! is a RAW in every sense a warp can preserve — camera-linear `u16`
|
||||
//! samples at the first source's own scale, its matrices, illuminants,
|
||||
//! as-shot neutral and body name — so the panorama is developed afterwards
|
||||
//! as one photograph, from the sensor's numbers.
|
||||
//!
|
||||
//! # Streamed, not buffered
|
||||
//!
|
||||
//! The composite is larger than any single photograph the pipeline renders
|
||||
//! and larger than the tablet's memory (FR-MRG-11), so the writer never
|
||||
//! holds it. Strips are pulled from the caller one at a time through a
|
||||
//! closure, in order, and written as they arrive; the caller renders a band
|
||||
//! of chunks, hands over its rows, and moves on.
|
||||
//!
|
||||
//! # Why the `tiff` crate after all
|
||||
//!
|
||||
//! S15.1's spike hand-rolled its IFD because the crate's encoder fixes
|
||||
//! `PhotometricInterpretation` to RGB when the image is opened. It does — but
|
||||
//! a directory is a map and a later `write_tag` on the same tag replaces the
|
||||
//! earlier, so `LinearRaw` goes in over the top and everything else the
|
||||
//! crate does (strips, offsets, sub-IFDs, the EXIF block `encode.rs` already
|
||||
//! knows how to write) is kept.
|
||||
|
||||
use std::io::{Seek, Write};
|
||||
|
||||
use tiff::encoder::{colortype, DirectoryEncoder, SRational, TiffEncoder, TiffKind, TiffValue};
|
||||
use tiff::tags::Tag;
|
||||
|
||||
use crate::encode::{sub_directories, tag_metadata, Ascii, Rationals};
|
||||
use crate::{ExportError, SourceMetadata};
|
||||
|
||||
/// What the DNG says about the camera that "took" the composite: the first
|
||||
/// source's profile, carried across so the composite develops through it.
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct DngProfile {
|
||||
/// `UniqueCameraModel`, the name the profile database matches on.
|
||||
pub unique_model: String,
|
||||
/// `(CalibrationIlluminant, ColorMatrix)`: the EXIF light-source code and
|
||||
/// the XYZ → camera matrix measured under it. One or two.
|
||||
pub calibrations: Vec<(u16, [[f32; 3]; 3])>,
|
||||
/// `AsShotNeutral`, camera RGB of the scene's white.
|
||||
pub as_shot_neutral: [f32; 3],
|
||||
/// `WhiteLevel`: the sample value that is clipping. The first source's
|
||||
/// white minus its black, since the samples are black-subtracted.
|
||||
pub white_level: u32,
|
||||
}
|
||||
|
||||
/// Write a linear DNG, pulling `rows_per_strip`-row strips from `strips`.
|
||||
///
|
||||
/// Each call to `strips` receives the strip index and a buffer to fill with
|
||||
/// `width × rows × 3` interleaved RGB `u16` samples (the last strip may be
|
||||
/// shorter). `source` supplies the `Make`, `Model`, dates and EXIF block
|
||||
/// exactly as an export does (FR-EXP-8 sanitising already applied by the
|
||||
/// caller).
|
||||
///
|
||||
/// `PhotometricInterpretation = LinearRaw`, `DNGVersion 1.4`, uncompressed,
|
||||
/// `Orientation = 1` — the composite is written upright (panorama.md §8).
|
||||
pub fn write_linear_dng<W, F>(
|
||||
out: W,
|
||||
width: u32,
|
||||
height: u32,
|
||||
rows_per_strip: u32,
|
||||
profile: &DngProfile,
|
||||
source: Option<&SourceMetadata>,
|
||||
mut strips: F,
|
||||
) -> Result<(), ExportError>
|
||||
where
|
||||
W: Write + Seek,
|
||||
F: FnMut(usize, &mut Vec<u16>) -> Result<(), ExportError>,
|
||||
{
|
||||
let enc = |e: tiff::TiffError| ExportError::Encode(e.to_string());
|
||||
let mut encoder = TiffEncoder::new(out).map_err(enc)?;
|
||||
let sub = sub_directories(&mut encoder, source, width, height)?;
|
||||
let mut image = encoder
|
||||
.new_image::<colortype::RGB16>(width, height)
|
||||
.map_err(enc)?;
|
||||
image.rows_per_strip(rows_per_strip.max(1)).map_err(enc)?;
|
||||
tag_metadata(image.encoder(), source, &sub)?;
|
||||
tag_dng(image.encoder(), profile).map_err(enc)?;
|
||||
|
||||
let rows = rows_per_strip.max(1);
|
||||
let strip_count = height.div_ceil(rows) as usize;
|
||||
let mut buf: Vec<u16> = Vec::with_capacity((width * rows * 3) as usize);
|
||||
for k in 0..strip_count {
|
||||
buf.clear();
|
||||
strips(k, &mut buf)?;
|
||||
let expected_rows = rows.min(height - k as u32 * rows);
|
||||
let expected = (width * expected_rows * 3) as usize;
|
||||
if buf.len() != expected {
|
||||
return Err(ExportError::Encode(format!(
|
||||
"strip {k} has {} samples, expected {expected}",
|
||||
buf.len()
|
||||
)));
|
||||
}
|
||||
image.write_strip(&buf).map_err(enc)?;
|
||||
}
|
||||
image.finish().map_err(enc)
|
||||
}
|
||||
|
||||
/// The tags that make a TIFF a DNG, and a linear one.
|
||||
fn tag_dng<W, K>(dir: &mut DirectoryEncoder<'_, W, K>, profile: &DngProfile) -> tiff::TiffResult<()>
|
||||
where
|
||||
W: Write + Seek,
|
||||
K: TiffKind,
|
||||
{
|
||||
// Over the top of what `new_image` wrote: this is the whole trick.
|
||||
dir.write_tag(Tag::PhotometricInterpretation, LINEAR_RAW)?;
|
||||
dir.write_tag(Tag::Orientation, 1u16)?;
|
||||
dir.write_tag(Tag::Unknown(tag::DNG_VERSION), &[1u8, 4, 0, 0][..])?;
|
||||
dir.write_tag(Tag::Unknown(tag::DNG_BACKWARD_VERSION), &[1u8, 4, 0, 0][..])?;
|
||||
dir.write_tag(Tag::Unknown(tag::UNIQUE_CAMERA_MODEL), Ascii(&profile.unique_model))?;
|
||||
dir.write_tag(Tag::Unknown(tag::WHITE_LEVEL), &[profile.white_level; 3][..])?;
|
||||
dir.write_tag(Tag::Unknown(tag::BLACK_LEVEL), &[0u32; 3][..])?;
|
||||
|
||||
for (slot, (illuminant, matrix)) in profile.calibrations.iter().take(2).enumerate() {
|
||||
let (ill_tag, mat_tag) = if slot == 0 {
|
||||
(tag::CALIBRATION_ILLUMINANT_1, tag::COLOR_MATRIX_1)
|
||||
} else {
|
||||
(tag::CALIBRATION_ILLUMINANT_2, tag::COLOR_MATRIX_2)
|
||||
};
|
||||
dir.write_tag(Tag::Unknown(ill_tag), *illuminant)?;
|
||||
let flat: Vec<SRational> = matrix
|
||||
.iter()
|
||||
.flatten()
|
||||
.map(|&v| SRational {
|
||||
n: (v * 10_000.0).round() as i32,
|
||||
d: 10_000,
|
||||
})
|
||||
.collect();
|
||||
dir.write_tag(Tag::Unknown(mat_tag), SRationals(&flat))?;
|
||||
}
|
||||
|
||||
let neutral: Vec<(u32, u32)> = profile
|
||||
.as_shot_neutral
|
||||
.iter()
|
||||
.map(|&v| ((v.max(0.0) * 1_000_000.0).round() as u32, 1_000_000))
|
||||
.collect();
|
||||
dir.write_tag(Tag::Unknown(tag::AS_SHOT_NEUTRAL), Rationals(&neutral))?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// `PhotometricInterpretation` for demosaiced, un-rendered sensor data.
|
||||
const LINEAR_RAW: u16 = 34892;
|
||||
|
||||
/// DNG tag numbers the `tiff` crate has no names for.
|
||||
mod tag {
|
||||
pub const DNG_VERSION: u16 = 50706;
|
||||
pub const DNG_BACKWARD_VERSION: u16 = 50707;
|
||||
pub const UNIQUE_CAMERA_MODEL: u16 = 50708;
|
||||
pub const BLACK_LEVEL: u16 = 50714;
|
||||
pub const WHITE_LEVEL: u16 = 50717;
|
||||
pub const COLOR_MATRIX_1: u16 = 50721;
|
||||
pub const COLOR_MATRIX_2: u16 = 50722;
|
||||
pub const AS_SHOT_NEUTRAL: u16 = 50728;
|
||||
pub const CALIBRATION_ILLUMINANT_1: u16 = 50778;
|
||||
pub const CALIBRATION_ILLUMINANT_2: u16 = 50779;
|
||||
}
|
||||
|
||||
/// A run of `SRATIONAL`s, as `encode::Rationals` is for `RATIONAL`.
|
||||
struct SRationals<'a>(&'a [SRational]);
|
||||
|
||||
impl TiffValue for SRationals<'_> {
|
||||
const BYTE_LEN: u8 = 8;
|
||||
const FIELD_TYPE: tiff::tags::Type = tiff::tags::Type::SRATIONAL;
|
||||
|
||||
fn count(&self) -> usize {
|
||||
self.0.len()
|
||||
}
|
||||
|
||||
fn data(&self) -> std::borrow::Cow<'_, [u8]> {
|
||||
let mut out = Vec::with_capacity(self.0.len() * 8);
|
||||
for r in self.0 {
|
||||
out.extend_from_slice(&r.n.to_ne_bytes());
|
||||
out.extend_from_slice(&r.d.to_ne_bytes());
|
||||
}
|
||||
std::borrow::Cow::Owned(out)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn profile() -> DngProfile {
|
||||
DngProfile {
|
||||
unique_model: "Canon EOS 6D".into(),
|
||||
calibrations: vec![
|
||||
(17, [[0.8, -0.2, 0.1], [-0.3, 1.1, 0.2], [0.0, -0.1, 0.9]]),
|
||||
(21, [[0.7, -0.1, 0.0], [-0.2, 1.0, 0.1], [0.0, -0.2, 0.8]]),
|
||||
],
|
||||
as_shot_neutral: [0.5, 1.0, 0.6],
|
||||
white_level: 13_023,
|
||||
}
|
||||
}
|
||||
|
||||
fn write(width: u32, height: u32, rows: u32) -> Vec<u8> {
|
||||
let mut bytes = std::io::Cursor::new(Vec::new());
|
||||
let source = SourceMetadata {
|
||||
make: Some("Canon".into()),
|
||||
model: Some("Canon EOS 6D".into()),
|
||||
..Default::default()
|
||||
};
|
||||
write_linear_dng(&mut bytes, width, height, rows, &profile(), Some(&source), |k, buf| {
|
||||
let first = k as u32 * rows;
|
||||
let n = rows.min(height - first);
|
||||
for y in first..first + n {
|
||||
for x in 0..width {
|
||||
buf.extend([(x + y * width) as u16, 1000, 2000]);
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
})
|
||||
.expect("written");
|
||||
bytes.into_inner()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rawler_reads_it_back_as_linear_raw() {
|
||||
let bytes = write(20, 13, 4);
|
||||
let source = rawler::rawsource::RawSource::new_from_slice(&bytes);
|
||||
let decoder = rawler::get_decoder(&source).expect("a DNG");
|
||||
let image = decoder
|
||||
.raw_image(&source, &Default::default(), false)
|
||||
.expect("decodes");
|
||||
assert_eq!((image.width, image.height, image.cpp), (20, 13, 3));
|
||||
assert_eq!(image.whitelevel.0[0], 13_023);
|
||||
// Pixel (3, 2) is (3 + 2·20, 1000, 2000) — samples in order, strips
|
||||
// joined without a seam.
|
||||
let rawler::RawImageData::Integer(data) = &image.data else {
|
||||
panic!("integer samples")
|
||||
};
|
||||
let i = (2 * 20 + 3) * 3;
|
||||
assert_eq!(&data[i..i + 3], &[43, 1000, 2000]);
|
||||
// Last row, from the short final strip.
|
||||
let i = (12 * 20 + 19) * 3;
|
||||
assert_eq!(data[i], (19 + 12 * 20) as u16);
|
||||
// The profile came through as the camera's.
|
||||
assert!(image.camera.color_matrix.len() >= 1);
|
||||
assert_eq!(image.model, "Canon EOS 6D");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_strip_of_the_wrong_length_is_refused() {
|
||||
let mut bytes = std::io::Cursor::new(Vec::new());
|
||||
let err = write_linear_dng(&mut bytes, 8, 8, 8, &profile(), None, |_, buf| {
|
||||
buf.extend([0u16; 10]);
|
||||
Ok(())
|
||||
})
|
||||
.unwrap_err();
|
||||
assert!(matches!(err, ExportError::Encode(_)));
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user