dtourolle/DarkRoom
1
0
Fork 0
Code Issues 39 Pull Requests Actions Packages Projects Releases 20 Wiki Activity

Spec DCP camera profiles (D20)

Browse Source 
The library's Canon 6D DNGs were written by Lightroom and embed Adobe
Standard with its HueSatMap and LookTable; DarkRoom renders them through
the matrix alone, which is most of why the same file looks flatter here
than in Lightroom.

camera-profiles.md designs the deferred half of FR-DEV-3e: the tables
applied by a camera_profile scene operation after exposure, the profile
taken from the DNG or from a matched .dcp, tables carried with the
decoded image like the matrix, a look-strength control, and copying an
embedded profile out where its policy allows. FR-DEV-3e gains item 4
and D20 records the placement and what was rejected.
This commit is contained in:
dtourolle
2026-10-02 22:37:58 -04:00
parent 0b06e31bf3
commit 05ac2416c6
2 changed files with 278 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/dev/camera-profiles.md
+238
View File
@@ -0,0 +1,238 @@
# Camera profiles — DCP tables on top of the matrix
Design for the deferred half of **FR-DEV-3e** ([requirements.md](requirements.md)): the
`HueSatMap` and `LookTable` of a DNG camera profile, read from the DNG that carries one or from a
`.dcp` file, and applied after the matrix. Draft of 2026-10-02, recorded as **D20**.
---
## 1. What we are matching
Lightroom renders a raw through a *profile* before any slider moves. A profile is the matrix
DarkRoom already applies, plus two lookup tables indexed by hue, saturation and value:
- **`ProfileHueSatMap`** — a calibration. It corrects what a 3×3 cannot: a sensor whose reds and
oranges sit in the wrong place relative to its blues, which no linear map fixes. Two copies, one
per calibration illuminant, interpolated like the matrices.
- **`ProfileLookTable`** — a rendering intent. The "Adobe Standard" look: richer blues and greens,
warmer yellows, skin held where it is. The difference between Adobe Standard, Adobe Color and
Adobe Vivid is mostly this table.
Without them a raw renders through the matrix alone, which is accurate on a ColorChecker and reads
as flat next to Lightroom. The first complaint FR-DEV-3e's rationale names is exactly this, and
the per-body base curves D19 retired were the cheaper stand-in for it.
**What the library holds** (catalog of 2026-10-02): 17,286 of its raws are Canon EOS 6D. The
9,348 DNGs were written by Lightroom 6.14 and every one sampled embeds *Adobe Standard* with both
tables — `HueSatMapDims 90 30 1`, `LookTableDims 36 8 16`, `ProfileEmbedPolicy 0` ("allow
copying"), no `ProfileToneCurve`. The 7,938 CR2s from the same body carry no profile. So the
tables Lightroom used are already on disk for half the library, and their licence lets them be
applied to the other half.
## 2. The model, as the DNG specification states it
Each table is a grid of `(hueShift°, satScale, valScale)` triples over HSV, stored with
saturation varying fastest, then hue, then value:
```
index = v · (hueDivs · satDivs) + h · satDivs + s
```
Lookup follows the DNG SDK's `RefBaselineHueSatMap`:
- **HSV** is the SDK's: `v = max(r,g,b)`, `s = (v − min)/v`, `h ∈ [0, 6)` from which channel
leads. Grey has `s = 0` and is untouched by construction.
- **Hue wraps**: `hueDivs` samples over 360°, the last interpolating to the first.
- **Saturation** samples `0..=1` at `satDivs` points; linear between.
- **Value** samples `0..=1` at `valDivs` points when `valDivs > 1`; a table with `valDivs = 1` is
"2.5-D" and ignores value. `ProfileHueSatMapEncoding` / `ProfileLookTableEncoding` = 1 means the
value axis is indexed by the sRGB-encoded value; 0 (the default, and the 6D's) means linear.
- **Apply**: `h += hueShift · 6/360`, `s = min(s · satScale, 1)`, `v ·= valScale`; back to RGB.
- **Space**: linear ProPhoto (ROMM) primaries, D50 white — the space the forward matrix lands in.
- **Two illuminants**: `HueSatMapData1/2` are interpolated entry by entry, with the same mired
weight the matrices use. `LookTable` is single.
Two departures, both forced by D19's unbounded scene-linear values (the SDK runs these on `[0, 1]`):
1. **Value is not clamped.** The SDK writes `min(v · valScale, 1)`; here `v · valScale`, unbounded.
For lookup only, the value axis reads `min(v, 1)`, so a highlight above 1.0 uses the table's
brightest row. Where the encoding is sRGB the scale is defined on the encoded value; it is
applied as the ratio `decode(enc(v′)·valScale)/v′` at `v′ = min(v, 1)`, so a value above 1.0
gets the brightest row's ratio rather than a clip.
2. **A colour outside ProPhoto passes through.** A negative component has no SDK HSV. Such a
colour is outside every surface colour a camera records under normal light. It is left
unmodified rather than floored, because flooring it clips a value D19 says nothing may clip.
## 3. Where it sits
```
… camera matrix ─► vignetting(5) ─► exposure(20) ─► camera_profile(25) ─► contrast(30) ─► … ─► view transform
│
working → ProPhoto ─► HueSatMap ─► LookTable ─► ProPhoto → working
```
**A scene operation at order 25, not part of the matrix snippet.** Three reasons:
- **The matrix stays what it is.** `cam_to_srgb` is unchanged and still runs where D19 put it, and
so does every reader of it: the mask pass's copy, the white-balance picker's, the camera-space
tap. The tables add a conversion into ProPhoto and back *inside* their own fragment, through two
constant matrices (§3.1). A photograph with no profile composes exactly the shader it does today.
- **It commutes with what runs before it.** HSV hue and saturation are invariant under a uniform
gain, and vignetting and exposure are uniform gains. So a 2.5-D table — every Adobe HueSatMap
seen, and the 6D's — gives the same answer before or after them. That lets one position serve
both tables, which is where the second reason matters:
- **The look sees exposure.** The SDK applies `LookTable` after its exposure ramp, so a look that
desaturates highlights finds the highlights the photographer chose. At 25 it does too. Contrast,
tone and the colour controls come after it, as they do in Camera Raw.
**Tables are per source, like the matrix.** They are decoded with the raw, interpolated once at
decode (the HueSatMap blend uses the as-shot neutral, as the matrix does) and carried on
`DemosaicedImage` next to `color_matrix`. `dr-gpu` uploads them to a storage buffer at
`@binding(8)` and writes their dimensions into the base uniform block. Every render path that
reaches `AdjustPass` therefore gets them without being told: develop, export, previews, the tablet.
A path that had to call a setter on the graph would be a path that one day forgot to, and an export
that differed from the screen would be the result.
### 3.1 The two constants
`P = row-normalise(XYZ→sRGB · Bradford(D50→D65) · ProPhoto→XYZ(D50))` takes ProPhoto to the
working space; the fragment uses `P⁻¹` going in and `P` coming out. These are the same constants
`forward_to_srgb` composes for the forward-matrix route, so for a profile with forward matrices the
round trip is exact: the working colour is `P · FM`-space colour and `P⁻¹` recovers the SDK's
ProPhoto. Row normalisation keeps working-space white at ProPhoto white, so a neutral stays at
`s = 0`. For the colour-matrix route, `P⁻¹` gives the ProPhoto rendering of the same colour, which
is what the SDK derives too.
## 4. Where a profile comes from
In this order, first match wins:
1. **The profile embedded in the DNG being opened.** It is what the file says, and it was made for
the matrices the file carries. Read from the root IFD through rawler's parsed `IFD`, as
`read_dng_matrices` already reads the forward matrices — no second TIFF parser.
2. **A `.dcp` in the profiles directory** whose `UniqueCameraModel` matches the body — compared
case-insensitively against the DNG's `UniqueCameraModel` where there is one, and against
`make + " " + model` otherwise ("Canon EOS 6D"). A DCP is a whole profile: its matrices replace
the file's, because its tables were built against its forward matrix. The file's as-shot neutral
is kept. If several match, the first by file name wins, so the choice is stable.
3. **None.** The matrix alone, as today.
The profiles directory is `profiles/` under the platform data directory (`dr_plat::dirs`), loaded
once per process. A DCP is a TIFF with the magic `IIRC` (0x4352) in place of 42; rawler's
`GenericTiffReader` already accepts it.
**Copying an embedded profile out.** A DNG whose profile has `ProfileEmbedPolicy` 0 ("allow
copying") or 3 ("no restrictions") can have that profile saved as a `.dcp` into the profiles
directory. That is how the 6D's CR2s get Adobe Standard: open a 6D DNG, choose *Use this profile
for every Canon EOS 6D*. Policies 1 ("embed if used") and 2 ("embed never") offer no such action.
The written file carries the profile's name, copyright and policy unchanged.
**Nothing is shipped.** Adobe's profiles are Adobe's; the application ships no `.dcp` and copies
none on its own. A profile reaches the directory because the photographer put it there or asked
for it to be copied from their own file.
## 5. The control
A develop operation, `camera_profile`, `[colour]`, order 25, hand-written (`rust:`) because it
reads a buffer no declaration can name:
| Parameter | Kind | Default | Meaning |
|---|---|---|---|
| `apply` | Bool | on | Use the profile's tables, or the matrix alone |
| `look` | Scalar 0–200 | 100 | Strength of the `LookTable`, as Lightroom's *Amount* |
`look` scales the look's deltas: `hueShift · a`, `1 + (satScale − 1)·a`, `1 + (valScale − 1)·a`,
with `a = look/100`, scales floored at 0. At 200 the look is twice as strong, which is the
"more vivid than Adobe Standard" this started from. The HueSatMap is a calibration and is not
scaled: `apply` is its only switch.
**Always composed while `apply` is on**, as the view transform is: a profile at its defaults *is*
the rendering, not an edit, so an untouched photograph writes no parameters and still renders
through its profile. The fragment branches on the uniform that says whether the source has tables,
so a JPEG, or a raw with none, pays one uniform read. The branch is uniform across the dispatch.
**Mask layers.** A layer may offset `look` (blended as a setting, which is linear) but not `apply`;
the photograph has one profile.
**The panel says which profile is in use**, as the lens line does: *Adobe Standard (in the file)*,
*Adobe Standard (Canon EOS 6D.dcp)*, or *No profile for this camera — matrix only*. The copy-out
action sits on that line.
## 6. Not done, and why
- **`ProfileToneCurve` is read and ignored.** D19 gives tone to the view transform, one for every
body, and rejected per-body curves as its defaults. A profile's curve is a per-body curve. If it
comes back, it comes back as an option of the view transform, not as a stage here. The 6D's
Adobe Standard has none, so the case that matters today loses nothing.
- **`BaselineExposure` is not applied** (it is not today either). Adobe Standard was tuned with it,
and the 6D's is +0.25 EV. Separate change; it moves every photograph's brightness.
- **The interpolation follows the as-shot neutral, not the white-balance slider**, as the matrix
does. Camera Raw re-blends on every temperature change; doing so here means the matrix moves too,
which is its own change.
- **Masks select on the matrix's colour.** A colour-range mask sees colour before the profile, as
it sees colour before every other operation. Deterministic, and a mask is drawn on the picture
the user sees only approximately anyway.
- **The profiles directory does not sync.** A CR2 rendered on a desktop with a copied 6D profile
and on a tablet without one will differ. The panel line says which profile each device used, so
the difference is visible rather than silent. Syncing the directory with the library is the
follow-up.
- **Rec.2020 working primaries** stay deferred (D19); nothing here depends on them.
## 7. What it costs
- **Every DNG with an embedded profile renders differently** — more saturated, which is the point.
Previews rendered before the change keep the old look until rendered again, as with D19.
- **Tablet and desktop must be released together.** No schema change, and the sidecar gains only
ordinary parameters, but two builds render the same DNG differently.
- **One storage-buffer binding** in every generated shader's layout (a one-entry placeholder when
there are no tables), and two vec4 slots in the base uniform block.
- **Per pixel**: two 3×3 multiplies, two HSV round trips, and 4 + 8 buffer reads (bilinear
HueSatMap, trilinear LookTable). Small next to the fused pass it joins.
## 8. Acceptance
- **Parsing.** The 6D DNG's embedded profile parses to `90×30×1` and `36×8×16`, its policy to 0,
its name to "Adobe Standard"; a `.dcp` written from it parses back to the same tables bit for
bit.
- **The CPU reference matches the SDK's algorithm**: grey passes through; a table of
`(0°, 1, 1)` everywhere is the identity to 1e-6; a uniform `satScale` of 1.2 scales HSV
saturation by 1.2; hue interpolation wraps between the last and first column.
- **The shader agrees with the CPU reference** within 1e-4 on a ramp across hue, saturation and
values up to 16.0, on a device.
- **Scene-referred.** A value above 1.0 leaves the stage above 1.0 (`scene_referred_until_the_view`
covers the operation).
- **Neutral.** `apply` off, or no tables, composes a shader whose output equals today's to the bit.
- **Two illuminants.** A HueSatMap at blend weight 0 is Data1, at 1 is Data2.
- **Matching.** An embedded profile beats a directory one; a DCP for "Canon EOS 6D" matches a CR2
whose rawler make/model is "Canon"/"EOS 6D"; no match leaves the matrix.
- **Subjective.** A 6D DNG rendered here at defaults is visibly closer to the same file in
Lightroom 6 with Adobe Standard than the matrix-only render, side by side.
## 9. Vivid presets
Independent of the tables, and shipped in the same change: a *Vivid* section of read-only presets
(`presets/vivid.drpl`) for the "more colourful than the default" request. They use only operations
every photograph has — vibrance, saturation, the colour mixer, colour grading, contrast — so they
work on JPEGs and on bodies with no profile, and change only what they name (FR-DEV-6):
- **Vivid** — vibrance and a little saturation and contrast: the general-purpose one.
- **Vivid, strong** — the same, pushed, with deeper blacks.
- **Vivid landscape** — greens, blues and azure skies, skin bands left alone.
- **Vivid warm** — oranges and yellows up, a warm highlight cast: golden hour.
- **Vivid portrait** — vibrance (which protects skin) with the orange and red bands held back.
They are bounded by the existing `bundled.rs` tests: every key names a real parameter, every value
is inside its control's range, and every preset changes something.
## 10. Build order
1. `dr-decode`: parse the tables (embedded and `.dcp`), the profiles directory, matching, blending,
the `.dcp` writer. CPU reference of the lookup. Unit tests against the library's 6D DNG,
skipped when it is absent.
2. `dr-pipeline`: the `camera_profile` operation, the base-block slots, `@binding(8)`, the WGSL
lookup; composition tests.
3. `dr-gpu`: carry the tables on `DemosaicedImage`, upload and bind them; the shader-versus-CPU
test on a device.
4. `dr-ui`: profile line, copy-out action, labels; the profiles directory set at start-up on
desktop and Android.
5. The *Vivid* presets.
docs/dev/requirements.md
+40 -2
View File
@@ -433,9 +433,18 @@ camera RGB, where its multipliers are defined, and every other operation receive
colour. Before D19 the edits ran in camera RGB and the matrix came after them, so a hue in the
colour mixer and the weights in `luminance()` meant something different on every body.
**Deferred but not foreclosed:** full `.dcp` support with `HueSatDeltas`, `ProfileLookTable`, and
~~**Deferred but not foreclosed:** full `.dcp` support with `HueSatDeltas`, `ProfileLookTable`, and
dual-illuminant interpolation. The stage shall be structured so these are additions rather than a
pipeline reordering.
pipeline reordering.~~ *Amended 2026-10-02 (D20):* dual-illuminant interpolation of the matrices
was built with item 1. The tables follow, designed in [camera-profiles.md](camera-profiles.md):
4. **DCP tables.** `ProfileHueSatMap` (both illuminants, blended as the matrices are) and
`ProfileLookTable`, read from the profile embedded in a DNG or from a `.dcp` file in the
profiles directory matched by `UniqueCameraModel`, the embedded one first. They are applied by a
`camera_profile` scene operation after exposure, with a switch and a look strength (0–200 %),
on by default where a profile exists. `ProfileToneCurve` is read and not applied: tone is the
view transform's (D19). An embedded profile whose `ProfileEmbedPolicy` allows copying can be
saved as a `.dcp` for other files from the same body. The application ships no profile.
Rationale for the reduced scope: a bare 3×3 matrix produces the flat, poor-skin-tone rendering
characteristic of dcraw defaults, which is the documented reason people abandon darktable in the
@@ -448,6 +457,9 @@ measurements, and their provenance was not known well enough to keep them as def
*Acceptance:* the default render is subjectively comparable to the camera's own JPEG — through
FR-DEV-3j's default, for every body. ΔE2000 validation against ColorChecker references applies
once DCP support lands.
For item 4: the lookup follows the DNG SDK's on `[0, 1]` and leaves values above 1.0 above it;
grey and an identity table pass through unchanged; the shader agrees with the CPU reference; with
the switch off, or no profile, the render is unchanged to the bit (camera-profiles.md §8).
**FR-DEV-3f — Look emulation.** Support HaldCLUT import, which inherits the existing free film
simulation ecosystem at near-zero implementation cost, plus reading the in-RAF film simulation tag
@@ -2425,6 +2437,7 @@ Rationale, evidence, and the eliminated alternatives are recorded in
| D12 | Scope versus pace | **DECIDED 2026-09-19** — settled by events; full scope stands, no v1 date |
| D18 | Derived images | **DECIDED 2026-09-19** — a merge writes a new source file; no multi-source Version |
| D19 | Scene-referred pipeline | **DECIDED 2026-09-27** — edits on unbounded scene-linear colour; one view transform, last; per-body base curves retired |
| D20 | DCP camera profiles | **DECIDED 2026-10-02** — HueSatMap and LookTable as a scene operation after exposure; embedded profile first, then a matched `.dcp`; tone curve not applied; none shipped |
### D11 — product positioning
@@ -2730,6 +2743,31 @@ unbounded, but several fragments floor at zero, which clips a colour outside sRG
mixer's bands and the colour grading wheel would need their hues re-measured. Gamut compression
beyond the output transform's clip goes with it.
### D20 — DCP camera profiles · **DECIDED 2026-10-02**
**A camera profile's `HueSatMap` and `LookTable` are applied by a `camera_profile` scene
operation at order 25, after exposure, converting into linear ProPhoto and back inside its own
fragment.** Design and the full argument: [camera-profiles.md](camera-profiles.md).
*Why now.* The library's 9,348 Canon 6D DNGs carry Adobe Standard's tables, which Lightroom
rendered them through, and DarkRoom ignored them. That gap is most of why the same file looks
flatter here.
*Why there.* The matrix snippet stays what D19 made it, and every copy of it (masks, picker,
camera-space tap) stays correct without changing. Hue and saturation are invariant under the
uniform gains that precede order 25, so a 2.5-D HueSatMap gives the same answer there as straight
after the matrix, and the LookTable sees the photographer's exposure, as it does in the SDK.
*Rejected.* Extending the matrix snippet: every duplicate of it would have had to follow. Two
operations, one per table: the HueSatMap has no control of its own and commutes to the same place.
Applying `ProfileToneCurve`: a per-body tone curve is what D19 retired. Shipping Adobe's profiles:
they are not ours to ship. Handing the tables to the graph through a setter, as lens profiles are:
every render path would have to remember to call it. They travel with the decoded image, as the
matrix does.
*What it costs.* Every DNG with an embedded profile renders differently; previews refresh only when
rendered again; tablet and desktop release together. The profiles directory does not sync yet.
### D16 — plugin licensing · **OPEN, post-v1**
> Deferred with §3.10 on 2026-09-19. Still to be answered before the format is published as