dtourolle
c02b401a9a
The second half of D20: a camera_profile scene operation at order 25 that converts working colour into linear ProPhoto, runs the DNG SDK's HSV lookup through the HueSatMap and then the LookTable, and converts back. Hue and saturation do not change under the uniform gains before it, so a 2.5-D HueSatMap gives the same answer as straight after the matrix, and the look sees the photographer's exposure as it does in the SDK. Two departures for scene-referred values: value is not clamped on the way out, and a colour outside ProPhoto passes through. The operation holds only the switch (on by default) and a look strength of 0-200 %. It is composed while the switch is on — a new Operation::composes() separates "does something" from "moved from the defaults", so an untouched raw renders through its profile and still writes nothing. The tables come from the source: dr-gpu uploads the ones DemosaicedImage carries into a storage buffer at @binding(8), whose two-entry header tells the fragment whether there is anything to apply, and binds a header of zeros for every other source. apply_reference is the lookup on the CPU. The GPU test holds the shader to it over 256 colours, through synthetic tables strong enough that a wrong index shows, and through the library's real Adobe Standard tables when the 6D DNG is present.
241 lines
15 KiB
Markdown
241 lines
15 KiB
Markdown
# Camera profiles — DCP tables on top of the matrix
|
||
|
||
Design for the deferred half of **FR-DEV-3e** ([requirements.md](requirements.md)): the
|
||
`HueSatMap` and `LookTable` of a DNG camera profile, read from the DNG that carries one or from a
|
||
`.dcp` file, and applied after the matrix. Draft of 2026-10-02, recorded as **D20**.
|
||
|
||
---
|
||
|
||
## 1. What we are matching
|
||
|
||
Lightroom renders a raw through a *profile* before any slider moves. A profile is the matrix
|
||
DarkRoom already applies, plus two lookup tables indexed by hue, saturation and value:
|
||
|
||
- **`ProfileHueSatMap`** — a calibration. It corrects what a 3×3 cannot: a sensor whose reds and
|
||
oranges sit in the wrong place relative to its blues, which no linear map fixes. Two copies, one
|
||
per calibration illuminant, interpolated like the matrices.
|
||
- **`ProfileLookTable`** — a rendering intent. The "Adobe Standard" look: richer blues and greens,
|
||
warmer yellows, skin held where it is. The difference between Adobe Standard, Adobe Color and
|
||
Adobe Vivid is mostly this table.
|
||
|
||
Without them a raw renders through the matrix alone, which is accurate on a ColorChecker and reads
|
||
as flat next to Lightroom. The first complaint FR-DEV-3e's rationale names is exactly this, and
|
||
the per-body base curves D19 retired were the cheaper stand-in for it.
|
||
|
||
**What the library holds** (catalog of 2026-10-02): 17,286 of its raws are Canon EOS 6D. The
|
||
9,348 DNGs were written by Lightroom 6.14 and every one sampled embeds *Adobe Standard* with both
|
||
tables — `HueSatMapDims 90 30 1`, `LookTableDims 36 8 16`, `ProfileEmbedPolicy 0` ("allow
|
||
copying"), no `ProfileToneCurve`. The 7,938 CR2s from the same body carry no profile. So the
|
||
tables Lightroom used are already on disk for half the library, and their licence lets them be
|
||
applied to the other half.
|
||
|
||
## 2. The model, as the DNG specification states it
|
||
|
||
Each table is a grid of `(hueShift°, satScale, valScale)` triples over HSV, stored with
|
||
saturation varying fastest, then hue, then value:
|
||
|
||
```
|
||
index = v · (hueDivs · satDivs) + h · satDivs + s
|
||
```
|
||
|
||
Lookup follows the DNG SDK's `RefBaselineHueSatMap`:
|
||
|
||
- **HSV** is the SDK's: `v = max(r,g,b)`, `s = (v − min)/v`, `h ∈ [0, 6)` from which channel
|
||
leads. Grey has `s = 0` and is untouched by construction.
|
||
- **Hue wraps**: `hueDivs` samples over 360°, the last interpolating to the first.
|
||
- **Saturation** samples `0..=1` at `satDivs` points; linear between.
|
||
- **Value** samples `0..=1` at `valDivs` points when `valDivs > 1`; a table with `valDivs = 1` is
|
||
"2.5-D" and ignores value. `ProfileHueSatMapEncoding` / `ProfileLookTableEncoding` = 1 means the
|
||
value axis is indexed by the sRGB-encoded value; 0 (the default, and the 6D's) means linear.
|
||
- **Apply**: `h += hueShift · 6/360`, `s = min(s · satScale, 1)`, `v ·= valScale`; back to RGB.
|
||
- **Space**: linear ProPhoto (ROMM) primaries, D50 white — the space the forward matrix lands in.
|
||
- **Two illuminants**: `HueSatMapData1/2` are interpolated entry by entry, with the same mired
|
||
weight the matrices use. `LookTable` is single.
|
||
|
||
Two departures, both forced by D19's unbounded scene-linear values (the SDK runs these on `[0, 1]`):
|
||
|
||
1. **Value is not clamped.** The SDK writes `min(v · valScale, 1)`; here `v · valScale`, unbounded.
|
||
For lookup only, the value axis reads `min(v, 1)`, so a highlight above 1.0 uses the table's
|
||
brightest row. Where the encoding is sRGB the scale is defined on the encoded value; it is
|
||
applied as the ratio `decode(enc(v′)·valScale)/v′` at `v′ = min(v, 1)`, so a value above 1.0
|
||
gets the brightest row's ratio rather than a clip.
|
||
2. **A colour outside ProPhoto passes through.** A negative component has no SDK HSV. Such a
|
||
colour is outside every surface colour a camera records under normal light. It is left
|
||
unmodified rather than floored, because flooring it clips a value D19 says nothing may clip.
|
||
|
||
## 3. Where it sits
|
||
|
||
```
|
||
… camera matrix ─► vignetting(5) ─► exposure(20) ─► camera_profile(25) ─► contrast(30) ─► … ─► view transform
|
||
│
|
||
working → ProPhoto ─► HueSatMap ─► LookTable ─► ProPhoto → working
|
||
```
|
||
|
||
**A scene operation at order 25, not part of the matrix snippet.** Three reasons:
|
||
|
||
- **The matrix stays what it is.** `cam_to_srgb` is unchanged and still runs where D19 put it, and
|
||
so does every reader of it: the mask pass's copy, the white-balance picker's, the camera-space
|
||
tap. The tables add a conversion into ProPhoto and back *inside* their own fragment, through two
|
||
constant matrices (§3.1). A photograph with no profile composes exactly the shader it does today.
|
||
- **It commutes with what runs before it.** HSV hue and saturation are invariant under a uniform
|
||
gain, and vignetting and exposure are uniform gains. So a 2.5-D table — every Adobe HueSatMap
|
||
seen, and the 6D's — gives the same answer before or after them. That lets one position serve
|
||
both tables, which is where the second reason matters:
|
||
- **The look sees exposure.** The SDK applies `LookTable` after its exposure ramp, so a look that
|
||
desaturates highlights finds the highlights the photographer chose. At 25 it does too. Contrast,
|
||
tone and the colour controls come after it, as they do in Camera Raw.
|
||
|
||
**Tables are per source, like the matrix.** They are decoded with the raw, interpolated once at
|
||
decode (the HueSatMap blend uses the as-shot neutral, as the matrix does) and carried on
|
||
`DemosaicedImage` next to `color_matrix`. `dr-gpu` uploads them to a storage buffer at
|
||
`@binding(8)` and writes their dimensions into the base uniform block. Every render path that
|
||
reaches `AdjustPass` therefore gets them without being told: develop, export, previews, the tablet.
|
||
A path that had to call a setter on the graph would be a path that one day forgot to, and an export
|
||
that differed from the screen would be the result.
|
||
|
||
### 3.1 The two constants
|
||
|
||
`P⁻¹` is `ColourSpace::ProPhoto.from_linear_srgb()` — the conversion `dr-types` already derives
|
||
from the two spaces' chromaticities, adapting D65 to D50 by Bradford, which the export path uses to
|
||
write ProPhoto files — and `P` is its inverse. The fragment uses `P⁻¹` going in and `P` coming
|
||
out. Each row of both is scaled to sum to one, so working-space white is ProPhoto white exactly and
|
||
a neutral reaches the tables at `s = 0`. For a profile with forward matrices this recovers the
|
||
SDK's ProPhoto colour to within the difference between that derivation and `forward_to_srgb`'s
|
||
published Bradford constants, which is rounding.
|
||
|
||
## 4. Where a profile comes from
|
||
|
||
In this order, first match wins:
|
||
|
||
1. **The profile embedded in the DNG being opened.** It is what the file says, and it was made for
|
||
the matrices the file carries. Read from the root IFD through rawler's parsed `IFD`, as
|
||
`read_dng_matrices` already reads the forward matrices — no second TIFF parser.
|
||
2. **A `.dcp` in the profiles directory** whose `UniqueCameraModel` matches the body — compared
|
||
case-insensitively against the DNG's `UniqueCameraModel` where there is one, and against
|
||
`make + " " + model` otherwise ("Canon EOS 6D"). A DCP is a whole profile: its matrices replace
|
||
the file's, because its tables were built against its forward matrix. The file's as-shot neutral
|
||
is kept. If several match, the first by file name wins, so the choice is stable.
|
||
3. **None.** The matrix alone, as today.
|
||
|
||
The profiles directory is `profiles/` under the platform data directory (`dr_plat::dirs`), loaded
|
||
once per process. A DCP is a TIFF with the magic `IIRC` (0x4352) in place of 42; rawler's
|
||
`GenericTiffReader` already accepts it.
|
||
|
||
**Copying an embedded profile out.** A DNG whose profile has `ProfileEmbedPolicy` 0 ("allow
|
||
copying") or 3 ("no restrictions") can have that profile saved as a `.dcp` into the profiles
|
||
directory. That is how the 6D's CR2s get Adobe Standard: open a 6D DNG, choose *Use this profile
|
||
for every Canon EOS 6D*. Policies 1 ("embed if used") and 2 ("embed never") offer no such action.
|
||
The written file carries the profile's name, copyright and policy unchanged.
|
||
|
||
**Nothing is shipped.** Adobe's profiles are Adobe's; the application ships no `.dcp` and copies
|
||
none on its own. A profile reaches the directory because the photographer put it there or asked
|
||
for it to be copied from their own file.
|
||
|
||
## 5. The control
|
||
|
||
A develop operation, `camera_profile`, `[colour]`, order 25, hand-written (`rust:`) because it
|
||
reads a buffer no declaration can name:
|
||
|
||
| Parameter | Kind | Default | Meaning |
|
||
|---|---|---|---|
|
||
| `apply` | Bool | on | Use the profile's tables, or the matrix alone |
|
||
| `look` | Scalar 0–200 | 100 | Strength of the `LookTable`, as Lightroom's *Amount* |
|
||
|
||
`look` scales the look's deltas: `hueShift · a`, `1 + (satScale − 1)·a`, `1 + (valScale − 1)·a`,
|
||
with `a = look/100`, scales floored at 0. At 200 the look is twice as strong, which is the
|
||
"more vivid than Adobe Standard" this started from. The HueSatMap is a calibration and is not
|
||
scaled: `apply` is its only switch.
|
||
|
||
**Always composed while `apply` is on**, as the view transform is: a profile at its defaults *is*
|
||
the rendering, not an edit, so an untouched photograph writes no parameters and still renders
|
||
through its profile. The fragment branches on the uniform that says whether the source has tables,
|
||
so a JPEG, or a raw with none, pays one uniform read. The branch is uniform across the dispatch.
|
||
|
||
**Mask layers.** A layer may offset `look` (blended as a setting, which is linear) but not `apply`;
|
||
the photograph has one profile.
|
||
|
||
**The panel says which profile is in use**, as the lens line does: *Adobe Standard (in the file)*,
|
||
*Adobe Standard (Canon EOS 6D.dcp)*, or *No profile for this camera — matrix only*. The copy-out
|
||
action sits on that line.
|
||
|
||
## 6. Not done, and why
|
||
|
||
- **`ProfileToneCurve` is read and ignored.** D19 gives tone to the view transform, one for every
|
||
body, and rejected per-body curves as its defaults. A profile's curve is a per-body curve. If it
|
||
comes back, it comes back as an option of the view transform, not as a stage here. The 6D's
|
||
Adobe Standard has none, so the case that matters today loses nothing.
|
||
- **`BaselineExposure` is not applied** (it is not today either). Adobe Standard was tuned with it,
|
||
and the 6D's is +0.25 EV. Separate change; it moves every photograph's brightness.
|
||
- **The interpolation follows the as-shot neutral, not the white-balance slider**, as the matrix
|
||
does. Camera Raw re-blends on every temperature change; doing so here means the matrix moves too,
|
||
which is its own change.
|
||
- **Masks select on the matrix's colour.** A colour-range mask sees colour before the profile, as
|
||
it sees colour before every other operation. Deterministic, and a mask is drawn on the picture
|
||
the user sees only approximately anyway.
|
||
- **The profiles directory does not sync.** A CR2 rendered on a desktop with a copied 6D profile
|
||
and on a tablet without one will differ. The panel line says which profile each device used, so
|
||
the difference is visible rather than silent. Syncing the directory with the library is the
|
||
follow-up.
|
||
- **Rec.2020 working primaries** stay deferred (D19); nothing here depends on them.
|
||
|
||
## 7. What it costs
|
||
|
||
- **Every DNG with an embedded profile renders differently** — more saturated, which is the point.
|
||
Previews rendered before the change keep the old look until rendered again, as with D19.
|
||
- **Tablet and desktop must be released together.** No schema change, and the sidecar gains only
|
||
ordinary parameters, but two builds render the same DNG differently.
|
||
- **One storage-buffer binding** in every generated shader's layout (a one-entry placeholder when
|
||
there are no tables), and two vec4 slots in the base uniform block.
|
||
- **Per pixel**: two 3×3 multiplies, two HSV round trips, and 4 + 8 buffer reads (bilinear
|
||
HueSatMap, trilinear LookTable). Small next to the fused pass it joins.
|
||
|
||
## 8. Acceptance
|
||
|
||
- **Parsing.** The 6D DNG's embedded profile parses to `90×30×1` and `36×8×16`, its policy to 0,
|
||
its name to "Adobe Standard"; a `.dcp` written from it parses back to the same tables bit for
|
||
bit.
|
||
- **The CPU reference matches the SDK's algorithm**: grey passes through; a table of
|
||
`(0°, 1, 1)` everywhere is the identity to 1e-6; a uniform `satScale` of 1.2 scales HSV
|
||
saturation by 1.2; hue interpolation wraps between the last and first column.
|
||
- **The shader agrees with the CPU reference** on a device, within two 8-bit codes of the
|
||
display-encoded readback (the only readback the adjust pass has), over 256 colours that tables
|
||
of tens of degrees and ±30 % saturation move, and over the library's real Adobe Standard tables
|
||
(`dr-gpu/tests/camera_profile.rs`).
|
||
- **Scene-referred.** A value above 1.0 leaves the stage above 1.0 (`scene_referred_until_the_view`
|
||
covers the operation).
|
||
- **Neutral.** `apply` off renders to the bit what a source with no tables renders.
|
||
- **Two illuminants.** A HueSatMap at blend weight 0 is Data1, at 1 is Data2.
|
||
- **Matching.** An embedded profile beats a directory one; a DCP for "Canon EOS 6D" matches a CR2
|
||
whose rawler make/model is "Canon"/"EOS 6D"; no match leaves the matrix.
|
||
- **Subjective.** A 6D DNG rendered here at defaults is visibly closer to the same file in
|
||
Lightroom 6 with Adobe Standard than the matrix-only render, side by side.
|
||
|
||
## 9. Vivid presets
|
||
|
||
Independent of the tables, and shipped in the same change: a *Vivid* section of read-only presets
|
||
(`presets/vivid.drpl`) for the "more colourful than the default" request. They use only operations
|
||
every photograph has — vibrance, saturation, the colour mixer, colour grading, contrast — so they
|
||
work on JPEGs and on bodies with no profile, and change only what they name (FR-DEV-6):
|
||
|
||
- **Vivid** — vibrance and a little saturation and contrast: the general-purpose one.
|
||
- **Vivid, strong** — the same, pushed, with deeper blacks.
|
||
- **Vivid landscape** — greens, blues and azure skies, skin bands left alone.
|
||
- **Vivid warm** — oranges and yellows up, a warm highlight cast: golden hour.
|
||
- **Vivid portrait** — vibrance (which protects skin) with the orange and red bands held back.
|
||
|
||
They are bounded by the existing `bundled.rs` tests: every key names a real parameter, every value
|
||
is inside its control's range, and every preset changes something.
|
||
|
||
## 10. Build order
|
||
|
||
1. `dr-decode`: parse the tables (embedded and `.dcp`), the profiles directory, matching, blending,
|
||
the `.dcp` writer. CPU reference of the lookup. Unit tests against the library's 6D DNG,
|
||
skipped when it is absent.
|
||
2. `dr-pipeline`: the `camera_profile` operation, the base-block slots, `@binding(8)`, the WGSL
|
||
lookup; composition tests.
|
||
3. `dr-gpu`: carry the tables on `DemosaicedImage`, upload and bind them; the shader-versus-CPU
|
||
test on a device.
|
||
4. `dr-ui`: profile line, copy-out action, labels; the profiles directory set at start-up on
|
||
desktop and Android.
|
||
5. The *Vivid* presets.
|