dtourolle/DarkRoom
1
0
Fork 0
Code Issues 39 Pull Requests Actions Packages Projects Releases 20 Wiki Activity

Store what the model found, so a reopened photograph keeps its masks

Browse Source 
A subject or category layer was written to the sidecar as identity alone —
which run, which instance, which category — on the reasoning that the pixels
are reproducible by running the same model over the same image. They are, but
only by *running the model*, and nothing runs one except a photographer
pressing "find subjects". So on every path that did not already have a run in
memory the layer resolved to no coverage, `MaskPass::render` logged "has no
distance field; skipping", and the adjustment was silently absent:

  - reopening an edited photograph rendered it without its local adjustments,
    and then saved that state back on the way out;
  - a batch export from the grid could not have them at any point, because
    `render_from_library` opens a session, applies a version and renders, and
    there is no model anywhere on that path. Three hundred files written
    without the edits their photographer made, over a log warning.

Neither failure announced itself. The generated shader still emits the layer's
block and the empty placeholder multiplies it by zero, so the result is a
well-formed frame that is simply missing an edit — `mask_is_stale` already
named the state and called it "not stale, just unrenderable".

The coverage now travels in the file, as one `coverage = w h levels payload`
line at the end of the layer's block.

Two levels, and that is not a compromise. The model hands out a byte per pixel
but `Shaped::build` measures its distance field from `coverage >= 128` and
throws the shoulder away on the first line; everything soft about the rendered
edge comes afterwards from the layer's feather and falloff, which are read off
the distance. So one bit per pixel is not an approximation of what the model
said — it is exactly the part of it that reaches a pixel, and the stored mask
renders the identical frame. Storing all 256 levels would have stored 1.7 MB
of bilinear interpolation to reconstruct a predicate, and would not even have
compressed: a model mask is a bilinear upsample of a coarse grid, so almost no
two adjacent bytes are alike. Measured on a simulated sky and a simulated
figure at 1600x1067, against 1.71 MB raw: 4.0 kB and 6.5 kB at two levels,
46 kB and 76 kB at sixteen, 835 kB and 1.43 MB at all 256. The level count is
still written into the line, so a later build that finds a use for the
shoulder can write sixteen and this one will read them rather than misreading
a stream of lengths as pairs.

The coder is hand-rolled — run-length pairs in a base-64 varint — because
`dr-pipeline` links nothing, which is the property that lets the descriptor
and codegen logic be tested without a device. `flate2` would have been fewer
lines and a dependency in the one crate that has none.

Where it lives matters more than how it is coded. The raster sits on
`MaskLayer` beside the source, not inside `MaskSource::Subject`: the source is
*identity*, which is what makes it diff as a handful of numbers and merge per
field under FR-NC-9, and a raster in there would have given the merge a binary
blob to arbitrate. It takes no part in `MaskLayer`'s equality for the same
reason — a device that has run the model and one that has not hold the same
edit, and counting the difference would raise a conflict over a cache and let
`remote_wins` answer it by discarding the only copy of the pixels.

Encoding happens in `masks_for_storage`, on the save path, rather than in
`ensure_subject_fields` where every coverage already funnels through.
`ensure_subject_fields` runs on a drag — dilating a mask with a compound
morphology rebuilds the field every frame — and encoding a megapixel raster
per frame is the kind of work NFR-P5 exists to keep off a gesture. Saving
happens once, when the photograph stops being the open one, and already costs
a network round trip.

Version skew holds both ways. A file with no `coverage` line reads exactly as
it did before, which is a layer that needs the model run; an unreadable one
costs the pixels and not the layer, because the layer is the edit and the
raster is a cache of it. An old build reading a new file drops the key it does
not understand, which costs a model run and no work. And a payload that will
not compress is refused rather than truncated: a checkerboard would encode to
twice the raster it came from, so past 64 kB nothing is stored and the
behaviour falls back to what it was — half a mask would render as a mask that
is confidently wrong, which is the failure that tells nobody.
This commit is contained in:
dtourolle
2026-08-30 21:28:54 +02:00
parent 6acc98baad
commit 89c4ff1820
9 changed files with 1688 additions and 91 deletions
Download Patch File Download Diff File Expand all files Collapse all files
core/dr-pipeline/src/coverage.rs
+634
View File
@@ -0,0 +1,634 @@
//! TRACES: FR-DEV-3 | FR-CAT-8
//! A model's mask, in a form a sidecar can carry.
//!
//! [`MaskSource::Subject`](crate::mask::MaskSource::Subject) and
//! [`MaskSource::Category`](crate::mask::MaskSource::Category) name what they
//! cover — an index, a class, a category — and naming is enough only while
//! the run that produced the numbers is still in memory. Reopen the
//! photograph, or export it from the grid, and there is no run: the layer
//! resolves to nothing and the local adjustment silently is not applied. That
//! is what this module exists to stop. It stores the *pixels* the layer
//! covered, beside the identity rather than instead of it, so a second model
//! pass is an optimisation rather than a precondition.
//!
//! # Two levels, and why that is not a compromise
//!
//! The model hands out a byte per pixel, but nothing downstream reads more
//! than one bit of it. A subject or category layer becomes a mask by way of
//! an exact Euclidean distance field, and that field is measured from
//! `coverage >= threshold` — the soft shoulder the model produced is
//! discarded on the first line of the transform. Everything soft about the
//! rendered edge comes afterwards, from [`MaskLayer::feather`] and
//! [`MaskLayer::falloff`], which are read off the *distance*.
//!
//! [`MaskLayer::feather`]: crate::mask::MaskLayer::feather
//! [`MaskLayer::falloff`]: crate::mask::MaskLayer::falloff
//!
//! So [`RENDERED_LEVELS`] is two, and the result is not an approximation of
//! what the model said: it is exactly the part of what the model said that
//! reaches a pixel. Storing all 256 levels would be storing 1.7 MB of
//! interpolation to reconstruct a predicate — and it would not even compress,
//! because a model mask is a bilinear upsample of a coarse grid and therefore
//! has almost no two adjacent bytes alike. Measured on a simulated sky and a
//! simulated figure at 1600x1067, against 1.71 MB raw: **4.0 kB and 6.5 kB at
//! two levels**, 46 kB and 76 kB at sixteen, and 835 kB and 1.43 MB at all
//! 256 — the last two being over [`MAX_PAYLOAD`] and therefore not storable
//! at all.
//!
//! [`Coverage::encode`] still takes the level count, and it is written into
//! the line, so a later build that finds a use for the shoulder can write
//! sixteen levels and this one will read them back correctly rather than
//! misreading a stream of lengths as pairs.
//!
//! # One line, because a node is a line
//!
//! FR-NC-9 merges the edit graph per node and [`Version::merge`] does that by
//! comparing lines, so a stored mask is one `coverage = ...` line inside the
//! layer's block — the same shape the `regions = ...` line already had, for
//! the same reason. Splitting it over many lines would put a single opaque
//! blob into the merge as several independently-winnable keys, which is a
//! merge that can produce a mask neither device ever had.
//!
//! [`Version::merge`]: crate::sidecar::Version::merge
//!
//! # Hand-rolled, and it has to be
//!
//! `dr-pipeline` links nothing (ARCH §6.5a), which is what lets the descriptor
//! and codegen logic be tested without a device. That rules out `flate2`,
//! `serde` and `base64`, so the run-length coder and the digits below are
//! written out. It is forty lines, and the alternative was a dependency in the
//! one crate that has none.
use std::fmt::Write as _;
/// The number of coverage levels the renderer can actually tell apart.
///
/// Two. See the module header: the distance field is built from a threshold,
/// so a second level is the whole of the information that survives into a
/// rendered frame. Named rather than written as `2` at the call site because
/// the number is a *claim about the render path*, and a claim wants somewhere
/// to be explained.
pub const RENDERED_LEVELS: u32 = 2;
/// The most encoded payload a stored coverage may take, in bytes.
///
/// Sidecars sync over WebDAV and are read whole by every device that opens the
/// photograph, so a mask that will not compress must not be allowed to make
/// the file enormous — it is a *cache* of something a model can produce again,
/// and no cache is worth a megabyte of sync traffic per layer.
///
/// A realistic mask lands between 4 and 7 kB, so this is roughly ten times the
/// worst case anyone has measured: enough for genuinely awkward subjects —
/// foliage, chain-link, hair against a busy background — and far short of a
/// file a human cannot open. Past it [`Coverage::encode`] returns `None`, the
/// layer stores nothing, and the behaviour falls back to what it was before
/// this module existed: the mask needs the model run. Refusing rather than
/// truncating, because half a mask renders as a *wrong* mask, which is the
/// failure that announces itself to nobody.
pub const MAX_PAYLOAD: usize = 64 * 1024;
/// The most pixels a coverage read from a file may claim.
///
/// A file is not trusted. The proxy a mask is built at is bounded by the long
/// edge the segmentation runs on — under three megapixels — so this is ample
/// headroom, and it is here so that `width * height` from a corrupt line
/// cannot ask for an allocation measured in gigabytes.
pub const MAX_PIXELS: usize = 16 << 20;
/// Digits of the payload's base-64 varint. Ordered so the alphabet is stable
/// and contains nothing a line-oriented format would have to escape — no
/// whitespace, no `=`, no `#`.
const DIGITS: &[u8; 64] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_";
/// Bit set in a digit that means "another digit follows".
const CONTINUE: u32 = 32;
/// Value bits carried by one digit.
const CHUNK: u32 = 5;
const fn reverse_digits() -> [u8; 256] {
let mut table = [255u8; 256];
let mut i = 0;
while i < 64 {
table[DIGITS[i] as usize] = i as u8;
i += 1;
}
table
}
/// Digit value by byte, `255` for anything that is not a digit.
const REVERSE: [u8; 256] = reverse_digits();
/// TRACES: FR-DEV-3
/// One layer's pixel coverage, held in the form it is stored in.
///
/// **Encoded, not expanded.** The struct owns the payload text rather than the
/// 1.7 MB raster it decodes to, because that raster is wanted exactly once —
/// when a distance field is built — and is held by nothing afterwards. Keeping
/// it expanded would put a megabyte and a half per layer into every undo
/// snapshot the history stack holds, to save a decode that costs far less than
/// the exact Euclidean transform immediately following it.
///
/// It also makes the round trip byte-identical for free: a coverage read from
/// a file and written back is the same characters, which is the property that
/// lets a caller skip an upload by comparing content
/// ([`Sidecar`](crate::Sidecar)).
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Coverage {
width: usize,
height: usize,
levels: u32,
payload: String,
}
impl Coverage {
/// Encode one byte-per-pixel coverage, or `None` where it will not fit.
///
/// `levels` is what the bytes are quantised to on the way in; see
/// [`RENDERED_LEVELS`] for why two is the honest answer for a mask that is
/// going to be thresholded.
///
/// `None` for a mismatched length, a nonsensical level count, or a payload
/// over [`MAX_PAYLOAD`] — all three meaning "do not store this", which the
/// caller can act on identically because the fallback is the same in every
/// case.
pub fn encode(values: &[u8], width: usize, height: usize, levels: u32) -> Option<Self> {
if width == 0 || height == 0 || values.len() != width.checked_mul(height)? {
return None;
}
if !(2..=256).contains(&levels) {
return None;
}
let top = levels - 1;
let mut payload = String::new();
let mut index = 0;
while index < values.len() {
let level = quantise(values[index], top);
let mut run = 1;
while index + run < values.len() && quantise(values[index + run], top) == level {
run += 1;
}
push_varint(&mut payload, level as u64);
push_varint(&mut payload, run as u64);
index += run;
// Checked inside the loop rather than after it: the pathological
// input is one that runs to a payload larger than the raster, and
// building the whole of that before deciding to throw it away is
// the allocation this bound exists to prevent.
if payload.len() > MAX_PAYLOAD {
return None;
}
}
Some(Self {
width,
height,
levels,
payload,
})
}
/// Read the value of a sidecar `coverage` line.
///
/// `None` for anything that does not describe a complete raster. A
/// coverage is a cache, so refusing it costs a model run; accepting a
/// partial one costs a photograph rendered with a mask that is wrong in a
/// way nothing reports.
pub fn parse(value: &str) -> Option<Self> {
let mut tokens = value.split_whitespace();
let width: usize = tokens.next()?.parse().ok()?;
let height: usize = tokens.next()?.parse().ok()?;
let levels: u32 = tokens.next()?.parse().ok()?;
let payload = tokens.next()?;
let pixels = width.checked_mul(height)?;
if pixels == 0 || pixels > MAX_PIXELS || !(2..=256).contains(&levels) {
return None;
}
if payload.len() > MAX_PAYLOAD {
return None;
}
// Measured rather than expanded. The payload has to be checked here —
// failing at the point of use would put the error in the renderer,
// where there is no longer a file to name in the message — but a
// library scan parses thousands of sidecars, and materialising a
// megabyte and a half per layer to establish that the arithmetic adds
// up would make opening the grid pay for masks nobody is rendering.
if measure(payload, levels)? != pixels {
return None;
}
Some(Self {
width,
height,
levels,
payload: payload.to_string(),
})
}
/// The value to write after `coverage = `.
pub fn to_text(&self) -> String {
format!(
"{} {} {} {}",
self.width, self.height, self.levels, self.payload
)
}
pub fn width(&self) -> usize {
self.width
}
pub fn height(&self) -> usize {
self.height
}
pub fn levels(&self) -> u32 {
self.levels
}
/// The encoded payload's length in bytes — what this costs a sidecar.
pub fn encoded_len(&self) -> usize {
self.payload.len()
}
/// Expand back to one byte per pixel, at the size it was stored at.
pub fn decode(&self) -> Vec<u8> {
decode(&self.payload, self.width * self.height, self.levels)
.expect("a Coverage only exists once its payload has been decoded once")
}
/// Expand to `width` x `height`, resampling if that is not the size it was
/// stored at.
///
/// Nearest neighbour, and deliberately: the values are a threshold's two
/// sides, so interpolating between them would invent coverage levels that
/// mean nothing and move the boundary by a rounding rule rather than by a
/// measurement. The resample only runs at all when a build reads a mask
/// stored against a different proxy edge — in the ordinary case the sizes
/// match and this is the decode.
pub fn decode_at(&self, width: usize, height: usize) -> Vec<u8> {
let source = self.decode();
if (width, height) == (self.width, self.height) {
return source;
}
if width == 0 || height == 0 {
return Vec::new();
}
let mut out = vec![0u8; width * height];
for y in 0..height {
let sy = ((y * self.height) / height).min(self.height - 1);
let row = sy * self.width;
for x in 0..width {
let sx = ((x * self.width) / width).min(self.width - 1);
out[y * width + x] = source[row + sx];
}
}
out
}
}
/// One byte to its level, rounding to nearest.
fn quantise(value: u8, top: u32) -> u32 {
((value as u32 * top) + 127) / 255
}
/// One level back to a byte, so that the top level is exactly 255.
fn dequantise(level: u32, top: u32) -> u8 {
(((level * 255) + top / 2) / top).min(255) as u8
}
/// Little-endian base-64 varint: five value bits per digit, the sixth saying
/// whether another follows.
fn push_varint(out: &mut String, mut value: u64) {
loop {
let chunk = (value & (CONTINUE - 1) as u64) as u32;
value >>= CHUNK;
let more = if value != 0 { CONTINUE } else { 0 };
let _ = out.write_char(DIGITS[(chunk | more) as usize] as char);
if value == 0 {
return;
}
}
}
/// Read one varint, returning it and how many digits it took.
fn read_varint(bytes: &[u8]) -> Option<(u64, usize)> {
let mut value: u64 = 0;
let mut shift = 0;
for (taken, &byte) in bytes.iter().enumerate() {
let digit = REVERSE[byte as usize];
if digit == 255 {
return None;
}
// A run cannot exceed MAX_PIXELS and a level cannot exceed 255, so a
// varint past this width is a corrupt line rather than a large number.
if shift >= 64 {
return None;
}
value |= ((digit as u64) & (CONTINUE - 1) as u64) << shift;
if digit as u32 & CONTINUE == 0 {
return Some((value, taken + 1));
}
shift += CHUNK;
}
None
}
/// Walk a payload's runs, handing each `(level, length)` to `take`.
///
/// Returns the total length, or `None` for a payload that is not well formed:
/// a digit that is not one, a truncated varint, a zero-length run, or a level
/// the declared count does not contain.
fn walk(payload: &str, levels: u32, mut take: impl FnMut(u32, usize)) -> Option<usize> {
let top = levels - 1;
let bytes = payload.as_bytes();
let mut total: usize = 0;
let mut at = 0;
while at < bytes.len() {
let (level, used) = read_varint(&bytes[at..])?;
at += used;
let (run, used) = read_varint(&bytes[at..])?;
at += used;
let level = u32::try_from(level).ok()?;
if level > top {
return None;
}
let run = usize::try_from(run).ok()?;
// A zero-length run is not a shorter way of saying anything, so it is
// a corrupt line rather than a run to skip — and left in, two of them
// would encode the same raster two ways and break the byte-identical
// round trip the sidecar relies on.
if run == 0 {
return None;
}
total = total.checked_add(run)?;
if total > MAX_PIXELS {
return None;
}
take(level, run);
}
Some(total)
}
/// How many pixels a payload covers, without building any of them.
fn measure(payload: &str, levels: u32) -> Option<usize> {
walk(payload, levels, |_, _| {})
}
/// Expand a payload to `pixels` bytes, or `None` if it does not describe
/// exactly that many.
fn decode(payload: &str, pixels: usize, levels: u32) -> Option<Vec<u8>> {
let top = levels - 1;
let mut out = Vec::with_capacity(pixels);
let total = walk(payload, levels, |level, run| {
out.resize(out.len() + run, dequantise(level, top));
})?;
(total == pixels).then_some(out)
}
#[cfg(test)]
mod tests {
use super::*;
/// Round trip at the level count the renderer actually uses.
fn round_trip(values: &[u8], width: usize, height: usize) -> Vec<u8> {
let coverage = Coverage::encode(values, width, height, RENDERED_LEVELS)
.expect("this mask should encode");
let text = coverage.to_text();
let read = Coverage::parse(&text).expect("what was written should parse");
assert_eq!(read, coverage, "the round trip changed the encoding");
assert_eq!(read.to_text(), text, "re-writing must be byte-identical");
read.decode()
}
/// Two levels is exactly the predicate the distance transform applies, so
/// the round trip must agree with it on every pixel.
fn thresholded(values: &[u8]) -> Vec<bool> {
values.iter().map(|&v| v >= 128).collect()
}
#[test]
fn an_empty_mask_round_trips() {
let values = vec![0u8; 64 * 32];
let back = round_trip(&values, 64, 32);
assert_eq!(back, values);
}
#[test]
fn a_full_mask_round_trips() {
let values = vec![255u8; 64 * 32];
let back = round_trip(&values, 64, 32);
assert_eq!(back, values);
}
#[test]
fn a_single_pixel_mask_round_trips() {
let mut values = vec![0u8; 64 * 32];
values[17 * 64 + 33] = 255;
let back = round_trip(&values, 64, 32);
assert_eq!(back, values);
}
#[test]
fn a_one_pixel_raster_round_trips() {
assert_eq!(round_trip(&[255], 1, 1), vec![255]);
assert_eq!(round_trip(&[0], 1, 1), vec![0]);
}
/// The whole of the fidelity claim: two levels loses nothing the renderer
/// could have used, because the renderer thresholds.
#[test]
fn two_levels_preserve_the_threshold_exactly() {
let values: Vec<u8> = (0..=255u8).collect();
let back = round_trip(&values, 16, 16);
assert_eq!(thresholded(&back), thresholded(&values));
// And the shoulder really is gone, which is the cost being paid.
assert!(back.iter().all(|&v| v == 0 || v == 255));
}
/// A soft edge quantised to sixteen levels stays within one step of what
/// went in, so a later build that wants the shoulder can have it.
#[test]
fn sixteen_levels_are_within_one_step() {
let values: Vec<u8> = (0..256).map(|i| i as u8).collect();
let coverage = Coverage::encode(&values, 16, 16, 16).expect("should encode");
let back = Coverage::parse(&coverage.to_text())
.expect("should parse")
.decode();
for (a, b) in values.iter().zip(&back) {
assert!(
(*a as i32 - *b as i32).abs() <= 255 / 15 / 2 + 1,
"{a} came back as {b}"
);
}
}
/// The case run-length coding is worst at. It must refuse rather than
/// write a payload larger than the raster it came from.
#[test]
fn alternating_detail_is_refused_rather_than_expanded() {
let (w, h) = (512, 512);
let values: Vec<u8> = (0..w * h)
.map(|i| if i % 2 == 0 { 0 } else { 255 })
.collect();
assert!(
Coverage::encode(&values, w, h, RENDERED_LEVELS).is_none(),
"a checkerboard must not be stored"
);
}
/// Small enough to fit, and still exact — the bound is on size, not on
/// shape, so awkward detail that *does* fit must survive intact.
#[test]
fn alternating_detail_that_fits_is_exact() {
let (w, h) = (64, 64);
let values: Vec<u8> = (0..w * h)
.map(|i| if i % 2 == 0 { 0 } else { 255 })
.collect();
let back = round_trip(&values, w, h);
assert_eq!(back, values);
}
#[test]
fn a_mask_of_the_wrong_length_is_refused() {
assert!(Coverage::encode(&[0u8; 10], 4, 4, RENDERED_LEVELS).is_none());
assert!(Coverage::encode(&[], 0, 0, RENDERED_LEVELS).is_none());
}
#[test]
fn a_payload_that_does_not_cover_the_raster_is_refused() {
let values = vec![0u8; 32];
let coverage = Coverage::encode(&values, 8, 4, RENDERED_LEVELS).expect("should encode");
let payload = coverage.to_text();
let payload = payload.rsplit_once(' ').expect("a payload").1;
// The same payload, against a raster twice the size it covers.
assert!(Coverage::parse(&format!("8 4 2 {payload}")).is_some());
assert!(Coverage::parse(&format!("8 8 2 {payload}")).is_none());
}
#[test]
fn nonsense_is_refused_rather_than_guessed_at() {
assert!(Coverage::parse("").is_none());
assert!(Coverage::parse("8 4 2").is_none(), "no payload");
assert!(
Coverage::parse("8 4 1 AA").is_none(),
"one level is not a mask"
);
assert!(Coverage::parse("8 4 2 ****").is_none(), "not digits");
assert!(
Coverage::parse(&format!("{} {} 2 AA", usize::MAX, usize::MAX)).is_none(),
"a size that overflows must not be believed"
);
assert!(
Coverage::parse("100000 100000 2 A_____").is_none(),
"a raster past the cap must not be allocated"
);
}
/// A level a payload is not allowed to name, in a file that names it.
#[test]
fn a_level_outside_the_range_is_refused() {
// "BB" is level 1, run 1 — the shortest legal payload there is.
assert_eq!(decode("BB", 1, 2), Some(vec![255]));
// "DB" is level 3, run 1, and a two-level coverage has no level 3.
assert!(decode("DB", 1, 2).is_none());
// A run of zero says nothing and is refused rather than skipped.
assert!(decode("BA", 1, 2).is_none());
}
#[test]
fn resampling_lands_on_the_same_shape() {
let (w, h) = (32, 32);
let mut values = vec![0u8; w * h];
for y in 8..24 {
for x in 8..24 {
values[y * w + x] = 255;
}
}
let coverage = Coverage::encode(&values, w, h, RENDERED_LEVELS).expect("should encode");
let same = coverage.decode_at(w, h);
assert_eq!(same, values, "the matching size must not resample at all");
let half = coverage.decode_at(16, 16);
assert_eq!(half.len(), 256);
assert_eq!(half.iter().filter(|&&v| v == 255).count(), 64);
let double = coverage.decode_at(64, 64);
assert_eq!(double.len(), 4096);
assert_eq!(double.iter().filter(|&&v| v == 255).count(), 1024);
}
/// Long runs cross rows, which is what makes a flat mask cost almost
/// nothing: a 1600x1067 empty frame is two numbers.
#[test]
fn a_flat_mask_costs_almost_nothing() {
let values = vec![0u8; 1600 * 1067];
let coverage =
Coverage::encode(&values, 1600, 1067, RENDERED_LEVELS).expect("should encode");
assert!(
coverage.encoded_len() < 16,
"an empty mask took {} bytes",
coverage.encoded_len()
);
}
/// What a real one costs. The shape is a bilinear upsample of a coarse
/// grid, which is what both models produce, so the run structure is the
/// one a photograph actually gives.
#[test]
fn a_realistic_mask_fits_in_a_sidecar() {
let (w, h) = (1600usize, 1067usize);
let (gw, gh) = (160usize, 107usize);
let mut grid = vec![0f32; gw * gh];
for y in 0..gh {
for x in 0..gw {
let dx = (x as f32 - 80.0) / 26.0;
let dy = (y as f32 - 60.0) / 42.0;
let r = (dx * dx + dy * dy).sqrt()
+ 0.06 * ((y as f32 * 0.9).sin() * (x as f32 * 0.7).cos());
grid[y * gw + x] = 1.0 / (1.0 + ((r - 1.0) * 9.0).exp());
}
}
let mut values = vec![0u8; w * h];
for y in 0..h {
let fy = ((y as f32 + 0.5) / h as f32 * gh as f32 - 0.5).max(0.0);
let (y0, ty) = (fy.floor() as usize, fy.fract());
let y1 = (y0 + 1).min(gh - 1);
for x in 0..w {
let fx = ((x as f32 + 0.5) / w as f32 * gw as f32 - 0.5).max(0.0);
let (x0, tx) = (fx.floor() as usize, fx.fract());
let x1 = (x0 + 1).min(gw - 1);
let a = grid[y0 * gw + x0] * (1.0 - tx) + grid[y0 * gw + x1] * tx;
let b = grid[y1 * gw + x0] * (1.0 - tx) + grid[y1 * gw + x1] * tx;
values[y * w + x] = ((a * (1.0 - ty) + b * ty) * 255.0) as u8;
}
}
let coverage = Coverage::encode(&values, w, h, RENDERED_LEVELS).expect("should encode");
let expected: Vec<u8> = values
.iter()
.map(|&v| if v >= 128 { 255 } else { 0 })
.collect();
assert_eq!(
coverage.decode(),
expected,
"the stored mask must threshold identically to the model's"
);
// Measured at 6,464 bytes; the bound is loose enough not to fail over
// a change of rounding and tight enough to catch a coder that has
// stopped coding. Against 1,707,200 bytes raw.
assert!(
coverage.encoded_len() < 8 * 1024,
"a realistic subject took {} bytes",
coverage.encoded_len()
);
}
}
core/dr-pipeline/src/lib.rs
+2
View File
@@ -32,6 +32,7 @@
//! single multiply and white balance a per-channel scale; on gamma-encoded
//! data neither would be physically meaningful (ARCH §5.2).
pub mod coverage;
pub mod declared;
pub mod descriptor;
pub mod detail;
@@ -48,6 +49,7 @@ pub mod spot;
pub mod starter;
pub mod state;
pub use coverage::Coverage;
pub use declared::{Declaration, DeclaredOp};
pub use descriptor::{
Attribute, Facet, LocalizedKey, OpDescriptor, OpId, ParamDescriptor, ParamId, ParamKind,
core/dr-pipeline/src/mask.rs
+75 -7
View File
@@ -34,10 +34,27 @@
//! The cost is that the ids only mean anything alongside the segmentation that
//! produced them, so [`MaskSource::Regions::signature`] records which one —
//! see there for what happens when it does not match.
//!
//! # And the raster that had to come back anyway
//!
//! The same reasoning was applied to [`MaskSource::Subject`] and
//! [`MaskSource::Category`], and there it went one step too far. A model's
//! coverage is reproducible in principle, but only by running the model — and
//! nothing runs one except a photographer pressing a button. So a stored
//! subject layer resolved to no pixels on every path that did not have a run
//! already in memory: reopening the photograph, and exporting it from the
//! grid, which never runs one at all.
//!
//! [`MaskLayer::coverage`] is the answer, and note what it is *not*: the
//! source still stores identity, still diffs as a handful of numbers, and
//! still merges per field. The raster sits beside it as a cache, takes no part
//! in equality, and is thrown away rather than trusted when it does not fit.
//! See [`crate::coverage`].
use std::fmt::Write as _;
use std::sync::Arc;
use crate::coverage::Coverage;
use crate::descriptor::{OpDescriptor, ParamId};
use crate::operation::Operation;
use crate::ops;
@@ -527,10 +544,12 @@ pub enum MaskSource {
/// right and soft, and dilation, erosion and a chosen falloff are how it
/// is made to fit.
///
/// Stored as *identity*, not as pixels. The mask itself is several
/// megabytes and is reproducible by running the same model over the same
/// image, so the sidecar carries what is needed to find it again and the
/// session carries the pixels.
/// Stored as *identity*, not as pixels: the mask is several megabytes and
/// the fields below are what is needed to find it again. The pixels do go
/// in the sidecar as well, run-length coded beside the layer rather than
/// inside this variant, because "reproducible by running the model again"
/// turned out to mean "absent everywhere a model has not been run" — see
/// [`MaskLayer::coverage`].
Subject {
/// Which segmentation run produced it, so a layer can tell whether
/// the index below still means what it meant.
@@ -559,9 +578,8 @@ pub enum MaskSource {
/// before the mask ever existed, and [`Self::Subject`] is the source for
/// that question.
///
/// Stored as identity like a subject, and for the same reason: the
/// coverage is megabytes and is reproducible from the same model over the
/// same image.
/// Stored as identity like a subject, and the coverage travels beside it
/// for the same reason — see [`MaskLayer::coverage`].
Category {
/// Which segmentation run produced it, so a layer can tell whether
/// the name below still refers to something that was computed.
@@ -715,6 +733,41 @@ pub struct MaskLayer {
/// the same reason: two layers may sit on the same category and want
/// different amounts of it, and the model ran once for both.
pub refine: f32,
/// The pixels this layer covered, when a model produced them and they
/// were worth storing.
///
/// # Beside the source, not inside it
///
/// [`MaskSource::Subject`] and [`MaskSource::Category`] are *identity* —
/// which run, which instance, which category — and that is what makes them
/// diffable, small, and mergeable per field under FR-NC-9. Putting a
/// raster inside either variant would make two devices that selected the
/// same dog hold different values for the same selection, and the merge
/// would then have a binary blob to arbitrate rather than an index.
///
/// So this sits alongside as what it actually is: a **materialisation** of
/// the source, produced by a run of a model this crate has never heard of
/// and knows nothing about. `MaskSource` still says what the layer means;
/// this says what that meant last time anybody worked it out. The
/// distinction is why it takes no part in [`PartialEq`] — a layer with the
/// pixels cached and one without are the same edit, and a merge that
/// called them different would raise a conflict over a cache.
///
/// # Why it exists at all
///
/// Without it a stored subject or category layer renders as nothing until
/// somebody presses "find subjects" — so reopening a photograph dropped
/// its local adjustments, and a batch export, which never runs a model,
/// could not have them at any point. See [`crate::coverage`].
///
/// Shared rather than owned because the undo stack holds a snapshot per
/// step and a layer is cloned by value; an `Arc` makes recording a slider
/// drag cost a refcount rather than a copy of every mask in the stack.
/// Only ever set for the two model sources — nothing else has a model
/// behind it to cache.
pub coverage: Option<Arc<Coverage>>,
/// This layer's adjustments.
///
/// A full chain, the same one [`crate::EditGraph`] holds. That is the
@@ -772,6 +825,8 @@ impl Clone for MaskLayer {
morphology: self.morphology,
morph_radius: self.morph_radius,
refine: self.refine,
// A refcount, not a raster. See the field.
coverage: self.coverage.clone(),
ops,
}
}
@@ -789,12 +844,22 @@ impl std::fmt::Debug for MaskLayer {
.field("feather", &self.feather)
.field("falloff", &self.falloff)
.field("morphology", &self.morphology)
.field("coverage", &self.coverage.as_ref().map(|c| c.encoded_len()))
.field("active_ops", &self.active_ops().count())
.finish()
}
}
impl PartialEq for MaskLayer {
/// Every field that is the *edit*, and deliberately not
/// [`Self::coverage`].
///
/// This comparison is what [`crate::sidecar::Version::merge`] uses to
/// decide whether a device changed a layer (FR-NC-9). A cached raster is
/// not something a photographer changed: one device that has run the model
/// and one that has not hold the same edit, and counting the difference
/// would raise a conflict over a cache — and, with `remote_wins`, could
/// answer it by discarding the only copy of the pixels.
fn eq(&self, other: &Self) -> bool {
self.id == other.id
&& self.name == other.name
@@ -833,6 +898,9 @@ impl MaskLayer {
// `dr_segment`'s number to state and this crate does not depend on
// it — `Session::add_category_mask` sets it on the way in.
refine: 0.0,
// Nothing has run yet. Filled in the first time a model's coverage
// is turned into a distance field — see [`Self::coverage`].
coverage: None,
ops: layer_chain(),
}
}
core/dr-pipeline/src/sidecar.rs
+45
View File
@@ -67,6 +67,7 @@ use std::collections::BTreeMap;
use std::fmt;
use std::fmt::Write as _;
use crate::coverage::Coverage;
use crate::graph::EditGraph;
use crate::mask::{Falloff, MaskLayer, MaskSource, MaskStack, Morphology, Stroke, DEFAULT_FEATHER};
use crate::preset::Preset;
@@ -1043,6 +1044,25 @@ fn write_mask(out: &mut String, version: &str, layer: &MaskLayer) {
for (op, param, value) in layer.params() {
let _ = writeln!(out, "{op}.{param} = {}", format_value(value));
}
// TRACES: FR-DEV-3 | FR-CAT-8
// The pixels a model found, so that opening the photograph again — or
// exporting it from the grid, where no model is ever run — renders the
// layer instead of silently dropping it. See [`crate::coverage`] for the
// encoding and for why it is one line.
//
// **Last in the block, and that is on purpose.** It is thousands of
// characters against a dozen elsewhere, and a sidecar is read by hand when
// an edit has gone wrong (ARCH §6.12); everything a human is looking for
// should be above it rather than after it.
//
// Omitted, not truncated, when it will not encode: a layer with no stored
// coverage behaves exactly as every layer did before this existed, which
// is a mask that needs the model run — where a *partial* one would be a
// mask that is confidently wrong.
if let Some(coverage) = layer.coverage.as_ref() {
let _ = writeln!(out, "coverage = {}", coverage.to_text());
}
}
/// Write a brush layer's strokes, one line each.
@@ -1150,6 +1170,7 @@ struct PartialMask {
morphology: Morphology,
morph_radius: f32,
refine: f32,
coverage: Option<Coverage>,
strokes: Vec<Stroke>,
params: Vec<(String, String, f32)>,
}
@@ -1181,6 +1202,7 @@ impl PartialMask {
morphology: Morphology::default(),
morph_radius: 0.0,
refine: 0.0,
coverage: None,
strokes: Vec::new(),
params: Vec::new(),
}
@@ -1217,6 +1239,19 @@ impl PartialMask {
"angle" => self.angle = value.parse().unwrap_or(0.0),
"width" => self.width = value.parse().unwrap_or(0.0),
"feather" => self.feather = value.parse().unwrap_or(0.0),
// A cache, so an unreadable one is dropped rather than refused:
// the layer still says what it selects, and the worst a `None`
// here costs is a model run. Refusing the layer over it would
// throw away an edit to protect a copy of something reproducible.
"coverage" => {
self.coverage = Coverage::parse(value);
if self.coverage.is_none() {
log::warn!(
"sidecar: mask {} has unreadable coverage; it will need the model run",
self.id
);
}
}
// Appended rather than assigned: a brush layer is a list of these,
// and the file's line order is the order they were painted in.
"stroke" => self.strokes.extend(parse_stroke(value)),
@@ -1321,6 +1356,16 @@ impl PartialMask {
layer.morphology = self.morphology;
layer.morph_radius = self.morph_radius;
layer.refine = self.refine;
// Only where there is a model behind the layer to have produced it. A
// gradient or a brush that arrived carrying one is a file that has
// been hand-edited or written by a build that means something else by
// the key, and honouring it would upload a raster nothing samples.
layer.coverage = match layer.source {
MaskSource::Subject { .. } | MaskSource::Category { .. } => {
self.coverage.map(std::sync::Arc::new)
}
_ => None,
};
for (op, param, value) in &self.params {
// `ParamId` holds a `&'static str` and this one came off disk, so
// it is matched against the descriptors and the *static* id is
core/dr-pipeline/tests/mask_sidecar.rs
+346
View File
@@ -778,3 +778,349 @@ fn a_category_from_another_run_is_stale() {
assert!(layer.is_stale(2), "a different run must invalidate it");
assert!(!layer.is_stale(1), "the run it was built against must not");
}
// ---------------------------------------------------------------------------
// Stored coverage (dr_pipeline::coverage)
// ---------------------------------------------------------------------------
//
// A subject or a category is stored as *identity* — which run, which instance,
// which category — and identity alone is only enough while the run is still in
// memory. These pin down the raster that goes beside it, which is what lets a
// reopened photograph and a batch export render the layer without a model.
use dr_pipeline::coverage::{Coverage, RENDERED_LEVELS};
use std::sync::Arc;
const PROXY: (usize, usize) = (96, 64);
fn subject(signature: u64, index: u32) -> MaskSource {
MaskSource::Subject {
signature,
index,
class: "dog".into(),
score: 0.94,
}
}
/// A soft-edged disc, which is the shape a model actually hands out: a coarse
/// sigmoid with a shoulder several pixels wide.
fn a_disc() -> Vec<u8> {
let (w, h) = PROXY;
let mut values = vec![0u8; w * h];
for y in 0..h {
for x in 0..w {
let dx = (x as f32 - 40.0) / 20.0;
let dy = (y as f32 - 30.0) / 20.0;
let r = (dx * dx + dy * dy).sqrt();
values[y * w + x] = ((1.0 / (1.0 + ((r - 1.0) * 4.0).exp())) * 255.0) as u8;
}
}
values
}
/// What the renderer would make of a coverage: the distance transform reads
/// `>= 128` and nothing else, so this is the whole of the information a stored
/// mask has to preserve.
fn inside(values: &[u8]) -> Vec<bool> {
values.iter().map(|&v| v >= 128).collect()
}
fn with_coverage(id: &str, source: MaskSource, values: &[u8]) -> MaskLayer {
let mut layer = MaskLayer::new(id, source);
layer.set_param("exposure", ParamId("exposure"), 0.75);
layer.coverage = Some(Arc::new(
Coverage::encode(values, PROXY.0, PROXY.1, RENDERED_LEVELS).expect("a disc should encode"),
));
layer
}
/// The bug this whole thing exists for: without the raster, reopening the
/// photograph gave the layer nothing to be, and the local adjustment was
/// silently absent until somebody pressed "find subjects".
#[test]
fn a_subjects_coverage_survives_a_round_trip() {
let values = a_disc();
let mut graph = EditGraph::default_chain();
graph
.masks_mut()
.push(with_coverage("m1", subject(0x1234, 2), &values));
let restored = round_trip(&graph);
let layer = &restored.masks().layers()[0];
assert_eq!(
layer.source,
subject(0x1234, 2),
"the identity is still there"
);
let coverage = layer.coverage.as_ref().expect("the pixels came back too");
assert_eq!((coverage.width(), coverage.height()), PROXY);
assert_eq!(
inside(&coverage.decode()),
inside(&values),
"every pixel must fall on the same side of the threshold"
);
}
#[test]
fn a_categorys_coverage_survives_a_round_trip() {
let values = a_disc();
let source = MaskSource::Category {
signature: 0x5678,
name: "sky".into(),
};
let mut graph = EditGraph::default_chain();
graph
.masks_mut()
.push(with_coverage("m1", source.clone(), &values));
let restored = round_trip(&graph);
let layer = &restored.masks().layers()[0];
assert_eq!(layer.source, source);
assert_eq!(
inside(&layer.coverage.as_ref().expect("coverage").decode()),
inside(&values)
);
}
/// The property that lets a caller skip an upload by comparing content — now
/// with several kilobytes of run-length payload in the file. An encoder that
/// re-coded the same raster differently on the way out would put a spurious
/// upload on every save.
#[test]
fn a_stored_coverage_writes_the_same_bytes_every_time() {
let mut graph = EditGraph::default_chain();
graph
.masks_mut()
.push(with_coverage("m1", subject(1, 0), &a_disc()));
let mut sidecar = Sidecar::new();
sidecar.put(Version::from_graph("default", "Default", &graph));
let once = sidecar.to_text();
let twice = Sidecar::parse(&once).expect("reparse").to_text();
assert_eq!(once, twice);
}
/// The line goes last in its block, and that is a claim about reading the file
/// by hand: it is thousands of characters against a dozen everywhere else, and
/// a sidecar is what somebody opens when an edit has gone wrong (ARCH §6.12).
#[test]
fn the_coverage_line_comes_after_everything_a_human_is_looking_for() {
let mut graph = EditGraph::default_chain();
graph
.masks_mut()
.push(with_coverage("m1", subject(1, 0), &a_disc()));
let mut sidecar = Sidecar::new();
sidecar.put(Version::from_graph("default", "Default", &graph));
let text = sidecar.to_text();
let block = text.split("[mask ").nth(1).expect("a mask block");
let keys: Vec<&str> = block
.lines()
.filter_map(|l| l.split_once(" = "))
.map(|(k, _)| k)
.collect();
assert_eq!(
keys.last(),
Some(&"coverage"),
"coverage should be the last key in the block: {keys:?}"
);
assert!(
keys.contains(&"class") && keys.contains(&"exposure.exposure"),
"and everything else should still be above it: {keys:?}"
);
assert_eq!(
block.lines().filter(|l| l.starts_with("coverage")).count(),
1,
"one line, because a node is a line and the merge is key-wise"
);
}
/// Version skew, backwards: a file written before any of this existed.
///
/// The layer must load and behave exactly as it did then — which is to say it
/// needs the model run — rather than reading as a mask with no pixels.
#[test]
fn a_sidecar_written_before_coverage_existed_still_loads() {
let text = "\
drsc 1
[version default]
name = Default
default = 1
revision = 3
[mask default m1]
name = Dog
source = subject
signature = 4660
index = 2
class = dog
score = 0.94
exposure.exposure = 0.75
";
let sidecar = Sidecar::parse(text).expect("an old file must still parse");
let mut graph = EditGraph::default_chain();
sidecar
.versions
.get("default")
.expect("version")
.apply(&mut graph)
.expect_no_film();
let layer = &graph.masks().layers()[0];
assert_eq!(layer.source, subject(4660, 2));
assert_eq!(layer.name, "Dog");
assert!(
layer.coverage.is_none(),
"absent means absent, not an empty mask"
);
}
/// Version skew, forwards: a coverage this build cannot make sense of.
///
/// The stand-in for a payload written by a build that means something else by
/// the key. It costs the pixels — a model run — and must not cost the layer,
/// because the layer is the edit and the raster is a cache of it.
#[test]
fn an_unreadable_coverage_costs_the_pixels_and_not_the_layer() {
let text = "\
drsc 1
[version default]
name = Default
default = 1
[mask default m1]
name = Dog
source = subject
signature = 4660
index = 2
class = dog
score = 0.94
exposure.exposure = 0.75
coverage = 96 64 999 not-a-payload
";
let sidecar = Sidecar::parse(text).expect("parse");
let mut graph = EditGraph::default_chain();
sidecar
.versions
.get("default")
.expect("version")
.apply(&mut graph)
.expect_no_film();
let layer = &graph.masks().layers()[0];
assert_eq!(layer.source, subject(4660, 2));
assert_eq!(layer.name, "Dog");
assert!(layer.coverage.is_none());
assert_eq!(
layer
.ops
.iter()
.find(|o| o.descriptor().id.0 == "exposure")
.map(|o| o.param(ParamId("exposure"))),
Some(0.75),
"the adjustment is the thing that must not be lost"
);
}
/// A raster only means anything against a source a model produced. A gradient
/// carrying one is a hand-edited or mis-written file, and honouring it would
/// upload a buffer nothing samples.
#[test]
fn coverage_is_not_loaded_onto_a_source_with_no_model_behind_it() {
let payload = Coverage::encode(&a_disc(), PROXY.0, PROXY.1, RENDERED_LEVELS)
.expect("encode")
.to_text();
let text = format!(
"drsc 1\n\n[version default]\nname = Default\ndefault = 1\n\n\
[mask default m1]\nsource = radial\ncentre = 0.5 0.5\nradii = 0.25 0.25\n\
coverage = {payload}\n"
);
let sidecar = Sidecar::parse(&text).expect("parse");
let mut graph = EditGraph::default_chain();
sidecar
.versions
.get("default")
.expect("version")
.apply(&mut graph)
.expect_no_film();
let layer = &graph.masks().layers()[0];
assert!(matches!(layer.source, MaskSource::Radial { .. }));
assert!(layer.coverage.is_none());
}
/// TRACES: FR-NC-9
/// One device has run the model and the other has not. That is the same edit.
///
/// The merge decides "did this device change the layer" by comparing layers,
/// so a cached raster taking part would make a photograph opened on the phone
/// conflict with itself on the desktop — and, with `remote_wins`, resolve the
/// conflict by discarding the only copy of the pixels.
#[test]
fn a_coverage_one_device_has_and_the_other_lacks_is_not_a_conflict() {
let layer = |with: bool| {
let mut l = lit("m1", &[1], 1.0);
l.source = subject(7, 0);
if with {
l.coverage = Some(Arc::new(
Coverage::encode(&a_disc(), PROXY.0, PROXY.1, RENDERED_LEVELS).expect("encode"),
));
}
l
};
let base = version_with("default", 1, |g| {
g.masks_mut().push(layer(false));
});
let mut ours = version_with("default", 2, |g| {
g.masks_mut().push(layer(true));
});
let theirs = version_with("default", 9, |g| {
g.masks_mut().push(layer(false));
});
let conflicts = ours.merge(&theirs, Some(&base));
assert!(
conflicts.is_empty(),
"running the model is not an edit: {conflicts:?}"
);
assert!(
ours.masks.get("m1").expect("layer").coverage.is_some(),
"and the side that has the pixels keeps them"
);
}
/// The other half of the same claim: a real edit is still a conflict when both
/// sides also happen to hold coverage.
#[test]
fn a_real_edit_is_still_a_conflict_with_coverage_present() {
let stored = || {
Some(Arc::new(
Coverage::encode(&a_disc(), PROXY.0, PROXY.1, RENDERED_LEVELS).expect("encode"),
))
};
let layer = |ev: f32| {
let mut l = lit("m1", &[1], ev);
l.source = subject(7, 0);
l.coverage = stored();
l
};
let base = version_with("default", 1, |g| {
g.masks_mut().push(layer(0.5));
});
let mut ours = version_with("default", 2, |g| {
g.masks_mut().push(layer(1.0));
});
let theirs = version_with("default", 9, |g| {
g.masks_mut().push(layer(-1.0));
});
assert_eq!(
ours.merge(&theirs, Some(&base)),
vec![("mask".to_string(), "m1".to_string())]
);
}
docs/traceability.md
+35 -35
View File
File diff suppressed because one or more lines are too long
ui/dr-ui/src/develop.rs
+508 -38
View File
@@ -9,6 +9,7 @@
//! declared [`ParamKind`], not from which parameter it is (ARCH §4.3), so a
//! new operation appears in the panel with no change here (FR-DEV-3c).
use std::borrow::Cow;
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
use std::sync::Arc;
@@ -1868,63 +1869,172 @@ impl DevelopSession {
h
}
/// Rebuild the distance fields if anything they depend on moved.
fn ensure_subject_fields(&mut self, ctx: &GpuContext) {
/// One layer's coverage: what the model says now, or what the sidecar
/// remembered it saying.
///
/// **The model first, always.** It is the live answer, it is the only one
/// that can respond to the refine control, and a run in this sitting is by
/// definition newer than anything a file was holding.
///
/// The fallback is the point of the stored raster. A photograph reopened,
/// and a batch export from the grid — which opens a session, applies a
/// version and never runs a model at all — have no segmentation to ask, so
/// before this they resolved every subject and category layer to nothing
/// and wrote out a file missing the local adjustments, with a line in the
/// log as the only sign. See [`dr_pipeline::coverage`].
///
/// `None` for every other source: a gradient and a brush are rasterised
/// from their own geometry and have no coverage to fetch, and the caller
/// gives them a placeholder field so the slot indices still line up.
fn layer_coverage<'a>(
&'a self,
layer: &'a dr_pipeline::mask::MaskLayer,
width: usize,
height: usize,
) -> Option<Cow<'a, [u8]>> {
use dr_pipeline::mask::MaskSource;
let live = self
.segmentation
.as_ref()
.and_then(|seg| match &layer.source {
MaskSource::Category { name, .. } => seg.category_mask_at(name, layer.refine),
MaskSource::Subject { index, .. } => {
seg.instance_mask(*index as usize).map(Cow::Borrowed)
}
_ => None,
});
if live.is_some() {
return live;
}
match layer.source {
// Resampled where it was written against a different proxy edge;
// in the ordinary case the sizes match and this is the decode.
MaskSource::Subject { .. } | MaskSource::Category { .. } => layer
.coverage
.as_ref()
.map(|stored| Cow::Owned(stored.decode_at(width, height))),
_ => None,
}
}
/// TRACES: FR-CAT-8 | FR-DEV-3
/// The mask stack as it should be written to a sidecar.
///
/// The stack the graph holds, with each model layer's coverage brought up
/// to what the segmentation now says it is. That raster is what lets the
/// *next* opening of this photograph render the layer without a model run
/// — the whole of [`dr_pipeline::coverage`]'s reason to exist.
///
/// # Why here, and not where the field is built
///
/// `ensure_subject_fields` is the tempting place: it is the one funnel
/// every coverage passes through, so recording it there would catch every
/// route automatically. But it runs on a *drag* — dilating a mask with a
/// compound morphology rebuilds the field every frame — and encoding a
/// megapixel raster per frame is exactly the kind of work NFR-P5 is about.
///
/// Saving happens when the photograph stops being the open one, once, and
/// already costs a network round trip. So the encoding is done here, where
/// nothing is waiting on it, and the session's own rendering goes on
/// reading the model directly.
///
/// Returns the stack by value rather than mutating: the caller is
/// serialising, not editing, and a graph that quietly gained a field on
/// the way past would be a mutation nobody asked for and undo would not
/// know about.
pub fn masks_for_storage(&self) -> dr_pipeline::mask::MaskStack {
use dr_pipeline::coverage::{Coverage, RENDERED_LEVELS};
use dr_pipeline::mask::MaskSource;
let mut stack = self.graph.masks().clone();
let Some(seg) = self.segmentation.as_ref() else {
// No model has run this sitting, so whatever the layers arrived
// holding is still the best answer anyone has. Handing the stack
// back untouched is what stops a photograph that was opened,
// glanced at and closed from losing the coverage its own sidecar
// gave it.
return stack;
};
let (pw, ph) = seg.proxy_size();
for layer in stack.layers_mut() {
let values = match &layer.source {
MaskSource::Category { name, .. } => seg.category_mask_at(name, layer.refine),
MaskSource::Subject { index, .. } => {
seg.instance_mask(*index as usize).map(Cow::Borrowed)
}
// Nothing else has a model behind it. Left alone rather than
// cleared, so a hand-written file's key survives a round trip
// even though nothing samples it.
_ => continue,
};
// The layer names something this run does not contain — a stale
// index, a category the scene model no longer offers. Keeping what
// was stored is right: it is a mask that was once correct, and the
// panel is already telling the user the layer is stale.
let Some(values) = values else { continue };
// `None` from the encoder means "will not fit in a sidecar", and
// the stored raster is then cleared rather than left standing. It
// would describe the layer at some earlier refine, and a mask of
// the wrong shape presented as authoritative is worse than the
// honest state, which is that this one needs the model run.
layer.coverage = Coverage::encode(&values, pw, ph, RENDERED_LEVELS).map(Arc::new);
}
stack
}
/// Rebuild the distance fields if anything they depend on moved.
fn ensure_subject_fields(&mut self, ctx: &GpuContext) {
let key = self.subject_signature();
if key == self.subject_key && self.subjects.is_some() {
return;
}
let Some(seg) = self.segmentation.as_ref() else {
return;
// The proxy the fields are measured in: the segmentation's own where
// one has been run, and otherwise the size the mask array is
// rasterised at. `mask_raster_size` derives that from the photograph
// rather than from a segmentation for exactly this case, and the two
// are the same number by construction — see there.
let (pw, ph) = match self.segmentation.as_ref() {
Some(seg) => seg.proxy_size(),
None => {
let (w, h) = self.mask_raster_size();
(w as usize, h as usize)
}
};
let (pw, ph) = seg.proxy_size();
// In `active()` order, because that is the order the rasteriser walks
// and the order it indexes these by.
let mut fields: Vec<Vec<f32>> = Vec::new();
for layer in self.graph.masks().active() {
let field = match &layer.source {
MaskSource::Category { name, .. } => seg
.category_mask_at(name, layer.refine)
.map(|coverage| {
let field = match self.layer_coverage(layer, pw, ph) {
Some(coverage) => {
dr_segment::Shaped::build(
&coverage,
pw,
ph,
128,
morphology_for(layer.morphology),
// Radii are fractions of the shorter edge; the field
// is in proxy pixels.
layer.morph_radius * pw.min(ph) as f32,
)
.distance
})
}
// Full size, never `unwrap_or_default`: an empty vec is a
// wrong-sized field, `SubjectMasks::upload` rejects the
// whole batch on one, and every other layer in the stack
// then loses its mask too. One stale name should cost one
// layer, not all of them.
.unwrap_or_else(|| vec![-1.0; pw * ph]),
MaskSource::Subject { index, .. } => seg
.instance_mask(*index as usize)
.map(|coverage| {
dr_segment::Shaped::build(
coverage,
pw,
ph,
128,
morphology_for(layer.morphology),
// Radii are fractions of the shorter edge; the
// field is in proxy pixels.
layer.morph_radius * pw.min(ph) as f32,
)
.distance
})
.unwrap_or_else(|| vec![-1.0; pw * ph]),
// A placeholder of the right size, so the slot indices line up
// with `active()` whatever mix of sources the stack holds.
_ => vec![-1.0; pw * ph],
// wrong-sized field, `SubjectMasks::upload` rejects the whole
// batch on one, and every other layer in the stack then loses
// its mask too. One stale name should cost one layer, not all
// of them.
//
// It is also the placeholder a gradient or a brush gets, so
// the slot indices line up with `active()` whatever mix of
// sources the stack holds.
None => vec![-1.0; pw * ph],
};
fields.push(field);
}
@@ -2978,10 +3088,11 @@ impl DevelopSession {
};
match self.segmentation.as_ref() {
Some(seg) => layer.is_stale(seg.signature()),
// Nothing loaded to compare against. Not stale, just unrenderable
// — the distinction matters because "stale" invites the user to
// recompute the selection and this only needs the segmentation
// running.
// Nothing loaded to compare against, and nothing to report: the
// layer renders from the raster its sidecar stored (see
// `layer_coverage`). It was already not *stale* before that — the
// word invites the user to recompute a selection that is fine —
// and now it is not unrenderable either.
None => false,
}
}
@@ -4757,6 +4868,365 @@ mod tests {
);
}
// ----------------------------------------------------------------------
// Stored coverage (dr_pipeline::coverage)
// ----------------------------------------------------------------------
//
// A subject or category layer is stored in the sidecar as identity — which
// run, which instance, which category — and identity resolves to pixels
// only while that run is in memory. So reopening an edited photograph
// dropped every model-backed local adjustment, and a batch export, which
// never runs a model at all, could not have them at any point. Both
// failures were silent: the shader still emitted the layer's block, the
// placeholder multiplied it by zero, and the result was a well-formed
// frame with the adjustment simply absent.
/// A stored edit holding one subject layer over the left half of the
/// frame, as a session that *had* run the model would have written it.
///
/// `stored` is the whole variable: with the raster, this is a sidecar
/// written by a build that persists coverage; without it, one written
/// before that existed. Everything else about the two is identical, which
/// is what makes the pair of tests below a measurement rather than an
/// assertion about two different edits.
fn version_with_a_subject(proxy: (u32, u32), stored: bool) -> dr_pipeline::Version {
use dr_pipeline::coverage::{Coverage, RENDERED_LEVELS};
let (pw, ph) = (proxy.0 as usize, proxy.1 as usize);
let mut values = vec![0u8; pw * ph];
for y in 0..ph {
for x in 0..pw / 2 {
values[y * pw + x] = 255;
}
}
let mut layer = MaskLayer::new(
"m1",
MaskSource::Subject {
signature: 0xfeed,
index: 0,
class: "dog".into(),
score: 0.9,
},
);
layer.set_param("exposure", ParamId("exposure"), 2.0);
if stored {
layer.coverage = Some(Arc::new(
Coverage::encode(&values, pw, ph, RENDERED_LEVELS).expect("a half frame encodes"),
));
}
let mut graph = EditGraph::default_chain();
graph.masks_mut().push(layer);
dr_pipeline::Version::from_graph("default", "Default", &graph)
}
/// A flat grey session with nothing segmented, and its render.
fn grey_session(ctx: &GpuContext) -> (DevelopSession, Vec<u8>) {
let rgba: Vec<u8> = (0..64 * 64).flat_map(|_| [128u8, 128, 128, 255]).collect();
let mut session =
DevelopSession::open_rgb(ctx, &rgba, 64, 64, dr_types::Orientation::NORMAL)
.expect("session");
assert!(
!session.has_segmentation(),
"the premise: no model has been run"
);
let before = read_back(ctx, &session.render(64, 64).expect("render"));
(session, before)
}
/// TRACES: FR-DEV-3 | FR-CAT-8
/// Reopening an edited photograph renders its subject mask, with no model.
///
/// This is the failure the stored raster exists for. `apply_version` is
/// exactly what opening a photograph from the library does with the
/// sidecar it fetched, and until the coverage went into the file the layer
/// resolved to nothing every time.
#[test]
fn a_stored_subject_mask_renders_with_no_model_run() {
let Some(ctx) = headless() else { return };
let (mut session, before) = grey_session(&ctx);
let proxy = session.mask_raster_size();
session.apply_version(&version_with_a_subject(proxy, true));
let after = read_back(&ctx, &session.render(64, 64).expect("render"));
let at = |px: &[u8], x: usize, y: usize| px[(y * 64 + x) * 4];
assert!(
at(&after, 16, 32) > at(&before, 16, 32) + 20,
"the covered half must brighten: {} against {}",
at(&after, 16, 32),
at(&before, 16, 32)
);
// And only that half. A mask that failed the other way — covering
// everything — would pass the assertion above and is the louder bug.
assert_eq!(
at(&after, 56, 32),
at(&before, 56, 32),
"the uncovered half must not move"
);
}
/// The control for the test above: the same edit with the raster left out
/// is the behaviour every build had before this, which is no mask at all.
///
/// Worth pinning down in both directions. It is what a sidecar written by
/// an older build looks like, and reading one must go on being harmless —
/// the layer needs the model run, exactly as it always did, rather than
/// rendering as an empty mask or as the whole frame.
#[test]
fn a_subject_mask_with_no_stored_coverage_still_needs_the_model() {
let Some(ctx) = headless() else { return };
let (mut session, before) = grey_session(&ctx);
let proxy = session.mask_raster_size();
session.apply_version(&version_with_a_subject(proxy, false));
let after = read_back(&ctx, &session.render(64, 64).expect("render"));
assert_eq!(
after, before,
"with no coverage the layer contributes nothing"
);
}
/// TRACES: FR-EXP-9 | FR-CAT-8
/// A batch export renders the stored mask too.
///
/// `export::render_from_library` fetches the RAW and the sidecar, opens a
/// session, applies the version and calls `render_for_export` — and it
/// runs no model on the way, so before the coverage was stored a batch
/// wrote out files with the photographer's local adjustments missing, over
/// a log warning nobody was reading. These two lines are that path with
/// the fetch taken out.
#[test]
fn an_export_renders_a_stored_subject_mask() {
let Some(ctx) = headless() else { return };
let rgba: Vec<u8> = (0..64 * 64).flat_map(|_| [128u8, 128, 128, 255]).collect();
let mut session =
DevelopSession::open_rgb(&ctx, &rgba, 64, 64, dr_types::Orientation::NORMAL)
.expect("session");
let plain = session
.render_for_export(dr_types::ColourSpace::Srgb)
.expect("export");
let proxy = session.mask_raster_size();
session.apply_version(&version_with_a_subject(proxy, true));
let masked = session
.render_for_export(dr_types::ColourSpace::Srgb)
.expect("export");
let at = |f: &dr_export::Frame, x: usize, y: usize| f.rgba[(y * 64 + x) * 4];
assert!(
at(&masked, 16, 32) > at(&plain, 16, 32) + 20,
"the exported file must carry the local adjustment: {} against {}",
at(&masked, 16, 32),
at(&plain, 16, 32)
);
assert_eq!(
at(&masked, 56, 32),
at(&plain, 56, 32),
"and only where the mask covers"
);
}
/// What a session hands the sidecar writer when no model has run.
///
/// Untouched, and that is the whole of it: a photograph opened, looked at
/// and closed must not have the coverage its own sidecar gave it stripped
/// out on the way past. Saving is automatic, so a stack that lost the
/// raster here would lose it on disk within seconds of being opened.
#[test]
fn saving_without_a_model_keeps_the_coverage_that_was_loaded() {
let Some(ctx) = headless() else { return };
let (mut session, _) = grey_session(&ctx);
let proxy = session.mask_raster_size();
session.apply_version(&version_with_a_subject(proxy, true));
let stored = session.masks_for_storage();
assert_eq!(stored.len(), 1);
assert!(
stored.layers()[0].coverage.is_some(),
"the raster the sidecar gave us must go back to the sidecar"
);
}
/// A session holding a segmentation with one instance over the left half.
///
/// Built rather than detected. What these tests need is coverage of a
/// *known* shape, so that "the stored raster renders what the model's did"
/// is a comparison rather than a hope — and asking a real run what it
/// happened to find in a synthetic frame would make the assertion depend
/// on the weights. `segmented_session` above is the one that runs a model,
/// and it goes on doing so.
fn session_with_a_left_half_subject(ctx: &GpuContext) -> DevelopSession {
let rgba: Vec<u8> = (0..64 * 64).flat_map(|_| [128u8, 128, 128, 255]).collect();
let mut session =
DevelopSession::open_rgb(ctx, &rgba, 64, 64, dr_types::Orientation::NORMAL)
.expect("session");
let (pw, ph) = session.mask_raster_size();
let (pw, ph) = (pw as usize, ph as usize);
let mut mask = vec![0u8; pw * ph];
for y in 0..ph {
for x in 0..pw / 2 {
mask[y * pw + x] = 255;
}
}
session.segmentation = Some(segmentation::Segmentation::for_test(
vec![segmentation::InstanceSummary {
class_name: "dog".into(),
score: 0.9,
mask,
bbox: (0.0, 0.0, (pw / 2) as f32, ph as f32),
}],
Vec::new(),
0xfeed,
(pw, ph),
));
session.subjects = None;
session.subject_key = 0;
session
}
/// TRACES: FR-DEV-3 | FR-CAT-8
/// The whole claim, end to end: what is stored renders what was rendered.
///
/// A session with a model's coverage in hand draws the mask; the stack it
/// hands the sidecar writer goes through the file and into a session with
/// no model at all; and the two frames must be the same. Anything weaker
/// — that the coverage is present, that it round-trips as bytes — would
/// still pass if the raster came back at the wrong scale, upside down, or
/// a threshold out.
#[test]
fn a_stored_mask_renders_exactly_what_the_model_rendered() {
let Some(ctx) = headless() else { return };
let mut live = session_with_a_left_half_subject(&ctx);
let id = live.add_subject_mask(0).expect("a subject layer");
live.graph
.masks_mut()
.get_mut(&id)
.expect("the layer")
.set_param("exposure", ParamId("exposure"), 2.0);
let with_model = read_back(&ctx, &live.render(64, 64).expect("render"));
// Through the sidecar, the way `presets::save_open_edit` does it: the
// graph's own parameters, with the stack that carries the coverage
// substituted for the one the graph is holding.
let mut version = dr_pipeline::Version::from_graph("default", "Default", &live.graph);
version.masks = live.masks_for_storage();
let mut sidecar = dr_pipeline::Sidecar::new();
sidecar.put(version);
let text = sidecar.to_text();
let parsed = dr_pipeline::Sidecar::parse(&text).expect("reparse");
let rgba: Vec<u8> = (0..64 * 64).flat_map(|_| [128u8, 128, 128, 255]).collect();
let mut reopened =
DevelopSession::open_rgb(&ctx, &rgba, 64, 64, dr_types::Orientation::NORMAL)
.expect("session");
assert!(
!reopened.has_segmentation(),
"the point: nothing has run a model in this session"
);
reopened.apply_version(parsed.default_version().expect("a version"));
let from_the_file = read_back(&ctx, &reopened.render(64, 64).expect("render"));
assert_eq!(
from_the_file, with_model,
"the reopened frame must be the frame the model produced"
);
// And that both are actually a mask rather than both being nothing.
let at = |px: &[u8], x: usize| px[(32 * 64 + x) * 4];
assert!(
at(&with_model, 16) > at(&with_model, 56) + 20,
"the premise: the live render really is masked ({} against {})",
at(&with_model, 16),
at(&with_model, 56)
);
}
/// And what a session hands over when a model *has* run: the coverage
/// folded in, at the proxy the distance field is measured in.
///
/// The encoding happens here rather than where the field is built because
/// that runs on a drag — see `masks_for_storage`.
#[test]
fn saving_after_a_model_run_folds_the_coverage_in() {
let Some(ctx) = headless() else { return };
let mut session = session_with_a_left_half_subject(&ctx);
let id = session.add_subject_mask(0).expect("a subject layer");
let stored = session.masks_for_storage();
let coverage = stored
.get(&id)
.expect("the layer is in the stack")
.coverage
.as_ref()
.expect("a subject the model found has coverage to store");
let (pw, ph) = session.mask_raster_size();
assert_eq!(
(coverage.width(), coverage.height()),
(pw as usize, ph as usize)
);
let decoded = coverage.decode();
assert_eq!(decoded[32 * pw as usize + 8], 255, "inside the subject");
assert_eq!(decoded[32 * pw as usize + 56], 0, "outside it");
assert!(
coverage.encoded_len() < 512,
"a half-frame mask is a handful of runs, not {} bytes",
coverage.encoded_len()
);
}
/// A layer whose coverage came from the file must still take the model's
/// once a model runs — the live answer is the only one that responds to
/// the refine control, and a run in this sitting is newer than a file.
#[test]
fn a_model_run_takes_precedence_over_what_was_stored() {
let Some(ctx) = headless() else { return };
let mut session = session_with_a_left_half_subject(&ctx);
// Stored coverage saying the *right* half, against a segmentation
// saying the left.
let (pw, ph) = session.mask_raster_size();
let (pw, ph) = (pw as usize, ph as usize);
let mut mirrored = vec![0u8; pw * ph];
for y in 0..ph {
for x in pw / 2..pw {
mirrored[y * pw + x] = 255;
}
}
let id = session.add_subject_mask(0).expect("a subject layer");
{
let layer = session.graph.masks_mut().get_mut(&id).expect("the layer");
layer.set_param("exposure", ParamId("exposure"), 2.0);
layer.coverage = Some(Arc::new(
dr_pipeline::Coverage::encode(
&mirrored,
pw,
ph,
dr_pipeline::coverage::RENDERED_LEVELS,
)
.expect("encode"),
));
}
let frame = read_back(&ctx, &session.render(64, 64).expect("render"));
let at = |x: usize| frame[(32 * 64 + x) * 4];
assert!(
at(16) > at(56) + 20,
"the model's left half must win over the file's right half: \
{} against {}",
at(16),
at(56)
);
}
/// TRACES: FR-DEV-3
/// Multi-select: one slider, applied to every selected layer.
///
ui/dr-ui/src/presets.rs
+8 -1
View File
@@ -391,7 +391,14 @@ pub fn save_open_edit(
let Some((preset, masks, film)) = session.borrow().as_ref().map(|s| {
(
s.copy_settings(),
s.masks().clone(),
// TRACES: FR-DEV-3
// Not `masks().clone()`: this is the stack *with the model's
// coverage folded in*, so a subject or category layer survives
// being closed and reopened. Without it the sidecar stored the
// layer's identity and nothing else, and the next session — or a
// batch export, which never runs a model — rendered the
// photograph with the local adjustment silently missing.
s.masks_for_storage(),
// TRACES: FR-DEV-3f
// Taken in the same borrow as the other two, for the reason the
// note above gives: they are one edit, and a stock read from a
ui/dr-ui/src/segmentation.rs
+25
View File
@@ -113,6 +113,31 @@ pub struct Segmentation {
}
impl Segmentation {
/// TRACES: FR-DEV-3
/// A segmentation assembled by hand, for tests that need one without a
/// model.
///
/// Test-only, and it exists because the alternative is worse: the tests
/// that need coverage to be *a particular shape* — that a stored raster
/// renders the same pixels as the model's own did — cannot ask a real run
/// for a shape, and running the model to find out what it happened to
/// detect in a synthetic frame makes the assertion depend on the weights.
/// Every test that is genuinely about the model still runs it.
#[cfg(test)]
pub(crate) fn for_test(
instances: Vec<InstanceSummary>,
categories: Vec<CategorySummary>,
signature: u64,
proxy: (usize, usize),
) -> Self {
Self {
instances,
categories,
signature,
proxy,
}
}
pub fn signature(&self) -> u64 {
self.signature
}