Store what the model found, so a reopened photograph keeps its masks
A subject or category layer was written to the sidecar as identity alone —
which run, which instance, which category — on the reasoning that the pixels
are reproducible by running the same model over the same image. They are, but
only by *running the model*, and nothing runs one except a photographer
pressing "find subjects". So on every path that did not already have a run in
memory the layer resolved to no coverage, `MaskPass::render` logged "has no
distance field; skipping", and the adjustment was silently absent:
- reopening an edited photograph rendered it without its local adjustments,
and then saved that state back on the way out;
- a batch export from the grid could not have them at any point, because
`render_from_library` opens a session, applies a version and renders, and
there is no model anywhere on that path. Three hundred files written
without the edits their photographer made, over a log warning.
Neither failure announced itself. The generated shader still emits the layer's
block and the empty placeholder multiplies it by zero, so the result is a
well-formed frame that is simply missing an edit — `mask_is_stale` already
named the state and called it "not stale, just unrenderable".
The coverage now travels in the file, as one `coverage = w h levels payload`
line at the end of the layer's block.
Two levels, and that is not a compromise. The model hands out a byte per pixel
but `Shaped::build` measures its distance field from `coverage >= 128` and
throws the shoulder away on the first line; everything soft about the rendered
edge comes afterwards from the layer's feather and falloff, which are read off
the distance. So one bit per pixel is not an approximation of what the model
said — it is exactly the part of it that reaches a pixel, and the stored mask
renders the identical frame. Storing all 256 levels would have stored 1.7 MB
of bilinear interpolation to reconstruct a predicate, and would not even have
compressed: a model mask is a bilinear upsample of a coarse grid, so almost no
two adjacent bytes are alike. Measured on a simulated sky and a simulated
figure at 1600x1067, against 1.71 MB raw: 4.0 kB and 6.5 kB at two levels,
46 kB and 76 kB at sixteen, 835 kB and 1.43 MB at all 256. The level count is
still written into the line, so a later build that finds a use for the
shoulder can write sixteen and this one will read them rather than misreading
a stream of lengths as pairs.
The coder is hand-rolled — run-length pairs in a base-64 varint — because
`dr-pipeline` links nothing, which is the property that lets the descriptor
and codegen logic be tested without a device. `flate2` would have been fewer
lines and a dependency in the one crate that has none.
Where it lives matters more than how it is coded. The raster sits on
`MaskLayer` beside the source, not inside `MaskSource::Subject`: the source is
*identity*, which is what makes it diff as a handful of numbers and merge per
field under FR-NC-9, and a raster in there would have given the merge a binary
blob to arbitrate. It takes no part in `MaskLayer`'s equality for the same
reason — a device that has run the model and one that has not hold the same
edit, and counting the difference would raise a conflict over a cache and let
`remote_wins` answer it by discarding the only copy of the pixels.
Encoding happens in `masks_for_storage`, on the save path, rather than in
`ensure_subject_fields` where every coverage already funnels through.
`ensure_subject_fields` runs on a drag — dilating a mask with a compound
morphology rebuilds the field every frame — and encoding a megapixel raster
per frame is the kind of work NFR-P5 exists to keep off a gesture. Saving
happens once, when the photograph stops being the open one, and already costs
a network round trip.
Version skew holds both ways. A file with no `coverage` line reads exactly as
it did before, which is a layer that needs the model run; an unreadable one
costs the pixels and not the layer, because the layer is the edit and the
raster is a cache of it. An old build reading a new file drops the key it does
not understand, which costs a model run and no work. And a payload that will
not compress is refused rather than truncated: a checkerboard would encode to
twice the raster it came from, so past 64 kB nothing is stored and the
behaviour falls back to what it was — half a mask would render as a mask that
is confidently wrong, which is the failure that tells nobody.
This commit is contained in:
@@ -34,10 +34,27 @@
|
||||
//! The cost is that the ids only mean anything alongside the segmentation that
|
||||
//! produced them, so [`MaskSource::Regions::signature`] records which one —
|
||||
//! see there for what happens when it does not match.
|
||||
//!
|
||||
//! # And the raster that had to come back anyway
|
||||
//!
|
||||
//! The same reasoning was applied to [`MaskSource::Subject`] and
|
||||
//! [`MaskSource::Category`], and there it went one step too far. A model's
|
||||
//! coverage is reproducible in principle, but only by running the model — and
|
||||
//! nothing runs one except a photographer pressing a button. So a stored
|
||||
//! subject layer resolved to no pixels on every path that did not have a run
|
||||
//! already in memory: reopening the photograph, and exporting it from the
|
||||
//! grid, which never runs one at all.
|
||||
//!
|
||||
//! [`MaskLayer::coverage`] is the answer, and note what it is *not*: the
|
||||
//! source still stores identity, still diffs as a handful of numbers, and
|
||||
//! still merges per field. The raster sits beside it as a cache, takes no part
|
||||
//! in equality, and is thrown away rather than trusted when it does not fit.
|
||||
//! See [`crate::coverage`].
|
||||
|
||||
use std::fmt::Write as _;
|
||||
use std::sync::Arc;
|
||||
|
||||
use crate::coverage::Coverage;
|
||||
use crate::descriptor::{OpDescriptor, ParamId};
|
||||
use crate::operation::Operation;
|
||||
use crate::ops;
|
||||
@@ -527,10 +544,12 @@ pub enum MaskSource {
|
||||
/// right and soft, and dilation, erosion and a chosen falloff are how it
|
||||
/// is made to fit.
|
||||
///
|
||||
/// Stored as *identity*, not as pixels. The mask itself is several
|
||||
/// megabytes and is reproducible by running the same model over the same
|
||||
/// image, so the sidecar carries what is needed to find it again and the
|
||||
/// session carries the pixels.
|
||||
/// Stored as *identity*, not as pixels: the mask is several megabytes and
|
||||
/// the fields below are what is needed to find it again. The pixels do go
|
||||
/// in the sidecar as well, run-length coded beside the layer rather than
|
||||
/// inside this variant, because "reproducible by running the model again"
|
||||
/// turned out to mean "absent everywhere a model has not been run" — see
|
||||
/// [`MaskLayer::coverage`].
|
||||
Subject {
|
||||
/// Which segmentation run produced it, so a layer can tell whether
|
||||
/// the index below still means what it meant.
|
||||
@@ -559,9 +578,8 @@ pub enum MaskSource {
|
||||
/// before the mask ever existed, and [`Self::Subject`] is the source for
|
||||
/// that question.
|
||||
///
|
||||
/// Stored as identity like a subject, and for the same reason: the
|
||||
/// coverage is megabytes and is reproducible from the same model over the
|
||||
/// same image.
|
||||
/// Stored as identity like a subject, and the coverage travels beside it
|
||||
/// for the same reason — see [`MaskLayer::coverage`].
|
||||
Category {
|
||||
/// Which segmentation run produced it, so a layer can tell whether
|
||||
/// the name below still refers to something that was computed.
|
||||
@@ -715,6 +733,41 @@ pub struct MaskLayer {
|
||||
/// the same reason: two layers may sit on the same category and want
|
||||
/// different amounts of it, and the model ran once for both.
|
||||
pub refine: f32,
|
||||
|
||||
/// The pixels this layer covered, when a model produced them and they
|
||||
/// were worth storing.
|
||||
///
|
||||
/// # Beside the source, not inside it
|
||||
///
|
||||
/// [`MaskSource::Subject`] and [`MaskSource::Category`] are *identity* —
|
||||
/// which run, which instance, which category — and that is what makes them
|
||||
/// diffable, small, and mergeable per field under FR-NC-9. Putting a
|
||||
/// raster inside either variant would make two devices that selected the
|
||||
/// same dog hold different values for the same selection, and the merge
|
||||
/// would then have a binary blob to arbitrate rather than an index.
|
||||
///
|
||||
/// So this sits alongside as what it actually is: a **materialisation** of
|
||||
/// the source, produced by a run of a model this crate has never heard of
|
||||
/// and knows nothing about. `MaskSource` still says what the layer means;
|
||||
/// this says what that meant last time anybody worked it out. The
|
||||
/// distinction is why it takes no part in [`PartialEq`] — a layer with the
|
||||
/// pixels cached and one without are the same edit, and a merge that
|
||||
/// called them different would raise a conflict over a cache.
|
||||
///
|
||||
/// # Why it exists at all
|
||||
///
|
||||
/// Without it a stored subject or category layer renders as nothing until
|
||||
/// somebody presses "find subjects" — so reopening a photograph dropped
|
||||
/// its local adjustments, and a batch export, which never runs a model,
|
||||
/// could not have them at any point. See [`crate::coverage`].
|
||||
///
|
||||
/// Shared rather than owned because the undo stack holds a snapshot per
|
||||
/// step and a layer is cloned by value; an `Arc` makes recording a slider
|
||||
/// drag cost a refcount rather than a copy of every mask in the stack.
|
||||
/// Only ever set for the two model sources — nothing else has a model
|
||||
/// behind it to cache.
|
||||
pub coverage: Option<Arc<Coverage>>,
|
||||
|
||||
/// This layer's adjustments.
|
||||
///
|
||||
/// A full chain, the same one [`crate::EditGraph`] holds. That is the
|
||||
@@ -772,6 +825,8 @@ impl Clone for MaskLayer {
|
||||
morphology: self.morphology,
|
||||
morph_radius: self.morph_radius,
|
||||
refine: self.refine,
|
||||
// A refcount, not a raster. See the field.
|
||||
coverage: self.coverage.clone(),
|
||||
ops,
|
||||
}
|
||||
}
|
||||
@@ -789,12 +844,22 @@ impl std::fmt::Debug for MaskLayer {
|
||||
.field("feather", &self.feather)
|
||||
.field("falloff", &self.falloff)
|
||||
.field("morphology", &self.morphology)
|
||||
.field("coverage", &self.coverage.as_ref().map(|c| c.encoded_len()))
|
||||
.field("active_ops", &self.active_ops().count())
|
||||
.finish()
|
||||
}
|
||||
}
|
||||
|
||||
impl PartialEq for MaskLayer {
|
||||
/// Every field that is the *edit*, and deliberately not
|
||||
/// [`Self::coverage`].
|
||||
///
|
||||
/// This comparison is what [`crate::sidecar::Version::merge`] uses to
|
||||
/// decide whether a device changed a layer (FR-NC-9). A cached raster is
|
||||
/// not something a photographer changed: one device that has run the model
|
||||
/// and one that has not hold the same edit, and counting the difference
|
||||
/// would raise a conflict over a cache — and, with `remote_wins`, could
|
||||
/// answer it by discarding the only copy of the pixels.
|
||||
fn eq(&self, other: &Self) -> bool {
|
||||
self.id == other.id
|
||||
&& self.name == other.name
|
||||
@@ -833,6 +898,9 @@ impl MaskLayer {
|
||||
// `dr_segment`'s number to state and this crate does not depend on
|
||||
// it — `Session::add_category_mask` sets it on the way in.
|
||||
refine: 0.0,
|
||||
// Nothing has run yet. Filled in the first time a model's coverage
|
||||
// is turned into a distance field — see [`Self::coverage`].
|
||||
coverage: None,
|
||||
ops: layer_chain(),
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user