Start a category mask where the photograph can bear it
Clicking "architecture" made a layer whose mask was gone. Every category layer began at STRICTNESS_DEFAULT, and that constant was fitted on the synthetic sky the refine tests build — its own note warns that a real photograph's noise "moves every crossing down together", which turns out to be a considerable understatement. Measured over seven ordinary frames, half scale removes 76% to 99.5% of `architecture`, 36% to 93% of `ground` and 18% to 91% of `vegetation`. Only sky, the category the number was calibrated against, survives it. An empty mask is indistinguishable from a broken one: the layer is listed, the adjustment moves, and no pixel changes. So what this looks like from outside is that the segmentation does not make masks at all. No smaller constant fixes it either, because a nat of evidence means different things over a smooth sky and over a stone facade — the useful position is above 5 on one frame and below 1 on the next. So the frame is asked instead: `Refinement::gentle` walks down from half scale and takes the first rung whose gate removes no more than a sixth of the category's weight, and the model's own outline when none of them does. One `apply` on a friendly photograph and four on an unfriendly one, paid when a layer is made rather than for eight categories nobody masked. The slider's reset went to 4 as well, so taking the control back to its "default" emptied the mask. It goes to zero now, which is the one position documented to mean something: exactly what the model weighted.
This commit is contained in:
@@ -299,6 +299,13 @@ pub const STRICTNESS_MAX: f32 = 8.0;
|
||||
/// smooth enough to sit on [`VARIANCE_FLOOR`], where a real one has noise and
|
||||
/// therefore a real spread, which moves every crossing down together. The
|
||||
/// ordering survives that; the exact placement is what the slider is for.
|
||||
///
|
||||
/// **Not where a new layer starts.** That warning turned out to be an
|
||||
/// understatement — on a real photograph this position removes most of every
|
||||
/// category that is not sky, and the measurements are in
|
||||
/// [`Refinement::gentle`], which is what a layer starts at instead. What this
|
||||
/// constant still names is the midpoint of the control's travel, and the
|
||||
/// synthetic frame the tests hold it against.
|
||||
pub const STRICTNESS_DEFAULT: f32 = 4.0;
|
||||
|
||||
/// Why a refinement could not be built.
|
||||
@@ -773,8 +780,88 @@ impl Refinement {
|
||||
.map(|&w| (w * 255.0).round().clamp(0.0, 255.0) as u8)
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// The strongest strictness this photograph can be started at without the
|
||||
/// category disappearing.
|
||||
///
|
||||
/// # Why a constant could not do this job
|
||||
///
|
||||
/// [`STRICTNESS_DEFAULT`] was measured on the synthetic frame the tests
|
||||
/// build, and its own note warns that a real photograph's noise "moves
|
||||
/// every crossing down together". It moves them a great deal further than
|
||||
/// that reads. Measured on seven ordinary frames, `4.0` — half scale, the
|
||||
/// position a control would naturally start at — removes:
|
||||
///
|
||||
/// | category | weight removed at 4.0 |
|
||||
/// |----------|----------------------|
|
||||
/// | sky | 0% – 6% |
|
||||
/// | vegetation | 18% – 91% |
|
||||
/// | ground | 36% – 93% |
|
||||
/// | architecture | 76% – **99.5%** |
|
||||
///
|
||||
/// So a layer created at the constant is an *empty mask* on most
|
||||
/// photographs that are not mostly sky, and empty is indistinguishable
|
||||
/// from broken: the adjustment moves and no pixel changes. That is the
|
||||
/// whole of the fault, and it cannot be fixed by choosing a smaller
|
||||
/// constant — the useful position is 5.0 on one frame and below 1.0 on the
|
||||
/// next, because a nat of evidence means different things over a smooth
|
||||
/// sky and over a stone facade.
|
||||
///
|
||||
/// # What this does instead
|
||||
///
|
||||
/// It asks the photograph. Walking down from half scale, the first rung
|
||||
/// whose gate takes no more than [`GENTLE_CUT`] of the category's weight
|
||||
/// is the answer, and [`STRICTNESS_OFF`] is the answer when none of them
|
||||
/// does — the model's own outline, which is never wrong about *where the
|
||||
/// category is*, only about where it stops.
|
||||
///
|
||||
/// Downwards rather than upwards because the friendly case is the common
|
||||
/// one and it exits on the first rung: a sky costs one `apply`, and only a
|
||||
/// frame the refinement disagrees with pays for all four.
|
||||
///
|
||||
/// This is a starting position and not a limit. The slider still offers
|
||||
/// the whole range, and it is the control's job to let a photographer go
|
||||
/// past what this considered safe.
|
||||
pub fn gentle(&self, coverage: &[u8]) -> f32 {
|
||||
let total: f64 = coverage.iter().map(|&c| c as f64).sum();
|
||||
if total <= 0.0 {
|
||||
return STRICTNESS_OFF;
|
||||
}
|
||||
for &strictness in GENTLE_LADDER {
|
||||
let kept: f64 = self
|
||||
.apply_coverage(coverage, strictness)
|
||||
.iter()
|
||||
.map(|&c| c as f64)
|
||||
.sum();
|
||||
if (total - kept) / total <= GENTLE_CUT as f64 {
|
||||
return strictness;
|
||||
}
|
||||
}
|
||||
STRICTNESS_OFF
|
||||
}
|
||||
}
|
||||
|
||||
/// How much of a category's weight a *starting* strictness may take.
|
||||
///
|
||||
/// A sixth, and the number is doing one job: separating "the gate tidied the
|
||||
/// edge" from "the gate ate the category". A twenty-proxy-pixel boundary
|
||||
/// around a subject covering a fifth of the frame is a few percent of its
|
||||
/// weight, so a refinement doing what it is for lands well under this; the
|
||||
/// failures measured on [`Refinement::gentle`]'s table are all at 76% and
|
||||
/// above. Nothing sits near the line, which is what makes it safe to state as
|
||||
/// a constant rather than fit.
|
||||
const GENTLE_CUT: f32 = 1.0 / 6.0;
|
||||
|
||||
/// The rungs [`Refinement::gentle`] tries, strongest first.
|
||||
///
|
||||
/// Whole nats, because that is the unit the evidence is denominated in and the
|
||||
/// spacing the sweep in `examples/scene.rs` is read at. Stopping at half scale
|
||||
/// rather than at [`STRICTNESS_MAX`]: past there a category's own dominant
|
||||
/// colours have gone, and a photograph on which 8.0 removed under a sixth of
|
||||
/// the weight would be one where the gate is finding nothing to cut and the
|
||||
/// starting position may as well be gentler.
|
||||
const GENTLE_LADDER: &[f32] = &[4.0, 3.0, 2.0, 1.0];
|
||||
|
||||
/// The eight-neighbourhood, as offsets.
|
||||
///
|
||||
/// Eight rather than four because a watershed on a four-neighbourhood produces
|
||||
@@ -1478,6 +1565,90 @@ mod tests {
|
||||
);
|
||||
}
|
||||
|
||||
/// The coverage buffer as the application holds it: one byte a pixel.
|
||||
fn quantised(weights: &[f32]) -> Vec<u8> {
|
||||
weights
|
||||
.iter()
|
||||
.map(|&w| (w.clamp(0.0, 1.0) * 255.0).round() as u8)
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// What fraction of a category's weight a strictness takes away.
|
||||
fn removed(r: &Refinement, coverage: &[u8], strictness: f32) -> f32 {
|
||||
let total: f64 = coverage.iter().map(|&c| c as f64).sum();
|
||||
let kept: f64 = r
|
||||
.apply_coverage(coverage, strictness)
|
||||
.iter()
|
||||
.map(|&c| c as f64)
|
||||
.sum();
|
||||
((total - kept) / total) as f32
|
||||
}
|
||||
|
||||
/// The promise a starting position has to keep: whatever it chooses, the
|
||||
/// category is still there afterwards.
|
||||
///
|
||||
/// This is the fault it exists to fix, stated as an assertion. A fixed
|
||||
/// strictness removed 76% to 99.5% of `architecture` on real photographs
|
||||
/// — a mask that is empty on arrival, and indistinguishable from a broken
|
||||
/// one, because the adjustment moves and no pixel changes.
|
||||
#[test]
|
||||
fn a_starting_strictness_never_empties_the_category() {
|
||||
let opts = RefineOptions::default();
|
||||
for (rect, colour, what) in [(FLAG, RED, "flag"), (CLOUD, WHITE, "cloud")] {
|
||||
let (rgb, weights) = with(rect, colour);
|
||||
let refinement = Refinement::compute(&weights, &rgb, EDGE, EDGE, CELL, &opts)
|
||||
.expect("the frame has both sides");
|
||||
let coverage = quantised(&weights);
|
||||
|
||||
let start = refinement.gentle(&coverage);
|
||||
let cut = removed(&refinement, &coverage, start);
|
||||
assert!(
|
||||
cut <= GENTLE_CUT + 1e-3,
|
||||
"the {what} frame starts at {start}, which takes {:.1}% of the category",
|
||||
cut * 100.0
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// And it must not answer with zero out of caution.
|
||||
///
|
||||
/// A starting position that is always "off" would be a safe way of not
|
||||
/// having the feature. On the frame the module was built for — a red flag
|
||||
/// inside the sky — there *is* a strictness that takes the flag and leaves
|
||||
/// the sky, and this has to find it.
|
||||
#[test]
|
||||
fn a_gentle_start_still_takes_the_flag_out_of_the_sky() {
|
||||
let (rgb, weights) = with(FLAG, RED);
|
||||
let refinement =
|
||||
Refinement::compute(&weights, &rgb, EDGE, EDGE, CELL, &RefineOptions::default())
|
||||
.expect("the flag frame has both sides");
|
||||
let coverage = quantised(&weights);
|
||||
|
||||
let start = refinement.gentle(&coverage);
|
||||
assert!(start > STRICTNESS_OFF, "gave up rather than choosing");
|
||||
|
||||
let refined = refinement.apply(&weights, start);
|
||||
let inside = mean(&refined, EDGE, FLAG_CORE);
|
||||
assert!(inside < 0.2, "the flag should be cut out, got {inside}");
|
||||
let sky = mean(&refined, EDGE, (8, 8, 60, 60));
|
||||
assert!(sky > 0.8, "the sky around it should survive, got {sky}");
|
||||
}
|
||||
|
||||
/// A category nothing has claimed has nothing to judge, and asking must
|
||||
/// not divide by its zero total.
|
||||
#[test]
|
||||
fn an_empty_category_starts_at_off() {
|
||||
let (rgb, weights) = with(FLAG, RED);
|
||||
let refinement =
|
||||
Refinement::compute(&weights, &rgb, EDGE, EDGE, CELL, &RefineOptions::default())
|
||||
.expect("the flag frame has both sides");
|
||||
assert_eq!(
|
||||
refinement.gentle(&vec![0u8; EDGE * EDGE]),
|
||||
STRICTNESS_OFF,
|
||||
"nothing to cut back"
|
||||
);
|
||||
}
|
||||
|
||||
/// Zero strictness is exactly the model's own weighting.
|
||||
///
|
||||
/// The control's off position has to be the old behaviour bit for bit, or
|
||||
|
||||
+40
-39
File diff suppressed because one or more lines are too long
+15
-6
@@ -3228,7 +3228,7 @@ impl DevelopSession {
|
||||
},
|
||||
);
|
||||
layer.name = name.to_string();
|
||||
// Refined from the start, where there is anything to refine with.
|
||||
// Refined from the start, by as much as this photograph will bear.
|
||||
//
|
||||
// A category's edges are twenty proxy pixels wide before this runs, so
|
||||
// the unrefined mask is the wrong default for the common case — a
|
||||
@@ -3236,16 +3236,25 @@ impl DevelopSession {
|
||||
// chimney in it. Zero is still one drag away, and it is exactly the
|
||||
// model's own weighting when they get there.
|
||||
//
|
||||
// **Asked of the frame rather than taken from a constant**, and that
|
||||
// is the correction rather than a refinement of the idea. The constant
|
||||
// was `STRICTNESS_DEFAULT`, fitted on a synthetic sky; on real
|
||||
// photographs it removes three quarters to all of `architecture`,
|
||||
// `ground` and `vegetation`, so clicking a category produced an empty
|
||||
// mask — the failure that reads as the feature not working at all,
|
||||
// because the adjustment moves and no pixel changes. The measurements
|
||||
// are on `dr_segment::Refinement::gentle`, which is what picks the
|
||||
// number now.
|
||||
//
|
||||
// Set here rather than in `MaskLayer::new` because the number belongs
|
||||
// to `dr_segment` and `dr-pipeline` does not depend on it — see
|
||||
// `dr_pipeline::mask::MAX_REFINE`.
|
||||
if self
|
||||
layer.base_mut().refine = self
|
||||
.segmentation
|
||||
.as_ref()
|
||||
.is_some_and(|seg| seg.category_is_refinable(name))
|
||||
{
|
||||
layer.base_mut().refine = dr_segment::STRICTNESS_DEFAULT;
|
||||
}
|
||||
.map_or(dr_segment::STRICTNESS_OFF, |seg| {
|
||||
seg.category_default_refine(name)
|
||||
});
|
||||
if !self.graph.masks_mut().push(layer) {
|
||||
return None;
|
||||
}
|
||||
|
||||
@@ -201,6 +201,28 @@ impl Segmentation {
|
||||
.is_some_and(|c| c.refinement.is_some())
|
||||
}
|
||||
|
||||
/// Where this category's refine control should start on *this*
|
||||
/// photograph.
|
||||
///
|
||||
/// [`dr_segment::STRICTNESS_OFF`] where nothing was fitted, and otherwise
|
||||
/// whatever [`dr_segment::Refinement::gentle`] finds the frame will bear —
|
||||
/// see there for why a constant could not do it and what a constant cost.
|
||||
///
|
||||
/// Costs one to four `apply` passes, tens of milliseconds each. That is
|
||||
/// why it is asked here, when a layer is made, rather than for every
|
||||
/// category during the precompute: eight categories nobody masked would be
|
||||
/// seconds added to a wait, and the answer is only wanted for the one that
|
||||
/// was clicked.
|
||||
pub fn category_default_refine(&self, name: &str) -> f32 {
|
||||
self.categories
|
||||
.iter()
|
||||
.find(|c| &*c.name == name)
|
||||
.and_then(|c| Some((c.refinement.as_ref()?, &c.mask)))
|
||||
.map_or(dr_segment::STRICTNESS_OFF, |(refinement, mask)| {
|
||||
refinement.gentle(mask)
|
||||
})
|
||||
}
|
||||
|
||||
/// Replace one instance in place, keeping every other index and the
|
||||
/// signature unchanged.
|
||||
///
|
||||
|
||||
+10
-2
@@ -412,7 +412,15 @@ component MaskEntry inherits Rectangle {
|
||||
+ "outline onto the edge the photograph actually has. "
|
||||
+ "Zero is the model's own outline.";
|
||||
value: root.data.refine;
|
||||
default-value: 4;
|
||||
// Zero, which is the model's own outline — and *not* the
|
||||
// position a layer starts at, which is chosen per photograph
|
||||
// because a nat of evidence means different things over a
|
||||
// smooth sky and over a stone facade (see
|
||||
// `dr_segment::Refinement::gentle`). A fixed default here used
|
||||
// to be 4, which is half travel and looks like the natural
|
||||
// resting place; on an ordinary frame it takes three quarters
|
||||
// of the category away, so "reset" emptied the mask.
|
||||
default-value: 0;
|
||||
minimum: 0;
|
||||
maximum: 8;
|
||||
// Tenths, not whole nats. The interval between a flag going
|
||||
@@ -421,7 +429,7 @@ component MaskEntry inherits Rectangle {
|
||||
// the range it exists to explore.
|
||||
precision: 1;
|
||||
changed(v) => { root.refine-changed(v); }
|
||||
reset => { root.refine-changed(4); }
|
||||
reset => { root.refine-changed(0); }
|
||||
}
|
||||
|
||||
// The band (FR-DEV-10). Above the edge controls because for a
|
||||
|
||||
Reference in New Issue
Block a user