Weigh the categories in the same precompute, and mask with them
The scene model shipped with a decoder and no caller. This runs it. ## Beside the instance pass, not instead of it `compute` now does both on the same upright frame and lays both back down the same way, so instance masks and category masks index into one grid — the sensor's. A failure in the scene half is logged and dropped rather than propagated: no scene model is an ordinary state, and a photograph that can still be masked by subject should not become unopenable because the categories are missing. Categories under half a percent of the frame never reach the cache. A control that does nothing when moved is worse than an absent one, and each one it skips is a proxy-sized buffer not allocated. ## The shader needed nothing A category reaches `dr-gpu` as a soft coverage buffer at proxy resolution, turned into a distance field — which is exactly what a subject is. So they share `MODE_SUBJECT`. That is not a shortcut taken for speed: the shader has no way to tell them apart and no reason to want one. What differs is only which model produced the coverage, and that has already happened by then. Feather, falloff, dilation and erosion therefore work on a category on the day it arrives, because they were never subject-specific. ## Where the weights come from `scene-model` compiles the graph in and the desktop app takes it; Android leaves it off and reads the copy `install_bundled_models` unpacks, because 24 MB of constant is worth avoiding in a mobile install and not worth the plumbing to avoid on a desktop one. Embedded is tried first — a build that has the weights compiled in should not be silently overridden by a stale file in a data directory. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
+82
-1
@@ -1795,6 +1795,23 @@ impl DevelopSession {
|
||||
MaskSource::Subject { index, .. } => {
|
||||
mix(1);
|
||||
mix(*index as u64);
|
||||
if layer.morphology.is_compound() {
|
||||
mix(match layer.morphology {
|
||||
Morphology::Close => 2,
|
||||
Morphology::Open => 3,
|
||||
_ => 0,
|
||||
});
|
||||
mix(layer.morph_radius.to_bits() as u64);
|
||||
}
|
||||
}
|
||||
// Hashed by *name*, and `4` rather than `1` so a category
|
||||
// named the same as an instance index could never collide with
|
||||
// it. The field has to be rebuilt when either changes.
|
||||
MaskSource::Category { name, .. } => {
|
||||
mix(4);
|
||||
for b in name.as_bytes() {
|
||||
mix(*b as u64);
|
||||
}
|
||||
// Only the compound operations change the field itself.
|
||||
if layer.morphology.is_compound() {
|
||||
mix(match layer.morphology {
|
||||
@@ -1830,6 +1847,20 @@ impl DevelopSession {
|
||||
let mut fields: Vec<Vec<f32>> = Vec::new();
|
||||
for layer in self.graph.masks().active() {
|
||||
let field = match &layer.source {
|
||||
MaskSource::Category { name, .. } => seg
|
||||
.category_mask(name)
|
||||
.map(|coverage| {
|
||||
dr_segment::Shaped::build(
|
||||
coverage,
|
||||
pw,
|
||||
ph,
|
||||
128,
|
||||
morphology_for(layer.morphology),
|
||||
layer.morph_radius * pw.min(ph) as f32,
|
||||
)
|
||||
.distance
|
||||
})
|
||||
.unwrap_or_default(),
|
||||
MaskSource::Subject { index, .. } => seg
|
||||
.instance_mask(*index as usize)
|
||||
.map(|coverage| {
|
||||
@@ -2594,6 +2625,54 @@ impl DevelopSession {
|
||||
self.add_subject_mask(index)
|
||||
}
|
||||
|
||||
/// Add a layer covering one photographic category.
|
||||
///
|
||||
/// The counterpart to [`Self::add_subject_mask`], and it takes a *name*
|
||||
/// rather than an index for the reason `MaskSource::Category` stores one:
|
||||
/// the descriptor grouping ADE20K's classes is editable, so an index would
|
||||
/// silently repoint every stored layer the first time a category was
|
||||
/// added to it.
|
||||
///
|
||||
/// The layer is named after the category, because "sky" is a better name
|
||||
/// for a layer than "Mask 3" and the user can rename it anyway.
|
||||
pub fn add_category_mask(&mut self, name: &str) -> Option<String> {
|
||||
use dr_pipeline::mask::MaskSource;
|
||||
|
||||
let seg = self.segmentation.as_ref()?;
|
||||
let signature = seg.signature();
|
||||
// Refuse a category this run did not produce rather than creating a
|
||||
// layer that renders empty: an empty mask looks like a broken
|
||||
// adjustment, where a button that does nothing at least says so.
|
||||
if seg.category_mask(name).is_none() {
|
||||
return None;
|
||||
}
|
||||
|
||||
let id = self.graph.masks().next_id();
|
||||
let mut layer = MaskLayer::new(
|
||||
id.clone(),
|
||||
MaskSource::Category {
|
||||
signature,
|
||||
name: name.to_string(),
|
||||
},
|
||||
);
|
||||
layer.name = name.to_string();
|
||||
if !self.graph.masks_mut().push(layer) {
|
||||
return None;
|
||||
}
|
||||
self.active_masks = vec![id.clone()];
|
||||
self.history
|
||||
.record(&self.graph, Edit::Action(labels::step::MASK_ADDED));
|
||||
Some(id)
|
||||
}
|
||||
|
||||
/// What the scene model found in this frame, largest category first.
|
||||
///
|
||||
/// Empty when no scene model was available, which is an ordinary state —
|
||||
/// see `segmentation::scene_categories`.
|
||||
pub fn categories(&self) -> &[segmentation::CategorySummary] {
|
||||
self.segmentation.as_ref().map_or(&[], |s| s.categories())
|
||||
}
|
||||
|
||||
/// Add a layer selecting one detected subject.
|
||||
///
|
||||
/// The instance's own coverage is the mask, rather than the watershed
|
||||
@@ -2768,7 +2847,9 @@ impl DevelopSession {
|
||||
self.graph.masks().get(id).is_some_and(|l| {
|
||||
matches!(
|
||||
l.source,
|
||||
MaskSource::Subject { .. } | MaskSource::Regions { .. }
|
||||
MaskSource::Subject { .. }
|
||||
| MaskSource::Category { .. }
|
||||
| MaskSource::Regions { .. }
|
||||
)
|
||||
})
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user