dtourolle
696bafa9d5
§7 still listed "AI subject masking — deferred per D11" while MaskSource::Subject and MaskSource::Category, backed by dr-segment's instance and semantic models, had been the primary way a local adjustment is made for weeks. The code was tagged FR-DEV-3, which names gradients and brushes and says nothing about a model. FR-DEV-3i now states what exists: a subject or a category found by a local model, stored as identity with the run's signature so that it merges per field and reads as stale rather than wrong, then treated as any other layer by the edge, stroke, composition and reveal clauses. The one place it departs from FR-DEV-19 — coverage written run-length coded beside the layer, so a stored subject renders without a model — is recorded in the clause instead of left for the next audit to find. The segmentation crate and the UI's selection module are tagged to it.
101 lines
4.6 KiB
Rust
101 lines
4.6 KiB
Rust
//! TRACES: FR-DEV-3i
|
||
//! Region segmentation for local masking (S15, docs/segmentation.md).
|
||
//!
|
||
//! Local adjustments need to know where the image's regions are before they
|
||
//! can snap a mask to one. This crate is that map, and it is deliberately
|
||
//! **device-free**: the watershed's pixel passes live in `dr-gpu` because they
|
||
//! are shaders, and everything that reasons about *regions* rather than
|
||
//! *pixels* lives here, where it can be tested on hand-built inputs with no
|
||
//! adapter present (ARCH §6.5a).
|
||
//!
|
||
//! # The three arms
|
||
//!
|
||
//! [`hierarchy`] is **arm A** — a watershed over-segments the image and the
|
||
//! recorded merge order becomes a granularity ladder. Deterministic, needs no
|
||
//! model, works on any picture, and knows nothing about what anything *is*.
|
||
//!
|
||
//! [`semantic`] is **arm B** — a YOLO instance-segmentation model naming the
|
||
//! subjects it recognises. Knows what things are, and is vague about exactly
|
||
//! where their edges fall (its prototypes are quarter-resolution).
|
||
//!
|
||
//! [`prior`] is **arm C**, and it is the one that ships. Arm B's instances
|
||
//! *re-weight* arm A's merge order, so coarse levels of the ladder line up
|
||
//! with real objects while every boundary stays exactly where the watershed
|
||
//! put it. The model contributes what it is good at — knowing what things are
|
||
//! — and the watershed contributes what it is good at, which is knowing where
|
||
//! the edge is, to the pixel, at every scale.
|
||
//!
|
||
//! That combination is also what repairs the vocabulary problem. The shipped
|
||
//! model is COCO-trained, so it recognises subjects and has no class for sky,
|
||
//! foliage or wall (`models/LICENCE.md`). Selecting those falls to arm A,
|
||
//! which never needed a vocabulary to begin with.
|
||
//!
|
||
//! # And [`scene`], which is not one of the arms
|
||
//!
|
||
//! The three arms all serve *local* adjustment: they exist so a mask can be
|
||
//! snapped to one region of the picture. [`scene`] serves the opposite move —
|
||
//! one grade applied to every pixel of a category at once, sky or foliage or
|
||
//! water — and reads a second, ADE20K-trained model to do it. It shares this
|
||
//! crate because it shares the runtime and the letterbox, not because it is
|
||
//! another way of doing the same thing.
|
||
//!
|
||
//! [`refine`] is what the scene model needs and the instance model does not.
|
||
//! A category's weights come off an 80×80 grid, so one cell is twenty pixels
|
||
//! of a 1600px proxy and anything smaller than that — a flag in the sky, a
|
||
//! chimney, a bare branch — is averaged into whatever surrounds it. There is
|
||
//! no tiling answer here the way there is for an instance, because a category
|
||
//! has no bounding box to tile over. So the fix is the same one arm C makes:
|
||
//! the model says *what*, and the photograph's own pixels say *which* of them
|
||
//! belong to it.
|
||
//!
|
||
//! It shares no code with the arms and it is not a fourth one — it sharpens a
|
||
//! mask that already exists rather than proposing regions — but it is built on
|
||
//! [`distance`] for the same reason arm C is built on the watershed, which is
|
||
//! that the useful question is always "how far inside am I".
|
||
|
||
pub mod distance;
|
||
pub mod hierarchy;
|
||
pub mod prior;
|
||
pub mod refine;
|
||
#[cfg(feature = "semantic")]
|
||
pub mod scene;
|
||
#[cfg(feature = "semantic")]
|
||
pub mod semantic;
|
||
|
||
pub use distance::{signed_distance, Falloff, Morphology, Shaped};
|
||
pub use hierarchy::{Edge, Merge, MergeTree, RegionField};
|
||
pub use prior::{Membership, PriorOptions};
|
||
pub use refine::{
|
||
refine_category, RefineOptions, Refined, Refinement, SkipReason, STRICTNESS_DEFAULT,
|
||
STRICTNESS_MAX, STRICTNESS_OFF,
|
||
};
|
||
#[cfg(feature = "semantic")]
|
||
pub use scene::{Category, Scene, SceneModel};
|
||
#[cfg(feature = "semantic")]
|
||
pub use semantic::{Instance, SemanticModel, SemanticOptions, Tiling};
|
||
|
||
/// What can go wrong between an image and a region map.
|
||
#[derive(Debug, thiserror::Error)]
|
||
pub enum SegmentError {
|
||
#[error("could not read model file: {0}")]
|
||
ModelRead(#[source] std::io::Error),
|
||
|
||
#[cfg(feature = "semantic")]
|
||
#[error("inference failed: {0}")]
|
||
Inference(#[source] ort::Error),
|
||
|
||
#[error("image buffer is {got} floats, expected {expected} (RGB, three per pixel)")]
|
||
ImageShape { expected: usize, got: usize },
|
||
|
||
/// The graph produced something the decoder does not recognise — a
|
||
/// different model, or a different export of the same one.
|
||
#[error("model output '{0}' did not have the expected shape")]
|
||
OutputShape(&'static str),
|
||
|
||
/// `models/scene/categories.txt` and the model disagree, or the descriptor
|
||
/// is malformed. Its own variant rather than a parse error because every
|
||
/// case carries a specific sentence about what to fix.
|
||
#[error("category descriptor: {0}")]
|
||
CategoryDescriptor(String),
|
||
}
|