//! TRACES: FR-DEV-3i //! Region segmentation for local masking (S15, docs/dev/segmentation.md). //! //! Local adjustments need to know where the image's regions are before they //! can snap a mask to one. This crate is that map, and it is deliberately //! **device-free**: the watershed's pixel passes live in `dr-gpu` because they //! are shaders, and everything that reasons about *regions* rather than //! *pixels* lives here, where it can be tested on hand-built inputs with no //! adapter present (ARCH §6.5a). //! //! # The three arms //! //! [`hierarchy`] is **arm A** — a watershed over-segments the image and the //! recorded merge order becomes a granularity ladder. Deterministic, needs no //! model, works on any picture, and knows nothing about what anything *is*. //! //! [`semantic`] is **arm B** — a YOLO instance-segmentation model naming the //! subjects it recognises. Knows what things are, and is vague about exactly //! where their edges fall (its prototypes are quarter-resolution). //! //! [`prior`] is **arm C**, and it is the one that ships. Arm B's instances //! *re-weight* arm A's merge order, so coarse levels of the ladder line up //! with real objects while every boundary stays exactly where the watershed //! put it. The model contributes what it is good at — knowing what things are //! — and the watershed contributes what it is good at, which is knowing where //! the edge is, to the pixel, at every scale. //! //! That combination is also what repairs the vocabulary problem. The shipped //! model is COCO-trained, so it recognises subjects and has no class for sky, //! foliage or wall (`models/LICENCE.md`). Selecting those falls to arm A, //! which never needed a vocabulary to begin with. //! //! # And [`scene`], which is not one of the arms //! //! The three arms all serve *local* adjustment: they exist so a mask can be //! snapped to one region of the picture. [`scene`] serves the opposite move — //! one grade applied to every pixel of a category at once, sky or foliage or //! water — and reads a second, ADE20K-trained model to do it. It shares this //! crate because it shares the runtime and the letterbox, not because it is //! another way of doing the same thing. //! //! [`refine`] is what the scene model needs and the instance model does not. //! A category's weights come off an 80×80 grid, so one cell is twenty pixels //! of a 1600px proxy and anything smaller than that — a flag in the sky, a //! chimney, a bare branch — is averaged into whatever surrounds it. There is //! no tiling answer here the way there is for an instance, because a category //! has no bounding box to tile over. So the fix is the same one arm C makes: //! the model says *what*, and the photograph's own pixels say *which* of them //! belong to it. //! //! It shares no code with the arms and it is not a fourth one — it sharpens a //! mask that already exists rather than proposing regions — but it is built on //! [`distance`] for the same reason arm C is built on the watershed, which is //! that the useful question is always "how far inside am I". pub mod distance; pub mod hierarchy; pub mod prior; pub mod refine; #[cfg(feature = "semantic")] pub mod scene; #[cfg(feature = "semantic")] pub mod semantic; pub use distance::{signed_distance, Falloff, Morphology, Shaped}; pub use hierarchy::{Edge, Merge, MergeTree, RegionField}; pub use prior::{Membership, PriorOptions}; pub use refine::{ refine_category, RefineOptions, Refined, Refinement, SkipReason, STRICTNESS_DEFAULT, STRICTNESS_MAX, STRICTNESS_OFF, }; #[cfg(feature = "semantic")] pub use scene::{Category, Scene, SceneModel}; #[cfg(feature = "embedded-model")] pub use semantic::embedded_model_bytes; #[cfg(feature = "semantic")] pub use semantic::{Instance, SemanticModel, SemanticOptions, Tiling}; /// What can go wrong between an image and a region map. #[derive(Debug, thiserror::Error)] pub enum SegmentError { #[error("could not read model file: {0}")] ModelRead(#[source] std::io::Error), #[cfg(feature = "semantic")] #[error("inference failed: {0}")] Inference(#[source] ort::Error), #[error("image buffer is {got} floats, expected {expected} (RGB, three per pixel)")] ImageShape { expected: usize, got: usize }, /// The graph produced something the decoder does not recognise — a /// different model, or a different export of the same one. #[error("model output '{0}' did not have the expected shape")] OutputShape(&'static str), /// `models/scene/categories.txt` and the model disagree, or the descriptor /// is malformed. Its own variant rather than a parse error because every /// case carries a specific sentence about what to fix. #[error("category descriptor: {0}")] CategoryDescriptor(String), } #[cfg(feature = "semantic")] impl From for SegmentError { fn from(e: dr_inference_engine::Error) -> Self { match e { dr_inference_engine::Error::Inference(e) => SegmentError::Inference(e), dr_inference_engine::Error::Io(e) => SegmentError::ModelRead(e), } } }