dtourolle
104e3a106f
dr_pano::fill owns everything the model does not — which tiles, what context, how to blend — behind an Inpainter trait, and dr_pano::migan is that trait over the shipped generator on the inference engine. The known content is mirrored across the coverage edge into the hole and a 256-px ring, the nearest 48 px folded, so the model interpolates between real and mirrored sky rather than extrapolating into nothing. A coarse pass at a quarter decides the structure with the whole border in a few tiles; fine passes in 96-px bands from the edge outward texture it; the seam is blended over a feather inside the real edge. Every knob is a Params field, and an Observer hears each stage for whoever is looking at why a fill went wrong.
87 lines
3.0 KiB
Rust
87 lines
3.0 KiB
Rust
//! TRACES: FR-MRG-1 | FR-MRG-10
|
|
//! Panorama geometry — from several frames to the rotations that relate
|
|
//! them, and the projections that lay them out.
|
|
//!
|
|
//! This is the CPU half of a merge (FR-MRG-10): keypoints, matching, the
|
|
//! rotation solve and the choice of output surface. The per-pixel half —
|
|
//! rendering, warping, seams, blending — is the GPU's and lives in
|
|
//! `dr-gpu`, driven from above; nothing here touches a full-resolution
|
|
//! pixel. The split is the whole design (panorama.md §4): everything in
|
|
//! this crate is bounded by the number of frames, not the size of the
|
|
//! composite, and runs on proxies.
|
|
//!
|
|
//! # Layout
|
|
//!
|
|
//! - [`image`] — the grayscale proxy a detector reads: oriented, resampled.
|
|
//! - [`features`] — keypoints with descriptors, and the XFeat decoder.
|
|
//! - [`xfeat`] — the network under tract (feature `xfeat`).
|
|
//! - [`matching`] — mutual nearest neighbours.
|
|
//! - [`homography`] — a robust pairwise homography, the focal length read
|
|
//! off it, and the rotation it implies.
|
|
//! - [`bundle`] — every rotation and the focal length refined together.
|
|
//! - [`align`] — the whole thing, from features to cameras, honest about
|
|
//! what it could not place.
|
|
//! - [`projection`] — perspective, cylindrical, spherical.
|
|
//! - [`linalg`] — the small dense algebra all of it uses.
|
|
//!
|
|
//! # What it depends on
|
|
//!
|
|
//! Nothing, without the `xfeat` feature: the geometry is pure Rust with
|
|
//! hand-rolled linear algebra (`linalg` says why) so that it tests without
|
|
//! a model, a GPU or a device, on synthetic sets whose answer is known
|
|
//! exactly. With the feature it adds the same `ort`-over-tract runtime the
|
|
//! rest of the application already carries.
|
|
|
|
pub mod align;
|
|
pub mod bundle;
|
|
pub mod features;
|
|
pub mod fill;
|
|
pub mod homography;
|
|
pub mod image;
|
|
pub mod linalg;
|
|
pub mod matching;
|
|
#[cfg(feature = "xfeat")]
|
|
pub mod migan;
|
|
pub mod projection;
|
|
#[cfg(feature = "xfeat")]
|
|
pub mod xfeat;
|
|
|
|
pub use align::{align, AlignOptions, Alignment, Link, Unaligned};
|
|
pub use bundle::Cameras;
|
|
pub use features::{Features, Keypoint};
|
|
pub use fill::{fill_border, Inpainter, Observer, Params as FillParams};
|
|
pub use image::Gray;
|
|
pub use projection::Projection;
|
|
|
|
#[derive(Debug, thiserror::Error)]
|
|
pub enum PanoError {
|
|
#[error("bad input: {0}")]
|
|
Input(String),
|
|
#[error("geometry: {0}")]
|
|
Geometry(String),
|
|
#[error("model: {0}")]
|
|
Model(String),
|
|
#[error("could not read the model: {0}")]
|
|
ModelRead(#[source] std::io::Error),
|
|
#[cfg(feature = "xfeat")]
|
|
#[error("inference: {0}")]
|
|
Inference(#[source] ort::Error),
|
|
}
|
|
|
|
#[cfg(feature = "xfeat")]
|
|
impl From<ort::Error> for PanoError {
|
|
fn from(e: ort::Error) -> Self {
|
|
PanoError::Inference(e)
|
|
}
|
|
}
|
|
|
|
#[cfg(feature = "xfeat")]
|
|
impl From<dr_inference_engine::Error> for PanoError {
|
|
fn from(e: dr_inference_engine::Error) -> Self {
|
|
match e {
|
|
dr_inference_engine::Error::Inference(e) => PanoError::Inference(e),
|
|
dr_inference_engine::Error::Io(e) => PanoError::ModelRead(e),
|
|
}
|
|
}
|
|
}
|