dtourolle
6f33517b35
The merge weighted every overlap pixel by its distance from each frame's edge, a 200 px linear cross-fade. Anything the frames disagreed on — parallax in the near foreground, grass in the wind, a walker — came out twice at half strength: a soft double edge at 1:1. dr_pano::seam picks, per output texel at proxy resolution, which frame a pixel comes from. Where a new frame overlaps the composite the cost is the gain-corrected difference plus local detail plus nearness to either frame's edge, taken as the worst over a small window, and the cut is a dynamic-programming path across the overlap. merge.wgsl weights each frame by its tent-filtered share of that map, a 64 px blend that follows the seam, with the edge feather kept as the fallback. The page's preview uses the same map, and examples/merge.rs takes --feather-only for comparison.
90 lines
3.1 KiB
Rust
90 lines
3.1 KiB
Rust
//! TRACES: FR-MRG-1 | FR-MRG-10
|
|
//! Panorama geometry — from several frames to the rotations that relate
|
|
//! them, and the projections that lay them out.
|
|
//!
|
|
//! This is the CPU half of a merge (FR-MRG-10): keypoints, matching, the
|
|
//! rotation solve and the choice of output surface. The per-pixel half —
|
|
//! rendering, warping, seams, blending — is the GPU's and lives in
|
|
//! `dr-gpu`, driven from above; nothing here touches a full-resolution
|
|
//! pixel. The split is the whole design (panorama.md §4): everything in
|
|
//! this crate is bounded by the number of frames, not the size of the
|
|
//! composite, and runs on proxies.
|
|
//!
|
|
//! # Layout
|
|
//!
|
|
//! - [`image`] — the grayscale proxy a detector reads: oriented, resampled.
|
|
//! - [`features`] — keypoints with descriptors, and the XFeat decoder.
|
|
//! - [`xfeat`] — the network under tract (feature `xfeat`).
|
|
//! - [`matching`] — mutual nearest neighbours.
|
|
//! - [`homography`] — a robust pairwise homography, the focal length read
|
|
//! off it, and the rotation it implies.
|
|
//! - [`bundle`] — every rotation and the focal length refined together.
|
|
//! - [`align`] — the whole thing, from features to cameras, honest about
|
|
//! what it could not place.
|
|
//! - [`projection`] — perspective, cylindrical, spherical.
|
|
//! - [`seam`] — which frame each output pixel is taken from.
|
|
//! - [`linalg`] — the small dense algebra all of it uses.
|
|
//!
|
|
//! # What it depends on
|
|
//!
|
|
//! Nothing, without the `xfeat` feature: the geometry is pure Rust with
|
|
//! hand-rolled linear algebra (`linalg` says why) so that it tests without
|
|
//! a model, a GPU or a device, on synthetic sets whose answer is known
|
|
//! exactly. With the feature it adds the same `ort`-over-tract runtime the
|
|
//! rest of the application already carries.
|
|
|
|
pub mod align;
|
|
pub mod bundle;
|
|
pub mod features;
|
|
pub mod fill;
|
|
pub mod homography;
|
|
pub mod image;
|
|
pub mod linalg;
|
|
pub mod matching;
|
|
#[cfg(feature = "xfeat")]
|
|
pub mod migan;
|
|
pub mod projection;
|
|
pub mod seam;
|
|
#[cfg(feature = "xfeat")]
|
|
pub mod xfeat;
|
|
|
|
pub use align::{align, match_pairs, solve, AlignOptions, Alignment, Link, Pairs, Unaligned};
|
|
pub use bundle::Cameras;
|
|
pub use features::{Features, Keypoint};
|
|
pub use fill::{fill_border, Inpainter, Observer, Params as FillParams};
|
|
pub use image::Gray;
|
|
pub use projection::Projection;
|
|
pub use seam::{SeamMap, SeamOptions};
|
|
|
|
#[derive(Debug, thiserror::Error)]
|
|
pub enum PanoError {
|
|
#[error("bad input: {0}")]
|
|
Input(String),
|
|
#[error("geometry: {0}")]
|
|
Geometry(String),
|
|
#[error("model: {0}")]
|
|
Model(String),
|
|
#[error("could not read the model: {0}")]
|
|
ModelRead(#[source] std::io::Error),
|
|
#[cfg(feature = "xfeat")]
|
|
#[error("inference: {0}")]
|
|
Inference(#[source] ort::Error),
|
|
}
|
|
|
|
#[cfg(feature = "xfeat")]
|
|
impl From<ort::Error> for PanoError {
|
|
fn from(e: ort::Error) -> Self {
|
|
PanoError::Inference(e)
|
|
}
|
|
}
|
|
|
|
#[cfg(feature = "xfeat")]
|
|
impl From<dr_inference_engine::Error> for PanoError {
|
|
fn from(e: dr_inference_engine::Error) -> Self {
|
|
match e {
|
|
dr_inference_engine::Error::Inference(e) => PanoError::Inference(e),
|
|
dr_inference_engine::Error::Io(e) => PanoError::ModelRead(e),
|
|
}
|
|
}
|
|
}
|