dtourolle/DarkRoom
1
0
Fork 0
Code Issues 39 Pull Requests Actions 9 Packages Projects Releases 20 Wiki Activity

Cut panorama overlaps along seams instead of averaging them

Browse Source 
The merge weighted every overlap pixel by its distance from each frame's
edge, a 200 px linear cross-fade. Anything the frames disagreed on —
parallax in the near foreground, grass in the wind, a walker — came out
twice at half strength: a soft double edge at 1:1.

dr_pano::seam picks, per output texel at proxy resolution, which frame a
pixel comes from. Where a new frame overlaps the composite the cost is the
gain-corrected difference plus local detail plus nearness to either
frame's edge, taken as the worst over a small window, and the cut is a
dynamic-programming path across the overlap. merge.wgsl weights each frame
by its tent-filtered share of that map, a 64 px blend that follows the
seam, with the edge feather kept as the fallback. The page's preview uses
the same map, and examples/merge.rs takes --feather-only for comparison.
This commit is contained in:
dtourolle
2026-09-30 21:57:44 -04:00
parent 1d7115437b
commit 6f33517b35
8 changed files with 1012 additions and 32 deletions
Download Patch File Download Diff File Expand all files Collapse all files
core/dr-gpu/src/merge.rs
+109 -8
View File
@@ -30,18 +30,27 @@
//! are the caller's to provide and cache — `source` is asked for frame `k`
//! as it is needed, and a caller short of memory may demosaic on demand.
//!
//! # The blend
//!
//! With a seam map (`dr_pano::seam`), a frame's weight at a pixel is its
//! share of the map about that pixel — whole on its own side of a seam,
//! nothing on the other, and a ramp across a window `seam_blend` pixels
//! wide that follows the seam. Without one, or where the map has nothing
//! to say, the weight is the distance to the frame's edge over `feather`,
//! which hides exposure steps and does not hide parallax: the average draws
//! anything the frames disagree on twice.
//!
//! # What is not here yet
//!
//! A feathered blend, not seams and a Laplacian pyramid: the weight is the
//! distance to the frame's edge, which hides exposure steps and small
//! misalignments and does not hide parallax. Gain is a scalar per frame
//! the caller supplies. Both are panorama.md §10's step 5, after the path
//! writes a file end to end.
//! A Laplacian pyramid, which would let the seam's blend be narrow for
//! detail and wide for exposure at once. Gain is a scalar per frame the
//! caller supplies.
use std::sync::Arc;
use dr_pano::bundle::Cameras;
use dr_pano::projection::{Bounds, Projection};
use dr_pano::seam::SeamMap;
use wgpu::util::DeviceExt;
use crate::readback::await_mapping;
@@ -58,7 +67,7 @@ pub struct MergeFrame {
}
/// The output the merge produces.
#[derive(Debug, Clone, Copy, PartialEq)]
#[derive(Debug, Clone, PartialEq)]
pub struct MergeOutput {
pub projection: Projection,
/// The projection's scale in output pixels: the cylinder's radius, the
@@ -69,6 +78,11 @@ pub struct MergeOutput {
pub bounds: Bounds,
/// Pixels over which a frame's weight ramps up from its edge.
pub feather: f32,
/// Which frame each part of the output is taken from, laid out at the
/// proxies' scale; `None` for the feathered average everywhere.
pub seams: Option<Arc<SeamMap>>,
/// The width, in output pixels, of the blend across a seam.
pub seam_blend: f32,
/// Chunk size: the unit of GPU work and of memory.
pub chunk: (u32, u32),
/// Multiplies a normalised sample (1.0 = white) to the sensor's scale.
@@ -115,6 +129,12 @@ struct WarpParams {
feather: f32,
clip_onset: f32,
balance: [f32; 4],
seam_origin: [f32; 2],
seam_size: [u32; 2],
seam_px: f32,
seam_radius: f32,
frame_index: u32,
seam_on: u32,
}
#[repr(C)]
@@ -182,6 +202,16 @@ impl MergePass {
count: None,
},
storage(2, false),
wgpu::BindGroupLayoutEntry {
binding: 3,
visibility: wgpu::ShaderStages::COMPUTE,
ty: wgpu::BindingType::Texture {
sample_type: wgpu::TextureSampleType::Uint,
view_dimension: wgpu::TextureViewDimension::D2,
multisampled: false,
},
count: None,
},
],
});
let resolve_layout =
@@ -279,6 +309,32 @@ impl MergePass {
let mut band_cov = vec![false; (out_w * ch) as usize];
let mut chunk_px: Vec<u32> = Vec::new();
// The seam map, once for the whole output, and where it sits in
// this output's coordinates. A one-texel stand-in when there is
// none, because the binding is not optional.
let (seam_tex, seam_origin, seam_px, seam_radius, seam_size) = match &output.seams {
Some(m) => {
let ((ou, ov), px) = m.at_scale(output.scale);
let radius = m.blend_radius(output.scale, f64::from(output.seam_blend));
(
self.label_texture(m.width as u32, m.height as u32, &m.labels),
[ou as f32, ov as f32],
px as f32,
radius as f32,
[m.width as u32, m.height as u32],
)
}
None => (
self.label_texture(1, 1, &[dr_pano::seam::NONE]),
[0.0; 2],
1.0,
1.0,
[1, 1],
),
};
let seam_view = seam_tex.create_view(&Default::default());
let seam_on = u32::from(output.seams.is_some());
let mut y = 0u32;
while y < out_h {
let rows = ch.min(out_h - y);
@@ -346,8 +402,14 @@ impl MergePass {
output.balance[2].max(1e-3),
0.0,
],
seam_origin,
seam_size,
seam_px,
seam_radius,
frame_index: k as u32,
seam_on,
};
self.accumulate(&params, tile);
self.accumulate(&params, tile, &seam_view);
}
self.resolve_chunk((cols, rows), output.sample_scale, &mut chunk_px)?;
@@ -385,7 +447,42 @@ impl MergePass {
self.ctx.queue.submit(Some(enc.finish()));
}
fn accumulate(&mut self, params: &WarpParams, tile: &wgpu::Texture) {
/// The seam map's labels as an `r8uint` texture.
fn label_texture(&self, width: u32, height: u32, labels: &[u8]) -> wgpu::Texture {
let size = wgpu::Extent3d {
width,
height,
depth_or_array_layers: 1,
};
let tex = self.ctx.device.create_texture(&wgpu::TextureDescriptor {
label: Some("merge-seams"),
size,
mip_level_count: 1,
sample_count: 1,
dimension: wgpu::TextureDimension::D2,
format: wgpu::TextureFormat::R8Uint,
usage: wgpu::TextureUsages::TEXTURE_BINDING | wgpu::TextureUsages::COPY_DST,
view_formats: &[],
});
self.ctx.queue.write_texture(
wgpu::TexelCopyTextureInfo {
texture: &tex,
mip_level: 0,
origin: wgpu::Origin3d::ZERO,
aspect: wgpu::TextureAspect::All,
},
labels,
wgpu::TexelCopyBufferLayout {
offset: 0,
bytes_per_row: Some(width),
rows_per_image: Some(height),
},
size,
);
tex
}
fn accumulate(&mut self, params: &WarpParams, tile: &wgpu::Texture, seams: &wgpu::TextureView) {
let chunk = (params.chunk_size[0], params.chunk_size[1]);
let uniforms = self
.ctx
@@ -417,6 +514,10 @@ impl MergePass {
binding: 2,
resource: acc.as_entire_binding(),
},
wgpu::BindGroupEntry {
binding: 3,
resource: wgpu::BindingResource::TextureView(seams),
},
],
});
let mut enc = self.ctx.device.create_command_encoder(&Default::default());