Leave a panorama frame out without leaving the page

A frame that did not fit ended the job with its name, and the only way
on was Back, a smaller selection and every frame read, demosaiced and
searched for keypoints again. Each row on the page now has a box. An
unticked frame is left out and the rest are solved again from what the
first pass measured.

dr_pano::align is split for it: match_pairs does the matching and the
pairwise RANSAC once over every frame (about 5 s for twelve), and solve
takes a subset of the frames and uses only the links among them (about
0.1 s). Solving a subset by re-aligning it also moved every RANSAC seed,
which are keyed on frame position, and on the fixture set that was
enough to lose a marginal link and strand a neighbour of the frame left
out.

A frame that cannot be placed no longer stops the job either. Its row
names it and Merge stays off until it is unticked; the headless example
leaves such frames out the same way, and takes --leave-out N to try it.

core/dr-pano/src/align.rs

@@ -12,6 +12,11 @@
 //!    best-connected frame; rotations chained along it.
 //! 5. Bundle adjustment over every link's inliers (`bundle`).
 //!
+//! Steps 1 and 2 are [`match_pairs`] and most of the time; 3 to 5 are
+//! [`solve`], which takes a subset of the frames. Leaving a frame out is
+//! then a solve over the pairs already measured — the same links, not a
+//! fresh RANSAC whose seeds would move with the frames' positions.
+//!
 //! What it refuses to do is guess. A frame the tree does not reach is
 //! reported by index with the reason (FR-MRG-5) and left out of the
 //! cameras; the caller decides whether a set with a hole is worth
@@ -125,13 +130,45 @@ impl Alignment {
     }
 }
 
-/// Align a set of frames from their features.
+/// Every pair of a set measured: steps 1 and 2, the expensive part, kept
+/// so that a solve over a subset reuses it.
+#[derive(Debug, Clone, PartialEq)]
+pub struct Pairs {
+    /// Each frame's long edge, for the focal length's clamp.
+    long_edges: Vec<f64>,
+    /// Pairs with enough matches to try a geometry, whether or not it held.
+    matched: Vec<(usize, usize)>,
+    links: Vec<Link>,
+    /// Every link's inliers, in pixels, centred.
+    observations: Vec<Observation>,
+}
+
+impl Pairs {
+    /// How many frames were measured.
+    pub fn len(&self) -> usize {
+        self.long_edges.len()
+    }
+
+    pub fn is_empty(&self) -> bool {
+        self.long_edges.is_empty()
+    }
+}
+
+/// Align a set of frames from their features: [`match_pairs`], then
+/// [`solve`] over all of them.
 ///
 /// Every `Features` must be in its own frame's pixel coordinates with the
 /// image size filled in; points are centred on the image centre here. The
 /// frames must all come from the same lens at the same focal length, which
 /// is the panorama assumption and not checked — the caller has the EXIF.
 pub fn align(frames: &[Features], opts: &AlignOptions) -> Result<Alignment, PanoError> {
+    let pairs = match_pairs(frames, opts)?;
+    solve(&pairs, &vec![true; frames.len()], opts)
+}
+
+/// Steps 1 and 2: every pair matched, and a robust homography for each
+/// pair with enough matches.
+pub fn match_pairs(frames: &[Features], opts: &AlignOptions) -> Result<Pairs, PanoError> {
     let n = frames.len();
     if n < 2 {
         return Err(PanoError::Input(
@@ -156,7 +193,7 @@ pub fn align(frames: &[Features], opts: &AlignOptions) -> Result<Alignment, Pano
     // 1 + 2: every pair.
     let mut links = Vec::new();
     let mut observations: Vec<Observation> = Vec::new();
-    let mut matched_any = vec![false; n];
+    let mut matched = Vec::new();
     let t_match = std::time::Instant::now();
     for i in 0..n {
         for j in i + 1..n {
@@ -165,8 +202,7 @@ pub fn align(frames: &[Features], opts: &AlignOptions) -> Result<Alignment, Pano
             if matches.len() < 4 {
                 continue;
             }
-            matched_any[i] = true;
-            matched_any[j] = true;
+            matched.push((i, j));
             let pairs: Vec<((f64, f64), (f64, f64))> = matches
                 .iter()
                 .map(|m| {
@@ -214,6 +250,69 @@ pub fn align(frames: &[Features], opts: &AlignOptions) -> Result<Alignment, Pano
     }
 
     log::debug!("matching and pairwise geometry in {:?}", t_match.elapsed());
+    Ok(Pairs {
+        long_edges: frames
+            .iter()
+            .map(|f| f.width.max(f.height) as f64)
+            .collect(),
+        matched,
+        links,
+        observations,
+    })
+}
+
+/// Steps 3 to 5 over the frames `keep` marks, from pairs already measured.
+///
+/// The result is indexed by the kept frames in order: its frame `k` is the
+/// `k`-th frame `keep` marks. Only pairs whose frames are both kept take
+/// part, so a frame whose only overlap was with one left out is reported
+/// as unaligned, as it would be had it never been measured with it.
+pub fn solve(pairs: &Pairs, keep: &[bool], opts: &AlignOptions) -> Result<Alignment, PanoError> {
+    if keep.len() != pairs.len() {
+        return Err(PanoError::Input(format!(
+            "{} flags for {} frames",
+            keep.len(),
+            pairs.len()
+        )));
+    }
+    // Input index to the solve's.
+    let mut slot = vec![None; keep.len()];
+    let mut n = 0usize;
+    for (k, &kept) in keep.iter().enumerate() {
+        if kept {
+            slot[k] = Some(n);
+            n += 1;
+        }
+    }
+    if n < 2 {
+        return Err(PanoError::Input(
+            "a panorama needs at least two frames".into(),
+        ));
+    }
+    let both = |i: usize, j: usize| Some((slot[i]?, slot[j]?));
+    let mut matched_any = vec![false; n];
+    for &(i, j) in &pairs.matched {
+        if let Some((i, j)) = both(i, j) {
+            matched_any[i] = true;
+            matched_any[j] = true;
+        }
+    }
+    let links: Vec<Link> = pairs
+        .links
+        .iter()
+        .filter_map(|l| {
+            let (i, j) = both(l.i, l.j)?;
+            Some(Link { i, j, ..l.clone() })
+        })
+        .collect();
+    let observations: Vec<Observation> = pairs
+        .observations
+        .iter()
+        .filter_map(|o| {
+            let (i, j) = both(o.i, o.j)?;
+            Some(Observation { i, j, ..*o })
+        })
+        .collect();
 
     // 3: the focal length.
     let mut estimates: Vec<f64> = links
@@ -221,9 +320,12 @@ pub fn align(frames: &[Features], opts: &AlignOptions) -> Result<Alignment, Pano
         .filter_map(|l| homography::focal_from_homography(&l.h))
         .filter(|f| f.is_finite() && *f > 0.0)
         .collect();
-    let longest = frames
+    let longest = pairs
+        .long_edges
         .iter()
-        .map(|f| f.width.max(f.height) as f64)
+        .zip(keep)
+        .filter(|(_, &kept)| kept)
+        .map(|(&e, _)| e)
         .fold(0.0, f64::max);
     let focal = if !estimates.is_empty() {
         estimates.sort_by(f64::total_cmp);
@@ -448,6 +550,40 @@ mod tests {
         assert!(out.rotations[..3].iter().all(Option::is_some));
     }
 
+    #[test]
+    fn a_frame_left_out_is_solved_without_measuring_again() {
+        let (frames, truth) = synthetic_sweep(6, 0.3, 1400.0, 1024, 768);
+        let opts = AlignOptions::default();
+        let pairs = match_pairs(&frames, &opts).expect("measured");
+        // The first frame left out: five cameras, indexed as the kept
+        // frames, and the links among them only.
+        let keep = [false, true, true, true, true, true];
+        let out = solve(&pairs, &keep, &opts).expect("solved");
+        assert!(out.is_complete(), "unaligned: {:?}", out.unaligned);
+        assert_eq!(out.rotations.len(), 5);
+        assert_eq!(out.links.len(), 4 + 3, "links: {}", out.links.len());
+        let root = out
+            .rotations
+            .iter()
+            .position(|r| *r == Some(Mat3::IDENTITY))
+            .unwrap();
+        for k in 0..5 {
+            let rel_truth = truth.rotations[root + 1].transpose() * truth.rotations[k + 1];
+            let err = angle_between(rel_truth, out.rotations[k].unwrap());
+            assert!(err < 2e-3, "frame {k} off by {err} rad");
+        }
+        // A frame in the middle left out splits the sweep only if nothing
+        // spans the gap; at 0.3 rad steps its neighbours still overlap.
+        let keep = [true, true, false, true, true, true];
+        let out = solve(&pairs, &keep, &opts).expect("solved");
+        assert!(out.is_complete(), "unaligned: {:?}", out.unaligned);
+        // And the whole set solved from the pairs is `align`'s answer.
+        assert_eq!(
+            solve(&pairs, &[true; 6], &opts).expect("solved"),
+            align(&frames, &opts).expect("aligned")
+        );
+    }
+
     #[test]
     fn one_frame_is_refused() {
         let (frames, _) = synthetic_sweep(1, 0.3, 1400.0, 640, 480);

core/dr-pano/src/lib.rs

@@ -46,7 +46,7 @@ pub mod projection;
 #[cfg(feature = "xfeat")]
 pub mod xfeat;
 
-pub use align::{align, AlignOptions, Alignment, Link, Unaligned};
+pub use align::{align, match_pairs, solve, AlignOptions, Alignment, Link, Pairs, Unaligned};
 pub use bundle::Cameras;
 pub use features::{Features, Keypoint};
 pub use fill::{fill_border, Inpainter, Observer, Params as FillParams};