dr-pano: the geometry, from features to cameras

A new crate holding the CPU half of a merge (FR-MRG-10): the grayscale
proxy with orientation, the XFeat decoder ported step for step from the
reference detectAndCompute, mutual-nearest-neighbour matching, a robust
pairwise homography with the focal length read off it, a hand-rolled
Levenberg–Marquardt bundle adjustment over every rotation and the focal,
the three output projections, and align(), which chains it all and names
the frames it could not place rather than guessing (FR-MRG-5).

Dependency-free without the xfeat feature — linalg.rs says why the dense
algebra is hand-rolled — and tested on synthetic sweeps whose answer is
known exactly. The noise test records the single-row degeneracy: one
pixel of noise is a tenth of a percent of focal, which is a uniform
stretch of the sweep, not a misalignment.

core/dr-pano/src/matching.rs

@@ -0,0 +1,142 @@
+//! Descriptor matching between two images.
+//!
+//! Mutual nearest neighbour on cosine similarity, with a floor on the
+//! similarity — the reference XFeat's own matcher (`match_mkpts`,
+//! `min_cossim = 0.82`). For a panorama that is enough: one lens, one
+//! scene, near-pure rotation and 20–40 % overlap make the matching problem
+//! easy, and what is hard — sky, repeated structure, exposure drift — is
+//! handled by the detector's descriptors and by RANSAC downstream, not by a
+//! cleverer matcher. A learned matcher (LightGlue) is the step after this
+//! one fails on a real set, and it has not (panorama.md §6).
+//!
+//! Brute force. `4096 × 4096 × 64` multiply-adds is a billion, which is
+//! tens of milliseconds a pair on one core, and there are at most a few
+//! dozen pairs. Not worth an index.
+
+use crate::features::{Features, DESCRIPTOR_LEN};
+
+/// A correspondence: keypoint `a` in the first image matches keypoint `b`
+/// in the second, with the cosine similarity of their descriptors.
+#[derive(Debug, Clone, Copy, PartialEq)]
+pub struct Match {
+    pub a: usize,
+    pub b: usize,
+    pub similarity: f32,
+}
+
+/// Match two sets of features.
+///
+/// A pair is kept when each is the other's nearest neighbour and their
+/// similarity is at least `min_similarity`.
+pub fn match_features(a: &Features, b: &Features, min_similarity: f32) -> Vec<Match> {
+    if a.is_empty() || b.is_empty() {
+        return Vec::new();
+    }
+    let best_ab = nearest(a, b);
+    let best_ba = nearest(b, a);
+    best_ab
+        .iter()
+        .enumerate()
+        .filter_map(|(ia, &(ib, sim))| {
+            (best_ba[ib].0 == ia && sim >= min_similarity).then_some(Match {
+                a: ia,
+                b: ib,
+                similarity: sim,
+            })
+        })
+        .collect()
+}
+
+/// For each descriptor in `from`, the index of its nearest in `to` and the
+/// similarity.
+fn nearest(from: &Features, to: &Features) -> Vec<(usize, f32)> {
+    (0..from.len())
+        .map(|i| {
+            let d = from.descriptor(i);
+            let mut best = (0usize, f32::MIN);
+            for j in 0..to.len() {
+                let s = dot(d, to.descriptor(j));
+                if s > best.1 {
+                    best = (j, s);
+                }
+            }
+            best
+        })
+        .collect()
+}
+
+#[inline]
+fn dot(a: &[f32], b: &[f32]) -> f32 {
+    // Written as a plain loop over a fixed length so the compiler
+    // vectorises it; the length is a constant and the slices are exact.
+    let mut s = 0.0f32;
+    for k in 0..DESCRIPTOR_LEN {
+        s += a[k] * b[k];
+    }
+    s
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    use crate::features::Keypoint;
+
+    /// Features whose descriptors are unit vectors along the given axes.
+    fn along(axes: &[usize]) -> Features {
+        let mut descriptors = vec![0.0; axes.len() * DESCRIPTOR_LEN];
+        for (i, &ax) in axes.iter().enumerate() {
+            descriptors[i * DESCRIPTOR_LEN + ax] = 1.0;
+        }
+        Features {
+            keypoints: axes
+                .iter()
+                .map(|_| Keypoint {
+                    x: 0.0,
+                    y: 0.0,
+                    score: 1.0,
+                })
+                .collect(),
+            descriptors,
+            width: 1,
+            height: 1,
+        }
+    }
+
+    #[test]
+    fn identical_descriptors_match_mutually() {
+        let a = along(&[0, 1, 2]);
+        let b = along(&[2, 0, 1]);
+        let m = match_features(&a, &b, 0.8);
+        let mut pairs: Vec<(usize, usize)> = m.iter().map(|m| (m.a, m.b)).collect();
+        pairs.sort();
+        assert_eq!(pairs, vec![(0, 1), (1, 2), (2, 0)]);
+        assert!(m.iter().all(|m| (m.similarity - 1.0).abs() < 1e-6));
+    }
+
+    #[test]
+    fn a_descriptor_with_no_counterpart_is_unmatched() {
+        let a = along(&[0, 1, 5]);
+        let b = along(&[0, 1]);
+        let m = match_features(&a, &b, 0.8);
+        assert_eq!(m.len(), 2);
+        assert!(m.iter().all(|m| m.a != 2));
+    }
+
+    #[test]
+    fn mutuality_breaks_a_one_sided_match() {
+        // b0 is the nearest to both a0 and a1, but a0 is its nearest — a1
+        // must not be matched to it.
+        let mut a = along(&[0, 0]);
+        a.descriptors[DESCRIPTOR_LEN] = 0.9;
+        a.descriptors[DESCRIPTOR_LEN + 1] = (1.0f32 - 0.81).sqrt();
+        let b = along(&[0]);
+        let m = match_features(&a, &b, 0.0);
+        assert_eq!(m.len(), 1);
+        assert_eq!((m[0].a, m[0].b), (0, 0));
+    }
+
+    #[test]
+    fn empty_input_is_empty_output() {
+        assert!(match_features(&along(&[]), &along(&[1]), 0.5).is_empty());
+    }
+}