dr-pano: a second XFeat shape for portrait frames, and a matcher that takes seconds

Twelve real frames from the fixture set now align in 4.5 s — 4.4 s of
matching, 118 ms of bundle adjustment — where the first run took 51 s and
left the first two frames out.

The matcher computes each pair's similarity matrix once, across the
cores, with a dot product written to vectorise; both nearest-neighbour
directions read it. The frames that failed were portrait: fitted into the
landscape input they used 512 of 1024 px, and their thin overlap did not
survive at half resolution. The same weights are now exported at 768×1024
as well and the detector picks the shape by aspect. The example aligns
from embedded previews and draws the set on a cylinder; on the fixture the
sweep is 152° at a fitted 47.9 mm against the EXIF's 50, RMS 1.5 px, and
the overlaps show no ghosting.
This commit is contained in:
2026-09-19 15:24:12 +02:00
parent 231b4a54ab
commit 54290b9540
9 changed files with 358 additions and 103 deletions
+16 -11
View File
@@ -6,8 +6,12 @@
# someone once produced and nobody can regenerate. Run it when bumping the
# model or changing its input size.
#
# ./tools/export-xfeat.sh # 768×1024 -> models/keypoints/xfeat-1024.onnx
# ./tools/export-xfeat.sh 576 768 # another fixed size
# ./tools/export-xfeat.sh # both shapes -> models/keypoints/xfeat-{1024,768}.onnx
#
# Two files from one set of weights: 1024 wide × 768 tall for landscape
# frames and 768 × 1024 for portrait, chosen by the frame's aspect at run
# time. One landscape file would fit a portrait frame at half the width and
# waste half the detector on padding, which is what the 6D fixture did.
#
# Requires `uv`. Everything else is fetched into a throwaway venv, including
# a CPU-only torch — the export needs no GPU and the CUDA wheels are 2 GB.
@@ -37,23 +41,25 @@ set -euo pipefail
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO="$(cd "${HERE}/.." && pwd)"
H="${1:-768}"
W="${2:-1024}"
OUT="${REPO}/models/keypoints"
NAME="xfeat-${W}"
# Not `mktemp -d` under /tmp: a tmpfs, and torch is a gigabyte.
WORK="$(mktemp -d -p "${TMPDIR:-/var/tmp}")"
trap 'rm -rf "${WORK}"' EXIT
echo "==> exporting XFeat at ${H}×${W} in ${WORK}"
echo "==> exporting XFeat in ${WORK}"
cd "${WORK}"
git clone -q --depth 1 https://github.com/verlab/accelerated_features.git xfeat
uv venv -q --python 3.12 venv
VIRTUAL_ENV="${WORK}/venv" uv pip install -q --index-url https://download.pytorch.org/whl/cpu torch
VIRTUAL_ENV="${WORK}/venv" uv pip install -q onnx onnxslim
VIRTUAL_ENV="${WORK}/venv" "${WORK}/venv/bin/python" - "${WORK}/xfeat" "${WORK}/${NAME}.onnx" "${H}" "${W}" <<'PY'
mkdir -p "${OUT}"
for shape in "768 1024" "1024 768"; do
set -- $shape
H="$1"; W="$2"
NAME="xfeat-${W}"
VIRTUAL_ENV="${WORK}/venv" "${WORK}/venv/bin/python" - "${WORK}/xfeat" "${WORK}/${NAME}.onnx" "${H}" "${W}" <<'PY'
import sys, torch, onnx, onnxslim
sys.path.insert(0, sys.argv[1])
from modules.model import XFeatModel
@@ -72,7 +78,6 @@ print("ops:", sorted({n.op_type for n in m.graph.node}))
for o in m.graph.output:
print("out", o.name, [d.dim_value for d in o.type.tensor_type.shape.dim])
PY
mkdir -p "${OUT}"
cp "${WORK}/${NAME}.onnx" "${OUT}/${NAME}.onnx"
echo "==> ${OUT}/${NAME}.onnx"
cp "${WORK}/${NAME}.onnx" "${OUT}/${NAME}.onnx"
echo "==> ${OUT}/${NAME}.onnx"
done