The engine knew f32 and int8, and gave the Hexagon int8 for every role it
served. Measured on the tablet itself (inference.md §1.5), int8 lost
5% of the detector's faces at 40-80 px, moved the landmarks 1.5 px,
emptied the segmenter's scores and cost the denoiser 5-9 dB; fp16 the HTP
refuses outright. `Form` gains A16W8 and A16W16, and `Rung::form` now
names one per role: detectors and landmarks A16W8, the segmenter, scene
model, border filler and denoiser A16W16, XFeat int8. The embedder and
the eye classifiers stay on the CPU.
Each loader resolves its `<stem>.<form>.onnx` sibling; the segmenter and
XFeat, compiled into the binary, embed their quantised forms on Android
only and pick through `choose_embedded`. The probe, the compile step and
the cache fingerprint follow the form instead of assuming int8. Detectors
on the new form write `scrfd_*_a16+w600k_mbf`, and `model_ids` answers
for all three spellings.
On the tablet (ORT 1.29 + QNN 2.42), each shipped file against f32 on the
same inputs, and against the CPU's f32 time:
SCRFD 500m/2.5g/10g A16W8 100% of faces in every band 4.2/5.1/9.0 ms vs 17/56/198
landmarks A16W8 0.25 px in the 192 crop 0.5 ms vs 2.8
YOLO26n-seg A16W16 98.2% found, mask IoU 0.994 12.9 ms vs 90
scene model A16W16 98.9% of cells agree 15 ms vs 151
MI-GAN A16W16 41 dB from f32 in the fill 87 ms vs 488
XFeat int8 pano alignment 0.45 px (f32's own spread 0.41) 6.5 ms vs 58
denoiser A16W16 0.00 dB at every ISO 95 ms vs 1510 a tile
Face numbers are over public COCO val2017 photographs, not a library.
The APK carries the siblings (BUNDLED 15 -> 19; the old int8 detectors
removed), about 43 MB more. The Windows installer and its CI count skip
them; the Arch and Flatpak packages list their files and never had them.
The ladder example takes a role per model, which is how the per-role
forms above were seen landing on the NPU from the real probe.
The merge weighted every overlap pixel by its distance from each frame's
edge, a 200 px linear cross-fade. Anything the frames disagreed on —
parallax in the near foreground, grass in the wind, a walker — came out
twice at half strength: a soft double edge at 1:1.
dr_pano::seam picks, per output texel at proxy resolution, which frame a
pixel comes from. Where a new frame overlaps the composite the cost is the
gain-corrected difference plus local detail plus nearness to either
frame's edge, taken as the worst over a small window, and the cut is a
dynamic-programming path across the overlap. merge.wgsl weights each frame
by its tent-filtered share of that map, a 64 px blend that follows the
seam, with the edge feather kept as the fallback. The page's preview uses
the same map, and examples/merge.rs takes --feather-only for comparison.
A frame that did not fit ended the job with its name, and the only way
on was Back, a smaller selection and every frame read, demosaiced and
searched for keypoints again. Each row on the page now has a box. An
unticked frame is left out and the rest are solved again from what the
first pass measured.
dr_pano::align is split for it: match_pairs does the matching and the
pairwise RANSAC once over every frame (about 5 s for twelve), and solve
takes a subset of the frames and uses only the links among them (about
0.1 s). Solving a subset by re-aligning it also moved every RANSAC seed,
which are keyed on frame position, and on the fixture set that was
enough to lose a marginal link and strand a neighbour of the frame left
out.
A frame that cannot be placed no longer stops the job either. Its row
names it and Merge stays off until it is unticked; the headless example
leaves such frames out the same way, and takes --leave-out N to try it.
docs/ had 26 developer documents flat beside the manual, and the two
audiences are very differently sized: most readers want the manual and
the gesture reference, a few want the register, the designs and the
measurements. The manual and gestures.md stay at the top; everything for
someone changing the code moves to docs/dev/, and the two documents that
name their own successors — the v0.1 milestone and the UI-refinement plan
— go to docs/dev/archive/ rather than being deleted, since both are still
cited. docs/README.md is the index, users first.
Every reference follows: code comments, Cargo manifests, the workflows,
the pre-commit hook, the bench and traceability tools (which locate the
repo root by docs/dev/requirements.md now), packaging, the Docker READMEs,
CLAUDE.md, CONTRIBUTING.md and the README. The matrix links one level
deeper and is regenerated. Links out of the moved documents into the tree
gain a level; a link checker over every Markdown file finds none broken.
dr_pano::fill owns everything the model does not — which tiles, what
context, how to blend — behind an Inpainter trait, and dr_pano::migan is
that trait over the shipped generator on the inference engine.
The known content is mirrored across the coverage edge into the hole and
a 256-px ring, the nearest 48 px folded, so the model interpolates between
real and mirrored sky rather than extrapolating into nothing. A coarse
pass at a quarter decides the structure with the whole border in a few
tiles; fine passes in 96-px bands from the edge outward texture it; the
seam is blended over a feather inside the real edge. Every knob is a
Params field, and an Observer hears each stage for whoever is looking at
why a fill went wrong.
XFeat's two exports are a Keypoints role now; the crate no longer names
tract, and the app compiles TensorRT engines for both ahead of the
first merge. The probe picks the smallest *detector* rather than the
smallest file: the tablet's first run chose the 112 KB eye classifier,
which has no int8 form, and reported the Hexagon as failed for want of
one.
Twelve real frames from the fixture set now align in 4.5 s — 4.4 s of
matching, 118 ms of bundle adjustment — where the first run took 51 s and
left the first two frames out.
The matcher computes each pair's similarity matrix once, across the
cores, with a dot product written to vectorise; both nearest-neighbour
directions read it. The frames that failed were portrait: fitted into the
landscape input they used 512 of 1024 px, and their thin overlap did not
survive at half resolution. The same weights are now exported at 768×1024
as well and the detector picks the shape by aspect. The example aligns
from embedded previews and draws the set on a cylinder; on the fixture the
sweep is 152° at a fitted 47.9 mm against the EXIF's 50, RMS 1.5 px, and
the overlaps show no ghosting.
A new crate holding the CPU half of a merge (FR-MRG-10): the grayscale
proxy with orientation, the XFeat decoder ported step for step from the
reference detectAndCompute, mutual-nearest-neighbour matching, a robust
pairwise homography with the focal length read off it, a hand-rolled
Levenberg–Marquardt bundle adjustment over every rotation and the focal,
the three output projections, and align(), which chains it all and names
the frames it could not place rather than guessing (FR-MRG-5).
Dependency-free without the xfeat feature — linalg.rs says why the dense
algebra is hand-rolled — and tested on synthetic sweeps whose answer is
known exactly. The noise test records the single-row degeneracy: one
pixel of noise is a tenth of a percent of focal, which is a uniform
stretch of the sweep, not a misalignment.
dtourolle