Add the int8 detectors for the Hexagon, calibrated on real photographs

tools/quantise-models.sh writes the QDQ form QNN's HTP backend takes
whole: opset 17, per-channel int8 weights, uint8 activations, ranges
from running the f32 graph over photographs fed exactly as the app
feeds them. The calibration is strided, four images at a time, because
every ONNX Runtime calibrator holds each image's whole set of
activations until it folds them — a gigabyte an image on the 10g
detector, and an OOM kill with no message when folded once at the end.

Release-time, never on the device (docs/inference.md §5): it needs
real photographs and a person reading the recall measurement that
gates whether each file is offered.
This commit is contained in:
2026-09-19 16:02:37 +02:00
parent 05508741af
commit 4ed29b9d81
6 changed files with 194 additions and 0 deletions
+35
View File
@@ -0,0 +1,35 @@
#!/usr/bin/env bash
# Produce the int8 form of a model for the Hexagon (docs/inference.md §5).
#
# ./tools/quantise-models.sh PHOTO_DIR MODEL.onnx [MODEL.onnx ...]
#
# Writes `MODEL.int8.onnx` beside each input: a QDQ graph, per-channel int8
# weights, uint8 activations — the form QNN's HTP backend takes whole. The
# activations' ranges come from running the f32 model over the photographs in
# PHOTO_DIR, fed exactly as the app feeds them (letterboxed to the model's
# input, the detector's `(x - 127.5) / 128` normalisation), which is why
# this is a release-time step and not something the device does: it needs
# real photographs and, after it, a person reading §10 M2's numbers.
#
# The SCRFD and ArcFace exports are opset 11; per-channel QDQ needs 13, so a
# model below 13 is first upgraded to 17. That changes only the graph's
# spelling, not a weight — and it is what `tools/fix-face-model-shapes.sh`
# will do to the canonical files in the same model release.
#
# A venv per run, like fix-face-model-shapes.sh: the tools are not a build
# input and nothing in the tree should have them on its path.
set -euo pipefail
if [ "$#" -lt 2 ]; then
sed -n '2,20p' "$0" >&2
exit 2
fi
PHOTOS="$1"; shift
[ -d "${PHOTOS}" ] || { echo "no such directory: ${PHOTOS}" >&2; exit 1; }
WORK="$(mktemp -d -p /var/tmp quantise-models.XXXXXX)"
trap 'rm -rf "${WORK}"' EXIT
echo "==> venv in ${WORK}"
uv venv --python 3.12 "${WORK}/venv" >/dev/null
VIRTUAL_ENV="${WORK}/venv" uv pip install --quiet onnx onnxruntime pillow numpy sympy
exec "${WORK}/venv/bin/python" "$(dirname "$0")/quantise-models.py" "${PHOTOS}" "$@"