Add the int8 detectors for the Hexagon, calibrated on real photographs
tools/quantise-models.sh writes the QDQ form QNN's HTP backend takes whole: opset 17, per-channel int8 weights, uint8 activations, ranges from running the f32 graph over photographs fed exactly as the app feeds them. The calibration is strided, four images at a time, because every ONNX Runtime calibrator holds each image's whole set of activations until it folds them — a gigabyte an image on the 10g detector, and an OOM kill with no message when folded once at the end. Release-time, never on the device (docs/inference.md §5): it needs real photographs and a person reading the recall measurement that gates whether each file is offered.
This commit is contained in:
Executable
+35
@@ -0,0 +1,35 @@
|
||||
#!/usr/bin/env bash
|
||||
# Produce the int8 form of a model for the Hexagon (docs/inference.md §5).
|
||||
#
|
||||
# ./tools/quantise-models.sh PHOTO_DIR MODEL.onnx [MODEL.onnx ...]
|
||||
#
|
||||
# Writes `MODEL.int8.onnx` beside each input: a QDQ graph, per-channel int8
|
||||
# weights, uint8 activations — the form QNN's HTP backend takes whole. The
|
||||
# activations' ranges come from running the f32 model over the photographs in
|
||||
# PHOTO_DIR, fed exactly as the app feeds them (letterboxed to the model's
|
||||
# input, the detector's `(x - 127.5) / 128` normalisation), which is why
|
||||
# this is a release-time step and not something the device does: it needs
|
||||
# real photographs and, after it, a person reading §10 M2's numbers.
|
||||
#
|
||||
# The SCRFD and ArcFace exports are opset 11; per-channel QDQ needs 13, so a
|
||||
# model below 13 is first upgraded to 17. That changes only the graph's
|
||||
# spelling, not a weight — and it is what `tools/fix-face-model-shapes.sh`
|
||||
# will do to the canonical files in the same model release.
|
||||
#
|
||||
# A venv per run, like fix-face-model-shapes.sh: the tools are not a build
|
||||
# input and nothing in the tree should have them on its path.
|
||||
set -euo pipefail
|
||||
if [ "$#" -lt 2 ]; then
|
||||
sed -n '2,20p' "$0" >&2
|
||||
exit 2
|
||||
fi
|
||||
PHOTOS="$1"; shift
|
||||
[ -d "${PHOTOS}" ] || { echo "no such directory: ${PHOTOS}" >&2; exit 1; }
|
||||
|
||||
WORK="$(mktemp -d -p /var/tmp quantise-models.XXXXXX)"
|
||||
trap 'rm -rf "${WORK}"' EXIT
|
||||
echo "==> venv in ${WORK}"
|
||||
uv venv --python 3.12 "${WORK}/venv" >/dev/null
|
||||
VIRTUAL_ENV="${WORK}/venv" uv pip install --quiet onnx onnxruntime pillow numpy sympy
|
||||
|
||||
exec "${WORK}/venv/bin/python" "$(dirname "$0")/quantise-models.py" "${PHOTOS}" "$@"
|
||||
Reference in New Issue
Block a user