dtourolle
05508741af
The desktop names where a package may have put libonnxruntime — an override variable, beside the executable, the package's own library directory, the Flatpak prefix, the system library directory — and Android points at the APK's native library directory, which is also what Qualcomm's DSP loader must be told for the Hexagon skel. Android starts the engine at the end of the model unpack rather than at launch, because the probe fingerprints the model files and a first launch has none until then. The About panel gains an Inference row beside Graphics, re-read every two seconds while the probe runs and engines land, and faces.model_id carries the detector's form: an int8 detector finds a different set of faces and is a different population (docs/inference.md §7). A low-memory signal drops every idle session with the GPU caches. The APK assembly bundles ONNX Runtime and the Qualcomm HTP libraries from Maven, fetched by tools/fetch-android-runtime.sh with their published checksums; RUNTIME_DIR=none builds the tract-only APK, which is a slower app and not a broken one. The desktop packages carry no runtime yet. Two probe fixes from the first desktop run: the floor must not be built with CPU fallback disabled, and a versioned libonnxruntime.so is a runtime too. On the reference desktop the probe now loads ONNX Runtime 1.30, measures 30 ms on the CPU provider, and selects TensorRT at 1.5 ms.
97 lines
4.6 KiB
Bash
Executable File
97 lines
4.6 KiB
Bash
Executable File
#!/usr/bin/env bash
|
||
# Fetch the inference runtime the Android APK carries (docs/inference.md §3).
|
||
#
|
||
# ./tools/fetch-android-runtime.sh [DEST]
|
||
#
|
||
# Two Maven artefacts, pinned to each other by ONNX Runtime's own POM:
|
||
#
|
||
# com.microsoft.onnxruntime:onnxruntime-android-qnn MIT
|
||
# com.qualcomm.qti:qnn-runtime Qualcomm AI Engine Direct SDK licence
|
||
#
|
||
# The first is ONNX Runtime built with the CPU, QNN, XNNPACK, NNAPI and WebGPU
|
||
# providers; the second is Qualcomm's HTP backend — the ARM-side compiler and
|
||
# the per-generation Hexagon "skel" the DSP loads. Both ship as AARs whose
|
||
# `jni/arm64-v8a/` is what an APK's `lib/arm64-v8a/` wants, so this script
|
||
# unpacks exactly that and nothing else, plus the licence texts, which travel
|
||
# with the libraries (§3.1).
|
||
#
|
||
# ## What is and is not taken from the Qualcomm package
|
||
#
|
||
# Every HTP generation has its own skel and stub, 16 MB a pair. `QNN_HTP_ARCHS`
|
||
# names the ones to ship; the default is the generations in devices sold since
|
||
# the 8 Gen 2 (V73 — the MagicPad 2's 8s Gen 3 is one), through the 8 Elite
|
||
# (V79) and its successor (V81). V66–V69 are 2020–2021 silicon and are left
|
||
# out, which is 50 MB the tablet never loads. `libQnnHtpPrepare.so`, the
|
||
# on-device graph compiler, is 84 MB and cannot be left out: it is what turns
|
||
# an int8 ONNX graph into something the DSP runs, once per device (§5).
|
||
#
|
||
# `libQnnGpu.so` and `libQnnDsp*.so` are not taken: the Adreno rung measured
|
||
# slower than the Hexagon everywhere the Hexagon exists (§1.1), and the cDSP
|
||
# path is the pre-HTP generation.
|
||
#
|
||
# ## Why a script and not a checked-in copy
|
||
#
|
||
# 150 MB of vendor binaries in LFS, re-fetched by every clone, for files that
|
||
# Maven serves with checksums. The cache directory keeps them across builds;
|
||
# CI's is the target cache it already mounts.
|
||
set -euo pipefail
|
||
|
||
ORT_VERSION="${ORT_VERSION:-1.29.0}"
|
||
QNN_VERSION="${QNN_VERSION:-2.42.0}"
|
||
QNN_HTP_ARCHS="${QNN_HTP_ARCHS:-73 75 79 81}"
|
||
DEST="${1:-${PWD}/target-android/runtime}"
|
||
MAVEN="https://repo1.maven.org/maven2"
|
||
|
||
fetch() {
|
||
# A Maven artefact and its SHA-1, verified before anything is unpacked.
|
||
local group="$1" artefact="$2" version="$3" out="$4"
|
||
local url="${MAVEN}/${group//.//}/${artefact}/${version}/${artefact}-${version}.aar"
|
||
if [[ -f "${out}" && -f "${out}.sha1" ]] \
|
||
&& [[ "$(sha1sum "${out}" | cut -d' ' -f1)" == "$(cut -c1-40 "${out}.sha1")" ]]; then
|
||
return
|
||
fi
|
||
echo "==> fetching ${artefact} ${version}"
|
||
curl -fsSL -o "${out}.sha1" "${url}.sha1"
|
||
curl -fsSL -o "${out}" "${url}"
|
||
[[ "$(sha1sum "${out}" | cut -d' ' -f1)" == "$(cut -c1-40 "${out}.sha1")" ]] || {
|
||
echo "error: ${artefact}-${version}.aar does not match its published SHA-1" >&2
|
||
rm -f "${out}"
|
||
exit 1
|
||
}
|
||
}
|
||
|
||
mkdir -p "${DEST}/aar" "${DEST}/lib"
|
||
fetch com.microsoft.onnxruntime onnxruntime-android-qnn "${ORT_VERSION}" \
|
||
"${DEST}/aar/onnxruntime-android-qnn-${ORT_VERSION}.aar"
|
||
fetch com.qualcomm.qti qnn-runtime "${QNN_VERSION}" \
|
||
"${DEST}/aar/qnn-runtime-${QNN_VERSION}.aar"
|
||
|
||
# The ONNX Runtime POM names the QNN version it was built against; a pair
|
||
# that disagrees loads and then fails at the first graph, which is the kind
|
||
# of failure the probe would only report as "Hexagon failed".
|
||
pom="${DEST}/aar/onnxruntime-android-qnn-${ORT_VERSION}.pom"
|
||
[[ -f "${pom}" ]] || curl -fsSL -o "${pom}" \
|
||
"${MAVEN}/com/microsoft/onnxruntime/onnxruntime-android-qnn/${ORT_VERSION}/onnxruntime-android-qnn-${ORT_VERSION}.pom"
|
||
wanted="$(sed -n '/<artifactId>qnn-runtime<\/artifactId>/{n;s/.*<version>\(.*\)<\/version>.*/\1/p}' "${pom}")"
|
||
if [[ -n "${wanted}" && "${wanted}" != "${QNN_VERSION}" ]]; then
|
||
echo "error: ONNX Runtime ${ORT_VERSION} was built against QNN ${wanted}, not ${QNN_VERSION}" >&2
|
||
exit 1
|
||
fi
|
||
|
||
rm -rf "${DEST}/lib"
|
||
mkdir -p "${DEST}/lib"
|
||
unzip -q -o -j "${DEST}/aar/onnxruntime-android-qnn-${ORT_VERSION}.aar" \
|
||
'jni/arm64-v8a/libonnxruntime.so' -d "${DEST}/lib"
|
||
members=(jni/arm64-v8a/libQnnHtp.so jni/arm64-v8a/libQnnHtpPrepare.so jni/arm64-v8a/libQnnSystem.so)
|
||
for arch in ${QNN_HTP_ARCHS}; do
|
||
members+=("jni/arm64-v8a/libQnnHtpV${arch}Skel.so" "jni/arm64-v8a/libQnnHtpV${arch}Stub.so")
|
||
done
|
||
unzip -q -o -j "${DEST}/aar/qnn-runtime-${QNN_VERSION}.aar" "${members[@]}" -d "${DEST}/lib"
|
||
unzip -q -o -j "${DEST}/aar/qnn-runtime-${QNN_VERSION}.aar" 'LICENSE.pdf' 'NOTICE.txt' -d "${DEST}" 2>/dev/null || true
|
||
mv -f "${DEST}/LICENSE.pdf" "${DEST}/QNN-LICENSE.pdf" 2>/dev/null || true
|
||
mv -f "${DEST}/NOTICE.txt" "${DEST}/QNN-NOTICE.txt" 2>/dev/null || true
|
||
|
||
echo "==> runtime in ${DEST}/lib:"
|
||
du -sh "${DEST}/lib" | cut -f1 | sed 's/^/ /'
|
||
ls "${DEST}/lib" | sed 's/^/ /'
|