Search the user's own runtime directory before the system library
The reference desktop's only system ONNX Runtime is Arch's onnxruntime-opt-cuda: 1.29, built without TensorRT and against cuDNN 8 on a cuDNN 9 machine. The probe rejects both providers correctly and the app runs on the CPU provider, which is right and not what anyone wants. runtime/ beside the models is now searched ahead of /usr/lib, tools/fetch-desktop-runtime.sh fills it with the four libraries from the current onnxruntime-gpu wheel (cuDNN 9, TensorRT 10), and the About caption lists every rung that lost and why, not only the first. Verified: the app selects TensorRT from that directory with no environment variable set.
This commit is contained in:
Executable
+32
@@ -0,0 +1,32 @@
|
||||
#!/usr/bin/env bash
|
||||
# Put a GPU-capable ONNX Runtime where the desktop app looks for one
|
||||
# (docs/inference.md §3): `runtime/` beside the models in the user data
|
||||
# directory, ahead of the system library.
|
||||
#
|
||||
# ./tools/fetch-desktop-runtime.sh [DEST]
|
||||
#
|
||||
# The source is the `onnxruntime-gpu` wheel: the one build that carries the
|
||||
# CUDA *and* TensorRT providers against the cuDNN and TensorRT majors current
|
||||
# on this machine. Distribution packages tend to have neither — Arch's
|
||||
# `onnxruntime-opt-cuda` is built without TensorRT and against cuDNN 8 — and
|
||||
# the probe rejects them correctly and leaves the app on the CPU provider,
|
||||
# which is what this script exists to fix. Nothing NVIDIA is bundled here:
|
||||
# the providers load CUDA, cuDNN and TensorRT from the system, and if those
|
||||
# are missing the probe says so and the app stays on the CPU.
|
||||
set -euo pipefail
|
||||
DEST="${1:-${XDG_DATA_HOME:-${HOME}/.local/share}/darkroom/runtime}"
|
||||
WORK="$(mktemp -d -p /var/tmp fetch-desktop-runtime.XXXXXX)"
|
||||
trap 'rm -rf "${WORK}"' EXIT
|
||||
|
||||
echo "==> downloading the onnxruntime-gpu wheel"
|
||||
uv venv --python 3.12 "${WORK}/venv" >/dev/null
|
||||
VIRTUAL_ENV="${WORK}/venv" uv pip install --quiet onnxruntime-gpu
|
||||
CAPI="$(find "${WORK}/venv" -type d -path '*/onnxruntime/capi' | head -1)"
|
||||
[[ -n "${CAPI}" ]] || { echo "error: no capi directory in the wheel" >&2; exit 1; }
|
||||
|
||||
mkdir -p "${DEST}"
|
||||
# The runtime and its provider libraries; not the Python binding.
|
||||
cp "${CAPI}"/libonnxruntime.so* "${CAPI}"/libonnxruntime_providers_*.so "${DEST}/"
|
||||
echo "==> runtime in ${DEST}:"
|
||||
ls -1 "${DEST}" | sed 's/^/ /'
|
||||
echo " (the app finds it on its next launch; Settings › About › Inference says what it chose)"
|
||||
Reference in New Issue
Block a user