dtourolle
c73743394f
The macOS ladder was the CPU provider alone, with CoreML listed as a gap. It is now CoreML, then the CPU, then tract — unmeasured, since nobody here has a Mac, and safe to ship unmeasured because the probe's clock rejects a CoreML slower than the CPU and `attempt` refuses one that crashes. - `Rung::CoreMl`, a compiling rung like TensorRT: an ML Program with every compute unit allowed, falling back to the CPU until each model's program is built. The embedder stays on the CPU, as on the Hexagon (§7). - The cache is one directory per model and runtime version. CoreML keys a model committed from memory on its input and node names, not its weights (ONNX Runtime 1.29, coreml_execution_provider.cc), so two exports of one architecture would otherwise share a program. - The fingerprint on macOS is the chip and the OS release, which ships CoreML. - The desktop looks for the runtime in the bundle's Contents/Frameworks and Homebrew's prefixes; fetch-desktop-runtime.sh on a Mac downloads ONNX Runtime 1.29.0 for Apple silicon, which carries CoreML. docs/dev/macos.md says what exists, how to build it, and which log lines to ask a Mac user for.
64 lines
3.0 KiB
Bash
Executable File
64 lines
3.0 KiB
Bash
Executable File
#!/usr/bin/env bash
|
||
# Put a GPU-capable ONNX Runtime where the desktop app looks for one
|
||
# (docs/dev/inference.md §3): `runtime/` beside the models in the user data
|
||
# directory, ahead of the system library.
|
||
#
|
||
# ./tools/fetch-desktop-runtime.sh [DEST]
|
||
#
|
||
# The source is the `onnxruntime-gpu` wheel: the one build that carries the
|
||
# CUDA *and* TensorRT providers against the cuDNN and TensorRT majors current
|
||
# on this machine. Distribution packages tend to have neither — Arch's
|
||
# `onnxruntime-opt-cuda` is built without TensorRT and against cuDNN 8 — and
|
||
# the probe rejects them correctly and leaves the app on the CPU provider,
|
||
# which is what this script exists to fix. Nothing NVIDIA is bundled here:
|
||
# the providers load CUDA, cuDNN and TensorRT from the system, and if those
|
||
# are missing the probe says so and the app stays on the CPU.
|
||
#
|
||
# This is the NVIDIA script. On AMD there is nothing to fetch: the
|
||
# distribution's ROCm build of ONNX Runtime (Arch's `onnxruntime-rocm`)
|
||
# carries the MIGraphX provider, and the app finds it in the system library
|
||
# directory (docs/inference.md §1.3).
|
||
set -euo pipefail
|
||
DEST="${1:-${XDG_DATA_HOME:-${HOME}/.local/share}/darkroom/runtime}"
|
||
|
||
# macOS: Microsoft's release archive, which carries the CoreML provider in the
|
||
# one library. Pinned, because the CoreML options the engine sets were read
|
||
# from this version's source (docs/dev/macos.md, CLAUDE.md "Providers").
|
||
# Apple silicon only: no Intel archive is published since 1.29; an Intel Mac
|
||
# takes Homebrew's `onnxruntime` or stays on tract.
|
||
if [[ "$(uname -s)" == Darwin ]]; then
|
||
ORT_VERSION=1.29.0
|
||
[[ "$(uname -m)" == arm64 ]] || {
|
||
echo "error: no ONNX Runtime ${ORT_VERSION} archive for $(uname -m); try: brew install onnxruntime" >&2
|
||
exit 1
|
||
}
|
||
NAME="onnxruntime-osx-arm64-${ORT_VERSION}"
|
||
WORK="$(mktemp -d)"
|
||
trap 'rm -rf "${WORK}"' EXIT
|
||
echo "==> downloading ${NAME}"
|
||
curl -fsSL "https://github.com/microsoft/onnxruntime/releases/download/v${ORT_VERSION}/${NAME}.tgz" \
|
||
| tar xz -C "${WORK}"
|
||
mkdir -p "${DEST}"
|
||
cp "${WORK}/${NAME}/lib/libonnxruntime.dylib" "${WORK}/${NAME}/LICENSE" "${DEST}/"
|
||
echo "==> runtime in ${DEST}:"
|
||
ls -1 "${DEST}" | sed 's/^/ /'
|
||
echo " (the app finds it on its next launch; Settings › About › Inference says what it chose)"
|
||
exit 0
|
||
fi
|
||
|
||
WORK="$(mktemp -d -p /var/tmp fetch-desktop-runtime.XXXXXX)"
|
||
trap 'rm -rf "${WORK}"' EXIT
|
||
|
||
echo "==> downloading the onnxruntime-gpu wheel"
|
||
uv venv --python 3.12 "${WORK}/venv" >/dev/null
|
||
VIRTUAL_ENV="${WORK}/venv" uv pip install --quiet onnxruntime-gpu
|
||
CAPI="$(find "${WORK}/venv" -type d -path '*/onnxruntime/capi' | head -1)"
|
||
[[ -n "${CAPI}" ]] || { echo "error: no capi directory in the wheel" >&2; exit 1; }
|
||
|
||
mkdir -p "${DEST}"
|
||
# The runtime and its provider libraries; not the Python binding.
|
||
cp "${CAPI}"/libonnxruntime.so* "${CAPI}"/libonnxruntime_providers_*.so "${DEST}/"
|
||
echo "==> runtime in ${DEST}:"
|
||
ls -1 "${DEST}" | sed 's/^/ /'
|
||
echo " (the app finds it on its next launch; Settings › About › Inference says what it chose)"
|