dtourolleandClaude Opus 5
2e09906a08
`mktemp -d` lands in `/tmp`, which on this and most current Linux
distributions is a tmpfs — memory, not disk, sized at half of RAM. The
venv this script builds installs torch into it, several gigabytes, and
the failure mode is not subtle:
error: Failed to install: torch-2.13.0-...whl
Caused by: No space left on device (os error 28)
on a machine with 102 GB free on the filesystem holding `/var/tmp`. The
quieter version of the same bug is worse: when it does fit, it evicts
whatever the user had in page cache to make room.
`${TMPDIR:-/var/tmp}` respects an explicit TMPDIR and otherwise picks the
disk-backed directory, which is what a multi-gigabyte throwaway wants.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
70 lines
2.7 KiB
Bash
Executable File
70 lines
2.7 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Re-export the segmentation model that ships in models/ at the repository root.
|
|
#
|
|
# The .onnx is committed (D14), so this is not part of any build — it exists so
|
|
# the committed artefact is reproducible rather than a binary someone once
|
|
# produced and nobody can regenerate. Run it when bumping the model.
|
|
#
|
|
# ./tools/export-seg-model.sh
|
|
#
|
|
# Requires `uv`. Everything else is fetched into a throwaway venv.
|
|
#
|
|
# ## Why these export flags
|
|
#
|
|
# `dynamic=False` is not a default we failed to change: **tract cannot parse
|
|
# the dynamic-shape graph at all**, failing shape inference on the neck's
|
|
# Concat. A fixed input shape is a hard requirement of the pure-Rust backend
|
|
# (see the workspace manifest for why that backend was chosen), and it is what
|
|
# makes the tiling option in `semantic.rs` the only route to more resolution.
|
|
#
|
|
# `imgsz` square rather than a rectangle matched to 3:2: one graph has to
|
|
# serve portrait, landscape, square crops and panoramas. A landscape-shaped
|
|
# graph trades letterbox waste on 3:2 for worse waste on everything else.
|
|
set -euo pipefail
|
|
|
|
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
REPO="$(cd "${HERE}/.." && pwd)"
|
|
OUT="${REPO}/models/segment"
|
|
|
|
MODEL="${1:-yolo26n-seg}"
|
|
IMGSZ="${2:-640}"
|
|
# Not `mktemp -d`: the default TMPDIR is `/tmp`, which on most current Linux
|
|
# distributions is a tmpfs — RAM, sized at half of physical memory. The venv
|
|
# below pulls torch, several gigabytes of it, and installing that into RAM
|
|
# either evicts the user's page cache or fails outright with ENOSPC on a
|
|
# machine that has hundreds of gigabytes of actual disk free.
|
|
WORK="$(mktemp -d -p "${TMPDIR:-/var/tmp}")"
|
|
trap 'rm -rf "${WORK}"' EXIT
|
|
|
|
echo "==> exporting ${MODEL} at imgsz=${IMGSZ} in ${WORK}"
|
|
cd "${WORK}"
|
|
uv venv --python 3.12 venv
|
|
VIRTUAL_ENV="${WORK}/venv" uv pip install ultralytics onnx onnxslim
|
|
|
|
VIRTUAL_ENV="${WORK}/venv" "${WORK}/venv/bin/python" - "${MODEL}" "${IMGSZ}" <<'PY'
|
|
import sys, json
|
|
from ultralytics import YOLO
|
|
|
|
name = sys.argv[1]
|
|
imgsz = int(sys.argv[2])
|
|
m = YOLO(f"{name}.pt")
|
|
path = m.export(format="onnx", opset=17, simplify=True, imgsz=imgsz, dynamic=False)
|
|
print("ONNX:", path)
|
|
|
|
# The class names travel with the model rather than being retyped into Rust —
|
|
# a hand-copied vocabulary is a silent mismatch waiting to happen when the
|
|
# model is bumped.
|
|
with open("classes.json", "w") as f:
|
|
json.dump([m.names[i] for i in range(len(m.names))], f, indent=1)
|
|
print("classes:", len(m.names))
|
|
PY
|
|
|
|
mkdir -p "${OUT}"
|
|
cp "${WORK}/${MODEL}.onnx" "${OUT}/${MODEL}.onnx"
|
|
cp "${WORK}/classes.json" "${OUT}/${MODEL}.classes.json"
|
|
|
|
echo "==> wrote:"
|
|
ls -la "${OUT}"
|
|
echo
|
|
echo "Remember: these weights are AGPL-3.0 (see ${OUT}/LICENCE.md)."
|