Add a CoreML rung on macOS
The macOS ladder was the CPU provider alone, with CoreML listed as a gap. It is now CoreML, then the CPU, then tract — unmeasured, since nobody here has a Mac, and safe to ship unmeasured because the probe's clock rejects a CoreML slower than the CPU and `attempt` refuses one that crashes. - `Rung::CoreMl`, a compiling rung like TensorRT: an ML Program with every compute unit allowed, falling back to the CPU until each model's program is built. The embedder stays on the CPU, as on the Hexagon (§7). - The cache is one directory per model and runtime version. CoreML keys a model committed from memory on its input and node names, not its weights (ONNX Runtime 1.29, coreml_execution_provider.cc), so two exports of one architecture would otherwise share a program. - The fingerprint on macOS is the chip and the OS release, which ships CoreML. - The desktop looks for the runtime in the bundle's Contents/Frameworks and Homebrew's prefixes; fetch-desktop-runtime.sh on a Mac downloads ONNX Runtime 1.29.0 for Apple silicon, which carries CoreML. docs/dev/macos.md says what exists, how to build it, and which log lines to ask a Mac user for.
This commit is contained in:
@@ -151,10 +151,14 @@ winning:
|
||||
| Linux / Windows, NVIDIA GPU | TensorRT, f32 model, fp16 engine | CUDA provider, f32 | ORT CPU, f32 | tract |
|
||||
| Linux, AMD GPU with ROCm | MIGraphX, f32 model, fp16 program | ORT CPU, f32 | — | tract |
|
||||
| Linux / Windows, no GPU stack | ORT CPU, f32 | — | — | tract |
|
||||
| macOS ⁵ | ORT CPU, f32 | — | — | tract |
|
||||
| macOS ⁵ | CoreML, f32 model, ML Program | ORT CPU, f32 | — | tract |
|
||||
|
||||
⁵ CoreML is the obvious rung and is unmeasured; it is listed so its absence is a gap and not an
|
||||
oversight.
|
||||
⁵ **Unmeasured**, and the one exception to the rule below: nobody here has a Mac. The rung is on
|
||||
the ladder because the probe makes a wrong guess cheap — a CoreML that is slower than the CPU is
|
||||
rejected by §4's clock, one that errors is recorded as failed, and one that takes the process
|
||||
down is refused on the third launch (§4, `attempt`). The embedder stays on the CPU (§7). The first
|
||||
macOS log that shows a probe line is this row's measurement; [macos.md](macos.md) says what to
|
||||
ask for.
|
||||
|
||||
Deliberately **not** on any ladder, with the measurement that excluded each: NNAPI (no driver),
|
||||
XNNPACK (slower than CPU, aborts on SCRFD), WebGPU (slower than CPU), the Adreno through QNN (works,
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
# macOS
|
||||
|
||||
macOS is out of scope for v1 ([requirements.md](requirements.md)), and nobody working on
|
||||
DarkRoom has a Mac. This page records what exists anyway, and how a macOS build is set up so
|
||||
that someone who does have one can send back enough to fix what they hit.
|
||||
|
||||
## 1. What exists
|
||||
|
||||
- **Inference** ([inference.md §2](inference.md)). The macOS ladder is CoreML, then ONNX
|
||||
Runtime's CPU provider, then tract. CoreML is unmeasured. The probe decides whether it is used,
|
||||
and the crash guard (§4, `attempt`) covers the case where the provider takes the process down.
|
||||
The device fingerprint is the chip (`machdep.cpu.brand_string`) and the OS release, because
|
||||
CoreML ships with the OS.
|
||||
- **Where files go** ([`dr_plat::dirs`](../../platform/dr-plat/src/dirs.rs)). The Unix rules,
|
||||
except the state directory (the log and crash records), which is `~/Library/Logs/darkroom`.
|
||||
- **A diagnostic build**, described in §3.
|
||||
|
||||
The rest is not built, packaged or run on macOS by anyone here. This covers the window,
|
||||
Metal through wgpu, the display profile (FR-DSP-8 asks X11 and Wayland), the keyring, the
|
||||
bundle, and signing. `dr-plat` sends every non-Android Unix to the X11/Wayland dependencies.
|
||||
|
||||
## 2. Building
|
||||
|
||||
The Rust side compile-checks from Linux:
|
||||
|
||||
rustup target add aarch64-apple-darwin
|
||||
cargo check --target aarch64-apple-darwin -p dr-inference-engine --features native
|
||||
|
||||
Linking needs Apple's SDK, which means a Mac. On one:
|
||||
|
||||
cargo build --profile diagnostic -p darkroom-desktop
|
||||
./tools/fetch-desktop-runtime.sh # ONNX Runtime 1.29.0 with CoreML, Apple silicon only
|
||||
|
||||
The fetch script puts `libonnxruntime.dylib` in the user's `runtime/` directory, next to the
|
||||
models. The app also looks in `Contents/Frameworks` of its own bundle, and in Homebrew's
|
||||
`/opt/homebrew/lib` and `/usr/local/lib`. Homebrew's build may not include CoreML; the probe
|
||||
reports that as a failed rung and uses the CPU.
|
||||
|
||||
**For whoever packages it.** A notarised app runs with the hardened runtime, whose library
|
||||
validation refuses to `dlopen` a library signed by another team. A bundled
|
||||
`Contents/Frameworks/libonnxruntime.dylib` must be signed with the app. A runtime the user
|
||||
fetched needs the `com.apple.security.cs.disable-library-validation` entitlement, or it will not
|
||||
load, and the app will be the tract build without saying why beyond one log line.
|
||||
|
||||
## 3. The diagnostic build
|
||||
|
||||
Every macOS build is in the hands of someone who can send a log but cannot attach a debugger,
|
||||
so it is set up to log like a debug build while running at release speed.
|
||||
|
||||
- **The log says more.** With no `RUST_LOG`, the desktop's default filter is `debug` for every
|
||||
`dr_*` crate, for `darkroom_desktop`, and for `onnxruntime`. That last one is ONNX Runtime's
|
||||
own session log, which the engine forwards into `log` on every platform (`session.rs`,
|
||||
`with_runtime_log`). At `debug` it includes how many nodes each provider took. At `trace`
|
||||
(`RUST_LOG=onnxruntime=trace`) it lists every node's placement, which is long. The log cap is
|
||||
the same as everywhere (two files of 4 MiB).
|
||||
- **Backtraces have line numbers.** `--profile diagnostic` is release plus line tables. On
|
||||
macOS the tables go into a `.dSYM` beside the executable, and the backtrace in a crash record
|
||||
finds them only if the `.dSYM` stays next to the binary. Keep it in the bundle.
|
||||
|
||||
## 4. What to ask a Mac user for
|
||||
|
||||
`~/Library/Logs/darkroom/darkroom.log`, plus `darkroom.log.1` if present, after the first launch
|
||||
and after the first scan with faces. Console.app lists it under *Log Reports*. The Settings
|
||||
diagnostics bundle collects the same files. The lines that answer the open questions are:
|
||||
|
||||
| Line | What it tells us |
|
||||
|---|---|
|
||||
| `inference: ONNX Runtime … from …` / `inference: runtime tract` | Whether a runtime was found, and which one |
|
||||
| `inference: floor … ms on the CPU provider` | The CPU number for §2's table |
|
||||
| `inference: CoreML session built in … s` | CoreML's first compile of the probe model |
|
||||
| `inference: CoreML rejected: …` / `failed: …` | Why the CPU was kept |
|
||||
| `onnxruntime` lines naming `CoreMLExecutionProvider::GetCapability` | How much of the graph CoreML took |
|
||||
| `inference: the app died during …` | The crash guard fired, and on what |
|
||||
| `inference: compiling … for CoreML` / `ready on CoreML in … s` | Each model's compile, and any that CoreML refused |
|
||||
|
||||
Also ask for the settings row (*Settings › About › Inference*), which is one line and says the
|
||||
same in short. When one of these logs comes back with CoreML numbers, they go into
|
||||
[inference.md §1–2](inference.md), and footnote ⁵ becomes a measurement.
|
||||
Reference in New Issue
Block a user