Add OpenVINO and WebGPU rungs to the inference ladder

OpenVINO is the Intel rung: the integrated or Arc GPU, fp16 for every
role but the embedder, a compiled program per model kept in a directory
per model, precision and runtime version. On the Iris Xe it beats ONNX
Runtime's CPU provider on every shipped model — scrfd_10g 23 ms against
58, the scene model 17 against 57, MI-GAN 57 against 330, a denoise tile
40 against 158.

WebGPU is the generic rung for a GPU no vendor rung covers. It was slower
than the CPU on the Iris Xe, the RTX 3050 and the Adreno, so it is on the
ladder for the GPUs it has not been timed on, behind the probe's clock.

MIGraphX's registration becomes one generic key/value helper that all
three share, with option names read from each runtime's own source.
This commit is contained in:
2026-10-04 21:00:15 -04:00
parent 555ec0efb3
commit 87c405eb46
4 changed files with 182 additions and 20 deletions
+14 -5
View File
@@ -14,18 +14,27 @@ use crate::{api::Runtime, state, Cache, Config, Form, Role, Rung};
/// The rungs to try on this platform, best first, under the user's ceiling.
fn ladder(ceiling: Option<Rung>) -> Vec<Rung> {
// WebGPU is the generic rung (§2): it is reached only on a runtime that
// carries it, which `api` loads where no vendor's runtime fits the
// device, and kept only where it beats the CPU.
#[cfg(target_os = "android")]
let all = [Rung::Hexagon];
let all = [Rung::Hexagon, Rung::WebGpu];
// Unmeasured (§2 ⁵): it is on the ladder because the probe's clock and
// `attempt` make a wrong guess cost one slow or failed probe, not a
// slow or crashing app.
#[cfg(target_os = "macos")]
let all = [Rung::CoreMl];
// A desktop has one vendor's GPU; the other vendor's providers are
// "not enabled in this build" or a library that fails to load, and
// either answer arrives in milliseconds.
// A runtime carries one vendor's providers, chosen for this device's
// GPU (`api`); the others are "not enabled in this build", and that
// answer arrives in milliseconds.
#[cfg(not(any(target_os = "android", target_os = "macos")))]
let all = [Rung::TensorRt, Rung::Cuda, Rung::MiGraphX];
let all = [
Rung::TensorRt,
Rung::Cuda,
Rung::MiGraphX,
Rung::OpenVino,
Rung::WebGpu,
];
all.into_iter()
.filter(|r| ceiling.is_none_or(|c| *r <= c))
.collect()