Load the runtime that fits the GPU, not the first one found
A runtime carries one vendor's providers, only one loads per process, and a device can now hold several: the package's OpenVINO or WebGPU build, a CUDA build the user fetched, the distribution's ROCm build. `api::install` opens each it finds, lists its providers with GetAvailableProviders, and installs the one scoring highest against the GPUs `hardware::detect` reads from files — a vendor rung on its own vendor's GPU above OpenVINO on an Intel one above the generic WebGPU rung above a CPU-only build. Equal scores keep the old first-found order, and DARKROOM_ORT_DIR still wins outright. The losers stay mapped rather than unloaded. The Linux fingerprint now names the OpenCL drivers too, so installing Intel's re-probes. `ladder` takes DARKROOM_ORT_DIRS to show the choice.
This commit is contained in:
@@ -6,6 +6,9 @@
|
||||
//! cargo run --release -p dr-inference-engine --features native,tract \
|
||||
//! --example ladder -- CACHE_DIR models/face/scrfd_500m_640.onnx [ROLE=MODEL.onnx ...]
|
||||
//!
|
||||
//! `DARKROOM_ORT_DIRS=a:b:c` offers several runtimes, as the app's search
|
||||
//! list does, and shows which the engine chose for this device's GPU.
|
||||
//!
|
||||
//! A bare path is a `Detector`; `denoiser=…`, `scene=…`, `inpainter=…`,
|
||||
//! `landmarks=…` (any `Role`, lower case) says otherwise, so a device can
|
||||
//! show each role taking its own form (inference.md §1.5). Each is opened
|
||||
@@ -31,9 +34,16 @@ fn main() {
|
||||
std::process::exit(2);
|
||||
}
|
||||
|
||||
// DARKROOM_ORT_DIRS lists several, colon-separated, as the app's search
|
||||
// does: the engine loads the one that fits the GPU (§3.2).
|
||||
let runtime_dirs: Vec<PathBuf> = std::env::var_os("DARKROOM_ORT_DIR")
|
||||
.map(PathBuf::from)
|
||||
.into_iter()
|
||||
.chain(
|
||||
std::env::var_os("DARKROOM_ORT_DIRS")
|
||||
.map(|v| std::env::split_paths(&v).collect::<Vec<_>>())
|
||||
.unwrap_or_default(),
|
||||
)
|
||||
.collect();
|
||||
let started = Instant::now();
|
||||
dr_inference_engine::init(dr_inference_engine::Config {
|
||||
|
||||
Reference in New Issue
Block a user