Add dr-inference-engine and route every model session through it

One crate names the runtime, the providers and the devices; dr-face and
dr-segment ask it for a session by role. It hands ort an API table once
per process — from a libonnxruntime it dlopens when the app names a
directory holding one, otherwise from tract — so the Rust build stays
free of C on every target and a package can install the runtime as a
file (docs/inference.md §3).

Sessions live in a registry behind a Model handle that holds the bytes,
not the session: every use refreshes a timestamp and a reaper unloads
whatever sat idle past the decay. A scan that runs the detector on each
image never lets it go idle; a click in the develop view lets the
segmenter go after thirty seconds; a handle used after that reloads,
and reloads on a higher rung if a compiled engine has landed meanwhile.

The probe walks the platform's ladder by building strict sessions and
timing them against the CPU provider, caches the choice against a
fingerprint of the runtime, driver, hardware and models, and compiles
engines for the selected rung in the background, smallest model first.
Nothing in this commit turns the native path on: the apps still run on
tract until they call init with a runtime directory.
This commit is contained in:
2026-09-19 16:02:37 +02:00
parent caf21bea64
commit d15c41e699
16 changed files with 1368 additions and 90 deletions
+11 -15
View File
@@ -124,25 +124,21 @@ pub enum FaceError {
ImageShape { expected: usize, got: usize },
}
/// Install tract as `ort`'s backend.
///
/// Idempotent, and it must happen before any other `ort` call: with
/// `alternative-backend` there is no linked runtime to fall back on, so an
/// un-set API is a panic rather than a slow path. Same helper as
/// `dr-segment::semantic`, for the same reason.
#[cfg(feature = "inference")]
pub(crate) fn install_backend() {
use std::sync::Once;
static ONCE: Once = Once::new();
ONCE.call_once(|| {
let _ = ort::set_api(ort_tract::api());
});
impl From<dr_inference_engine::Error> for FaceError {
fn from(e: dr_inference_engine::Error) -> Self {
match e {
dr_inference_engine::Error::Inference(e) => FaceError::Inference(e),
dr_inference_engine::Error::Io(e) => FaceError::ModelRead(e),
}
}
}
/// [`install_backend`] for the M1 probe example, which drives `ort` directly
/// rather than through [`detect::Detector`] so it can report the raw error.
/// Make sure `ort` has a backend, for the M1 probe example, which drives
/// `ort` directly rather than through [`detect::Detector`] so it can report
/// the raw error. Every other path goes through `dr-inference-engine`.
#[cfg(feature = "inference")]
#[doc(hidden)]
pub fn install_backend_for_probe() {
install_backend();
dr_inference_engine::ensure_runtime();
}