Add dr-inference-engine and route every model session through it

One crate names the runtime, the providers and the devices; dr-face and
dr-segment ask it for a session by role. It hands ort an API table once
per process — from a libonnxruntime it dlopens when the app names a
directory holding one, otherwise from tract — so the Rust build stays
free of C on every target and a package can install the runtime as a
file (docs/inference.md §3).

Sessions live in a registry behind a Model handle that holds the bytes,
not the session: every use refreshes a timestamp and a reaper unloads
whatever sat idle past the decay. A scan that runs the detector on each
image never lets it go idle; a click in the develop view lets the
segmenter go after thirty seconds; a handle used after that reloads,
and reloads on a higher rung if a compiled engine has landed meanwhile.

The probe walks the platform's ladder by building strict sessions and
timing them against the CPU provider, caches the choice against a
fingerprint of the runtime, driver, hardware and models, and compiles
engines for the selected rung in the background, smallest model first.
Nothing in this commit turns the native path on: the apps still run on
tract until they call init with a runtime directory.
This commit is contained in:
2026-09-19 16:02:37 +02:00
parent caf21bea64
commit d15c41e699
16 changed files with 1368 additions and 90 deletions
+11 -15
View File
@@ -59,7 +59,7 @@ use ndarray::ArrayView3;
#[cfg(test)]
use crate::semantic::INPUT_EDGE;
use crate::semantic::{install_backend, Letterbox, Window};
use crate::semantic::{Letterbox, Window};
use crate::SegmentError;
/// Classes in the ADE20K vocabulary the scene model was trained on.
@@ -84,7 +84,7 @@ pub struct Category {
/// The scene model, and the categories it has been told to report.
pub struct SceneModel {
session: ort::session::Session,
session: dr_inference_engine::Model,
categories: Vec<Category>,
}
@@ -132,12 +132,12 @@ impl SceneModel {
}
pub fn from_bytes(bytes: &[u8], categories: Vec<Category>) -> Result<Self, SegmentError> {
install_backend();
let session = ort::session::Session::builder()
.map_err(SegmentError::Inference)?
.commit_from_memory(bytes)
.map_err(SegmentError::Inference)?;
// f32, as for `SemanticModel`; see there.
let session = dr_inference_engine::open(
dr_inference_engine::Role::Scene,
dr_inference_engine::Form::F32,
bytes,
)?;
Ok(Self {
session,
@@ -170,13 +170,9 @@ impl SceneModel {
});
}
// Split the borrow: `run` needs the session mutably while
// `marginalise` needs the categories, and going through `self` for
// both at once is what the borrow checker objects to.
let Self {
session,
categories,
} = self;
let categories = &self.categories;
let acquired = self.session.acquire()?;
let mut session = acquired.lock();
let window = Window {
x: 0.0,