Add dr-inference-engine and route every model session through it
One crate names the runtime, the providers and the devices; dr-face and dr-segment ask it for a session by role. It hands ort an API table once per process — from a libonnxruntime it dlopens when the app names a directory holding one, otherwise from tract — so the Rust build stays free of C on every target and a package can install the runtime as a file (docs/inference.md §3). Sessions live in a registry behind a Model handle that holds the bytes, not the session: every use refreshes a timestamp and a reaper unloads whatever sat idle past the decay. A scan that runs the detector on each image never lets it go idle; a click in the develop view lets the segmenter go after thirty seconds; a handle used after that reloads, and reloads on a higher rung if a compiled engine has landed meanwhile. The probe walks the platform's ladder by building strict sessions and timing them against the CPU provider, caches the choice against a fingerprint of the runtime, driver, hardware and models, and compiles engines for the selected rung in the background, smallest model first. Nothing in this commit turns the native path on: the apps still run on tract until they call init with a runtime directory.
This commit is contained in:
Generated
+16
-2
@@ -1475,11 +1475,11 @@ dependencies = [
|
||||
name = "dr-face"
|
||||
version = "0.12.2"
|
||||
dependencies = [
|
||||
"dr-inference-engine",
|
||||
"env_logger",
|
||||
"log",
|
||||
"ndarray",
|
||||
"ort",
|
||||
"ort-tract",
|
||||
"thiserror 2.0.20",
|
||||
"zune-jpeg 0.4.21",
|
||||
]
|
||||
@@ -1511,6 +1511,20 @@ dependencies = [
|
||||
"wgpu",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "dr-inference-engine"
|
||||
version = "0.12.2"
|
||||
dependencies = [
|
||||
"libloading",
|
||||
"log",
|
||||
"ort",
|
||||
"ort-sys",
|
||||
"ort-tract",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"thiserror 2.0.20",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "dr-ingest"
|
||||
version = "0.12.2"
|
||||
@@ -1584,11 +1598,11 @@ dependencies = [
|
||||
name = "dr-segment"
|
||||
version = "0.12.2"
|
||||
dependencies = [
|
||||
"dr-inference-engine",
|
||||
"env_logger",
|
||||
"log",
|
||||
"ndarray",
|
||||
"ort",
|
||||
"ort-tract",
|
||||
"thiserror 2.0.20",
|
||||
"zune-jpeg 0.4.21",
|
||||
]
|
||||
|
||||
Reference in New Issue
Block a user