Add dr-inference-engine and route every model session through it

One crate names the runtime, the providers and the devices; dr-face and
dr-segment ask it for a session by role. It hands ort an API table once
per process — from a libonnxruntime it dlopens when the app names a
directory holding one, otherwise from tract — so the Rust build stays
free of C on every target and a package can install the runtime as a
file (docs/inference.md §3).

Sessions live in a registry behind a Model handle that holds the bytes,
not the session: every use refreshes a timestamp and a reaper unloads
whatever sat idle past the decay. A scan that runs the detector on each
image never lets it go idle; a click in the develop view lets the
segmenter go after thirty seconds; a handle used after that reloads,
and reloads on a higher rung if a compiled engine has landed meanwhile.

The probe walks the platform's ladder by building strict sessions and
timing them against the CPU provider, caches the choice against a
fingerprint of the runtime, driver, hardware and models, and compiles
engines for the selected rung in the background, smallest model first.
Nothing in this commit turns the native path on: the apps still run on
tract until they call init with a runtime directory.
This commit is contained in:
2026-09-19 16:02:37 +02:00
parent caf21bea64
commit d15c41e699
16 changed files with 1368 additions and 90 deletions
+4
View File
@@ -8,6 +8,7 @@ members = [
"core/dr-export",
"core/dr-face",
"core/dr-film",
"core/dr-inference-engine",
"core/dr-ingest",
"core/dr-gpu",
"core/dr-lens",
@@ -46,6 +47,9 @@ dr-export = { path = "core/dr-export" }
# `features = ["inference"]`.
dr-face = { path = "core/dr-face", default-features = false }
dr-film = { path = "core/dr-film" }
# `tract` on by default so a test binary can open a session with nothing
# installed; the apps add `native` to look for a runtime file (docs/inference.md §3).
dr-inference-engine = { path = "core/dr-inference-engine" }
dr-ingest = { path = "core/dr-ingest" }
dr-gpu = { path = "core/dr-gpu" }
dr-lens = { path = "core/dr-lens" }