Add dr-inference-engine and route every model session through it
One crate names the runtime, the providers and the devices; dr-face and dr-segment ask it for a session by role. It hands ort an API table once per process — from a libonnxruntime it dlopens when the app names a directory holding one, otherwise from tract — so the Rust build stays free of C on every target and a package can install the runtime as a file (docs/inference.md §3). Sessions live in a registry behind a Model handle that holds the bytes, not the session: every use refreshes a timestamp and a reaper unloads whatever sat idle past the decay. A scan that runs the detector on each image never lets it go idle; a click in the develop view lets the segmenter go after thirty seconds; a handle used after that reloads, and reloads on a higher rung if a compiled engine has landed meanwhile. The probe walks the platform's ladder by building strict sessions and timing them against the CPU provider, caches the choice against a fingerprint of the runtime, driver, hardware and models, and compiles engines for the selected rung in the background, smallest model first. Nothing in this commit turns the native path on: the apps still run on tract until they call init with a runtime directory.
This commit is contained in:
@@ -9,10 +9,11 @@ license.workspace = true
|
||||
thiserror.workspace = true
|
||||
log.workspace = true
|
||||
|
||||
# Inference. `ort` is the API; **tract is the engine** — see the workspace
|
||||
# manifest, and docs/faces.md §3, for why the C++ ONNX Runtime is not linked.
|
||||
# Inference. `ort` is the API; **what runs it is `dr-inference-engine`'s
|
||||
# business** — tract, or an ONNX Runtime the app found on disk, on whichever
|
||||
# provider the device has (docs/inference.md). This crate never names either.
|
||||
ort = { workspace = true, optional = true }
|
||||
ort-tract = { workspace = true, optional = true }
|
||||
dr-inference-engine = { workspace = true, optional = true }
|
||||
ndarray = { workspace = true, optional = true }
|
||||
|
||||
[dev-dependencies]
|
||||
@@ -20,7 +21,7 @@ zune-jpeg.workspace = true
|
||||
env_logger.workspace = true
|
||||
# The M1 probe drives `ort` directly so it can print the raw load error.
|
||||
ort = { workspace = true }
|
||||
ort-tract = { workspace = true }
|
||||
dr-inference-engine = { workspace = true }
|
||||
|
||||
[[example]]
|
||||
name = "probe"
|
||||
@@ -48,4 +49,4 @@ default = []
|
||||
# must be testable against synthetic embeddings on a machine with no weights on
|
||||
# it — a test suite that needs a research-licensed download is a test suite
|
||||
# that does not run in CI.
|
||||
inference = ["dep:ort", "dep:ort-tract", "dep:ndarray"]
|
||||
inference = ["dep:ort", "dep:dr-inference-engine", "dep:ndarray"]
|
||||
|
||||
Reference in New Issue
Block a user