dtourolle
d15c41e699
One crate names the runtime, the providers and the devices; dr-face and dr-segment ask it for a session by role. It hands ort an API table once per process — from a libonnxruntime it dlopens when the app names a directory holding one, otherwise from tract — so the Rust build stays free of C on every target and a package can install the runtime as a file (docs/inference.md §3). Sessions live in a registry behind a Model handle that holds the bytes, not the session: every use refreshes a timestamp and a reaper unloads whatever sat idle past the decay. A scan that runs the detector on each image never lets it go idle; a click in the develop view lets the segmenter go after thirty seconds; a handle used after that reloads, and reloads on a higher rung if a compiled engine has landed meanwhile. The probe walks the platform's ladder by building strict sessions and timing them against the CPU provider, caches the choice against a fingerprint of the runtime, driver, hardware and models, and compiles engines for the selected rung in the background, smallest model first. Nothing in this commit turns the native path on: the apps still run on tract until they call init with a runtime directory.
45 lines
1.8 KiB
TOML
45 lines
1.8 KiB
TOML
[package]
|
|
name = "dr-inference-engine"
|
|
version.workspace = true
|
|
edition.workspace = true
|
|
rust-version.workspace = true
|
|
license.workspace = true
|
|
|
|
# The one crate that names a runtime, a provider, a vendor library or a
|
|
# device (docs/inference.md §8). `dr-face` and `dr-segment` ask it for a
|
|
# session by role and never see which of these answered.
|
|
|
|
[dependencies]
|
|
thiserror.workspace = true
|
|
log.workspace = true
|
|
serde.workspace = true
|
|
serde_json.workspace = true
|
|
|
|
# `ort` is the API; what supplies it is decided once per process (§3):
|
|
# `libonnxruntime` found on disk, or `tract`. Both are behind
|
|
# `alternative-backend`, so nothing here links C on any target.
|
|
ort = { workspace = true }
|
|
ort-tract = { workspace = true, optional = true }
|
|
# dlopen, and the C types of the table it fetches. Both pure Rust;
|
|
# `libloading` is already in the tree through wgpu.
|
|
libloading = { version = "0.8", optional = true }
|
|
ort-sys = { version = "2.0.0-rc.13", default-features = false, features = ["disable-linking"], optional = true }
|
|
|
|
# The NVIDIA rungs exist on the desktop only. These features add `ort`'s
|
|
# option builders and nothing else — no linking under `alternative-backend` —
|
|
# but an Android binary has no business carrying even the option names, and
|
|
# the packaging must never be tempted to (§2, §3.1).
|
|
[target.'cfg(not(target_os = "android"))'.dependencies]
|
|
ort = { workspace = true, features = ["cuda", "tensorrt"] }
|
|
|
|
[target.'cfg(target_os = "android")'.dependencies]
|
|
ort = { workspace = true, features = ["qnn"] }
|
|
|
|
[features]
|
|
# The floor: `tract` supplies the API table when no runtime file is found, or
|
|
# always, in a build without `native`. Tests want this and nothing else.
|
|
default = ["tract"]
|
|
tract = ["dep:ort-tract"]
|
|
# Look for `libonnxruntime` on disk and hand its table to `ort`.
|
|
native = ["dep:libloading", "dep:ort-sys"]
|