Run each model on the Hexagon in the form measured to hold it
The engine knew f32 and int8, and gave the Hexagon int8 for every role it served. Measured on the tablet itself (inference.md §1.5), int8 lost 5% of the detector's faces at 40-80 px, moved the landmarks 1.5 px, emptied the segmenter's scores and cost the denoiser 5-9 dB; fp16 the HTP refuses outright. `Form` gains A16W8 and A16W16, and `Rung::form` now names one per role: detectors and landmarks A16W8, the segmenter, scene model, border filler and denoiser A16W16, XFeat int8. The embedder and the eye classifiers stay on the CPU. Each loader resolves its `<stem>.<form>.onnx` sibling; the segmenter and XFeat, compiled into the binary, embed their quantised forms on Android only and pick through `choose_embedded`. The probe, the compile step and the cache fingerprint follow the form instead of assuming int8. Detectors on the new form write `scrfd_*_a16+w600k_mbf`, and `model_ids` answers for all three spellings. On the tablet (ORT 1.29 + QNN 2.42), each shipped file against f32 on the same inputs, and against the CPU's f32 time: SCRFD 500m/2.5g/10g A16W8 100% of faces in every band 4.2/5.1/9.0 ms vs 17/56/198 landmarks A16W8 0.25 px in the 192 crop 0.5 ms vs 2.8 YOLO26n-seg A16W16 98.2% found, mask IoU 0.994 12.9 ms vs 90 scene model A16W16 98.9% of cells agree 15 ms vs 151 MI-GAN A16W16 41 dB from f32 in the fill 87 ms vs 488 XFeat int8 pano alignment 0.45 px (f32's own spread 0.41) 6.5 ms vs 58 denoiser A16W16 0.00 dB at every ISO 95 ms vs 1510 a tile Face numbers are over public COCO val2017 photographs, not a library. The APK carries the siblings (BUNDLED 15 -> 19; the old int8 detectors removed), about 43 MB more. The Windows installer and its CI count skip them; the Arch and Flatpak packages list their files and never had them. The ladder example takes a role per model, which is how the per-role forms above were seen landing on the NPU from the real probe.
This commit is contained in:
@@ -4,12 +4,14 @@
|
||||
//!
|
||||
//! DARKROOM_ORT_DIR=/usr/lib \
|
||||
//! cargo run --release -p dr-inference-engine --features native,tract \
|
||||
//! --example ladder -- CACHE_DIR models/face/scrfd_500m_640.onnx [MODEL.onnx ...]
|
||||
//! --example ladder -- CACHE_DIR models/face/scrfd_500m_640.onnx [ROLE=MODEL.onnx ...]
|
||||
//!
|
||||
//! Every model named is a `Detector` for the config's purposes, which is
|
||||
//! enough to see the rung taken, the engines compiled and a session land
|
||||
//! on it. Delete `CACHE_DIR` to see the first run again; keep it to see the
|
||||
//! second.
|
||||
//! A bare path is a `Detector`; `denoiser=…`, `scene=…`, `inpainter=…`,
|
||||
//! `landmarks=…` (any `Role`, lower case) says otherwise, so a device can
|
||||
//! show each role taking its own form (inference.md §1.5). Each is opened
|
||||
//! through `resolve_model`, as the app opens it, and the line says which
|
||||
//! form and which rung it landed on. Delete `CACHE_DIR` to see the first
|
||||
//! run again; keep it to see the second.
|
||||
|
||||
use std::path::PathBuf;
|
||||
use std::time::{Duration, Instant};
|
||||
@@ -17,7 +19,10 @@ use std::time::{Duration, Instant};
|
||||
fn main() {
|
||||
env_logger::Builder::from_env(env_logger::Env::default().default_filter_or("info")).init();
|
||||
let mut args = std::env::args_os().skip(1).map(PathBuf::from);
|
||||
let (Some(cache_dir), models) = (args.next(), args.collect::<Vec<_>>()) else {
|
||||
let (Some(cache_dir), models) = (
|
||||
args.next(),
|
||||
args.map(|a| role_and_path(&a)).collect::<Vec<_>>(),
|
||||
) else {
|
||||
eprintln!("usage: ladder CACHE_DIR MODEL.onnx [MODEL.onnx ...]");
|
||||
std::process::exit(2);
|
||||
};
|
||||
@@ -34,10 +39,7 @@ fn main() {
|
||||
dr_inference_engine::init(dr_inference_engine::Config {
|
||||
runtime_dirs,
|
||||
cache_dir: cache_dir.clone(),
|
||||
models: models
|
||||
.iter()
|
||||
.map(|p| (dr_inference_engine::Role::Detector, p.clone()))
|
||||
.collect(),
|
||||
models: models.clone(),
|
||||
embedded: Vec::new(),
|
||||
ceiling: None,
|
||||
threads: 0,
|
||||
@@ -80,21 +82,39 @@ fn main() {
|
||||
std::thread::sleep(Duration::from_millis(500));
|
||||
}
|
||||
|
||||
for path in &models {
|
||||
let bytes = std::fs::read(path).expect("read model");
|
||||
for (role, path) in &models {
|
||||
let (path, form) = dr_inference_engine::resolve_model(*role, path);
|
||||
let bytes = std::fs::read(&path).expect("read model");
|
||||
let t = Instant::now();
|
||||
let model = dr_inference_engine::open(
|
||||
dr_inference_engine::Role::Detector,
|
||||
dr_inference_engine::Form::F32,
|
||||
&bytes,
|
||||
)
|
||||
.expect("open model");
|
||||
let model = dr_inference_engine::open(*role, form, &bytes).expect("open model");
|
||||
let acquired = model.acquire().expect("acquire session");
|
||||
println!(
|
||||
"{} on {} in {:.2} s",
|
||||
"{role:?}: {} ({form:?}) on {} in {:.2} s",
|
||||
path.file_name().unwrap().to_string_lossy(),
|
||||
acquired.rung().label(),
|
||||
t.elapsed().as_secs_f64()
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// `denoiser=path` → (Denoiser, path); a bare path is a detector.
|
||||
fn role_and_path(arg: &std::path::Path) -> (dr_inference_engine::Role, PathBuf) {
|
||||
use dr_inference_engine::Role::*;
|
||||
let s = arg.to_string_lossy();
|
||||
let Some((name, path)) = s.split_once('=') else {
|
||||
return (Detector, arg.to_path_buf());
|
||||
};
|
||||
let role = match name {
|
||||
"detector" => Detector,
|
||||
"embedder" => Embedder,
|
||||
"segmenter" => Segmenter,
|
||||
"scene" => Scene,
|
||||
"landmarks" => Landmarks,
|
||||
"eyes" => EyeClassifier,
|
||||
"keypoints" => Keypoints,
|
||||
"inpainter" => Inpainter,
|
||||
"denoiser" => Denoiser,
|
||||
other => panic!("no role {other:?}"),
|
||||
};
|
||||
(role, PathBuf::from(path))
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user