Let the probe's clock be its proof, not disable_cpu_ep_fallback
The strict flag refused the Hexagon over the ten quantise/dequantise nodes at the graph's edges that QNN declines by policy, which cost microseconds. A provider that hands real work to the CPU is slower than the CPU floor and the timing already rejects it; the tablet measured 2.3 ms on the NPU against a 29.7 ms floor.
This commit is contained in:
@@ -1,9 +1,9 @@
|
||||
//! Walk the ladder, once, by building real sessions (docs/inference.md §4).
|
||||
//!
|
||||
//! A rung is taken when a strict session builds on it, runs, and is faster
|
||||
//! than the floor. Both halves matter: a provider can register and then fail
|
||||
//! at partition time, and a provider can take a graph and run it slower than
|
||||
//! the CPU would have. The outcome is cached against a fingerprint of the
|
||||
//! A rung is taken when a session builds on it, runs, and is faster than
|
||||
//! the floor. Both halves matter: a provider can register and then fail at
|
||||
//! partition time, and a provider can take a graph — or quietly hand most
|
||||
//! of it back to the CPU — and run it slower than the CPU would have. The outcome is cached against a fingerprint of the
|
||||
//! runtime, the driver, the hardware and the models, and trusted until any
|
||||
//! of those changes.
|
||||
|
||||
@@ -135,9 +135,9 @@ fn probe_model(cfg: &Config) -> Option<(Role, PathBuf)> {
|
||||
smallest(Some(Role::Detector)).or_else(|| smallest(None))
|
||||
}
|
||||
|
||||
/// Build strictly, run once for the engine, then time three runs; the
|
||||
/// median in milliseconds and, for a compiling rung, the cache key of the
|
||||
/// engine this just built.
|
||||
/// Build, run once for the engine, then time three runs; the median in
|
||||
/// milliseconds and, for a compiling rung, the cache key of the engine this
|
||||
/// just built.
|
||||
fn time_rung(
|
||||
rung: Rung,
|
||||
role: Role,
|
||||
@@ -157,8 +157,8 @@ fn time_rung(
|
||||
};
|
||||
let bytes = std::fs::read(&path).map_err(|e| e.to_string())?;
|
||||
let started = Instant::now();
|
||||
let mut session = crate::session::build(rung, role, &bytes, cfg, true)
|
||||
.map_err(|e| first_line(&e.to_string()))?;
|
||||
let mut session =
|
||||
crate::session::build(rung, role, &bytes, cfg).map_err(|e| first_line(&e.to_string()))?;
|
||||
log::info!(
|
||||
"inference: {} session built in {:.1} s",
|
||||
rung.label(),
|
||||
|
||||
Reference in New Issue
Block a user