Answer M1: tract loads both face graphs once their dims are pinned

Neither InsightFace export parses as shipped -- SCRFD fails at its input node, ArcFace at the first Conv -- which is the same wall dr-segment hit on YOLO's dynamic export. Both load cleanly with the input dims frozen, so the pure-Rust runtime holds for the face pipeline too. tools/fix-face-model-shapes.sh does the freezing, and exists so the artefact is reproducible rather than a binary someone once produced. It takes two forms because the two graphs need different ones: ArcFace's batch is a named dim_param, SCRFD's H and W are dynamic but unnamed. Also notes YuNet loading with no intervention, which matters for the licence question in faces.md 2.3. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 19:51:02 +02:00
parent ec740115b6
commit 72410f39c6
9 changed files with 435 additions and 1 deletions
@@ -0,0 +1,87 @@
+//! SCRFD face detection (docs/faces.md §4).
+//!
+//! For now: enough of the loader to answer M1 — whether tract will parse these
+//! graphs at all — plus the load-time shape validation that keeps a YuNet file
+//! from being decoded as an SCRFD one.
+
+use crate::{install_backend, FaceError};
+
+/// A loaded SCRFD graph.
+pub struct Detector {
+    session: ort::session::Session,
+    /// Feature-map count: 3 for strides {8,16,32}, 4 for {8,16,32,64}.
+    ///
+    /// Discovered from the output count rather than assumed, because both
+    /// exports exist and hardcoding 3 silently ignores the largest faces in a
+    /// four-stride model.
+    strides: usize,
+}
+
+/// Strides, in the order SCRFD emits them.
+pub const ALL_STRIDES: [usize; 4] = [8, 16, 32, 64];
+
+/// Anchors per feature-map location.
+pub const ANCHORS: usize = 2;
+
+impl Detector {
+    pub fn from_path(path: impl AsRef<std::path::Path>) -> Result<Self, FaceError> {
+        let bytes = std::fs::read(path).map_err(FaceError::ModelRead)?;
+        Self::from_bytes(&bytes)
+    }
+
+    pub fn from_bytes(bytes: &[u8]) -> Result<Self, FaceError> {
+        install_backend();
+
+        let session = ort::session::Session::builder()
+            .map_err(FaceError::Inference)?
+            .commit_from_memory(bytes)
+            .map_err(FaceError::Inference)?;
+
+        let n_out = session.outputs().len();
+        if n_out % 3 != 0 || !(9..=12).contains(&n_out) {
+            return Err(FaceError::WrongModel {
+                expected: "InsightFace SCRFD",
+                detail: format!("expected 9 or 12 outputs, got {n_out}"),
+            });
+        }
+        let strides = n_out / 3;
+
+        // The check that actually distinguishes the models: score, box and
+        // landmark groups end in 1, 4 and 10 respectively. YuNet also has
+        // twelve outputs, so the count alone proves nothing.
+        for (group, expected_last) in [1_i64, 4, 10].into_iter().enumerate() {
+            for s in 0..strides {
+                let idx = group * strides + s;
+                let out = &session.outputs()[idx];
+                let last: Option<i64> =
+                    out.dtype().tensor_shape().and_then(|d| d.last().copied());
+                if last != Some(expected_last) {
+                    return Err(FaceError::WrongModel {
+                        expected: "InsightFace SCRFD",
+                        detail: format!(
+                            "output '{}' last dim is {:?}, expected {expected_last}",
+                            out.name(), last
+                        ),
+                    });
+                }
+            }
+        }
+
+        Ok(Self { session, strides })
+    }
+
+    /// Number of stride levels this graph emits.
+    pub fn strides(&self) -> &'static [usize] {
+        &ALL_STRIDES[..self.strides]
+    }
+
+    /// The graph's declared input shape, for diagnosing a dynamic export.
+    pub fn input_shape(&self) -> Option<Vec<i64>> {
+        self.session
+            .inputs()
+            .first()?
+            .dtype()
+            .tensor_shape()
+            .map(|s| s.to_vec())
+    }
+}
@@ -0,0 +1,83 @@
+//! Faces and identity (S14, docs/faces.md).
+//!
+//! Two models, run over the proxy tier, producing per face a box, five
+//! landmarks, a confidence and a 512-d embedding (FR-CULL-8) — and then the
+//! arithmetic that turns embeddings into people (FR-CULL-9, FR-CULL-10).
+//!
+//! Like `dr-segment`, this crate is **device-free**: no GPU adapter, no
+//! Slint, nothing that needs a display. Unlike `dr-segment`, it carries **no
+//! weights at all**, and the absence is deliberate — see [`the licence
+//! note`](#the-weights-are-not-in-this-repository) below.
+//!
+//! # The weights are not in this repository
+//!
+//! The models this crate is built for — SCRFD-500MF and ArcFace/MobileFaceNet
+//! — are InsightFace's, and their pretrained weights carry a **non-commercial
+//! research-only** grant. That is incompatible with GPL-3.0-or-later and with
+//! every channel DarkRoom ships through, so the weights cannot be committed
+//! here the way `dr-segment`'s can, and there is no `embedded-model` feature
+//! for a packaging script to switch on. The application obtains a model at
+//! runtime; this crate takes bytes and never fetches anything.
+//!
+//! docs/faces.md §2 is the full reading, including what would have to change
+//! for that to stop being true.
+//!
+//! # Why the runtime is split behind a feature
+//!
+//! [`calibrate`] and [`cluster`] are where this subsystem's accuracy actually
+//! lives, and both are pure arithmetic over embeddings with no model in them.
+//! They build and test without `inference`, on synthetic embeddings, on a
+//! machine with no weights on it — which is what lets CI cover the part most
+//! likely to be subtly wrong.
+
+#[cfg(feature = "inference")]
+pub mod detect;
+
+/// What can go wrong between an image and a face.
+#[derive(Debug, thiserror::Error)]
+pub enum FaceError {
+    #[error("could not read model file: {0}")]
+    ModelRead(#[source] std::io::Error),
+
+    #[cfg(feature = "inference")]
+    #[error("inference failed: {0}")]
+    Inference(#[source] ort::Error),
+
+    /// The graph is not the one this decoder was written for.
+    ///
+    /// Worth a distinct variant rather than a generic failure: the models in
+    /// this space have interchangeable *shapes* and incompatible *layouts*
+    /// (a YuNet export also has twelve outputs), so the failure this catches
+    /// is not a crash but a page of plausible numbers.
+    #[error("model does not look like {expected}: {detail}")]
+    WrongModel {
+        expected: &'static str,
+        detail: String,
+    },
+
+    #[error("image buffer is {got} floats, expected {expected} (RGB, three per pixel)")]
+    ImageShape { expected: usize, got: usize },
+}
+
+/// Install tract as `ort`'s backend.
+///
+/// Idempotent, and it must happen before any other `ort` call: with
+/// `alternative-backend` there is no linked runtime to fall back on, so an
+/// un-set API is a panic rather than a slow path. Same helper as
+/// `dr-segment::semantic`, for the same reason.
+#[cfg(feature = "inference")]
+pub(crate) fn install_backend() {
+    use std::sync::Once;
+    static ONCE: Once = Once::new();
+    ONCE.call_once(|| {
+        let _ = ort::set_api(ort_tract::api());
+    });
+}
+
+/// [`install_backend`] for the M1 probe example, which drives `ort` directly
+/// rather than through [`detect::Detector`] so it can report the raw error.
+#[cfg(feature = "inference")]
+#[doc(hidden)]
+pub fn install_backend_for_probe() {
+    install_backend();
+}