The similarity scan picks its dot product per machine, and the NEON one
is the kernel that ships to the phone and the tablet — and the one a
desktop cargo test never executes. A wrong lane index or a mishandled
tail there is a silent wrong answer on exactly the devices nobody runs
the suite on, which is a poor place for the only untested code path.
dr-face carries no weights and touches no display, so its tests are a
plain ARM64 binary that runs under adb shell with nothing installed.
The script builds it against the SDK's newest NDK, pushes it, runs it
and cleans up. It checks for the device first, so a tablet that is not
plugged in costs a second rather than the two minutes it takes to
compile for it.
Not wired into CI, which has no device attached.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The §9 note said the scan was half a regroup and implied the
agglomeration was an irreducible sequential walk. Both halves of that
are now wrong, and the numbers are the point of the section.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The scan is O(n²) dot products and nothing else, so its speed is the
face subsystem's speed — and it was running at 0.7 flops per cycle.
Two separate faults, both measured over the reference 18,143-face
library on twenty cores. It walked the whole embedding array once per
row, ~336 GB of traffic, where a column tile that fits in L2 is read
once per tile of rows: 4.64s → 2.81s. And the workspace builds for
baseline x86-64 — SSE2, no FMA — into which the portable loop was not
being vectorised at all: 2.81s → 0.86s, 195 GFLOP/s.
So the dot product is now chosen per machine. AVX2 + FMA where
is_x86_feature_detected! finds it; NEON unconditionally on aarch64,
since Advanced SIMD is in that baseline and every Android device the app
builds for has it — with the explicit vfmaq, because LLVM will not fuse
a multiply and an add without being told to. The portable loop stays as
the definition the others are tested against, and
the_fastest_kernel_agrees_with_the_portable_one is the only check the
NEON path gets on a machine that is not aarch64.
Faces::embeddings is one flat buffer rather than a Vec per face: the
pointer chase defeated both the prefetcher and the tiling, and it is
also the layout a GPU pass would want.
Behaviour is unchanged and that is checked rather than asserted — the
same 1,531,969 pairs from all three kernels, and on the real library the
same 2,518 groups holding the same 16,246 faces with the same confidence
distribution. A full regroup there goes from 10.0s to 5.9s; the rest is
the agglomeration, which is a sequential heap walk and is where the next
look should go.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The number beside a suggestion was the mean calibrated probability
between the face and the rest of its group, which measures the wrong
thing twice. It punishes coverage: a person with two hundred faces over
fifteen years is *meant* to have members a given photograph is
orthogonal to, so a correct suggestion onto a well-photographed person
scored low for being well photographed. And it never asked who else the
face might be — a face matching Anna at 0.95 and nobody else, and one
matching Anna at 0.95 and her sister at 0.93, came out identical, when
the second is the only one worth the user's attention.
dr_face::assign answers both, and multiplies them: the mean of the best
ten calibrated matches into the identity (the old mean, capped, which is
what stops coverage counting against it), times that identity's share of
the evidence against every *named* rival.
Only named people compete, and per person rather than per group. Both
halves of that had to be measured on a real 18,000-face library rather
than reasoned about. Normalising across every group made the number
useless — median suggestion 21%, four in five under half — because
clustering leaves one person spread over many groups, so a face competed
against itself; and keying rivals by group left Catherine competing with
Catherine, median 39%. Per named person: median 99.5%.
Rivals are gathered below the merge threshold, down to even odds: a
named person matching at 0.6 will never be merged into but is exactly
the competition to discount for. That would be a second similarity scan,
the expensive half of regrouping a library, so cluster_scored scans once
at the looser floor and hands the merge engine the subset at or above
the threshold — pair for pair what it would have scanned for itself,
held to that by a test.
Leave-one-out over that library's 2,702 confirmations across 54 named
people: 99.33% of faces placed on the right person against the old
mean's 99.15%, and the number shown for the right person moves from a
median of 90.4% to 99.3%. It errs low — 100% correct wherever it states
80% or more — which is the safe direction, and docs/faces.md §9.1 says
plainly that the low bands are not calibrated.
The example that measures it comes too: this is a claim about a
library's numbers, and nobody should have to take it on faith.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
FR-CULL-9 was read as "no fit, no number", so every library without 200
confirmed positive pairs showed "Confidence unavailable" on every
suggestion — which is every library, until enough confirmations exist to
fit one. The confirmations are made on this screen, ranked by the number
it was withholding, so the degraded state was also the permanent one.
There has always been a curve: Calibration::default is the reference
implementation's fitted MBF sigmoid, which is what clustering already
operates at. It is a published operating point, not an invention, and
what the requirement forbids is presenting it *as though it were
measured on this library*. So the percentage is shown, and the screen
says once, above the grid, where the curve came from.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Face indexing was compiled into the APK all along — dr-ui takes dr-face with
`inference` on every target, so SCRFD, alignment, MBF, calibration and
clustering were all in there. What was missing was the weights, and on Android
there was no way to supply them.
Route C (docs/faces.md §2.2) says the user obtains the model and the app loads
it. On a desktop that is a real gesture: drop two files in
~/.local/share/darkroom/models/ and indexing starts working. On Android it is
not a gesture at all. `internal_data_path` is app-private, `run-as` needs a
debuggable build, and the in-app fetch route C specifies was never built — so
the settings page reported "no face model is installed" on every launch with
nothing behind the message. Not "off until you supply weights"; off.
So the shape-fixed pair goes into LFS under the APK's assets, assemble-apk.sh
copies it into the package, and `android_main` unpacks it to the shared models
directory before anything asks whether a model is present.
Three things that are not incidental:
The models directory is now shared across accounts rather than per-account.
Weights are identified by `faces.model_id`, not by who is signed in, so two
accounts had no reason to hold two copies — and the unpack runs before any
session exists to key a per-account path off. `face_models` still prefers a
per-account directory when one is populated, so anyone mid-migration keeps the
ability to pin one library to its own pair.
The unpack writes under a temporary name and renames. `face_models` decides
availability on `is_file()` alone, so a copy truncated by the process being
killed would leave a file that passes that test and fails inside tract —
reported to the user as a broken model rather than a missing one.
assemble-apk.sh refuses an LFS pointer. At ~130 bytes it looks exactly like a
model to `cp`, and unchecked it reaches the device and fails in the graph
loader instead of telling someone to run `git lfs pull` — the same guard
dr-segment's build script applies to yolo26n-seg.onnx.
The licensing half is unchanged and recorded in §2.2a: the InsightFace grant is
research-only, this is a private repository and a self-installed build, and
these files come back out before anything is published. The weights are still
not a cargo build input — dr-face has no `models/` directory and no
`embedded-model` feature, and nothing in the build reads them. The APK assembly
step copies two files and is the only thing in the tree that knows they exist.
Verified on device: both models unpack on first launch (2524817 and 13616095
bytes) and the APK carries them at assets/models/.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An image with no faces in it was indistinguishable from one that had
never been looked at, so every landscape, still life and document scan in
the library was re-detected on every pass, for ever. In a real library
that is most of it: on the 23,527-image test library, 64 of the first 110
images indexed contain no face at all.
Schema v9 adds face_index, a run marker per (image, model) carrying the
face count and the proxy edge it read. Keyed on the model, so a model
change puts every image back in the queue by itself.
That makes a coverage figure possible, which is the thing a user actually
wants to see. The audit also splits the outstanding set by whether a
proxy exists, because 23,417 awaiting a proxy and 110 ready to index are
different problems, and telling the user to run indexing again would not
fix the first.
The Identity screen gains Index faces, Stop, and the coverage line.
examples/face_index.rs is the same check and sweep without a window,
which is the right shape for an overnight pass.
Measured on the real library in release: 3.5 images/second, 110 images
and 125 faces in 30 seconds, and a second run correctly finds nothing
left to do.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Also notes what building it taught the design: a split has to reject
before it confirms, or the next clustering pass undoes it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
FR-CULL-9 forbids thresholding a bare cosine anywhere in the subsystem,
so calibrate fits P(same person) per library and reports whether the fit
is trustworthy. Two details carry most of the weight.
The fit runs against a 200-bin histogram rather than a pair list: a
25,000-face library has ~3e8 pairs and no gradient descent is running
over that. And a fresh library has no valid calibration, because the
positives have to come from user confirmations or burst siblings --
bootstrapping them from high cosine would fit the calibration to the
belief it was supposed to test.
Clustering defends against the over-merging FR-CULL-10 warns about with
constraints rather than a better threshold: two faces in one photograph
never merge, and two groups confirmed as different people never merge.
Average link rather than single link, so one strong edge cannot weld two
families together.
Calibration is defined once, in dr-face, and dr-catalog re-exports it.
Two implementations of one probability model is exactly how a number
comes to mean the wrong thing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Ports the pipeline from the C++ reference in ../scene-actor-extraction
(MIT, same author). End to end on real portraits it separates identities
the way the reference's fitted calibration says it should: 0.596 between
distinct photographs of one person, 0.05 between different people, either
side of MBF's 0.267 boundary.
Three things are structural rather than incidental:
Aligned112 can only be built by align::warp, so Embedder::embed cannot be
handed an unaligned bounding-box crop. That mistake yields 512 plausible
unit-norm numbers and no error, so the type system refuses it instead.
Embedding carries its ModelId and cosine() returns None across models,
because a cross-model similarity is the one mistake that produces
plausible garbage rather than a failure.
The model-free half -- alignment, embedding arithmetic, f16 storage --
sits outside the inference feature and is covered by 11 tests that need
no weights on the machine.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Neither InsightFace export parses as shipped -- SCRFD fails at its input
node, ArcFace at the first Conv -- which is the same wall dr-segment hit
on YOLO's dynamic export. Both load cleanly with the input dims frozen,
so the pure-Rust runtime holds for the face pipeline too.
tools/fix-face-model-shapes.sh does the freezing, and exists so the
artefact is reproducible rather than a binary someone once produced. It
takes two forms because the two graphs need different ones: ArcFace's
batch is a named dim_param, SCRFD's H and W are dynamic but unnamed.
Also notes YuNet loading with no intervention, which matters for the
licence question in faces.md 2.3.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
FR-CULL-8..12 specify the subsystem in terms of "a 512-dimension embedding
from a stated model" and stop there, because D13 was open. This names the
models, and grounds them in the measurements and the working C++ pipeline in
../scene-actor-extraction rather than in a literature reading.
The licensing half of D13 stays open, but with a route through it: the
InsightFace weights are non-commercial and cannot be committed, so the app
ships the code and the user fetches the model. faces.model_id already makes
that a survivable choice.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
dtourolleandClaude Opus 5