Fuse every detector's faces into one population per embedder

Choosing "Thorough" made the library look empty. The detector setting
writes under its own faces.model_id, and every reader of "the faces"
keyed on that exact id: the clustering pass, the coverage figure, the
sweep's work list, the shard export and import, and the sync merge's
face matching. On the reference library that restarted coverage at
1,834 of 19,140, drew a People rail of 36 faces for a person with 520,
queued a ~400 GB re-fetch on each device, and stranded the desktop's
3,583 confirmations under the old id: the tablet held the same faces
under the new one and the merge refused to match them. Same photograph,
same box, same embedder, two ids — that is one face, not two libraries.

The embedder half of the id is now the key. embedder_of and embedder_sql
give it to every query; writes keep the full id, so which detector drew
a box stays on record. record_detections is unchanged and is where the
generations meet: an image holds one pipeline's faces at a time, and a
re-detection carries confirmations across by box overlap. The merge's
match_faces applies the same rule within an embedder. The calibration
is keyed on the embedder too, since the similarity space did not change.

Shards travel every generation, each under its own id, and a peer adopts
whichever it is sent — including a stronger detector's pass over an
image it indexed itself with a weaker one, which is the re-detection its
own sweep would otherwise queue, already done. Never downwards: a tablet
on Fast keeps the desktop's Thorough faces. The sweep gains the same
tail — images a weaker detector indexed, after the ones nothing has —
driven by FaceDetector::supersedes, so choosing a stronger detector still
improves the library over time without first making it disappear.
This commit is contained in:
2026-09-19 10:49:28 +02:00
parent f8addbee53
commit 327decfab1
9 changed files with 703 additions and 136 deletions
+13 -10
View File
@@ -91,14 +91,15 @@ pub struct FaceRequest {
/// examined, so every landscape in the library would be re-detected on every
/// run, for ever. See the V9 migration.
///
/// Keyed on the model, so a model upgrade re-indexes rather than leaving the
/// library half-described by weights that are no longer comparable.
/// Keyed on the embedder, so an embedder upgrade re-indexes rather than
/// leaving the library half-described by weights that are no longer
/// comparable — and a detector change, which keeps the embedder, does not.
pub fn faces_outstanding(
catalog: &Catalog,
store: &ThumbStore,
model_id: &str,
) -> Result<Vec<FaceRequest>, dr_catalog::CatalogError> {
let mut stmt = catalog.connection().prepare(
let mut stmt = catalog.connection().prepare(&format!(
"SELECT i.id, r.file_id
FROM images i
JOIN remote r ON r.image_id = i.id
@@ -107,12 +108,13 @@ pub fn faces_outstanding(
AND i.shadowed_by IS NULL
AND NOT EXISTS (
SELECT 1 FROM face_index fi
WHERE fi.image_id = i.id AND fi.model_id = ?1
WHERE fi.image_id = i.id AND {} = ?1
)
ORDER BY i.id",
)?;
faces::embedder_sql("fi.model_id")
))?;
let rows = stmt
.query_map([model_id], |r| {
.query_map([faces::embedder_of(model_id)], |r| {
Ok(FaceRequest {
image_id: ImageId(r.get::<_, i64>(0)? as u64),
file_id: r.get::<_, i64>(1)? as u64,
@@ -209,7 +211,7 @@ pub fn audit(
// for the same reason `faces::coverage` excludes them: they are the JPEG
// half of a pair, no sweep will ever index one, and counting them makes
// the outstanding figure a number that cannot reach zero.
let mut stmt = conn.prepare(
let mut stmt = conn.prepare(&format!(
"SELECT r.file_id
FROM images i
JOIN remote r ON r.image_id = i.id
@@ -218,12 +220,13 @@ pub fn audit(
AND i.shadowed_by IS NULL
AND NOT EXISTS (
SELECT 1 FROM face_index fi
WHERE fi.image_id = i.id AND fi.model_id = ?1
WHERE fi.image_id = i.id AND {} = ?1
)",
)?;
faces::embedder_sql("fi.model_id")
))?;
let (mut ready, mut awaiting) = (0u64, 0u64);
for file_id in stmt
.query_map([model_id], |r| r.get::<_, i64>(0))?
.query_map([faces::embedder_of(model_id)], |r| r.get::<_, i64>(0))?
.filter_map(Result::ok)
{
if store.contains(file_id as u64, FACE_TIER) {