From b2f3936a53ea6a8297c572a5e5154b3d6ad20dc9 Mon Sep 17 00:00:00 2001 From: Duncan Tourolle Date: Sat, 26 Sep 2026 18:29:42 -0400 Subject: [PATCH] Say how synced faces and same-name people merge since #77 and #78 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit catalog.md §8.2 still said confirmed and rejected faces were matched to local faces "by box", and that the merge reads only a remote face's box and model; faces.md said `match_faces` "still matches by overlap alone across devices". Since #77 the match falls back to embeddings, on the photographs where a box leaves a remote face over, and since #78 `dedup_people` folds people of one name whose faces agree after every sync. Both now say so, with the thresholds and the reason a less decisive pair stays unmatched, taken from the code's own documentation. --- docs/dev/catalog.md | 10 ++++++++-- docs/dev/faces.md | 36 ++++++++++++++++++++++++++++++++++-- 2 files changed, 42 insertions(+), 4 deletions(-) diff --git a/docs/dev/catalog.md b/docs/dev/catalog.md index 0bf900b..b947304 100644 --- a/docs/dev/catalog.md +++ b/docs/dev/catalog.md @@ -723,7 +723,12 @@ merges reuses those rules or keys on the same identities, and each has no other - **Collections and their membership** — by uuid and revision, membership as a set union. - **Keywords** — the vocabulary by the same verdict, the assignments as a union. - **People and identity judgements** — people by uuid and revision, and the confirmed and rejected - face assignments matched to local faces by box (`merge::match_faces`). + face assignments matched to local faces (`merge::match_faces`): by box first, and — since 0.18.0, + only on the photographs where a remote face is left over — by embedding, a pair being accepted + at cosine ≥ 0.7 when each is the other's best by a lead of ≥ 0.2 (#77; [faces.md §18.2](faces.md)). After every merge, + `dedup_people` folds people of one name whose confirmed faces agree, and a face held twice in + one photograph, through the ordinary `merged_into` redirect, which older builds already honour + (#78; [faces.md §19](faces.md)). - **Albums** (FR-EXP-10, 0.17.0) — by uuid and revision with tombstones, and what went into each as a set union keyed on the server's file id (a content hash on a folder library). An album's server folder is a column of its row and travels with it; a folder on *this device* is in @@ -754,7 +759,7 @@ it goes anywhere. **The face crops stay out of the upload.** A crop is a ~5 KB JPEG on each `faces` row. On a 19k-face library they are 96 MB of a 158 MB catalog. The face shards carry them to other devices, once each. -The merge reads a remote face's box and model to match it to a local one, never its pixels. No +The merge reads a remote face's box and model to match it to a local one — and, where the boxes cannot decide, its embedding — never its pixels. No device adopts a downloaded catalog as its own: a fresh device starts empty and takes faces, crops included, from the shards. So the snapshot's `crop` is NULL, and a merge never writes a local crop. They were first stripped (2026-08) by copying the whole file with the backup API, setting @@ -778,6 +783,7 @@ integer ids stay local and are never compared across catalogs. | Deletion | Tombstone (`deleted = 1`) carrying a revision | Without it, merging against a device that still holds the collection resurrects it. With a revision, deletion competes on equal footing with a rename | | An image the remote has and we do not | Skip the membership row | It joins on a later merge, once a scan has catalogued the file. Not an error | | A remote from a newer schema | Decline before attaching | Attempting it would fail mid-transaction rather than declining cleanly | +| People with the same name | Folded after each merge when their faces agree ([faces.md §19](faces.md)) | Names typed separately on two devices otherwise stay two people for ever | Merging is idempotent: running it twice reports no changes the second time. That property is tested, because a merge that oscillates would upload on every sync forever. diff --git a/docs/dev/faces.md b/docs/dev/faces.md index 92b16cb..0e3a0e4 100644 --- a/docs/dev/faces.md +++ b/docs/dev/faces.md @@ -1575,5 +1575,37 @@ It is a match, not an update in place, and that is why the per-face repairs exis detection: where nothing about a face but one field needs doing, `record_updates` keeps the id and there is nothing to judge. -The merge's `match_faces` still matches by overlap alone across devices. It is the same question, -and the same answer would serve it; it is not changed here. +Since #77 (0.18.0) the merge's `match_faces` answers it too, within a photograph's `file_id` and +one embedder: box IoU ≥ 0.5, unique on both sides, first; then, only for photographs where a remote +face is left over and a local face is free, embedding cosine ≥ 0.7, mutual best, with a lead of +≥ 0.2 over the runner-up on both sides. A box match is never overruled by a low cosine (about 150 +genuine cross-device pairs of tiny faces score below 0.45). On the reference desktop/tablet pair this +recovers 20 of 631 unmatched faces with no false matches; the rest are faces one device alone found. +The merge also keeps one person to one face per photograph: an incoming assignment is refused when +another local face already holds that person, unless it is a remote confirmation over a local +suggestion, which moves the suggestion. Refusals are counted in `faces_one_per_photograph`. + +The threshold differs from `SAME_FACE_COSINE` (0.45) above on purpose: re-detection additionally +requires the boxes to overlap, while the merge's embedding route exists for boxes that don't. + +## 19. Deduplicating people · 2026-09-26 + +`dr_catalog::dedup_people::run` runs after every successful sync merge (`sync::merge_remote`, on the +sync worker), in one transaction, and logs one `dedup:` line (#78). + +**People.** Named people with the same name, trimmed and case-folded, merge into the one with the +most confirmed faces (ties go to the smaller uuid) when every shared embedder's confirmed-face +centroids agree at cosine ≥ 0.7 (distance < 0.3). Each side needs at least two confirmed faces to +compare; a namesake holding no faces merges outright; a face confirmed as one and rejected as the +other keeps them apart; unnamed and set-aside people are never touched. On the reference library the +same-person centroid median is 0.91, and different named people have a 99.9th percentile of 0.41. + +**Faces.** Two faces in the same image and embedder with IoU ≥ 0.5 and cosine ≥ 0.7 are one: the +job keeps the stronger detector's face (`FaceDetector::outranks`), then the confirmed one, then the +lower id, and it takes both faces' assignment and rejections. + +**Propagation.** The merge is `faces::merge_people`, whose `merged_into` redirect a 0.17.0 peer +already honours, so an older device never resurrects the duplicate. The job also follows redirects +left by earlier manual merges, moving this device's own assignments onto the person kept, and +breaks a mutual redirect at the smaller uuid, which every device computes alike. A merge now also +carries the merged-away person's rejections to the person kept.