dtourolle
c78b798cf0
Seven names are two or three live people on both devices: Ian (756 confirmed faces, and a second Ian with none), Jessie three times, Claudine, Mathias, Noemi, Pascal and PJ. Each was typed on its own device and carried across by sync, which keys people on their uuid and so keeps both. Each half of a person shows half their photographs. dedup_people::run, in one transaction: - Same-name people (trimmed, case-folded as the Identity screen folds them) merge into the one with the most confirmed faces, ties to the smaller uuid, through faces::merge_people_within, so confirmations, rejections and the survivor's name are kept. A person holding no faces at all merges: there is nothing to compare or to carry. Anyone else needs >= 2 confirmed faces per shared embedder on both sides and centroids at cosine >= 0.7 in each. A face confirmed as one and rejected as the other keeps them apart. Unnamed and set-aside people are never merged by name. - Faces held twice (one image, one embedder, IoU >= 0.5, cosine >= 0.7) keep the stronger detector's row (FaceDetector::outranks), then the confirmed one, then the older. The survivor takes the confirmed assignment and both rows' rejections. A pair confirmed as two different people is left and counted. - Judgements still on a merged-away person move to the person at the end of its redirects, and a redirect cycle (two devices merging one pair in opposite directions) is broken at the smaller uuid. Measured on copies of the desktop catalog and the tablet's server snapshot, w600k_mbf, confirmed faces only: - Centroids of differently named people: 2,699 pairs, median 0.02, 99.9th percentile 0.41. One pair reaches 0.70 (0.700 desktop, 0.705 tablet), "Michelle Casanonve" and "Michelle Casanova", one person typed two ways. Next is 0.62/0.64, "Boris Jost" and "Boris". The highest pair that is plainly two people is 0.43/0.44. - One person split in random halves: minimum 0.69, median 0.91 over 72 people. Four faces against twenty-two reach 0.7 in 97% of draws. One face against twenty of somebody else's reached 0.74 in 3,000 draws, and two faces reached 0.61, hence the two-face minimum. - Pascal (22 and 4 confirmed) is at 0.57 and PJ (14 and 7) at 0.50, under 0.7 on both devices, so both pairs stay apart and are logged. The desktop's second Ian holds 4 suggestions and no confirmations, at 0.38 against Ian's centroid, and stays apart. On the tablet it holds nothing and merges. Why a merge made here survives a peer on 0.17.0: the merged-away person stays as a merged_into redirect with a bumped revision, which the catalog merge has always taken on revision. The peer hides the duplicate and never sends it back as a live person. Its own confirmations of that person stay on the redirect, because a merge never overwrites a local confirmation. The manual merge has always left them there too. They follow the redirect when the peer runs this job. A test syncs two catalog files through the previous merge code and back, and the people converge and stay converged. Once a catalog is clean the job reads 80 redirects, the named people, and the face boxes from the covering faces_box index. That is ~10 ms on the reference library. There is no schema change. The index is created IF NOT EXISTS, as the merge already does.
3029 lines
124 KiB
Rust
3029 lines
124 KiB
Rust
//! TRACES: FR-CAT-7 | FR-CAT-5 | FR-NC-9
|
|
//! Merging a remote catalog's collections and keywords into the local one.
|
|
//!
|
|
//! # Why this is a merge and not a copy
|
|
//!
|
|
//! The catalog file syncs to Nextcloud, and a device that finds a newer remote
|
|
//! copy must not simply replace its own — whichever device synced second would
|
|
//! lose everything the first did not have. So the remote file is downloaded to
|
|
//! a side path, `ATTACH`ed, and merged table by table.
|
|
//!
|
|
//! Row-level merging needs identities that are stable across devices, and
|
|
//! `collections.id INTEGER PRIMARY KEY` is not: two devices independently
|
|
//! allocate id 1 for different collections. Hence `collections.uuid`, which is
|
|
//! what everything here keys on. The integer id stays local and is never
|
|
//! compared across catalogs.
|
|
//!
|
|
//! # Conflict rule
|
|
//!
|
|
//! Per collection, by `revision` — a monotonic counter bumped on every local
|
|
//! edit — with `modified` timestamp only as a tiebreak. Comparing revisions
|
|
//! rather than mtimes means a device with a skewed clock cannot silently win
|
|
//! (the failure mode FR-NC-9 avoids for sidecars, applied here).
|
|
//!
|
|
//! Membership merges as a **set union**, not last-writer-wins: two devices
|
|
//! each adding different images to the same collection keep both sets. That
|
|
//! is almost always what the user meant, and the exception — a removal racing
|
|
//! an addition — resolves in favour of the addition, which is recoverable by
|
|
//! removing it again. Silently losing an addition is not.
|
|
//!
|
|
//! # Deletion
|
|
//!
|
|
//! A deleted collection leaves a tombstone (`deleted = 1`), because a merge
|
|
//! against a device that still holds it would otherwise resurrect it. The
|
|
//! tombstone carries a revision like any other edit, so deletion competes on
|
|
//! the same footing as a rename.
|
|
//!
|
|
//! # Keywords merge on the same three rules
|
|
//!
|
|
//! [`merge_keywords`] reuses all of the above rather than inventing a second
|
|
//! set of rules, because a keyword is the same shape of problem as a
|
|
//! collection: a named thing with a device-independent identity, and a
|
|
//! many-to-many join to images.
|
|
//!
|
|
//! - The **vocabulary** (`keyword_terms`) is decided per row by [`verdict`],
|
|
//! exactly as collections are.
|
|
//! - The **assignments** (`keywords`) are a set union, exactly as membership
|
|
//! is: two devices each keywording different photographs "puffin" keep both
|
|
//! sets, and two devices each keywording the *same* photograph converge on
|
|
//! one row rather than one of them winning.
|
|
//! - **Deletion** tombstones, and takes the assignments with it.
|
|
//!
|
|
//! Two things are genuinely different, and both are consequences of assignments
|
|
//! storing the *word* rather than a row id:
|
|
//!
|
|
//! 1. A tombstone deletes assignments **by name**, so a deletion still lands on
|
|
//! a device that had minted its own identity for the same word. The union
|
|
//! then refuses to readmit a word a winning tombstone has just removed —
|
|
//! without that filter, the other device's live assignments would resurrect
|
|
//! it on the very same pass.
|
|
//! 2. Two devices that independently typed the same word arrive with two uuids
|
|
//! for one keyword. [`crate::keywords::fuse_duplicates`] collapses them onto
|
|
//! the lexicographically smaller one, which both devices compute identically.
|
|
//! A unique index on the name would instead abort the merge transaction at
|
|
//! that moment, which is the ordinary case rather than a corner one.
|
|
|
|
use rusqlite::{Connection, OptionalExtension};
|
|
|
|
use crate::error::CatalogError;
|
|
|
|
/// How a collection differed between the two catalogs.
|
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
|
pub enum MergeVerdict {
|
|
/// Present only remotely — insert it.
|
|
InsertedFromRemote,
|
|
/// Remote revision is higher — take its fields.
|
|
UpdatedFromRemote,
|
|
/// Local revision is at least as high — keep ours.
|
|
KeptLocal,
|
|
/// Remote says deleted, and wins on revision.
|
|
DeletedByRemote,
|
|
}
|
|
|
|
/// What a merge did, for logging and for telling the user.
|
|
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
|
|
pub struct MergeReport {
|
|
pub inserted: usize,
|
|
pub updated: usize,
|
|
pub kept_local: usize,
|
|
pub deleted: usize,
|
|
pub members_added: usize,
|
|
|
|
// Keywords are counted separately from collections rather than summed into
|
|
// the same fields. The report is shown to the user — "3 collections, 11
|
|
// keywords" is a sentence; "14 things" is not — and a merge that went wrong
|
|
// is far easier to place when the counts say which half it went wrong in.
|
|
/// Keywords the remote had and this device did not.
|
|
pub keywords_inserted: usize,
|
|
/// Keywords the remote had renamed, or brought back from a tombstone.
|
|
pub keywords_updated: usize,
|
|
/// Keywords the remote deleted, and this device has now deleted too.
|
|
pub keywords_deleted: usize,
|
|
/// Keywords where this device's revision was at least as high.
|
|
pub keywords_kept_local: usize,
|
|
/// People the remote had and this device did not.
|
|
pub people_inserted: usize,
|
|
/// People the remote had renamed, set aside, or brought back.
|
|
pub people_updated: usize,
|
|
/// People where this device's revision was at least as high.
|
|
pub people_kept_local: usize,
|
|
/// Faces this device now agrees belong to somebody.
|
|
pub faces_assigned: usize,
|
|
/// Faces whose local confirmation outranked the remote's.
|
|
pub faces_kept_local: usize,
|
|
/// "Not this person" judgements taken from the remote.
|
|
pub faces_rejected: usize,
|
|
/// Remote faces placed on a local one by their embedding, where the
|
|
/// boxes disagreed or were ambiguous (see `match_faces`).
|
|
pub faces_matched_by_embedding: usize,
|
|
/// Assignments from the remote refused because this device already has
|
|
/// that person on another face of the same photograph.
|
|
pub faces_one_per_photograph: usize,
|
|
|
|
/// Redundant identities for one word, retired by
|
|
/// [`crate::keywords::fuse_duplicates`].
|
|
pub keywords_fused: usize,
|
|
/// Keyword assignments taken from the remote.
|
|
pub keywords_assigned: usize,
|
|
/// Images whose capture metadata was taken from the remote.
|
|
pub metadata_adopted: usize,
|
|
|
|
/// Albums the remote had and this device did not, or had renamed, moved
|
|
/// or deleted with the higher revision.
|
|
pub albums_taken: usize,
|
|
/// Albums where this device's revision was at least as high.
|
|
pub albums_kept_local: usize,
|
|
/// Exported files the remote had recorded into an album and this device
|
|
/// had not.
|
|
pub album_exports_added: usize,
|
|
}
|
|
|
|
impl MergeReport {
|
|
/// Whether the local catalog changed, and so needs re-uploading.
|
|
pub fn local_changed(&self) -> bool {
|
|
self.inserted > 0
|
|
|| self.updated > 0
|
|
|| self.deleted > 0
|
|
|| self.members_added > 0
|
|
|| self.keywords_inserted > 0
|
|
|| self.keywords_updated > 0
|
|
|| self.keywords_deleted > 0
|
|
|| self.keywords_fused > 0
|
|
|| self.keywords_assigned > 0
|
|
|| self.metadata_adopted > 0
|
|
|| self.albums_taken > 0
|
|
|| self.album_exports_added > 0
|
|
}
|
|
|
|
/// Whether the local catalog holds anything the remote did not, and so
|
|
/// must be uploaded even if nothing was taken from the remote.
|
|
pub fn should_upload(&self) -> bool {
|
|
self.kept_local > 0
|
|
|| self.keywords_kept_local > 0
|
|
|| self.albums_kept_local > 0
|
|
|| self.local_changed()
|
|
}
|
|
}
|
|
|
|
/// Decide one collection, given both sides' revisions.
|
|
///
|
|
/// Split out from the SQL so the rule is testable on its own — it is the part
|
|
/// that decides whether a user loses a collection.
|
|
pub fn verdict(
|
|
local: Option<(i64, i64)>, // (revision, modified)
|
|
remote: (i64, i64),
|
|
remote_deleted: bool,
|
|
) -> MergeVerdict {
|
|
let (r_rev, r_mod) = remote;
|
|
match local {
|
|
None if remote_deleted => {
|
|
// A tombstone for something we never had. Recording it still
|
|
// matters: without it, a third device could reintroduce the
|
|
// collection through us.
|
|
MergeVerdict::DeletedByRemote
|
|
}
|
|
None => MergeVerdict::InsertedFromRemote,
|
|
Some((l_rev, l_mod)) => {
|
|
// Revision first; timestamp only to break an exact tie. Equal
|
|
// revisions with equal timestamps keep local, so a merge that
|
|
// changes nothing is stable and repeatable.
|
|
let remote_wins = r_rev > l_rev || (r_rev == l_rev && r_mod > l_mod);
|
|
if !remote_wins {
|
|
MergeVerdict::KeptLocal
|
|
} else if remote_deleted {
|
|
MergeVerdict::DeletedByRemote
|
|
} else {
|
|
MergeVerdict::UpdatedFromRemote
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
/// Merge everything that syncs, from an attached catalog.
|
|
///
|
|
/// The remote catalog must already be attached under the schema name
|
|
/// `remote_cat`; [`crate::sync::merge_remote`] handles that.
|
|
///
|
|
/// **One transaction over both halves.** Keywords and collections are
|
|
/// independent as data, but a merge that landed the collections and then failed
|
|
/// on the keywords would leave a catalog that has already taken the remote's
|
|
/// revisions for half of itself — and the next attempt, seeing those revisions,
|
|
/// would decline to take them again. Half a merge is not a state that can be
|
|
/// resumed, so it is not a state that can be reached.
|
|
pub fn merge_all(conn: &Connection) -> Result<MergeReport, CatalogError> {
|
|
let tx = conn.unchecked_transaction()?;
|
|
let mut report = MergeReport::default();
|
|
merge_collections_within(&tx, &mut report)?;
|
|
merge_keywords_within(&tx, &mut report)?;
|
|
merge_people_within(&tx, &mut report)?;
|
|
merge_metadata_within(&tx, &mut report)?;
|
|
merge_albums_within(&tx, &mut report)?;
|
|
tx.commit()?;
|
|
Ok(report)
|
|
}
|
|
|
|
/// Merge albums and what was exported into them, from an attached catalog.
|
|
pub fn merge_albums(conn: &Connection) -> Result<MergeReport, CatalogError> {
|
|
let tx = conn.unchecked_transaction()?;
|
|
let mut report = MergeReport::default();
|
|
merge_albums_within(&tx, &mut report)?;
|
|
tx.commit()?;
|
|
Ok(report)
|
|
}
|
|
|
|
/// The album half: rows by [`verdict`], exports as a set union.
|
|
///
|
|
/// `album_folders` is not read. It is this device's choice of a local folder
|
|
/// and means nothing on the device that sent the snapshot — which has in any
|
|
/// case dropped it before uploading (see [`crate::albums`]).
|
|
fn merge_albums_within(tx: &Connection, report: &mut MergeReport) -> Result<(), CatalogError> {
|
|
// A snapshot from a build before albums, or from a device that never
|
|
// made one, has no tables to read. Ours are made on demand so the
|
|
// statements below have somewhere to write.
|
|
if !remote_has(tx, "albums")? {
|
|
return Ok(());
|
|
}
|
|
crate::albums::ensure_tables(tx)?;
|
|
|
|
/// One album as the remote has it, and what the verdict made of it.
|
|
struct IncomingAlbum {
|
|
uuid: String,
|
|
name: String,
|
|
server_path: Option<String>,
|
|
created: i64,
|
|
revision: i64,
|
|
modified: i64,
|
|
deleted: bool,
|
|
verdict: MergeVerdict,
|
|
}
|
|
|
|
let rows: Vec<IncomingAlbum> = {
|
|
let mut stmt = tx.prepare(
|
|
"SELECT r.uuid, r.name, r.server_path, r.created, r.revision, r.modified,
|
|
r.deleted, l.revision, l.modified
|
|
FROM remote_cat.albums r
|
|
LEFT JOIN main.albums l ON l.uuid = r.uuid",
|
|
)?;
|
|
let rows = stmt
|
|
.query_map([], |r| {
|
|
let revision: i64 = r.get(4)?;
|
|
let modified: i64 = r.get(5)?;
|
|
let deleted: bool = r.get::<_, i64>(6)? != 0;
|
|
let local_rev: Option<i64> = r.get(7)?;
|
|
let local_mod: Option<i64> = r.get(8)?;
|
|
Ok(IncomingAlbum {
|
|
uuid: r.get(0)?,
|
|
name: r.get(1)?,
|
|
server_path: r.get(2)?,
|
|
created: r.get(3)?,
|
|
revision,
|
|
modified,
|
|
deleted,
|
|
verdict: verdict(local_rev.zip(local_mod), (revision, modified), deleted),
|
|
})
|
|
})?
|
|
.collect::<Result<Vec<_>, _>>()?;
|
|
rows
|
|
};
|
|
|
|
{
|
|
// One upsert covers insert, update and tombstone: the verdict has
|
|
// already decided the remote row wins, so its fields are the answer
|
|
// whichever of the three it is.
|
|
let mut take = tx.prepare(
|
|
"INSERT INTO main.albums
|
|
(uuid, name, server_path, created, revision, modified, deleted)
|
|
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7)
|
|
ON CONFLICT(uuid) DO UPDATE SET
|
|
name = excluded.name, server_path = excluded.server_path,
|
|
revision = excluded.revision, modified = excluded.modified,
|
|
deleted = excluded.deleted",
|
|
)?;
|
|
let mut forget = tx.prepare(
|
|
"DELETE FROM main.album_exports
|
|
WHERE album_id = (SELECT id FROM main.albums WHERE uuid = ?1)",
|
|
)?;
|
|
let mut unfold = tx.prepare(
|
|
"DELETE FROM main.album_folders
|
|
WHERE album_id = (SELECT id FROM main.albums WHERE uuid = ?1)",
|
|
)?;
|
|
for row in rows {
|
|
if row.verdict == MergeVerdict::KeptLocal {
|
|
report.albums_kept_local += 1;
|
|
continue;
|
|
}
|
|
take.execute(rusqlite::params![
|
|
row.uuid,
|
|
row.name,
|
|
row.server_path,
|
|
row.created,
|
|
row.revision,
|
|
row.modified,
|
|
row.deleted as i64
|
|
])?;
|
|
if row.deleted {
|
|
forget.execute([&row.uuid])?;
|
|
unfold.execute([&row.uuid])?;
|
|
} else if row.server_path.is_some() {
|
|
// Another device moved the album to the server with the newer
|
|
// revision; a local folder chosen here is no longer where it
|
|
// goes.
|
|
unfold.execute([&row.uuid])?;
|
|
}
|
|
report.albums_taken += 1;
|
|
}
|
|
}
|
|
|
|
report.album_exports_added = tx.execute(ALBUM_EXPORTS_BY_FILE_ID, [])?
|
|
+ tx.execute(ALBUM_EXPORTS_BY_CONTENT_HASH, [])?;
|
|
Ok(())
|
|
}
|
|
|
|
/// Exported files, matched to local images by the server's file id — the
|
|
/// identity [`MEMBERS_BY_FILE_ID`] explains. Tombstoned albums are excluded,
|
|
/// or a merge would refill an album it had just deleted.
|
|
const ALBUM_EXPORTS_BY_FILE_ID: &str = "
|
|
INSERT OR IGNORE INTO main.album_exports(album_id, file_name, image_id, exported_at)
|
|
SELECT la.id, re.file_name, li.id, re.exported_at
|
|
FROM remote_cat.album_exports re
|
|
JOIN remote_cat.albums ra ON ra.id = re.album_id
|
|
JOIN main.albums la ON la.uuid = ra.uuid AND la.deleted = 0
|
|
JOIN remote_cat.remote rr ON rr.image_id = re.image_id
|
|
JOIN main.remote lr ON lr.file_id = rr.file_id
|
|
JOIN main.images li ON li.id = lr.image_id";
|
|
|
|
/// The same union by content hash, for a library with no server behind it.
|
|
const ALBUM_EXPORTS_BY_CONTENT_HASH: &str = "
|
|
INSERT OR IGNORE INTO main.album_exports(album_id, file_name, image_id, exported_at)
|
|
SELECT la.id, re.file_name, li.id, re.exported_at
|
|
FROM remote_cat.album_exports re
|
|
JOIN remote_cat.albums ra ON ra.id = re.album_id
|
|
JOIN main.albums la ON la.uuid = ra.uuid AND la.deleted = 0
|
|
JOIN remote_cat.images ri ON ri.id = re.image_id
|
|
JOIN main.images li ON li.content_hash = ri.content_hash
|
|
WHERE ri.content_hash IS NOT NULL";
|
|
|
|
/// Adopt capture metadata from an attached catalog, on its own.
|
|
pub fn merge_metadata(conn: &Connection) -> Result<MergeReport, CatalogError> {
|
|
let tx = conn.unchecked_transaction()?;
|
|
let mut report = MergeReport::default();
|
|
merge_metadata_within(&tx, &mut report)?;
|
|
tx.commit()?;
|
|
Ok(report)
|
|
}
|
|
|
|
/// Capture metadata a peer's sweep already read, for images this device has
|
|
/// not dated yet.
|
|
///
|
|
/// The `images` table is local state and the merge leaves it alone — except
|
|
/// for these columns, which are not: a capture time, an offset, a camera, a
|
|
/// lens and an ISO are facts about the file's bytes, identical on every
|
|
/// device, and read by fetching a header per image across the whole library
|
|
/// (`dr_ui::library::spawn_sweep`). A fresh device inherits its peers'
|
|
/// thumbnails and faces from the shards and then spent hours re-reading
|
|
/// every header for the timeline; the snapshot it had just merged held
|
|
/// every one of those dates.
|
|
///
|
|
/// Matched by `oc:fileid`, as collection membership is. Only rows still at
|
|
/// `metadata_state < 2` take anything, and only from a remote row at 2: a
|
|
/// date this device read for itself is never overwritten, and a peer that
|
|
/// has not read one has nothing to give. The sweep's own query
|
|
/// (`metadata_state < 2`) then finds nothing left to do for them.
|
|
const METADATA_BY_FILE_ID: &str = "
|
|
UPDATE main.images
|
|
SET captured_at = r.captured_at,
|
|
captured_offset = coalesce(main.images.captured_offset, r.captured_offset),
|
|
camera = coalesce(main.images.camera, r.camera),
|
|
lens = coalesce(main.images.lens, r.lens),
|
|
iso = coalesce(main.images.iso, r.iso),
|
|
metadata_state = 2
|
|
FROM (SELECT lr.image_id, ri.captured_at, ri.captured_offset,
|
|
ri.camera, ri.lens, ri.iso
|
|
FROM remote_cat.images ri
|
|
JOIN remote_cat.remote rr ON rr.image_id = ri.id
|
|
JOIN main.remote lr ON lr.file_id = rr.file_id
|
|
WHERE ri.metadata_state >= 2 AND ri.captured_at IS NOT NULL) AS r
|
|
WHERE main.images.id = r.image_id
|
|
AND main.images.metadata_state < 2";
|
|
|
|
fn merge_metadata_within(tx: &Connection, report: &mut MergeReport) -> Result<(), CatalogError> {
|
|
// A snapshot from before these columns, or from a library with no server
|
|
// behind it, has nothing to join on.
|
|
if !remote_has(tx, "remote")?
|
|
|| !remote_has_column(tx, "images", "metadata_state")?
|
|
|| !remote_has_column(tx, "images", "captured_offset")?
|
|
{
|
|
return Ok(());
|
|
}
|
|
report.metadata_adopted = tx.execute(METADATA_BY_FILE_ID, [])?;
|
|
Ok(())
|
|
}
|
|
|
|
/// Merge people and identity judgements from an attached catalog.
|
|
///
|
|
/// The people half of [`merge_all`], on its own, for the same reason the other
|
|
/// two have one: the rules are independent and worth exercising alone.
|
|
pub fn merge_people(conn: &Connection) -> Result<MergeReport, CatalogError> {
|
|
let tx = conn.unchecked_transaction()?;
|
|
let mut report = MergeReport::default();
|
|
merge_people_within(&tx, &mut report)?;
|
|
tx.commit()?;
|
|
Ok(report)
|
|
}
|
|
|
|
/// Merge collections and membership from an attached catalog.
|
|
///
|
|
/// The collections half of [`merge_all`], on its own. Kept as a public entry
|
|
/// point because the two halves are genuinely independent, and because the
|
|
/// rules for this one are worth being able to exercise without a keyword in
|
|
/// sight.
|
|
///
|
|
/// Runs in one transaction: a merge either lands whole or not at all.
|
|
pub fn merge_collections(conn: &Connection) -> Result<MergeReport, CatalogError> {
|
|
let tx = conn.unchecked_transaction()?;
|
|
let mut report = MergeReport::default();
|
|
merge_collections_within(&tx, &mut report)?;
|
|
tx.commit()?;
|
|
Ok(report)
|
|
}
|
|
|
|
/// Merge the keyword vocabulary and its assignments from an attached catalog.
|
|
///
|
|
/// The keywords half of [`merge_all`], on its own. See the module header for
|
|
/// the three rules and the two places keywords differ from collections.
|
|
pub fn merge_keywords(conn: &Connection) -> Result<MergeReport, CatalogError> {
|
|
let tx = conn.unchecked_transaction()?;
|
|
let mut report = MergeReport::default();
|
|
merge_keywords_within(&tx, &mut report)?;
|
|
tx.commit()?;
|
|
Ok(report)
|
|
}
|
|
|
|
fn merge_collections_within(tx: &Connection, report: &mut MergeReport) -> Result<(), CatalogError> {
|
|
// ---- collections ------------------------------------------------------
|
|
{
|
|
let mut stmt = tx.prepare(
|
|
// The parent arrives as a *uuid*, not `r.parent_id`: row ids are
|
|
// local to a catalog, so the remote's integer means nothing here.
|
|
"SELECT r.uuid, r.name, rp.uuid, r.kind, r.selector_json,
|
|
r.created, r.revision, r.modified, r.deleted,
|
|
l.revision, l.modified
|
|
FROM remote_cat.collections r
|
|
LEFT JOIN main.collections l ON l.uuid = r.uuid
|
|
LEFT JOIN remote_cat.collections rp ON rp.id = r.parent_id",
|
|
)?;
|
|
|
|
struct Incoming {
|
|
uuid: String,
|
|
name: String,
|
|
kind: i64,
|
|
/// The parent's uuid, resolved to a local row id once every
|
|
/// incoming collection exists — a child can arrive before its
|
|
/// parent, so this cannot be applied inline.
|
|
parent_uuid: Option<String>,
|
|
selector_json: Option<String>,
|
|
created: i64,
|
|
revision: i64,
|
|
modified: i64,
|
|
// No `deleted` field: the verdict already encodes it, and keeping
|
|
// both invites the two disagreeing.
|
|
verdict: MergeVerdict,
|
|
}
|
|
|
|
let rows: Vec<Incoming> = stmt
|
|
.query_map([], |r| {
|
|
let deleted: i64 = r.get(8)?;
|
|
let local_rev: Option<i64> = r.get(9)?;
|
|
let local_mod: Option<i64> = r.get(10)?;
|
|
let revision: i64 = r.get(6)?;
|
|
let modified: i64 = r.get(7)?;
|
|
Ok(Incoming {
|
|
uuid: r.get(0)?,
|
|
name: r.get(1)?,
|
|
kind: r.get(3)?,
|
|
parent_uuid: r.get(2)?,
|
|
selector_json: r.get(4)?,
|
|
created: r.get(5)?,
|
|
revision,
|
|
modified,
|
|
verdict: verdict(local_rev.zip(local_mod), (revision, modified), deleted != 0),
|
|
})
|
|
})?
|
|
.collect::<Result<_, _>>()?;
|
|
|
|
// Parentage is applied after the loop: a child can arrive before its
|
|
// parent, so resolving the uuid inline would find nothing and silently
|
|
// flatten the tree.
|
|
let mut reparent: Vec<(String, Option<String>)> = Vec::new();
|
|
|
|
for row in rows {
|
|
match row.verdict {
|
|
MergeVerdict::KeptLocal => {
|
|
report.kept_local += 1;
|
|
}
|
|
MergeVerdict::InsertedFromRemote => {
|
|
tx.execute(
|
|
"INSERT INTO main.collections
|
|
(uuid, name, parent_id, kind, selector_json,
|
|
created, revision, modified, deleted)
|
|
VALUES (?1, ?2, NULL, ?3, ?4, ?5, ?6, ?7, 0)",
|
|
rusqlite::params![
|
|
row.uuid,
|
|
row.name,
|
|
row.kind,
|
|
row.selector_json,
|
|
row.created,
|
|
row.revision,
|
|
row.modified,
|
|
],
|
|
)?;
|
|
reparent.push((row.uuid.clone(), row.parent_uuid.clone()));
|
|
report.inserted += 1;
|
|
}
|
|
MergeVerdict::UpdatedFromRemote => {
|
|
tx.execute(
|
|
"UPDATE main.collections
|
|
SET name = ?2, kind = ?3, selector_json = ?4,
|
|
revision = ?5, modified = ?6, deleted = 0
|
|
WHERE uuid = ?1",
|
|
rusqlite::params![
|
|
row.uuid,
|
|
row.name,
|
|
row.kind,
|
|
row.selector_json,
|
|
row.revision,
|
|
row.modified,
|
|
],
|
|
)?;
|
|
reparent.push((row.uuid.clone(), row.parent_uuid.clone()));
|
|
report.updated += 1;
|
|
}
|
|
MergeVerdict::DeletedByRemote => {
|
|
// Tombstone rather than DELETE: the row must outlive the
|
|
// deletion or a third device reintroduces it.
|
|
tx.execute(
|
|
"INSERT INTO main.collections
|
|
(uuid, name, kind, created, revision, modified, deleted)
|
|
VALUES (?1, ?2, ?3, ?4, ?5, ?6, 1)
|
|
ON CONFLICT(uuid) DO UPDATE SET
|
|
deleted = 1, revision = ?5, modified = ?6",
|
|
rusqlite::params![
|
|
row.uuid,
|
|
row.name,
|
|
row.kind,
|
|
row.created,
|
|
row.revision,
|
|
row.modified,
|
|
],
|
|
)?;
|
|
tx.execute(
|
|
"DELETE FROM main.collection_members
|
|
WHERE collection_id = (SELECT id FROM main.collections WHERE uuid = ?1)",
|
|
[&row.uuid],
|
|
)?;
|
|
report.deleted += 1;
|
|
}
|
|
}
|
|
}
|
|
|
|
// Second pass: every incoming collection now exists locally, so a
|
|
// parent uuid can be resolved to a row id. A parent we have never seen
|
|
// resolves to NULL, which leaves the collection at the top level —
|
|
// wrong, but visible and recoverable, where a dangling id would not be.
|
|
//
|
|
// Without this the tree flattened on every sync: `parent_id` is a local
|
|
// row id and was written as NULL rather than translated, so a nested
|
|
// collection came back from a round trip at the top level.
|
|
for (uuid, parent_uuid) in reparent {
|
|
// Remote's tree is acyclic and so is ours, but the union of the two
|
|
// need not be: if we hold A above B and the remote holds B above A,
|
|
// applying only the winning half closes a loop, and `descendants`
|
|
// would then spin. Walk up from the proposed parent first; reaching
|
|
// the collection itself means this edge would close a cycle, so the
|
|
// safe move is to leave it where it is.
|
|
if let Some(ref parent) = parent_uuid {
|
|
let closes_cycle: bool = tx.query_row(
|
|
"WITH RECURSIVE up(id) AS (
|
|
SELECT id FROM main.collections WHERE uuid = ?2
|
|
UNION
|
|
SELECT c.parent_id FROM main.collections c
|
|
JOIN up ON c.id = up.id
|
|
WHERE c.parent_id IS NOT NULL
|
|
)
|
|
SELECT EXISTS(
|
|
SELECT 1 FROM up
|
|
WHERE id = (SELECT id FROM main.collections WHERE uuid = ?1))",
|
|
rusqlite::params![uuid, parent],
|
|
|r| r.get(0),
|
|
)?;
|
|
if closes_cycle {
|
|
continue;
|
|
}
|
|
}
|
|
tx.execute(
|
|
"UPDATE main.collections
|
|
SET parent_id = (SELECT id FROM main.collections WHERE uuid = ?2)
|
|
WHERE uuid = ?1",
|
|
rusqlite::params![uuid, parent_uuid],
|
|
)?;
|
|
}
|
|
}
|
|
|
|
// ---- membership -------------------------------------------------------
|
|
//
|
|
// Set union, keyed on (collection uuid, image identity). Not the image id,
|
|
// for the same reason collections use a uuid: image ids are local. An image
|
|
// the remote has and we do not is skipped — it will join when a scan or
|
|
// sync catalogues it, and the next merge picks it up.
|
|
//
|
|
// Both identities are tried, and the file id first. See
|
|
// [`MEMBERS_BY_FILE_ID`] for why keying on the content hash alone made this
|
|
// whole union a no-op on every ordinary library.
|
|
//
|
|
// Tombstoned collections are excluded, or a merge would repopulate a
|
|
// collection it had just deleted.
|
|
let added = tx.execute(MEMBERS_BY_FILE_ID, [])? + tx.execute(MEMBERS_BY_CONTENT_HASH, [])?;
|
|
report.members_added = added;
|
|
|
|
Ok(())
|
|
}
|
|
|
|
/// Take membership for images both devices know by the server's file id.
|
|
///
|
|
/// **This is the identity that exists.** Membership was keyed on
|
|
/// `images.content_hash` alone, and the schema is explicit that the column is
|
|
/// "computed only when something needs it (import dedup, reconnect-by-hash),
|
|
/// never in a scan" — so on an ordinary library it is NULL for every row, the
|
|
/// join matched nothing, and `WHERE ri.content_hash IS NOT NULL` discarded what
|
|
/// little was left. Collections synced their names, because those are keyed on
|
|
/// a uuid, and arrived empty on every device. A 23,000-image library had a
|
|
/// content hash for none of them and an `oc:fileid` for all of them.
|
|
///
|
|
/// `oc:fileid` is recorded for every image the moment a remote scan sees it, is
|
|
/// stable across server-side rename and move (FR-NC-5), and is the same integer
|
|
/// on every device pointed at the same Nextcloud — which is exactly the
|
|
/// situation where two devices share collections. It is already what the
|
|
/// thumbnail shards are keyed by, and what [`ASSIGN_BY_FILE_ID`] uses for
|
|
/// keywords; membership was the one thing left behind.
|
|
const MEMBERS_BY_FILE_ID: &str = "
|
|
INSERT OR IGNORE INTO main.collection_members(collection_id, image_id, position, added)
|
|
SELECT lc.id, li.id, rm.position, rm.added
|
|
FROM remote_cat.collection_members rm
|
|
JOIN remote_cat.collections rc ON rc.id = rm.collection_id
|
|
JOIN main.collections lc ON lc.uuid = rc.uuid AND lc.deleted = 0
|
|
JOIN remote_cat.remote rr ON rr.image_id = rm.image_id
|
|
JOIN main.remote lr ON lr.file_id = rr.file_id
|
|
JOIN main.images li ON li.id = lr.image_id";
|
|
|
|
/// The same union for a library with no server behind it.
|
|
///
|
|
/// A local-only library has no `remote` rows at all, so [`MEMBERS_BY_FILE_ID`]
|
|
/// matches nothing and the content hash is the only identity available. Kept
|
|
/// rather than replaced: where a hash *has* been computed — an imported card,
|
|
/// a reconnect — it is a true identity, and one that survives a library moving
|
|
/// between servers.
|
|
///
|
|
/// Both statements run. `INSERT OR IGNORE` against the
|
|
/// `(collection_id, image_id)` primary key makes the overlap free.
|
|
const MEMBERS_BY_CONTENT_HASH: &str = "
|
|
INSERT OR IGNORE INTO main.collection_members(collection_id, image_id, position, added)
|
|
SELECT lc.id, li.id, rm.position, rm.added
|
|
FROM remote_cat.collection_members rm
|
|
JOIN remote_cat.collections rc ON rc.id = rm.collection_id
|
|
JOIN main.collections lc ON lc.uuid = rc.uuid AND lc.deleted = 0
|
|
JOIN remote_cat.images ri ON ri.id = rm.image_id
|
|
JOIN main.images li ON li.content_hash = ri.content_hash
|
|
WHERE ri.content_hash IS NOT NULL";
|
|
|
|
/// Schema name the downloaded remote catalog is attached under.
|
|
///
|
|
/// Repeated from [`crate::sync`] rather than shared, because the SQL below
|
|
/// spells it inline and a constant that only half the file used would be worse
|
|
/// than no constant at all.
|
|
const REMOTE: &str = "remote_cat";
|
|
|
|
/// The keyword half. See the module header.
|
|
fn merge_keywords_within(tx: &Connection, report: &mut MergeReport) -> Result<(), CatalogError> {
|
|
// ---- the vocabulary ---------------------------------------------------
|
|
//
|
|
// A remote written before schema v6 has no `keyword_terms` at all, and
|
|
// `remote_is_mergeable` deliberately admits it: the check is that the
|
|
// remote is not *newer* than us. So the table's absence is a normal state
|
|
// and not an error. Its assignments still merge below — those have been in
|
|
// the schema since v1 — and its words gain identities on that device the
|
|
// next time it opens the catalog and backfills.
|
|
if attached_has_table(tx, REMOTE, "keyword_terms")? {
|
|
struct Incoming {
|
|
uuid: String,
|
|
name: String,
|
|
/// What this device currently calls the same identity, if it has
|
|
/// it. A rename is applied to the assignment rows by rewriting this
|
|
/// text, so it has to be read before the term row is overwritten.
|
|
local_name: Option<String>,
|
|
created: i64,
|
|
revision: i64,
|
|
modified: i64,
|
|
verdict: MergeVerdict,
|
|
}
|
|
|
|
let rows: Vec<Incoming> = {
|
|
let mut stmt = tx.prepare(
|
|
"SELECT r.uuid, r.name, r.created, r.revision, r.modified, r.deleted,
|
|
l.name, l.revision, l.modified
|
|
FROM remote_cat.keyword_terms r
|
|
LEFT JOIN main.keyword_terms l ON l.uuid = r.uuid",
|
|
)?;
|
|
let found = stmt
|
|
.query_map([], |r| {
|
|
let deleted: i64 = r.get(5)?;
|
|
let local_rev: Option<i64> = r.get(7)?;
|
|
let local_mod: Option<i64> = r.get(8)?;
|
|
let revision: i64 = r.get(3)?;
|
|
let modified: i64 = r.get(4)?;
|
|
Ok(Incoming {
|
|
uuid: r.get(0)?,
|
|
name: r.get(1)?,
|
|
local_name: r.get(6)?,
|
|
created: r.get(2)?,
|
|
revision,
|
|
modified,
|
|
verdict: verdict(
|
|
local_rev.zip(local_mod),
|
|
(revision, modified),
|
|
deleted != 0,
|
|
),
|
|
})
|
|
})?
|
|
.collect::<Result<Vec<_>, _>>()?;
|
|
found
|
|
};
|
|
|
|
for row in rows {
|
|
match row.verdict {
|
|
MergeVerdict::KeptLocal => {
|
|
report.keywords_kept_local += 1;
|
|
}
|
|
MergeVerdict::InsertedFromRemote => {
|
|
tx.execute(
|
|
"INSERT INTO main.keyword_terms
|
|
(uuid, name, created, revision, modified, deleted)
|
|
VALUES (?1, ?2, ?3, ?4, ?5, 0)",
|
|
rusqlite::params![
|
|
row.uuid,
|
|
row.name,
|
|
row.created,
|
|
row.revision,
|
|
row.modified,
|
|
],
|
|
)?;
|
|
report.keywords_inserted += 1;
|
|
}
|
|
MergeVerdict::UpdatedFromRemote => {
|
|
// The assignments carry the *word*, so taking a new name
|
|
// for an identity we already hold means rewriting every row
|
|
// spelt the old way. Without this the vocabulary would show
|
|
// the new spelling and the search would only find the old.
|
|
if let Some(old) = row.local_name.filter(|n| *n != row.name) {
|
|
tx.execute(
|
|
"INSERT OR IGNORE INTO main.keywords(version_id, keyword)
|
|
SELECT version_id, ?2 FROM main.keywords WHERE keyword = ?1",
|
|
rusqlite::params![old, row.name],
|
|
)?;
|
|
tx.execute("DELETE FROM main.keywords WHERE keyword = ?1", [&old])?;
|
|
}
|
|
tx.execute(
|
|
"UPDATE main.keyword_terms
|
|
SET name = ?2, revision = ?3, modified = ?4, deleted = 0
|
|
WHERE uuid = ?1",
|
|
rusqlite::params![row.uuid, row.name, row.revision, row.modified],
|
|
)?;
|
|
report.keywords_updated += 1;
|
|
}
|
|
MergeVerdict::DeletedByRemote => {
|
|
// Tombstone rather than DELETE, or a third device
|
|
// reintroduces the keyword through us.
|
|
tx.execute(
|
|
"INSERT INTO main.keyword_terms
|
|
(uuid, name, created, revision, modified, deleted)
|
|
VALUES (?1, ?2, ?3, ?4, ?5, 1)
|
|
ON CONFLICT(uuid) DO UPDATE SET
|
|
deleted = 1, revision = ?4, modified = ?5",
|
|
rusqlite::params![
|
|
row.uuid,
|
|
row.name,
|
|
row.created,
|
|
row.revision,
|
|
row.modified,
|
|
],
|
|
)?;
|
|
// **By name, not by identity.** This device may well have
|
|
// minted its own uuid for the same word before the two ever
|
|
// synced, in which case deleting by uuid would tombstone a
|
|
// row that nothing is assigned to and leave every
|
|
// photograph still carrying the word.
|
|
tx.execute("DELETE FROM main.keywords WHERE keyword = ?1", [&row.name])?;
|
|
report.keywords_deleted += 1;
|
|
}
|
|
}
|
|
}
|
|
|
|
// Two devices that each typed "Iceland" now hold two identities for one
|
|
// word. Collapse them before the assignments arrive, so the vocabulary
|
|
// the user sees after a sync has one row per word.
|
|
report.keywords_fused = crate::keywords::fuse_duplicates(tx)?;
|
|
}
|
|
|
|
// ---- assignments ------------------------------------------------------
|
|
//
|
|
// Set union, and the union is the whole point: FR-NC-9's principle applied
|
|
// to metadata rather than to edit nodes. Two devices that keyworded
|
|
// different frames "puffin" both keep their work, and neither loses it to
|
|
// whichever synced second.
|
|
//
|
|
// A removal therefore does not propagate — the remote's assignment simply
|
|
// reappears. That is the same trade-off collection membership makes above,
|
|
// and for the same reason: an unwanted keyword is removed again in a
|
|
// second, and a silently lost afternoon of keywording is not recoverable at
|
|
// all. Making removal propagate needs a tombstone per assignment, which is
|
|
// a schema change and a merge rule of its own.
|
|
//
|
|
// An incoming keyword lands on the local default version, so an image that
|
|
// has not got one yet would silently drop it. That is not a rare state: the
|
|
// invariant is maintained by a backfill on open, and a scan that ran since
|
|
// has added rows it has not covered. Losing a word the user typed on
|
|
// another device, for a bookkeeping reason, would be the wrong answer —
|
|
// this is idempotent and writes nothing once the invariant holds.
|
|
crate::rating::ensure_default_versions_within(tx)?;
|
|
|
|
// Two passes rather than one statement with an `OR`, because they resolve
|
|
// *different identities* for the same photograph and each wants its own
|
|
// index. See [`ASSIGN_BY_FILE_ID`] for why there are two at all.
|
|
for sql in [ASSIGN_BY_FILE_ID, ASSIGN_BY_CONTENT_HASH] {
|
|
report.keywords_assigned += tx.execute(sql, [])?;
|
|
}
|
|
|
|
Ok(())
|
|
}
|
|
|
|
/// Take assignments for images both devices know by the server's file id.
|
|
///
|
|
/// **Preferred over the content hash**, and the reason is that `content_hash`
|
|
/// is expensive — the schema says so, and it is computed only when import
|
|
/// dedup or a reconnect asks for it, which for most libraries is never. Keying
|
|
/// keywords on it alone would mean the union quietly did nothing for the
|
|
/// ordinary image, which is the exact failure this merge exists to prevent.
|
|
///
|
|
/// `oc:fileid` is the opposite: it is recorded for every image the moment a
|
|
/// remote scan sees it, it is stable across server-side renames and moves, and
|
|
/// it is the same integer on every device pointed at the same Nextcloud — which
|
|
/// is precisely the situation where two devices are keywording one library.
|
|
///
|
|
/// The keyword lands on the local image's **default version**, not on the
|
|
/// version it came from. Version uuids do not reconcile across devices in the
|
|
/// catalog: [`crate::rating::ensure_default_versions`] mints a fresh one per
|
|
/// device, so the same photograph's default versions have different uuids on
|
|
/// two machines and a uuid-keyed join would union nothing at all. Version
|
|
/// identity is reconciled in the *sidecar* (FR-NC-8), and until a merged
|
|
/// version arrives through there, the default version is both where
|
|
/// [`crate::keywords::assign`] writes and where the panel reads — so it is the
|
|
/// one place the word can land and be seen.
|
|
///
|
|
/// A word is refused when this device holds it only as a tombstone: deleted
|
|
/// under some identity and live under none. That is a set of words, the same
|
|
/// for every row, so it is asked once -- a list SQLite builds before the walk
|
|
/// -- rather than as a correlated `NOT EXISTS ... OR EXISTS` per incoming
|
|
/// assignment. The `deleted = 1` half of that could use no index
|
|
/// (`keyword_terms_name` holds only live rows) and scanned the whole
|
|
/// vocabulary for each of 10,800 assignments: 70 ms of a sync pass that
|
|
/// changed nothing, on the reference library, against 11 ms now.
|
|
const ASSIGN_BY_FILE_ID: &str = "
|
|
INSERT OR IGNORE INTO main.keywords(version_id, keyword)
|
|
SELECT lv.id, rk.keyword
|
|
FROM remote_cat.keywords rk
|
|
JOIN remote_cat.versions rv ON rv.id = rk.version_id
|
|
JOIN remote_cat.remote rr ON rr.image_id = rv.image_id
|
|
JOIN main.remote lr ON lr.file_id = rr.file_id
|
|
JOIN main.versions lv ON lv.image_id = lr.image_id AND lv.is_default = 1
|
|
WHERE rk.keyword NOT IN (SELECT name FROM main.keyword_terms WHERE deleted = 1
|
|
EXCEPT
|
|
SELECT name FROM main.keyword_terms WHERE deleted = 0)";
|
|
|
|
/// The same union for a library with no server behind it.
|
|
///
|
|
/// A local-only library has no `remote` rows at all, so [`ASSIGN_BY_FILE_ID`]
|
|
/// matches nothing and this is the only identity available — the same pair
|
|
/// collection membership uses, so a library where membership merges has
|
|
/// keywords that merge too.
|
|
const ASSIGN_BY_CONTENT_HASH: &str = "
|
|
INSERT OR IGNORE INTO main.keywords(version_id, keyword)
|
|
SELECT lv.id, rk.keyword
|
|
FROM remote_cat.keywords rk
|
|
JOIN remote_cat.versions rv ON rv.id = rk.version_id
|
|
JOIN remote_cat.images ri ON ri.id = rv.image_id
|
|
JOIN main.images li ON li.content_hash = ri.content_hash
|
|
JOIN main.versions lv ON lv.image_id = li.id AND lv.is_default = 1
|
|
WHERE ri.content_hash IS NOT NULL
|
|
AND rk.keyword NOT IN (SELECT name FROM main.keyword_terms WHERE deleted = 1
|
|
EXCEPT
|
|
SELECT name FROM main.keyword_terms WHERE deleted = 0)";
|
|
|
|
/// Whether an attached database holds a table of this name.
|
|
///
|
|
/// The schema name and the table name are both literals from this file, never
|
|
/// user text — but they are still bound rather than formatted where SQLite
|
|
/// allows it, because the habit is what keeps the one that eventually is user
|
|
/// text from being formatted by accident.
|
|
fn attached_has_table(conn: &Connection, schema: &str, table: &str) -> Result<bool, CatalogError> {
|
|
let sql =
|
|
format!("SELECT count(*) FROM {schema}.sqlite_master WHERE type = 'table' AND name = ?1");
|
|
let n: i64 = conn.query_row(&sql, [table], |r| r.get(0))?;
|
|
Ok(n > 0)
|
|
}
|
|
|
|
// ── people, and who the user said they are ────────────────────────────────
|
|
|
|
/// Merge people and the user's identity judgements from an attached catalog.
|
|
///
|
|
/// # Why this is here at all
|
|
///
|
|
/// Face *data* syncs as sealed shards ([`crate::face_shard`]) — boxes,
|
|
/// landmarks, embeddings, the run marker. What the shards deliberately do not
|
|
/// carry is who anybody **is**: the person rows, their names, and the
|
|
/// assignments joining the two. Those were supposed to travel in the catalog
|
|
/// snapshot, which is a whole-file copy and therefore does contain them — but
|
|
/// the snapshot is *merged*, not adopted, and this merge only ever looked at
|
|
/// collections and keywords. So a second device received every face and no
|
|
/// people at all, and drew an empty People screen over a full catalog.
|
|
///
|
|
/// # What travels, and what is recomputed
|
|
///
|
|
/// The rule this module already follows for the rest of the catalog: user
|
|
/// judgements travel, inference is rebuilt. Concretely (docs/dev/faces.md, and the
|
|
/// asymmetry `crate::faces` opens with):
|
|
///
|
|
/// - **People** — uuid, name, and whether the user set them aside. Merged by
|
|
/// uuid on `revision`, exactly as a collection is.
|
|
/// - **Confirmations** — the user said this face is this person.
|
|
/// - **Rejections** — the user said it is *not*, which is equally a fact and
|
|
/// is why re-clustering does not put it back.
|
|
/// - **The assignments inside an ignored group** — carried even though they are
|
|
/// only suggestions, because they are what anchors the ignore. Without them
|
|
/// a group set aside on one device reappears on the other, which is the same
|
|
/// fault that made "Not interested" not stick locally.
|
|
///
|
|
/// Ordinary suggestions are *not* carried. They are this pass's own output,
|
|
/// clustering is deterministic, and both devices hold the same embeddings — so
|
|
/// each recomputes them and arrives at the same answer. Shipping them would
|
|
/// double the merge for no new information.
|
|
///
|
|
/// # Faces have no cross-device identity, so one is derived
|
|
///
|
|
/// `faces.id` is a local row id and means nothing in another catalog; there is
|
|
/// no uuid to fall back on. What both devices *do* agree on is `oc:fileid`,
|
|
/// the box, and the embedding, so a remote face is matched to the local face
|
|
/// on the same photograph whose box overlaps it, above a floor of 0.5 IoU —
|
|
/// or, where no box or two boxes do, whose vector it decisively resembles
|
|
/// ([`match_faces`]).
|
|
///
|
|
/// That is not a new rule: it is the one
|
|
/// [`crate::faces::record_detections`] already uses to carry a confirmation
|
|
/// across a re-index, and it is loose on purpose — the question is "is this the
|
|
/// same face in the frame", not "is this the same rectangle", and a device
|
|
/// running a newer detector is entitled to have moved the box.
|
|
fn merge_people_within(tx: &Connection, report: &mut MergeReport) -> Result<(), CatalogError> {
|
|
// A remote written before faces existed has none of these tables, and one
|
|
// written before V10 has no `ignored`. Both are ordinary — `remote_is_
|
|
// mergeable` admits any catalog at or below this schema version — so they
|
|
// are probed for rather than assumed, and an absent one skips this half
|
|
// instead of aborting a merge that would otherwise have succeeded.
|
|
if !remote_has(tx, "people")? || !remote_has(tx, "faces")? {
|
|
return Ok(());
|
|
}
|
|
let ignored_col = remote_has_column(tx, "people", "ignored")?;
|
|
// Spelled into the SQL rather than branched around it: the two queries
|
|
// below would otherwise each need a second copy.
|
|
let ignored_sel = if ignored_col { "r.ignored" } else { "0" };
|
|
|
|
// ---- the people themselves -------------------------------------------
|
|
struct Incoming {
|
|
uuid: String,
|
|
name: String,
|
|
ignored: bool,
|
|
created: i64,
|
|
revision: i64,
|
|
modified: i64,
|
|
/// Resolved after every person exists, since a merge target can arrive
|
|
/// after the person redirecting to it.
|
|
merged_into_uuid: Option<String>,
|
|
verdict: MergeVerdict,
|
|
}
|
|
|
|
let rows: Vec<Incoming> = {
|
|
let mut stmt = tx.prepare(&format!(
|
|
"SELECT r.uuid, r.name, {ignored_sel}, r.created, r.revision, r.modified,
|
|
rm.uuid, l.revision, l.modified
|
|
FROM remote_cat.people r
|
|
LEFT JOIN main.people l ON l.uuid = r.uuid
|
|
LEFT JOIN remote_cat.people rm ON rm.id = r.merged_into"
|
|
))?;
|
|
let mapped = stmt.query_map([], |r| {
|
|
let revision: i64 = r.get(4)?;
|
|
let modified: i64 = r.get(5)?;
|
|
let local_rev: Option<i64> = r.get(7)?;
|
|
let local_mod: Option<i64> = r.get(8)?;
|
|
Ok(Incoming {
|
|
uuid: r.get(0)?,
|
|
name: r.get(1)?,
|
|
ignored: r.get(2)?,
|
|
created: r.get(3)?,
|
|
revision,
|
|
modified,
|
|
merged_into_uuid: r.get(6)?,
|
|
// People have no tombstone: a person is merged away rather
|
|
// than deleted, and `merged_into` is that redirect.
|
|
verdict: verdict(local_rev.zip(local_mod), (revision, modified), false),
|
|
})
|
|
})?;
|
|
mapped.collect::<Result<_, _>>()?
|
|
};
|
|
|
|
for p in &rows {
|
|
match p.verdict {
|
|
MergeVerdict::InsertedFromRemote => {
|
|
tx.execute(
|
|
"INSERT INTO people (uuid, name, ignored, created, revision, modified)
|
|
VALUES (?1, ?2, ?3, ?4, ?5, ?6)",
|
|
rusqlite::params![p.uuid, p.name, p.ignored, p.created, p.revision, p.modified],
|
|
)?;
|
|
report.people_inserted += 1;
|
|
}
|
|
MergeVerdict::UpdatedFromRemote | MergeVerdict::DeletedByRemote => {
|
|
tx.execute(
|
|
"UPDATE people
|
|
SET name = ?2, ignored = ?3, revision = ?4, modified = ?5
|
|
WHERE uuid = ?1",
|
|
rusqlite::params![p.uuid, p.name, p.ignored, p.revision, p.modified],
|
|
)?;
|
|
report.people_updated += 1;
|
|
}
|
|
MergeVerdict::KeptLocal => report.people_kept_local += 1,
|
|
}
|
|
}
|
|
|
|
// Redirects, once every person on both sides exists locally.
|
|
for p in rows.iter().filter(|p| p.merged_into_uuid.is_some()) {
|
|
if p.verdict == MergeVerdict::KeptLocal {
|
|
continue;
|
|
}
|
|
tx.execute(
|
|
"UPDATE people
|
|
SET merged_into = (SELECT id FROM people WHERE uuid = ?2)
|
|
WHERE uuid = ?1",
|
|
rusqlite::params![p.uuid, p.merged_into_uuid],
|
|
)?;
|
|
}
|
|
|
|
// ---- match the remote's faces onto this device's ----------------------
|
|
let matched = match_faces(tx)?;
|
|
report.faces_matched_by_embedding += matched.by_embedding;
|
|
let FaceMatch {
|
|
map: face_map,
|
|
on_file,
|
|
file_of,
|
|
..
|
|
} = matched;
|
|
if face_map.is_empty() {
|
|
return Ok(());
|
|
}
|
|
|
|
// ---- confirmations, and the anchors under an ignored group -----------
|
|
{
|
|
// The person is resolved in the same statement, by the local
|
|
// `people.uuid` key, and what this device already holds is read once
|
|
// for the whole pass and looked up in memory. A steady-state pass
|
|
// walks every confirmed and every ignored face the other device
|
|
// holds -- 13,000 on the reference library -- and three statements
|
|
// per face, even cached, were 52 ms of it. In the key's order, which
|
|
// is the order the table is walked in anyway: when two of its faces
|
|
// match one of ours, which one is applied last decides the answer.
|
|
let mut stmt = tx.prepare(&format!(
|
|
"SELECT fp.face_id, lp.id, fp.probability, fp.confirmed
|
|
FROM remote_cat.face_person fp
|
|
JOIN remote_cat.people p ON p.id = fp.person_id
|
|
JOIN main.people lp ON lp.uuid = p.uuid
|
|
WHERE fp.confirmed = 1 OR {} = 1
|
|
ORDER BY fp.face_id",
|
|
if ignored_col { "p.ignored" } else { "0" }
|
|
))?;
|
|
let incoming: Vec<(i64, i64, f64, bool)> = stmt
|
|
.query_map([], |r| Ok((r.get(0)?, r.get(1)?, r.get(2)?, r.get(3)?)))?
|
|
.collect::<Result<_, _>>()?;
|
|
|
|
// Kept current as the loop writes: two of the other device's faces
|
|
// can match one of ours, and the second must see what the first left.
|
|
let mut held: std::collections::HashMap<i64, (i64, f64, i64)> = tx
|
|
.prepare("SELECT face_id, person_id, probability, confirmed FROM main.face_person")?
|
|
.query_map([], |r| Ok((r.get(0)?, (r.get(1)?, r.get(2)?, r.get(3)?))))?
|
|
.collect::<Result<_, _>>()?;
|
|
let rejected: std::collections::HashSet<(i64, i64)> = tx
|
|
.prepare("SELECT face_id, person_id FROM main.face_person_rejected")?
|
|
.query_map([], |r| Ok((r.get(0)?, r.get(1)?)))?
|
|
.collect::<Result<_, _>>()?;
|
|
let mut assign = tx.prepare_cached(
|
|
"INSERT INTO face_person (face_id, person_id, probability, confirmed)
|
|
VALUES (?1, ?2, ?3, ?4)
|
|
ON CONFLICT(face_id) DO UPDATE SET
|
|
person_id = excluded.person_id,
|
|
probability = excluded.probability,
|
|
confirmed = excluded.confirmed",
|
|
)?;
|
|
let mut withdraw =
|
|
tx.prepare_cached("DELETE FROM face_person WHERE face_id = ?1 AND confirmed = 0")?;
|
|
|
|
for (remote_face, person, probability, confirmed) in incoming {
|
|
let Some(&local_face) = face_map.get(&remote_face) else {
|
|
continue;
|
|
};
|
|
let current = held.get(&local_face).copied();
|
|
|
|
// A local confirmation is never overwritten, in either direction.
|
|
// Two devices confirming the same face as different people is a
|
|
// genuine disagreement and there is no revision on an assignment to
|
|
// settle it with; silently taking the remote's answer would let a
|
|
// sync undo something the user did here. It stays as it is, and the
|
|
// user can change it on the device they are looking at.
|
|
if current.is_some_and(|(_, _, confirmed)| confirmed == 1) {
|
|
report.faces_kept_local += 1;
|
|
continue;
|
|
}
|
|
|
|
// A rejection here outranks an assignment from elsewhere: it is
|
|
// this user's judgement about this pair, and re-suggesting what
|
|
// they pushed away is the behaviour that makes the feature feel
|
|
// broken.
|
|
if rejected.contains(&(local_face, person)) {
|
|
continue;
|
|
}
|
|
|
|
// One person, one face per photograph -- the cannot-link the
|
|
// grouping pass already keeps (`dr_face::cluster`), which the
|
|
// merge did not. When the two devices disagree about *which*
|
|
// face in a frame is somebody, taking the remote's answer
|
|
// beside this device's own puts the person on both. The
|
|
// reference library holds 80 such pairs (79 set-aside
|
|
// strangers, one named person), the same on both devices. The
|
|
// face this device already gave the person keeps them, unless
|
|
// the remote's is a confirmation and this device's only a
|
|
// suggestion.
|
|
//
|
|
// A face that already holds the person adds nothing beside it,
|
|
// whatever else the photograph holds.
|
|
let adds_person = current.is_none_or(|(held_person, ..)| held_person != person);
|
|
let rival = file_of
|
|
.get(&local_face)
|
|
.filter(|_| adds_person)
|
|
.and_then(|file| {
|
|
on_file[file].iter().copied().find(|&other| {
|
|
other != local_face && held.get(&other).is_some_and(|h| h.0 == person)
|
|
})
|
|
});
|
|
if let Some(rival) = rival {
|
|
if !confirmed || held[&rival].2 == 1 {
|
|
report.faces_one_per_photograph += 1;
|
|
continue;
|
|
}
|
|
withdraw.execute([rival])?;
|
|
held.remove(&rival);
|
|
}
|
|
|
|
// Written only when it differs. Rewriting a row with the values it
|
|
// already holds dirtied a page per face, every pass, for nothing;
|
|
// the report still counts it, as it always has.
|
|
let wanted = (person, probability, i64::from(confirmed));
|
|
if current != Some(wanted) {
|
|
assign.execute(rusqlite::params![
|
|
local_face,
|
|
person,
|
|
probability,
|
|
confirmed
|
|
])?;
|
|
held.insert(local_face, wanted);
|
|
}
|
|
report.faces_assigned += 1;
|
|
}
|
|
}
|
|
|
|
// ---- rejections, as a set union --------------------------------------
|
|
{
|
|
let mut stmt = tx.prepare(
|
|
"SELECT fr.face_id, p.uuid
|
|
FROM remote_cat.face_person_rejected fr
|
|
JOIN remote_cat.people p ON p.id = fr.person_id",
|
|
)?;
|
|
let incoming: Vec<(i64, String)> = stmt
|
|
.query_map([], |r| Ok((r.get(0)?, r.get(1)?)))?
|
|
.collect::<Result<_, _>>()?;
|
|
|
|
let mut person_of = tx.prepare_cached("SELECT id FROM people WHERE uuid = ?1")?;
|
|
let mut reject = tx.prepare_cached(
|
|
"INSERT OR IGNORE INTO face_person_rejected (face_id, person_id)
|
|
VALUES (?1, ?2)",
|
|
)?;
|
|
let mut unsuggest = tx.prepare_cached(
|
|
"DELETE FROM face_person
|
|
WHERE face_id = ?1 AND person_id = ?2 AND confirmed = 0",
|
|
)?;
|
|
|
|
for (remote_face, uuid) in incoming {
|
|
let Some(&local_face) = face_map.get(&remote_face) else {
|
|
continue;
|
|
};
|
|
let person: Option<i64> = person_of.query_row([&uuid], |r| r.get(0)).optional()?;
|
|
let Some(person) = person else { continue };
|
|
|
|
let n = reject.execute([local_face, person])?;
|
|
report.faces_rejected += n;
|
|
|
|
// A rejection that lands on a face currently *suggested* to be
|
|
// that person has to take the suggestion with it, or the screen
|
|
// keeps offering exactly what the other device just refused.
|
|
unsuggest.execute([local_face, person])?;
|
|
}
|
|
}
|
|
|
|
Ok(())
|
|
}
|
|
|
|
/// Whether the attached remote holds a table.
|
|
fn remote_has(tx: &Connection, table: &str) -> Result<bool, CatalogError> {
|
|
Ok(tx.query_row(
|
|
"SELECT EXISTS(SELECT 1 FROM remote_cat.sqlite_master
|
|
WHERE type = 'table' AND name = ?1)",
|
|
[table],
|
|
|r| r.get::<_, bool>(0),
|
|
)?)
|
|
}
|
|
|
|
/// Whether a table in the attached remote holds a column.
|
|
fn remote_has_column(tx: &Connection, table: &str, column: &str) -> Result<bool, CatalogError> {
|
|
// `pragma_table_info` takes the schema as a second argument, which is the
|
|
// only way to ask about an attached database rather than the main one.
|
|
let mut stmt =
|
|
tx.prepare("SELECT 1 FROM pragma_table_info(?1, 'remote_cat') WHERE name = ?2")?;
|
|
Ok(stmt.exists(rusqlite::params![table, column])?)
|
|
}
|
|
|
|
/// The cosine above which two vectors from one embedder, on one
|
|
/// photograph, on two devices, are taken to be the same face when the boxes
|
|
/// do not say so.
|
|
///
|
|
/// Measured on the reference library against the tablet's snapshot
|
|
/// (2026-09-26, #77), both `w600k_mbf`: of the 169,548 pairs of *different*
|
|
/// faces in one photograph, four reach 0.7 and none 0.83 -- lookalikes in
|
|
/// one frame, a parent and child. Of the 18,348 pairs the boxes match, 94%
|
|
/// are above 0.9; the tail below is one face cut by two detectors, which is
|
|
/// why this never overrules a box that matches on its own. At 0.6 two of
|
|
/// the pairs it would claim carry different people on the two devices; at
|
|
/// 0.7 none of the twenty it claims does, and ten carry the same person on
|
|
/// both. Stricter than [`crate::faces::SAME_FACE_COSINE`] because that one
|
|
/// is only asked about boxes that overlap, and this one is asked about boxes
|
|
/// that do not.
|
|
const SAME_FACE_ACROSS_DEVICES: f32 = 0.7;
|
|
|
|
/// How far a face's best counterpart must lead its second best, on both
|
|
/// sides, for the embedding to decide. A face that two others resemble
|
|
/// almost equally is exactly the one a merge must not guess at; every pair
|
|
/// the rule claims on the reference library leads by more than 0.5.
|
|
const DECISIVE_MARGIN: f32 = 0.2;
|
|
|
|
/// What [`match_faces`] found.
|
|
#[derive(Default)]
|
|
struct FaceMatch {
|
|
/// Remote face row id to local face row id; one-to-one.
|
|
map: std::collections::HashMap<i64, i64>,
|
|
/// Every local face on a synced photograph, by the photograph's
|
|
/// cross-device id -- what "another face in the same photograph" means
|
|
/// to the merge.
|
|
on_file: std::collections::HashMap<i64, Vec<i64>>,
|
|
/// The inverse of `on_file`.
|
|
file_of: std::collections::HashMap<i64, i64>,
|
|
/// Pairs the boxes could not settle and the embeddings did.
|
|
by_embedding: usize,
|
|
}
|
|
|
|
/// Remote face row id to local face row id, by photograph, box and vector.
|
|
///
|
|
/// See [`merge_people_within`] for why a face has no shared identity and this
|
|
/// has to be derived. Faces are compared within an *embedder*
|
|
/// (`faces::embedder_of`), not within an exact pipeline id: two detectors in
|
|
/// front of the same embedder draw boxes around the same faces, and a
|
|
/// confirmation made on one device's box is about the face, not the
|
|
/// rectangle — the same judgement `faces::record_detections` makes when it
|
|
/// carries a confirmation across a re-detection. Keying on the exact id was
|
|
/// what let a detector change strand every name on the device that made it.
|
|
///
|
|
/// # Two passes, and the second is rare
|
|
///
|
|
/// **By box.** A remote face and a local one on the same photograph are the
|
|
/// same face when their boxes overlap by at least 0.5 IoU and neither has
|
|
/// another such candidate. That settles 18,348 of the reference library's
|
|
/// 19,052 remote faces, reads no vector, and is the whole of a steady-state
|
|
/// pass.
|
|
///
|
|
/// **By embedding**, only on the photographs where a remote face is left
|
|
/// over -- no box overlapped it, or two did. Their vectors are read (a few
|
|
/// hundred photographs, not the library's 19 MB of them) and a remote face is
|
|
/// paired with the local face it resembles most when the cosine is at least
|
|
/// [`SAME_FACE_ACROSS_DEVICES`], the pair is each other's best, each leads
|
|
/// its runner-up by [`DECISIVE_MARGIN`], and the local face was not already
|
|
/// claimed by a box. That is the face whose box one device drew somewhere
|
|
/// else -- twenty on the reference library, boxes at IoU 0 with cosines of
|
|
/// 0.72 to 0.96 -- and the face between two overlapping boxes. Anything less
|
|
/// decisive stays unmatched, which is what a new face is: a name that fails
|
|
/// to cross can be given again, a name put on the wrong face is a false
|
|
/// merge the user has to find.
|
|
fn match_faces(tx: &Connection) -> Result<FaceMatch, CatalogError> {
|
|
use std::collections::HashMap;
|
|
|
|
/// Loose on purpose — "the same face in the frame", not "the same
|
|
/// rectangle". The figure `record_detections` uses for the same job.
|
|
const MIN_IOU: f32 = 0.5;
|
|
|
|
type Boxed = (i64, f32, f32, f32, f32);
|
|
type Key = (i64, String);
|
|
|
|
ensure_face_box_index(tx);
|
|
|
|
let read_boxes = |sql: &str| -> Result<HashMap<Key, Vec<Boxed>>, CatalogError> {
|
|
let mut out: HashMap<Key, Vec<Boxed>> = HashMap::new();
|
|
let mut stmt = tx.prepare(sql)?;
|
|
let rows = stmt.query_map([], |r| {
|
|
Ok((
|
|
r.get::<_, i64>(1)?,
|
|
r.get::<_, String>(2)?,
|
|
(
|
|
r.get::<_, i64>(0)?,
|
|
r.get::<_, f64>(3)? as f32,
|
|
r.get::<_, f64>(4)? as f32,
|
|
r.get::<_, f64>(5)? as f32,
|
|
r.get::<_, f64>(6)? as f32,
|
|
),
|
|
))
|
|
})?;
|
|
for row in rows {
|
|
let (file_id, model, boxed) = row?;
|
|
let embedder = crate::faces::embedder_of(&model).to_string();
|
|
out.entry((file_id, embedder)).or_default().push(boxed);
|
|
}
|
|
Ok(out)
|
|
};
|
|
|
|
// Local faces, grouped by the photograph's cross-device id.
|
|
let local = read_boxes(
|
|
"SELECT f.id, r.file_id, f.model_id, f.x, f.y, f.w, f.h
|
|
FROM main.faces f
|
|
JOIN main.remote r ON r.image_id = f.image_id
|
|
WHERE r.file_id IS NOT NULL",
|
|
)?;
|
|
if local.is_empty() {
|
|
return Ok(FaceMatch::default());
|
|
}
|
|
let mut out = FaceMatch::default();
|
|
for ((file_id, _), faces) in &local {
|
|
let on = out.on_file.entry(*file_id).or_default();
|
|
for &(id, ..) in faces {
|
|
on.push(id);
|
|
out.file_of.insert(id, *file_id);
|
|
}
|
|
}
|
|
|
|
let remote = read_boxes(
|
|
"SELECT f.id, r.file_id, f.model_id, f.x, f.y, f.w, f.h
|
|
FROM remote_cat.faces f
|
|
JOIN remote_cat.remote r ON r.image_id = f.image_id
|
|
WHERE r.file_id IS NOT NULL",
|
|
)?;
|
|
|
|
// ---- by box -----------------------------------------------------------
|
|
// Per group, which local face (by index) each remote face took.
|
|
let mut left_over: Vec<(&Key, Vec<Option<usize>>)> = Vec::new();
|
|
for (key, theirs) in &remote {
|
|
let Some(ours) = local.get(key) else {
|
|
continue;
|
|
};
|
|
let overlaps: Vec<Vec<bool>> = theirs
|
|
.iter()
|
|
.map(|&(_, x, y, w, h)| {
|
|
ours.iter()
|
|
.map(|&(_, lx, ly, lw, lh)| iou((x, y, w, h), (lx, ly, lw, lh)) >= MIN_IOU)
|
|
.collect()
|
|
})
|
|
.collect();
|
|
let mut taken: Vec<Option<usize>> = vec![None; theirs.len()];
|
|
for (i, row) in overlaps.iter().enumerate() {
|
|
let mut hits = row.iter().enumerate().filter(|(_, &hit)| hit);
|
|
let (Some((j, _)), None) = (hits.next(), hits.next()) else {
|
|
continue;
|
|
};
|
|
if overlaps.iter().filter(|other| other[j]).count() == 1 {
|
|
taken[i] = Some(j);
|
|
out.map.insert(theirs[i].0, ours[j].0);
|
|
}
|
|
}
|
|
// Worth reading vectors for only where a remote face is still
|
|
// unplaced and a local face is still free to be its counterpart.
|
|
let free = ours.len() > taken.iter().flatten().count();
|
|
if free && taken.iter().any(Option::is_none) {
|
|
left_over.push((key, taken));
|
|
}
|
|
}
|
|
if left_over.is_empty() {
|
|
return Ok(out);
|
|
}
|
|
|
|
// ---- by embedding, for what the boxes left --------------------------
|
|
let wanted = |side: &HashMap<Key, Vec<Boxed>>| -> String {
|
|
let ids: Vec<String> = left_over
|
|
.iter()
|
|
.flat_map(|(key, _)| side[*key].iter().map(|b| b.0.to_string()))
|
|
.collect();
|
|
format!("[{}]", ids.join(","))
|
|
};
|
|
// One statement per side, keyed by row id, for the faces of those
|
|
// photographs only.
|
|
let read_vectors = |schema: &str, ids: String| -> Result<HashMap<i64, Vec<u8>>, CatalogError> {
|
|
let mut stmt = tx.prepare(&format!(
|
|
"SELECT f.id, f.embedding
|
|
FROM json_each(?1) j
|
|
JOIN {schema}.faces f ON f.id = j.value"
|
|
))?;
|
|
let rows = stmt.query_map([ids], |r| Ok((r.get(0)?, r.get(1)?)))?;
|
|
Ok(rows.collect::<Result<_, _>>()?)
|
|
};
|
|
let our_vectors = read_vectors("main", wanted(&local))?;
|
|
let their_vectors = read_vectors("remote_cat", wanted(&remote))?;
|
|
|
|
for (key, taken) in left_over {
|
|
let model = dr_face::ModelId::new(key.1.as_str());
|
|
let decode =
|
|
|vectors: &HashMap<i64, Vec<u8>>, faces: &[Boxed]| -> Vec<Option<dr_face::Embedding>> {
|
|
faces
|
|
.iter()
|
|
.map(|b| {
|
|
let blob = vectors.get(&b.0)?;
|
|
dr_face::Embedding::from_f16_bytes(model.clone(), blob)
|
|
})
|
|
.collect()
|
|
};
|
|
let (theirs, ours) = (&remote[key], &local[key]);
|
|
let pairs = pair_by_embedding(
|
|
&decode(&their_vectors, theirs),
|
|
&decode(&our_vectors, ours),
|
|
&taken,
|
|
);
|
|
for (i, j) in pairs {
|
|
out.map.insert(theirs[i].0, ours[j].0);
|
|
out.by_embedding += 1;
|
|
}
|
|
}
|
|
Ok(out)
|
|
}
|
|
|
|
/// The pairs the embeddings decide, as `(remote index, local index)`, for the
|
|
/// remote faces the boxes left unplaced (`taken[i] == None`).
|
|
///
|
|
/// Both sides are compared in full -- a local face a box already claimed can
|
|
/// still be a remote face's best resemblance, and then that remote face is
|
|
/// not placed elsewhere, because its best counterpart is spoken for and its
|
|
/// second best is not decisive. Vectors that are missing or of another
|
|
/// embedder compare as nothing ([`dr_face::Embedding::cosine`]).
|
|
fn pair_by_embedding(
|
|
theirs: &[Option<dr_face::Embedding>],
|
|
ours: &[Option<dr_face::Embedding>],
|
|
taken: &[Option<usize>],
|
|
) -> Vec<(usize, usize)> {
|
|
let cos: Vec<Vec<f32>> = theirs
|
|
.iter()
|
|
.map(|t| {
|
|
ours.iter()
|
|
.map(|o| match (t, o) {
|
|
(Some(t), Some(o)) => t.cosine(o).unwrap_or(f32::NEG_INFINITY),
|
|
_ => f32::NEG_INFINITY,
|
|
})
|
|
.collect()
|
|
})
|
|
.collect();
|
|
|
|
/// The index of the largest value, and by how much it leads the next.
|
|
fn best(values: impl Iterator<Item = f32>) -> Option<(usize, f32, f32)> {
|
|
let mut first: Option<(usize, f32)> = None;
|
|
let mut second = f32::NEG_INFINITY;
|
|
for (at, v) in values.enumerate() {
|
|
match first {
|
|
Some((_, top)) if v <= top => second = second.max(v),
|
|
_ => {
|
|
if let Some((_, top)) = first {
|
|
second = top;
|
|
}
|
|
first = Some((at, v));
|
|
}
|
|
}
|
|
}
|
|
first.map(|(at, top)| (at, top, second))
|
|
}
|
|
let decisive =
|
|
|top: f32, second: f32| top >= SAME_FACE_ACROSS_DEVICES && top - second >= DECISIVE_MARGIN;
|
|
|
|
let claimed: std::collections::HashSet<usize> = taken.iter().flatten().copied().collect();
|
|
let mut pairs = Vec::new();
|
|
for (i, row) in cos.iter().enumerate() {
|
|
if taken[i].is_some() {
|
|
continue;
|
|
}
|
|
let Some((j, top, second)) = best(row.iter().copied()) else {
|
|
continue;
|
|
};
|
|
if claimed.contains(&j) || !decisive(top, second) {
|
|
continue;
|
|
}
|
|
let Some((back, top, second)) = best(cos.iter().map(|row| row[j])) else {
|
|
continue;
|
|
};
|
|
if back == i && decisive(top, second) {
|
|
pairs.push((i, j));
|
|
}
|
|
}
|
|
pairs
|
|
}
|
|
|
|
/// The index the local half of [`match_faces`] is read from: every column
|
|
/// it asks of a face, so the walk touches no `faces` row.
|
|
///
|
|
/// A face row is eight kilobytes and `model_id` sits past the embedding, so
|
|
/// reading it opened the row's overflow pages: 38 ms of every sync pass on
|
|
/// the reference library, to read 19,000 boxes. From this index, 8 ms. The
|
|
/// other device's half has no such index to use -- it is a snapshot made by
|
|
/// whatever build that device runs -- but its rows have had their crops
|
|
/// stripped, which is most of their width.
|
|
///
|
|
/// Created on first use rather than by a migration, like
|
|
/// `keywords::ensure_term_index` and for the reason given there: a new
|
|
/// schema version makes older builds refuse this catalog's snapshot, and an
|
|
/// extra index is invisible to them. The first merge after an upgrade pays
|
|
/// for building it, once. A failure is logged and the merge goes on reading
|
|
/// rows, as it did before.
|
|
pub(crate) fn ensure_face_box_index(tx: &Connection) {
|
|
if let Err(e) = tx.execute_batch(
|
|
"CREATE INDEX IF NOT EXISTS main.faces_box ON faces(image_id, model_id, x, y, w, h);",
|
|
) {
|
|
log::warn!("merge: could not create faces_box: {e}");
|
|
}
|
|
}
|
|
|
|
/// Intersection over union of two `(x, y, w, h)` boxes.
|
|
pub(crate) fn iou(a: (f32, f32, f32, f32), b: (f32, f32, f32, f32)) -> f32 {
|
|
let x0 = a.0.max(b.0);
|
|
let y0 = a.1.max(b.1);
|
|
let x1 = (a.0 + a.2).min(b.0 + b.2);
|
|
let y1 = (a.1 + a.3).min(b.1 + b.3);
|
|
let inter = (x1 - x0).max(0.0) * (y1 - y0).max(0.0);
|
|
let union = a.2 * a.3 + b.2 * b.3 - inter;
|
|
if union <= 0.0 {
|
|
0.0
|
|
} else {
|
|
inter / union
|
|
}
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
use crate::schema;
|
|
|
|
#[test]
|
|
fn a_collection_we_lack_is_taken_from_remote() {
|
|
assert_eq!(
|
|
verdict(None, (1, 100), false),
|
|
MergeVerdict::InsertedFromRemote
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn higher_remote_revision_wins() {
|
|
assert_eq!(
|
|
verdict(Some((3, 100)), (4, 50), false),
|
|
MergeVerdict::UpdatedFromRemote
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn a_skewed_clock_cannot_beat_a_higher_local_revision() {
|
|
// The remote's timestamp is far in the future, but it has seen fewer
|
|
// edits. Revision decides, so the skewed device does not silently
|
|
// overwrite real work.
|
|
assert_eq!(
|
|
verdict(Some((9, 100)), (2, 999_999), false),
|
|
MergeVerdict::KeptLocal
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn equal_revisions_break_on_timestamp() {
|
|
assert_eq!(
|
|
verdict(Some((3, 100)), (3, 200), false),
|
|
MergeVerdict::UpdatedFromRemote
|
|
);
|
|
assert_eq!(
|
|
verdict(Some((3, 200)), (3, 100), false),
|
|
MergeVerdict::KeptLocal
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn an_identical_collection_is_stable() {
|
|
// Merging twice must not oscillate or report spurious changes.
|
|
assert_eq!(
|
|
verdict(Some((3, 100)), (3, 100), false),
|
|
MergeVerdict::KeptLocal
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn deletion_competes_on_revision_like_any_other_edit() {
|
|
// Remote deleted it at revision 5; we renamed it at revision 4. The
|
|
// deletion is newer, so it wins.
|
|
assert_eq!(
|
|
verdict(Some((4, 100)), (5, 100), true),
|
|
MergeVerdict::DeletedByRemote
|
|
);
|
|
// But a stale deletion does not undo a newer local edit.
|
|
assert_eq!(
|
|
verdict(Some((6, 100)), (5, 100), true),
|
|
MergeVerdict::KeptLocal
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn a_tombstone_for_something_we_never_had_is_recorded() {
|
|
// Otherwise this device could reintroduce the collection to a third.
|
|
assert_eq!(verdict(None, (2, 100), true), MergeVerdict::DeletedByRemote);
|
|
}
|
|
|
|
// ---- integration over two real catalogs ------------------------------
|
|
|
|
/// A fresh device takes the capture dates a peer's sweep read, matched by
|
|
/// `oc:fileid`, and never overwrites a date it read for itself.
|
|
#[test]
|
|
fn capture_metadata_arrives_for_undated_images_only() {
|
|
let c = two_catalogs();
|
|
// Three photographs on both devices: 1 undated here and dated there;
|
|
// 2 dated on both, differently; 3 undated on both.
|
|
for id in 1..=3 {
|
|
add_image_without_hash(&c, "main", id);
|
|
add_image_without_hash(&c, "remote_cat", id + 10);
|
|
add_remote_id(&c, "main", id, 100 + id);
|
|
add_remote_id(&c, "remote_cat", id + 10, 100 + id);
|
|
}
|
|
c.execute(
|
|
"UPDATE remote_cat.images
|
|
SET captured_at = 1000, captured_offset = 60, camera = 'X', metadata_state = 2
|
|
WHERE id = 11",
|
|
[],
|
|
)
|
|
.unwrap();
|
|
c.execute(
|
|
"UPDATE remote_cat.images SET captured_at = 2000, metadata_state = 2 WHERE id = 12",
|
|
[],
|
|
)
|
|
.unwrap();
|
|
c.execute(
|
|
"UPDATE main.images SET captured_at = 2222, metadata_state = 2 WHERE id = 2",
|
|
[],
|
|
)
|
|
.unwrap();
|
|
|
|
let report = merge_metadata(&c).unwrap();
|
|
assert_eq!(report.metadata_adopted, 1);
|
|
|
|
let row = |id: i64| -> (Option<i64>, Option<i64>, Option<String>, i64) {
|
|
c.query_row(
|
|
"SELECT captured_at, captured_offset, camera, metadata_state
|
|
FROM main.images WHERE id = ?1",
|
|
[id],
|
|
|r| Ok((r.get(0)?, r.get(1)?, r.get(2)?, r.get(3)?)),
|
|
)
|
|
.unwrap()
|
|
};
|
|
assert_eq!(row(1), (Some(1000), Some(60), Some("X".into()), 2));
|
|
assert_eq!(row(2), (Some(2222), None, None, 2));
|
|
assert_eq!(row(3), (None, None, None, 0));
|
|
|
|
// Idempotent: a second pass finds nothing left to take.
|
|
assert_eq!(merge_metadata(&c).unwrap().metadata_adopted, 0);
|
|
}
|
|
|
|
fn two_catalogs() -> Connection {
|
|
attached_remote(schema::for_attached("remote_cat"))
|
|
}
|
|
|
|
/// The same pair, but with the remote stopped at v1 — a device running a
|
|
/// build from before keywords had identities.
|
|
fn two_catalogs_with_a_v1_remote() -> Connection {
|
|
attached_remote(schema::v1_for_attached("remote_cat"))
|
|
}
|
|
|
|
fn attached_remote(remote_schema: String) -> Connection {
|
|
let c = Connection::open_in_memory().unwrap();
|
|
schema::configure(&c).unwrap();
|
|
schema::migrate(&c).unwrap();
|
|
// A second in-memory database standing in for the downloaded remote.
|
|
c.execute_batch("ATTACH ':memory:' AS remote_cat").unwrap();
|
|
c.execute_batch(&remote_schema).unwrap();
|
|
c
|
|
}
|
|
|
|
/// An image with no content hash — which is every image in a library that
|
|
/// has only ever been scanned.
|
|
fn add_image_without_hash(c: &Connection, db: &str, id: i64) {
|
|
c.execute(
|
|
&format!(
|
|
"INSERT INTO {db}.roots(id, kind, label) VALUES (1, 'remote', 'lib')
|
|
ON CONFLICT(id) DO NOTHING"
|
|
),
|
|
[],
|
|
)
|
|
.unwrap();
|
|
c.execute(
|
|
&format!(
|
|
"INSERT INTO {db}.images(id, root_id, source_ref, added_at)
|
|
VALUES (?1, 1, ?2, 0)"
|
|
),
|
|
rusqlite::params![id, format!("img{id}.CR3")],
|
|
)
|
|
.unwrap();
|
|
}
|
|
|
|
fn count(c: &Connection, sql: &str) -> i64 {
|
|
c.query_row(sql, [], |r| r.get(0)).unwrap()
|
|
}
|
|
|
|
/// Give an image the `oc:fileid` a remote scan records for it.
|
|
///
|
|
/// What every image in a real synced library has, and what none of them
|
|
/// has a content hash for.
|
|
fn add_remote_id(c: &Connection, db: &str, image_id: i64, file_id: i64) {
|
|
c.execute(
|
|
&format!("INSERT INTO {db}.remote(image_id, file_id) VALUES (?1, ?2)"),
|
|
rusqlite::params![image_id, file_id],
|
|
)
|
|
.unwrap();
|
|
}
|
|
|
|
fn add_image(c: &Connection, db: &str, id: i64, hash: &str) {
|
|
c.execute(
|
|
&format!(
|
|
"INSERT INTO {db}.roots(id, kind, label) VALUES (1, 'local', 'r')
|
|
ON CONFLICT(id) DO NOTHING"
|
|
),
|
|
[],
|
|
)
|
|
.unwrap();
|
|
c.execute(
|
|
&format!(
|
|
"INSERT INTO {db}.images(id, root_id, source_ref, content_hash, added_at)
|
|
VALUES (?1, 1, ?2, ?3, 0)"
|
|
),
|
|
rusqlite::params![id, format!("img{id}.CR3"), hash],
|
|
)
|
|
.unwrap();
|
|
}
|
|
|
|
fn add_collection(c: &Connection, db: &str, id: i64, uuid: &str, name: &str, rev: i64) {
|
|
c.execute(
|
|
&format!(
|
|
"INSERT INTO {db}.collections(id, uuid, name, kind, created, revision, modified)
|
|
VALUES (?1, ?2, ?3, 0, 0, ?4, ?4)"
|
|
),
|
|
rusqlite::params![id, uuid, name, rev],
|
|
)
|
|
.unwrap();
|
|
}
|
|
|
|
/// The parent's uuid for `uuid`, or None if it sits at the top level.
|
|
fn parent_of(c: &Connection, uuid: &str) -> Option<String> {
|
|
c.query_row(
|
|
"SELECT p.uuid FROM main.collections ch
|
|
JOIN main.collections p ON p.id = ch.parent_id
|
|
WHERE ch.uuid = ?1",
|
|
[uuid],
|
|
|r| r.get(0),
|
|
)
|
|
.ok()
|
|
}
|
|
|
|
#[test]
|
|
fn a_nested_collection_arrives_still_nested() {
|
|
// The tree used to flatten on every sync: `parent_id` is a local row id,
|
|
// and the merge wrote NULL rather than translating it through the uuid.
|
|
let c = two_catalogs();
|
|
add_collection(&c, "remote_cat", 1, "uuid-holidays", "holidays", 1);
|
|
add_collection(&c, "remote_cat", 2, "uuid-arosa", "arosa", 1);
|
|
c.execute(
|
|
"UPDATE remote_cat.collections SET parent_id = 1 WHERE id = 2",
|
|
[],
|
|
)
|
|
.unwrap();
|
|
|
|
merge_collections(&c).unwrap();
|
|
|
|
assert_eq!(
|
|
parent_of(&c, "uuid-arosa").as_deref(),
|
|
Some("uuid-holidays"),
|
|
"a sub-collection must not be promoted to the top level by a merge"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn a_parent_arriving_after_its_child_still_adopts_it() {
|
|
// Row order is whatever the query returns, so the child can be inserted
|
|
// first. Parentage is applied in a second pass for exactly this reason.
|
|
let c = two_catalogs();
|
|
// Child has the lower id, so it is seen before the parent exists.
|
|
add_collection(&c, "remote_cat", 1, "uuid-child", "arosa", 1);
|
|
add_collection(&c, "remote_cat", 2, "uuid-parent", "holidays", 1);
|
|
c.execute(
|
|
"UPDATE remote_cat.collections SET parent_id = 2 WHERE id = 1",
|
|
[],
|
|
)
|
|
.unwrap();
|
|
|
|
merge_collections(&c).unwrap();
|
|
|
|
assert_eq!(
|
|
parent_of(&c, "uuid-child").as_deref(),
|
|
Some("uuid-parent"),
|
|
"resolution must not depend on the order rows happen to arrive in"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn disagreeing_devices_cannot_close_a_parent_cycle() {
|
|
// Local holds A above B; the remote holds B above A and wins on
|
|
// revision. Applying that blindly makes each the other's parent, and
|
|
// every tree walk then spins.
|
|
let c = two_catalogs();
|
|
add_collection(&c, "main", 1, "uuid-a", "A", 1);
|
|
add_collection(&c, "main", 2, "uuid-b", "B", 1);
|
|
c.execute("UPDATE main.collections SET parent_id = 1 WHERE id = 2", [])
|
|
.unwrap();
|
|
|
|
// Remote: A under B, at a higher revision so it is the winner.
|
|
add_collection(&c, "remote_cat", 1, "uuid-a", "A", 9);
|
|
add_collection(&c, "remote_cat", 2, "uuid-b", "B", 9);
|
|
c.execute(
|
|
"UPDATE remote_cat.collections SET parent_id = 2 WHERE id = 1",
|
|
[],
|
|
)
|
|
.unwrap();
|
|
|
|
merge_collections(&c).unwrap();
|
|
|
|
let a = parent_of(&c, "uuid-a");
|
|
let b = parent_of(&c, "uuid-b");
|
|
assert!(
|
|
!(a.as_deref() == Some("uuid-b") && b.as_deref() == Some("uuid-a")),
|
|
"merge closed a cycle: A parented under B and B under A"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn disjoint_collections_from_two_devices_both_survive() {
|
|
// The property the whole design exists for: neither device loses work.
|
|
let c = two_catalogs();
|
|
add_collection(&c, "main", 1, "uuid-local", "Iceland", 1);
|
|
add_collection(&c, "remote_cat", 1, "uuid-remote", "Portugal", 1);
|
|
|
|
let report = merge_collections(&c).unwrap();
|
|
assert_eq!(report.inserted, 1);
|
|
|
|
let names: Vec<String> = c
|
|
.prepare("SELECT name FROM main.collections ORDER BY name")
|
|
.unwrap()
|
|
.query_map([], |r| r.get(0))
|
|
.unwrap()
|
|
.collect::<Result<_, _>>()
|
|
.unwrap();
|
|
assert_eq!(names, vec!["Iceland", "Portugal"]);
|
|
}
|
|
|
|
#[test]
|
|
fn membership_unions_rather_than_replacing() {
|
|
// Two devices each added a different image to the same collection.
|
|
let c = two_catalogs();
|
|
add_collection(&c, "main", 1, "shared", "Trip", 1);
|
|
add_collection(&c, "remote_cat", 1, "shared", "Trip", 1);
|
|
add_image(&c, "main", 1, "hash-a");
|
|
add_image(&c, "main", 2, "hash-b");
|
|
add_image(&c, "remote_cat", 1, "hash-b");
|
|
|
|
c.execute(
|
|
"INSERT INTO main.collection_members(collection_id, image_id, added) VALUES (1, 1, 0)",
|
|
[],
|
|
)
|
|
.unwrap();
|
|
c.execute(
|
|
"INSERT INTO remote_cat.collection_members(collection_id, image_id, added)
|
|
VALUES (1, 1, 0)",
|
|
[],
|
|
)
|
|
.unwrap();
|
|
|
|
merge_collections(&c).unwrap();
|
|
|
|
let n: i64 = c
|
|
.query_row("SELECT count(*) FROM main.collection_members", [], |r| {
|
|
r.get(0)
|
|
})
|
|
.unwrap();
|
|
assert_eq!(n, 2, "both devices' additions survive");
|
|
}
|
|
|
|
#[test]
|
|
fn a_library_that_never_computed_a_hash_still_merges_membership() {
|
|
// The bug this replaces. `content_hash` is computed only by import
|
|
// dedup or a reconnect — never by a scan — so a synced library has one
|
|
// for no image at all. Keying membership on it alone meant collections
|
|
// arrived with their names and none of their contents, on every device,
|
|
// for every user.
|
|
let c = two_catalogs();
|
|
add_collection(&c, "main", 1, "shared", "Trip", 1);
|
|
add_collection(&c, "remote_cat", 1, "shared", "Trip", 1);
|
|
|
|
// No hashes anywhere, and a file id for both — a real library.
|
|
add_image_without_hash(&c, "main", 77);
|
|
add_image_without_hash(&c, "remote_cat", 3);
|
|
add_remote_id(&c, "main", 77, 90210);
|
|
add_remote_id(&c, "remote_cat", 3, 90210);
|
|
|
|
c.execute(
|
|
"INSERT INTO remote_cat.collection_members(collection_id, image_id, added)
|
|
VALUES (1, 3, 0)",
|
|
[],
|
|
)
|
|
.unwrap();
|
|
|
|
let report = merge_collections(&c).unwrap();
|
|
assert_eq!(report.members_added, 1);
|
|
|
|
let img: i64 = c
|
|
.query_row("SELECT image_id FROM main.collection_members", [], |r| {
|
|
r.get(0)
|
|
})
|
|
.unwrap();
|
|
assert_eq!(img, 77, "resolved through the server's file id");
|
|
}
|
|
|
|
#[test]
|
|
fn a_different_photograph_on_the_server_is_not_adopted() {
|
|
// The file id is an identity, so two different ids must not join.
|
|
let c = two_catalogs();
|
|
add_collection(&c, "main", 1, "shared", "Trip", 1);
|
|
add_collection(&c, "remote_cat", 1, "shared", "Trip", 1);
|
|
add_image_without_hash(&c, "main", 77);
|
|
add_image_without_hash(&c, "remote_cat", 3);
|
|
add_remote_id(&c, "main", 77, 111);
|
|
add_remote_id(&c, "remote_cat", 3, 222);
|
|
|
|
c.execute(
|
|
"INSERT INTO remote_cat.collection_members(collection_id, image_id, added)
|
|
VALUES (1, 3, 0)",
|
|
[],
|
|
)
|
|
.unwrap();
|
|
|
|
let report = merge_collections(&c).unwrap();
|
|
assert_eq!(report.members_added, 0);
|
|
assert_eq!(count(&c, "SELECT COUNT(*) FROM main.collection_members"), 0);
|
|
}
|
|
|
|
#[test]
|
|
fn an_image_identified_both_ways_is_added_once() {
|
|
// Both statements run, and their overlap must be free rather than a
|
|
// constraint violation or a double count.
|
|
let c = two_catalogs();
|
|
add_collection(&c, "main", 1, "shared", "Trip", 1);
|
|
add_collection(&c, "remote_cat", 1, "shared", "Trip", 1);
|
|
add_image(&c, "main", 77, "same-photo");
|
|
add_image(&c, "remote_cat", 3, "same-photo");
|
|
add_remote_id(&c, "main", 77, 90210);
|
|
add_remote_id(&c, "remote_cat", 3, 90210);
|
|
|
|
c.execute(
|
|
"INSERT INTO remote_cat.collection_members(collection_id, image_id, added)
|
|
VALUES (1, 3, 0)",
|
|
[],
|
|
)
|
|
.unwrap();
|
|
|
|
merge_collections(&c).unwrap();
|
|
assert_eq!(count(&c, "SELECT COUNT(*) FROM main.collection_members"), 1);
|
|
}
|
|
|
|
#[test]
|
|
fn membership_maps_across_devices_by_content_hash() {
|
|
// The same photograph carries different integer ids on each device.
|
|
// Keying on the id would attach the wrong image.
|
|
let c = two_catalogs();
|
|
add_collection(&c, "main", 1, "shared", "Trip", 1);
|
|
add_collection(&c, "remote_cat", 1, "shared", "Trip", 1);
|
|
add_image(&c, "main", 77, "same-photo");
|
|
add_image(&c, "remote_cat", 3, "same-photo");
|
|
|
|
c.execute(
|
|
"INSERT INTO remote_cat.collection_members(collection_id, image_id, added)
|
|
VALUES (1, 3, 0)",
|
|
[],
|
|
)
|
|
.unwrap();
|
|
|
|
merge_collections(&c).unwrap();
|
|
|
|
let img: i64 = c
|
|
.query_row("SELECT image_id FROM main.collection_members", [], |r| {
|
|
r.get(0)
|
|
})
|
|
.unwrap();
|
|
assert_eq!(img, 77, "resolved to the local id for the same photo");
|
|
}
|
|
|
|
#[test]
|
|
fn an_image_we_do_not_have_yet_is_skipped_not_errored() {
|
|
let c = two_catalogs();
|
|
add_collection(&c, "main", 1, "shared", "Trip", 1);
|
|
add_collection(&c, "remote_cat", 1, "shared", "Trip", 1);
|
|
add_image(&c, "remote_cat", 1, "not-here-yet");
|
|
c.execute(
|
|
"INSERT INTO remote_cat.collection_members(collection_id, image_id, added)
|
|
VALUES (1, 1, 0)",
|
|
[],
|
|
)
|
|
.unwrap();
|
|
|
|
let report = merge_collections(&c).unwrap();
|
|
assert_eq!(report.members_added, 0);
|
|
// It joins on a later merge, once a scan has catalogued the file.
|
|
}
|
|
|
|
#[test]
|
|
fn a_remote_deletion_does_not_resurrect_via_membership() {
|
|
let c = two_catalogs();
|
|
add_collection(&c, "main", 1, "doomed", "Old", 1);
|
|
add_image(&c, "main", 1, "hash-a");
|
|
add_image(&c, "remote_cat", 1, "hash-a");
|
|
c.execute(
|
|
"INSERT INTO remote_cat.collections(id, uuid, name, kind, created, revision, modified, deleted)
|
|
VALUES (1, 'doomed', 'Old', 0, 0, 5, 5, 1)",
|
|
[],
|
|
)
|
|
.unwrap();
|
|
c.execute(
|
|
"INSERT INTO remote_cat.collection_members(collection_id, image_id, added)
|
|
VALUES (1, 1, 0)",
|
|
[],
|
|
)
|
|
.unwrap();
|
|
|
|
let report = merge_collections(&c).unwrap();
|
|
assert_eq!(report.deleted, 1);
|
|
let n: i64 = c
|
|
.query_row("SELECT count(*) FROM main.collection_members", [], |r| {
|
|
r.get(0)
|
|
})
|
|
.unwrap();
|
|
assert_eq!(n, 0, "membership must not repopulate a deleted collection");
|
|
}
|
|
|
|
#[test]
|
|
fn merging_twice_changes_nothing_the_second_time() {
|
|
let c = two_catalogs();
|
|
add_collection(&c, "remote_cat", 1, "uuid-r", "Portugal", 1);
|
|
|
|
let first = merge_collections(&c).unwrap();
|
|
assert!(first.local_changed());
|
|
|
|
let second = merge_collections(&c).unwrap();
|
|
assert!(!second.local_changed(), "merge must be idempotent");
|
|
}
|
|
|
|
// ---- keywords --------------------------------------------------------
|
|
|
|
/// Give an image a default version, as every write path assumes it has.
|
|
fn add_version(c: &Connection, db: &str, image: i64, uuid: &str) -> i64 {
|
|
c.execute(
|
|
&format!(
|
|
"INSERT INTO {db}.versions(image_id, uuid, name, is_default)
|
|
VALUES (?1, ?2, 'Default', 1)"
|
|
),
|
|
rusqlite::params![image, uuid],
|
|
)
|
|
.unwrap();
|
|
c.last_insert_rowid()
|
|
}
|
|
|
|
/// Put a word on an image's default version, creating the version.
|
|
///
|
|
/// The version uuid is derived from the database *and* the image, so the
|
|
/// two catalogs never accidentally agree on one — which is the real
|
|
/// situation, and the reason the assignment union cannot key on it.
|
|
fn keyword(c: &Connection, db: &str, image: i64, word: &str) {
|
|
let existing: Option<i64> = c
|
|
.query_row(
|
|
&format!("SELECT id FROM {db}.versions WHERE image_id = ?1 AND is_default = 1"),
|
|
[image],
|
|
|r| r.get(0),
|
|
)
|
|
.ok();
|
|
let version =
|
|
existing.unwrap_or_else(|| add_version(c, db, image, &format!("v-{db}-{image}")));
|
|
c.execute(
|
|
&format!("INSERT OR IGNORE INTO {db}.keywords(version_id, keyword) VALUES (?1, ?2)"),
|
|
rusqlite::params![version, word],
|
|
)
|
|
.unwrap();
|
|
}
|
|
|
|
fn add_term(c: &Connection, db: &str, uuid: &str, name: &str, rev: i64, deleted: i64) {
|
|
c.execute(
|
|
&format!(
|
|
"INSERT INTO {db}.keyword_terms(uuid, name, created, revision, modified, deleted)
|
|
VALUES (?1, ?2, 0, ?3, ?3, ?4)"
|
|
),
|
|
rusqlite::params![uuid, name, rev, deleted],
|
|
)
|
|
.unwrap();
|
|
}
|
|
|
|
/// Map an image to a server file id, as a remote scan does.
|
|
fn add_file_id(c: &Connection, db: &str, image: i64, file_id: i64) {
|
|
c.execute(
|
|
&format!("INSERT INTO {db}.remote(image_id, file_id) VALUES (?1, ?2)"),
|
|
rusqlite::params![image, file_id],
|
|
)
|
|
.unwrap();
|
|
}
|
|
|
|
/// Every word on an image locally, sorted.
|
|
fn words_on(c: &Connection, image: i64) -> Vec<String> {
|
|
let mut stmt = c
|
|
.prepare(
|
|
"SELECT DISTINCT k.keyword FROM main.keywords k
|
|
JOIN main.versions v ON v.id = k.version_id
|
|
WHERE v.image_id = ?1 ORDER BY k.keyword",
|
|
)
|
|
.unwrap();
|
|
let rows = stmt.query_map([image], |r| r.get(0)).unwrap();
|
|
rows.collect::<Result<Vec<_>, _>>().unwrap()
|
|
}
|
|
|
|
fn live_terms(c: &Connection) -> Vec<String> {
|
|
let mut stmt = c
|
|
.prepare("SELECT name FROM main.keyword_terms WHERE deleted = 0 ORDER BY name")
|
|
.unwrap();
|
|
let rows = stmt.query_map([], |r| r.get(0)).unwrap();
|
|
rows.collect::<Result<Vec<_>, _>>().unwrap()
|
|
}
|
|
|
|
#[test]
|
|
fn two_devices_keywording_different_photographs_both_survive() {
|
|
// FR-NC-9's principle applied to metadata: disjoint work merges to the
|
|
// union, and neither device loses an afternoon to whoever synced last.
|
|
let c = two_catalogs();
|
|
for db in ["main", "remote_cat"] {
|
|
add_image(&c, db, 1, "hash-a");
|
|
add_image(&c, db, 2, "hash-b");
|
|
}
|
|
keyword(&c, "main", 1, "puffin");
|
|
keyword(&c, "remote_cat", 2, "gannet");
|
|
|
|
merge_keywords(&c).unwrap();
|
|
|
|
assert_eq!(words_on(&c, 1), ["puffin"]);
|
|
assert_eq!(words_on(&c, 2), ["gannet"]);
|
|
}
|
|
|
|
#[test]
|
|
fn two_devices_keywording_one_photograph_keep_both_words() {
|
|
// The case the union is really for: the same frame, two different
|
|
// words, and last-writer-wins would silently drop one of them.
|
|
let c = two_catalogs();
|
|
for db in ["main", "remote_cat"] {
|
|
add_image(&c, db, 1, "hash-a");
|
|
}
|
|
keyword(&c, "main", 1, "puffin");
|
|
keyword(&c, "remote_cat", 1, "Iceland");
|
|
|
|
let report = merge_keywords(&c).unwrap();
|
|
assert_eq!(report.keywords_assigned, 1);
|
|
assert_eq!(words_on(&c, 1), ["Iceland", "puffin"]);
|
|
}
|
|
|
|
#[test]
|
|
fn a_word_both_devices_already_had_is_not_duplicated() {
|
|
let c = two_catalogs();
|
|
for db in ["main", "remote_cat"] {
|
|
add_image(&c, db, 1, "hash-a");
|
|
keyword(&c, db, 1, "puffin");
|
|
}
|
|
|
|
let report = merge_keywords(&c).unwrap();
|
|
assert_eq!(report.keywords_assigned, 0);
|
|
assert_eq!(words_on(&c, 1), ["puffin"]);
|
|
}
|
|
|
|
#[test]
|
|
fn keywords_reach_an_image_the_server_names_but_no_one_has_hashed() {
|
|
// `content_hash` is computed only when import dedup or a reconnect asks
|
|
// for it, so for most images it is NULL — and a union keyed on it alone
|
|
// would quietly do nothing for the ordinary photograph. The file id is
|
|
// recorded by every remote scan, which is exactly the situation where
|
|
// two devices are keywording one library.
|
|
let c = two_catalogs();
|
|
c.execute(
|
|
"INSERT INTO main.roots(id, kind, label) VALUES (1, 'remote', 'r')",
|
|
[],
|
|
)
|
|
.unwrap();
|
|
c.execute(
|
|
"INSERT INTO remote_cat.roots(id, kind, label) VALUES (1, 'remote', 'r')",
|
|
[],
|
|
)
|
|
.unwrap();
|
|
// Different row ids for one photograph, and no hash on either side.
|
|
c.execute(
|
|
"INSERT INTO main.images(id, root_id, source_ref, added_at)
|
|
VALUES (77, 1, 'IMG_1.CR3', 0)",
|
|
[],
|
|
)
|
|
.unwrap();
|
|
c.execute(
|
|
"INSERT INTO remote_cat.images(id, root_id, source_ref, added_at)
|
|
VALUES (3, 1, 'IMG_1.CR3', 0)",
|
|
[],
|
|
)
|
|
.unwrap();
|
|
add_file_id(&c, "main", 77, 9001);
|
|
add_file_id(&c, "remote_cat", 3, 9001);
|
|
add_version(&c, "main", 77, "v-main");
|
|
keyword(&c, "remote_cat", 3, "puffin");
|
|
|
|
merge_keywords(&c).unwrap();
|
|
assert_eq!(words_on(&c, 77), ["puffin"]);
|
|
}
|
|
|
|
#[test]
|
|
fn keywords_map_across_devices_by_content_hash_where_there_is_no_server() {
|
|
// A local-only library has no `remote` rows at all, so the hash is the
|
|
// only identity available — and it is the one membership already uses.
|
|
let c = two_catalogs();
|
|
add_image(&c, "main", 77, "same-photo");
|
|
add_image(&c, "remote_cat", 3, "same-photo");
|
|
add_version(&c, "main", 77, "v-main");
|
|
keyword(&c, "remote_cat", 3, "puffin");
|
|
|
|
merge_keywords(&c).unwrap();
|
|
assert_eq!(words_on(&c, 77), ["puffin"]);
|
|
}
|
|
|
|
#[test]
|
|
fn the_vocabulary_merges_by_uuid_and_a_skewed_clock_cannot_win() {
|
|
let c = two_catalogs();
|
|
add_term(&c, "main", "u-1", "Iceland", 9, 0);
|
|
add_term(&c, "remote_cat", "u-1", "iceland", 2, 0);
|
|
add_term(&c, "remote_cat", "u-2", "puffin", 1, 0);
|
|
|
|
let report = merge_keywords(&c).unwrap();
|
|
assert_eq!(report.keywords_kept_local, 1);
|
|
assert_eq!(report.keywords_inserted, 1);
|
|
assert_eq!(live_terms(&c), ["Iceland", "puffin"]);
|
|
}
|
|
|
|
#[test]
|
|
fn a_remote_rename_moves_this_device_s_assignments_too() {
|
|
// The failure this exists to stop: the vocabulary shows the corrected
|
|
// spelling and the search still only finds the old one.
|
|
let c = two_catalogs();
|
|
add_image(&c, "main", 1, "hash-a");
|
|
add_term(&c, "main", "u-1", "Icland", 1, 0);
|
|
keyword(&c, "main", 1, "Icland");
|
|
add_term(&c, "remote_cat", "u-1", "Iceland", 4, 0);
|
|
|
|
let report = merge_keywords(&c).unwrap();
|
|
assert_eq!(report.keywords_updated, 1);
|
|
assert_eq!(live_terms(&c), ["Iceland"]);
|
|
assert_eq!(words_on(&c, 1), ["Iceland"]);
|
|
}
|
|
|
|
#[test]
|
|
fn a_remote_deletion_takes_the_word_off_every_photograph() {
|
|
let c = two_catalogs();
|
|
for db in ["main", "remote_cat"] {
|
|
add_image(&c, db, 1, "hash-a");
|
|
}
|
|
add_term(&c, "main", "u-1", "blurry", 1, 0);
|
|
keyword(&c, "main", 1, "blurry");
|
|
add_term(&c, "remote_cat", "u-1", "blurry", 5, 1);
|
|
|
|
let report = merge_keywords(&c).unwrap();
|
|
assert_eq!(report.keywords_deleted, 1);
|
|
assert!(live_terms(&c).is_empty());
|
|
assert!(words_on(&c, 1).is_empty());
|
|
}
|
|
|
|
#[test]
|
|
fn a_deletion_is_not_undone_by_the_union_on_the_same_pass() {
|
|
// The remote deleted the word *and* still carries assignments for it —
|
|
// it has not yet had the chance to sweep them, or a third device put
|
|
// them there. Without the tombstone filter the union would put the word
|
|
// straight back on the photograph the deletion had just cleared.
|
|
let c = two_catalogs();
|
|
for db in ["main", "remote_cat"] {
|
|
add_image(&c, db, 1, "hash-a");
|
|
}
|
|
add_term(&c, "main", "u-1", "blurry", 1, 0);
|
|
keyword(&c, "main", 1, "blurry");
|
|
add_term(&c, "remote_cat", "u-1", "blurry", 5, 1);
|
|
keyword(&c, "remote_cat", 1, "blurry");
|
|
|
|
merge_keywords(&c).unwrap();
|
|
assert!(words_on(&c, 1).is_empty(), "a deleted keyword came back");
|
|
}
|
|
|
|
#[test]
|
|
fn a_deletion_lands_even_when_the_two_devices_minted_different_uuids() {
|
|
// Both typed "blurry" before they ever synced, so this device's row has
|
|
// a uuid the remote has never heard of. Deleting by identity would
|
|
// tombstone nothing and leave every photograph still carrying the word.
|
|
let c = two_catalogs();
|
|
add_image(&c, "main", 1, "hash-a");
|
|
add_term(&c, "main", "mine", "blurry", 1, 0);
|
|
keyword(&c, "main", 1, "blurry");
|
|
add_term(&c, "remote_cat", "theirs", "blurry", 5, 1);
|
|
|
|
merge_keywords(&c).unwrap();
|
|
assert!(words_on(&c, 1).is_empty());
|
|
}
|
|
|
|
#[test]
|
|
fn two_devices_that_typed_one_word_end_up_with_one_keyword() {
|
|
// Neither is wrong until they meet, which is why the name carries no
|
|
// unique index — a constraint would abort the merge at this moment.
|
|
let c = two_catalogs();
|
|
add_term(&c, "main", "zzzz", "Iceland", 3, 0);
|
|
add_term(&c, "remote_cat", "aaaa", "Iceland", 1, 0);
|
|
|
|
let report = merge_keywords(&c).unwrap();
|
|
assert_eq!(report.keywords_fused, 1);
|
|
assert_eq!(live_terms(&c), ["Iceland"]);
|
|
|
|
let survivor: String = c
|
|
.query_row("SELECT uuid FROM main.keyword_terms", [], |r| r.get(0))
|
|
.unwrap();
|
|
assert_eq!(
|
|
survivor, "aaaa",
|
|
"both devices must pick the same survivor without asking each other"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn merging_keywords_twice_changes_nothing_the_second_time() {
|
|
let c = two_catalogs();
|
|
for db in ["main", "remote_cat"] {
|
|
add_image(&c, db, 1, "hash-a");
|
|
}
|
|
add_term(&c, "remote_cat", "u-1", "puffin", 1, 0);
|
|
keyword(&c, "remote_cat", 1, "puffin");
|
|
|
|
let first = merge_keywords(&c).unwrap();
|
|
assert!(first.local_changed());
|
|
let second = merge_keywords(&c).unwrap();
|
|
assert!(!second.local_changed(), "merge must be idempotent");
|
|
}
|
|
|
|
#[test]
|
|
fn a_remote_from_before_keyword_identities_still_contributes_its_words() {
|
|
// `remote_is_mergeable` admits an older remote on purpose — the check
|
|
// is that it is not *newer* than us. A missing table is therefore a
|
|
// normal state and must not fail the merge.
|
|
let c = two_catalogs_with_a_v1_remote();
|
|
for db in ["main", "remote_cat"] {
|
|
add_image(&c, db, 1, "hash-a");
|
|
}
|
|
keyword(&c, "remote_cat", 1, "puffin");
|
|
|
|
let report = merge_keywords(&c).unwrap();
|
|
assert_eq!(report.keywords_assigned, 1);
|
|
assert_eq!(words_on(&c, 1), ["puffin"]);
|
|
}
|
|
|
|
#[test]
|
|
fn merge_all_lands_both_halves() {
|
|
let c = two_catalogs();
|
|
add_collection(&c, "remote_cat", 1, "u-coll", "Portugal", 1);
|
|
for db in ["main", "remote_cat"] {
|
|
add_image(&c, db, 1, "hash-a");
|
|
}
|
|
keyword(&c, "remote_cat", 1, "puffin");
|
|
|
|
let report = merge_all(&c).unwrap();
|
|
assert_eq!(report.inserted, 1);
|
|
assert_eq!(report.keywords_assigned, 1);
|
|
}
|
|
|
|
#[test]
|
|
fn keeping_local_still_marks_the_catalog_for_upload() {
|
|
// We hold something the remote does not, so the remote is stale even
|
|
// though we took nothing from it.
|
|
let c = two_catalogs();
|
|
add_collection(&c, "main", 1, "shared", "Renamed here", 5);
|
|
add_collection(&c, "remote_cat", 1, "shared", "Old name", 2);
|
|
|
|
let report = merge_collections(&c).unwrap();
|
|
assert_eq!(report.kept_local, 1);
|
|
assert!(report.should_upload());
|
|
}
|
|
|
|
// ── people, and who the user said they are ────────────────────────────
|
|
|
|
/// An image present in `db` and carrying the cross-device file id both
|
|
/// catalogs agree on.
|
|
fn add_synced_image(c: &Connection, db: &str, id: i64, file_id: i64) {
|
|
add_image_without_hash(c, db, id);
|
|
c.execute(
|
|
&format!("INSERT INTO {db}.remote(image_id, file_id) VALUES (?1, ?2)"),
|
|
rusqlite::params![id, file_id],
|
|
)
|
|
.unwrap();
|
|
}
|
|
|
|
/// A face on `image`, at a box the caller can nudge to test the matching.
|
|
fn add_face(c: &Connection, db: &str, id: i64, image: i64, x: f64) -> i64 {
|
|
c.execute(
|
|
&format!(
|
|
"INSERT INTO {db}.faces
|
|
(id, image_id, x, y, w, h, landmarks, detector_confidence,
|
|
embedding, crop_px, model_id, detected_at)
|
|
VALUES (?1, ?2, ?3, 0.2, 0.2, 0.2, X'00', 0.9, X'00', 150.0,
|
|
'w600k_mbf', 0)"
|
|
),
|
|
rusqlite::params![id, image, x],
|
|
)
|
|
.unwrap();
|
|
id
|
|
}
|
|
|
|
fn add_person(c: &Connection, db: &str, id: i64, uuid: &str, name: &str, ignored: bool) {
|
|
c.execute(
|
|
&format!(
|
|
"INSERT INTO {db}.people(id, uuid, name, ignored, created, revision, modified)
|
|
VALUES (?1, ?2, ?3, ?4, 0, 1, 1)"
|
|
),
|
|
rusqlite::params![id, uuid, name, ignored],
|
|
)
|
|
.unwrap();
|
|
}
|
|
|
|
fn assign(c: &Connection, db: &str, face: i64, person: i64, confirmed: bool) {
|
|
c.execute(
|
|
&format!(
|
|
"INSERT INTO {db}.face_person(face_id, person_id, probability, confirmed)
|
|
VALUES (?1, ?2, 0.9, ?3)"
|
|
),
|
|
rusqlite::params![face, person, confirmed],
|
|
)
|
|
.unwrap();
|
|
}
|
|
|
|
fn person_of(c: &Connection, face: i64) -> Option<(String, bool)> {
|
|
c.query_row(
|
|
"SELECT p.name, fp.confirmed
|
|
FROM main.face_person fp JOIN main.people p ON p.id = fp.person_id
|
|
WHERE fp.face_id = ?1",
|
|
[face],
|
|
|r| Ok((r.get(0)?, r.get(1)?)),
|
|
)
|
|
.optional()
|
|
.unwrap()
|
|
}
|
|
|
|
/// The bug: a second device received every face through the shards and no
|
|
/// people at all, because this merge only ever looked at collections and
|
|
/// keywords. It drew an empty People screen over a full catalog.
|
|
#[test]
|
|
fn a_named_person_and_their_confirmed_face_cross_over() {
|
|
let c = two_catalogs();
|
|
for db in ["main", "remote_cat"] {
|
|
add_synced_image(&c, db, 1, 5000);
|
|
}
|
|
// The same face, found independently on each device, so the row ids
|
|
// differ — which is the whole difficulty.
|
|
let local = add_face(&c, "main", 7, 1, 0.30);
|
|
let remote = add_face(&c, "remote_cat", 42, 1, 0.31);
|
|
add_person(&c, "remote_cat", 3, "u-anna", "Anna", false);
|
|
assign(&c, "remote_cat", remote, 3, true);
|
|
|
|
let report = merge_all(&c).unwrap();
|
|
assert_eq!(report.people_inserted, 1);
|
|
assert_eq!(report.faces_assigned, 1);
|
|
assert_eq!(person_of(&c, local), Some(("Anna".to_string(), true)));
|
|
}
|
|
|
|
/// The bug this rule exists for: the desktop switched to a stronger
|
|
/// detector and confirmed 3,500 faces under the old pipeline id; the
|
|
/// tablet held the same faces under the new one, and not one name
|
|
/// crossed, because the match demanded the exact id. Same photograph,
|
|
/// same box, same embedder — that is the same face.
|
|
#[test]
|
|
fn a_confirmation_crosses_a_detector_change() {
|
|
let c = two_catalogs();
|
|
for db in ["main", "remote_cat"] {
|
|
add_synced_image(&c, db, 1, 5000);
|
|
}
|
|
let local = add_face(&c, "main", 7, 1, 0.30);
|
|
c.execute(
|
|
"UPDATE main.faces SET model_id = 'scrfd_10g+w600k_mbf' WHERE id = ?1",
|
|
[local],
|
|
)
|
|
.unwrap();
|
|
let remote = add_face(&c, "remote_cat", 42, 1, 0.31);
|
|
add_person(&c, "remote_cat", 3, "u-anna", "Anna", false);
|
|
assign(&c, "remote_cat", remote, 3, true);
|
|
|
|
let report = merge_all(&c).unwrap();
|
|
assert_eq!(report.faces_assigned, 1);
|
|
assert_eq!(person_of(&c, local), Some(("Anna".to_string(), true)));
|
|
}
|
|
|
|
/// A different embedder is a different space, and a box there is a face
|
|
/// nobody here has a vector for.
|
|
#[test]
|
|
fn a_confirmation_does_not_cross_an_embedder_change() {
|
|
let c = two_catalogs();
|
|
for db in ["main", "remote_cat"] {
|
|
add_synced_image(&c, db, 1, 5000);
|
|
}
|
|
let local = add_face(&c, "main", 7, 1, 0.30);
|
|
c.execute(
|
|
"UPDATE main.faces SET model_id = 'scrfd_10g+other_embedder' WHERE id = ?1",
|
|
[local],
|
|
)
|
|
.unwrap();
|
|
let remote = add_face(&c, "remote_cat", 42, 1, 0.31);
|
|
add_person(&c, "remote_cat", 3, "u-anna", "Anna", false);
|
|
assign(&c, "remote_cat", remote, 3, true);
|
|
|
|
merge_all(&c).unwrap();
|
|
assert_eq!(person_of(&c, local), None, "matched across embedders");
|
|
}
|
|
|
|
/// Boxes from two devices are close but not identical. Matching has to be
|
|
/// by overlap, not equality, or nothing ever lines up.
|
|
#[test]
|
|
fn a_face_in_a_different_photograph_is_not_matched() {
|
|
let c = two_catalogs();
|
|
for db in ["main", "remote_cat"] {
|
|
add_synced_image(&c, db, 1, 5000);
|
|
add_synced_image(&c, db, 2, 6000);
|
|
}
|
|
let elsewhere = add_face(&c, "main", 7, 2, 0.30);
|
|
let remote = add_face(&c, "remote_cat", 42, 1, 0.30);
|
|
add_person(&c, "remote_cat", 3, "u-anna", "Anna", false);
|
|
assign(&c, "remote_cat", remote, 3, true);
|
|
|
|
merge_all(&c).unwrap();
|
|
assert_eq!(person_of(&c, elsewhere), None, "matched across photographs");
|
|
}
|
|
|
|
/// Two faces in one frame, and the judgement must land on the right one.
|
|
#[test]
|
|
fn the_overlapping_face_is_the_one_that_gets_the_name() {
|
|
let c = two_catalogs();
|
|
for db in ["main", "remote_cat"] {
|
|
add_synced_image(&c, db, 1, 5000);
|
|
}
|
|
let left = add_face(&c, "main", 7, 1, 0.10);
|
|
let right = add_face(&c, "main", 8, 1, 0.70);
|
|
let remote = add_face(&c, "remote_cat", 42, 1, 0.71);
|
|
add_person(&c, "remote_cat", 3, "u-bob", "Bob", false);
|
|
assign(&c, "remote_cat", remote, 3, true);
|
|
|
|
merge_all(&c).unwrap();
|
|
assert_eq!(person_of(&c, right), Some(("Bob".to_string(), true)));
|
|
assert_eq!(person_of(&c, left), None);
|
|
}
|
|
|
|
/// A group set aside on one device stays set aside on the other — which
|
|
/// needs its *suggestions* to travel, since that is what anchors it.
|
|
#[test]
|
|
fn a_group_set_aside_stays_set_aside_on_the_other_device() {
|
|
let c = two_catalogs();
|
|
for db in ["main", "remote_cat"] {
|
|
add_synced_image(&c, db, 1, 5000);
|
|
}
|
|
let local = add_face(&c, "main", 7, 1, 0.30);
|
|
let remote = add_face(&c, "remote_cat", 42, 1, 0.30);
|
|
add_person(&c, "remote_cat", 3, "u-stranger", "", true);
|
|
// Only ever a suggestion, which is exactly why it needs carrying.
|
|
assign(&c, "remote_cat", remote, 3, false);
|
|
|
|
merge_all(&c).unwrap();
|
|
let ignored: bool = c
|
|
.query_row(
|
|
"SELECT ignored FROM main.people WHERE uuid = 'u-stranger'",
|
|
[],
|
|
|r| r.get(0),
|
|
)
|
|
.unwrap();
|
|
assert!(ignored, "the set-aside flag did not travel");
|
|
assert_eq!(person_of(&c, local), Some((String::new(), false)));
|
|
}
|
|
|
|
/// An ordinary suggestion is this pass's own output. Both devices hold the
|
|
/// same embeddings and clustering is deterministic, so each recomputes it —
|
|
/// shipping it would double the merge for no new information.
|
|
#[test]
|
|
fn an_ordinary_suggestion_does_not_travel() {
|
|
let c = two_catalogs();
|
|
for db in ["main", "remote_cat"] {
|
|
add_synced_image(&c, db, 1, 5000);
|
|
}
|
|
let local = add_face(&c, "main", 7, 1, 0.30);
|
|
let remote = add_face(&c, "remote_cat", 42, 1, 0.30);
|
|
add_person(&c, "remote_cat", 3, "u-guess", "", false);
|
|
assign(&c, "remote_cat", remote, 3, false);
|
|
|
|
merge_all(&c).unwrap();
|
|
assert_eq!(person_of(&c, local), None);
|
|
}
|
|
|
|
/// A sync must not undo what the user did on the device they are holding.
|
|
#[test]
|
|
fn a_local_confirmation_outranks_the_remotes() {
|
|
let c = two_catalogs();
|
|
for db in ["main", "remote_cat"] {
|
|
add_synced_image(&c, db, 1, 5000);
|
|
}
|
|
let local = add_face(&c, "main", 7, 1, 0.30);
|
|
let remote = add_face(&c, "remote_cat", 42, 1, 0.30);
|
|
add_person(&c, "main", 1, "u-anna", "Anna", false);
|
|
add_person(&c, "remote_cat", 3, "u-bob", "Bob", false);
|
|
assign(&c, "main", local, 1, true);
|
|
assign(&c, "remote_cat", remote, 3, true);
|
|
|
|
let report = merge_all(&c).unwrap();
|
|
assert_eq!(report.faces_kept_local, 1);
|
|
assert_eq!(person_of(&c, local), Some(("Anna".to_string(), true)));
|
|
}
|
|
|
|
/// "Not this person" is a judgement too, and it has to outrank an
|
|
/// assignment arriving from elsewhere.
|
|
#[test]
|
|
fn a_rejection_travels_and_removes_the_suggestion_it_contradicts() {
|
|
let c = two_catalogs();
|
|
for db in ["main", "remote_cat"] {
|
|
add_synced_image(&c, db, 1, 5000);
|
|
}
|
|
let local = add_face(&c, "main", 7, 1, 0.30);
|
|
let remote = add_face(&c, "remote_cat", 42, 1, 0.30);
|
|
add_person(&c, "main", 1, "u-anna", "Anna", false);
|
|
add_person(&c, "remote_cat", 3, "u-anna", "Anna", false);
|
|
// Locally suggested; the other device says it is not her.
|
|
assign(&c, "main", local, 1, false);
|
|
c.execute(
|
|
"INSERT INTO remote_cat.face_person_rejected(face_id, person_id)
|
|
VALUES (?1, 3)",
|
|
[remote],
|
|
)
|
|
.unwrap();
|
|
|
|
merge_all(&c).unwrap();
|
|
assert_eq!(person_of(&c, local), None, "the refused suggestion stayed");
|
|
let rejected: bool = c
|
|
.query_row(
|
|
"SELECT EXISTS(SELECT 1 FROM main.face_person_rejected WHERE face_id = ?1)",
|
|
[local],
|
|
|r| r.get(0),
|
|
)
|
|
.unwrap();
|
|
assert!(rejected);
|
|
}
|
|
|
|
/// A rename on the other device wins on revision, like everything else.
|
|
#[test]
|
|
fn a_higher_revision_renames_a_person() {
|
|
let c = two_catalogs();
|
|
add_person(&c, "main", 1, "u-anna", "Ana", false);
|
|
add_person(&c, "remote_cat", 3, "u-anna", "Anna", false);
|
|
c.execute(
|
|
"UPDATE remote_cat.people SET revision = 5, modified = 5 WHERE uuid = 'u-anna'",
|
|
[],
|
|
)
|
|
.unwrap();
|
|
|
|
let report = merge_all(&c).unwrap();
|
|
assert_eq!(report.people_updated, 1);
|
|
let name: String = c
|
|
.query_row(
|
|
"SELECT name FROM main.people WHERE uuid = 'u-anna'",
|
|
[],
|
|
|r| r.get(0),
|
|
)
|
|
.unwrap();
|
|
assert_eq!(name, "Anna");
|
|
}
|
|
|
|
#[test]
|
|
fn a_lower_revision_does_not_rename_a_person() {
|
|
let c = two_catalogs();
|
|
add_person(&c, "main", 1, "u-anna", "Anna", false);
|
|
add_person(&c, "remote_cat", 3, "u-anna", "Ana", false);
|
|
c.execute("UPDATE main.people SET revision = 9, modified = 9", [])
|
|
.unwrap();
|
|
|
|
let report = merge_all(&c).unwrap();
|
|
assert_eq!(report.people_kept_local, 1);
|
|
let name: String = c
|
|
.query_row(
|
|
"SELECT name FROM main.people WHERE uuid = 'u-anna'",
|
|
[],
|
|
|r| r.get(0),
|
|
)
|
|
.unwrap();
|
|
assert_eq!(name, "Anna");
|
|
}
|
|
|
|
/// Merging twice must not double anything — a sync runs on every pass.
|
|
#[test]
|
|
fn merging_people_twice_changes_nothing_the_second_time() {
|
|
let c = two_catalogs();
|
|
for db in ["main", "remote_cat"] {
|
|
add_synced_image(&c, db, 1, 5000);
|
|
}
|
|
add_face(&c, "main", 7, 1, 0.30);
|
|
let remote = add_face(&c, "remote_cat", 42, 1, 0.30);
|
|
add_person(&c, "remote_cat", 3, "u-anna", "Anna", false);
|
|
assign(&c, "remote_cat", remote, 3, true);
|
|
|
|
merge_all(&c).unwrap();
|
|
let second = merge_all(&c).unwrap();
|
|
assert_eq!(second.people_inserted, 0);
|
|
let people: i64 = c
|
|
.query_row("SELECT COUNT(*) FROM main.people", [], |r| r.get(0))
|
|
.unwrap();
|
|
assert_eq!(people, 1);
|
|
}
|
|
|
|
// ── faces the boxes cannot place, and their vectors ───────────────────
|
|
|
|
/// A unit vector in the embedder's space, the same for the same seed.
|
|
/// Two seeds are near-orthogonal, as two strangers' faces are.
|
|
fn vector(seed: u32) -> Vec<f32> {
|
|
let mut s = seed.wrapping_mul(2_654_435_761).wrapping_add(1);
|
|
let mut v: Vec<f32> = (0..dr_face::EMBEDDING_DIM)
|
|
.map(|_| {
|
|
s = s.wrapping_mul(1_664_525).wrapping_add(1_013_904_223);
|
|
(s >> 8) as f32 / (1u32 << 23) as f32 - 0.5
|
|
})
|
|
.collect();
|
|
let norm = v.iter().map(|x| x * x).sum::<f32>().sqrt();
|
|
v.iter_mut().for_each(|x| *x /= norm);
|
|
v
|
|
}
|
|
|
|
/// Store `v` as `face`'s embedding, as the embedder would.
|
|
fn embed(c: &Connection, db: &str, face: i64, v: &[f32]) {
|
|
let e = dr_face::Embedding {
|
|
model: dr_face::ModelId::new("w600k_mbf"),
|
|
v: Box::new(v.try_into().unwrap()),
|
|
};
|
|
c.execute(
|
|
&format!("UPDATE {db}.faces SET embedding = ?2 WHERE id = ?1"),
|
|
rusqlite::params![face, e.to_f16_bytes()],
|
|
)
|
|
.unwrap();
|
|
}
|
|
|
|
/// Anna confirmed on the remote's face 42.
|
|
fn anna_on(c: &Connection, remote: i64) {
|
|
add_person(c, "remote_cat", 3, "u-anna", "Anna", false);
|
|
assign(c, "remote_cat", remote, 3, true);
|
|
}
|
|
|
|
/// The case #77 was opened for: one device drew the box somewhere else —
|
|
/// on the reference library, whole photographs whose boxes sit at IoU 0
|
|
/// with cosines above 0.9 — and the name stayed behind. The vector says
|
|
/// it is the same face.
|
|
#[test]
|
|
fn a_shifted_box_with_the_same_embedding_matches() {
|
|
let c = two_catalogs();
|
|
for db in ["main", "remote_cat"] {
|
|
add_synced_image(&c, db, 1, 5000);
|
|
}
|
|
let local = add_face(&c, "main", 7, 1, 0.10);
|
|
let remote = add_face(&c, "remote_cat", 42, 1, 0.60);
|
|
embed(&c, "main", local, &vector(1));
|
|
embed(&c, "remote_cat", remote, &vector(1));
|
|
anna_on(&c, remote);
|
|
|
|
let report = merge_all(&c).unwrap();
|
|
assert_eq!(report.faces_matched_by_embedding, 1);
|
|
assert_eq!(person_of(&c, local), Some(("Anna".to_string(), true)));
|
|
}
|
|
|
|
/// Two faces close enough that both boxes overlap the remote's by more
|
|
/// than half: the box cannot say which, and must not guess. The vector
|
|
/// can.
|
|
#[test]
|
|
fn two_overlapping_faces_are_told_apart_by_embedding() {
|
|
let c = two_catalogs();
|
|
for db in ["main", "remote_cat"] {
|
|
add_synced_image(&c, db, 1, 5000);
|
|
}
|
|
let front = add_face(&c, "main", 7, 1, 0.30);
|
|
let behind = add_face(&c, "main", 8, 1, 0.36);
|
|
let remote = add_face(&c, "remote_cat", 42, 1, 0.33);
|
|
embed(&c, "main", front, &vector(1));
|
|
embed(&c, "main", behind, &vector(2));
|
|
embed(&c, "remote_cat", remote, &vector(2));
|
|
anna_on(&c, remote);
|
|
|
|
merge_all(&c).unwrap();
|
|
assert_eq!(person_of(&c, behind), Some(("Anna".to_string(), true)));
|
|
assert_eq!(person_of(&c, front), None);
|
|
}
|
|
|
|
/// The same two overlapping boxes, and vectors that do not decide: the
|
|
/// face stays unmatched rather than going to the larger overlap.
|
|
#[test]
|
|
fn an_ambiguous_box_with_no_decisive_vector_stays_unmatched() {
|
|
let c = two_catalogs();
|
|
for db in ["main", "remote_cat"] {
|
|
add_synced_image(&c, db, 1, 5000);
|
|
}
|
|
let front = add_face(&c, "main", 7, 1, 0.30);
|
|
let behind = add_face(&c, "main", 8, 1, 0.35);
|
|
let remote = add_face(&c, "remote_cat", 42, 1, 0.33);
|
|
embed(&c, "main", front, &vector(1));
|
|
embed(&c, "main", behind, &vector(2));
|
|
// Equally like both: 0.71 each, no margin.
|
|
let between: Vec<f32> = vector(1)
|
|
.iter()
|
|
.zip(vector(2))
|
|
.map(|(a, b)| (a + b) / 2f32.sqrt())
|
|
.collect();
|
|
embed(&c, "remote_cat", remote, &between);
|
|
anna_on(&c, remote);
|
|
|
|
merge_all(&c).unwrap();
|
|
assert_eq!(person_of(&c, front), None);
|
|
assert_eq!(person_of(&c, behind), None);
|
|
}
|
|
|
|
/// The same person in another photograph has the same vector — the
|
|
/// worst lookalike there is — and is not the same face. Nor is a face
|
|
/// in the right photograph that neither box nor vector ties to it: that
|
|
/// is a face this device found and the other did not, and it stays new.
|
|
#[test]
|
|
fn a_similar_embedding_in_a_different_photograph_never_matches() {
|
|
let c = two_catalogs();
|
|
for db in ["main", "remote_cat"] {
|
|
add_synced_image(&c, db, 1, 5000);
|
|
add_synced_image(&c, db, 2, 6000);
|
|
}
|
|
let elsewhere = add_face(&c, "main", 7, 2, 0.10);
|
|
let stranger = add_face(&c, "main", 8, 1, 0.10);
|
|
let remote = add_face(&c, "remote_cat", 42, 1, 0.60);
|
|
embed(&c, "main", elsewhere, &vector(1));
|
|
embed(&c, "main", stranger, &vector(2));
|
|
embed(&c, "remote_cat", remote, &vector(1));
|
|
anna_on(&c, remote);
|
|
|
|
let report = merge_all(&c).unwrap();
|
|
assert_eq!(report.faces_matched_by_embedding, 0);
|
|
assert_eq!(person_of(&c, elsewhere), None, "matched across photographs");
|
|
assert_eq!(person_of(&c, stranger), None, "a new face was matched");
|
|
}
|
|
|
|
/// Vectors from two embedders live in two spaces; a cosine between
|
|
/// them is a number that means nothing.
|
|
#[test]
|
|
fn different_embedders_never_compare() {
|
|
let c = two_catalogs();
|
|
for db in ["main", "remote_cat"] {
|
|
add_synced_image(&c, db, 1, 5000);
|
|
}
|
|
let local = add_face(&c, "main", 7, 1, 0.10);
|
|
c.execute(
|
|
"UPDATE main.faces SET model_id = 'scrfd_10g+other_embedder' WHERE id = ?1",
|
|
[local],
|
|
)
|
|
.unwrap();
|
|
let remote = add_face(&c, "remote_cat", 42, 1, 0.60);
|
|
embed(&c, "main", local, &vector(1));
|
|
embed(&c, "remote_cat", remote, &vector(1));
|
|
anna_on(&c, remote);
|
|
|
|
merge_all(&c).unwrap();
|
|
assert_eq!(person_of(&c, local), None, "matched across embedders");
|
|
}
|
|
|
|
/// A local face its box already placed is not handed to a second remote
|
|
/// face because that one resembles it: one face, one counterpart.
|
|
#[test]
|
|
fn a_face_the_box_placed_is_not_taken_again_by_a_vector() {
|
|
let c = two_catalogs();
|
|
for db in ["main", "remote_cat"] {
|
|
add_synced_image(&c, db, 1, 5000);
|
|
}
|
|
let placed = add_face(&c, "main", 7, 1, 0.10);
|
|
let free = add_face(&c, "main", 8, 1, 0.70);
|
|
let by_box = add_face(&c, "remote_cat", 41, 1, 0.10);
|
|
let remote = add_face(&c, "remote_cat", 42, 1, 0.40);
|
|
embed(&c, "main", placed, &vector(1));
|
|
embed(&c, "main", free, &vector(3));
|
|
embed(&c, "remote_cat", by_box, &vector(2));
|
|
embed(&c, "remote_cat", remote, &vector(1));
|
|
anna_on(&c, remote);
|
|
|
|
merge_all(&c).unwrap();
|
|
assert_eq!(person_of(&c, placed), None);
|
|
assert_eq!(person_of(&c, free), None);
|
|
}
|
|
|
|
// ── one person, one face per photograph ───────────────────────────────
|
|
|
|
/// Two faces far apart in one photograph, one each side's remote
|
|
/// counterpart can be matched to by box: (local 7, local 8, remote 42
|
|
/// over 8).
|
|
fn two_faces_one_photograph(c: &Connection) -> (i64, i64, i64) {
|
|
for db in ["main", "remote_cat"] {
|
|
add_synced_image(c, db, 1, 5000);
|
|
}
|
|
let here = add_face(c, "main", 7, 1, 0.10);
|
|
let there = add_face(c, "main", 8, 1, 0.60);
|
|
let remote = add_face(c, "remote_cat", 42, 1, 0.60);
|
|
(here, there, remote)
|
|
}
|
|
|
|
/// The devices disagree about which stranger in a crowd a set-aside
|
|
/// group holds. Taking the remote's anchor beside this device's own put
|
|
/// one person on two faces of one frame.
|
|
#[test]
|
|
fn a_set_aside_anchor_does_not_land_beside_this_devices_own() {
|
|
let c = two_catalogs();
|
|
let (here, there, remote) = two_faces_one_photograph(&c);
|
|
add_person(&c, "main", 1, "u-stranger", "", true);
|
|
add_person(&c, "remote_cat", 3, "u-stranger", "", true);
|
|
assign(&c, "main", here, 1, false);
|
|
assign(&c, "remote_cat", remote, 3, false);
|
|
|
|
let report = merge_all(&c).unwrap();
|
|
assert_eq!(report.faces_one_per_photograph, 1);
|
|
assert_eq!(person_of(&c, here), Some((String::new(), false)));
|
|
assert_eq!(person_of(&c, there), None);
|
|
}
|
|
|
|
/// A confirmation from the other device outranks a suggestion here for
|
|
/// the same person on another face, which gives the person up.
|
|
#[test]
|
|
fn a_remote_confirmation_moves_a_local_suggestion_off_the_other_face() {
|
|
let c = two_catalogs();
|
|
let (here, there, remote) = two_faces_one_photograph(&c);
|
|
add_person(&c, "main", 1, "u-anna", "Anna", false);
|
|
add_person(&c, "remote_cat", 3, "u-anna", "Anna", false);
|
|
assign(&c, "main", here, 1, false);
|
|
assign(&c, "remote_cat", remote, 3, true);
|
|
|
|
merge_all(&c).unwrap();
|
|
assert_eq!(person_of(&c, there), Some(("Anna".to_string(), true)));
|
|
assert_eq!(person_of(&c, here), None);
|
|
}
|
|
|
|
/// Two confirmations of one person on two faces of one photograph is a
|
|
/// disagreement no merge can settle; this device's stands, and the
|
|
/// second is not added beside it.
|
|
#[test]
|
|
fn a_remote_confirmation_does_not_double_a_local_one() {
|
|
let c = two_catalogs();
|
|
let (here, there, remote) = two_faces_one_photograph(&c);
|
|
add_person(&c, "main", 1, "u-anna", "Anna", false);
|
|
add_person(&c, "remote_cat", 3, "u-anna", "Anna", false);
|
|
assign(&c, "main", here, 1, true);
|
|
assign(&c, "remote_cat", remote, 3, true);
|
|
|
|
let report = merge_all(&c).unwrap();
|
|
assert_eq!(report.faces_one_per_photograph, 1);
|
|
assert_eq!(person_of(&c, here), Some(("Anna".to_string(), true)));
|
|
assert_eq!(person_of(&c, there), None);
|
|
}
|
|
}
|