Find catalog duplicates and fold a group onto one copy in one transaction

The library holds the same RAW in several folders: a dated folder, a
bck/ beside it, a renamed Darktable export tree. dr_catalog::duplicates
is the catalog half of consolidating them (#67).

candidates() is one grouped query over root, camera, capture instant and
size, joined back for the rows; count() is the same grouping under
COUNT. On a copy of the reference catalog (23,582 images) both take
10-35 ms and find 1,836 groups holding 3,379 spare copies.

survivor() prefers a copy outside a backup-looking folder, then one
still named the way the camera named it, then the oldest, then the
lowest id.

consolidate() re-checks the plan against the catalog, merges the copies'
judgements onto the survivor (highest rating, keywords unioned,
collections unioned with the survivor keeping its place, a flag or label
the copies agree on, faces via faces::carry_onto_copy) and records the
copies as trashed, all in one transaction, so a failure part way leaves
the group untouched. preview() runs the same code and rolls it back.

Sameness probes are kept in dedup_probes, created on first use rather
than by a migration: a schema bump would make older builds refuse this
catalog's snapshot at sync. trash::record_trashed_within lets the trash
write share the merge's transaction.
This commit is contained in:
2026-09-26 07:19:13 -04:00
parent d430ec9528
commit 3c2eacbf3f
5 changed files with 1302 additions and 18 deletions
+30 -18
View File
@@ -129,28 +129,40 @@ pub fn record_trashed(
return Ok(0);
}
let tx = conn.unchecked_transaction()?;
let mut n = 0;
{
let mut stmt = tx.prepare(
"UPDATE images
SET trashed_from = CASE
WHEN trashed_at IS NULL THEN source_ref
ELSE trashed_from
END,
source_ref = ?2,
trashed_at = coalesce(trashed_at, ?3)
WHERE id = ?1",
)?;
for (image, path) in moved {
n += stmt.execute(rusqlite::params![image.0 as i64, path, now])?;
}
}
let n = record_trashed_within(&tx, moved, now)?;
tx.commit()?;
Ok(n)
}
/// [`record_trashed`] inside a transaction the caller owns.
///
/// For a caller whose trash is one half of a larger write that must land
/// whole or not at all — consolidating duplicates (`crate::duplicates`)
/// merges a copy's judgements onto the survivor and trashes the copy in one
/// commit. `unchecked_transaction` cannot nest, so this is offered here
/// rather than wrapped from above.
pub fn record_trashed_within(
tx: &Connection,
moved: &[(ImageId, String)],
now: i64,
) -> Result<usize, CatalogError> {
let mut n = 0;
let mut stmt = tx.prepare(
"UPDATE images
SET trashed_from = CASE
WHEN trashed_at IS NULL THEN source_ref
ELSE trashed_from
END,
source_ref = ?2,
trashed_at = coalesce(trashed_at, ?3)
WHERE id = ?1",
)?;
for (image, path) in moved {
n += stmt.execute(rusqlite::params![image.0 as i64, path, now])?;
}
Ok(n)
}
/// Record that images have been moved back out of the trash.
///
/// Call after the move succeeds, for the same reason as [`record_trashed`].