Find catalog duplicates and fold a group onto one copy in one transaction
The library holds the same RAW in several folders: a dated folder, a bck/ beside it, a renamed Darktable export tree. dr_catalog::duplicates is the catalog half of consolidating them (#67). candidates() is one grouped query over root, camera, capture instant and size, joined back for the rows; count() is the same grouping under COUNT. On a copy of the reference catalog (23,582 images) both take 10-35 ms and find 1,836 groups holding 3,379 spare copies. survivor() prefers a copy outside a backup-looking folder, then one still named the way the camera named it, then the oldest, then the lowest id. consolidate() re-checks the plan against the catalog, merges the copies' judgements onto the survivor (highest rating, keywords unioned, collections unioned with the survivor keeping its place, a flag or label the copies agree on, faces via faces::carry_onto_copy) and records the copies as trashed, all in one transaction, so a failure part way leaves the group untouched. preview() runs the same code and rolls it back. Sameness probes are kept in dedup_probes, created on first use rather than by a migration: a schema bump would make older builds refuse this catalog's snapshot at sync. trash::record_trashed_within lets the trash write share the merge's transaction.
This commit is contained in:
@@ -129,28 +129,40 @@ pub fn record_trashed(
|
||||
return Ok(0);
|
||||
}
|
||||
let tx = conn.unchecked_transaction()?;
|
||||
let mut n = 0;
|
||||
|
||||
{
|
||||
let mut stmt = tx.prepare(
|
||||
"UPDATE images
|
||||
SET trashed_from = CASE
|
||||
WHEN trashed_at IS NULL THEN source_ref
|
||||
ELSE trashed_from
|
||||
END,
|
||||
source_ref = ?2,
|
||||
trashed_at = coalesce(trashed_at, ?3)
|
||||
WHERE id = ?1",
|
||||
)?;
|
||||
for (image, path) in moved {
|
||||
n += stmt.execute(rusqlite::params![image.0 as i64, path, now])?;
|
||||
}
|
||||
}
|
||||
|
||||
let n = record_trashed_within(&tx, moved, now)?;
|
||||
tx.commit()?;
|
||||
Ok(n)
|
||||
}
|
||||
|
||||
/// [`record_trashed`] inside a transaction the caller owns.
|
||||
///
|
||||
/// For a caller whose trash is one half of a larger write that must land
|
||||
/// whole or not at all — consolidating duplicates (`crate::duplicates`)
|
||||
/// merges a copy's judgements onto the survivor and trashes the copy in one
|
||||
/// commit. `unchecked_transaction` cannot nest, so this is offered here
|
||||
/// rather than wrapped from above.
|
||||
pub fn record_trashed_within(
|
||||
tx: &Connection,
|
||||
moved: &[(ImageId, String)],
|
||||
now: i64,
|
||||
) -> Result<usize, CatalogError> {
|
||||
let mut n = 0;
|
||||
let mut stmt = tx.prepare(
|
||||
"UPDATE images
|
||||
SET trashed_from = CASE
|
||||
WHEN trashed_at IS NULL THEN source_ref
|
||||
ELSE trashed_from
|
||||
END,
|
||||
source_ref = ?2,
|
||||
trashed_at = coalesce(trashed_at, ?3)
|
||||
WHERE id = ?1",
|
||||
)?;
|
||||
for (image, path) in moved {
|
||||
n += stmt.execute(rusqlite::params![image.0 as i64, path, now])?;
|
||||
}
|
||||
Ok(n)
|
||||
}
|
||||
|
||||
/// Record that images have been moved back out of the trash.
|
||||
///
|
||||
/// Call after the move succeeds, for the same reason as [`record_trashed`].
|
||||
|
||||
Reference in New Issue
Block a user