Prove duplicate originals the same and consolidate them on workers

dr_ui::duplicates is the half of #67 that touches files. The check reads
each copy's first and last megabyte by range through the backend and
hashes them (or compares stored content hashes where every copy has one),
keeps the probes in the catalog, and reads each copy's sidecar: a group
whose bytes differ, whose copies cannot be read, or whose develop edits
differ is left out and the review says why.

Consolidating a group carries the one edit onto the survivor's sidecar
where it has none, moves the other copies into the trash, and then
commits dr_catalog::duplicates::consolidate. A failure after the first
move puts the files and the sidecar back; a run that died between the
moves and the commit is finished by the next one, which finds each moved
file at its trash path.

Tested end to end on a folder library of real files: the copies land in
.darkroom-trash, the skipped groups are untouched, the edit reaches the
survivor, a catalog failure moves everything back, and restore returns
the copies byte for byte.
This commit is contained in:
2026-09-26 07:19:13 -04:00
parent 3c2eacbf3f
commit 8d4ecb75c1
4 changed files with 1220 additions and 0 deletions
+3
View File
@@ -37,6 +37,9 @@ dr-export.workspace = true
# runtime is the same tract the faces and masks already carry.
dr-pano = { workspace = true, features = ["xfeat", "embedded-model"] }
dr-ingest.workspace = true
# The sameness probe of a catalog duplicate (FR-CAT-11a): SHA-256 over the
# ends of each copy, the digest the import already uses for whole files.
sha2 = "0.10"
dr-film.workspace = true
# The lens profile database, here for the same reason dr-film is: dr-pipeline
# knows the maths of lens correction and deliberately has no dependency with