docs/ had 26 developer documents flat beside the manual, and the two audiences are very differently sized: most readers want the manual and the gesture reference, a few want the register, the designs and the measurements. The manual and gestures.md stay at the top; everything for someone changing the code moves to docs/dev/, and the two documents that name their own successors — the v0.1 milestone and the UI-refinement plan — go to docs/dev/archive/ rather than being deleted, since both are still cited. docs/README.md is the index, users first. Every reference follows: code comments, Cargo manifests, the workflows, the pre-commit hook, the bench and traceability tools (which locate the repo root by docs/dev/requirements.md now), packaging, the Docker READMEs, CLAUDE.md, CONTRIBUTING.md and the README. The matrix links one level deeper and is regenerated. Links out of the moved documents into the tree gain a level; a link checker over every Markdown file finds none broken.
1310 lines
52 KiB
Rust
1310 lines
52 KiB
Rust
//! TRACES: FR-CULL-5
|
||
//! Bursts: frames that are one moment, grouped so they can be judged as one.
|
||
//!
|
||
//! A burst is the commonest thing in a cull and the least interesting. Twelve
|
||
//! frames of the same gull, shot at 10 fps, occupy twelve cells of the grid,
|
||
//! are scrolled past twelve times, and end with the photographer keeping one.
|
||
//! Collapsing them to a single cell with a count on it is the whole feature:
|
||
//! the twelve are still there, still reachable, still individually ratable —
|
||
//! they simply stop costing twelve decisions where one was meant.
|
||
//!
|
||
//! # What this deliberately does not do
|
||
//!
|
||
//! **Nothing here ranks a frame.** FR-CULL-5 names the failure it is avoiding:
|
||
//! automated *selection* is distrusted because the documented way it fails is
|
||
//! rejecting the only frame of an important moment because somebody blinked.
|
||
//! So there is no sharpness score, no eye detector, no "best of burst". The
|
||
//! representative of a group is the **earliest frame**, which is a fact about
|
||
//! the clock and not a judgement about the photograph, and the user can name a
|
||
//! different one at any time ([`choose_representative`]).
|
||
//!
|
||
//! That is also why this sits beside [`crate::dedup`] rather than inside it.
|
||
//! Dedup answers "are these the same file?" with a whole-file digest, for
|
||
//! import collision; two bytes of difference make two different answers, which
|
||
//! is exactly right there and useless here. A burst is a set of frames that are
|
||
//! *deliberately* different — the wing is in another place in each one.
|
||
//!
|
||
//! # Two signals, and why neither alone will do
|
||
//!
|
||
//! **Time alone** groups a wedding ceremony into one burst. Frames arrive
|
||
//! seconds apart for an hour, and a photographer shooting steadily never
|
||
//! produces the gap that would end the run.
|
||
//!
|
||
//! **Similarity alone** groups the same subject photographed on two different
|
||
//! days — a studio setup, a copy stand, a hundred frames of the same document.
|
||
//! Those are not a burst, they are a project, and collapsing them hides work
|
||
//! the user did on purpose.
|
||
//!
|
||
//! Together they are specific: *adjacent in time* **and** *looks like the frame
|
||
//! before it*. Both bounds are in [`Rules`].
|
||
//!
|
||
//! # Chained, not compared to the first frame
|
||
//!
|
||
//! Each frame is tested against the one immediately before it, and a burst is a
|
||
//! run of frames that each joined the last. Comparing every frame to the *first*
|
||
//! would split a pan: by frame twenty a camera following a bird has nothing in
|
||
//! common with frame one, and yet no two adjacent frames in that sequence differ
|
||
//! by much. Chaining follows the subject; a fixed anchor loses it half way.
|
||
//!
|
||
//! The accepted consequence is that the first and last frames of a long burst
|
||
//! may be quite unlike each other. That is what a burst *is*, and the time bound
|
||
//! is what stops the chain running away: a sequence that has drifted a long way
|
||
//! has almost always paused somewhere, and the pause ends the run.
|
||
//!
|
||
//! # The similarity signal is a 64-bit difference hash
|
||
//!
|
||
//! [`Signature`] is a dHash taken on a 9×8 box-averaged reduction of the image:
|
||
//! 64 comparisons of a cell against its right-hand neighbour, one bit each. It
|
||
//! is scale-independent, survives JPEG artefacts and mild exposure changes,
|
||
//! costs nothing to store, and compares in one `xor` and a `popcount` — which
|
||
//! matters, because the grouping pass is a whole-library operation.
|
||
//!
|
||
//! What it is bad at is worth stating plainly: **a featureless frame hashes to
|
||
//! zero**, and so do all the other featureless frames. A lens cap, a black
|
||
//! frame, a white wall and an overexposed sky are mutually indistinguishable to
|
||
//! it. The capture-time bound is what keeps that from grouping every mistake in
|
||
//! the library into one enormous burst, and it is sufficient in practice — but
|
||
//! it is the reason this is grouping and not deduplication.
|
||
//!
|
||
//! # Where the pixels come from, and why the hash is stored
|
||
//!
|
||
//! This module never decodes an image. It takes a luma or RGBA buffer somebody
|
||
//! else already had in hand ([`signature_of_luma`], [`signature_of_rgba`]) and
|
||
//! `images.perceptual_hash` keeps the answer, the same bargain
|
||
//! [`crate::dedup::set_content_hash`] makes: whoever is already holding the
|
||
//! pixels pays nothing, and everybody afterwards pays nothing at all. In
|
||
//! practice the caller is the thumbnail store — a 256px thumbnail is far more
|
||
//! resolution than a 9×8 reduction needs — so a library that has been browsed
|
||
//! has already paid for its signatures.
|
||
//!
|
||
//! # A pass, not a job
|
||
//!
|
||
//! Grouping has no natural `subject_id`: it is a property of a *run* of frames,
|
||
//! so a per-image job would rebuild the world once per photograph. It is
|
||
//! therefore a debounced library-level pass, for exactly the reasons
|
||
//! docs/dev/catalog.md §10.2 gives for face clustering, and [`regroup`] is the whole
|
||
//! of it — one ordered walk, no per-pair comparison beyond adjacent frames.
|
||
//!
|
||
//! # Grouping is not hiding
|
||
//!
|
||
//! A burst this pass has never seen before is recorded **open**: the grid keeps
|
||
//! every row it had, and all that appears is a mark saying how many frames each
|
||
//! run holds. Only the user folds one up. That is a deliberate refusal of the
|
||
//! obvious default — collapsing on discovery would be tidier and would also
|
||
//! mean a background pass removing photographs from under somebody part way
|
||
//! through a cull, which is the same class of surprise FR-CULL-5 is written to
|
||
//! avoid. A burst that is already known keeps whatever state it is in, so a
|
||
//! pass after the next import does not spring open a morning's work.
|
||
//!
|
||
//! # What is stored, and what a merge does with it
|
||
//!
|
||
//! `burst_members` is derived: deleting it costs one pass and loses nothing.
|
||
//! `burst_pick` is not — it is the user saying which frame stands for the
|
||
//! moment, and it lives in its own table precisely so [`regroup`] can rewrite
|
||
//! the grouping without erasing the judgement. That is the same argument
|
||
//! `people.ignored` makes one subsystem over: nothing short of remembering a
|
||
//! decision survives re-clustering.
|
||
//!
|
||
//! Both tables travel in the uploaded catalog snapshot, and nothing on the far
|
||
//! side reads them — [`crate::sync`] merges collections and keywords only — so
|
||
//! a second device rebuilds its own grouping from its own signatures. Bursts
|
||
//! are not yet cross-device state, and pretending otherwise would mean syncing
|
||
//! `burst_pick` as user data, which is a merge question this does not answer.
|
||
|
||
use std::collections::{HashMap, HashSet};
|
||
|
||
use dr_types::ImageId;
|
||
use rusqlite::Connection;
|
||
|
||
use crate::CatalogError;
|
||
|
||
/// A 64-bit perceptual signature of one image.
|
||
///
|
||
/// Comparable only to other signatures from this build: the reduction grid and
|
||
/// the bit order are part of the definition, and changing either silently
|
||
/// changes what "similar" means. If that ever happens the column has to be
|
||
/// cleared, the way a face `model_id` change forces a re-index.
|
||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord)]
|
||
pub struct Signature(pub u64);
|
||
|
||
impl Signature {
|
||
/// Differing bits, 0..=64. Small is similar.
|
||
pub fn distance(self, other: Signature) -> u32 {
|
||
(self.0 ^ other.0).count_ones()
|
||
}
|
||
|
||
/// The stored form. SQLite integers are signed, so the bits are reinterpreted
|
||
/// rather than converted — `as` in both directions is exact and lossless.
|
||
pub fn to_stored(self) -> i64 {
|
||
self.0 as i64
|
||
}
|
||
|
||
/// The inverse of [`Self::to_stored`].
|
||
pub fn from_stored(v: i64) -> Self {
|
||
Signature(v as u64)
|
||
}
|
||
}
|
||
|
||
/// Columns in the reduction grid. One more than the number of comparisons per
|
||
/// row, because each bit compares a cell against its right-hand neighbour.
|
||
const GRID_W: usize = 9;
|
||
/// Rows in the reduction grid. `(GRID_W - 1) * GRID_H` is 64 — the width of the
|
||
/// signature, and the reason these two numbers are what they are.
|
||
const GRID_H: usize = 8;
|
||
|
||
/// Reduce a greyscale buffer to a [`Signature`].
|
||
///
|
||
/// `luma` is row-major, one byte a pixel, `width * height` of them. Returns
|
||
/// `None` for an empty or short buffer rather than panicking: the caller is
|
||
/// usually handing over the result of a decode, and a truncated JPEG is a
|
||
/// normal thing to find in a library rather than a programming error.
|
||
///
|
||
/// # Why box-averaged rather than sampled
|
||
///
|
||
/// Point-sampling a 9×8 grid out of a 256px thumbnail reads 72 pixels of
|
||
/// several hundred thousand, and two frames of a burst that differ by one pixel
|
||
/// of camera shake can sample entirely different detail. Averaging the whole
|
||
/// cell is what makes the signature stable under the small movements a burst is
|
||
/// made of — which is the property the whole feature rests on.
|
||
pub fn signature_of_luma(luma: &[u8], width: u32, height: u32) -> Option<Signature> {
|
||
let (w, h) = (width as usize, height as usize);
|
||
if w == 0 || h == 0 || luma.len() < w * h {
|
||
return None;
|
||
}
|
||
|
||
let mut cells = [0f32; GRID_W * GRID_H];
|
||
for gy in 0..GRID_H {
|
||
// Cell bounds by proportion, so every source pixel lands in exactly one
|
||
// cell whatever the aspect ratio. The `max` keeps a cell non-empty when
|
||
// the source is narrower or shorter than the grid — a 4px-wide preview
|
||
// is degenerate but must not divide by zero.
|
||
let y0 = gy * h / GRID_H;
|
||
let y1 = (((gy + 1) * h) / GRID_H).max(y0 + 1).min(h);
|
||
for gx in 0..GRID_W {
|
||
let x0 = gx * w / GRID_W;
|
||
let x1 = (((gx + 1) * w) / GRID_W).max(x0 + 1).min(w);
|
||
|
||
let mut sum = 0u32;
|
||
let mut n = 0u32;
|
||
for y in y0..y1 {
|
||
let row = &luma[y * w..y * w + w];
|
||
for px in &row[x0..x1] {
|
||
sum += *px as u32;
|
||
n += 1;
|
||
}
|
||
}
|
||
cells[gy * GRID_W + gx] = sum as f32 / n as f32;
|
||
}
|
||
}
|
||
|
||
// Each bit: is this cell brighter than the one to its right? A *difference*
|
||
// rather than a level, which is what makes the signature indifferent to the
|
||
// exposure drifting a third of a stop through a burst.
|
||
let mut bits = 0u64;
|
||
for gy in 0..GRID_H {
|
||
for gx in 0..GRID_W - 1 {
|
||
bits <<= 1;
|
||
if cells[gy * GRID_W + gx] > cells[gy * GRID_W + gx + 1] {
|
||
bits |= 1;
|
||
}
|
||
}
|
||
}
|
||
Some(Signature(bits))
|
||
}
|
||
|
||
/// [`signature_of_luma`] for the packed RGBA a decoded thumbnail arrives as.
|
||
///
|
||
/// Alpha is ignored: a thumbnail is opaque, and a signature that changed with
|
||
/// it would make two renderings of one frame look like two photographs.
|
||
pub fn signature_of_rgba(rgba: &[u8], width: u32, height: u32) -> Option<Signature> {
|
||
let (w, h) = (width as usize, height as usize);
|
||
if w == 0 || h == 0 || rgba.len() < w * h * 4 {
|
||
return None;
|
||
}
|
||
// Rec. 601 luma in fixed point. The exact weights matter less than their
|
||
// being the same weights every time — see [`Signature`].
|
||
let luma: Vec<u8> = rgba
|
||
.chunks_exact(4)
|
||
.take(w * h)
|
||
.map(|p| ((77 * p[0] as u32 + 150 * p[1] as u32 + 29 * p[2] as u32) >> 8) as u8)
|
||
.collect();
|
||
signature_of_luma(&luma, width, height)
|
||
}
|
||
|
||
/// The two bounds that decide what counts as one burst.
|
||
///
|
||
/// Held together in a struct rather than passed as two numbers so a caller
|
||
/// cannot supply one and default the other, and so the pair can be logged as
|
||
/// what a given grouping was produced under.
|
||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||
pub struct Rules {
|
||
/// Longest gap, in seconds, between two frames of one burst.
|
||
pub max_gap: i64,
|
||
/// Largest [`Signature::distance`] between two frames of one burst.
|
||
pub max_distance: u32,
|
||
}
|
||
|
||
impl Default for Rules {
|
||
/// # Why two seconds, when a burst fires ten frames in one
|
||
///
|
||
/// Because `images.captured_at` is **whole seconds**. EXIF `DateTimeOriginal`
|
||
/// has no sub-second field — `SubSecTimeOriginal` is a separate, optional tag
|
||
/// that plenty of bodies omit — so a 10 fps burst arrives in the catalog as
|
||
/// ten frames sharing one timestamp, and any threshold below a second is a
|
||
/// threshold on information that is not there.
|
||
///
|
||
/// Two seconds is therefore the smallest bound that can distinguish
|
||
/// anything: it holds a burst together across the second boundary it will
|
||
/// certainly straddle, and it is short enough that a photographer working at
|
||
/// a considered pace — a frame every three or four seconds — is never
|
||
/// grouped. Where the pace really is faster than that, the similarity bound
|
||
/// is what separates the frames.
|
||
///
|
||
/// # Why eight bits of sixty-four
|
||
///
|
||
/// A difference hash of two frames of one burst typically differs by nought
|
||
/// to four bits; two unrelated photographs differ by twenty to thirty-two,
|
||
/// with thirty-two being the expected distance between two *random*
|
||
/// signatures. Eight sits in the empty ground between those, near enough to
|
||
/// the burst end of it that a subject change inside one second — the case
|
||
/// FR-CULL-5's failure mode is really about — does not merge.
|
||
///
|
||
/// Erring low is the right direction: a burst left ungrouped costs the user
|
||
/// a scroll, and two moments wrongly merged hide a photograph behind a cell
|
||
/// that does not look like it.
|
||
fn default() -> Self {
|
||
Rules {
|
||
max_gap: 2,
|
||
max_distance: 8,
|
||
}
|
||
}
|
||
}
|
||
|
||
/// One frame as the grouping pass sees it.
|
||
///
|
||
/// Deliberately not a catalog row: the whole decision is made from these four
|
||
/// fields, which is what lets [`group`] be tested against constructed frames
|
||
/// with no database at all.
|
||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||
pub struct Frame {
|
||
pub id: ImageId,
|
||
/// UTC seconds. A frame without one cannot be placed in a run and is never
|
||
/// read by the pass.
|
||
pub captured_at: i64,
|
||
/// The joined make-and-model string, as the scan stores it.
|
||
pub camera: Option<String>,
|
||
/// `None` until something has hashed this image's pixels.
|
||
pub signature: Option<Signature>,
|
||
}
|
||
|
||
/// Whether `b` continues the burst that `a` is part of.
|
||
///
|
||
/// `b` is assumed not to precede `a`.
|
||
fn continues(a: &Frame, b: &Frame, rules: Rules) -> bool {
|
||
if b.captured_at - a.captured_at > rules.max_gap {
|
||
return false;
|
||
}
|
||
|
||
// Two bodies firing at the same instant are two photographs of one moment,
|
||
// not one burst — a second shooter at a wedding, or a camera on a tripod
|
||
// triggered alongside the one in your hands. Where either camera is unknown
|
||
// the frames are not separated on that account: an unread EXIF header is
|
||
// absence of evidence, and the similarity bound still has to be satisfied.
|
||
if let (Some(x), Some(y)) = (&a.camera, &b.camera) {
|
||
if x != y {
|
||
return false;
|
||
}
|
||
}
|
||
|
||
// A missing signature never groups. It would be easy to fall back to time
|
||
// alone here and it would be wrong: an unhashed library would collapse
|
||
// every steadily-shot sequence in it, and the user would have no way to see
|
||
// that the reason was a missing hash rather than a resemblance.
|
||
match (a.signature, b.signature) {
|
||
(Some(p), Some(q)) => p.distance(q) <= rules.max_distance,
|
||
_ => false,
|
||
}
|
||
}
|
||
|
||
/// Group frames into bursts. The whole of the algorithm.
|
||
///
|
||
/// Returns only groups of two or more, each ordered by capture time, in the
|
||
/// order the bursts themselves begin. A lone frame is not a burst and gets no
|
||
/// row anywhere: "is this image in a burst" is then answerable as "does it have
|
||
/// a `burst_members` row", and the grid pays nothing for the overwhelming
|
||
/// majority of a library that is not bursts.
|
||
///
|
||
/// Sorts its own input. The caller usually reads frames in capture order
|
||
/// anyway, but the run structure is only meaningful in that order and a
|
||
/// mis-ordered caller would get silently wrong groups rather than an error.
|
||
pub fn group(frames: &[Frame], rules: Rules) -> Vec<Vec<ImageId>> {
|
||
let mut ordered: Vec<&Frame> = frames.iter().collect();
|
||
// Ties on the second broken by id, which is the order the frames were
|
||
// catalogued in and therefore the order the camera wrote them. It is the
|
||
// same tiebreak the grid uses, so a burst is a contiguous run on screen.
|
||
ordered.sort_by_key(|f| (f.captured_at, f.id.0));
|
||
|
||
let mut out: Vec<Vec<ImageId>> = Vec::new();
|
||
let mut run: Vec<ImageId> = Vec::new();
|
||
|
||
for pair in ordered.windows(2) {
|
||
let (a, b) = (pair[0], pair[1]);
|
||
if continues(a, b, rules) {
|
||
if run.is_empty() {
|
||
run.push(a.id);
|
||
}
|
||
run.push(b.id);
|
||
} else if !run.is_empty() {
|
||
out.push(std::mem::take(&mut run));
|
||
}
|
||
}
|
||
if !run.is_empty() {
|
||
out.push(run);
|
||
}
|
||
out
|
||
}
|
||
|
||
/// What one [`regroup`] did, for the log and for a progress line.
|
||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
|
||
pub struct Report {
|
||
/// Groups of two or more frames.
|
||
pub bursts: usize,
|
||
/// Frames inside them. Never the size of the library.
|
||
pub frames: usize,
|
||
/// Frames in the largest single burst.
|
||
pub largest: usize,
|
||
}
|
||
|
||
/// Rebuild the whole library's grouping.
|
||
///
|
||
/// One ordered read of the dated, visible images, one pass over it, one
|
||
/// transaction. The cost is linear in library size and the comparison count is
|
||
/// one per *adjacent pair* — there is no all-pairs step here and there must
|
||
/// never be one, because that is what turns a grouping pass into a job nobody
|
||
/// can afford to run.
|
||
///
|
||
/// Existing membership is replaced rather than amended. Amending would need to
|
||
/// know which frames had changed since the last pass, and the answer is
|
||
/// "possibly any of them, because a signature arrives long after the row does";
|
||
/// a full rebuild is a few milliseconds and cannot drift.
|
||
pub fn regroup(conn: &Connection, rules: Rules) -> Result<Report, CatalogError> {
|
||
let frames = read_frames(conn)?;
|
||
let groups = group(&frames, rules);
|
||
let picked = user_picks(conn)?;
|
||
let known = known_bursts(conn)?;
|
||
|
||
let tx = conn.unchecked_transaction()?;
|
||
tx.execute("DELETE FROM burst_members", [])?;
|
||
{
|
||
let mut insert = tx.prepare(
|
||
"INSERT INTO burst_members(image_id, burst_id, representative)
|
||
VALUES (?1, ?2, ?3)",
|
||
)?;
|
||
for members in &groups {
|
||
// The earliest frame's own id names the burst. It is stable across
|
||
// a regroup that leaves the burst alone, which is what lets the UI
|
||
// remember that this group is open; it is not a durable identity,
|
||
// and a burst that gains an earlier frame is a new group as far as
|
||
// anything keyed on this is concerned.
|
||
let burst_id = members[0].0 as i64;
|
||
let representative = members
|
||
.iter()
|
||
.find(|id| picked.contains(*id))
|
||
.copied()
|
||
.unwrap_or(members[0]);
|
||
for id in members {
|
||
insert.execute(rusqlite::params![
|
||
id.0 as i64,
|
||
burst_id,
|
||
(*id == representative) as i64
|
||
])?;
|
||
}
|
||
|
||
// **A burst the user has never seen arrives open.** Nothing is
|
||
// hidden by this pass running: the grid holds exactly the
|
||
// photographs it held before, and the only new thing is a mark on
|
||
// each burst saying how many frames it is part of. Folding one up
|
||
// is then something the user did, which is the difference between
|
||
// a tool that groups and a tool that decides — and it is the only
|
||
// arrangement in which a background pass cannot make a row
|
||
// disappear from under somebody mid-cull.
|
||
//
|
||
// A burst that is already known keeps whatever state it is in, so
|
||
// the pass that runs after the next import does not spring open
|
||
// everything the user spent the morning folding away.
|
||
if !known.contains(&burst_id) {
|
||
tx.execute(
|
||
"INSERT OR IGNORE INTO burst_expanded(burst_id) VALUES (?1)",
|
||
[burst_id],
|
||
)?;
|
||
}
|
||
}
|
||
}
|
||
// A group that no longer exists must not leave an expansion behind: the id
|
||
// is an image id, and the next pass could hand it to a different burst.
|
||
tx.execute(
|
||
"DELETE FROM burst_expanded
|
||
WHERE burst_id NOT IN (SELECT burst_id FROM burst_members)",
|
||
[],
|
||
)?;
|
||
tx.commit()?;
|
||
|
||
Ok(Report {
|
||
bursts: groups.len(),
|
||
frames: groups.iter().map(|g| g.len()).sum(),
|
||
largest: groups.iter().map(|g| g.len()).max().unwrap_or(0),
|
||
})
|
||
}
|
||
|
||
/// The frames a grouping pass considers, in capture order.
|
||
///
|
||
/// The same two exclusions the grid makes, for the same two reasons: a shadowed
|
||
/// JPEG is its RAW's frame rather than a second photograph, and a trashed image
|
||
/// is not in the library. Undated frames are excluded because a burst is a
|
||
/// statement about time and there is nothing to say about a frame with none.
|
||
fn read_frames(conn: &Connection) -> Result<Vec<Frame>, CatalogError> {
|
||
let mut stmt = conn.prepare(
|
||
"SELECT id, captured_at, camera, perceptual_hash
|
||
FROM images
|
||
WHERE captured_at IS NOT NULL
|
||
AND shadowed_by IS NULL
|
||
AND trashed_at IS NULL
|
||
ORDER BY captured_at ASC, id ASC",
|
||
)?;
|
||
let rows = stmt.query_map([], |r| {
|
||
Ok(Frame {
|
||
id: ImageId(r.get::<_, i64>(0)? as u64),
|
||
captured_at: r.get(1)?,
|
||
camera: r.get(2)?,
|
||
signature: r.get::<_, Option<i64>>(3)?.map(Signature::from_stored),
|
||
})
|
||
})?;
|
||
Ok(rows.collect::<Result<Vec<_>, _>>()?)
|
||
}
|
||
|
||
/// The bursts the previous pass left behind, so this one can tell a group the
|
||
/// user has already met from a group that is new.
|
||
fn known_bursts(conn: &Connection) -> Result<HashSet<i64>, CatalogError> {
|
||
let mut stmt = conn.prepare("SELECT DISTINCT burst_id FROM burst_members")?;
|
||
let rows = stmt.query_map([], |r| r.get::<_, i64>(0))?;
|
||
Ok(rows.collect::<Result<HashSet<_>, _>>()?)
|
||
}
|
||
|
||
fn user_picks(conn: &Connection) -> Result<HashSet<ImageId>, CatalogError> {
|
||
let mut stmt = conn.prepare("SELECT image_id FROM burst_pick")?;
|
||
let rows = stmt.query_map([], |r| Ok(ImageId(r.get::<_, i64>(0)? as u64)))?;
|
||
Ok(rows.collect::<Result<HashSet<_>, _>>()?)
|
||
}
|
||
|
||
/// Record the signature of an image somebody has just decoded.
|
||
///
|
||
/// Returns rows updated: zero means the image is no longer in the catalog,
|
||
/// which is a normal race with a rescan rather than an error — the same answer
|
||
/// [`crate::dedup::set_content_hash`] gives.
|
||
pub fn set_signature(
|
||
conn: &Connection,
|
||
image: ImageId,
|
||
signature: Signature,
|
||
) -> Result<usize, CatalogError> {
|
||
Ok(conn.execute(
|
||
"UPDATE images SET perceptual_hash = ?2 WHERE id = ?1",
|
||
rusqlite::params![image.0 as i64, signature.to_stored()],
|
||
)?)
|
||
}
|
||
|
||
/// What [`set_signature`] stored, if anything.
|
||
pub fn signature(conn: &Connection, image: ImageId) -> Result<Option<Signature>, CatalogError> {
|
||
let stored: Option<i64> = conn.query_row(
|
||
"SELECT perceptual_hash FROM images WHERE id = ?1",
|
||
[image.0 as i64],
|
||
|r| r.get(0),
|
||
)?;
|
||
Ok(stored.map(Signature::from_stored))
|
||
}
|
||
|
||
/// Visible, dated images that nothing has hashed yet.
|
||
///
|
||
/// What the pass that fills the column iterates. Dated, because an undated
|
||
/// frame can never join a burst however well it hashes, and hashing it would be
|
||
/// work with no possible consequence.
|
||
pub fn images_without_signature(conn: &Connection) -> Result<Vec<ImageId>, CatalogError> {
|
||
let mut stmt = conn.prepare(
|
||
"SELECT id FROM images
|
||
WHERE perceptual_hash IS NULL
|
||
AND captured_at IS NOT NULL
|
||
AND shadowed_by IS NULL
|
||
AND trashed_at IS NULL
|
||
ORDER BY captured_at ASC, id ASC",
|
||
)?;
|
||
let rows = stmt.query_map([], |r| Ok(ImageId(r.get::<_, i64>(0)? as u64)))?;
|
||
Ok(rows.collect::<Result<Vec<_>, _>>()?)
|
||
}
|
||
|
||
/// What the grid needs to know about one cell's burst.
|
||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||
pub struct Membership {
|
||
/// The group's id, which is also the image id of its earliest frame.
|
||
pub burst_id: ImageId,
|
||
/// Frames in the group, this one included. Always two or more.
|
||
pub size: u32,
|
||
/// Whether this is the frame the group collapses to.
|
||
pub representative: bool,
|
||
/// Whether the group is currently open in the grid.
|
||
pub expanded: bool,
|
||
}
|
||
|
||
/// Burst membership for one window of the grid, in a single query.
|
||
///
|
||
/// One statement for the whole window rather than one per cell, which is the
|
||
/// same discipline the collection badges and the rating counts follow: a grid
|
||
/// that asks the catalog a question per cell is a grid that stutters while a
|
||
/// finger is on it.
|
||
pub fn memberships(
|
||
conn: &Connection,
|
||
images: &[ImageId],
|
||
) -> Result<HashMap<ImageId, Membership>, CatalogError> {
|
||
if images.is_empty() {
|
||
return Ok(HashMap::new());
|
||
}
|
||
// Placeholders are generated from the *count* of ids, never from anything
|
||
// the user typed.
|
||
let placeholders = std::iter::repeat_n("?", images.len())
|
||
.collect::<Vec<_>>()
|
||
.join(",");
|
||
let sql = format!(
|
||
"SELECT bm.image_id,
|
||
bm.burst_id,
|
||
bm.representative,
|
||
(SELECT count(*) FROM burst_members m WHERE m.burst_id = bm.burst_id),
|
||
EXISTS (SELECT 1 FROM burst_expanded be WHERE be.burst_id = bm.burst_id)
|
||
FROM burst_members bm
|
||
WHERE bm.image_id IN ({placeholders})"
|
||
);
|
||
let params: Vec<rusqlite::types::Value> = images
|
||
.iter()
|
||
.map(|i| rusqlite::types::Value::Integer(i.0 as i64))
|
||
.collect();
|
||
|
||
let mut stmt = conn.prepare(&sql)?;
|
||
let rows = stmt.query_map(rusqlite::params_from_iter(params.iter()), |r| {
|
||
Ok((
|
||
ImageId(r.get::<_, i64>(0)? as u64),
|
||
Membership {
|
||
burst_id: ImageId(r.get::<_, i64>(1)? as u64),
|
||
size: r.get::<_, i64>(3)? as u32,
|
||
representative: r.get::<_, i64>(2)? != 0,
|
||
expanded: r.get::<_, i64>(4)? != 0,
|
||
},
|
||
))
|
||
})?;
|
||
Ok(rows.collect::<Result<HashMap<_, _>, _>>()?)
|
||
}
|
||
|
||
/// The frames of one burst, in the order the grid lists them.
|
||
pub fn members(conn: &Connection, burst: ImageId) -> Result<Vec<ImageId>, CatalogError> {
|
||
let mut stmt = conn.prepare(
|
||
"SELECT bm.image_id
|
||
FROM burst_members bm
|
||
JOIN images i ON i.id = bm.image_id
|
||
WHERE bm.burst_id = ?1
|
||
ORDER BY i.captured_at ASC, i.id ASC",
|
||
)?;
|
||
let rows = stmt.query_map([burst.0 as i64], |r| {
|
||
Ok(ImageId(r.get::<_, i64>(0)? as u64))
|
||
})?;
|
||
Ok(rows.collect::<Result<Vec<_>, _>>()?)
|
||
}
|
||
|
||
/// Open or close one burst in the grid.
|
||
///
|
||
/// Kept in the catalog rather than in the interface's own memory for one
|
||
/// reason: the grid is a *window* over an ordered query, and what is collapsed
|
||
/// has to be decided by that query or the window's row count stops matching the
|
||
/// scrollbar. Once the state has to be visible to SQL, the catalog is where it
|
||
/// lives — and a culling session that survives closing the app is the shape
|
||
/// FR-CULL-4 already asks for elsewhere.
|
||
pub fn set_expanded(conn: &Connection, burst: ImageId, expanded: bool) -> Result<(), CatalogError> {
|
||
if expanded {
|
||
conn.execute(
|
||
// `OR IGNORE` rather than an upsert: the toggle is wired to a
|
||
// callback that can fire twice, and re-opening an open burst must
|
||
// be a no-op rather than an error.
|
||
"INSERT OR IGNORE INTO burst_expanded(burst_id) VALUES (?1)",
|
||
[burst.0 as i64],
|
||
)?;
|
||
} else {
|
||
conn.execute(
|
||
"DELETE FROM burst_expanded WHERE burst_id = ?1",
|
||
[burst.0 as i64],
|
||
)?;
|
||
}
|
||
Ok(())
|
||
}
|
||
|
||
/// Whether this burst is currently open.
|
||
pub fn is_expanded(conn: &Connection, burst: ImageId) -> Result<bool, CatalogError> {
|
||
let n: i64 = conn.query_row(
|
||
"SELECT count(*) FROM burst_expanded WHERE burst_id = ?1",
|
||
[burst.0 as i64],
|
||
|r| r.get(0),
|
||
)?;
|
||
Ok(n > 0)
|
||
}
|
||
|
||
/// Close every open burst. Returns how many were open.
|
||
pub fn collapse_all(conn: &Connection) -> Result<usize, CatalogError> {
|
||
Ok(conn.execute("DELETE FROM burst_expanded", [])?)
|
||
}
|
||
|
||
/// The user names the frame their burst collapses to.
|
||
///
|
||
/// The one *choice* in this module, and the reason the rest of it makes none.
|
||
/// Recorded against the image rather than the group so that it survives the
|
||
/// next [`regroup`]: a group is rebuilt from scratch every pass, and a
|
||
/// representative stored on it would be forgotten every time a frame arrived.
|
||
///
|
||
/// Any previous pick within the same burst is dropped — a burst collapses to
|
||
/// one frame, and two picks would make the choice depend on row order.
|
||
pub fn choose_representative(conn: &Connection, image: ImageId) -> Result<(), CatalogError> {
|
||
let id = image.0 as i64;
|
||
let tx = conn.unchecked_transaction()?;
|
||
|
||
// Only meaningful for an image that is actually in a burst; anything else
|
||
// would leave a pick that no group can ever honour.
|
||
let burst: Option<i64> = tx
|
||
.query_row(
|
||
"SELECT burst_id FROM burst_members WHERE image_id = ?1",
|
||
[id],
|
||
|r| r.get(0),
|
||
)
|
||
.ok();
|
||
let Some(burst) = burst else {
|
||
return Ok(());
|
||
};
|
||
|
||
tx.execute(
|
||
"DELETE FROM burst_pick
|
||
WHERE image_id IN (SELECT image_id FROM burst_members WHERE burst_id = ?1)",
|
||
[burst],
|
||
)?;
|
||
tx.execute("INSERT INTO burst_pick(image_id) VALUES (?1)", [id])?;
|
||
// The grouping already exists, so the flag it carries is corrected now
|
||
// rather than at the next pass — the user expects the cell to change under
|
||
// the pointer, not after a background sweep.
|
||
tx.execute(
|
||
"UPDATE burst_members SET representative = (image_id = ?2) WHERE burst_id = ?1",
|
||
[burst, id],
|
||
)?;
|
||
tx.commit()?;
|
||
Ok(())
|
||
}
|
||
|
||
/// SQL for "this row is not a frame a collapsed burst is standing in for".
|
||
///
|
||
/// Handed out as a predicate rather than as a list of ids because the grid is a
|
||
/// window: the rows a collapsed burst hides are mostly not loaded, so the
|
||
/// question can only be answered where the ordering and the `LIMIT` are — in
|
||
/// the query itself. `image` names the table or alias the image row is in,
|
||
/// because the grid aliases `images` as `i` and other callers do not.
|
||
///
|
||
/// Both subqueries are probes on a primary key, so this costs one index lookup
|
||
/// per row the query walks and nothing at all for a library with no bursts in
|
||
/// it, where `burst_members` is empty.
|
||
///
|
||
/// Never interpolate anything user-supplied as `image`.
|
||
pub fn not_collapsed_away(image: &str) -> String {
|
||
format!(
|
||
"NOT EXISTS (SELECT 1 FROM burst_members bm
|
||
WHERE bm.image_id = {image}.id
|
||
AND bm.representative = 0
|
||
AND NOT EXISTS (SELECT 1 FROM burst_expanded be
|
||
WHERE be.burst_id = bm.burst_id))"
|
||
)
|
||
}
|
||
|
||
#[cfg(test)]
|
||
mod tests {
|
||
use super::*;
|
||
use crate::Catalog;
|
||
|
||
/// A frame with everything the grouping looks at.
|
||
fn frame(id: u64, at: i64, sig: u64) -> Frame {
|
||
Frame {
|
||
id: ImageId(id),
|
||
captured_at: at,
|
||
camera: Some("Canon EOS R5".into()),
|
||
signature: Some(Signature(sig)),
|
||
}
|
||
}
|
||
|
||
// ── the signature ─────────────────────────────────────────────────────
|
||
|
||
/// A blocky pseudo-random scene, deterministic and free of any dependency.
|
||
///
|
||
/// Blocks rather than per-pixel noise, because per-pixel noise averages to a
|
||
/// flat grey in the 9×8 reduction and every such image would hash alike —
|
||
/// which would make the tests below pass for the wrong reason.
|
||
fn scene(w: usize, h: usize, seed: u64, shift: usize, brighter: i32) -> Vec<u8> {
|
||
let block = 8;
|
||
let mut out = vec![0u8; w * h];
|
||
for y in 0..h {
|
||
for x in 0..w {
|
||
let bx = (x + shift) / block;
|
||
let by = y / block;
|
||
// A small LCG, evaluated per block: reproducible, and nothing
|
||
// in the workspace has to provide it.
|
||
let mut s = seed
|
||
.wrapping_add(bx as u64)
|
||
.wrapping_mul(6_364_136_223_846_793_005)
|
||
.wrapping_add((by as u64).wrapping_mul(1_442_695_040_888_963_407));
|
||
s ^= s >> 33;
|
||
let v = (s % 256) as i32 + brighter;
|
||
out[y * w + x] = v.clamp(0, 255) as u8;
|
||
}
|
||
}
|
||
out
|
||
}
|
||
|
||
#[test]
|
||
fn two_frames_of_one_burst_hash_almost_alike() {
|
||
// The property the whole feature rests on: the same scene, moved by a
|
||
// pixel and a third of a stop brighter, is the same signature or very
|
||
// nearly.
|
||
let a = signature_of_luma(&scene(64, 64, 7, 0, 0), 64, 64).unwrap();
|
||
let b = signature_of_luma(&scene(64, 64, 7, 1, 6), 64, 64).unwrap();
|
||
assert!(
|
||
a.distance(b) <= Rules::default().max_distance,
|
||
"a burst pair differed by {} bits, which the default rules would \
|
||
not group",
|
||
a.distance(b)
|
||
);
|
||
}
|
||
|
||
#[test]
|
||
fn two_different_subjects_do_not_hash_alike() {
|
||
let a = signature_of_luma(&scene(64, 64, 7, 0, 0), 64, 64).unwrap();
|
||
let b = signature_of_luma(&scene(64, 64, 99, 0, 0), 64, 64).unwrap();
|
||
assert!(
|
||
a.distance(b) > Rules::default().max_distance,
|
||
"two unrelated scenes differed by only {} bits",
|
||
a.distance(b)
|
||
);
|
||
}
|
||
|
||
#[test]
|
||
fn the_signature_does_not_change_with_scale() {
|
||
// Detection runs against whichever preview tier is in hand, and the
|
||
// cache is entitled to regenerate one at another size. A signature that
|
||
// moved with the resolution would silently stop matching its own
|
||
// library.
|
||
let big = scene(128, 128, 21, 0, 0);
|
||
// Nearest-neighbour halving, which is the harshest thing a real
|
||
// downscale could do to it.
|
||
let mut small = vec![0u8; 64 * 64];
|
||
for y in 0..64 {
|
||
for x in 0..64 {
|
||
small[y * 64 + x] = big[(y * 2) * 128 + x * 2];
|
||
}
|
||
}
|
||
let a = signature_of_luma(&big, 128, 128).unwrap();
|
||
let b = signature_of_luma(&small, 64, 64).unwrap();
|
||
assert!(
|
||
a.distance(b) <= Rules::default().max_distance,
|
||
"halving the image moved {} bits, enough to stop it matching itself",
|
||
a.distance(b)
|
||
);
|
||
}
|
||
|
||
#[test]
|
||
fn a_short_or_empty_buffer_is_not_a_signature() {
|
||
assert!(signature_of_luma(&[], 0, 0).is_none());
|
||
assert!(signature_of_luma(&[1, 2, 3], 64, 64).is_none());
|
||
assert!(signature_of_rgba(&[1, 2, 3, 255], 64, 64).is_none());
|
||
}
|
||
|
||
#[test]
|
||
fn rgba_and_luma_agree() {
|
||
let luma = scene(64, 64, 3, 0, 0);
|
||
let rgba: Vec<u8> = luma.iter().flat_map(|v| [*v, *v, *v, 255]).collect();
|
||
// Grey is grey: the Rec. 601 weights sum to 256/256, so a neutral pixel
|
||
// survives the conversion within a rounding step.
|
||
let a = signature_of_luma(&luma, 64, 64).unwrap();
|
||
let b = signature_of_rgba(&rgba, 64, 64).unwrap();
|
||
assert!(a.distance(b) <= 2, "{} bits apart", a.distance(b));
|
||
}
|
||
|
||
#[test]
|
||
fn a_signature_survives_the_trip_through_sqlite() {
|
||
// The top bit is the one at risk: SQLite integers are signed.
|
||
let s = Signature(u64::MAX);
|
||
assert_eq!(Signature::from_stored(s.to_stored()), s);
|
||
assert_eq!(
|
||
Signature::from_stored(Signature(0).to_stored()),
|
||
Signature(0)
|
||
);
|
||
}
|
||
|
||
// ── the grouping ──────────────────────────────────────────────────────
|
||
|
||
#[test]
|
||
fn frames_a_second_apart_and_alike_are_one_burst() {
|
||
let g = group(
|
||
&[
|
||
frame(1, 1000, 0xFF00),
|
||
frame(2, 1001, 0xFF00),
|
||
frame(3, 1001, 0xFF01),
|
||
],
|
||
Rules::default(),
|
||
);
|
||
assert_eq!(g, vec![vec![ImageId(1), ImageId(2), ImageId(3)]]);
|
||
}
|
||
|
||
#[test]
|
||
fn a_pause_ends_the_burst() {
|
||
// The boundary itself: `max_gap` is inclusive, so two seconds continues
|
||
// a run and three begins a new one.
|
||
let rules = Rules::default();
|
||
let g = group(
|
||
&[
|
||
frame(1, 1000, 0xFF00),
|
||
frame(2, 1002, 0xFF00),
|
||
frame(3, 1005, 0xFF00),
|
||
frame(4, 1006, 0xFF00),
|
||
],
|
||
rules,
|
||
);
|
||
assert_eq!(
|
||
g,
|
||
vec![vec![ImageId(1), ImageId(2)], vec![ImageId(3), ImageId(4)],],
|
||
"the three-second pause did not end the first burst"
|
||
);
|
||
}
|
||
|
||
#[test]
|
||
fn two_subjects_one_second_apart_are_not_a_burst() {
|
||
// The failure FR-CULL-5 names, in its mildest form: turning round and
|
||
// photographing something else does not make the two frames one moment,
|
||
// however quickly it was done.
|
||
let bird = signature_of_luma(&scene(64, 64, 11, 0, 0), 64, 64).unwrap();
|
||
let sign = signature_of_luma(&scene(64, 64, 404, 0, 0), 64, 64).unwrap();
|
||
let g = group(
|
||
&[
|
||
Frame {
|
||
id: ImageId(1),
|
||
captured_at: 1000,
|
||
camera: Some("X-T5".into()),
|
||
signature: Some(bird),
|
||
},
|
||
Frame {
|
||
id: ImageId(2),
|
||
captured_at: 1001,
|
||
camera: Some("X-T5".into()),
|
||
signature: Some(sign),
|
||
},
|
||
],
|
||
Rules::default(),
|
||
);
|
||
assert!(g.is_empty(), "two different subjects were grouped: {g:?}");
|
||
}
|
||
|
||
#[test]
|
||
fn a_lone_frame_is_not_a_burst() {
|
||
let g = group(&[frame(1, 1000, 0xAB)], Rules::default());
|
||
assert!(g.is_empty());
|
||
}
|
||
|
||
#[test]
|
||
fn a_burst_chains_through_a_pan() {
|
||
// Frame one and frame four have nothing in common; each frame resembles
|
||
// the one before it. That is a camera following a bird, and it is one
|
||
// burst.
|
||
let rules = Rules::default();
|
||
let g = group(
|
||
&[
|
||
frame(1, 1000, 0x0000_0000_0000_0000),
|
||
frame(2, 1000, 0x0000_0000_0000_00FF),
|
||
frame(3, 1001, 0x0000_0000_0000_FFFF),
|
||
frame(4, 1001, 0x0000_0000_00FF_FFFF),
|
||
],
|
||
rules,
|
||
);
|
||
assert_eq!(g.len(), 1, "the pan was split: {g:?}");
|
||
assert_eq!(g[0].len(), 4);
|
||
// And the ends really are far apart — otherwise this test would pass
|
||
// without exercising the chaining at all.
|
||
assert!(Signature(0).distance(Signature(0x00FF_FFFF)) > rules.max_distance);
|
||
}
|
||
|
||
#[test]
|
||
fn frames_from_two_cameras_at_one_instant_stay_apart() {
|
||
// A second shooter, or a tethered body beside the one in your hands.
|
||
let g = group(
|
||
&[
|
||
Frame {
|
||
id: ImageId(1),
|
||
captured_at: 1000,
|
||
camera: Some("Canon EOS R5".into()),
|
||
signature: Some(Signature(0xFF00)),
|
||
},
|
||
Frame {
|
||
id: ImageId(2),
|
||
captured_at: 1000,
|
||
camera: Some("NIKON Z 9".into()),
|
||
signature: Some(Signature(0xFF00)),
|
||
},
|
||
],
|
||
Rules::default(),
|
||
);
|
||
assert!(g.is_empty(), "two bodies were merged into one burst: {g:?}");
|
||
}
|
||
|
||
#[test]
|
||
fn an_unknown_camera_does_not_split_a_burst() {
|
||
// metadata_state 1 is a normal resting state for a freshly scanned
|
||
// library; refusing to group on it would mean bursts appear only after
|
||
// a full EXIF sweep.
|
||
let g = group(
|
||
&[
|
||
Frame {
|
||
id: ImageId(1),
|
||
captured_at: 1000,
|
||
camera: None,
|
||
signature: Some(Signature(0xFF00)),
|
||
},
|
||
Frame {
|
||
id: ImageId(2),
|
||
captured_at: 1000,
|
||
camera: Some("Canon EOS R5".into()),
|
||
signature: Some(Signature(0xFF00)),
|
||
},
|
||
],
|
||
Rules::default(),
|
||
);
|
||
assert_eq!(g.len(), 1);
|
||
}
|
||
|
||
#[test]
|
||
fn an_unhashed_frame_never_groups() {
|
||
// Time alone would put these three together, which is precisely the
|
||
// wedding-ceremony failure.
|
||
let g = group(
|
||
&[
|
||
Frame {
|
||
id: ImageId(1),
|
||
captured_at: 1000,
|
||
camera: None,
|
||
signature: None,
|
||
},
|
||
Frame {
|
||
id: ImageId(2),
|
||
captured_at: 1001,
|
||
camera: None,
|
||
signature: None,
|
||
},
|
||
Frame {
|
||
id: ImageId(3),
|
||
captured_at: 1002,
|
||
camera: None,
|
||
signature: None,
|
||
},
|
||
],
|
||
Rules::default(),
|
||
);
|
||
assert!(g.is_empty(), "unhashed frames were grouped on time alone");
|
||
}
|
||
|
||
#[test]
|
||
fn input_order_does_not_change_the_answer() {
|
||
let frames = vec![
|
||
frame(3, 1002, 0xFF00),
|
||
frame(1, 1000, 0xFF00),
|
||
frame(2, 1001, 0xFF00),
|
||
];
|
||
let g = group(&frames, Rules::default());
|
||
assert_eq!(g, vec![vec![ImageId(1), ImageId(2), ImageId(3)]]);
|
||
}
|
||
|
||
// ── the pass, against a catalog ───────────────────────────────────────
|
||
|
||
/// A catalog holding `frames` as (id, captured_at, hash).
|
||
fn seeded(frames: &[(i64, i64, Option<u64>)]) -> Catalog {
|
||
let cat = Catalog::in_memory().unwrap();
|
||
let c = cat.connection();
|
||
c.execute(
|
||
"INSERT INTO roots(id, kind, label) VALUES (1, 'local', 'lib')",
|
||
[],
|
||
)
|
||
.unwrap();
|
||
for (id, at, hash) in frames {
|
||
c.execute(
|
||
"INSERT INTO images(id, root_id, source_ref, captured_at, camera,
|
||
perceptual_hash, added_at)
|
||
VALUES (?1, 1, ?2, ?3, 'Canon EOS R5', ?4, 0)",
|
||
rusqlite::params![
|
||
id,
|
||
format!("IMG_{id}.CR3"),
|
||
at,
|
||
hash.map(|h| Signature(h).to_stored())
|
||
],
|
||
)
|
||
.unwrap();
|
||
}
|
||
cat
|
||
}
|
||
|
||
#[test]
|
||
fn a_pass_records_the_bursts_it_found() {
|
||
let cat = seeded(&[
|
||
(1, 1000, Some(0xFF00)),
|
||
(2, 1001, Some(0xFF00)),
|
||
(3, 2000, Some(0xFF00)),
|
||
]);
|
||
let report = regroup(cat.connection(), Rules::default()).unwrap();
|
||
assert_eq!(report.bursts, 1);
|
||
assert_eq!(report.frames, 2);
|
||
assert_eq!(report.largest, 2);
|
||
|
||
let m = memberships(cat.connection(), &[ImageId(1), ImageId(2), ImageId(3)]).unwrap();
|
||
assert_eq!(m.len(), 2, "the lone frame should have no row");
|
||
assert_eq!(m[&ImageId(1)].burst_id, ImageId(1));
|
||
assert!(m[&ImageId(1)].representative);
|
||
assert!(!m[&ImageId(2)].representative);
|
||
assert_eq!(m[&ImageId(2)].size, 2);
|
||
}
|
||
|
||
#[test]
|
||
fn a_second_pass_replaces_the_first() {
|
||
let cat = seeded(&[(1, 1000, Some(0xFF00)), (2, 1001, Some(0xFF00))]);
|
||
regroup(cat.connection(), Rules::default()).unwrap();
|
||
// The frames turn out to be seconds apart after all — an EXIF sweep
|
||
// corrected the timestamp.
|
||
cat.connection()
|
||
.execute("UPDATE images SET captured_at = 3000 WHERE id = 2", [])
|
||
.unwrap();
|
||
let report = regroup(cat.connection(), Rules::default()).unwrap();
|
||
assert_eq!(report.bursts, 0);
|
||
assert!(memberships(cat.connection(), &[ImageId(1), ImageId(2)])
|
||
.unwrap()
|
||
.is_empty());
|
||
}
|
||
|
||
#[test]
|
||
fn the_representative_is_the_earliest_frame_and_not_a_judgement() {
|
||
let cat = seeded(&[
|
||
(7, 1000, Some(0xFF00)),
|
||
(8, 1000, Some(0xFF00)),
|
||
(9, 1001, Some(0xFF00)),
|
||
]);
|
||
regroup(cat.connection(), Rules::default()).unwrap();
|
||
let m = memberships(cat.connection(), &[ImageId(7), ImageId(8), ImageId(9)]).unwrap();
|
||
assert!(m[&ImageId(7)].representative);
|
||
assert!(!m[&ImageId(8)].representative);
|
||
assert!(!m[&ImageId(9)].representative);
|
||
}
|
||
|
||
#[test]
|
||
fn the_users_choice_of_representative_survives_a_regroup() {
|
||
// The same guarantee `people.ignored` has, for the same reason: a pass
|
||
// that forgot the decision would ask for it again after every scan.
|
||
let cat = seeded(&[
|
||
(1, 1000, Some(0xFF00)),
|
||
(2, 1001, Some(0xFF00)),
|
||
(3, 1002, Some(0xFF00)),
|
||
]);
|
||
regroup(cat.connection(), Rules::default()).unwrap();
|
||
choose_representative(cat.connection(), ImageId(2)).unwrap();
|
||
|
||
let m = memberships(cat.connection(), &[ImageId(2)]).unwrap();
|
||
assert!(
|
||
m[&ImageId(2)].representative,
|
||
"the pick did not take effect"
|
||
);
|
||
|
||
regroup(cat.connection(), Rules::default()).unwrap();
|
||
let m = memberships(cat.connection(), &[ImageId(1), ImageId(2)]).unwrap();
|
||
assert!(m[&ImageId(2)].representative, "the regroup forgot the pick");
|
||
assert!(!m[&ImageId(1)].representative);
|
||
}
|
||
|
||
#[test]
|
||
fn choosing_a_representative_outside_a_burst_does_nothing() {
|
||
let cat = seeded(&[(1, 1000, Some(0xFF00))]);
|
||
regroup(cat.connection(), Rules::default()).unwrap();
|
||
choose_representative(cat.connection(), ImageId(1)).unwrap();
|
||
let n: i64 = cat
|
||
.connection()
|
||
.query_row("SELECT count(*) FROM burst_pick", [], |r| r.get(0))
|
||
.unwrap();
|
||
assert_eq!(n, 0);
|
||
}
|
||
|
||
#[test]
|
||
fn a_newly_found_burst_arrives_open() {
|
||
// The pass must not take rows off the screen. It marks them; the user
|
||
// folds them.
|
||
let cat = seeded(&[(1, 1000, Some(0xFF00)), (2, 1001, Some(0xFF00))]);
|
||
regroup(cat.connection(), Rules::default()).unwrap();
|
||
assert!(is_expanded(cat.connection(), ImageId(1)).unwrap());
|
||
}
|
||
|
||
#[test]
|
||
fn a_burst_the_user_folded_up_stays_folded_through_the_next_pass() {
|
||
// Otherwise every import springs open a morning's culling.
|
||
let cat = seeded(&[
|
||
(1, 1000, Some(0xFF00)),
|
||
(2, 1001, Some(0xFF00)),
|
||
(3, 9000, Some(0x00FF)),
|
||
]);
|
||
regroup(cat.connection(), Rules::default()).unwrap();
|
||
set_expanded(cat.connection(), ImageId(1), false).unwrap();
|
||
|
||
// A frame arrives elsewhere in the library, so the pass runs again.
|
||
cat.connection()
|
||
.execute(
|
||
"INSERT INTO images(id, root_id, source_ref, captured_at, camera,
|
||
perceptual_hash, added_at)
|
||
VALUES (4, 1, 'IMG_4.CR3', 9001, 'Canon EOS R5', ?1, 0)",
|
||
[Signature(0x00FF).to_stored()],
|
||
)
|
||
.unwrap();
|
||
regroup(cat.connection(), Rules::default()).unwrap();
|
||
|
||
assert!(!is_expanded(cat.connection(), ImageId(1)).unwrap());
|
||
assert!(
|
||
is_expanded(cat.connection(), ImageId(3)).unwrap(),
|
||
"the burst nobody has seen yet should have arrived open"
|
||
);
|
||
}
|
||
|
||
#[test]
|
||
fn expansion_is_remembered_and_reversible() {
|
||
let cat = seeded(&[(1, 1000, Some(0xFF00)), (2, 1001, Some(0xFF00))]);
|
||
regroup(cat.connection(), Rules::default()).unwrap();
|
||
|
||
set_expanded(cat.connection(), ImageId(1), false).unwrap();
|
||
assert!(!is_expanded(cat.connection(), ImageId(1)).unwrap());
|
||
set_expanded(cat.connection(), ImageId(1), true).unwrap();
|
||
assert!(is_expanded(cat.connection(), ImageId(1)).unwrap());
|
||
// Idempotent: the UI toggles this from a callback that can fire twice.
|
||
set_expanded(cat.connection(), ImageId(1), true).unwrap();
|
||
assert_eq!(collapse_all(cat.connection()).unwrap(), 1);
|
||
assert!(!is_expanded(cat.connection(), ImageId(1)).unwrap());
|
||
}
|
||
|
||
#[test]
|
||
fn a_burst_that_no_longer_exists_leaves_no_expansion_behind() {
|
||
// The id is an image id, so a stale row would eventually re-open some
|
||
// unrelated group.
|
||
let cat = seeded(&[(1, 1000, Some(0xFF00)), (2, 1001, Some(0xFF00))]);
|
||
regroup(cat.connection(), Rules::default()).unwrap();
|
||
assert!(is_expanded(cat.connection(), ImageId(1)).unwrap());
|
||
|
||
cat.connection()
|
||
.execute("UPDATE images SET captured_at = 9000 WHERE id = 2", [])
|
||
.unwrap();
|
||
regroup(cat.connection(), Rules::default()).unwrap();
|
||
|
||
assert!(!is_expanded(cat.connection(), ImageId(1)).unwrap());
|
||
}
|
||
|
||
#[test]
|
||
fn a_collapsed_burst_hides_everything_but_its_representative() {
|
||
let cat = seeded(&[
|
||
(1, 1000, Some(0xFF00)),
|
||
(2, 1001, Some(0xFF00)),
|
||
(3, 1002, Some(0xFF00)),
|
||
(4, 9000, Some(0xFF00)),
|
||
]);
|
||
regroup(cat.connection(), Rules::default()).unwrap();
|
||
|
||
let sql = format!(
|
||
"SELECT id FROM images i WHERE {} ORDER BY id",
|
||
not_collapsed_away("i")
|
||
);
|
||
let visible = |c: &Connection| -> Vec<i64> {
|
||
let mut stmt = c.prepare(&sql).unwrap();
|
||
let rows = stmt.query_map([], |r| r.get::<_, i64>(0)).unwrap();
|
||
rows.collect::<Result<Vec<_>, _>>().unwrap()
|
||
};
|
||
|
||
// Open, as a new burst always is: nothing has been taken away.
|
||
assert_eq!(visible(cat.connection()), vec![1, 2, 3, 4]);
|
||
set_expanded(cat.connection(), ImageId(1), false).unwrap();
|
||
assert_eq!(visible(cat.connection()), vec![1, 4]);
|
||
set_expanded(cat.connection(), ImageId(1), true).unwrap();
|
||
assert_eq!(visible(cat.connection()), vec![1, 2, 3, 4]);
|
||
}
|
||
|
||
#[test]
|
||
fn a_library_with_no_bursts_hides_nothing() {
|
||
// The predicate is in every grid query, so its cost and its effect on a
|
||
// library that has never been grouped both matter.
|
||
let cat = seeded(&[(1, 1000, Some(0xFF00)), (2, 9000, Some(0xFF00))]);
|
||
let sql = format!(
|
||
"SELECT count(*) FROM images i WHERE {}",
|
||
not_collapsed_away("i")
|
||
);
|
||
let n: i64 = cat.connection().query_row(&sql, [], |r| r.get(0)).unwrap();
|
||
assert_eq!(n, 2);
|
||
}
|
||
|
||
#[test]
|
||
fn an_unhashed_image_is_offered_for_hashing_once() {
|
||
let cat = seeded(&[(1, 1000, None), (2, 1001, Some(0xFF00))]);
|
||
assert_eq!(
|
||
images_without_signature(cat.connection()).unwrap(),
|
||
vec![ImageId(1)]
|
||
);
|
||
set_signature(cat.connection(), ImageId(1), Signature(0x1234)).unwrap();
|
||
assert!(images_without_signature(cat.connection())
|
||
.unwrap()
|
||
.is_empty());
|
||
assert_eq!(
|
||
signature(cat.connection(), ImageId(1)).unwrap(),
|
||
Some(Signature(0x1234))
|
||
);
|
||
}
|
||
|
||
#[test]
|
||
fn hashing_an_image_the_scan_has_dropped_is_not_an_error() {
|
||
let cat = seeded(&[(1, 1000, None)]);
|
||
assert_eq!(
|
||
set_signature(cat.connection(), ImageId(404), Signature(1)).unwrap(),
|
||
0
|
||
);
|
||
}
|
||
|
||
#[test]
|
||
fn a_trashed_or_shadowed_frame_is_not_part_of_a_burst() {
|
||
// The same two exclusions the grid makes. A shadowed JPEG beside its RAW
|
||
// would otherwise be a two-frame burst with the frame it *is*.
|
||
let cat = seeded(&[
|
||
(1, 1000, Some(0xFF00)),
|
||
(2, 1000, Some(0xFF00)),
|
||
(3, 1001, Some(0xFF00)),
|
||
]);
|
||
cat.connection()
|
||
.execute("UPDATE images SET shadowed_by = 1 WHERE id = 2", [])
|
||
.unwrap();
|
||
cat.connection()
|
||
.execute("UPDATE images SET trashed_at = 99 WHERE id = 3", [])
|
||
.unwrap();
|
||
let report = regroup(cat.connection(), Rules::default()).unwrap();
|
||
assert_eq!(report.bursts, 0);
|
||
}
|
||
|
||
#[test]
|
||
fn members_come_back_in_the_order_the_grid_lists_them() {
|
||
let cat = seeded(&[
|
||
(5, 1002, Some(0xFF00)),
|
||
(6, 1000, Some(0xFF00)),
|
||
(7, 1001, Some(0xFF00)),
|
||
]);
|
||
regroup(cat.connection(), Rules::default()).unwrap();
|
||
assert_eq!(
|
||
members(cat.connection(), ImageId(6)).unwrap(),
|
||
vec![ImageId(6), ImageId(7), ImageId(5)]
|
||
);
|
||
}
|
||
}
|