From d3dadbd725ced5dd5733901aa94541557953fdec Mon Sep 17 00:00:00 2001 From: Duncan Tourolle Date: Sat, 29 Aug 2026 20:37:54 +0200 Subject: [PATCH] Group the frames of one moment, by when they were taken and what they look like A burst is the commonest thing in a cull and the least interesting: twelve frames of the same gull at 10 fps occupy twelve cells, are scrolled past twelve times, and end with the photographer keeping one. FR-CULL-5 asks for them to collapse to one representative and be judged as a unit. Two signals, because neither alone survives a real library. Time alone groups a whole wedding ceremony -- a photographer working steadily never leaves the gap that would end the run. Similarity alone groups a studio setup shot across two days, which is a project rather than a moment. Together they are specific: adjacent in time *and* looks like the frame before it. Two seconds is the time bound, and the reason is worth recording because the figure looks absurd next to a 10 fps camera. `images.captured_at` is whole seconds -- EXIF's DateTimeOriginal has no sub-second field and SubSecTimeOriginal is optional and widely omitted -- so a burst arrives in the catalog as ten frames sharing one timestamp. Any threshold finer than a second is a threshold on information that is not there. Where the pace really is faster, the similarity bound is what separates the frames. Similarity is a 64-bit difference hash over a 9x8 box-averaged reduction, compared between *adjacent* frames only. Chained rather than anchored on the first frame, because by frame twenty a camera following a bird has nothing in common with frame one while no two neighbours differ by much; the time bound is what stops the chain running away. There is no all-pairs step and there must never be one -- that is what turns a grouping pass into something nobody can afford to run over 50k images. Nothing here ranks a frame. FR-CULL-5 names the failure it is avoiding, which is rejecting the only frame of an important moment because somebody blinked, so there is no sharpness score and no best-of-burst. The representative is the earliest frame -- a fact about the clock, not a judgement about the photograph -- and the user's own choice lives in its own table so that rebuilding the grouping cannot erase it. Same argument `people.ignored` makes one subsystem over: nothing short of remembering a decision survives re-clustering. A newly found burst is recorded *open*. Collapsing on discovery would be tidier, and would also mean a background pass taking photographs off the screen part way through a cull. The pass marks; the user folds. It is a pass rather than a job kind for the reason catalog.md 10.2 gives for face clustering: a burst is a property of a run of frames and has no natural subject_id, so a per-image job would rebuild the world once per photograph. Co-Authored-By: Claude Opus 5 (1M context) --- core/dr-catalog/src/bursts.rs | 1300 +++++++++++++++++++++++++++++++++ core/dr-catalog/src/lib.rs | 2 + core/dr-catalog/src/schema.rs | 81 +- docs/catalog.md | 55 ++ 4 files changed, 1437 insertions(+), 1 deletion(-) create mode 100644 core/dr-catalog/src/bursts.rs diff --git a/core/dr-catalog/src/bursts.rs b/core/dr-catalog/src/bursts.rs new file mode 100644 index 0000000..3aa628e --- /dev/null +++ b/core/dr-catalog/src/bursts.rs @@ -0,0 +1,1300 @@ +//! TRACES: FR-CULL-5 +//! Bursts: frames that are one moment, grouped so they can be judged as one. +//! +//! A burst is the commonest thing in a cull and the least interesting. Twelve +//! frames of the same gull, shot at 10 fps, occupy twelve cells of the grid, +//! are scrolled past twelve times, and end with the photographer keeping one. +//! Collapsing them to a single cell with a count on it is the whole feature: +//! the twelve are still there, still reachable, still individually ratable — +//! they simply stop costing twelve decisions where one was meant. +//! +//! # What this deliberately does not do +//! +//! **Nothing here ranks a frame.** FR-CULL-5 names the failure it is avoiding: +//! automated *selection* is distrusted because the documented way it fails is +//! rejecting the only frame of an important moment because somebody blinked. +//! So there is no sharpness score, no eye detector, no "best of burst". The +//! representative of a group is the **earliest frame**, which is a fact about +//! the clock and not a judgement about the photograph, and the user can name a +//! different one at any time ([`choose_representative`]). +//! +//! That is also why this sits beside [`crate::dedup`] rather than inside it. +//! Dedup answers "are these the same file?" with a whole-file digest, for +//! import collision; two bytes of difference make two different answers, which +//! is exactly right there and useless here. A burst is a set of frames that are +//! *deliberately* different — the wing is in another place in each one. +//! +//! # Two signals, and why neither alone will do +//! +//! **Time alone** groups a wedding ceremony into one burst. Frames arrive +//! seconds apart for an hour, and a photographer shooting steadily never +//! produces the gap that would end the run. +//! +//! **Similarity alone** groups the same subject photographed on two different +//! days — a studio setup, a copy stand, a hundred frames of the same document. +//! Those are not a burst, they are a project, and collapsing them hides work +//! the user did on purpose. +//! +//! Together they are specific: *adjacent in time* **and** *looks like the frame +//! before it*. Both bounds are in [`Rules`]. +//! +//! # Chained, not compared to the first frame +//! +//! Each frame is tested against the one immediately before it, and a burst is a +//! run of frames that each joined the last. Comparing every frame to the *first* +//! would split a pan: by frame twenty a camera following a bird has nothing in +//! common with frame one, and yet no two adjacent frames in that sequence differ +//! by much. Chaining follows the subject; a fixed anchor loses it half way. +//! +//! The accepted consequence is that the first and last frames of a long burst +//! may be quite unlike each other. That is what a burst *is*, and the time bound +//! is what stops the chain running away: a sequence that has drifted a long way +//! has almost always paused somewhere, and the pause ends the run. +//! +//! # The similarity signal is a 64-bit difference hash +//! +//! [`Signature`] is a dHash taken on a 9×8 box-averaged reduction of the image: +//! 64 comparisons of a cell against its right-hand neighbour, one bit each. It +//! is scale-independent, survives JPEG artefacts and mild exposure changes, +//! costs nothing to store, and compares in one `xor` and a `popcount` — which +//! matters, because the grouping pass is a whole-library operation. +//! +//! What it is bad at is worth stating plainly: **a featureless frame hashes to +//! zero**, and so do all the other featureless frames. A lens cap, a black +//! frame, a white wall and an overexposed sky are mutually indistinguishable to +//! it. The capture-time bound is what keeps that from grouping every mistake in +//! the library into one enormous burst, and it is sufficient in practice — but +//! it is the reason this is grouping and not deduplication. +//! +//! # Where the pixels come from, and why the hash is stored +//! +//! This module never decodes an image. It takes a luma or RGBA buffer somebody +//! else already had in hand ([`signature_of_luma`], [`signature_of_rgba`]) and +//! `images.perceptual_hash` keeps the answer, the same bargain +//! [`crate::dedup::set_content_hash`] makes: whoever is already holding the +//! pixels pays nothing, and everybody afterwards pays nothing at all. In +//! practice the caller is the thumbnail store — a 256px thumbnail is far more +//! resolution than a 9×8 reduction needs — so a library that has been browsed +//! has already paid for its signatures. +//! +//! # A pass, not a job +//! +//! Grouping has no natural `subject_id`: it is a property of a *run* of frames, +//! so a per-image job would rebuild the world once per photograph. It is +//! therefore a debounced library-level pass, for exactly the reasons +//! docs/catalog.md §10.2 gives for face clustering, and [`regroup`] is the whole +//! of it — one ordered walk, no per-pair comparison beyond adjacent frames. +//! +//! # Grouping is not hiding +//! +//! A burst this pass has never seen before is recorded **open**: the grid keeps +//! every row it had, and all that appears is a mark saying how many frames each +//! run holds. Only the user folds one up. That is a deliberate refusal of the +//! obvious default — collapsing on discovery would be tidier and would also +//! mean a background pass removing photographs from under somebody part way +//! through a cull, which is the same class of surprise FR-CULL-5 is written to +//! avoid. A burst that is already known keeps whatever state it is in, so a +//! pass after the next import does not spring open a morning's work. +//! +//! # What is stored, and what a merge does with it +//! +//! `burst_members` is derived: deleting it costs one pass and loses nothing. +//! `burst_pick` is not — it is the user saying which frame stands for the +//! moment, and it lives in its own table precisely so [`regroup`] can rewrite +//! the grouping without erasing the judgement. That is the same argument +//! `people.ignored` makes one subsystem over: nothing short of remembering a +//! decision survives re-clustering. +//! +//! Both tables travel in the uploaded catalog snapshot, and nothing on the far +//! side reads them — [`crate::sync`] merges collections and keywords only — so +//! a second device rebuilds its own grouping from its own signatures. Bursts +//! are not yet cross-device state, and pretending otherwise would mean syncing +//! `burst_pick` as user data, which is a merge question this does not answer. + +use std::collections::{HashMap, HashSet}; + +use dr_types::ImageId; +use rusqlite::Connection; + +use crate::CatalogError; + +/// A 64-bit perceptual signature of one image. +/// +/// Comparable only to other signatures from this build: the reduction grid and +/// the bit order are part of the definition, and changing either silently +/// changes what "similar" means. If that ever happens the column has to be +/// cleared, the way a face `model_id` change forces a re-index. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord)] +pub struct Signature(pub u64); + +impl Signature { + /// Differing bits, 0..=64. Small is similar. + pub fn distance(self, other: Signature) -> u32 { + (self.0 ^ other.0).count_ones() + } + + /// The stored form. SQLite integers are signed, so the bits are reinterpreted + /// rather than converted — `as` in both directions is exact and lossless. + pub fn to_stored(self) -> i64 { + self.0 as i64 + } + + /// The inverse of [`Self::to_stored`]. + pub fn from_stored(v: i64) -> Self { + Signature(v as u64) + } +} + +/// Columns in the reduction grid. One more than the number of comparisons per +/// row, because each bit compares a cell against its right-hand neighbour. +const GRID_W: usize = 9; +/// Rows in the reduction grid. `(GRID_W - 1) * GRID_H` is 64 — the width of the +/// signature, and the reason these two numbers are what they are. +const GRID_H: usize = 8; + +/// Reduce a greyscale buffer to a [`Signature`]. +/// +/// `luma` is row-major, one byte a pixel, `width * height` of them. Returns +/// `None` for an empty or short buffer rather than panicking: the caller is +/// usually handing over the result of a decode, and a truncated JPEG is a +/// normal thing to find in a library rather than a programming error. +/// +/// # Why box-averaged rather than sampled +/// +/// Point-sampling a 9×8 grid out of a 256px thumbnail reads 72 pixels of +/// several hundred thousand, and two frames of a burst that differ by one pixel +/// of camera shake can sample entirely different detail. Averaging the whole +/// cell is what makes the signature stable under the small movements a burst is +/// made of — which is the property the whole feature rests on. +pub fn signature_of_luma(luma: &[u8], width: u32, height: u32) -> Option { + let (w, h) = (width as usize, height as usize); + if w == 0 || h == 0 || luma.len() < w * h { + return None; + } + + let mut cells = [0f32; GRID_W * GRID_H]; + for gy in 0..GRID_H { + // Cell bounds by proportion, so every source pixel lands in exactly one + // cell whatever the aspect ratio. The `max` keeps a cell non-empty when + // the source is narrower or shorter than the grid — a 4px-wide preview + // is degenerate but must not divide by zero. + let y0 = gy * h / GRID_H; + let y1 = (((gy + 1) * h) / GRID_H).max(y0 + 1).min(h); + for gx in 0..GRID_W { + let x0 = gx * w / GRID_W; + let x1 = (((gx + 1) * w) / GRID_W).max(x0 + 1).min(w); + + let mut sum = 0u32; + let mut n = 0u32; + for y in y0..y1 { + let row = &luma[y * w..y * w + w]; + for px in &row[x0..x1] { + sum += *px as u32; + n += 1; + } + } + cells[gy * GRID_W + gx] = sum as f32 / n as f32; + } + } + + // Each bit: is this cell brighter than the one to its right? A *difference* + // rather than a level, which is what makes the signature indifferent to the + // exposure drifting a third of a stop through a burst. + let mut bits = 0u64; + for gy in 0..GRID_H { + for gx in 0..GRID_W - 1 { + bits <<= 1; + if cells[gy * GRID_W + gx] > cells[gy * GRID_W + gx + 1] { + bits |= 1; + } + } + } + Some(Signature(bits)) +} + +/// [`signature_of_luma`] for the packed RGBA a decoded thumbnail arrives as. +/// +/// Alpha is ignored: a thumbnail is opaque, and a signature that changed with +/// it would make two renderings of one frame look like two photographs. +pub fn signature_of_rgba(rgba: &[u8], width: u32, height: u32) -> Option { + let (w, h) = (width as usize, height as usize); + if w == 0 || h == 0 || rgba.len() < w * h * 4 { + return None; + } + // Rec. 601 luma in fixed point. The exact weights matter less than their + // being the same weights every time — see [`Signature`]. + let luma: Vec = rgba + .chunks_exact(4) + .take(w * h) + .map(|p| ((77 * p[0] as u32 + 150 * p[1] as u32 + 29 * p[2] as u32) >> 8) as u8) + .collect(); + signature_of_luma(&luma, width, height) +} + +/// The two bounds that decide what counts as one burst. +/// +/// Held together in a struct rather than passed as two numbers so a caller +/// cannot supply one and default the other, and so the pair can be logged as +/// what a given grouping was produced under. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct Rules { + /// Longest gap, in seconds, between two frames of one burst. + pub max_gap: i64, + /// Largest [`Signature::distance`] between two frames of one burst. + pub max_distance: u32, +} + +impl Default for Rules { + /// # Why two seconds, when a burst fires ten frames in one + /// + /// Because `images.captured_at` is **whole seconds**. EXIF `DateTimeOriginal` + /// has no sub-second field — `SubSecTimeOriginal` is a separate, optional tag + /// that plenty of bodies omit — so a 10 fps burst arrives in the catalog as + /// ten frames sharing one timestamp, and any threshold below a second is a + /// threshold on information that is not there. + /// + /// Two seconds is therefore the smallest bound that can distinguish + /// anything: it holds a burst together across the second boundary it will + /// certainly straddle, and it is short enough that a photographer working at + /// a considered pace — a frame every three or four seconds — is never + /// grouped. Where the pace really is faster than that, the similarity bound + /// is what separates the frames. + /// + /// # Why eight bits of sixty-four + /// + /// A difference hash of two frames of one burst typically differs by nought + /// to four bits; two unrelated photographs differ by twenty to thirty-two, + /// with thirty-two being the expected distance between two *random* + /// signatures. Eight sits in the empty ground between those, near enough to + /// the burst end of it that a subject change inside one second — the case + /// FR-CULL-5's failure mode is really about — does not merge. + /// + /// Erring low is the right direction: a burst left ungrouped costs the user + /// a scroll, and two moments wrongly merged hide a photograph behind a cell + /// that does not look like it. + fn default() -> Self { + Rules { + max_gap: 2, + max_distance: 8, + } + } +} + +/// One frame as the grouping pass sees it. +/// +/// Deliberately not a catalog row: the whole decision is made from these four +/// fields, which is what lets [`group`] be tested against constructed frames +/// with no database at all. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct Frame { + pub id: ImageId, + /// UTC seconds. A frame without one cannot be placed in a run and is never + /// read by the pass. + pub captured_at: i64, + /// The joined make-and-model string, as the scan stores it. + pub camera: Option, + /// `None` until something has hashed this image's pixels. + pub signature: Option, +} + +/// Whether `b` continues the burst that `a` is part of. +/// +/// `b` is assumed not to precede `a`. +fn continues(a: &Frame, b: &Frame, rules: Rules) -> bool { + if b.captured_at - a.captured_at > rules.max_gap { + return false; + } + + // Two bodies firing at the same instant are two photographs of one moment, + // not one burst — a second shooter at a wedding, or a camera on a tripod + // triggered alongside the one in your hands. Where either camera is unknown + // the frames are not separated on that account: an unread EXIF header is + // absence of evidence, and the similarity bound still has to be satisfied. + if let (Some(x), Some(y)) = (&a.camera, &b.camera) { + if x != y { + return false; + } + } + + // A missing signature never groups. It would be easy to fall back to time + // alone here and it would be wrong: an unhashed library would collapse + // every steadily-shot sequence in it, and the user would have no way to see + // that the reason was a missing hash rather than a resemblance. + match (a.signature, b.signature) { + (Some(p), Some(q)) => p.distance(q) <= rules.max_distance, + _ => false, + } +} + +/// Group frames into bursts. The whole of the algorithm. +/// +/// Returns only groups of two or more, each ordered by capture time, in the +/// order the bursts themselves begin. A lone frame is not a burst and gets no +/// row anywhere: "is this image in a burst" is then answerable as "does it have +/// a `burst_members` row", and the grid pays nothing for the overwhelming +/// majority of a library that is not bursts. +/// +/// Sorts its own input. The caller usually reads frames in capture order +/// anyway, but the run structure is only meaningful in that order and a +/// mis-ordered caller would get silently wrong groups rather than an error. +pub fn group(frames: &[Frame], rules: Rules) -> Vec> { + let mut ordered: Vec<&Frame> = frames.iter().collect(); + // Ties on the second broken by id, which is the order the frames were + // catalogued in and therefore the order the camera wrote them. It is the + // same tiebreak the grid uses, so a burst is a contiguous run on screen. + ordered.sort_by_key(|f| (f.captured_at, f.id.0)); + + let mut out: Vec> = Vec::new(); + let mut run: Vec = Vec::new(); + + for pair in ordered.windows(2) { + let (a, b) = (pair[0], pair[1]); + if continues(a, b, rules) { + if run.is_empty() { + run.push(a.id); + } + run.push(b.id); + } else if !run.is_empty() { + out.push(std::mem::take(&mut run)); + } + } + if !run.is_empty() { + out.push(run); + } + out +} + +/// What one [`regroup`] did, for the log and for a progress line. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] +pub struct Report { + /// Groups of two or more frames. + pub bursts: usize, + /// Frames inside them. Never the size of the library. + pub frames: usize, + /// Frames in the largest single burst. + pub largest: usize, +} + +/// Rebuild the whole library's grouping. +/// +/// One ordered read of the dated, visible images, one pass over it, one +/// transaction. The cost is linear in library size and the comparison count is +/// one per *adjacent pair* — there is no all-pairs step here and there must +/// never be one, because that is what turns a grouping pass into a job nobody +/// can afford to run. +/// +/// Existing membership is replaced rather than amended. Amending would need to +/// know which frames had changed since the last pass, and the answer is +/// "possibly any of them, because a signature arrives long after the row does"; +/// a full rebuild is a few milliseconds and cannot drift. +pub fn regroup(conn: &Connection, rules: Rules) -> Result { + let frames = read_frames(conn)?; + let groups = group(&frames, rules); + let picked = user_picks(conn)?; + let known = known_bursts(conn)?; + + let tx = conn.unchecked_transaction()?; + tx.execute("DELETE FROM burst_members", [])?; + { + let mut insert = tx.prepare( + "INSERT INTO burst_members(image_id, burst_id, representative) + VALUES (?1, ?2, ?3)", + )?; + for members in &groups { + // The earliest frame's own id names the burst. It is stable across + // a regroup that leaves the burst alone, which is what lets the UI + // remember that this group is open; it is not a durable identity, + // and a burst that gains an earlier frame is a new group as far as + // anything keyed on this is concerned. + let burst_id = members[0].0 as i64; + let representative = members + .iter() + .find(|id| picked.contains(*id)) + .copied() + .unwrap_or(members[0]); + for id in members { + insert.execute(rusqlite::params![ + id.0 as i64, + burst_id, + (*id == representative) as i64 + ])?; + } + + // **A burst the user has never seen arrives open.** Nothing is + // hidden by this pass running: the grid holds exactly the + // photographs it held before, and the only new thing is a mark on + // each burst saying how many frames it is part of. Folding one up + // is then something the user did, which is the difference between + // a tool that groups and a tool that decides — and it is the only + // arrangement in which a background pass cannot make a row + // disappear from under somebody mid-cull. + // + // A burst that is already known keeps whatever state it is in, so + // the pass that runs after the next import does not spring open + // everything the user spent the morning folding away. + if !known.contains(&burst_id) { + tx.execute( + "INSERT OR IGNORE INTO burst_expanded(burst_id) VALUES (?1)", + [burst_id], + )?; + } + } + } + // A group that no longer exists must not leave an expansion behind: the id + // is an image id, and the next pass could hand it to a different burst. + tx.execute( + "DELETE FROM burst_expanded + WHERE burst_id NOT IN (SELECT burst_id FROM burst_members)", + [], + )?; + tx.commit()?; + + Ok(Report { + bursts: groups.len(), + frames: groups.iter().map(|g| g.len()).sum(), + largest: groups.iter().map(|g| g.len()).max().unwrap_or(0), + }) +} + +/// The frames a grouping pass considers, in capture order. +/// +/// The same two exclusions the grid makes, for the same two reasons: a shadowed +/// JPEG is its RAW's frame rather than a second photograph, and a trashed image +/// is not in the library. Undated frames are excluded because a burst is a +/// statement about time and there is nothing to say about a frame with none. +fn read_frames(conn: &Connection) -> Result, CatalogError> { + let mut stmt = conn.prepare( + "SELECT id, captured_at, camera, perceptual_hash + FROM images + WHERE captured_at IS NOT NULL + AND shadowed_by IS NULL + AND trashed_at IS NULL + ORDER BY captured_at ASC, id ASC", + )?; + let rows = stmt.query_map([], |r| { + Ok(Frame { + id: ImageId(r.get::<_, i64>(0)? as u64), + captured_at: r.get(1)?, + camera: r.get(2)?, + signature: r.get::<_, Option>(3)?.map(Signature::from_stored), + }) + })?; + Ok(rows.collect::, _>>()?) +} + +/// The bursts the previous pass left behind, so this one can tell a group the +/// user has already met from a group that is new. +fn known_bursts(conn: &Connection) -> Result, CatalogError> { + let mut stmt = conn.prepare("SELECT DISTINCT burst_id FROM burst_members")?; + let rows = stmt.query_map([], |r| r.get::<_, i64>(0))?; + Ok(rows.collect::, _>>()?) +} + +fn user_picks(conn: &Connection) -> Result, CatalogError> { + let mut stmt = conn.prepare("SELECT image_id FROM burst_pick")?; + let rows = stmt.query_map([], |r| Ok(ImageId(r.get::<_, i64>(0)? as u64)))?; + Ok(rows.collect::, _>>()?) +} + +/// Record the signature of an image somebody has just decoded. +/// +/// Returns rows updated: zero means the image is no longer in the catalog, +/// which is a normal race with a rescan rather than an error — the same answer +/// [`crate::dedup::set_content_hash`] gives. +pub fn set_signature( + conn: &Connection, + image: ImageId, + signature: Signature, +) -> Result { + Ok(conn.execute( + "UPDATE images SET perceptual_hash = ?2 WHERE id = ?1", + rusqlite::params![image.0 as i64, signature.to_stored()], + )?) +} + +/// What [`set_signature`] stored, if anything. +pub fn signature(conn: &Connection, image: ImageId) -> Result, CatalogError> { + let stored: Option = conn.query_row( + "SELECT perceptual_hash FROM images WHERE id = ?1", + [image.0 as i64], + |r| r.get(0), + )?; + Ok(stored.map(Signature::from_stored)) +} + +/// Visible, dated images that nothing has hashed yet. +/// +/// What the pass that fills the column iterates. Dated, because an undated +/// frame can never join a burst however well it hashes, and hashing it would be +/// work with no possible consequence. +pub fn images_without_signature(conn: &Connection) -> Result, CatalogError> { + let mut stmt = conn.prepare( + "SELECT id FROM images + WHERE perceptual_hash IS NULL + AND captured_at IS NOT NULL + AND shadowed_by IS NULL + AND trashed_at IS NULL + ORDER BY captured_at ASC, id ASC", + )?; + let rows = stmt.query_map([], |r| Ok(ImageId(r.get::<_, i64>(0)? as u64)))?; + Ok(rows.collect::, _>>()?) +} + +/// What the grid needs to know about one cell's burst. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct Membership { + /// The group's id, which is also the image id of its earliest frame. + pub burst_id: ImageId, + /// Frames in the group, this one included. Always two or more. + pub size: u32, + /// Whether this is the frame the group collapses to. + pub representative: bool, + /// Whether the group is currently open in the grid. + pub expanded: bool, +} + +/// Burst membership for one window of the grid, in a single query. +/// +/// One statement for the whole window rather than one per cell, which is the +/// same discipline the collection badges and the rating counts follow: a grid +/// that asks the catalog a question per cell is a grid that stutters while a +/// finger is on it. +pub fn memberships( + conn: &Connection, + images: &[ImageId], +) -> Result, CatalogError> { + if images.is_empty() { + return Ok(HashMap::new()); + } + // Placeholders are generated from the *count* of ids, never from anything + // the user typed. + let placeholders = std::iter::repeat_n("?", images.len()) + .collect::>() + .join(","); + let sql = format!( + "SELECT bm.image_id, + bm.burst_id, + bm.representative, + (SELECT count(*) FROM burst_members m WHERE m.burst_id = bm.burst_id), + EXISTS (SELECT 1 FROM burst_expanded be WHERE be.burst_id = bm.burst_id) + FROM burst_members bm + WHERE bm.image_id IN ({placeholders})" + ); + let params: Vec = images + .iter() + .map(|i| rusqlite::types::Value::Integer(i.0 as i64)) + .collect(); + + let mut stmt = conn.prepare(&sql)?; + let rows = stmt.query_map(rusqlite::params_from_iter(params.iter()), |r| { + Ok(( + ImageId(r.get::<_, i64>(0)? as u64), + Membership { + burst_id: ImageId(r.get::<_, i64>(1)? as u64), + size: r.get::<_, i64>(3)? as u32, + representative: r.get::<_, i64>(2)? != 0, + expanded: r.get::<_, i64>(4)? != 0, + }, + )) + })?; + Ok(rows.collect::, _>>()?) +} + +/// The frames of one burst, in the order the grid lists them. +pub fn members(conn: &Connection, burst: ImageId) -> Result, CatalogError> { + let mut stmt = conn.prepare( + "SELECT bm.image_id + FROM burst_members bm + JOIN images i ON i.id = bm.image_id + WHERE bm.burst_id = ?1 + ORDER BY i.captured_at ASC, i.id ASC", + )?; + let rows = stmt.query_map([burst.0 as i64], |r| Ok(ImageId(r.get::<_, i64>(0)? as u64)))?; + Ok(rows.collect::, _>>()?) +} + +/// Open or close one burst in the grid. +/// +/// Kept in the catalog rather than in the interface's own memory for one +/// reason: the grid is a *window* over an ordered query, and what is collapsed +/// has to be decided by that query or the window's row count stops matching the +/// scrollbar. Once the state has to be visible to SQL, the catalog is where it +/// lives — and a culling session that survives closing the app is the shape +/// FR-CULL-4 already asks for elsewhere. +pub fn set_expanded(conn: &Connection, burst: ImageId, expanded: bool) -> Result<(), CatalogError> { + if expanded { + conn.execute( + // `OR IGNORE` rather than an upsert: the toggle is wired to a + // callback that can fire twice, and re-opening an open burst must + // be a no-op rather than an error. + "INSERT OR IGNORE INTO burst_expanded(burst_id) VALUES (?1)", + [burst.0 as i64], + )?; + } else { + conn.execute( + "DELETE FROM burst_expanded WHERE burst_id = ?1", + [burst.0 as i64], + )?; + } + Ok(()) +} + +/// Whether this burst is currently open. +pub fn is_expanded(conn: &Connection, burst: ImageId) -> Result { + let n: i64 = conn.query_row( + "SELECT count(*) FROM burst_expanded WHERE burst_id = ?1", + [burst.0 as i64], + |r| r.get(0), + )?; + Ok(n > 0) +} + +/// Close every open burst. Returns how many were open. +pub fn collapse_all(conn: &Connection) -> Result { + Ok(conn.execute("DELETE FROM burst_expanded", [])?) +} + +/// The user names the frame their burst collapses to. +/// +/// The one *choice* in this module, and the reason the rest of it makes none. +/// Recorded against the image rather than the group so that it survives the +/// next [`regroup`]: a group is rebuilt from scratch every pass, and a +/// representative stored on it would be forgotten every time a frame arrived. +/// +/// Any previous pick within the same burst is dropped — a burst collapses to +/// one frame, and two picks would make the choice depend on row order. +pub fn choose_representative(conn: &Connection, image: ImageId) -> Result<(), CatalogError> { + let id = image.0 as i64; + let tx = conn.unchecked_transaction()?; + + // Only meaningful for an image that is actually in a burst; anything else + // would leave a pick that no group can ever honour. + let burst: Option = tx + .query_row( + "SELECT burst_id FROM burst_members WHERE image_id = ?1", + [id], + |r| r.get(0), + ) + .ok(); + let Some(burst) = burst else { + return Ok(()); + }; + + tx.execute( + "DELETE FROM burst_pick + WHERE image_id IN (SELECT image_id FROM burst_members WHERE burst_id = ?1)", + [burst], + )?; + tx.execute("INSERT INTO burst_pick(image_id) VALUES (?1)", [id])?; + // The grouping already exists, so the flag it carries is corrected now + // rather than at the next pass — the user expects the cell to change under + // the pointer, not after a background sweep. + tx.execute( + "UPDATE burst_members SET representative = (image_id = ?2) WHERE burst_id = ?1", + [burst, id], + )?; + tx.commit()?; + Ok(()) +} + +/// SQL for "this row is not a frame a collapsed burst is standing in for". +/// +/// Handed out as a predicate rather than as a list of ids because the grid is a +/// window: the rows a collapsed burst hides are mostly not loaded, so the +/// question can only be answered where the ordering and the `LIMIT` are — in +/// the query itself. `image` names the table or alias the image row is in, +/// because the grid aliases `images` as `i` and other callers do not. +/// +/// Both subqueries are probes on a primary key, so this costs one index lookup +/// per row the query walks and nothing at all for a library with no bursts in +/// it, where `burst_members` is empty. +/// +/// Never interpolate anything user-supplied as `image`. +pub fn not_collapsed_away(image: &str) -> String { + format!( + "NOT EXISTS (SELECT 1 FROM burst_members bm + WHERE bm.image_id = {image}.id + AND bm.representative = 0 + AND NOT EXISTS (SELECT 1 FROM burst_expanded be + WHERE be.burst_id = bm.burst_id))" + ) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::Catalog; + + /// A frame with everything the grouping looks at. + fn frame(id: u64, at: i64, sig: u64) -> Frame { + Frame { + id: ImageId(id), + captured_at: at, + camera: Some("Canon EOS R5".into()), + signature: Some(Signature(sig)), + } + } + + // ── the signature ───────────────────────────────────────────────────── + + /// A blocky pseudo-random scene, deterministic and free of any dependency. + /// + /// Blocks rather than per-pixel noise, because per-pixel noise averages to a + /// flat grey in the 9×8 reduction and every such image would hash alike — + /// which would make the tests below pass for the wrong reason. + fn scene(w: usize, h: usize, seed: u64, shift: usize, brighter: i32) -> Vec { + let block = 8; + let mut out = vec![0u8; w * h]; + for y in 0..h { + for x in 0..w { + let bx = (x + shift) / block; + let by = y / block; + // A small LCG, evaluated per block: reproducible, and nothing + // in the workspace has to provide it. + let mut s = seed + .wrapping_add(bx as u64) + .wrapping_mul(6_364_136_223_846_793_005) + .wrapping_add(by as u64 * 1_442_695_040_888_963_407); + s ^= s >> 33; + let v = (s % 256) as i32 + brighter; + out[y * w + x] = v.clamp(0, 255) as u8; + } + } + out + } + + #[test] + fn two_frames_of_one_burst_hash_almost_alike() { + // The property the whole feature rests on: the same scene, moved by a + // pixel and a third of a stop brighter, is the same signature or very + // nearly. + let a = signature_of_luma(&scene(64, 64, 7, 0, 0), 64, 64).unwrap(); + let b = signature_of_luma(&scene(64, 64, 7, 1, 6), 64, 64).unwrap(); + assert!( + a.distance(b) <= Rules::default().max_distance, + "a burst pair differed by {} bits, which the default rules would \ + not group", + a.distance(b) + ); + } + + #[test] + fn two_different_subjects_do_not_hash_alike() { + let a = signature_of_luma(&scene(64, 64, 7, 0, 0), 64, 64).unwrap(); + let b = signature_of_luma(&scene(64, 64, 99, 0, 0), 64, 64).unwrap(); + assert!( + a.distance(b) > Rules::default().max_distance, + "two unrelated scenes differed by only {} bits", + a.distance(b) + ); + } + + #[test] + fn the_signature_does_not_change_with_scale() { + // Detection runs against whichever preview tier is in hand, and the + // cache is entitled to regenerate one at another size. A signature that + // moved with the resolution would silently stop matching its own + // library. + let big = scene(128, 128, 21, 0, 0); + // Nearest-neighbour halving, which is the harshest thing a real + // downscale could do to it. + let mut small = vec![0u8; 64 * 64]; + for y in 0..64 { + for x in 0..64 { + small[y * 64 + x] = big[(y * 2) * 128 + x * 2]; + } + } + let a = signature_of_luma(&big, 128, 128).unwrap(); + let b = signature_of_luma(&small, 64, 64).unwrap(); + assert!( + a.distance(b) <= Rules::default().max_distance, + "halving the image moved {} bits, enough to stop it matching itself", + a.distance(b) + ); + } + + #[test] + fn a_short_or_empty_buffer_is_not_a_signature() { + assert!(signature_of_luma(&[], 0, 0).is_none()); + assert!(signature_of_luma(&[1, 2, 3], 64, 64).is_none()); + assert!(signature_of_rgba(&[1, 2, 3, 255], 64, 64).is_none()); + } + + #[test] + fn rgba_and_luma_agree() { + let luma = scene(64, 64, 3, 0, 0); + let rgba: Vec = luma.iter().flat_map(|v| [*v, *v, *v, 255]).collect(); + // Grey is grey: the Rec. 601 weights sum to 256/256, so a neutral pixel + // survives the conversion within a rounding step. + let a = signature_of_luma(&luma, 64, 64).unwrap(); + let b = signature_of_rgba(&rgba, 64, 64).unwrap(); + assert!(a.distance(b) <= 2, "{} bits apart", a.distance(b)); + } + + #[test] + fn a_signature_survives_the_trip_through_sqlite() { + // The top bit is the one at risk: SQLite integers are signed. + let s = Signature(u64::MAX); + assert_eq!(Signature::from_stored(s.to_stored()), s); + assert_eq!(Signature::from_stored(Signature(0).to_stored()), Signature(0)); + } + + // ── the grouping ────────────────────────────────────────────────────── + + #[test] + fn frames_a_second_apart_and_alike_are_one_burst() { + let g = group( + &[frame(1, 1000, 0xFF00), frame(2, 1001, 0xFF00), frame(3, 1001, 0xFF01)], + Rules::default(), + ); + assert_eq!(g, vec![vec![ImageId(1), ImageId(2), ImageId(3)]]); + } + + #[test] + fn a_pause_ends_the_burst() { + // The boundary itself: `max_gap` is inclusive, so two seconds continues + // a run and three begins a new one. + let rules = Rules::default(); + let g = group( + &[ + frame(1, 1000, 0xFF00), + frame(2, 1002, 0xFF00), + frame(3, 1005, 0xFF00), + frame(4, 1006, 0xFF00), + ], + rules, + ); + assert_eq!( + g, + vec![ + vec![ImageId(1), ImageId(2)], + vec![ImageId(3), ImageId(4)], + ], + "the three-second pause did not end the first burst" + ); + } + + #[test] + fn two_subjects_one_second_apart_are_not_a_burst() { + // The failure FR-CULL-5 names, in its mildest form: turning round and + // photographing something else does not make the two frames one moment, + // however quickly it was done. + let bird = signature_of_luma(&scene(64, 64, 11, 0, 0), 64, 64).unwrap(); + let sign = signature_of_luma(&scene(64, 64, 404, 0, 0), 64, 64).unwrap(); + let g = group( + &[ + Frame { + id: ImageId(1), + captured_at: 1000, + camera: Some("X-T5".into()), + signature: Some(bird), + }, + Frame { + id: ImageId(2), + captured_at: 1001, + camera: Some("X-T5".into()), + signature: Some(sign), + }, + ], + Rules::default(), + ); + assert!(g.is_empty(), "two different subjects were grouped: {g:?}"); + } + + #[test] + fn a_lone_frame_is_not_a_burst() { + let g = group(&[frame(1, 1000, 0xAB)], Rules::default()); + assert!(g.is_empty()); + } + + #[test] + fn a_burst_chains_through_a_pan() { + // Frame one and frame four have nothing in common; each frame resembles + // the one before it. That is a camera following a bird, and it is one + // burst. + let rules = Rules::default(); + let g = group( + &[ + frame(1, 1000, 0x0000_0000_0000_0000), + frame(2, 1000, 0x0000_0000_0000_00FF), + frame(3, 1001, 0x0000_0000_0000_FFFF), + frame(4, 1001, 0x0000_0000_00FF_FFFF), + ], + rules, + ); + assert_eq!(g.len(), 1, "the pan was split: {g:?}"); + assert_eq!(g[0].len(), 4); + // And the ends really are far apart — otherwise this test would pass + // without exercising the chaining at all. + assert!(Signature(0).distance(Signature(0x00FF_FFFF)) > rules.max_distance); + } + + #[test] + fn frames_from_two_cameras_at_one_instant_stay_apart() { + // A second shooter, or a tethered body beside the one in your hands. + let g = group( + &[ + Frame { + id: ImageId(1), + captured_at: 1000, + camera: Some("Canon EOS R5".into()), + signature: Some(Signature(0xFF00)), + }, + Frame { + id: ImageId(2), + captured_at: 1000, + camera: Some("NIKON Z 9".into()), + signature: Some(Signature(0xFF00)), + }, + ], + Rules::default(), + ); + assert!(g.is_empty(), "two bodies were merged into one burst: {g:?}"); + } + + #[test] + fn an_unknown_camera_does_not_split_a_burst() { + // metadata_state 1 is a normal resting state for a freshly scanned + // library; refusing to group on it would mean bursts appear only after + // a full EXIF sweep. + let g = group( + &[ + Frame { + id: ImageId(1), + captured_at: 1000, + camera: None, + signature: Some(Signature(0xFF00)), + }, + Frame { + id: ImageId(2), + captured_at: 1000, + camera: Some("Canon EOS R5".into()), + signature: Some(Signature(0xFF00)), + }, + ], + Rules::default(), + ); + assert_eq!(g.len(), 1); + } + + #[test] + fn an_unhashed_frame_never_groups() { + // Time alone would put these three together, which is precisely the + // wedding-ceremony failure. + let g = group( + &[ + Frame { + id: ImageId(1), + captured_at: 1000, + camera: None, + signature: None, + }, + Frame { + id: ImageId(2), + captured_at: 1001, + camera: None, + signature: None, + }, + Frame { + id: ImageId(3), + captured_at: 1002, + camera: None, + signature: None, + }, + ], + Rules::default(), + ); + assert!(g.is_empty(), "unhashed frames were grouped on time alone"); + } + + #[test] + fn input_order_does_not_change_the_answer() { + let frames = vec![ + frame(3, 1002, 0xFF00), + frame(1, 1000, 0xFF00), + frame(2, 1001, 0xFF00), + ]; + let g = group(&frames, Rules::default()); + assert_eq!(g, vec![vec![ImageId(1), ImageId(2), ImageId(3)]]); + } + + // ── the pass, against a catalog ─────────────────────────────────────── + + /// A catalog holding `frames` as (id, captured_at, hash). + fn seeded(frames: &[(i64, i64, Option)]) -> Catalog { + let cat = Catalog::in_memory().unwrap(); + let c = cat.connection(); + c.execute( + "INSERT INTO roots(id, kind, label) VALUES (1, 'local', 'lib')", + [], + ) + .unwrap(); + for (id, at, hash) in frames { + c.execute( + "INSERT INTO images(id, root_id, source_ref, captured_at, camera, + perceptual_hash, added_at) + VALUES (?1, 1, ?2, ?3, 'Canon EOS R5', ?4, 0)", + rusqlite::params![ + id, + format!("IMG_{id}.CR3"), + at, + hash.map(|h| Signature(h).to_stored()) + ], + ) + .unwrap(); + } + cat + } + + #[test] + fn a_pass_records_the_bursts_it_found() { + let cat = seeded(&[ + (1, 1000, Some(0xFF00)), + (2, 1001, Some(0xFF00)), + (3, 2000, Some(0xFF00)), + ]); + let report = regroup(cat.connection(), Rules::default()).unwrap(); + assert_eq!(report.bursts, 1); + assert_eq!(report.frames, 2); + assert_eq!(report.largest, 2); + + let m = memberships(cat.connection(), &[ImageId(1), ImageId(2), ImageId(3)]).unwrap(); + assert_eq!(m.len(), 2, "the lone frame should have no row"); + assert_eq!(m[&ImageId(1)].burst_id, ImageId(1)); + assert!(m[&ImageId(1)].representative); + assert!(!m[&ImageId(2)].representative); + assert_eq!(m[&ImageId(2)].size, 2); + } + + #[test] + fn a_second_pass_replaces_the_first() { + let cat = seeded(&[(1, 1000, Some(0xFF00)), (2, 1001, Some(0xFF00))]); + regroup(cat.connection(), Rules::default()).unwrap(); + // The frames turn out to be seconds apart after all — an EXIF sweep + // corrected the timestamp. + cat.connection() + .execute("UPDATE images SET captured_at = 3000 WHERE id = 2", []) + .unwrap(); + let report = regroup(cat.connection(), Rules::default()).unwrap(); + assert_eq!(report.bursts, 0); + assert!(memberships(cat.connection(), &[ImageId(1), ImageId(2)]) + .unwrap() + .is_empty()); + } + + #[test] + fn the_representative_is_the_earliest_frame_and_not_a_judgement() { + let cat = seeded(&[ + (7, 1000, Some(0xFF00)), + (8, 1000, Some(0xFF00)), + (9, 1001, Some(0xFF00)), + ]); + regroup(cat.connection(), Rules::default()).unwrap(); + let m = memberships(cat.connection(), &[ImageId(7), ImageId(8), ImageId(9)]).unwrap(); + assert!(m[&ImageId(7)].representative); + assert!(!m[&ImageId(8)].representative); + assert!(!m[&ImageId(9)].representative); + } + + #[test] + fn the_users_choice_of_representative_survives_a_regroup() { + // The same guarantee `people.ignored` has, for the same reason: a pass + // that forgot the decision would ask for it again after every scan. + let cat = seeded(&[ + (1, 1000, Some(0xFF00)), + (2, 1001, Some(0xFF00)), + (3, 1002, Some(0xFF00)), + ]); + regroup(cat.connection(), Rules::default()).unwrap(); + choose_representative(cat.connection(), ImageId(2)).unwrap(); + + let m = memberships(cat.connection(), &[ImageId(2)]).unwrap(); + assert!(m[&ImageId(2)].representative, "the pick did not take effect"); + + regroup(cat.connection(), Rules::default()).unwrap(); + let m = memberships(cat.connection(), &[ImageId(1), ImageId(2)]).unwrap(); + assert!(m[&ImageId(2)].representative, "the regroup forgot the pick"); + assert!(!m[&ImageId(1)].representative); + } + + #[test] + fn choosing_a_representative_outside_a_burst_does_nothing() { + let cat = seeded(&[(1, 1000, Some(0xFF00))]); + regroup(cat.connection(), Rules::default()).unwrap(); + choose_representative(cat.connection(), ImageId(1)).unwrap(); + let n: i64 = cat + .connection() + .query_row("SELECT count(*) FROM burst_pick", [], |r| r.get(0)) + .unwrap(); + assert_eq!(n, 0); + } + + #[test] + fn a_newly_found_burst_arrives_open() { + // The pass must not take rows off the screen. It marks them; the user + // folds them. + let cat = seeded(&[(1, 1000, Some(0xFF00)), (2, 1001, Some(0xFF00))]); + regroup(cat.connection(), Rules::default()).unwrap(); + assert!(is_expanded(cat.connection(), ImageId(1)).unwrap()); + } + + #[test] + fn a_burst_the_user_folded_up_stays_folded_through_the_next_pass() { + // Otherwise every import springs open a morning's culling. + let cat = seeded(&[ + (1, 1000, Some(0xFF00)), + (2, 1001, Some(0xFF00)), + (3, 9000, Some(0x00FF)), + ]); + regroup(cat.connection(), Rules::default()).unwrap(); + set_expanded(cat.connection(), ImageId(1), false).unwrap(); + + // A frame arrives elsewhere in the library, so the pass runs again. + cat.connection() + .execute( + "INSERT INTO images(id, root_id, source_ref, captured_at, camera, + perceptual_hash, added_at) + VALUES (4, 1, 'IMG_4.CR3', 9001, 'Canon EOS R5', ?1, 0)", + [Signature(0x00FF).to_stored()], + ) + .unwrap(); + regroup(cat.connection(), Rules::default()).unwrap(); + + assert!(!is_expanded(cat.connection(), ImageId(1)).unwrap()); + assert!( + is_expanded(cat.connection(), ImageId(3)).unwrap(), + "the burst nobody has seen yet should have arrived open" + ); + } + + #[test] + fn expansion_is_remembered_and_reversible() { + let cat = seeded(&[(1, 1000, Some(0xFF00)), (2, 1001, Some(0xFF00))]); + regroup(cat.connection(), Rules::default()).unwrap(); + + set_expanded(cat.connection(), ImageId(1), false).unwrap(); + assert!(!is_expanded(cat.connection(), ImageId(1)).unwrap()); + set_expanded(cat.connection(), ImageId(1), true).unwrap(); + assert!(is_expanded(cat.connection(), ImageId(1)).unwrap()); + // Idempotent: the UI toggles this from a callback that can fire twice. + set_expanded(cat.connection(), ImageId(1), true).unwrap(); + assert_eq!(collapse_all(cat.connection()).unwrap(), 1); + assert!(!is_expanded(cat.connection(), ImageId(1)).unwrap()); + } + + #[test] + fn a_burst_that_no_longer_exists_leaves_no_expansion_behind() { + // The id is an image id, so a stale row would eventually re-open some + // unrelated group. + let cat = seeded(&[(1, 1000, Some(0xFF00)), (2, 1001, Some(0xFF00))]); + regroup(cat.connection(), Rules::default()).unwrap(); + assert!(is_expanded(cat.connection(), ImageId(1)).unwrap()); + + cat.connection() + .execute("UPDATE images SET captured_at = 9000 WHERE id = 2", []) + .unwrap(); + regroup(cat.connection(), Rules::default()).unwrap(); + + assert!(!is_expanded(cat.connection(), ImageId(1)).unwrap()); + } + + #[test] + fn a_collapsed_burst_hides_everything_but_its_representative() { + let cat = seeded(&[ + (1, 1000, Some(0xFF00)), + (2, 1001, Some(0xFF00)), + (3, 1002, Some(0xFF00)), + (4, 9000, Some(0xFF00)), + ]); + regroup(cat.connection(), Rules::default()).unwrap(); + + let sql = format!( + "SELECT id FROM images i WHERE {} ORDER BY id", + not_collapsed_away("i") + ); + let visible = |c: &Connection| -> Vec { + let mut stmt = c.prepare(&sql).unwrap(); + let rows = stmt.query_map([], |r| r.get::<_, i64>(0)).unwrap(); + rows.collect::, _>>().unwrap() + }; + + // Open, as a new burst always is: nothing has been taken away. + assert_eq!(visible(cat.connection()), vec![1, 2, 3, 4]); + set_expanded(cat.connection(), ImageId(1), false).unwrap(); + assert_eq!(visible(cat.connection()), vec![1, 4]); + set_expanded(cat.connection(), ImageId(1), true).unwrap(); + assert_eq!(visible(cat.connection()), vec![1, 2, 3, 4]); + } + + #[test] + fn a_library_with_no_bursts_hides_nothing() { + // The predicate is in every grid query, so its cost and its effect on a + // library that has never been grouped both matter. + let cat = seeded(&[(1, 1000, Some(0xFF00)), (2, 9000, Some(0xFF00))]); + let sql = format!( + "SELECT count(*) FROM images i WHERE {}", + not_collapsed_away("i") + ); + let n: i64 = cat.connection().query_row(&sql, [], |r| r.get(0)).unwrap(); + assert_eq!(n, 2); + } + + #[test] + fn an_unhashed_image_is_offered_for_hashing_once() { + let cat = seeded(&[(1, 1000, None), (2, 1001, Some(0xFF00))]); + assert_eq!( + images_without_signature(cat.connection()).unwrap(), + vec![ImageId(1)] + ); + set_signature(cat.connection(), ImageId(1), Signature(0x1234)).unwrap(); + assert!(images_without_signature(cat.connection()) + .unwrap() + .is_empty()); + assert_eq!( + signature(cat.connection(), ImageId(1)).unwrap(), + Some(Signature(0x1234)) + ); + } + + #[test] + fn hashing_an_image_the_scan_has_dropped_is_not_an_error() { + let cat = seeded(&[(1, 1000, None)]); + assert_eq!( + set_signature(cat.connection(), ImageId(404), Signature(1)).unwrap(), + 0 + ); + } + + #[test] + fn a_trashed_or_shadowed_frame_is_not_part_of_a_burst() { + // The same two exclusions the grid makes. A shadowed JPEG beside its RAW + // would otherwise be a two-frame burst with the frame it *is*. + let cat = seeded(&[ + (1, 1000, Some(0xFF00)), + (2, 1000, Some(0xFF00)), + (3, 1001, Some(0xFF00)), + ]); + cat.connection() + .execute("UPDATE images SET shadowed_by = 1 WHERE id = 2", []) + .unwrap(); + cat.connection() + .execute("UPDATE images SET trashed_at = 99 WHERE id = 3", []) + .unwrap(); + let report = regroup(cat.connection(), Rules::default()).unwrap(); + assert_eq!(report.bursts, 0); + } + + #[test] + fn members_come_back_in_the_order_the_grid_lists_them() { + let cat = seeded(&[ + (5, 1002, Some(0xFF00)), + (6, 1000, Some(0xFF00)), + (7, 1001, Some(0xFF00)), + ]); + regroup(cat.connection(), Rules::default()).unwrap(); + assert_eq!( + members(cat.connection(), ImageId(6)).unwrap(), + vec![ImageId(6), ImageId(7), ImageId(5)] + ); + } +} diff --git a/core/dr-catalog/src/lib.rs b/core/dr-catalog/src/lib.rs index c99c94c..de71325 100644 --- a/core/dr-catalog/src/lib.rs +++ b/core/dr-catalog/src/lib.rs @@ -16,6 +16,7 @@ //! - [`collections`] — the collection tree and membership the UI edits //! - [`keywords`] — the keyword vocabulary and what it is assigned to //! - [`faces`] — detected faces, the people they belong to, and who said so +//! - [`bursts`] — frames that are one moment, grouped so they judge as one //! - [`jobs`] — the durable background work queue //! - [`trash`] — soft delete to a folder, then permanent delete //! - [`merge`] / [`sync`] — cross-device merging of collections and keywords @@ -33,6 +34,7 @@ use std::path::Path; use dr_types::{Availability, ImageId}; use rusqlite::Connection; +pub mod bursts; pub mod cache; pub mod collections; pub mod dedup; diff --git a/core/dr-catalog/src/schema.rs b/core/dr-catalog/src/schema.rs index 98094f9..c621c37 100644 --- a/core/dr-catalog/src/schema.rs +++ b/core/dr-catalog/src/schema.rs @@ -15,7 +15,7 @@ use rusqlite::Connection; use crate::error::CatalogError; /// Schema version this build writes and understands. -pub const SCHEMA_VERSION: i64 = 10; +pub const SCHEMA_VERSION: i64 = 11; /// Apply migrations up to [`SCHEMA_VERSION`]. /// @@ -98,6 +98,13 @@ pub fn migrate(conn: &Connection) -> Result { tx.commit()?; } + if from < 11 { + let tx = conn.unchecked_transaction()?; + tx.execute_batch(V11)?; + tx.pragma_update(None, "user_version", 11)?; + tx.commit()?; + } + Ok(from) } @@ -205,6 +212,11 @@ pub fn v1_for_attached(schema_name: &str) -> String { /// creating it over there would fail on columns that are not there. Nothing is /// lost by its absence — it exists to make the *grid* page quickly, and the /// grid never reads across an attachment. +/// +/// V11 is excluded on the same grounds and for the plainer reason that a merge +/// has nothing to do with it: burst grouping is rebuilt locally from local +/// signatures, and no code reads a remote catalog's `burst_*` tables. Its +/// `ALTER TABLE` would fail here anyway, being unqualifiable by the rewrite. pub fn for_attached(schema_name: &str) -> String { // V10 is `ALTER TABLE`, which the textual rewrite cannot qualify, so its // columns are spelled out. A remote genuinely older than V10 is a real @@ -397,6 +409,73 @@ ALTER TABLE people ADD COLUMN ignored INTEGER NOT NULL DEFAULT 0; ALTER TABLE faces ADD COLUMN crop BLOB; "#; +/// TRACES: FR-CULL-5 +/// Burst grouping: which frames are one moment, and which one stands for it. +/// +/// The reasoning behind the grouping itself is in [`crate::bursts`]; what +/// belongs here is why it is stored in three pieces rather than one. +/// +/// **`images.perceptual_hash` is a column, not a table**, for the same reason +/// `content_hash` is: it is one number per image, NULL until something has had +/// the pixels in hand, and every query that wants it is already reading the +/// image row. It is local derived state — a rebuilt catalog recomputes it from +/// thumbnails — which is also why it is absent from [`for_attached`], alongside +/// the shadowing and trashing columns V2 through V5 add. +/// +/// **`burst_members` is rewritten whole by every pass.** No id of its own: the +/// group is named by the image id of its earliest frame, so a burst that has not +/// changed keeps its name across a regroup and the interface can remember that +/// this one is open. There is no `bursts` table to go with it because a group +/// has no properties beyond its members — inventing a row for it would create an +/// identity that survives the grouping being rebuilt, which is precisely what +/// must not happen. +/// +/// **`burst_pick` is the one thing here that is not derived**, and it is a +/// separate table so that rewriting the grouping cannot erase it. A +/// representative stored on `burst_members` would be forgotten every time a +/// frame arrived; the user would be asked the same question after every scan. +/// The same argument `people.ignored` makes in V10, one subsystem over. +/// +/// **`burst_expanded` is view state in the catalog**, which is unusual enough to +/// justify. The grid is a window over an ordered query — `LIMIT n OFFSET k` — +/// so what a collapsed burst hides has to be decided by the query, or the row +/// count stops agreeing with the scrollbar and the ordinals a scrub resolves to. +/// Once SQL has to see it, this is where it lives. Nothing else reads it, and it +/// is emptied of stale groups by every pass. +const V11: &str = r#" +-- A 64-bit perceptual signature. Local derived state: NULL until something has +-- decoded the image, recomputed from thumbnails if the catalog is rebuilt, and +-- comparable only to signatures produced by the same build (`bursts`). +ALTER TABLE images ADD COLUMN perceptual_hash INTEGER; + +CREATE TABLE burst_members ( + -- One burst at most per image: a frame belongs to the moment it was taken + -- in, and nothing else. + image_id INTEGER PRIMARY KEY REFERENCES images(id) ON DELETE CASCADE, + -- The image id of the burst's earliest frame. Not a foreign key by + -- accident: the leader is itself a member, so this genuinely references + -- images(id), and cascading its deletion is right. + burst_id INTEGER NOT NULL REFERENCES images(id) ON DELETE CASCADE, + -- The frame the group collapses to. Exactly one per burst. + representative INTEGER NOT NULL DEFAULT 0 +); +-- Counting a burst's frames and listing them are what the grid asks for, once +-- per window; without this both are a scan of every grouped frame in the +-- library. +CREATE INDEX burst_members_burst ON burst_members(burst_id); + +-- The user's own choice of representative. User data, never rewritten by a +-- grouping pass -- see the module doc above. +CREATE TABLE burst_pick ( + image_id INTEGER PRIMARY KEY REFERENCES images(id) ON DELETE CASCADE +); + +-- Bursts the grid is currently showing in full. +CREATE TABLE burst_expanded ( + burst_id INTEGER PRIMARY KEY REFERENCES images(id) ON DELETE CASCADE +); +"#; + const V9: &str = r#" -- TRACES: FR-CULL-8 -- A record that face detection has *run* on an image, distinct from what it diff --git a/docs/catalog.md b/docs/catalog.md index 1268bd3..5d1dee8 100644 --- a/docs/catalog.md +++ b/docs/catalog.md @@ -810,6 +810,60 @@ confidence is unavailable. It does not fall back to an untuned default dressed u --- +## 10a. Bursts and near-duplicates + +Specified by FR-CULL-5, implemented in `dr_catalog::bursts` (a v11 migration) with the pass that +feeds it in `dr_ui::bursts`. + +A burst is a run of frames that are **adjacent in time and look like the frame before them**. Both +halves are load-bearing. Time alone groups a whole wedding ceremony, because a photographer working +steadily never leaves the gap that would end the run. Similarity alone groups a studio setup shot +across two days, which is a project rather than a moment. The bounds are two seconds and eight bits +of a 64-bit difference hash, and the reasoning for each figure is in the module. + +**Two seconds, for a burst that fires ten frames in one.** `images.captured_at` is whole seconds: +EXIF's `DateTimeOriginal` has no sub-second field, and `SubSecTimeOriginal` is optional and widely +omitted. Ten frames of a burst therefore arrive sharing a timestamp, and any threshold finer than a +second is a threshold on information the catalog does not have. Where the pace really is faster than +two seconds, the similarity bound is what separates the frames. + +**The signal is a perceptual hash of the thumbnail, not of the original.** `images.perceptual_hash` +is filled from the 256px thumbnails §7 already stores — vastly more resolution than a 9×8 reduction +uses — so a library that has been browsed has already paid for its signatures and no RAW is decoded +for this. The consequence is stated rather than hidden: an image with no thumbnail gets no +signature, and a frame with no signature never joins a burst. It is picked up by the next pass. + +**It is a pass, not a job kind**, for exactly the reason §10.2 gives for face clustering: a burst is +a property of a *run* of frames and has no natural `subject_id`, so a per-image job would rebuild +the world once per photograph. It runs when the thumbnail sweep finishes, which is the first moment +the signatures can all be computed. + +**A newly found burst arrives open.** The pass marks frames; it never takes them off the screen. +Collapsing on discovery would be tidier and would also mean a background pass removing photographs +from under someone part way through a cull. Folding a burst up is the user's act, it is remembered +(`burst_expanded`), and a burst that is already known keeps whatever state it is in — so the pass +that follows the next import does not spring open a morning's work. + +**Nothing here ranks a frame.** The representative of a collapsed burst is its *earliest* frame, +which is a fact about the clock rather than a judgement about the photograph. FR-CULL-5 names the +failure this avoids — rejecting the only frame of an important moment because someone blinked — and +the only judgement in the subsystem is the user's own choice of representative, which lives in its +own table (`burst_pick`) so that rebuilding the grouping cannot erase it. Same argument as +`people.ignored` in §10. + +**What the collapse costs the grid.** Which rows a collapsed burst hides has to be decided by the +query rather than by the cells, because the grid is a window (`LIMIT n OFFSET k`) and the frames it +hides are mostly not loaded. So the predicate joins `VISIBLE` in every query that lists or counts +cells, under the same discipline: present in four places of five, the header's count, the +scrollbar, the shift-click range and the scrub's ordinal stop describing the same list. + +**What this does not settle.** Bursts are local: the tables ride along in the uploaded catalog +snapshot and nothing on the far side reads them, so a second device rebuilds its own grouping from +its own signatures. Making `burst_pick` cross-device is a merge question of the same shape as §8.4's +and is not answered here. + +--- + ## 11. Requirements touched | ID | How this document addresses it | @@ -828,6 +882,7 @@ confidence is unavailable. It does not fall back to an untuned default dressed u | NFR-P1 | §3.1 one stat per directory, not per file | | NFR-P3 | §7.1 on-demand generation | | NFR-ARCH-2 | §6.3 priority classes shared with the GPU scheduler | +| FR-CULL-5 | §10a burst grouping: capture-time proximity and image similarity, collapse without selection | | NFR-ARCH-3 | §4.3 query cancellation, §6 job cancellation | | NFR-RES-4 | §7.3 LRU cap, eviction order | | FR-CULL-8 | §10.1 `faces` schema, §6.1 `DetectFaces` job kind on the proxy tier |