//! TRACES: FR-CULL-5 //! Bursts: frames that are one moment, grouped so they can be judged as one. //! //! A burst is the commonest thing in a cull and the least interesting. Twelve //! frames of the same gull, shot at 10 fps, occupy twelve cells of the grid, //! are scrolled past twelve times, and end with the photographer keeping one. //! Collapsing them to a single cell with a count on it is the whole feature: //! the twelve are still there, still reachable, still individually ratable — //! they simply stop costing twelve decisions where one was meant. //! //! # What this deliberately does not do //! //! **Nothing here ranks a frame.** FR-CULL-5 names the failure it is avoiding: //! automated *selection* is distrusted because the documented way it fails is //! rejecting the only frame of an important moment because somebody blinked. //! So there is no sharpness score, no eye detector, no "best of burst". The //! representative of a group is the **earliest frame**, which is a fact about //! the clock and not a judgement about the photograph, and the user can name a //! different one at any time ([`choose_representative`]). //! //! That is also why this sits beside [`crate::dedup`] rather than inside it. //! Dedup answers "are these the same file?" with a whole-file digest, for //! import collision; two bytes of difference make two different answers, which //! is exactly right there and useless here. A burst is a set of frames that are //! *deliberately* different — the wing is in another place in each one. //! //! # Two signals, and why neither alone will do //! //! **Time alone** groups a wedding ceremony into one burst. Frames arrive //! seconds apart for an hour, and a photographer shooting steadily never //! produces the gap that would end the run. //! //! **Similarity alone** groups the same subject photographed on two different //! days — a studio setup, a copy stand, a hundred frames of the same document. //! Those are not a burst, they are a project, and collapsing them hides work //! the user did on purpose. //! //! Together they are specific: *adjacent in time* **and** *looks like the frame //! before it*. Both bounds are in [`Rules`]. //! //! # Chained, not compared to the first frame //! //! Each frame is tested against the one immediately before it, and a burst is a //! run of frames that each joined the last. Comparing every frame to the *first* //! would split a pan: by frame twenty a camera following a bird has nothing in //! common with frame one, and yet no two adjacent frames in that sequence differ //! by much. Chaining follows the subject; a fixed anchor loses it half way. //! //! The accepted consequence is that the first and last frames of a long burst //! may be quite unlike each other. That is what a burst *is*, and the time bound //! is what stops the chain running away: a sequence that has drifted a long way //! has almost always paused somewhere, and the pause ends the run. //! //! # The similarity signal is a 64-bit difference hash //! //! [`Signature`] is a dHash taken on a 9×8 box-averaged reduction of the image: //! 64 comparisons of a cell against its right-hand neighbour, one bit each. It //! is scale-independent, survives JPEG artefacts and mild exposure changes, //! costs nothing to store, and compares in one `xor` and a `popcount` — which //! matters, because the grouping pass is a whole-library operation. //! //! What it is bad at is worth stating plainly: **a featureless frame hashes to //! zero**, and so do all the other featureless frames. A lens cap, a black //! frame, a white wall and an overexposed sky are mutually indistinguishable to //! it. The capture-time bound is what keeps that from grouping every mistake in //! the library into one enormous burst, and it is sufficient in practice — but //! it is the reason this is grouping and not deduplication. //! //! # Where the pixels come from, and why the hash is stored //! //! This module never decodes an image. It takes a luma or RGBA buffer somebody //! else already had in hand ([`signature_of_luma`], [`signature_of_rgba`]) and //! `images.perceptual_hash` keeps the answer, the same bargain //! [`crate::dedup::set_content_hash`] makes: whoever is already holding the //! pixels pays nothing, and everybody afterwards pays nothing at all. In //! practice the caller is the thumbnail store — a 256px thumbnail is far more //! resolution than a 9×8 reduction needs — so a library that has been browsed //! has already paid for its signatures. //! //! # A pass, not a job //! //! Grouping has no natural `subject_id`: it is a property of a *run* of frames, //! so a per-image job would rebuild the world once per photograph. It is //! therefore a debounced library-level pass, for exactly the reasons //! docs/catalog.md §10.2 gives for face clustering, and [`regroup`] is the whole //! of it — one ordered walk, no per-pair comparison beyond adjacent frames. //! //! # Grouping is not hiding //! //! A burst this pass has never seen before is recorded **open**: the grid keeps //! every row it had, and all that appears is a mark saying how many frames each //! run holds. Only the user folds one up. That is a deliberate refusal of the //! obvious default — collapsing on discovery would be tidier and would also //! mean a background pass removing photographs from under somebody part way //! through a cull, which is the same class of surprise FR-CULL-5 is written to //! avoid. A burst that is already known keeps whatever state it is in, so a //! pass after the next import does not spring open a morning's work. //! //! # What is stored, and what a merge does with it //! //! `burst_members` is derived: deleting it costs one pass and loses nothing. //! `burst_pick` is not — it is the user saying which frame stands for the //! moment, and it lives in its own table precisely so [`regroup`] can rewrite //! the grouping without erasing the judgement. That is the same argument //! `people.ignored` makes one subsystem over: nothing short of remembering a //! decision survives re-clustering. //! //! Both tables travel in the uploaded catalog snapshot, and nothing on the far //! side reads them — [`crate::sync`] merges collections and keywords only — so //! a second device rebuilds its own grouping from its own signatures. Bursts //! are not yet cross-device state, and pretending otherwise would mean syncing //! `burst_pick` as user data, which is a merge question this does not answer. use std::collections::{HashMap, HashSet}; use dr_types::ImageId; use rusqlite::Connection; use crate::CatalogError; /// A 64-bit perceptual signature of one image. /// /// Comparable only to other signatures from this build: the reduction grid and /// the bit order are part of the definition, and changing either silently /// changes what "similar" means. If that ever happens the column has to be /// cleared, the way a face `model_id` change forces a re-index. #[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord)] pub struct Signature(pub u64); impl Signature { /// Differing bits, 0..=64. Small is similar. pub fn distance(self, other: Signature) -> u32 { (self.0 ^ other.0).count_ones() } /// The stored form. SQLite integers are signed, so the bits are reinterpreted /// rather than converted — `as` in both directions is exact and lossless. pub fn to_stored(self) -> i64 { self.0 as i64 } /// The inverse of [`Self::to_stored`]. pub fn from_stored(v: i64) -> Self { Signature(v as u64) } } /// Columns in the reduction grid. One more than the number of comparisons per /// row, because each bit compares a cell against its right-hand neighbour. const GRID_W: usize = 9; /// Rows in the reduction grid. `(GRID_W - 1) * GRID_H` is 64 — the width of the /// signature, and the reason these two numbers are what they are. const GRID_H: usize = 8; /// Reduce a greyscale buffer to a [`Signature`]. /// /// `luma` is row-major, one byte a pixel, `width * height` of them. Returns /// `None` for an empty or short buffer rather than panicking: the caller is /// usually handing over the result of a decode, and a truncated JPEG is a /// normal thing to find in a library rather than a programming error. /// /// # Why box-averaged rather than sampled /// /// Point-sampling a 9×8 grid out of a 256px thumbnail reads 72 pixels of /// several hundred thousand, and two frames of a burst that differ by one pixel /// of camera shake can sample entirely different detail. Averaging the whole /// cell is what makes the signature stable under the small movements a burst is /// made of — which is the property the whole feature rests on. pub fn signature_of_luma(luma: &[u8], width: u32, height: u32) -> Option { let (w, h) = (width as usize, height as usize); if w == 0 || h == 0 || luma.len() < w * h { return None; } let mut cells = [0f32; GRID_W * GRID_H]; for gy in 0..GRID_H { // Cell bounds by proportion, so every source pixel lands in exactly one // cell whatever the aspect ratio. The `max` keeps a cell non-empty when // the source is narrower or shorter than the grid — a 4px-wide preview // is degenerate but must not divide by zero. let y0 = gy * h / GRID_H; let y1 = (((gy + 1) * h) / GRID_H).max(y0 + 1).min(h); for gx in 0..GRID_W { let x0 = gx * w / GRID_W; let x1 = (((gx + 1) * w) / GRID_W).max(x0 + 1).min(w); let mut sum = 0u32; let mut n = 0u32; for y in y0..y1 { let row = &luma[y * w..y * w + w]; for px in &row[x0..x1] { sum += *px as u32; n += 1; } } cells[gy * GRID_W + gx] = sum as f32 / n as f32; } } // Each bit: is this cell brighter than the one to its right? A *difference* // rather than a level, which is what makes the signature indifferent to the // exposure drifting a third of a stop through a burst. let mut bits = 0u64; for gy in 0..GRID_H { for gx in 0..GRID_W - 1 { bits <<= 1; if cells[gy * GRID_W + gx] > cells[gy * GRID_W + gx + 1] { bits |= 1; } } } Some(Signature(bits)) } /// [`signature_of_luma`] for the packed RGBA a decoded thumbnail arrives as. /// /// Alpha is ignored: a thumbnail is opaque, and a signature that changed with /// it would make two renderings of one frame look like two photographs. pub fn signature_of_rgba(rgba: &[u8], width: u32, height: u32) -> Option { let (w, h) = (width as usize, height as usize); if w == 0 || h == 0 || rgba.len() < w * h * 4 { return None; } // Rec. 601 luma in fixed point. The exact weights matter less than their // being the same weights every time — see [`Signature`]. let luma: Vec = rgba .chunks_exact(4) .take(w * h) .map(|p| ((77 * p[0] as u32 + 150 * p[1] as u32 + 29 * p[2] as u32) >> 8) as u8) .collect(); signature_of_luma(&luma, width, height) } /// The two bounds that decide what counts as one burst. /// /// Held together in a struct rather than passed as two numbers so a caller /// cannot supply one and default the other, and so the pair can be logged as /// what a given grouping was produced under. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub struct Rules { /// Longest gap, in seconds, between two frames of one burst. pub max_gap: i64, /// Largest [`Signature::distance`] between two frames of one burst. pub max_distance: u32, } impl Default for Rules { /// # Why two seconds, when a burst fires ten frames in one /// /// Because `images.captured_at` is **whole seconds**. EXIF `DateTimeOriginal` /// has no sub-second field — `SubSecTimeOriginal` is a separate, optional tag /// that plenty of bodies omit — so a 10 fps burst arrives in the catalog as /// ten frames sharing one timestamp, and any threshold below a second is a /// threshold on information that is not there. /// /// Two seconds is therefore the smallest bound that can distinguish /// anything: it holds a burst together across the second boundary it will /// certainly straddle, and it is short enough that a photographer working at /// a considered pace — a frame every three or four seconds — is never /// grouped. Where the pace really is faster than that, the similarity bound /// is what separates the frames. /// /// # Why eight bits of sixty-four /// /// A difference hash of two frames of one burst typically differs by nought /// to four bits; two unrelated photographs differ by twenty to thirty-two, /// with thirty-two being the expected distance between two *random* /// signatures. Eight sits in the empty ground between those, near enough to /// the burst end of it that a subject change inside one second — the case /// FR-CULL-5's failure mode is really about — does not merge. /// /// Erring low is the right direction: a burst left ungrouped costs the user /// a scroll, and two moments wrongly merged hide a photograph behind a cell /// that does not look like it. fn default() -> Self { Rules { max_gap: 2, max_distance: 8, } } } /// One frame as the grouping pass sees it. /// /// Deliberately not a catalog row: the whole decision is made from these four /// fields, which is what lets [`group`] be tested against constructed frames /// with no database at all. #[derive(Debug, Clone, PartialEq, Eq)] pub struct Frame { pub id: ImageId, /// UTC seconds. A frame without one cannot be placed in a run and is never /// read by the pass. pub captured_at: i64, /// The joined make-and-model string, as the scan stores it. pub camera: Option, /// `None` until something has hashed this image's pixels. pub signature: Option, } /// Whether `b` continues the burst that `a` is part of. /// /// `b` is assumed not to precede `a`. fn continues(a: &Frame, b: &Frame, rules: Rules) -> bool { if b.captured_at - a.captured_at > rules.max_gap { return false; } // Two bodies firing at the same instant are two photographs of one moment, // not one burst — a second shooter at a wedding, or a camera on a tripod // triggered alongside the one in your hands. Where either camera is unknown // the frames are not separated on that account: an unread EXIF header is // absence of evidence, and the similarity bound still has to be satisfied. if let (Some(x), Some(y)) = (&a.camera, &b.camera) { if x != y { return false; } } // A missing signature never groups. It would be easy to fall back to time // alone here and it would be wrong: an unhashed library would collapse // every steadily-shot sequence in it, and the user would have no way to see // that the reason was a missing hash rather than a resemblance. match (a.signature, b.signature) { (Some(p), Some(q)) => p.distance(q) <= rules.max_distance, _ => false, } } /// Group frames into bursts. The whole of the algorithm. /// /// Returns only groups of two or more, each ordered by capture time, in the /// order the bursts themselves begin. A lone frame is not a burst and gets no /// row anywhere: "is this image in a burst" is then answerable as "does it have /// a `burst_members` row", and the grid pays nothing for the overwhelming /// majority of a library that is not bursts. /// /// Sorts its own input. The caller usually reads frames in capture order /// anyway, but the run structure is only meaningful in that order and a /// mis-ordered caller would get silently wrong groups rather than an error. pub fn group(frames: &[Frame], rules: Rules) -> Vec> { let mut ordered: Vec<&Frame> = frames.iter().collect(); // Ties on the second broken by id, which is the order the frames were // catalogued in and therefore the order the camera wrote them. It is the // same tiebreak the grid uses, so a burst is a contiguous run on screen. ordered.sort_by_key(|f| (f.captured_at, f.id.0)); let mut out: Vec> = Vec::new(); let mut run: Vec = Vec::new(); for pair in ordered.windows(2) { let (a, b) = (pair[0], pair[1]); if continues(a, b, rules) { if run.is_empty() { run.push(a.id); } run.push(b.id); } else if !run.is_empty() { out.push(std::mem::take(&mut run)); } } if !run.is_empty() { out.push(run); } out } /// What one [`regroup`] did, for the log and for a progress line. #[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] pub struct Report { /// Groups of two or more frames. pub bursts: usize, /// Frames inside them. Never the size of the library. pub frames: usize, /// Frames in the largest single burst. pub largest: usize, } /// Rebuild the whole library's grouping. /// /// One ordered read of the dated, visible images, one pass over it, one /// transaction. The cost is linear in library size and the comparison count is /// one per *adjacent pair* — there is no all-pairs step here and there must /// never be one, because that is what turns a grouping pass into a job nobody /// can afford to run. /// /// Existing membership is replaced rather than amended. Amending would need to /// know which frames had changed since the last pass, and the answer is /// "possibly any of them, because a signature arrives long after the row does"; /// a full rebuild is a few milliseconds and cannot drift. pub fn regroup(conn: &Connection, rules: Rules) -> Result { let frames = read_frames(conn)?; let groups = group(&frames, rules); let picked = user_picks(conn)?; let known = known_bursts(conn)?; let tx = conn.unchecked_transaction()?; tx.execute("DELETE FROM burst_members", [])?; { let mut insert = tx.prepare( "INSERT INTO burst_members(image_id, burst_id, representative) VALUES (?1, ?2, ?3)", )?; for members in &groups { // The earliest frame's own id names the burst. It is stable across // a regroup that leaves the burst alone, which is what lets the UI // remember that this group is open; it is not a durable identity, // and a burst that gains an earlier frame is a new group as far as // anything keyed on this is concerned. let burst_id = members[0].0 as i64; let representative = members .iter() .find(|id| picked.contains(*id)) .copied() .unwrap_or(members[0]); for id in members { insert.execute(rusqlite::params![ id.0 as i64, burst_id, (*id == representative) as i64 ])?; } // **A burst the user has never seen arrives open.** Nothing is // hidden by this pass running: the grid holds exactly the // photographs it held before, and the only new thing is a mark on // each burst saying how many frames it is part of. Folding one up // is then something the user did, which is the difference between // a tool that groups and a tool that decides — and it is the only // arrangement in which a background pass cannot make a row // disappear from under somebody mid-cull. // // A burst that is already known keeps whatever state it is in, so // the pass that runs after the next import does not spring open // everything the user spent the morning folding away. if !known.contains(&burst_id) { tx.execute( "INSERT OR IGNORE INTO burst_expanded(burst_id) VALUES (?1)", [burst_id], )?; } } } // A group that no longer exists must not leave an expansion behind: the id // is an image id, and the next pass could hand it to a different burst. tx.execute( "DELETE FROM burst_expanded WHERE burst_id NOT IN (SELECT burst_id FROM burst_members)", [], )?; tx.commit()?; Ok(Report { bursts: groups.len(), frames: groups.iter().map(|g| g.len()).sum(), largest: groups.iter().map(|g| g.len()).max().unwrap_or(0), }) } /// The frames a grouping pass considers, in capture order. /// /// The same two exclusions the grid makes, for the same two reasons: a shadowed /// JPEG is its RAW's frame rather than a second photograph, and a trashed image /// is not in the library. Undated frames are excluded because a burst is a /// statement about time and there is nothing to say about a frame with none. fn read_frames(conn: &Connection) -> Result, CatalogError> { let mut stmt = conn.prepare( "SELECT id, captured_at, camera, perceptual_hash FROM images WHERE captured_at IS NOT NULL AND shadowed_by IS NULL AND trashed_at IS NULL ORDER BY captured_at ASC, id ASC", )?; let rows = stmt.query_map([], |r| { Ok(Frame { id: ImageId(r.get::<_, i64>(0)? as u64), captured_at: r.get(1)?, camera: r.get(2)?, signature: r.get::<_, Option>(3)?.map(Signature::from_stored), }) })?; Ok(rows.collect::, _>>()?) } /// The bursts the previous pass left behind, so this one can tell a group the /// user has already met from a group that is new. fn known_bursts(conn: &Connection) -> Result, CatalogError> { let mut stmt = conn.prepare("SELECT DISTINCT burst_id FROM burst_members")?; let rows = stmt.query_map([], |r| r.get::<_, i64>(0))?; Ok(rows.collect::, _>>()?) } fn user_picks(conn: &Connection) -> Result, CatalogError> { let mut stmt = conn.prepare("SELECT image_id FROM burst_pick")?; let rows = stmt.query_map([], |r| Ok(ImageId(r.get::<_, i64>(0)? as u64)))?; Ok(rows.collect::, _>>()?) } /// Record the signature of an image somebody has just decoded. /// /// Returns rows updated: zero means the image is no longer in the catalog, /// which is a normal race with a rescan rather than an error — the same answer /// [`crate::dedup::set_content_hash`] gives. pub fn set_signature( conn: &Connection, image: ImageId, signature: Signature, ) -> Result { Ok(conn.execute( "UPDATE images SET perceptual_hash = ?2 WHERE id = ?1", rusqlite::params![image.0 as i64, signature.to_stored()], )?) } /// What [`set_signature`] stored, if anything. pub fn signature(conn: &Connection, image: ImageId) -> Result, CatalogError> { let stored: Option = conn.query_row( "SELECT perceptual_hash FROM images WHERE id = ?1", [image.0 as i64], |r| r.get(0), )?; Ok(stored.map(Signature::from_stored)) } /// Visible, dated images that nothing has hashed yet. /// /// What the pass that fills the column iterates. Dated, because an undated /// frame can never join a burst however well it hashes, and hashing it would be /// work with no possible consequence. pub fn images_without_signature(conn: &Connection) -> Result, CatalogError> { let mut stmt = conn.prepare( "SELECT id FROM images WHERE perceptual_hash IS NULL AND captured_at IS NOT NULL AND shadowed_by IS NULL AND trashed_at IS NULL ORDER BY captured_at ASC, id ASC", )?; let rows = stmt.query_map([], |r| Ok(ImageId(r.get::<_, i64>(0)? as u64)))?; Ok(rows.collect::, _>>()?) } /// What the grid needs to know about one cell's burst. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub struct Membership { /// The group's id, which is also the image id of its earliest frame. pub burst_id: ImageId, /// Frames in the group, this one included. Always two or more. pub size: u32, /// Whether this is the frame the group collapses to. pub representative: bool, /// Whether the group is currently open in the grid. pub expanded: bool, } /// Burst membership for one window of the grid, in a single query. /// /// One statement for the whole window rather than one per cell, which is the /// same discipline the collection badges and the rating counts follow: a grid /// that asks the catalog a question per cell is a grid that stutters while a /// finger is on it. pub fn memberships( conn: &Connection, images: &[ImageId], ) -> Result, CatalogError> { if images.is_empty() { return Ok(HashMap::new()); } // Placeholders are generated from the *count* of ids, never from anything // the user typed. let placeholders = std::iter::repeat_n("?", images.len()) .collect::>() .join(","); let sql = format!( "SELECT bm.image_id, bm.burst_id, bm.representative, (SELECT count(*) FROM burst_members m WHERE m.burst_id = bm.burst_id), EXISTS (SELECT 1 FROM burst_expanded be WHERE be.burst_id = bm.burst_id) FROM burst_members bm WHERE bm.image_id IN ({placeholders})" ); let params: Vec = images .iter() .map(|i| rusqlite::types::Value::Integer(i.0 as i64)) .collect(); let mut stmt = conn.prepare(&sql)?; let rows = stmt.query_map(rusqlite::params_from_iter(params.iter()), |r| { Ok(( ImageId(r.get::<_, i64>(0)? as u64), Membership { burst_id: ImageId(r.get::<_, i64>(1)? as u64), size: r.get::<_, i64>(3)? as u32, representative: r.get::<_, i64>(2)? != 0, expanded: r.get::<_, i64>(4)? != 0, }, )) })?; Ok(rows.collect::, _>>()?) } /// The frames of one burst, in the order the grid lists them. pub fn members(conn: &Connection, burst: ImageId) -> Result, CatalogError> { let mut stmt = conn.prepare( "SELECT bm.image_id FROM burst_members bm JOIN images i ON i.id = bm.image_id WHERE bm.burst_id = ?1 ORDER BY i.captured_at ASC, i.id ASC", )?; let rows = stmt.query_map([burst.0 as i64], |r| { Ok(ImageId(r.get::<_, i64>(0)? as u64)) })?; Ok(rows.collect::, _>>()?) } /// Open or close one burst in the grid. /// /// Kept in the catalog rather than in the interface's own memory for one /// reason: the grid is a *window* over an ordered query, and what is collapsed /// has to be decided by that query or the window's row count stops matching the /// scrollbar. Once the state has to be visible to SQL, the catalog is where it /// lives — and a culling session that survives closing the app is the shape /// FR-CULL-4 already asks for elsewhere. pub fn set_expanded(conn: &Connection, burst: ImageId, expanded: bool) -> Result<(), CatalogError> { if expanded { conn.execute( // `OR IGNORE` rather than an upsert: the toggle is wired to a // callback that can fire twice, and re-opening an open burst must // be a no-op rather than an error. "INSERT OR IGNORE INTO burst_expanded(burst_id) VALUES (?1)", [burst.0 as i64], )?; } else { conn.execute( "DELETE FROM burst_expanded WHERE burst_id = ?1", [burst.0 as i64], )?; } Ok(()) } /// Whether this burst is currently open. pub fn is_expanded(conn: &Connection, burst: ImageId) -> Result { let n: i64 = conn.query_row( "SELECT count(*) FROM burst_expanded WHERE burst_id = ?1", [burst.0 as i64], |r| r.get(0), )?; Ok(n > 0) } /// Close every open burst. Returns how many were open. pub fn collapse_all(conn: &Connection) -> Result { Ok(conn.execute("DELETE FROM burst_expanded", [])?) } /// The user names the frame their burst collapses to. /// /// The one *choice* in this module, and the reason the rest of it makes none. /// Recorded against the image rather than the group so that it survives the /// next [`regroup`]: a group is rebuilt from scratch every pass, and a /// representative stored on it would be forgotten every time a frame arrived. /// /// Any previous pick within the same burst is dropped — a burst collapses to /// one frame, and two picks would make the choice depend on row order. pub fn choose_representative(conn: &Connection, image: ImageId) -> Result<(), CatalogError> { let id = image.0 as i64; let tx = conn.unchecked_transaction()?; // Only meaningful for an image that is actually in a burst; anything else // would leave a pick that no group can ever honour. let burst: Option = tx .query_row( "SELECT burst_id FROM burst_members WHERE image_id = ?1", [id], |r| r.get(0), ) .ok(); let Some(burst) = burst else { return Ok(()); }; tx.execute( "DELETE FROM burst_pick WHERE image_id IN (SELECT image_id FROM burst_members WHERE burst_id = ?1)", [burst], )?; tx.execute("INSERT INTO burst_pick(image_id) VALUES (?1)", [id])?; // The grouping already exists, so the flag it carries is corrected now // rather than at the next pass — the user expects the cell to change under // the pointer, not after a background sweep. tx.execute( "UPDATE burst_members SET representative = (image_id = ?2) WHERE burst_id = ?1", [burst, id], )?; tx.commit()?; Ok(()) } /// SQL for "this row is not a frame a collapsed burst is standing in for". /// /// Handed out as a predicate rather than as a list of ids because the grid is a /// window: the rows a collapsed burst hides are mostly not loaded, so the /// question can only be answered where the ordering and the `LIMIT` are — in /// the query itself. `image` names the table or alias the image row is in, /// because the grid aliases `images` as `i` and other callers do not. /// /// Both subqueries are probes on a primary key, so this costs one index lookup /// per row the query walks and nothing at all for a library with no bursts in /// it, where `burst_members` is empty. /// /// Never interpolate anything user-supplied as `image`. pub fn not_collapsed_away(image: &str) -> String { format!( "NOT EXISTS (SELECT 1 FROM burst_members bm WHERE bm.image_id = {image}.id AND bm.representative = 0 AND NOT EXISTS (SELECT 1 FROM burst_expanded be WHERE be.burst_id = bm.burst_id))" ) } #[cfg(test)] mod tests { use super::*; use crate::Catalog; /// A frame with everything the grouping looks at. fn frame(id: u64, at: i64, sig: u64) -> Frame { Frame { id: ImageId(id), captured_at: at, camera: Some("Canon EOS R5".into()), signature: Some(Signature(sig)), } } // ── the signature ───────────────────────────────────────────────────── /// A blocky pseudo-random scene, deterministic and free of any dependency. /// /// Blocks rather than per-pixel noise, because per-pixel noise averages to a /// flat grey in the 9×8 reduction and every such image would hash alike — /// which would make the tests below pass for the wrong reason. fn scene(w: usize, h: usize, seed: u64, shift: usize, brighter: i32) -> Vec { let block = 8; let mut out = vec![0u8; w * h]; for y in 0..h { for x in 0..w { let bx = (x + shift) / block; let by = y / block; // A small LCG, evaluated per block: reproducible, and nothing // in the workspace has to provide it. let mut s = seed .wrapping_add(bx as u64) .wrapping_mul(6_364_136_223_846_793_005) .wrapping_add((by as u64).wrapping_mul(1_442_695_040_888_963_407)); s ^= s >> 33; let v = (s % 256) as i32 + brighter; out[y * w + x] = v.clamp(0, 255) as u8; } } out } #[test] fn two_frames_of_one_burst_hash_almost_alike() { // The property the whole feature rests on: the same scene, moved by a // pixel and a third of a stop brighter, is the same signature or very // nearly. let a = signature_of_luma(&scene(64, 64, 7, 0, 0), 64, 64).unwrap(); let b = signature_of_luma(&scene(64, 64, 7, 1, 6), 64, 64).unwrap(); assert!( a.distance(b) <= Rules::default().max_distance, "a burst pair differed by {} bits, which the default rules would \ not group", a.distance(b) ); } #[test] fn two_different_subjects_do_not_hash_alike() { let a = signature_of_luma(&scene(64, 64, 7, 0, 0), 64, 64).unwrap(); let b = signature_of_luma(&scene(64, 64, 99, 0, 0), 64, 64).unwrap(); assert!( a.distance(b) > Rules::default().max_distance, "two unrelated scenes differed by only {} bits", a.distance(b) ); } #[test] fn the_signature_does_not_change_with_scale() { // Detection runs against whichever preview tier is in hand, and the // cache is entitled to regenerate one at another size. A signature that // moved with the resolution would silently stop matching its own // library. let big = scene(128, 128, 21, 0, 0); // Nearest-neighbour halving, which is the harshest thing a real // downscale could do to it. let mut small = vec![0u8; 64 * 64]; for y in 0..64 { for x in 0..64 { small[y * 64 + x] = big[(y * 2) * 128 + x * 2]; } } let a = signature_of_luma(&big, 128, 128).unwrap(); let b = signature_of_luma(&small, 64, 64).unwrap(); assert!( a.distance(b) <= Rules::default().max_distance, "halving the image moved {} bits, enough to stop it matching itself", a.distance(b) ); } #[test] fn a_short_or_empty_buffer_is_not_a_signature() { assert!(signature_of_luma(&[], 0, 0).is_none()); assert!(signature_of_luma(&[1, 2, 3], 64, 64).is_none()); assert!(signature_of_rgba(&[1, 2, 3, 255], 64, 64).is_none()); } #[test] fn rgba_and_luma_agree() { let luma = scene(64, 64, 3, 0, 0); let rgba: Vec = luma.iter().flat_map(|v| [*v, *v, *v, 255]).collect(); // Grey is grey: the Rec. 601 weights sum to 256/256, so a neutral pixel // survives the conversion within a rounding step. let a = signature_of_luma(&luma, 64, 64).unwrap(); let b = signature_of_rgba(&rgba, 64, 64).unwrap(); assert!(a.distance(b) <= 2, "{} bits apart", a.distance(b)); } #[test] fn a_signature_survives_the_trip_through_sqlite() { // The top bit is the one at risk: SQLite integers are signed. let s = Signature(u64::MAX); assert_eq!(Signature::from_stored(s.to_stored()), s); assert_eq!( Signature::from_stored(Signature(0).to_stored()), Signature(0) ); } // ── the grouping ────────────────────────────────────────────────────── #[test] fn frames_a_second_apart_and_alike_are_one_burst() { let g = group( &[ frame(1, 1000, 0xFF00), frame(2, 1001, 0xFF00), frame(3, 1001, 0xFF01), ], Rules::default(), ); assert_eq!(g, vec![vec![ImageId(1), ImageId(2), ImageId(3)]]); } #[test] fn a_pause_ends_the_burst() { // The boundary itself: `max_gap` is inclusive, so two seconds continues // a run and three begins a new one. let rules = Rules::default(); let g = group( &[ frame(1, 1000, 0xFF00), frame(2, 1002, 0xFF00), frame(3, 1005, 0xFF00), frame(4, 1006, 0xFF00), ], rules, ); assert_eq!( g, vec![vec![ImageId(1), ImageId(2)], vec![ImageId(3), ImageId(4)],], "the three-second pause did not end the first burst" ); } #[test] fn two_subjects_one_second_apart_are_not_a_burst() { // The failure FR-CULL-5 names, in its mildest form: turning round and // photographing something else does not make the two frames one moment, // however quickly it was done. let bird = signature_of_luma(&scene(64, 64, 11, 0, 0), 64, 64).unwrap(); let sign = signature_of_luma(&scene(64, 64, 404, 0, 0), 64, 64).unwrap(); let g = group( &[ Frame { id: ImageId(1), captured_at: 1000, camera: Some("X-T5".into()), signature: Some(bird), }, Frame { id: ImageId(2), captured_at: 1001, camera: Some("X-T5".into()), signature: Some(sign), }, ], Rules::default(), ); assert!(g.is_empty(), "two different subjects were grouped: {g:?}"); } #[test] fn a_lone_frame_is_not_a_burst() { let g = group(&[frame(1, 1000, 0xAB)], Rules::default()); assert!(g.is_empty()); } #[test] fn a_burst_chains_through_a_pan() { // Frame one and frame four have nothing in common; each frame resembles // the one before it. That is a camera following a bird, and it is one // burst. let rules = Rules::default(); let g = group( &[ frame(1, 1000, 0x0000_0000_0000_0000), frame(2, 1000, 0x0000_0000_0000_00FF), frame(3, 1001, 0x0000_0000_0000_FFFF), frame(4, 1001, 0x0000_0000_00FF_FFFF), ], rules, ); assert_eq!(g.len(), 1, "the pan was split: {g:?}"); assert_eq!(g[0].len(), 4); // And the ends really are far apart — otherwise this test would pass // without exercising the chaining at all. assert!(Signature(0).distance(Signature(0x00FF_FFFF)) > rules.max_distance); } #[test] fn frames_from_two_cameras_at_one_instant_stay_apart() { // A second shooter, or a tethered body beside the one in your hands. let g = group( &[ Frame { id: ImageId(1), captured_at: 1000, camera: Some("Canon EOS R5".into()), signature: Some(Signature(0xFF00)), }, Frame { id: ImageId(2), captured_at: 1000, camera: Some("NIKON Z 9".into()), signature: Some(Signature(0xFF00)), }, ], Rules::default(), ); assert!(g.is_empty(), "two bodies were merged into one burst: {g:?}"); } #[test] fn an_unknown_camera_does_not_split_a_burst() { // metadata_state 1 is a normal resting state for a freshly scanned // library; refusing to group on it would mean bursts appear only after // a full EXIF sweep. let g = group( &[ Frame { id: ImageId(1), captured_at: 1000, camera: None, signature: Some(Signature(0xFF00)), }, Frame { id: ImageId(2), captured_at: 1000, camera: Some("Canon EOS R5".into()), signature: Some(Signature(0xFF00)), }, ], Rules::default(), ); assert_eq!(g.len(), 1); } #[test] fn an_unhashed_frame_never_groups() { // Time alone would put these three together, which is precisely the // wedding-ceremony failure. let g = group( &[ Frame { id: ImageId(1), captured_at: 1000, camera: None, signature: None, }, Frame { id: ImageId(2), captured_at: 1001, camera: None, signature: None, }, Frame { id: ImageId(3), captured_at: 1002, camera: None, signature: None, }, ], Rules::default(), ); assert!(g.is_empty(), "unhashed frames were grouped on time alone"); } #[test] fn input_order_does_not_change_the_answer() { let frames = vec![ frame(3, 1002, 0xFF00), frame(1, 1000, 0xFF00), frame(2, 1001, 0xFF00), ]; let g = group(&frames, Rules::default()); assert_eq!(g, vec![vec![ImageId(1), ImageId(2), ImageId(3)]]); } // ── the pass, against a catalog ─────────────────────────────────────── /// A catalog holding `frames` as (id, captured_at, hash). fn seeded(frames: &[(i64, i64, Option)]) -> Catalog { let cat = Catalog::in_memory().unwrap(); let c = cat.connection(); c.execute( "INSERT INTO roots(id, kind, label) VALUES (1, 'local', 'lib')", [], ) .unwrap(); for (id, at, hash) in frames { c.execute( "INSERT INTO images(id, root_id, source_ref, captured_at, camera, perceptual_hash, added_at) VALUES (?1, 1, ?2, ?3, 'Canon EOS R5', ?4, 0)", rusqlite::params![ id, format!("IMG_{id}.CR3"), at, hash.map(|h| Signature(h).to_stored()) ], ) .unwrap(); } cat } #[test] fn a_pass_records_the_bursts_it_found() { let cat = seeded(&[ (1, 1000, Some(0xFF00)), (2, 1001, Some(0xFF00)), (3, 2000, Some(0xFF00)), ]); let report = regroup(cat.connection(), Rules::default()).unwrap(); assert_eq!(report.bursts, 1); assert_eq!(report.frames, 2); assert_eq!(report.largest, 2); let m = memberships(cat.connection(), &[ImageId(1), ImageId(2), ImageId(3)]).unwrap(); assert_eq!(m.len(), 2, "the lone frame should have no row"); assert_eq!(m[&ImageId(1)].burst_id, ImageId(1)); assert!(m[&ImageId(1)].representative); assert!(!m[&ImageId(2)].representative); assert_eq!(m[&ImageId(2)].size, 2); } #[test] fn a_second_pass_replaces_the_first() { let cat = seeded(&[(1, 1000, Some(0xFF00)), (2, 1001, Some(0xFF00))]); regroup(cat.connection(), Rules::default()).unwrap(); // The frames turn out to be seconds apart after all — an EXIF sweep // corrected the timestamp. cat.connection() .execute("UPDATE images SET captured_at = 3000 WHERE id = 2", []) .unwrap(); let report = regroup(cat.connection(), Rules::default()).unwrap(); assert_eq!(report.bursts, 0); assert!(memberships(cat.connection(), &[ImageId(1), ImageId(2)]) .unwrap() .is_empty()); } #[test] fn the_representative_is_the_earliest_frame_and_not_a_judgement() { let cat = seeded(&[ (7, 1000, Some(0xFF00)), (8, 1000, Some(0xFF00)), (9, 1001, Some(0xFF00)), ]); regroup(cat.connection(), Rules::default()).unwrap(); let m = memberships(cat.connection(), &[ImageId(7), ImageId(8), ImageId(9)]).unwrap(); assert!(m[&ImageId(7)].representative); assert!(!m[&ImageId(8)].representative); assert!(!m[&ImageId(9)].representative); } #[test] fn the_users_choice_of_representative_survives_a_regroup() { // The same guarantee `people.ignored` has, for the same reason: a pass // that forgot the decision would ask for it again after every scan. let cat = seeded(&[ (1, 1000, Some(0xFF00)), (2, 1001, Some(0xFF00)), (3, 1002, Some(0xFF00)), ]); regroup(cat.connection(), Rules::default()).unwrap(); choose_representative(cat.connection(), ImageId(2)).unwrap(); let m = memberships(cat.connection(), &[ImageId(2)]).unwrap(); assert!( m[&ImageId(2)].representative, "the pick did not take effect" ); regroup(cat.connection(), Rules::default()).unwrap(); let m = memberships(cat.connection(), &[ImageId(1), ImageId(2)]).unwrap(); assert!(m[&ImageId(2)].representative, "the regroup forgot the pick"); assert!(!m[&ImageId(1)].representative); } #[test] fn choosing_a_representative_outside_a_burst_does_nothing() { let cat = seeded(&[(1, 1000, Some(0xFF00))]); regroup(cat.connection(), Rules::default()).unwrap(); choose_representative(cat.connection(), ImageId(1)).unwrap(); let n: i64 = cat .connection() .query_row("SELECT count(*) FROM burst_pick", [], |r| r.get(0)) .unwrap(); assert_eq!(n, 0); } #[test] fn a_newly_found_burst_arrives_open() { // The pass must not take rows off the screen. It marks them; the user // folds them. let cat = seeded(&[(1, 1000, Some(0xFF00)), (2, 1001, Some(0xFF00))]); regroup(cat.connection(), Rules::default()).unwrap(); assert!(is_expanded(cat.connection(), ImageId(1)).unwrap()); } #[test] fn a_burst_the_user_folded_up_stays_folded_through_the_next_pass() { // Otherwise every import springs open a morning's culling. let cat = seeded(&[ (1, 1000, Some(0xFF00)), (2, 1001, Some(0xFF00)), (3, 9000, Some(0x00FF)), ]); regroup(cat.connection(), Rules::default()).unwrap(); set_expanded(cat.connection(), ImageId(1), false).unwrap(); // A frame arrives elsewhere in the library, so the pass runs again. cat.connection() .execute( "INSERT INTO images(id, root_id, source_ref, captured_at, camera, perceptual_hash, added_at) VALUES (4, 1, 'IMG_4.CR3', 9001, 'Canon EOS R5', ?1, 0)", [Signature(0x00FF).to_stored()], ) .unwrap(); regroup(cat.connection(), Rules::default()).unwrap(); assert!(!is_expanded(cat.connection(), ImageId(1)).unwrap()); assert!( is_expanded(cat.connection(), ImageId(3)).unwrap(), "the burst nobody has seen yet should have arrived open" ); } #[test] fn expansion_is_remembered_and_reversible() { let cat = seeded(&[(1, 1000, Some(0xFF00)), (2, 1001, Some(0xFF00))]); regroup(cat.connection(), Rules::default()).unwrap(); set_expanded(cat.connection(), ImageId(1), false).unwrap(); assert!(!is_expanded(cat.connection(), ImageId(1)).unwrap()); set_expanded(cat.connection(), ImageId(1), true).unwrap(); assert!(is_expanded(cat.connection(), ImageId(1)).unwrap()); // Idempotent: the UI toggles this from a callback that can fire twice. set_expanded(cat.connection(), ImageId(1), true).unwrap(); assert_eq!(collapse_all(cat.connection()).unwrap(), 1); assert!(!is_expanded(cat.connection(), ImageId(1)).unwrap()); } #[test] fn a_burst_that_no_longer_exists_leaves_no_expansion_behind() { // The id is an image id, so a stale row would eventually re-open some // unrelated group. let cat = seeded(&[(1, 1000, Some(0xFF00)), (2, 1001, Some(0xFF00))]); regroup(cat.connection(), Rules::default()).unwrap(); assert!(is_expanded(cat.connection(), ImageId(1)).unwrap()); cat.connection() .execute("UPDATE images SET captured_at = 9000 WHERE id = 2", []) .unwrap(); regroup(cat.connection(), Rules::default()).unwrap(); assert!(!is_expanded(cat.connection(), ImageId(1)).unwrap()); } #[test] fn a_collapsed_burst_hides_everything_but_its_representative() { let cat = seeded(&[ (1, 1000, Some(0xFF00)), (2, 1001, Some(0xFF00)), (3, 1002, Some(0xFF00)), (4, 9000, Some(0xFF00)), ]); regroup(cat.connection(), Rules::default()).unwrap(); let sql = format!( "SELECT id FROM images i WHERE {} ORDER BY id", not_collapsed_away("i") ); let visible = |c: &Connection| -> Vec { let mut stmt = c.prepare(&sql).unwrap(); let rows = stmt.query_map([], |r| r.get::<_, i64>(0)).unwrap(); rows.collect::, _>>().unwrap() }; // Open, as a new burst always is: nothing has been taken away. assert_eq!(visible(cat.connection()), vec![1, 2, 3, 4]); set_expanded(cat.connection(), ImageId(1), false).unwrap(); assert_eq!(visible(cat.connection()), vec![1, 4]); set_expanded(cat.connection(), ImageId(1), true).unwrap(); assert_eq!(visible(cat.connection()), vec![1, 2, 3, 4]); } #[test] fn a_library_with_no_bursts_hides_nothing() { // The predicate is in every grid query, so its cost and its effect on a // library that has never been grouped both matter. let cat = seeded(&[(1, 1000, Some(0xFF00)), (2, 9000, Some(0xFF00))]); let sql = format!( "SELECT count(*) FROM images i WHERE {}", not_collapsed_away("i") ); let n: i64 = cat.connection().query_row(&sql, [], |r| r.get(0)).unwrap(); assert_eq!(n, 2); } #[test] fn an_unhashed_image_is_offered_for_hashing_once() { let cat = seeded(&[(1, 1000, None), (2, 1001, Some(0xFF00))]); assert_eq!( images_without_signature(cat.connection()).unwrap(), vec![ImageId(1)] ); set_signature(cat.connection(), ImageId(1), Signature(0x1234)).unwrap(); assert!(images_without_signature(cat.connection()) .unwrap() .is_empty()); assert_eq!( signature(cat.connection(), ImageId(1)).unwrap(), Some(Signature(0x1234)) ); } #[test] fn hashing_an_image_the_scan_has_dropped_is_not_an_error() { let cat = seeded(&[(1, 1000, None)]); assert_eq!( set_signature(cat.connection(), ImageId(404), Signature(1)).unwrap(), 0 ); } #[test] fn a_trashed_or_shadowed_frame_is_not_part_of_a_burst() { // The same two exclusions the grid makes. A shadowed JPEG beside its RAW // would otherwise be a two-frame burst with the frame it *is*. let cat = seeded(&[ (1, 1000, Some(0xFF00)), (2, 1000, Some(0xFF00)), (3, 1001, Some(0xFF00)), ]); cat.connection() .execute("UPDATE images SET shadowed_by = 1 WHERE id = 2", []) .unwrap(); cat.connection() .execute("UPDATE images SET trashed_at = 99 WHERE id = 3", []) .unwrap(); let report = regroup(cat.connection(), Rules::default()).unwrap(); assert_eq!(report.bursts, 0); } #[test] fn members_come_back_in_the_order_the_grid_lists_them() { let cat = seeded(&[ (5, 1002, Some(0xFF00)), (6, 1000, Some(0xFF00)), (7, 1001, Some(0xFF00)), ]); regroup(cat.connection(), Rules::default()).unwrap(); assert_eq!( members(cat.connection(), ImageId(6)).unwrap(), vec![ImageId(6), ImageId(7), ImageId(5)] ); } }