diff --git a/core/dr-catalog/src/bursts.rs b/core/dr-catalog/src/bursts.rs new file mode 100644 index 0000000..7402579 --- /dev/null +++ b/core/dr-catalog/src/bursts.rs @@ -0,0 +1,1309 @@ +//! TRACES: FR-CULL-5 +//! Bursts: frames that are one moment, grouped so they can be judged as one. +//! +//! A burst is the commonest thing in a cull and the least interesting. Twelve +//! frames of the same gull, shot at 10 fps, occupy twelve cells of the grid, +//! are scrolled past twelve times, and end with the photographer keeping one. +//! Collapsing them to a single cell with a count on it is the whole feature: +//! the twelve are still there, still reachable, still individually ratable — +//! they simply stop costing twelve decisions where one was meant. +//! +//! # What this deliberately does not do +//! +//! **Nothing here ranks a frame.** FR-CULL-5 names the failure it is avoiding: +//! automated *selection* is distrusted because the documented way it fails is +//! rejecting the only frame of an important moment because somebody blinked. +//! So there is no sharpness score, no eye detector, no "best of burst". The +//! representative of a group is the **earliest frame**, which is a fact about +//! the clock and not a judgement about the photograph, and the user can name a +//! different one at any time ([`choose_representative`]). +//! +//! That is also why this sits beside [`crate::dedup`] rather than inside it. +//! Dedup answers "are these the same file?" with a whole-file digest, for +//! import collision; two bytes of difference make two different answers, which +//! is exactly right there and useless here. A burst is a set of frames that are +//! *deliberately* different — the wing is in another place in each one. +//! +//! # Two signals, and why neither alone will do +//! +//! **Time alone** groups a wedding ceremony into one burst. Frames arrive +//! seconds apart for an hour, and a photographer shooting steadily never +//! produces the gap that would end the run. +//! +//! **Similarity alone** groups the same subject photographed on two different +//! days — a studio setup, a copy stand, a hundred frames of the same document. +//! Those are not a burst, they are a project, and collapsing them hides work +//! the user did on purpose. +//! +//! Together they are specific: *adjacent in time* **and** *looks like the frame +//! before it*. Both bounds are in [`Rules`]. +//! +//! # Chained, not compared to the first frame +//! +//! Each frame is tested against the one immediately before it, and a burst is a +//! run of frames that each joined the last. Comparing every frame to the *first* +//! would split a pan: by frame twenty a camera following a bird has nothing in +//! common with frame one, and yet no two adjacent frames in that sequence differ +//! by much. Chaining follows the subject; a fixed anchor loses it half way. +//! +//! The accepted consequence is that the first and last frames of a long burst +//! may be quite unlike each other. That is what a burst *is*, and the time bound +//! is what stops the chain running away: a sequence that has drifted a long way +//! has almost always paused somewhere, and the pause ends the run. +//! +//! # The similarity signal is a 64-bit difference hash +//! +//! [`Signature`] is a dHash taken on a 9×8 box-averaged reduction of the image: +//! 64 comparisons of a cell against its right-hand neighbour, one bit each. It +//! is scale-independent, survives JPEG artefacts and mild exposure changes, +//! costs nothing to store, and compares in one `xor` and a `popcount` — which +//! matters, because the grouping pass is a whole-library operation. +//! +//! What it is bad at is worth stating plainly: **a featureless frame hashes to +//! zero**, and so do all the other featureless frames. A lens cap, a black +//! frame, a white wall and an overexposed sky are mutually indistinguishable to +//! it. The capture-time bound is what keeps that from grouping every mistake in +//! the library into one enormous burst, and it is sufficient in practice — but +//! it is the reason this is grouping and not deduplication. +//! +//! # Where the pixels come from, and why the hash is stored +//! +//! This module never decodes an image. It takes a luma or RGBA buffer somebody +//! else already had in hand ([`signature_of_luma`], [`signature_of_rgba`]) and +//! `images.perceptual_hash` keeps the answer, the same bargain +//! [`crate::dedup::set_content_hash`] makes: whoever is already holding the +//! pixels pays nothing, and everybody afterwards pays nothing at all. In +//! practice the caller is the thumbnail store — a 256px thumbnail is far more +//! resolution than a 9×8 reduction needs — so a library that has been browsed +//! has already paid for its signatures. +//! +//! # A pass, not a job +//! +//! Grouping has no natural `subject_id`: it is a property of a *run* of frames, +//! so a per-image job would rebuild the world once per photograph. It is +//! therefore a debounced library-level pass, for exactly the reasons +//! docs/catalog.md §10.2 gives for face clustering, and [`regroup`] is the whole +//! of it — one ordered walk, no per-pair comparison beyond adjacent frames. +//! +//! # Grouping is not hiding +//! +//! A burst this pass has never seen before is recorded **open**: the grid keeps +//! every row it had, and all that appears is a mark saying how many frames each +//! run holds. Only the user folds one up. That is a deliberate refusal of the +//! obvious default — collapsing on discovery would be tidier and would also +//! mean a background pass removing photographs from under somebody part way +//! through a cull, which is the same class of surprise FR-CULL-5 is written to +//! avoid. A burst that is already known keeps whatever state it is in, so a +//! pass after the next import does not spring open a morning's work. +//! +//! # What is stored, and what a merge does with it +//! +//! `burst_members` is derived: deleting it costs one pass and loses nothing. +//! `burst_pick` is not — it is the user saying which frame stands for the +//! moment, and it lives in its own table precisely so [`regroup`] can rewrite +//! the grouping without erasing the judgement. That is the same argument +//! `people.ignored` makes one subsystem over: nothing short of remembering a +//! decision survives re-clustering. +//! +//! Both tables travel in the uploaded catalog snapshot, and nothing on the far +//! side reads them — [`crate::sync`] merges collections and keywords only — so +//! a second device rebuilds its own grouping from its own signatures. Bursts +//! are not yet cross-device state, and pretending otherwise would mean syncing +//! `burst_pick` as user data, which is a merge question this does not answer. + +use std::collections::{HashMap, HashSet}; + +use dr_types::ImageId; +use rusqlite::Connection; + +use crate::CatalogError; + +/// A 64-bit perceptual signature of one image. +/// +/// Comparable only to other signatures from this build: the reduction grid and +/// the bit order are part of the definition, and changing either silently +/// changes what "similar" means. If that ever happens the column has to be +/// cleared, the way a face `model_id` change forces a re-index. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord)] +pub struct Signature(pub u64); + +impl Signature { + /// Differing bits, 0..=64. Small is similar. + pub fn distance(self, other: Signature) -> u32 { + (self.0 ^ other.0).count_ones() + } + + /// The stored form. SQLite integers are signed, so the bits are reinterpreted + /// rather than converted — `as` in both directions is exact and lossless. + pub fn to_stored(self) -> i64 { + self.0 as i64 + } + + /// The inverse of [`Self::to_stored`]. + pub fn from_stored(v: i64) -> Self { + Signature(v as u64) + } +} + +/// Columns in the reduction grid. One more than the number of comparisons per +/// row, because each bit compares a cell against its right-hand neighbour. +const GRID_W: usize = 9; +/// Rows in the reduction grid. `(GRID_W - 1) * GRID_H` is 64 — the width of the +/// signature, and the reason these two numbers are what they are. +const GRID_H: usize = 8; + +/// Reduce a greyscale buffer to a [`Signature`]. +/// +/// `luma` is row-major, one byte a pixel, `width * height` of them. Returns +/// `None` for an empty or short buffer rather than panicking: the caller is +/// usually handing over the result of a decode, and a truncated JPEG is a +/// normal thing to find in a library rather than a programming error. +/// +/// # Why box-averaged rather than sampled +/// +/// Point-sampling a 9×8 grid out of a 256px thumbnail reads 72 pixels of +/// several hundred thousand, and two frames of a burst that differ by one pixel +/// of camera shake can sample entirely different detail. Averaging the whole +/// cell is what makes the signature stable under the small movements a burst is +/// made of — which is the property the whole feature rests on. +pub fn signature_of_luma(luma: &[u8], width: u32, height: u32) -> Option { + let (w, h) = (width as usize, height as usize); + if w == 0 || h == 0 || luma.len() < w * h { + return None; + } + + let mut cells = [0f32; GRID_W * GRID_H]; + for gy in 0..GRID_H { + // Cell bounds by proportion, so every source pixel lands in exactly one + // cell whatever the aspect ratio. The `max` keeps a cell non-empty when + // the source is narrower or shorter than the grid — a 4px-wide preview + // is degenerate but must not divide by zero. + let y0 = gy * h / GRID_H; + let y1 = (((gy + 1) * h) / GRID_H).max(y0 + 1).min(h); + for gx in 0..GRID_W { + let x0 = gx * w / GRID_W; + let x1 = (((gx + 1) * w) / GRID_W).max(x0 + 1).min(w); + + let mut sum = 0u32; + let mut n = 0u32; + for y in y0..y1 { + let row = &luma[y * w..y * w + w]; + for px in &row[x0..x1] { + sum += *px as u32; + n += 1; + } + } + cells[gy * GRID_W + gx] = sum as f32 / n as f32; + } + } + + // Each bit: is this cell brighter than the one to its right? A *difference* + // rather than a level, which is what makes the signature indifferent to the + // exposure drifting a third of a stop through a burst. + let mut bits = 0u64; + for gy in 0..GRID_H { + for gx in 0..GRID_W - 1 { + bits <<= 1; + if cells[gy * GRID_W + gx] > cells[gy * GRID_W + gx + 1] { + bits |= 1; + } + } + } + Some(Signature(bits)) +} + +/// [`signature_of_luma`] for the packed RGBA a decoded thumbnail arrives as. +/// +/// Alpha is ignored: a thumbnail is opaque, and a signature that changed with +/// it would make two renderings of one frame look like two photographs. +pub fn signature_of_rgba(rgba: &[u8], width: u32, height: u32) -> Option { + let (w, h) = (width as usize, height as usize); + if w == 0 || h == 0 || rgba.len() < w * h * 4 { + return None; + } + // Rec. 601 luma in fixed point. The exact weights matter less than their + // being the same weights every time — see [`Signature`]. + let luma: Vec = rgba + .chunks_exact(4) + .take(w * h) + .map(|p| ((77 * p[0] as u32 + 150 * p[1] as u32 + 29 * p[2] as u32) >> 8) as u8) + .collect(); + signature_of_luma(&luma, width, height) +} + +/// The two bounds that decide what counts as one burst. +/// +/// Held together in a struct rather than passed as two numbers so a caller +/// cannot supply one and default the other, and so the pair can be logged as +/// what a given grouping was produced under. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct Rules { + /// Longest gap, in seconds, between two frames of one burst. + pub max_gap: i64, + /// Largest [`Signature::distance`] between two frames of one burst. + pub max_distance: u32, +} + +impl Default for Rules { + /// # Why two seconds, when a burst fires ten frames in one + /// + /// Because `images.captured_at` is **whole seconds**. EXIF `DateTimeOriginal` + /// has no sub-second field — `SubSecTimeOriginal` is a separate, optional tag + /// that plenty of bodies omit — so a 10 fps burst arrives in the catalog as + /// ten frames sharing one timestamp, and any threshold below a second is a + /// threshold on information that is not there. + /// + /// Two seconds is therefore the smallest bound that can distinguish + /// anything: it holds a burst together across the second boundary it will + /// certainly straddle, and it is short enough that a photographer working at + /// a considered pace — a frame every three or four seconds — is never + /// grouped. Where the pace really is faster than that, the similarity bound + /// is what separates the frames. + /// + /// # Why eight bits of sixty-four + /// + /// A difference hash of two frames of one burst typically differs by nought + /// to four bits; two unrelated photographs differ by twenty to thirty-two, + /// with thirty-two being the expected distance between two *random* + /// signatures. Eight sits in the empty ground between those, near enough to + /// the burst end of it that a subject change inside one second — the case + /// FR-CULL-5's failure mode is really about — does not merge. + /// + /// Erring low is the right direction: a burst left ungrouped costs the user + /// a scroll, and two moments wrongly merged hide a photograph behind a cell + /// that does not look like it. + fn default() -> Self { + Rules { + max_gap: 2, + max_distance: 8, + } + } +} + +/// One frame as the grouping pass sees it. +/// +/// Deliberately not a catalog row: the whole decision is made from these four +/// fields, which is what lets [`group`] be tested against constructed frames +/// with no database at all. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct Frame { + pub id: ImageId, + /// UTC seconds. A frame without one cannot be placed in a run and is never + /// read by the pass. + pub captured_at: i64, + /// The joined make-and-model string, as the scan stores it. + pub camera: Option, + /// `None` until something has hashed this image's pixels. + pub signature: Option, +} + +/// Whether `b` continues the burst that `a` is part of. +/// +/// `b` is assumed not to precede `a`. +fn continues(a: &Frame, b: &Frame, rules: Rules) -> bool { + if b.captured_at - a.captured_at > rules.max_gap { + return false; + } + + // Two bodies firing at the same instant are two photographs of one moment, + // not one burst — a second shooter at a wedding, or a camera on a tripod + // triggered alongside the one in your hands. Where either camera is unknown + // the frames are not separated on that account: an unread EXIF header is + // absence of evidence, and the similarity bound still has to be satisfied. + if let (Some(x), Some(y)) = (&a.camera, &b.camera) { + if x != y { + return false; + } + } + + // A missing signature never groups. It would be easy to fall back to time + // alone here and it would be wrong: an unhashed library would collapse + // every steadily-shot sequence in it, and the user would have no way to see + // that the reason was a missing hash rather than a resemblance. + match (a.signature, b.signature) { + (Some(p), Some(q)) => p.distance(q) <= rules.max_distance, + _ => false, + } +} + +/// Group frames into bursts. The whole of the algorithm. +/// +/// Returns only groups of two or more, each ordered by capture time, in the +/// order the bursts themselves begin. A lone frame is not a burst and gets no +/// row anywhere: "is this image in a burst" is then answerable as "does it have +/// a `burst_members` row", and the grid pays nothing for the overwhelming +/// majority of a library that is not bursts. +/// +/// Sorts its own input. The caller usually reads frames in capture order +/// anyway, but the run structure is only meaningful in that order and a +/// mis-ordered caller would get silently wrong groups rather than an error. +pub fn group(frames: &[Frame], rules: Rules) -> Vec> { + let mut ordered: Vec<&Frame> = frames.iter().collect(); + // Ties on the second broken by id, which is the order the frames were + // catalogued in and therefore the order the camera wrote them. It is the + // same tiebreak the grid uses, so a burst is a contiguous run on screen. + ordered.sort_by_key(|f| (f.captured_at, f.id.0)); + + let mut out: Vec> = Vec::new(); + let mut run: Vec = Vec::new(); + + for pair in ordered.windows(2) { + let (a, b) = (pair[0], pair[1]); + if continues(a, b, rules) { + if run.is_empty() { + run.push(a.id); + } + run.push(b.id); + } else if !run.is_empty() { + out.push(std::mem::take(&mut run)); + } + } + if !run.is_empty() { + out.push(run); + } + out +} + +/// What one [`regroup`] did, for the log and for a progress line. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] +pub struct Report { + /// Groups of two or more frames. + pub bursts: usize, + /// Frames inside them. Never the size of the library. + pub frames: usize, + /// Frames in the largest single burst. + pub largest: usize, +} + +/// Rebuild the whole library's grouping. +/// +/// One ordered read of the dated, visible images, one pass over it, one +/// transaction. The cost is linear in library size and the comparison count is +/// one per *adjacent pair* — there is no all-pairs step here and there must +/// never be one, because that is what turns a grouping pass into a job nobody +/// can afford to run. +/// +/// Existing membership is replaced rather than amended. Amending would need to +/// know which frames had changed since the last pass, and the answer is +/// "possibly any of them, because a signature arrives long after the row does"; +/// a full rebuild is a few milliseconds and cannot drift. +pub fn regroup(conn: &Connection, rules: Rules) -> Result { + let frames = read_frames(conn)?; + let groups = group(&frames, rules); + let picked = user_picks(conn)?; + let known = known_bursts(conn)?; + + let tx = conn.unchecked_transaction()?; + tx.execute("DELETE FROM burst_members", [])?; + { + let mut insert = tx.prepare( + "INSERT INTO burst_members(image_id, burst_id, representative) + VALUES (?1, ?2, ?3)", + )?; + for members in &groups { + // The earliest frame's own id names the burst. It is stable across + // a regroup that leaves the burst alone, which is what lets the UI + // remember that this group is open; it is not a durable identity, + // and a burst that gains an earlier frame is a new group as far as + // anything keyed on this is concerned. + let burst_id = members[0].0 as i64; + let representative = members + .iter() + .find(|id| picked.contains(*id)) + .copied() + .unwrap_or(members[0]); + for id in members { + insert.execute(rusqlite::params![ + id.0 as i64, + burst_id, + (*id == representative) as i64 + ])?; + } + + // **A burst the user has never seen arrives open.** Nothing is + // hidden by this pass running: the grid holds exactly the + // photographs it held before, and the only new thing is a mark on + // each burst saying how many frames it is part of. Folding one up + // is then something the user did, which is the difference between + // a tool that groups and a tool that decides — and it is the only + // arrangement in which a background pass cannot make a row + // disappear from under somebody mid-cull. + // + // A burst that is already known keeps whatever state it is in, so + // the pass that runs after the next import does not spring open + // everything the user spent the morning folding away. + if !known.contains(&burst_id) { + tx.execute( + "INSERT OR IGNORE INTO burst_expanded(burst_id) VALUES (?1)", + [burst_id], + )?; + } + } + } + // A group that no longer exists must not leave an expansion behind: the id + // is an image id, and the next pass could hand it to a different burst. + tx.execute( + "DELETE FROM burst_expanded + WHERE burst_id NOT IN (SELECT burst_id FROM burst_members)", + [], + )?; + tx.commit()?; + + Ok(Report { + bursts: groups.len(), + frames: groups.iter().map(|g| g.len()).sum(), + largest: groups.iter().map(|g| g.len()).max().unwrap_or(0), + }) +} + +/// The frames a grouping pass considers, in capture order. +/// +/// The same two exclusions the grid makes, for the same two reasons: a shadowed +/// JPEG is its RAW's frame rather than a second photograph, and a trashed image +/// is not in the library. Undated frames are excluded because a burst is a +/// statement about time and there is nothing to say about a frame with none. +fn read_frames(conn: &Connection) -> Result, CatalogError> { + let mut stmt = conn.prepare( + "SELECT id, captured_at, camera, perceptual_hash + FROM images + WHERE captured_at IS NOT NULL + AND shadowed_by IS NULL + AND trashed_at IS NULL + ORDER BY captured_at ASC, id ASC", + )?; + let rows = stmt.query_map([], |r| { + Ok(Frame { + id: ImageId(r.get::<_, i64>(0)? as u64), + captured_at: r.get(1)?, + camera: r.get(2)?, + signature: r.get::<_, Option>(3)?.map(Signature::from_stored), + }) + })?; + Ok(rows.collect::, _>>()?) +} + +/// The bursts the previous pass left behind, so this one can tell a group the +/// user has already met from a group that is new. +fn known_bursts(conn: &Connection) -> Result, CatalogError> { + let mut stmt = conn.prepare("SELECT DISTINCT burst_id FROM burst_members")?; + let rows = stmt.query_map([], |r| r.get::<_, i64>(0))?; + Ok(rows.collect::, _>>()?) +} + +fn user_picks(conn: &Connection) -> Result, CatalogError> { + let mut stmt = conn.prepare("SELECT image_id FROM burst_pick")?; + let rows = stmt.query_map([], |r| Ok(ImageId(r.get::<_, i64>(0)? as u64)))?; + Ok(rows.collect::, _>>()?) +} + +/// Record the signature of an image somebody has just decoded. +/// +/// Returns rows updated: zero means the image is no longer in the catalog, +/// which is a normal race with a rescan rather than an error — the same answer +/// [`crate::dedup::set_content_hash`] gives. +pub fn set_signature( + conn: &Connection, + image: ImageId, + signature: Signature, +) -> Result { + Ok(conn.execute( + "UPDATE images SET perceptual_hash = ?2 WHERE id = ?1", + rusqlite::params![image.0 as i64, signature.to_stored()], + )?) +} + +/// What [`set_signature`] stored, if anything. +pub fn signature(conn: &Connection, image: ImageId) -> Result, CatalogError> { + let stored: Option = conn.query_row( + "SELECT perceptual_hash FROM images WHERE id = ?1", + [image.0 as i64], + |r| r.get(0), + )?; + Ok(stored.map(Signature::from_stored)) +} + +/// Visible, dated images that nothing has hashed yet. +/// +/// What the pass that fills the column iterates. Dated, because an undated +/// frame can never join a burst however well it hashes, and hashing it would be +/// work with no possible consequence. +pub fn images_without_signature(conn: &Connection) -> Result, CatalogError> { + let mut stmt = conn.prepare( + "SELECT id FROM images + WHERE perceptual_hash IS NULL + AND captured_at IS NOT NULL + AND shadowed_by IS NULL + AND trashed_at IS NULL + ORDER BY captured_at ASC, id ASC", + )?; + let rows = stmt.query_map([], |r| Ok(ImageId(r.get::<_, i64>(0)? as u64)))?; + Ok(rows.collect::, _>>()?) +} + +/// What the grid needs to know about one cell's burst. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct Membership { + /// The group's id, which is also the image id of its earliest frame. + pub burst_id: ImageId, + /// Frames in the group, this one included. Always two or more. + pub size: u32, + /// Whether this is the frame the group collapses to. + pub representative: bool, + /// Whether the group is currently open in the grid. + pub expanded: bool, +} + +/// Burst membership for one window of the grid, in a single query. +/// +/// One statement for the whole window rather than one per cell, which is the +/// same discipline the collection badges and the rating counts follow: a grid +/// that asks the catalog a question per cell is a grid that stutters while a +/// finger is on it. +pub fn memberships( + conn: &Connection, + images: &[ImageId], +) -> Result, CatalogError> { + if images.is_empty() { + return Ok(HashMap::new()); + } + // Placeholders are generated from the *count* of ids, never from anything + // the user typed. + let placeholders = std::iter::repeat_n("?", images.len()) + .collect::>() + .join(","); + let sql = format!( + "SELECT bm.image_id, + bm.burst_id, + bm.representative, + (SELECT count(*) FROM burst_members m WHERE m.burst_id = bm.burst_id), + EXISTS (SELECT 1 FROM burst_expanded be WHERE be.burst_id = bm.burst_id) + FROM burst_members bm + WHERE bm.image_id IN ({placeholders})" + ); + let params: Vec = images + .iter() + .map(|i| rusqlite::types::Value::Integer(i.0 as i64)) + .collect(); + + let mut stmt = conn.prepare(&sql)?; + let rows = stmt.query_map(rusqlite::params_from_iter(params.iter()), |r| { + Ok(( + ImageId(r.get::<_, i64>(0)? as u64), + Membership { + burst_id: ImageId(r.get::<_, i64>(1)? as u64), + size: r.get::<_, i64>(3)? as u32, + representative: r.get::<_, i64>(2)? != 0, + expanded: r.get::<_, i64>(4)? != 0, + }, + )) + })?; + Ok(rows.collect::, _>>()?) +} + +/// The frames of one burst, in the order the grid lists them. +pub fn members(conn: &Connection, burst: ImageId) -> Result, CatalogError> { + let mut stmt = conn.prepare( + "SELECT bm.image_id + FROM burst_members bm + JOIN images i ON i.id = bm.image_id + WHERE bm.burst_id = ?1 + ORDER BY i.captured_at ASC, i.id ASC", + )?; + let rows = stmt.query_map([burst.0 as i64], |r| { + Ok(ImageId(r.get::<_, i64>(0)? as u64)) + })?; + Ok(rows.collect::, _>>()?) +} + +/// Open or close one burst in the grid. +/// +/// Kept in the catalog rather than in the interface's own memory for one +/// reason: the grid is a *window* over an ordered query, and what is collapsed +/// has to be decided by that query or the window's row count stops matching the +/// scrollbar. Once the state has to be visible to SQL, the catalog is where it +/// lives — and a culling session that survives closing the app is the shape +/// FR-CULL-4 already asks for elsewhere. +pub fn set_expanded(conn: &Connection, burst: ImageId, expanded: bool) -> Result<(), CatalogError> { + if expanded { + conn.execute( + // `OR IGNORE` rather than an upsert: the toggle is wired to a + // callback that can fire twice, and re-opening an open burst must + // be a no-op rather than an error. + "INSERT OR IGNORE INTO burst_expanded(burst_id) VALUES (?1)", + [burst.0 as i64], + )?; + } else { + conn.execute( + "DELETE FROM burst_expanded WHERE burst_id = ?1", + [burst.0 as i64], + )?; + } + Ok(()) +} + +/// Whether this burst is currently open. +pub fn is_expanded(conn: &Connection, burst: ImageId) -> Result { + let n: i64 = conn.query_row( + "SELECT count(*) FROM burst_expanded WHERE burst_id = ?1", + [burst.0 as i64], + |r| r.get(0), + )?; + Ok(n > 0) +} + +/// Close every open burst. Returns how many were open. +pub fn collapse_all(conn: &Connection) -> Result { + Ok(conn.execute("DELETE FROM burst_expanded", [])?) +} + +/// The user names the frame their burst collapses to. +/// +/// The one *choice* in this module, and the reason the rest of it makes none. +/// Recorded against the image rather than the group so that it survives the +/// next [`regroup`]: a group is rebuilt from scratch every pass, and a +/// representative stored on it would be forgotten every time a frame arrived. +/// +/// Any previous pick within the same burst is dropped — a burst collapses to +/// one frame, and two picks would make the choice depend on row order. +pub fn choose_representative(conn: &Connection, image: ImageId) -> Result<(), CatalogError> { + let id = image.0 as i64; + let tx = conn.unchecked_transaction()?; + + // Only meaningful for an image that is actually in a burst; anything else + // would leave a pick that no group can ever honour. + let burst: Option = tx + .query_row( + "SELECT burst_id FROM burst_members WHERE image_id = ?1", + [id], + |r| r.get(0), + ) + .ok(); + let Some(burst) = burst else { + return Ok(()); + }; + + tx.execute( + "DELETE FROM burst_pick + WHERE image_id IN (SELECT image_id FROM burst_members WHERE burst_id = ?1)", + [burst], + )?; + tx.execute("INSERT INTO burst_pick(image_id) VALUES (?1)", [id])?; + // The grouping already exists, so the flag it carries is corrected now + // rather than at the next pass — the user expects the cell to change under + // the pointer, not after a background sweep. + tx.execute( + "UPDATE burst_members SET representative = (image_id = ?2) WHERE burst_id = ?1", + [burst, id], + )?; + tx.commit()?; + Ok(()) +} + +/// SQL for "this row is not a frame a collapsed burst is standing in for". +/// +/// Handed out as a predicate rather than as a list of ids because the grid is a +/// window: the rows a collapsed burst hides are mostly not loaded, so the +/// question can only be answered where the ordering and the `LIMIT` are — in +/// the query itself. `image` names the table or alias the image row is in, +/// because the grid aliases `images` as `i` and other callers do not. +/// +/// Both subqueries are probes on a primary key, so this costs one index lookup +/// per row the query walks and nothing at all for a library with no bursts in +/// it, where `burst_members` is empty. +/// +/// Never interpolate anything user-supplied as `image`. +pub fn not_collapsed_away(image: &str) -> String { + format!( + "NOT EXISTS (SELECT 1 FROM burst_members bm + WHERE bm.image_id = {image}.id + AND bm.representative = 0 + AND NOT EXISTS (SELECT 1 FROM burst_expanded be + WHERE be.burst_id = bm.burst_id))" + ) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::Catalog; + + /// A frame with everything the grouping looks at. + fn frame(id: u64, at: i64, sig: u64) -> Frame { + Frame { + id: ImageId(id), + captured_at: at, + camera: Some("Canon EOS R5".into()), + signature: Some(Signature(sig)), + } + } + + // ── the signature ───────────────────────────────────────────────────── + + /// A blocky pseudo-random scene, deterministic and free of any dependency. + /// + /// Blocks rather than per-pixel noise, because per-pixel noise averages to a + /// flat grey in the 9×8 reduction and every such image would hash alike — + /// which would make the tests below pass for the wrong reason. + fn scene(w: usize, h: usize, seed: u64, shift: usize, brighter: i32) -> Vec { + let block = 8; + let mut out = vec![0u8; w * h]; + for y in 0..h { + for x in 0..w { + let bx = (x + shift) / block; + let by = y / block; + // A small LCG, evaluated per block: reproducible, and nothing + // in the workspace has to provide it. + let mut s = seed + .wrapping_add(bx as u64) + .wrapping_mul(6_364_136_223_846_793_005) + .wrapping_add((by as u64).wrapping_mul(1_442_695_040_888_963_407)); + s ^= s >> 33; + let v = (s % 256) as i32 + brighter; + out[y * w + x] = v.clamp(0, 255) as u8; + } + } + out + } + + #[test] + fn two_frames_of_one_burst_hash_almost_alike() { + // The property the whole feature rests on: the same scene, moved by a + // pixel and a third of a stop brighter, is the same signature or very + // nearly. + let a = signature_of_luma(&scene(64, 64, 7, 0, 0), 64, 64).unwrap(); + let b = signature_of_luma(&scene(64, 64, 7, 1, 6), 64, 64).unwrap(); + assert!( + a.distance(b) <= Rules::default().max_distance, + "a burst pair differed by {} bits, which the default rules would \ + not group", + a.distance(b) + ); + } + + #[test] + fn two_different_subjects_do_not_hash_alike() { + let a = signature_of_luma(&scene(64, 64, 7, 0, 0), 64, 64).unwrap(); + let b = signature_of_luma(&scene(64, 64, 99, 0, 0), 64, 64).unwrap(); + assert!( + a.distance(b) > Rules::default().max_distance, + "two unrelated scenes differed by only {} bits", + a.distance(b) + ); + } + + #[test] + fn the_signature_does_not_change_with_scale() { + // Detection runs against whichever preview tier is in hand, and the + // cache is entitled to regenerate one at another size. A signature that + // moved with the resolution would silently stop matching its own + // library. + let big = scene(128, 128, 21, 0, 0); + // Nearest-neighbour halving, which is the harshest thing a real + // downscale could do to it. + let mut small = vec![0u8; 64 * 64]; + for y in 0..64 { + for x in 0..64 { + small[y * 64 + x] = big[(y * 2) * 128 + x * 2]; + } + } + let a = signature_of_luma(&big, 128, 128).unwrap(); + let b = signature_of_luma(&small, 64, 64).unwrap(); + assert!( + a.distance(b) <= Rules::default().max_distance, + "halving the image moved {} bits, enough to stop it matching itself", + a.distance(b) + ); + } + + #[test] + fn a_short_or_empty_buffer_is_not_a_signature() { + assert!(signature_of_luma(&[], 0, 0).is_none()); + assert!(signature_of_luma(&[1, 2, 3], 64, 64).is_none()); + assert!(signature_of_rgba(&[1, 2, 3, 255], 64, 64).is_none()); + } + + #[test] + fn rgba_and_luma_agree() { + let luma = scene(64, 64, 3, 0, 0); + let rgba: Vec = luma.iter().flat_map(|v| [*v, *v, *v, 255]).collect(); + // Grey is grey: the Rec. 601 weights sum to 256/256, so a neutral pixel + // survives the conversion within a rounding step. + let a = signature_of_luma(&luma, 64, 64).unwrap(); + let b = signature_of_rgba(&rgba, 64, 64).unwrap(); + assert!(a.distance(b) <= 2, "{} bits apart", a.distance(b)); + } + + #[test] + fn a_signature_survives_the_trip_through_sqlite() { + // The top bit is the one at risk: SQLite integers are signed. + let s = Signature(u64::MAX); + assert_eq!(Signature::from_stored(s.to_stored()), s); + assert_eq!( + Signature::from_stored(Signature(0).to_stored()), + Signature(0) + ); + } + + // ── the grouping ────────────────────────────────────────────────────── + + #[test] + fn frames_a_second_apart_and_alike_are_one_burst() { + let g = group( + &[ + frame(1, 1000, 0xFF00), + frame(2, 1001, 0xFF00), + frame(3, 1001, 0xFF01), + ], + Rules::default(), + ); + assert_eq!(g, vec![vec![ImageId(1), ImageId(2), ImageId(3)]]); + } + + #[test] + fn a_pause_ends_the_burst() { + // The boundary itself: `max_gap` is inclusive, so two seconds continues + // a run and three begins a new one. + let rules = Rules::default(); + let g = group( + &[ + frame(1, 1000, 0xFF00), + frame(2, 1002, 0xFF00), + frame(3, 1005, 0xFF00), + frame(4, 1006, 0xFF00), + ], + rules, + ); + assert_eq!( + g, + vec![vec![ImageId(1), ImageId(2)], vec![ImageId(3), ImageId(4)],], + "the three-second pause did not end the first burst" + ); + } + + #[test] + fn two_subjects_one_second_apart_are_not_a_burst() { + // The failure FR-CULL-5 names, in its mildest form: turning round and + // photographing something else does not make the two frames one moment, + // however quickly it was done. + let bird = signature_of_luma(&scene(64, 64, 11, 0, 0), 64, 64).unwrap(); + let sign = signature_of_luma(&scene(64, 64, 404, 0, 0), 64, 64).unwrap(); + let g = group( + &[ + Frame { + id: ImageId(1), + captured_at: 1000, + camera: Some("X-T5".into()), + signature: Some(bird), + }, + Frame { + id: ImageId(2), + captured_at: 1001, + camera: Some("X-T5".into()), + signature: Some(sign), + }, + ], + Rules::default(), + ); + assert!(g.is_empty(), "two different subjects were grouped: {g:?}"); + } + + #[test] + fn a_lone_frame_is_not_a_burst() { + let g = group(&[frame(1, 1000, 0xAB)], Rules::default()); + assert!(g.is_empty()); + } + + #[test] + fn a_burst_chains_through_a_pan() { + // Frame one and frame four have nothing in common; each frame resembles + // the one before it. That is a camera following a bird, and it is one + // burst. + let rules = Rules::default(); + let g = group( + &[ + frame(1, 1000, 0x0000_0000_0000_0000), + frame(2, 1000, 0x0000_0000_0000_00FF), + frame(3, 1001, 0x0000_0000_0000_FFFF), + frame(4, 1001, 0x0000_0000_00FF_FFFF), + ], + rules, + ); + assert_eq!(g.len(), 1, "the pan was split: {g:?}"); + assert_eq!(g[0].len(), 4); + // And the ends really are far apart — otherwise this test would pass + // without exercising the chaining at all. + assert!(Signature(0).distance(Signature(0x00FF_FFFF)) > rules.max_distance); + } + + #[test] + fn frames_from_two_cameras_at_one_instant_stay_apart() { + // A second shooter, or a tethered body beside the one in your hands. + let g = group( + &[ + Frame { + id: ImageId(1), + captured_at: 1000, + camera: Some("Canon EOS R5".into()), + signature: Some(Signature(0xFF00)), + }, + Frame { + id: ImageId(2), + captured_at: 1000, + camera: Some("NIKON Z 9".into()), + signature: Some(Signature(0xFF00)), + }, + ], + Rules::default(), + ); + assert!(g.is_empty(), "two bodies were merged into one burst: {g:?}"); + } + + #[test] + fn an_unknown_camera_does_not_split_a_burst() { + // metadata_state 1 is a normal resting state for a freshly scanned + // library; refusing to group on it would mean bursts appear only after + // a full EXIF sweep. + let g = group( + &[ + Frame { + id: ImageId(1), + captured_at: 1000, + camera: None, + signature: Some(Signature(0xFF00)), + }, + Frame { + id: ImageId(2), + captured_at: 1000, + camera: Some("Canon EOS R5".into()), + signature: Some(Signature(0xFF00)), + }, + ], + Rules::default(), + ); + assert_eq!(g.len(), 1); + } + + #[test] + fn an_unhashed_frame_never_groups() { + // Time alone would put these three together, which is precisely the + // wedding-ceremony failure. + let g = group( + &[ + Frame { + id: ImageId(1), + captured_at: 1000, + camera: None, + signature: None, + }, + Frame { + id: ImageId(2), + captured_at: 1001, + camera: None, + signature: None, + }, + Frame { + id: ImageId(3), + captured_at: 1002, + camera: None, + signature: None, + }, + ], + Rules::default(), + ); + assert!(g.is_empty(), "unhashed frames were grouped on time alone"); + } + + #[test] + fn input_order_does_not_change_the_answer() { + let frames = vec![ + frame(3, 1002, 0xFF00), + frame(1, 1000, 0xFF00), + frame(2, 1001, 0xFF00), + ]; + let g = group(&frames, Rules::default()); + assert_eq!(g, vec![vec![ImageId(1), ImageId(2), ImageId(3)]]); + } + + // ── the pass, against a catalog ─────────────────────────────────────── + + /// A catalog holding `frames` as (id, captured_at, hash). + fn seeded(frames: &[(i64, i64, Option)]) -> Catalog { + let cat = Catalog::in_memory().unwrap(); + let c = cat.connection(); + c.execute( + "INSERT INTO roots(id, kind, label) VALUES (1, 'local', 'lib')", + [], + ) + .unwrap(); + for (id, at, hash) in frames { + c.execute( + "INSERT INTO images(id, root_id, source_ref, captured_at, camera, + perceptual_hash, added_at) + VALUES (?1, 1, ?2, ?3, 'Canon EOS R5', ?4, 0)", + rusqlite::params![ + id, + format!("IMG_{id}.CR3"), + at, + hash.map(|h| Signature(h).to_stored()) + ], + ) + .unwrap(); + } + cat + } + + #[test] + fn a_pass_records_the_bursts_it_found() { + let cat = seeded(&[ + (1, 1000, Some(0xFF00)), + (2, 1001, Some(0xFF00)), + (3, 2000, Some(0xFF00)), + ]); + let report = regroup(cat.connection(), Rules::default()).unwrap(); + assert_eq!(report.bursts, 1); + assert_eq!(report.frames, 2); + assert_eq!(report.largest, 2); + + let m = memberships(cat.connection(), &[ImageId(1), ImageId(2), ImageId(3)]).unwrap(); + assert_eq!(m.len(), 2, "the lone frame should have no row"); + assert_eq!(m[&ImageId(1)].burst_id, ImageId(1)); + assert!(m[&ImageId(1)].representative); + assert!(!m[&ImageId(2)].representative); + assert_eq!(m[&ImageId(2)].size, 2); + } + + #[test] + fn a_second_pass_replaces_the_first() { + let cat = seeded(&[(1, 1000, Some(0xFF00)), (2, 1001, Some(0xFF00))]); + regroup(cat.connection(), Rules::default()).unwrap(); + // The frames turn out to be seconds apart after all — an EXIF sweep + // corrected the timestamp. + cat.connection() + .execute("UPDATE images SET captured_at = 3000 WHERE id = 2", []) + .unwrap(); + let report = regroup(cat.connection(), Rules::default()).unwrap(); + assert_eq!(report.bursts, 0); + assert!(memberships(cat.connection(), &[ImageId(1), ImageId(2)]) + .unwrap() + .is_empty()); + } + + #[test] + fn the_representative_is_the_earliest_frame_and_not_a_judgement() { + let cat = seeded(&[ + (7, 1000, Some(0xFF00)), + (8, 1000, Some(0xFF00)), + (9, 1001, Some(0xFF00)), + ]); + regroup(cat.connection(), Rules::default()).unwrap(); + let m = memberships(cat.connection(), &[ImageId(7), ImageId(8), ImageId(9)]).unwrap(); + assert!(m[&ImageId(7)].representative); + assert!(!m[&ImageId(8)].representative); + assert!(!m[&ImageId(9)].representative); + } + + #[test] + fn the_users_choice_of_representative_survives_a_regroup() { + // The same guarantee `people.ignored` has, for the same reason: a pass + // that forgot the decision would ask for it again after every scan. + let cat = seeded(&[ + (1, 1000, Some(0xFF00)), + (2, 1001, Some(0xFF00)), + (3, 1002, Some(0xFF00)), + ]); + regroup(cat.connection(), Rules::default()).unwrap(); + choose_representative(cat.connection(), ImageId(2)).unwrap(); + + let m = memberships(cat.connection(), &[ImageId(2)]).unwrap(); + assert!( + m[&ImageId(2)].representative, + "the pick did not take effect" + ); + + regroup(cat.connection(), Rules::default()).unwrap(); + let m = memberships(cat.connection(), &[ImageId(1), ImageId(2)]).unwrap(); + assert!(m[&ImageId(2)].representative, "the regroup forgot the pick"); + assert!(!m[&ImageId(1)].representative); + } + + #[test] + fn choosing_a_representative_outside_a_burst_does_nothing() { + let cat = seeded(&[(1, 1000, Some(0xFF00))]); + regroup(cat.connection(), Rules::default()).unwrap(); + choose_representative(cat.connection(), ImageId(1)).unwrap(); + let n: i64 = cat + .connection() + .query_row("SELECT count(*) FROM burst_pick", [], |r| r.get(0)) + .unwrap(); + assert_eq!(n, 0); + } + + #[test] + fn a_newly_found_burst_arrives_open() { + // The pass must not take rows off the screen. It marks them; the user + // folds them. + let cat = seeded(&[(1, 1000, Some(0xFF00)), (2, 1001, Some(0xFF00))]); + regroup(cat.connection(), Rules::default()).unwrap(); + assert!(is_expanded(cat.connection(), ImageId(1)).unwrap()); + } + + #[test] + fn a_burst_the_user_folded_up_stays_folded_through_the_next_pass() { + // Otherwise every import springs open a morning's culling. + let cat = seeded(&[ + (1, 1000, Some(0xFF00)), + (2, 1001, Some(0xFF00)), + (3, 9000, Some(0x00FF)), + ]); + regroup(cat.connection(), Rules::default()).unwrap(); + set_expanded(cat.connection(), ImageId(1), false).unwrap(); + + // A frame arrives elsewhere in the library, so the pass runs again. + cat.connection() + .execute( + "INSERT INTO images(id, root_id, source_ref, captured_at, camera, + perceptual_hash, added_at) + VALUES (4, 1, 'IMG_4.CR3', 9001, 'Canon EOS R5', ?1, 0)", + [Signature(0x00FF).to_stored()], + ) + .unwrap(); + regroup(cat.connection(), Rules::default()).unwrap(); + + assert!(!is_expanded(cat.connection(), ImageId(1)).unwrap()); + assert!( + is_expanded(cat.connection(), ImageId(3)).unwrap(), + "the burst nobody has seen yet should have arrived open" + ); + } + + #[test] + fn expansion_is_remembered_and_reversible() { + let cat = seeded(&[(1, 1000, Some(0xFF00)), (2, 1001, Some(0xFF00))]); + regroup(cat.connection(), Rules::default()).unwrap(); + + set_expanded(cat.connection(), ImageId(1), false).unwrap(); + assert!(!is_expanded(cat.connection(), ImageId(1)).unwrap()); + set_expanded(cat.connection(), ImageId(1), true).unwrap(); + assert!(is_expanded(cat.connection(), ImageId(1)).unwrap()); + // Idempotent: the UI toggles this from a callback that can fire twice. + set_expanded(cat.connection(), ImageId(1), true).unwrap(); + assert_eq!(collapse_all(cat.connection()).unwrap(), 1); + assert!(!is_expanded(cat.connection(), ImageId(1)).unwrap()); + } + + #[test] + fn a_burst_that_no_longer_exists_leaves_no_expansion_behind() { + // The id is an image id, so a stale row would eventually re-open some + // unrelated group. + let cat = seeded(&[(1, 1000, Some(0xFF00)), (2, 1001, Some(0xFF00))]); + regroup(cat.connection(), Rules::default()).unwrap(); + assert!(is_expanded(cat.connection(), ImageId(1)).unwrap()); + + cat.connection() + .execute("UPDATE images SET captured_at = 9000 WHERE id = 2", []) + .unwrap(); + regroup(cat.connection(), Rules::default()).unwrap(); + + assert!(!is_expanded(cat.connection(), ImageId(1)).unwrap()); + } + + #[test] + fn a_collapsed_burst_hides_everything_but_its_representative() { + let cat = seeded(&[ + (1, 1000, Some(0xFF00)), + (2, 1001, Some(0xFF00)), + (3, 1002, Some(0xFF00)), + (4, 9000, Some(0xFF00)), + ]); + regroup(cat.connection(), Rules::default()).unwrap(); + + let sql = format!( + "SELECT id FROM images i WHERE {} ORDER BY id", + not_collapsed_away("i") + ); + let visible = |c: &Connection| -> Vec { + let mut stmt = c.prepare(&sql).unwrap(); + let rows = stmt.query_map([], |r| r.get::<_, i64>(0)).unwrap(); + rows.collect::, _>>().unwrap() + }; + + // Open, as a new burst always is: nothing has been taken away. + assert_eq!(visible(cat.connection()), vec![1, 2, 3, 4]); + set_expanded(cat.connection(), ImageId(1), false).unwrap(); + assert_eq!(visible(cat.connection()), vec![1, 4]); + set_expanded(cat.connection(), ImageId(1), true).unwrap(); + assert_eq!(visible(cat.connection()), vec![1, 2, 3, 4]); + } + + #[test] + fn a_library_with_no_bursts_hides_nothing() { + // The predicate is in every grid query, so its cost and its effect on a + // library that has never been grouped both matter. + let cat = seeded(&[(1, 1000, Some(0xFF00)), (2, 9000, Some(0xFF00))]); + let sql = format!( + "SELECT count(*) FROM images i WHERE {}", + not_collapsed_away("i") + ); + let n: i64 = cat.connection().query_row(&sql, [], |r| r.get(0)).unwrap(); + assert_eq!(n, 2); + } + + #[test] + fn an_unhashed_image_is_offered_for_hashing_once() { + let cat = seeded(&[(1, 1000, None), (2, 1001, Some(0xFF00))]); + assert_eq!( + images_without_signature(cat.connection()).unwrap(), + vec![ImageId(1)] + ); + set_signature(cat.connection(), ImageId(1), Signature(0x1234)).unwrap(); + assert!(images_without_signature(cat.connection()) + .unwrap() + .is_empty()); + assert_eq!( + signature(cat.connection(), ImageId(1)).unwrap(), + Some(Signature(0x1234)) + ); + } + + #[test] + fn hashing_an_image_the_scan_has_dropped_is_not_an_error() { + let cat = seeded(&[(1, 1000, None)]); + assert_eq!( + set_signature(cat.connection(), ImageId(404), Signature(1)).unwrap(), + 0 + ); + } + + #[test] + fn a_trashed_or_shadowed_frame_is_not_part_of_a_burst() { + // The same two exclusions the grid makes. A shadowed JPEG beside its RAW + // would otherwise be a two-frame burst with the frame it *is*. + let cat = seeded(&[ + (1, 1000, Some(0xFF00)), + (2, 1000, Some(0xFF00)), + (3, 1001, Some(0xFF00)), + ]); + cat.connection() + .execute("UPDATE images SET shadowed_by = 1 WHERE id = 2", []) + .unwrap(); + cat.connection() + .execute("UPDATE images SET trashed_at = 99 WHERE id = 3", []) + .unwrap(); + let report = regroup(cat.connection(), Rules::default()).unwrap(); + assert_eq!(report.bursts, 0); + } + + #[test] + fn members_come_back_in_the_order_the_grid_lists_them() { + let cat = seeded(&[ + (5, 1002, Some(0xFF00)), + (6, 1000, Some(0xFF00)), + (7, 1001, Some(0xFF00)), + ]); + regroup(cat.connection(), Rules::default()).unwrap(); + assert_eq!( + members(cat.connection(), ImageId(6)).unwrap(), + vec![ImageId(6), ImageId(7), ImageId(5)] + ); + } +} diff --git a/core/dr-catalog/src/lib.rs b/core/dr-catalog/src/lib.rs index c99c94c..de71325 100644 --- a/core/dr-catalog/src/lib.rs +++ b/core/dr-catalog/src/lib.rs @@ -16,6 +16,7 @@ //! - [`collections`] — the collection tree and membership the UI edits //! - [`keywords`] — the keyword vocabulary and what it is assigned to //! - [`faces`] — detected faces, the people they belong to, and who said so +//! - [`bursts`] — frames that are one moment, grouped so they judge as one //! - [`jobs`] — the durable background work queue //! - [`trash`] — soft delete to a folder, then permanent delete //! - [`merge`] / [`sync`] — cross-device merging of collections and keywords @@ -33,6 +34,7 @@ use std::path::Path; use dr_types::{Availability, ImageId}; use rusqlite::Connection; +pub mod bursts; pub mod cache; pub mod collections; pub mod dedup; diff --git a/core/dr-catalog/src/schema.rs b/core/dr-catalog/src/schema.rs index 98094f9..c621c37 100644 --- a/core/dr-catalog/src/schema.rs +++ b/core/dr-catalog/src/schema.rs @@ -15,7 +15,7 @@ use rusqlite::Connection; use crate::error::CatalogError; /// Schema version this build writes and understands. -pub const SCHEMA_VERSION: i64 = 10; +pub const SCHEMA_VERSION: i64 = 11; /// Apply migrations up to [`SCHEMA_VERSION`]. /// @@ -98,6 +98,13 @@ pub fn migrate(conn: &Connection) -> Result { tx.commit()?; } + if from < 11 { + let tx = conn.unchecked_transaction()?; + tx.execute_batch(V11)?; + tx.pragma_update(None, "user_version", 11)?; + tx.commit()?; + } + Ok(from) } @@ -205,6 +212,11 @@ pub fn v1_for_attached(schema_name: &str) -> String { /// creating it over there would fail on columns that are not there. Nothing is /// lost by its absence — it exists to make the *grid* page quickly, and the /// grid never reads across an attachment. +/// +/// V11 is excluded on the same grounds and for the plainer reason that a merge +/// has nothing to do with it: burst grouping is rebuilt locally from local +/// signatures, and no code reads a remote catalog's `burst_*` tables. Its +/// `ALTER TABLE` would fail here anyway, being unqualifiable by the rewrite. pub fn for_attached(schema_name: &str) -> String { // V10 is `ALTER TABLE`, which the textual rewrite cannot qualify, so its // columns are spelled out. A remote genuinely older than V10 is a real @@ -397,6 +409,73 @@ ALTER TABLE people ADD COLUMN ignored INTEGER NOT NULL DEFAULT 0; ALTER TABLE faces ADD COLUMN crop BLOB; "#; +/// TRACES: FR-CULL-5 +/// Burst grouping: which frames are one moment, and which one stands for it. +/// +/// The reasoning behind the grouping itself is in [`crate::bursts`]; what +/// belongs here is why it is stored in three pieces rather than one. +/// +/// **`images.perceptual_hash` is a column, not a table**, for the same reason +/// `content_hash` is: it is one number per image, NULL until something has had +/// the pixels in hand, and every query that wants it is already reading the +/// image row. It is local derived state — a rebuilt catalog recomputes it from +/// thumbnails — which is also why it is absent from [`for_attached`], alongside +/// the shadowing and trashing columns V2 through V5 add. +/// +/// **`burst_members` is rewritten whole by every pass.** No id of its own: the +/// group is named by the image id of its earliest frame, so a burst that has not +/// changed keeps its name across a regroup and the interface can remember that +/// this one is open. There is no `bursts` table to go with it because a group +/// has no properties beyond its members — inventing a row for it would create an +/// identity that survives the grouping being rebuilt, which is precisely what +/// must not happen. +/// +/// **`burst_pick` is the one thing here that is not derived**, and it is a +/// separate table so that rewriting the grouping cannot erase it. A +/// representative stored on `burst_members` would be forgotten every time a +/// frame arrived; the user would be asked the same question after every scan. +/// The same argument `people.ignored` makes in V10, one subsystem over. +/// +/// **`burst_expanded` is view state in the catalog**, which is unusual enough to +/// justify. The grid is a window over an ordered query — `LIMIT n OFFSET k` — +/// so what a collapsed burst hides has to be decided by the query, or the row +/// count stops agreeing with the scrollbar and the ordinals a scrub resolves to. +/// Once SQL has to see it, this is where it lives. Nothing else reads it, and it +/// is emptied of stale groups by every pass. +const V11: &str = r#" +-- A 64-bit perceptual signature. Local derived state: NULL until something has +-- decoded the image, recomputed from thumbnails if the catalog is rebuilt, and +-- comparable only to signatures produced by the same build (`bursts`). +ALTER TABLE images ADD COLUMN perceptual_hash INTEGER; + +CREATE TABLE burst_members ( + -- One burst at most per image: a frame belongs to the moment it was taken + -- in, and nothing else. + image_id INTEGER PRIMARY KEY REFERENCES images(id) ON DELETE CASCADE, + -- The image id of the burst's earliest frame. Not a foreign key by + -- accident: the leader is itself a member, so this genuinely references + -- images(id), and cascading its deletion is right. + burst_id INTEGER NOT NULL REFERENCES images(id) ON DELETE CASCADE, + -- The frame the group collapses to. Exactly one per burst. + representative INTEGER NOT NULL DEFAULT 0 +); +-- Counting a burst's frames and listing them are what the grid asks for, once +-- per window; without this both are a scan of every grouped frame in the +-- library. +CREATE INDEX burst_members_burst ON burst_members(burst_id); + +-- The user's own choice of representative. User data, never rewritten by a +-- grouping pass -- see the module doc above. +CREATE TABLE burst_pick ( + image_id INTEGER PRIMARY KEY REFERENCES images(id) ON DELETE CASCADE +); + +-- Bursts the grid is currently showing in full. +CREATE TABLE burst_expanded ( + burst_id INTEGER PRIMARY KEY REFERENCES images(id) ON DELETE CASCADE +); +"#; + const V9: &str = r#" -- TRACES: FR-CULL-8 -- A record that face detection has *run* on an image, distinct from what it diff --git a/docs/catalog.md b/docs/catalog.md index 1268bd3..5d1dee8 100644 --- a/docs/catalog.md +++ b/docs/catalog.md @@ -810,6 +810,60 @@ confidence is unavailable. It does not fall back to an untuned default dressed u --- +## 10a. Bursts and near-duplicates + +Specified by FR-CULL-5, implemented in `dr_catalog::bursts` (a v11 migration) with the pass that +feeds it in `dr_ui::bursts`. + +A burst is a run of frames that are **adjacent in time and look like the frame before them**. Both +halves are load-bearing. Time alone groups a whole wedding ceremony, because a photographer working +steadily never leaves the gap that would end the run. Similarity alone groups a studio setup shot +across two days, which is a project rather than a moment. The bounds are two seconds and eight bits +of a 64-bit difference hash, and the reasoning for each figure is in the module. + +**Two seconds, for a burst that fires ten frames in one.** `images.captured_at` is whole seconds: +EXIF's `DateTimeOriginal` has no sub-second field, and `SubSecTimeOriginal` is optional and widely +omitted. Ten frames of a burst therefore arrive sharing a timestamp, and any threshold finer than a +second is a threshold on information the catalog does not have. Where the pace really is faster than +two seconds, the similarity bound is what separates the frames. + +**The signal is a perceptual hash of the thumbnail, not of the original.** `images.perceptual_hash` +is filled from the 256px thumbnails §7 already stores — vastly more resolution than a 9×8 reduction +uses — so a library that has been browsed has already paid for its signatures and no RAW is decoded +for this. The consequence is stated rather than hidden: an image with no thumbnail gets no +signature, and a frame with no signature never joins a burst. It is picked up by the next pass. + +**It is a pass, not a job kind**, for exactly the reason §10.2 gives for face clustering: a burst is +a property of a *run* of frames and has no natural `subject_id`, so a per-image job would rebuild +the world once per photograph. It runs when the thumbnail sweep finishes, which is the first moment +the signatures can all be computed. + +**A newly found burst arrives open.** The pass marks frames; it never takes them off the screen. +Collapsing on discovery would be tidier and would also mean a background pass removing photographs +from under someone part way through a cull. Folding a burst up is the user's act, it is remembered +(`burst_expanded`), and a burst that is already known keeps whatever state it is in — so the pass +that follows the next import does not spring open a morning's work. + +**Nothing here ranks a frame.** The representative of a collapsed burst is its *earliest* frame, +which is a fact about the clock rather than a judgement about the photograph. FR-CULL-5 names the +failure this avoids — rejecting the only frame of an important moment because someone blinked — and +the only judgement in the subsystem is the user's own choice of representative, which lives in its +own table (`burst_pick`) so that rebuilding the grouping cannot erase it. Same argument as +`people.ignored` in §10. + +**What the collapse costs the grid.** Which rows a collapsed burst hides has to be decided by the +query rather than by the cells, because the grid is a window (`LIMIT n OFFSET k`) and the frames it +hides are mostly not loaded. So the predicate joins `VISIBLE` in every query that lists or counts +cells, under the same discipline: present in four places of five, the header's count, the +scrollbar, the shift-click range and the scrub's ordinal stop describing the same list. + +**What this does not settle.** Bursts are local: the tables ride along in the uploaded catalog +snapshot and nothing on the far side reads them, so a second device rebuilds its own grouping from +its own signatures. Making `burst_pick` cross-device is a merge question of the same shape as §8.4's +and is not answered here. + +--- + ## 11. Requirements touched | ID | How this document addresses it | @@ -828,6 +882,7 @@ confidence is unavailable. It does not fall back to an untuned default dressed u | NFR-P1 | §3.1 one stat per directory, not per file | | NFR-P3 | §7.1 on-demand generation | | NFR-ARCH-2 | §6.3 priority classes shared with the GPU scheduler | +| FR-CULL-5 | §10a burst grouping: capture-time proximity and image similarity, collapse without selection | | NFR-ARCH-3 | §4.3 query cancellation, §6 job cancellation | | NFR-RES-4 | §7.3 LRU cap, eviction order | | FR-CULL-8 | §10.1 `faces` schema, §6.1 `DetectFaces` job kind on the proxy tier | diff --git a/ui/dr-ui/src/bursts.rs b/ui/dr-ui/src/bursts.rs new file mode 100644 index 0000000..b83253b --- /dev/null +++ b/ui/dr-ui/src/bursts.rs @@ -0,0 +1,531 @@ +//! TRACES: FR-CULL-5 +//! Burst grouping, as the library screen uses it. +//! +//! [`dr_catalog::bursts`] holds the grouping itself and knows nothing about +//! pixels. This is the other half: where the similarity signal comes from, and +//! how a group reaches a grid cell. +//! +//! # The signal comes out of the thumbnail store +//! +//! A perceptual signature needs pixels, and the cheapest pixels in the app are +//! the ones already sitting in `dr-thumbs`: a 256px JPEG per photograph, built +//! for the grid, shared between devices, and vastly more resolution than a 9×8 +//! reduction can use. So this pass decodes thumbnails, never originals. A +//! library that has been browsed — or that has synced somebody else's shards — +//! has already paid for every signature it is about to get. +//! +//! The consequence, stated rather than hidden: **an image with no thumbnail +//! gets no signature, and a frame with no signature never joins a burst.** That +//! is self-correcting rather than permanent — the next pass finds the thumbnail +//! the sweep has since built — and it is the reason this runs when the +//! thumbnail sweep finishes rather than on a timer. +//! +//! # Why a pass and not a job +//! +//! Hashing is per-image and would make a perfectly good job kind. Grouping is +//! not: a burst is a property of a *run* of frames, so a per-image job would +//! regroup the library once per photograph. Since the two have to happen in that +//! order and the second cannot be split, both live in one pass — the same +//! argument docs/catalog.md §10.2 makes for face clustering. +//! +//! # It is never on the UI thread +//! +//! Decoding tens of thousands of thumbnails is bounded only by library size, and +//! the one thing that must not grow with library size is how long the window +//! stops answering (NFR-P9). Cancellation is dropping the receiver; a pass +//! abandoned half way leaves the signatures it did compute — they are permanent +//! and correct — and the previous grouping intact. + +use std::cell::{Cell, RefCell}; +use std::collections::HashMap; +use std::path::PathBuf; +use std::sync::mpsc::Receiver; + +use dr_catalog::bursts::{self, Rules, Signature}; +use dr_catalog::Catalog; +use dr_thumbs::{ThumbSize, ThumbStore}; +use dr_types::ImageId; +use slint::Model as _; + +use crate::AppWindow; + +/// How many signatures are written per transaction. +/// +/// One transaction per image costs a WAL commit per thumbnail and turns a pass +/// over a real library into minutes of fsync; one transaction for the whole pass +/// holds a write lock for the duration and loses everything if the app closes. +/// A few hundred is the usual answer to that trade. +const WRITE_BATCH: usize = 256; + +/// Progress from a grouping pass. +#[derive(Debug, Clone, PartialEq)] +pub enum BurstMessage { + /// How many images still need a signature. Sent once, before any decoding. + Started { to_hash: usize }, + /// Cumulative signatures written. + Progress { hashed: usize }, + /// The pass finished, and this is what the library now looks like. + Finished { + hashed: usize, + bursts: usize, + frames: usize, + }, + /// It did not. + Failed(String), +} + +/// Hash whatever is missing a signature, then rebuild the grouping. +/// +/// Both halves run on a worker thread. The catalog is opened here rather than +/// shared with the UI's connection: SQLite connections are not `Send`, and WAL +/// is what makes a second one safe while the grid reads (NFR-R1). +pub fn spawn_grouping(catalog_path: PathBuf, thumbs_dir: PathBuf) -> Receiver { + let (tx, rx) = std::sync::mpsc::channel(); + + std::thread::spawn(move || { + let catalog = match Catalog::open(&catalog_path) { + Ok(c) => c, + Err(e) => { + let _ = tx.send(BurstMessage::Failed(format!("cannot open catalog: {e}"))); + return; + } + }; + let conn = catalog.connection(); + + let outstanding = match bursts::images_without_signature(conn) { + Ok(v) => v, + Err(e) => { + let _ = tx.send(BurstMessage::Failed(e.to_string())); + return; + } + }; + if tx + .send(BurstMessage::Started { + to_hash: outstanding.len(), + }) + .is_err() + { + return; + } + + // A broken or absent store costs signatures, never correctness: the + // grouping still runs over whatever is already hashed, and the images + // that missed out are picked up by the next pass. + let store = match ThumbStore::open(&thumbs_dir) { + Ok(s) => Some(s), + Err(e) => { + log::warn!("thumbnail store unavailable, not hashing: {e}"); + None + } + }; + + let hashed = match store { + Some(store) => hash_all(conn, &store, &outstanding, &tx), + None => 0, + }; + + match bursts::regroup(conn, Rules::default()) { + Ok(report) => { + log::info!( + "bursts: {hashed} signature(s) added, {} group(s) over {} frame(s), \ + largest {}", + report.bursts, + report.frames, + report.largest + ); + let _ = tx.send(BurstMessage::Finished { + hashed, + bursts: report.bursts, + frames: report.frames, + }); + } + Err(e) => { + let _ = tx.send(BurstMessage::Failed(e.to_string())); + } + } + }); + + rx +} + +thread_local! { + /// The running pass's drain timer, and whether one is running. + /// + /// Module-local rather than a pair of fields on the library controller, so + /// that everything this feature needs to run lives in this file and the + /// screen that starts it is left holding nothing. Safe as a thread local + /// because Slint's event loop is single-threaded (NFR-P9) and this is only + /// ever touched from it. + /// + /// The flag is separate because stopping a timer does not drop it: a slot + /// tested for emptiness would refuse every pass after the first. + static DRAIN: RefCell> = const { RefCell::new(None) }; + static RUNNING: Cell = const { Cell::new(false) }; +} + +/// Hash what has become hashable, then rebuild the library's burst grouping. +/// +/// Fired when the thumbnail sweep finishes, because that is the moment the +/// signatures can all be computed: the pass reads thumbnails, and until the +/// sweep has run most images have none. Not on a timer, and not after every +/// scan — a regroup is cheap but not free, and nothing is waiting on it. +/// +/// `grouped` is called once, with the number of bursts the library now has, if +/// the pass finishes. It is where the caller reloads the grid: the cells hold +/// the same photographs they held before — a new burst arrives open — but every +/// run of frames now carries a mark it did not have a moment ago, and only a +/// reload carries it. +/// +/// Deliberately silent otherwise. The sweeps around it report progress because +/// they run for tens of minutes; this is seconds, and a status line for it would +/// be a line the user must read in order to learn nothing. +pub fn start_pass(catalog_path: PathBuf, thumbs_dir: PathBuf, grouped: impl Fn(usize) + 'static) { + // A second pass would read the same rows and write the same answer over the + // first one's transactions. + if RUNNING.get() { + return; + } + RUNNING.set(true); + + let rx = spawn_grouping(catalog_path, thumbs_dir); + let timer = slint::Timer::default(); + timer.start( + slint::TimerMode::Repeated, + std::time::Duration::from_millis(400), + move || loop { + let msg = match rx.try_recv() { + Ok(m) => m, + Err(std::sync::mpsc::TryRecvError::Empty) => return, + Err(std::sync::mpsc::TryRecvError::Disconnected) => { + finish(); + return; + } + }; + + match msg { + BurstMessage::Started { to_hash } => { + log::info!("burst grouping: {to_hash} image(s) still to hash"); + } + // Nothing on screen is showing this. Drained rather than + // ignored, because an unread channel is a worker that stalls. + BurstMessage::Progress { hashed } => { + log::debug!("burst grouping: {hashed} hashed so far"); + } + BurstMessage::Finished { + hashed, + bursts, + frames, + } => { + log::info!( + "burst grouping: {hashed} signature(s) added, \ + {bursts} group(s) over {frames} frame(s)" + ); + finish(); + grouped(bursts); + return; + } + BurstMessage::Failed(e) => { + log::warn!("burst grouping: {e}"); + finish(); + return; + } + } + }, + ); + + DRAIN.with(|slot| *slot.borrow_mut() = Some(timer)); +} + +/// Stop draining, and let another pass start. +/// +/// Stops the timer without dropping it — dropping one from inside its own +/// callback is not something to rely on — which is why the flag beside it is +/// what actually says whether a pass is running. +fn finish() { + RUNNING.set(false); + DRAIN.with(|slot| { + if let Some(timer) = slot.borrow().as_ref() { + timer.stop(); + } + }); +} + +/// Decode each image's stored thumbnail and record its signature. +/// +/// Returns how many were written. Anything without a stored thumbnail, or whose +/// blob will not decode, is simply skipped — it keeps its NULL and comes back +/// next time, which is the same treatment `spawn_thumbnails` gives a corrupt +/// blob. +fn hash_all( + conn: &rusqlite::Connection, + store: &ThumbStore, + outstanding: &[ImageId], + tx: &std::sync::mpsc::Sender, +) -> usize { + let keys = thumbnail_keys(conn); + let mut pending: Vec<(ImageId, Signature)> = Vec::with_capacity(WRITE_BATCH); + let mut written = 0usize; + + for image in outstanding { + let Some(file_id) = keys.get(image).copied() else { + continue; + }; + // The grid class, not the large one: 256px is already thirty times the + // detail the reduction keeps, and asking for `Large` would miss most of + // the store, which is filled at `Grid`. + let Ok(Some(thumb)) = store.get(file_id, ThumbSize::Grid) else { + continue; + }; + let Ok((w, h, rgba)) = dr_thumbs::decode_rgba(&thumb.bytes) else { + continue; + }; + let Some(signature) = bursts::signature_of_rgba(&rgba, w, h) else { + continue; + }; + pending.push((*image, signature)); + + if pending.len() >= WRITE_BATCH { + written += flush(conn, &mut pending); + if tx.send(BurstMessage::Progress { hashed: written }).is_err() { + // The receiver is gone: the screen has moved on, and finishing + // the pass would be work nobody is waiting for. + return written; + } + } + } + written += flush(conn, &mut pending); + written +} + +/// Write one batch of signatures, emptying `pending`. +fn flush(conn: &rusqlite::Connection, pending: &mut Vec<(ImageId, Signature)>) -> usize { + if pending.is_empty() { + return 0; + } + let n = pending.len(); + let tx = match conn.unchecked_transaction() { + Ok(t) => t, + Err(e) => { + log::warn!("recording signatures: {e}"); + pending.clear(); + return 0; + } + }; + for (image, signature) in pending.drain(..) { + if let Err(e) = bursts::set_signature(&tx, image, signature) { + log::debug!("recording signature for {}: {e}", image.0); + } + } + match tx.commit() { + Ok(()) => n, + Err(e) => { + log::warn!("committing signatures: {e}"); + 0 + } + } +} + +/// Every image's thumbnail-store key, in one query. +/// +/// Read whole rather than asked per image: the table is one small row per +/// photograph, and a query per image would be tens of thousands of statements +/// to save a megabyte. +fn thumbnail_keys(conn: &rusqlite::Connection) -> HashMap { + let mut out = HashMap::new(); + let Ok(mut stmt) = conn.prepare("SELECT image_id, file_id FROM remote") else { + return out; + }; + let Ok(rows) = stmt.query_map([], |r| Ok((r.get::<_, i64>(0)?, r.get::<_, i64>(1)?))) else { + return out; + }; + for (image, file) in rows.flatten() { + out.insert(ImageId(image as u64), file as u64); + } + out +} + +/// Fill in the burst badge on every cell of the loaded window. +/// +/// One query for the window, in the same style as the collection badges and the +/// rating counts beside it — a grid that asks the catalog a question per cell is +/// a grid that stutters under a finger. +/// +/// `ids` is the window in grid order, so the row index into the model is the +/// index into it. +pub fn sync_badges(window: &AppWindow, catalog: &Catalog, ids: &[ImageId]) { + if ids.is_empty() { + return; + } + let found = match bursts::memberships(catalog.connection(), ids) { + Ok(m) => m, + Err(e) => { + log::debug!("reading burst membership: {e}"); + return; + } + }; + + let model = window.get_library_cells(); + for (row, id) in ids.iter().enumerate() { + let (count, expanded) = match found.get(id) { + Some(m) => (m.size as i32, m.expanded), + // Not in a burst at all, which is most of a library. + None => (0, false), + }; + if let Some(mut cell) = model.row_data(row) { + if cell.burst_count != count || cell.burst_expanded != expanded { + cell.burst_count = count; + cell.burst_expanded = expanded; + model.set_row_data(row, cell); + } + } + } +} + +/// Open or close the burst one cell belongs to. +/// +/// Which direction is decided from what the catalog says the group is currently +/// doing, rather than from the cell's own `burst-expanded`. The cell is a copy +/// of that state and can be one reload behind; the table cannot. +/// +/// The caller reloads the grid rather than repainting it, because collapsing +/// changes what the grid's *query* returns: the row count, the scrollbar and +/// the ordinal a scrub resolves all move together, and they can only stay in +/// step by being read again together. +/// +/// Returns whether anything changed, so a click on a cell that is in no burst — +/// an entirely normal thing to happen — costs no round trip through the grid. +pub fn toggle(catalog: &Catalog, image: ImageId) -> bool { + let conn = catalog.connection(); + let Ok(found) = bursts::memberships(conn, &[image]) else { + return false; + }; + let Some(membership) = found.get(&image) else { + return false; + }; + match bursts::set_expanded(conn, membership.burst_id, !membership.expanded) { + Ok(()) => true, + Err(e) => { + log::warn!("toggling burst {}: {e}", membership.burst_id.0); + false + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + /// A catalog with two frames of one burst, and a thumbnail store holding a + /// picture for each. + /// + /// `name` keeps two tests from sharing a directory, since they run in + /// parallel. Same shape as the store tests in `library`, which is also why + /// this reaches for `temp_dir` rather than a crate: nothing else here needs + /// one. + fn library(name: &str) -> (Catalog, PathBuf) { + let cat = Catalog::in_memory().unwrap(); + let c = cat.connection(); + c.execute( + "INSERT INTO roots(id, kind, label) VALUES (1, 'local', 'lib')", + [], + ) + .unwrap(); + for (id, at) in [(1i64, 1000i64), (2, 1001)] { + c.execute( + "INSERT INTO images(id, root_id, source_ref, captured_at, camera, added_at) + VALUES (?1, 1, ?2, ?3, 'Canon EOS R5', 0)", + rusqlite::params![id, format!("IMG_{id}.CR3"), at], + ) + .unwrap(); + c.execute( + "INSERT INTO remote(image_id, file_id) VALUES (?1, ?2)", + rusqlite::params![id, 100 + id], + ) + .unwrap(); + } + + let dir = std::env::temp_dir().join(format!("dr-ui-bursts-{name}-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&dir); + { + let mut store = ThumbStore::open(&dir).unwrap(); + for (id, shift) in [(1u64, 0usize), (2, 1)] { + let rgba = picture(shift); + let bytes = dr_thumbs::encode_rgba(64, 64, &rgba).unwrap(); + store + .put( + 100 + id, + ThumbSize::Grid, + &dr_thumbs::Thumbnail { + width: 64, + height: 64, + bytes, + }, + ) + .unwrap(); + } + } + (cat, dir) + } + + /// A blocky scene, moved sideways by `shift` pixels — one frame of a burst + /// and then the next. + fn picture(shift: usize) -> Vec { + let mut out = vec![0u8; 64 * 64 * 4]; + for y in 0..64 { + for x in 0..64 { + let v = (((x + shift) / 8) * 37 + (y / 8) * 91) as u8; + let p = (y * 64 + x) * 4; + out[p] = v; + out[p + 1] = v; + out[p + 2] = v; + out[p + 3] = 255; + } + } + out + } + + #[test] + fn the_pass_hashes_from_thumbnails_and_groups_what_it_hashed() { + let (cat, dir) = library("hashes"); + let conn = cat.connection(); + let store = ThumbStore::open(&dir).unwrap(); + let outstanding = bursts::images_without_signature(conn).unwrap(); + assert_eq!(outstanding.len(), 2); + + let (tx, _rx) = std::sync::mpsc::channel(); + let hashed = hash_all(conn, &store, &outstanding, &tx); + assert_eq!(hashed, 2, "both thumbnails should have yielded a signature"); + + let report = bursts::regroup(conn, Rules::default()).unwrap(); + assert_eq!(report.bursts, 1, "the two frames were not grouped"); + assert_eq!(report.frames, 2); + } + + #[test] + fn an_image_with_no_thumbnail_keeps_its_null() { + let (cat, dir) = library("no-thumb"); + let conn = cat.connection(); + conn.execute( + "INSERT INTO images(id, root_id, source_ref, captured_at, added_at) + VALUES (9, 1, 'IMG_9.CR3', 1002, 0)", + [], + ) + .unwrap(); + + let store = ThumbStore::open(&dir).unwrap(); + let outstanding = bursts::images_without_signature(conn).unwrap(); + let (tx, _rx) = std::sync::mpsc::channel(); + assert_eq!(hash_all(conn, &store, &outstanding, &tx), 2); + // And it is still offered next time, rather than being written off. + assert_eq!( + bursts::images_without_signature(conn).unwrap(), + vec![ImageId(9)] + ); + } + + #[test] + fn toggling_a_cell_that_is_in_no_burst_changes_nothing() { + let (cat, _dir) = library("no-burst"); + assert!(!toggle(&cat, ImageId(1))); + } +} diff --git a/ui/dr-ui/src/lib.rs b/ui/dr-ui/src/lib.rs index c10b19f..a101998 100644 --- a/ui/dr-ui/src/lib.rs +++ b/ui/dr-ui/src/lib.rs @@ -20,6 +20,7 @@ //! in `ui/` names an operation or knows a shader exists (FR-DEV-3a). mod activity; +mod bursts; mod collections_ui; mod derived_sync; mod develop; diff --git a/ui/dr-ui/src/library.rs b/ui/dr-ui/src/library.rs index 8bc230c..97a81d9 100644 --- a/ui/dr-ui/src/library.rs +++ b/ui/dr-ui/src/library.rs @@ -186,6 +186,26 @@ const VISIBLE_UNALIASED: &str = "shadowed_by IS NULL AND trashed_at IS NULL"; /// restore the same frame twice. const TRASHED: &str = "i.shadowed_by IS NULL AND i.trashed_at IS NOT NULL"; +/// TRACES: FR-CULL-5 +/// The clause that hides the frames a collapsed burst is standing in for. +/// +/// Subject to exactly the discipline [`VISIBLE`] is under, and for the same +/// reason: the header's count, the scrollbar's size, the run a shift-click +/// resolves and the ordinal a scrub lands on are four answers about one list. +/// A burst folded away in the cells but still counted in the total would leave +/// the grid ending in rows that draw nothing, with no clue why. +/// +/// The predicate itself is `dr_catalog::bursts`'s, not this file's, so the +/// interface and the pass that writes the table cannot come to disagree about +/// what collapsed means. +/// +/// A function rather than a constant because it has to name the image table, +/// and the grid aliases it as `i` where the timeline's queries do not. `image` +/// is a table name from this file and never anything a user supplied. +fn uncollapsed(image: &str) -> String { + format!(" AND {}", dr_catalog::bursts::not_collapsed_away(image)) +} + /// TRACES: FR-CAT-4 /// The order the grid lists photographs in: when they were taken. /// @@ -3814,10 +3834,11 @@ pub fn read_cells_scoped( .collect::>() .join(","); let rated = filter.sql(); + let folded = uncollapsed("i"); let sql = format!( "SELECT {CELL_COLUMNS} FROM images i - WHERE {VISIBLE}{rated} + WHERE {VISIBLE}{rated}{folded} AND i.id IN (SELECT image_id FROM collection_members WHERE collection_id IN ({placeholders})) {GRID_ORDER} @@ -3849,11 +3870,12 @@ fn read_cells_all( limit: usize, ) -> Result, dr_catalog::CatalogError> { let rated = filter.sql(); + let folded = uncollapsed("i"); let mut rows = { let mut stmt = catalog.connection().prepare(&format!( "SELECT {CELL_COLUMNS} FROM images i - WHERE {VISIBLE}{rated} + WHERE {VISIBLE}{rated}{folded} {GRID_ORDER} LIMIT ?1 OFFSET ?2" ))?; @@ -3954,10 +3976,11 @@ pub fn read_ids_span( } else { let (clause, params) = scope_clause(catalog, scope)?; let rated = filter.sql(); + let folded = uncollapsed("i"); ( format!( "SELECT i.id FROM images i - WHERE {VISIBLE}{rated}{clause} + WHERE {VISIBLE}{rated}{folded}{clause} {GRID_ORDER} LIMIT ? OFFSET ?" ), @@ -4086,9 +4109,10 @@ pub fn total_images_scoped( // Counted through `images` rather than over `collection_members` alone, so // `VISIBLE` applies — a trashed photograph is still a member row, and // counting it made the header claim images the grid would not draw. + let folded = uncollapsed("i"); let sql = format!( "SELECT count(DISTINCT i.id) FROM images i - WHERE {VISIBLE}{rated} + WHERE {VISIBLE}{rated}{folded} AND i.id IN (SELECT image_id FROM collection_members WHERE collection_id IN ({placeholders}))" ); @@ -4259,8 +4283,9 @@ fn total_images_filtered( filter: &RatingFilter, ) -> Result { let rated = filter.sql(); + let folded = uncollapsed("i"); let n: i64 = catalog.connection().query_row( - &format!("SELECT count(*) FROM images i WHERE {VISIBLE}{rated}"), + &format!("SELECT count(*) FROM images i WHERE {VISIBLE}{rated}{folded}"), [], |r| r.get(0), )?; @@ -4806,12 +4831,16 @@ mod tests { #[test] fn the_window_read_walks_the_ordering_index() { let catalog = with_images(20); + // Including the burst clause, because the grid includes it: a + // predicate that quietly cost the ordering index would put the sort + // back and this is the only place that would notice. + let folded = uncollapsed("i"); let plan: Vec = catalog .connection() .prepare(&format!( "EXPLAIN QUERY PLAN SELECT {CELL_COLUMNS} FROM images i - WHERE {VISIBLE} + WHERE {VISIBLE}{folded} {GRID_ORDER} LIMIT 10 OFFSET 5" )) @@ -4864,6 +4893,58 @@ mod tests { catalog } + /// TRACES: FR-CULL-5 + /// A folded burst takes rows out of the cells, the count and the range a + /// shift-click resolves — all three, together. + /// + /// This is the test that would fail if the clause were added to four of + /// the five queries that need it. That failure has no other symptom: the + /// header claims images the grid will not draw, the scrollbar sizes itself + /// for rows that are not there, and neither number looks wrong on its own. + #[test] + fn folding_a_burst_takes_the_same_rows_out_of_every_answer() { + use dr_catalog::bursts::{self, Rules, Signature}; + + let catalog = with_images(4); + // Three of the four are one burst: a second apart, one signature. + let ids = image_ids(&catalog); + for (n, id) in ids.iter().enumerate() { + let hash = if n < 3 { 0xFF00 } else { 0x00FF }; + catalog + .connection() + .execute( + "UPDATE images SET captured_at = ?2, camera = 'Canon EOS R5', + perceptual_hash = ?3 + WHERE id = ?1", + rusqlite::params![id.0 as i64, 1_000 + n as i64, Signature(hash).to_stored()], + ) + .unwrap(); + } + bursts::regroup(catalog.connection(), Rules::default()).unwrap(); + + let filter = RatingFilter::default(); + // Open, as a new burst is: nothing has been taken away yet. + assert_eq!(read_cells(&catalog, 0, 50).unwrap().len(), 4); + assert_eq!(total_images_filtered(&catalog, &filter).unwrap(), 4); + + bursts::set_expanded(catalog.connection(), ids[0], false).unwrap(); + + let cells = read_cells(&catalog, 0, 50).unwrap(); + assert_eq!(cells.len(), 2, "the folded frames are still in the cells"); + assert_eq!( + total_images_filtered(&catalog, &filter).unwrap(), + cells.len(), + "the header's count and the cells disagree" + ); + assert_eq!( + read_ids_span(&catalog, None, &filter, false, 0, 49) + .unwrap() + .len(), + cells.len(), + "a shift-click over the whole grid would select frames it cannot show" + ); + } + fn image_ids(catalog: &Catalog) -> Vec { let mut stmt = catalog .connection() diff --git a/ui/dr-ui/src/library_ui.rs b/ui/dr-ui/src/library_ui.rs index ae6844d..7d5383a 100644 --- a/ui/dr-ui/src/library_ui.rs +++ b/ui/dr-ui/src/library_ui.rs @@ -2214,6 +2214,11 @@ fn load_window(window: &AppWindow, ctl: &Rc) { // window, not one per cell. rating: 0, flag: 0, + // And by `bursts::sync_badges`, in one more query for the + // window. Zero is "not in a burst", which is what almost every + // photograph in a library is. + burst_count: 0, + burst_expanded: false, } }) .collect(); @@ -2246,6 +2251,9 @@ fn load_window(window: &AppWindow, ctl: &Rc) { .collect(); crate::collections_ui::sync_badges(window, catalog, &ids); sync_ratings(window, catalog, &ids); + // How many frames each cell stands for, where it stands for several + // (FR-CULL-5). + crate::bursts::sync_badges(window, catalog, &ids); // The rebuilt cells all carry `selected: false`, but the selection itself // is a set of image ids and survives untouched. Without this the ticks // vanished on every scroll — the selection was still there and still acted @@ -3769,6 +3777,28 @@ fn start_thumbnail_sweep(window: &AppWindow, ctl: &Rc) { if !offline { start_derived_sync(&w, &ctl_cb); } + // And now every photograph in the library has a + // thumbnail, which is the only moment all of its burst + // signatures can be computed. See `bursts::start_pass`, + // which owns the pass and everything it needs to drain + // itself; what it wants from here is the paths and a + // way to say the grid has something new to draw. + if let Some((conn, _)) = ctl_cb.session.borrow().clone() { + let weak_after = w.as_weak(); + let ctl_after = ctl_cb.clone(); + crate::bursts::start_pass( + library::catalog_path(&conn.account), + library::thumbs_dir(&conn.account), + move |bursts| { + let Some(w) = weak_after.upgrade() else { + return; + }; + if bursts > 0 && w.get_show_library() { + schedule_reload(&w, &ctl_after); + } + }, + ); + } return; } } @@ -4119,10 +4149,14 @@ fn capture_time_from_catalog(catalog: &Catalog, ordinal: usize) -> Option { catalog .connection() .query_row( - "SELECT captured_at FROM images - WHERE shadowed_by IS NULL AND captured_at IS NOT NULL - ORDER BY captured_at - LIMIT 1 OFFSET ?1", + &format!( + "SELECT captured_at FROM images + WHERE shadowed_by IS NULL AND captured_at IS NOT NULL + AND {} + ORDER BY captured_at + LIMIT 1 OFFSET ?1", + dr_catalog::bursts::not_collapsed_away("images") + ), [ordinal as i64], |r| r.get::<_, i64>(0), ) @@ -4153,13 +4187,21 @@ fn scrub_to(window: &AppWindow, ctl: &Rc, when: i64) { // BY), so they never precede a dated one and the predicate below stays // a simple `<`. Shadowed rows are excluded here exactly as the grid // excludes them. + // + // A burst folded up occupies one row of the grid, so it must occupy one + // row of this count as well: an ordinal taken over the unfolded library + // would overshoot by every frame hidden earlier in it. catalog .connection() .query_row( - "SELECT count(*) FROM images - WHERE shadowed_by IS NULL - AND captured_at IS NOT NULL - AND captured_at < ?1", + &format!( + "SELECT count(*) FROM images + WHERE shadowed_by IS NULL + AND captured_at IS NOT NULL + AND captured_at < ?1 + AND {}", + dr_catalog::bursts::not_collapsed_away("images") + ), [when], |r| r.get::<_, i64>(0), ) @@ -4946,6 +4988,34 @@ pub fn wire( }); } + // Fold a burst up, or open it out (FR-CULL-5). A reload rather than a repaint, because + // it changes what the grid's query returns — see [`crate::bursts::toggle`]. + { + let weak = window.as_weak(); + let ctl = ctl.clone(); + window.on_library_burst_toggled(move |row| { + let Some(w) = weak.upgrade() else { return }; + let id = ctl + .image_ids + .borrow() + .get(row as usize) + .map(|id| dr_types::ImageId(*id as u64)); + let Some(id) = id else { return }; + + let changed = { + let borrow = ctl.catalog.borrow(); + match borrow.as_ref() { + Some(catalog) => crate::bursts::toggle(catalog, id), + None => false, + } + }; + if changed { + ctl.requested.borrow_mut().clear(); + load_window(&w, &ctl); + } + }); + } + // A rating or flag key. Applies to the whole selection, which is what // makes judging a run of frames one keystroke rather than forty. { diff --git a/ui/dr-ui/ui/app.slint b/ui/dr-ui/ui/app.slint index 4fc9a0b..e8f67b9 100644 --- a/ui/dr-ui/ui/app.slint +++ b/ui/dr-ui/ui/app.slint @@ -585,6 +585,8 @@ export component AppWindow inherits Window { callback library-cell-rated(int, int); /// The trash target was clicked on one cell, by row. callback library-cell-trashed(int); + /// The burst mark was clicked on one cell, by row (FR-CULL-5). + callback library-burst-toggled(int); /// Move the grid selection to the trash — the `Delete` key. callback library-trash-selection(); /// A judgement key was pressed, applying to the whole selection. One of @@ -1548,6 +1550,7 @@ in property panel-visible: true; cell-rated(i, n) => { root.library-cell-rated(i, n); } cell-trashed(i) => { root.library-cell-trashed(i); } + burst-toggled(i) => { root.library-burst-toggled(i); } trash-selection() => { root.library-trash-selection(); } // Derived from the sidebar's own selection rather than // mirrored in a second property: `-1` is already the sentinel diff --git a/ui/dr-ui/ui/library.slint b/ui/dr-ui/ui/library.slint index 181f251..d0ce8ac 100644 --- a/ui/dr-ui/ui/library.slint +++ b/ui/dr-ui/ui/library.slint @@ -458,6 +458,16 @@ export struct LibraryCell { // 0 unflagged, 1 pick, 2 reject. Independent of the stars: rejecting a // four-star frame is a normal thing to do mid-cull. flag: int, + // Frames in the burst this cell belongs to (FR-CULL-5), itself included; 0 where it + // belongs to none, which is most of a library. A burst that is collapsed + // draws only its representative, so on that cell this is the count of what + // is hidden behind it — the reason it is shown at all. + burst-count: int, + // Whether the group is currently open. Drawn differently rather than + // hidden: a burst the user has expanded is the one thing on screen that + // needs a way back, and a control that disappears once used is a control + // nobody finds twice. + burst-expanded: bool, } // The photo roll: the grid's loaded window along the foot of the develop view. @@ -1155,6 +1165,11 @@ export component LibraryGrid inherits Rectangle { callback cell-clicked(int); /// A star was clicked on a cell: row, and the rating 0..5. callback cell-rated(int, int); + /// The burst mark on a cell was clicked (FR-CULL-5): open the group, or fold it back + /// up. Which of the two is decided in Rust, from what the catalog says the + /// group is currently doing, so the mark cannot get out of step with the + /// query that actually hides the frames. + callback burst-toggled(int); /// Whether the grid is currently listing the trash rather than the /// library. Suppresses the per-cell trash target, which would be inert /// there — `plan_trash` skips an already-trashed image — and offering a @@ -2925,6 +2940,75 @@ export component LibraryGrid inherits Rectangle { rate(n) => { root.cell-rated(i, n); } trash() => { root.cell-trashed(i); } } + + // The burst mark (FR-CULL-5): how many frames this + // moment holds, + // and the way in and out of them. + // + // A child of `cell-touch` for exactly the reason the + // stars above are: a click here must not also reach + // `cell-clicked` and throw the user into develop, and + // children are hit-tested before the element they sit + // in. Unlike the stars it is never hidden — a collapsed + // burst is standing in for frames that are not on + // screen, and there has to be something visible saying + // so whether or not a pointer is anywhere near. + // + // Bottom left, clear of the centred star strip and of + // both top corners, which the flag and the collection + // badge already have. + if cell.burst-count > 1: Rectangle { + x: 6px; + y: parent.height - self.height - 26px; + width: 30px; + height: 18px; + + // The pile behind the top card, drawn only while the + // group is folded up. It is the whole of the "there + // is more than one of these" cue; once the burst is + // open the frames themselves say it. + Rectangle { + x: 3px; + y: -3px; + width: parent.width - 3px; + height: parent.height; + visible: !cell.burst-expanded; + background: Theme.surface; + border-radius: Theme.radius-sm; + border-width: 1px; + border-color: Theme.rule; + } + + Rectangle { + width: 100%; + height: 100%; + background: cell.burst-expanded ? Theme.selected + : Theme.surface; + border-radius: Theme.radius-sm; + border-width: 1px; + border-color: cell.burst-expanded ? Theme.selected-ring + : Theme.rule; + + Text { + width: 100%; + height: 100%; + horizontal-alignment: center; + vertical-alignment: center; + // No "of": the number is the size of the + // group, and a cell this small cannot + // afford a word to say so. + text: cell.burst-count; + color: Theme.ink; + font-size: 10px; + font-weight: 700; + } + } + + TouchArea { + mouse-cursor: pointer; + clicked => { root.burst-toggled(i); } + } + } } } }