dtourolle/DarkRoom
1
0
Fork 0
Code Issues 39 Pull Requests Actions Packages Projects Releases 20 Wiki Activity

Add thumbnail size classes and grid zoom; fix the scrub ordinal

Browse Source 
The grid now zooms, which needs thumbnails at two resolutions rather than
one, and exposed a scrub that landed in the wrong place.

**Two thumbnail size classes.** `ThumbSize::Grid` (256px, ~10 KB) and
`Large` (1024px, ~45 KB), with the class part of the store key so both
coexist. Storing everything large would take the reference library from
~200 MB to ~860 MB, and shards sync, so that is transfer cost on every
device rather than only disk. A store written before the class existed
migrates in place: its entries are all grid-sized, which is what the
column defaults to, so nothing already fetched is discarded.

`forget` now drops every size for an image. Reading a single row left the
other class's bytes on the shard's tally for good, sealing it early on
space nothing occupied.

**Grid zoom.** Ctrl+wheel and pinch resize cells between 90px and 420px in
geometric steps, so the gesture feels the same at either end where a fixed
pixel step would be imperceptible at 400px and violent at 90px. Crossing
256px switches to the large class, so a zoomed cell is sharp rather than
upscaled. Columns and window capacity already derived from cell size, so
the grid reflows for free.

**The scrub landed about half a library too high.** It counted only dated
images while the grid shows all of them — 10,733 dated against 19,841
rows — and ignored `shadowed_by`. Verified against the live catalog: the
old formula gave 10,887, the new one 10,732, the true grid position
10,732. The scrub's count and the grid's window must use identical
predicates and ordering; a test now fails if they diverge.

**Timeline gestures are continuous.** Scrub and pan were quantised to
whole buckets, so a slow drag did nothing until it crossed a boundary and
then jumped a month. Both work in fractions of the visible span now, and
pinch-to-zoom arrives for tablet, where there is no wheel to reach the
axis with.

The pinch accumulator was wrong on first writing: it took at most one step
per update, so an 8x spread — three doublings — yielded one zoom level.
`log2().trunc()` now extracts every whole doubling and carries the
remainder. The original test asserted the wrong number and defended it in
a comment, which is worth remembering: a test can entrench a bug as
readily as catch one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
dtourolle
2026-08-09 21:58:32 +02:00
co-authored by Claude Opus 5
parent 75ce3846c3
commit d49b4b41de
13 changed files with 1020 additions and 156 deletions
Download Patch File Download Diff File Expand all files Collapse all files
core/dr-thumbs/src/lib.rs
+333 -94
View File
@@ -55,7 +55,58 @@ pub use error::ThumbError;
/// per shard, so the 17k-image reference library lands in ~14 shards.
pub const SHARD_MAX_BYTES: u64 = 25 * 1024 * 1024;
/// Long edge of a stored thumbnail.
/// Which resolution a stored thumbnail is.
///
/// Two classes rather than one, because the grid zooms: 256px is right for a
/// wall of small cells and soft on a large one, while storing everything large
/// would take the reference library from ~200 MB to ~860 MB — and the shards
/// **sync**, so that is transfer cost on every device, not just disk.
///
/// The discriminant is part of the store key, so both classes coexist and a
/// library thumbnailed at one size is not invalidated by the other appearing.
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]
#[repr(i64)]
pub enum ThumbSize {
/// Grid cells at their usual size. ~10 KB each.
Grid = 0,
/// Zoomed cells, the loupe, and the filmstrip. ~45 KB each, fetched only
/// where something actually asks for that detail.
Large = 1,
}
impl ThumbSize {
/// Long edge in pixels.
pub fn edge(self) -> u32 {
match self {
ThumbSize::Grid => 256,
ThumbSize::Large => 1024,
}
}
/// The smallest class that can fill a cell of this size without visibly
/// softening.
///
/// Compared against the *drawn* size, so a high-DPI display asking for
/// 300 logical pixels at 2x gets the large class, as it should.
pub fn for_cell(pixels: u32) -> Self {
if pixels > ThumbSize::Grid.edge() {
ThumbSize::Large
} else {
ThumbSize::Grid
}
}
fn from_i64(v: i64) -> Self {
match v {
1 => ThumbSize::Large,
_ => ThumbSize::Grid,
}
}
}
/// Long edge of a grid thumbnail.
///
/// Kept as the name callers already use; prefer [`ThumbSize::edge`].
pub const THUMBNAIL_EDGE: u32 = 256;
/// A thumbnail's bytes and dimensions.
@@ -84,6 +135,10 @@ impl ThumbStore {
index.pragma_update(None, "journal_mode", "WAL")?;
index.pragma_update(None, "synchronous", "NORMAL")?;
index.execute_batch(INDEX_SCHEMA)?;
// A store written before the size class existed holds grid-sized
// entries under a bare `file_id` key; bring it forward rather than
// discarding every thumbnail already fetched.
migrate_size_column(&index, "entries")?;
Ok(Self {
dir: dir.to_path_buf(),
@@ -92,12 +147,12 @@ impl ThumbStore {
}
/// Fetch a thumbnail by file id.
pub fn get(&self, file_id: u64) -> Result<Option<Thumbnail>, ThumbError> {
pub fn get(&self, file_id: u64, size: ThumbSize) -> Result<Option<Thumbnail>, ThumbError> {
let shard: Option<i64> = self
.index
.query_row(
"SELECT shard FROM entries WHERE file_id = ?1",
[file_id as i64],
"SELECT shard FROM entries WHERE file_id = ?1 AND size = ?2",
[file_id as i64, size as i64],
|r| r.get(0),
)
.ok();
@@ -109,8 +164,8 @@ impl ThumbStore {
let conn = self.open_shard(shard as u32, false)?;
let got = conn
.query_row(
"SELECT width, height, bytes FROM thumbs WHERE file_id = ?1",
[file_id as i64],
"SELECT width, height, bytes FROM thumbs WHERE file_id = ?1 AND size = ?2",
[file_id as i64, size as i64],
|r| {
Ok(Thumbnail {
width: r.get::<_, i64>(0)? as u32,
@@ -128,11 +183,11 @@ impl ThumbStore {
///
/// Cheaper than [`get`](Self::get) — the index alone answers it, with no
/// shard opened and no blob read.
pub fn contains(&self, file_id: u64) -> bool {
pub fn contains(&self, file_id: u64, size: ThumbSize) -> bool {
self.index
.query_row(
"SELECT 1 FROM entries WHERE file_id = ?1",
[file_id as i64],
"SELECT 1 FROM entries WHERE file_id = ?1 AND size = ?2",
[file_id as i64, size as i64],
|_| Ok(()),
)
.is_ok()
@@ -142,11 +197,11 @@ impl ThumbStore {
///
/// The grid's real question — "what must I fetch for these cells" — asked
/// in one pass rather than one query per cell.
pub fn missing(&self, file_ids: &[u64]) -> Vec<u64> {
pub fn missing(&self, file_ids: &[u64], size: ThumbSize) -> Vec<u64> {
file_ids
.iter()
.copied()
.filter(|id| !self.contains(*id))
.filter(|id| !self.contains(*id, size))
.collect()
}
@@ -155,13 +210,20 @@ impl ThumbStore {
/// Re-storing an existing id overwrites in place rather than migrating it
/// to the active shard: a sealed shard must stay byte-identical for other
/// clients, and rewriting one would force them all to re-sync it.
pub fn put(&mut self, file_id: u64, thumb: &Thumbnail) -> Result<u32, ThumbError> {
if let Some(existing) = self.shard_of(file_id) {
pub fn put(
&mut self,
file_id: u64,
size: ThumbSize,
thumb: &Thumbnail,
) -> Result<u32, ThumbError> {
if let Some(existing) = self.shard_of(file_id, size) {
let conn = self.open_shard(existing, true)?;
conn.execute(
"UPDATE thumbs SET width = ?2, height = ?3, bytes = ?4 WHERE file_id = ?1",
"UPDATE thumbs SET width = ?3, height = ?4, bytes = ?5
WHERE file_id = ?1 AND size = ?2",
rusqlite::params![
file_id as i64,
size as i64,
thumb.width as i64,
thumb.height as i64,
thumb.bytes
@@ -173,11 +235,13 @@ impl ThumbStore {
let shard = self.active_shard(thumb.bytes.len() as u64)?;
let conn = self.open_shard(shard, true)?;
conn.execute(
"INSERT INTO thumbs(file_id, width, height, bytes) VALUES (?1, ?2, ?3, ?4)
ON CONFLICT(file_id) DO UPDATE SET
"INSERT INTO thumbs(file_id, size, width, height, bytes)
VALUES (?1, ?2, ?3, ?4, ?5)
ON CONFLICT(file_id, size) DO UPDATE SET
width = excluded.width, height = excluded.height, bytes = excluded.bytes",
rusqlite::params![
file_id as i64,
size as i64,
thumb.width as i64,
thumb.height as i64,
thumb.bytes
@@ -185,9 +249,15 @@ impl ThumbStore {
)?;
self.index.execute(
"INSERT INTO entries(file_id, shard, bytes) VALUES (?1, ?2, ?3)
ON CONFLICT(file_id) DO UPDATE SET shard = excluded.shard, bytes = excluded.bytes",
rusqlite::params![file_id as i64, shard as i64, thumb.bytes.len() as i64],
"INSERT INTO entries(file_id, size, shard, bytes) VALUES (?1, ?2, ?3, ?4)
ON CONFLICT(file_id, size) DO UPDATE SET
shard = excluded.shard, bytes = excluded.bytes",
rusqlite::params![
file_id as i64,
size as i64,
shard as i64,
thumb.bytes.len() as i64
],
)?;
self.index.execute(
"INSERT INTO shards(id, bytes, sealed) VALUES (?1, ?2, 0)
@@ -198,11 +268,11 @@ impl ThumbStore {
Ok(shard)
}
fn shard_of(&self, file_id: u64) -> Option<u32> {
fn shard_of(&self, file_id: u64, size: ThumbSize) -> Option<u32> {
self.index
.query_row(
"SELECT shard FROM entries WHERE file_id = ?1",
[file_id as i64],
"SELECT shard FROM entries WHERE file_id = ?1 AND size = ?2",
[file_id as i64, size as i64],
|r| r.get::<_, i64>(0),
)
.ok()
@@ -299,31 +369,43 @@ impl ThumbStore {
let tx = self.index.unchecked_transaction()?;
for &file_id in file_ids {
let found: Option<(i64, i64, i64)> = tx
.query_row(
"SELECT e.shard, e.bytes, s.sealed
// Every size class, not just one: an image browsed at two sizes has
// two entries, and reading a single row would leave the other's
// bytes on the shard's tally forever — sealing it early on space
// nothing occupies.
let rows: Vec<(i64, i64, i64, i64)> = {
let mut stmt = tx.prepare(
"SELECT e.size, e.shard, e.bytes, s.sealed
FROM entries e JOIN shards s ON s.id = e.shard
WHERE e.file_id = ?1",
[file_id as i64],
|r| Ok((r.get(0)?, r.get(1)?, r.get(2)?)),
)
.ok();
let Some((shard, bytes, sealed)) = found else {
// Never stored, or already forgotten. Not an error: a purge runs
// over whatever the catalog knew about, and a thumbnail that was
// never fetched is the normal case.
continue;
)?;
let mapped = stmt.query_map([file_id as i64], |r| {
Ok((r.get(0)?, r.get(1)?, r.get(2)?, r.get(3)?))
})?;
mapped.collect::<Result<Vec<_>, _>>()?
};
tx.execute("DELETE FROM entries WHERE file_id = ?1", [file_id as i64])?;
forgotten += 1;
// Never stored, or already forgotten. Not an error: a purge runs
// over whatever the catalog knew about, and a thumbnail that was
// never fetched is the normal case.
if rows.is_empty() {
continue;
}
if sealed == 0 {
// The active shard is not yet anyone's cached copy, so its bytes
// can genuinely be reclaimed and the accounting corrected — which
// also means the shard does not seal prematurely on space that is
// no longer used.
for (size, shard, bytes, sealed) in rows {
tx.execute(
"DELETE FROM entries WHERE file_id = ?1 AND size = ?2",
[file_id as i64, size],
)?;
forgotten += 1;
if sealed != 0 {
continue;
}
// The active shard is not yet anyone's cached copy, so its
// bytes can genuinely be reclaimed and the accounting
// corrected — which also means the shard does not seal
// prematurely on space that is no longer used.
in_unsealed.push((file_id, shard as u32));
tx.execute(
"UPDATE shards SET bytes = max(0, bytes - ?2) WHERE id = ?1",
@@ -409,26 +491,41 @@ impl ThumbStore {
rusqlite::OpenFlags::SQLITE_OPEN_READ_ONLY | rusqlite::OpenFlags::SQLITE_OPEN_NO_MUTEX,
)?;
let mut stmt = src.prepare("SELECT file_id, width, height, bytes FROM thumbs")?;
// A shard from a client that predates the size class has no `size`
// column; its thumbnails are all grid-sized, which is what the
// fallback below assumes.
let has_size = src
.prepare("SELECT * FROM thumbs LIMIT 0")
.map(|stmt| stmt.column_names().iter().any(|c| *c == "size"))
.unwrap_or(false);
let sql = if has_size {
"SELECT file_id, size, width, height, bytes FROM thumbs"
} else {
"SELECT file_id, 0 AS size, width, height, bytes FROM thumbs"
};
let mut stmt = src.prepare(sql)?;
let incoming = stmt
.query_map([], |r| {
Ok((
r.get::<_, i64>(0)? as u64,
ThumbSize::from_i64(r.get::<_, i64>(1)?),
Thumbnail {
width: r.get::<_, i64>(1)? as u32,
height: r.get::<_, i64>(2)? as u32,
bytes: r.get(3)?,
width: r.get::<_, i64>(2)? as u32,
height: r.get::<_, i64>(3)? as u32,
bytes: r.get(4)?,
},
))
})?
.collect::<Result<Vec<_>, _>>()?;
let mut adopted = 0;
for (file_id, thumb) in incoming {
if self.contains(file_id) {
for (file_id, size, thumb) in incoming {
if self.contains(file_id, size) {
continue;
}
self.put(file_id, &thumb)?;
self.put(file_id, size, &thumb)?;
adopted += 1;
}
Ok(adopted)
@@ -446,9 +543,13 @@ pub struct ShardInfo {
const INDEX_SCHEMA: &str = r#"
CREATE TABLE IF NOT EXISTS entries (
file_id INTEGER PRIMARY KEY, -- oc:fileid, stable across rename/move
file_id INTEGER NOT NULL, -- oc:fileid, stable across rename/move
-- Which resolution. Part of the key so both classes coexist: adding the
-- large class must not invalidate a library already thumbnailed small.
size INTEGER NOT NULL DEFAULT 0,
shard INTEGER NOT NULL,
bytes INTEGER NOT NULL
bytes INTEGER NOT NULL,
PRIMARY KEY(file_id, size)
);
CREATE INDEX IF NOT EXISTS entries_shard ON entries(shard);
@@ -464,10 +565,65 @@ CREATE TABLE IF NOT EXISTS shards (
const SHARD_SCHEMA: &str = r#"
CREATE TABLE IF NOT EXISTS thumbs (
file_id INTEGER PRIMARY KEY,
file_id INTEGER NOT NULL,
size INTEGER NOT NULL DEFAULT 0,
width INTEGER NOT NULL,
height INTEGER NOT NULL,
bytes BLOB NOT NULL
bytes BLOB NOT NULL,
PRIMARY KEY(file_id, size)
);
"#;
/// Bring a store written before the size class existed up to date.
///
/// Those entries are all grid-sized, which is what the column defaults to, so
/// the migration is purely structural — no thumbnail is discarded and nothing
/// is re-fetched. A store that has never been opened by an older build runs
/// this as a no-op.
///
/// Sealed shards *are* rewritten here, which normally the design forbids
/// (their immutability is what makes syncing cheap). It is acceptable exactly
/// once: every client migrates the same way, and the alternative is discarding
/// every thumbnail already fetched.
fn migrate_size_column(conn: &Connection, table: &str) -> Result<(), ThumbError> {
let has_size: bool = conn
.prepare(&format!("SELECT * FROM {table} LIMIT 0"))
.map(|stmt| stmt.column_names().iter().any(|c| *c == "size"))
.unwrap_or(true);
if has_size {
return Ok(());
}
// SQLite cannot add a column to a primary key, so the table is rebuilt.
let tx = conn.unchecked_transaction()?;
tx.execute_batch(&format!(
"ALTER TABLE {table} RENAME TO {table}_old;
{}
INSERT INTO {table} SELECT file_id, 0, {cols} FROM {table}_old;
DROP TABLE {table}_old;",
if table == "entries" {
INDEX_ENTRIES_TABLE
} else {
SHARD_SCHEMA
},
cols = if table == "entries" {
"shard, bytes"
} else {
"width, height, bytes"
}
))?;
tx.commit()?;
Ok(())
}
/// The `entries` table alone, for the rebuild above.
const INDEX_ENTRIES_TABLE: &str = r#"
CREATE TABLE entries (
file_id INTEGER NOT NULL,
size INTEGER NOT NULL DEFAULT 0,
shard INTEGER NOT NULL,
bytes INTEGER NOT NULL,
PRIMARY KEY(file_id, size)
);
"#;
@@ -498,14 +654,14 @@ mod tests {
// The point of the whole method: a purged photograph must not keep a
// preview, on this device or any that syncs the shards.
let (mut s, _d) = store();
s.put(1, &thumb(1024)).unwrap();
s.put(2, &thumb(1024)).unwrap();
s.put(1, ThumbSize::Grid, &thumb(1024)).unwrap();
s.put(2, ThumbSize::Grid, &thumb(1024)).unwrap();
assert_eq!(s.forget(&[1]).unwrap(), 1);
assert!(s.get(1).unwrap().is_none());
assert!(!s.contains(1));
assert!(s.get(1, ThumbSize::Grid).unwrap().is_none());
assert!(!s.contains(1, ThumbSize::Grid));
// And its neighbour is untouched.
assert!(s.get(2).unwrap().is_some());
assert!(s.get(2, ThumbSize::Grid).unwrap().is_some());
assert_eq!(s.len(), 1);
}
@@ -514,7 +670,7 @@ mod tests {
// Otherwise the shard seals on space nothing is using, and the store
// grows a shard per purge.
let (mut s, _d) = store();
s.put(1, &thumb(4096)).unwrap();
s.put(1, ThumbSize::Grid, &thumb(4096)).unwrap();
let before = s.shards().unwrap()[0].bytes;
s.forget(&[1]).unwrap();
@@ -529,7 +685,7 @@ mod tests {
let (mut s, _d) = store();
let big = (SHARD_MAX_BYTES / 4) as usize;
for id in 0..5 {
s.put(id, &thumb(big)).unwrap();
s.put(id, ThumbSize::Grid, &thumb(big)).unwrap();
}
let shards = s.shards().unwrap();
assert!(shards[0].sealed, "precondition: shard 0 is sealed");
@@ -542,7 +698,7 @@ mod tests {
assert_eq!(s.forget(&[0]).unwrap(), 1);
// Unreachable through the index, which is what matters to a reader...
assert!(s.get(0).unwrap().is_none());
assert!(s.get(0, ThumbSize::Grid).unwrap().is_none());
// ...but the file itself is untouched.
assert_eq!(
std::fs::metadata(&sealed_path).unwrap().len(),
@@ -560,7 +716,7 @@ mod tests {
// A purge runs over whatever the catalog knew; most images never had a
// thumbnail fetched.
let (mut s, _d) = store();
s.put(1, &thumb(512)).unwrap();
s.put(1, ThumbSize::Grid, &thumb(512)).unwrap();
assert_eq!(s.forget(&[42, 43]).unwrap(), 0);
assert_eq!(s.forget(&[]).unwrap(), 0);
assert_eq!(s.len(), 1, "nothing else went");
@@ -570,7 +726,7 @@ mod tests {
fn forgetting_twice_is_idempotent() {
// An empty-trash retried after a partial failure runs over the same ids.
let (mut s, _d) = store();
s.put(1, &thumb(512)).unwrap();
s.put(1, ThumbSize::Grid, &thumb(512)).unwrap();
assert_eq!(s.forget(&[1]).unwrap(), 1);
assert_eq!(s.forget(&[1]).unwrap(), 0);
}
@@ -580,18 +736,101 @@ mod tests {
// Restoring from the server's own trashbin, or re-adding the same file:
// the id is stable, so the store must accept it back.
let (mut s, _d) = store();
s.put(1, &thumb(512)).unwrap();
s.put(1, ThumbSize::Grid, &thumb(512)).unwrap();
s.forget(&[1]).unwrap();
s.put(1, &thumb(512)).unwrap();
assert!(s.get(1).unwrap().is_some());
s.put(1, ThumbSize::Grid, &thumb(512)).unwrap();
assert!(s.get(1, ThumbSize::Grid).unwrap().is_some());
}
#[test]
fn the_two_size_classes_coexist() {
// Adding the large class must not evict or shadow the grid one: the
// same photograph is legitimately stored at both.
let (mut s, _d) = store();
s.put(7, ThumbSize::Grid, &thumb(100)).unwrap();
s.put(7, ThumbSize::Large, &thumb(900)).unwrap();
assert_eq!(s.get(7, ThumbSize::Grid).unwrap().unwrap().bytes.len(), 100);
assert_eq!(
s.get(7, ThumbSize::Large).unwrap().unwrap().bytes.len(),
900
);
assert_eq!(s.len(), 2, "counted separately");
}
#[test]
fn a_missing_large_is_not_satisfied_by_the_grid_one() {
// Otherwise a zoomed cell would silently show a 256px thumbnail
// upscaled, which is the softness the large class exists to avoid.
let (mut s, _d) = store();
s.put(7, ThumbSize::Grid, &thumb(100)).unwrap();
assert!(s.contains(7, ThumbSize::Grid));
assert!(!s.contains(7, ThumbSize::Large));
assert!(s.get(7, ThumbSize::Large).unwrap().is_none());
assert_eq!(s.missing(&[7], ThumbSize::Large), vec![7]);
}
#[test]
fn forgetting_an_image_drops_every_size() {
// A purged photograph must leave no preview at any size — the shards
// sync, so a survivor keeps appearing on every other client.
let (mut s, _d) = store();
s.put(7, ThumbSize::Grid, &thumb(100)).unwrap();
s.put(7, ThumbSize::Large, &thumb(900)).unwrap();
assert_eq!(s.forget(&[7]).unwrap(), 2, "both entries counted");
assert!(!s.contains(7, ThumbSize::Grid));
assert!(!s.contains(7, ThumbSize::Large));
assert_eq!(s.len(), 0);
}
#[test]
fn the_cell_size_picks_the_class() {
assert_eq!(ThumbSize::for_cell(180), ThumbSize::Grid);
assert_eq!(ThumbSize::for_cell(256), ThumbSize::Grid);
// Past the grid class's own edge, upscaling would show.
assert_eq!(ThumbSize::for_cell(257), ThumbSize::Large);
assert_eq!(ThumbSize::for_cell(400), ThumbSize::Large);
}
#[test]
fn a_store_written_before_the_size_class_keeps_its_thumbnails() {
// The migration case: an existing library must not lose the thumbnails
// it already paid to fetch, and its entries are all grid-sized.
let dir = std::env::temp_dir().join(format!("dr-thumbs-migrate-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
// An index in the old shape: no `size`, `file_id` alone as the key.
{
let c = Connection::open(dir.join("index.sqlite")).unwrap();
c.execute_batch(
"CREATE TABLE entries (
file_id INTEGER PRIMARY KEY, shard INTEGER NOT NULL, bytes INTEGER NOT NULL);
CREATE TABLE shards (
id INTEGER PRIMARY KEY, bytes INTEGER NOT NULL DEFAULT 0,
sealed INTEGER NOT NULL DEFAULT 0);
INSERT INTO shards(id, bytes, sealed) VALUES (0, 64, 0);
INSERT INTO entries(file_id, shard, bytes) VALUES (42, 0, 64);",
)
.unwrap();
}
let s = ThumbStore::open(&dir).unwrap();
assert!(
s.contains(42, ThumbSize::Grid),
"an existing entry survives as grid-sized"
);
assert_eq!(s.len(), 1);
}
#[test]
fn a_stored_thumbnail_round_trips() {
let (mut s, _d) = store();
s.put(1001, &thumb(1024)).unwrap();
s.put(1001, ThumbSize::Grid, &thumb(1024)).unwrap();
let got = s.get(1001).unwrap().expect("stored");
let got = s.get(1001, ThumbSize::Grid).unwrap().expect("stored");
assert_eq!(got.width, 256);
assert_eq!(got.bytes.len(), 1024);
}
@@ -599,30 +838,30 @@ mod tests {
#[test]
fn an_unknown_id_is_none_not_an_error() {
let (s, _d) = store();
assert!(s.get(9999).unwrap().is_none());
assert!(s.get(9999, ThumbSize::Grid).unwrap().is_none());
}
#[test]
fn contains_avoids_reading_the_blob() {
let (mut s, _d) = store();
s.put(1, &thumb(512)).unwrap();
assert!(s.contains(1));
assert!(!s.contains(2));
s.put(1, ThumbSize::Grid, &thumb(512)).unwrap();
assert!(s.contains(1, ThumbSize::Grid));
assert!(!s.contains(2, ThumbSize::Grid));
}
#[test]
fn missing_reports_only_what_is_absent() {
let (mut s, _d) = store();
s.put(1, &thumb(64)).unwrap();
s.put(3, &thumb(64)).unwrap();
assert_eq!(s.missing(&[1, 2, 3, 4]), vec![2, 4]);
s.put(1, ThumbSize::Grid, &thumb(64)).unwrap();
s.put(3, ThumbSize::Grid, &thumb(64)).unwrap();
assert_eq!(s.missing(&[1, 2, 3, 4], ThumbSize::Grid), vec![2, 4]);
}
#[test]
fn everything_small_lands_in_one_shard() {
let (mut s, _d) = store();
for id in 0..20 {
s.put(id, &thumb(1024)).unwrap();
s.put(id, ThumbSize::Grid, &thumb(1024)).unwrap();
}
assert_eq!(s.shards().unwrap().len(), 1);
assert_eq!(s.len(), 20);
@@ -634,7 +873,7 @@ mod tests {
// Quarter-cap blobs: the fifth cannot fit alongside four others.
let big = (SHARD_MAX_BYTES / 4) as usize;
for id in 0..5 {
s.put(id, &thumb(big)).unwrap();
s.put(id, ThumbSize::Grid, &thumb(big)).unwrap();
}
let shards = s.shards().unwrap();
@@ -651,15 +890,15 @@ mod tests {
let (mut s, _d) = store();
let big = (SHARD_MAX_BYTES / 4) as usize;
for id in 0..5 {
s.put(id, &thumb(big)).unwrap();
s.put(id, ThumbSize::Grid, &thumb(big)).unwrap();
}
// Small enough to fit in the sealed shard, but it must not go there.
s.put(100, &thumb(16)).unwrap();
s.put(100, ThumbSize::Grid, &thumb(16)).unwrap();
let shards = s.shards().unwrap();
assert!(shards[0].sealed);
assert_eq!(
s.shard_of(100),
s.shard_of(100, ThumbSize::Grid),
Some(1),
"new writes go to the open shard, not back into a sealed one"
);
@@ -672,13 +911,13 @@ mod tests {
let (mut s, _d) = store();
let big = (SHARD_MAX_BYTES / 4) as usize;
for id in 0..5 {
s.put(id, &thumb(big)).unwrap();
s.put(id, ThumbSize::Grid, &thumb(big)).unwrap();
}
let original = s.shard_of(0).unwrap();
let original = s.shard_of(0, ThumbSize::Grid).unwrap();
s.put(0, &thumb(32)).unwrap();
assert_eq!(s.shard_of(0), Some(original), "must stay put");
assert_eq!(s.get(0).unwrap().unwrap().bytes.len(), 32, "but update");
s.put(0, ThumbSize::Grid, &thumb(32)).unwrap();
assert_eq!(s.shard_of(0, ThumbSize::Grid), Some(original), "must stay put");
assert_eq!(s.get(0, ThumbSize::Grid).unwrap().unwrap().bytes.len(), 32, "but update");
}
#[test]
@@ -691,35 +930,35 @@ mod tests {
#[test]
fn reopening_finds_what_was_stored() {
let (mut s, dir) = store();
s.put(42, &thumb(256)).unwrap();
s.put(42, ThumbSize::Grid, &thumb(256)).unwrap();
drop(s);
let reopened = ThumbStore::open(&dir).unwrap();
assert!(reopened.contains(42));
assert!(reopened.contains(42, ThumbSize::Grid));
assert_eq!(reopened.len(), 1);
}
#[test]
fn merging_another_clients_shard_adopts_only_what_is_new() {
let (mut mine, _d1) = store();
mine.put(1, &thumb(64)).unwrap();
mine.put(1, ThumbSize::Grid, &thumb(64)).unwrap();
// A second store standing in for another device's downloaded shard.
let other_dir = std::env::temp_dir().join(format!("dr-thumbs-other-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&other_dir);
let mut theirs = ThumbStore::open(&other_dir).unwrap();
theirs.put(1, &thumb(999)).unwrap(); // we already have this one
theirs.put(2, &thumb(64)).unwrap();
theirs.put(3, &thumb(64)).unwrap();
theirs.put(1, ThumbSize::Grid, &thumb(999)).unwrap(); // we already have this one
theirs.put(2, ThumbSize::Grid, &thumb(64)).unwrap();
theirs.put(3, ThumbSize::Grid, &thumb(64)).unwrap();
let adopted = mine.merge_shard(&theirs.shard_path(0)).unwrap();
assert_eq!(adopted, 2, "only the two we lacked");
assert_eq!(
mine.get(1).unwrap().unwrap().bytes.len(),
mine.get(1, ThumbSize::Grid).unwrap().unwrap().bytes.len(),
64,
"ours is kept, not overwritten"
);
assert!(mine.contains(2) && mine.contains(3));
assert!(mine.contains(2, ThumbSize::Grid) && mine.contains(3, ThumbSize::Grid));
}
#[test]
@@ -729,7 +968,7 @@ mod tests {
std::env::temp_dir().join(format!("dr-thumbs-idem-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&other_dir);
let mut theirs = ThumbStore::open(&other_dir).unwrap();
theirs.put(10, &thumb(64)).unwrap();
theirs.put(10, ThumbSize::Grid, &thumb(64)).unwrap();
assert_eq!(mine.merge_shard(&theirs.shard_path(0)).unwrap(), 1);
assert_eq!(