dtourolle/DarkRoom
1
0
Fork 0
Code Issues 39 Pull Requests Actions Packages Projects Releases 20 Wiki Activity

Keep originals on this device, by pin and by use
Build and test / Desktop (Linux) (push) Failing after 1s
Details
Build and test / Android (aarch64) (push) Failing after 0s
Details
Build and test / Layer separation (push) Failing after 1s
Details
Traceability / Requirement traces (push) Failing after 2s
Details

Browse Source 
Fills in `image_cache`, which the previous commit's "On this device" filter
read but nothing wrote. Also carries in-flight work that shared these files:
the Android TLS root store, the settings page, and a regenerated
traceability report.

# Two populations, deliberately separate

An original is kept here for one of two reasons, and conflating them produces
the exact failure the feature exists to prevent.

**Pinned** originals were asked for. Pinning a collection before a trip is a
promise, so pinned rows are never evicted and never counted against the
budget — a cap that could silently delete a pinned trip would make pinning
worthless, because it could not be relied on without checking.

**Passively cached** originals are a side effect of working: develop already
downloads the whole file, so keeping it costs no bandwidth and saves the
entire transfer next time. This population is what the budget bounds, evicted
least-recently-used, because it otherwise grows until a day of culling fills
a disk.

Sharing one budget would let a large pin starve the passive cache, or let
browsing evict a pin. They are separate.

# What was built

`dr_catalog::cache` owns the bookkeeping — held tier, size, last use, pinned
— and writes the bytes; deciding to download stays with the caller, which is
what keeps a crate with no network out of the network's business. Files are
written to a temporary and renamed, so a dropped connection cannot leave a
truncated file recorded as a complete original. They are named by image id,
not filename: `Photos/IMG_0001.CR2` and `Trips/IMG_0001.CR2` are different
photographs, and a flat cache keyed on the name would serve one for the other.

`spawn_full_fetch` became read-through. A hit is a disk read; a miss stores
what it downloads and enforces the budget. A cache that cannot be opened is a
miss, not a failure to open the photograph.

Pinning writes intent — `tier_desired` — without downloading, so the button
responds immediately, and `spawn_pin_fetch` fills it in sequentially
afterwards. Sequential because these are tens of megabytes each: the lanes
that make the thumbnail sweep fast buy little against one connection's
bandwidth and cost a great deal of memory. A pin interrupted by a lost
connection resumes from where it stopped.

Schema v5 adds `pinned` and `path`. `pinned` is a column rather than something
inferred from `pinned_by_rule`, which is ON DELETE SET NULL and so cannot
answer for an image whose rule was deleted. A v4 catalog migrates in place;
existing rows default to unpinned, the safe direction.

The budget and "keep opened originals" come from the settings page rather than
a constant, and are applied at startup rather than only on change — a cache
capped at 2 GB last session would otherwise spend this one filling to the
default. Turning off keeping leaves what is already cached readable: those
bytes are paid for, and refusing them would re-download images sitting right
there, including pinned ones.

Also removes a doubled `#[test]` introduced in the previous commit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
dtourolle
2026-08-11 21:12:01 +02:00
co-authored by Claude Opus 5
parent cd75e5a4c6
commit fa12afed18
22 changed files with 4032 additions and 86 deletions
Download Patch File Download Diff File Expand all files Collapse all files
core/dr-catalog/src/cache.rs
+782
View File
@@ -0,0 +1,782 @@
//! TRACES: FR-NC-6a | FR-CAT-9 | NFR-RES-4
//! Which originals are kept on this device, and which may be evicted.
//!
//! # Two populations, one table
//!
//! An original ends up here two ways, and conflating them produces exactly the
//! failure the whole feature exists to prevent.
//!
//! **Pinned** originals were asked for. A user pins a collection before a trip
//! and expects those photographs to be there when there is no connection —
//! that is a promise, so pinned rows are never evicted and never counted
//! against the budget. A cap that could silently delete a pinned trip would
//! make pinning worthless, because the user could not rely on it without
//! checking.
//!
//! **Passively cached** originals are a side effect of working: opening an
//! image in develop downloads it, so keeping the bytes costs nothing extra and
//! saves the whole transfer next time. This population is bounded by
//! [`Budget`] and evicted least-recently-used, because it grows without limit
//! otherwise — a day of culling would fill a disk.
//!
//! The two budgets are separate rather than shared. Sharing them means a large
//! pin starves the passive cache, or worse, that browsing evicts a pin.
//!
//! # What this module does and does not own
//!
//! It owns the *bookkeeping*: which images are held, at what tier, how large,
//! when last used, and which are pinned. The bytes are files under a cache
//! directory, and [`store`](Cache::store) writes them; but deciding to
//! download something is the caller's business, because that needs a network
//! and this crate has none.
//!
//! # Why `tier_actual` is the truth
//!
//! `tier_desired` is what a pin asks for; `tier_actual` is what is on disk.
//! Only the second answers "can this be opened right now", which is the
//! question offline mode asks (FR-CAT-9). A pinned image whose download has
//! not run yet is precisely the one that would fail, so it must not report as
//! available.
use std::path::{Path, PathBuf};
use dr_types::{ImageId, Tier};
use rusqlite::Connection;
use crate::error::CatalogError;
/// Default ceiling for passively cached originals.
///
/// 1 GB holds roughly 30 full-frame RAWs — a working session's worth, which is
/// what this cache is for. It is deliberately modest: the passive cache is a
/// convenience that should not quietly consume a disk, and a user who wants
/// more kept is better served by pinning, which says so explicitly and is not
/// subject to eviction at all.
pub const DEFAULT_BUDGET_BYTES: u64 = 1024 * 1024 * 1024;
/// How much disk the passive cache may use.
///
/// A newtype rather than a bare `u64` so a byte count cannot be passed where a
/// budget belongs, and to give the "unlimited" case a name — some users have a
/// large disk and would rather never re-download.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Budget(Option<u64>);
impl Default for Budget {
fn default() -> Self {
Self::bytes(DEFAULT_BUDGET_BYTES)
}
}
impl Budget {
pub fn bytes(n: u64) -> Self {
Self(Some(n))
}
/// No ceiling: nothing is ever evicted for space.
pub fn unlimited() -> Self {
Self(None)
}
pub fn limit(self) -> Option<u64> {
self.0
}
/// How much must be freed to fit `used` within this budget.
fn overage(self, used: u64) -> u64 {
self.0.map_or(0, |cap| used.saturating_sub(cap))
}
}
/// What is held for one image.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Entry {
pub image: ImageId,
/// What is actually on disk.
pub tier: Tier,
/// What a pin has asked for, which may be ahead of `tier`.
pub desired: Tier,
pub bytes: u64,
/// Unix seconds, or `None` if never read back since being stored.
pub last_used: Option<i64>,
pub pinned: bool,
/// Path relative to the cache directory.
pub path: Option<String>,
}
/// How the cache is currently filled.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub struct Usage {
/// Bytes held by pinned originals. Not subject to the budget.
pub pinned_bytes: u64,
/// Bytes held by passively cached originals. What the budget bounds.
pub passive_bytes: u64,
pub pinned_count: usize,
pub passive_count: usize,
}
impl Usage {
pub fn total_bytes(self) -> u64 {
self.pinned_bytes + self.passive_bytes
}
}
/// The on-disk cache of originals, rooted at a directory.
pub struct Cache {
dir: PathBuf,
budget: Budget,
}
impl Cache {
/// Open a cache rooted at `dir`, creating it if needed.
pub fn open(dir: &Path, budget: Budget) -> Result<Self, CatalogError> {
std::fs::create_dir_all(dir)
.map_err(|e| CatalogError::Io(format!("creating {}: {e}", dir.display())))?;
Ok(Self {
dir: dir.to_path_buf(),
budget,
})
}
pub fn dir(&self) -> &Path {
&self.dir
}
pub fn budget(&self) -> Budget {
self.budget
}
/// Absolute path for a cached original.
///
/// Named by image id rather than by the remote filename: two folders on
/// the server may hold `IMG_0001.CR2`, and a flat cache keyed on the name
/// would have them overwrite each other. The extension is preserved so the
/// decoder's format probe sees what it expects.
fn relative_path(image: ImageId, source_ref: &str) -> String {
let ext = source_ref
.rsplit_once('.')
.map(|(_, e)| e.to_ascii_lowercase())
.filter(|e| !e.is_empty() && e.len() <= 8 && e.chars().all(|c| c.is_ascii_alphanumeric()))
.unwrap_or_else(|| "bin".to_string());
format!("{}.{ext}", image.0)
}
/// Store an original's bytes and record it.
///
/// `pinned` says which population this belongs to. Storing an image that
/// is already present updates it rather than duplicating — the same
/// photograph opened twice is one cache entry, and the second store simply
/// refreshes the bytes and the timestamp.
///
/// Does **not** evict. The caller runs [`enforce`](Self::enforce) once it
/// has finished storing, so a batch of downloads is trimmed once rather
/// than after every file.
pub fn store(
&self,
conn: &Connection,
image: ImageId,
source_ref: &str,
bytes: &[u8],
pinned: bool,
now: i64,
) -> Result<(), CatalogError> {
let rel = Self::relative_path(image, source_ref);
let abs = self.dir.join(&rel);
// Written to a temporary and renamed, so a crash or a dropped
// connection mid-write cannot leave a truncated file that the catalog
// records as a complete original — which would then fail to decode
// with no indication that the *cache* was at fault rather than the
// photograph.
let tmp = abs.with_extension("partial");
std::fs::write(&tmp, bytes)
.map_err(|e| CatalogError::Io(format!("writing {}: {e}", tmp.display())))?;
std::fs::rename(&tmp, &abs)
.map_err(|e| CatalogError::Io(format!("renaming {}: {e}", abs.display())))?;
// `pinned` is OR-ed rather than assigned: an image that was already
// pinned must not be demoted to evictable because it happened to be
// opened in develop, which is a passive store.
conn.execute(
"INSERT INTO image_cache
(image_id, tier_actual, tier_desired, bytes, last_used, pinned, path)
VALUES (?1, ?2, ?2, ?3, ?4, ?5, ?6)
ON CONFLICT(image_id) DO UPDATE SET
tier_actual = ?2,
tier_desired = max(tier_desired, ?2),
bytes = ?3,
last_used = ?4,
pinned = max(pinned, ?5),
path = ?6",
rusqlite::params![
image.0 as i64,
Tier::Original.stored(),
bytes.len() as i64,
now,
i64::from(pinned),
rel,
],
)?;
Ok(())
}
/// Read a cached original back, if it is here.
///
/// Touches `last_used`, which is what makes the eviction order reflect
/// actual use rather than download order. A read that finds the row but
/// not the file repairs the catalog rather than returning bytes it does
/// not have — the two can diverge if a user clears the directory by hand.
pub fn load(
&self,
conn: &Connection,
image: ImageId,
now: i64,
) -> Result<Option<Vec<u8>>, CatalogError> {
let path: Option<String> = conn
.query_row(
"SELECT path FROM image_cache
WHERE image_id = ?1 AND tier_actual >= ?2",
rusqlite::params![image.0 as i64, Tier::Original.stored()],
|r| r.get(0),
)
.ok()
.flatten();
let Some(rel) = path else { return Ok(None) };
let abs = self.dir.join(&rel);
match std::fs::read(&abs) {
Ok(bytes) => {
conn.execute(
"UPDATE image_cache SET last_used = ?2 WHERE image_id = ?1",
rusqlite::params![image.0 as i64, now],
)?;
Ok(Some(bytes))
}
Err(e) => {
// The file is gone but the row says it is here. Believing the
// row would report the image as locally available for ever
// while every open failed.
log::debug!("cached original {} missing, forgetting it: {e}", abs.display());
self.forget(conn, &[image])?;
Ok(None)
}
}
}
/// Whether an image's original is on this device.
pub fn holds_original(&self, conn: &Connection, image: ImageId) -> bool {
conn.query_row(
"SELECT 1 FROM image_cache
WHERE image_id = ?1 AND tier_actual >= ?2",
rusqlite::params![image.0 as i64, Tier::Original.stored()],
|_| Ok(()),
)
.is_ok()
}
/// Mark images as pinned, so they are kept regardless of the budget.
///
/// Pinning records the *intent* — `tier_desired` — without downloading
/// anything: the download needs a network, which belongs to the caller.
/// An image already cached passively becomes pinned in place, keeping its
/// bytes rather than re-fetching them.
pub fn pin(&self, conn: &Connection, images: &[ImageId]) -> Result<usize, CatalogError> {
self.set_pinned(conn, images, true)
}
/// Release a pin, returning those images to the evictable population.
///
/// The bytes stay until eviction needs the room. Deleting immediately
/// would make unpinning destructive, when it is meant only to withdraw a
/// guarantee.
pub fn unpin(&self, conn: &Connection, images: &[ImageId]) -> Result<usize, CatalogError> {
self.set_pinned(conn, images, false)
}
fn set_pinned(
&self,
conn: &Connection,
images: &[ImageId],
pinned: bool,
) -> Result<usize, CatalogError> {
if images.is_empty() {
return Ok(0);
}
let tx = conn.unchecked_transaction()?;
let mut n = 0;
for image in images {
n += tx.execute(
"INSERT INTO image_cache (image_id, tier_actual, tier_desired, bytes, pinned)
VALUES (?1, ?2, ?3, 0, ?4)
ON CONFLICT(image_id) DO UPDATE SET
pinned = ?4,
-- A pin raises the target; releasing one lowers it back to
-- whatever is actually held, so a released image is not
-- left permanently claiming it wants an original.
tier_desired = CASE WHEN ?4 = 1 THEN ?3 ELSE tier_actual END",
rusqlite::params![
image.0 as i64,
Tier::Metadata.stored(),
Tier::Original.stored(),
i64::from(pinned),
],
)?;
}
tx.commit()?;
Ok(n)
}
/// Images a pin wants but which are not yet downloaded.
///
/// The work list for whatever fetches originals. Ordered by id for a
/// stable, resumable sequence rather than an arbitrary one.
pub fn pending_pins(&self, conn: &Connection) -> Result<Vec<ImageId>, CatalogError> {
let mut stmt = conn.prepare(
"SELECT image_id FROM image_cache
WHERE pinned = 1 AND tier_actual < tier_desired
ORDER BY image_id",
)?;
let rows = stmt
.query_map([], |r| Ok(ImageId(r.get::<_, i64>(0)? as u64)))?
.collect::<Result<Vec<_>, _>>()?;
Ok(rows)
}
/// How full the cache is, split by population.
///
/// Counts only rows that actually hold an original: a pin that has not
/// downloaded yet occupies no disk, and counting its intent would evict
/// real files to make room for bytes that do not exist.
pub fn usage(&self, conn: &Connection) -> Result<Usage, CatalogError> {
let mut stmt = conn.prepare(
"SELECT pinned, count(*), coalesce(sum(bytes), 0)
FROM image_cache
WHERE tier_actual >= ?1
GROUP BY pinned",
)?;
let mut usage = Usage::default();
let rows = stmt.query_map(rusqlite::params![Tier::Original.stored()], |r| {
Ok((r.get::<_, i64>(0)?, r.get::<_, i64>(1)?, r.get::<_, i64>(2)?))
})?;
for row in rows {
let (pinned, count, bytes) = row?;
if pinned == 1 {
usage.pinned_count = count as usize;
usage.pinned_bytes = bytes as u64;
} else {
usage.passive_count = count as usize;
usage.passive_bytes = bytes as u64;
}
}
Ok(usage)
}
/// Evict least-recently-used passive entries until the budget is met.
///
/// Returns how many images were dropped. Pinned entries are never
/// candidates, which is the guarantee that makes a pin worth making.
///
/// A row whose file has already vanished is still dropped from the
/// catalog: it frees no disk, but leaving it would let a phantom entry
/// hold the cache permanently over budget and evict real files in its
/// place.
pub fn enforce(&self, conn: &Connection) -> Result<usize, CatalogError> {
let usage = self.usage(conn)?;
let mut over = self.budget.overage(usage.passive_bytes);
if over == 0 {
return Ok(0);
}
// Oldest first. `last_used IS NULL` sorts first deliberately: a row
// that has never been read back is the least valuable thing here.
let mut stmt = conn.prepare(
"SELECT image_id, bytes, path FROM image_cache
WHERE pinned = 0 AND tier_actual >= ?1
ORDER BY last_used IS NULL DESC, last_used ASC",
)?;
let candidates = stmt
.query_map(rusqlite::params![Tier::Original.stored()], |r| {
Ok((
ImageId(r.get::<_, i64>(0)? as u64),
r.get::<_, i64>(1)? as u64,
r.get::<_, Option<String>>(2)?,
))
})?
.collect::<Result<Vec<_>, _>>()?;
let mut evicted = Vec::new();
for (image, bytes, path) in candidates {
if over == 0 {
break;
}
if let Some(rel) = path {
let abs = self.dir.join(rel);
if let Err(e) = std::fs::remove_file(&abs) {
// Already gone is the common case and not a failure; the
// row still has to go, or it accounts for space nothing
// occupies.
log::debug!("evicting {}: {e}", abs.display());
}
}
over = over.saturating_sub(bytes);
evicted.push(image);
}
let n = evicted.len();
self.forget(conn, &evicted)?;
Ok(n)
}
/// Drop cache rows, without touching files.
///
/// The row is reduced to `Metadata` rather than deleted, so a pin recorded
/// against it survives: unpinning is the only thing that should clear a
/// pin, and eviction of the bytes is not unpinning.
fn forget(&self, conn: &Connection, images: &[ImageId]) -> Result<(), CatalogError> {
if images.is_empty() {
return Ok(());
}
let tx = conn.unchecked_transaction()?;
for image in images {
tx.execute(
"UPDATE image_cache
SET tier_actual = ?2, bytes = 0, path = NULL
WHERE image_id = ?1",
rusqlite::params![image.0 as i64, Tier::Metadata.stored()],
)?;
}
tx.commit()?;
Ok(())
}
/// Everything currently held, newest use first. For a cache management view.
pub fn entries(&self, conn: &Connection) -> Result<Vec<Entry>, CatalogError> {
let mut stmt = conn.prepare(
"SELECT image_id, tier_actual, tier_desired, bytes, last_used, pinned, path
FROM image_cache
WHERE tier_actual >= ?1
ORDER BY last_used IS NULL, last_used DESC",
)?;
let rows = stmt
.query_map(rusqlite::params![Tier::Original.stored()], |r| {
Ok(Entry {
image: ImageId(r.get::<_, i64>(0)? as u64),
tier: Tier::from_stored(r.get(1)?),
desired: Tier::from_stored(r.get(2)?),
bytes: r.get::<_, i64>(3)? as u64,
last_used: r.get(4)?,
pinned: r.get::<_, i64>(5)? == 1,
path: r.get(6)?,
})
})?
.collect::<Result<Vec<_>, _>>()?;
Ok(rows)
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::Catalog;
/// Distinguishes concurrent fixtures. The harness runs tests in parallel,
/// and a shared directory would have one test's eviction delete another's
/// files.
static SEQ: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(0);
/// A scratch directory that is fresh for each call.
fn tempdir() -> PathBuf {
let base = std::env::temp_dir().join(format!(
"dr-cache-test-{}-{}",
std::process::id(),
SEQ.fetch_add(1, std::sync::atomic::Ordering::Relaxed)
));
let _ = std::fs::remove_dir_all(&base);
std::fs::create_dir_all(&base).unwrap();
base
}
/// A catalog with `n` images, and a cache in a scratch directory.
fn fixture(n: usize) -> (Catalog, Cache, PathBuf, Vec<ImageId>) {
fixture_with(n, Budget::bytes(1000))
}
fn fixture_with(n: usize, budget: Budget) -> (Catalog, Cache, PathBuf, Vec<ImageId>) {
let catalog = Catalog::in_memory().unwrap();
catalog
.connection()
.execute(
"INSERT INTO roots (id, kind, label) VALUES (1, 'remote', 'test')",
[],
)
.unwrap();
let mut ids = Vec::new();
for i in 0..n {
catalog
.connection()
.execute(
"INSERT INTO images (root_id, source_ref, added_at)
VALUES (1, ?1, 0)",
rusqlite::params![format!("Photos/img{i:03}.CR2")],
)
.unwrap();
ids.push(ImageId(
catalog.connection().last_insert_rowid() as u64
));
}
let dir = tempdir();
let cache = Cache::open(&dir, budget).unwrap();
(catalog, cache, dir, ids)
}
#[test]
fn a_stored_original_reads_back() {
let (cat, cache, _dir, ids) = fixture(1);
cache
.store(cat.connection(), ids[0], "a.CR2", b"raw bytes", false, 10)
.unwrap();
assert!(cache.holds_original(cat.connection(), ids[0]));
assert_eq!(
cache.load(cat.connection(), ids[0], 20).unwrap().as_deref(),
Some(&b"raw bytes"[..])
);
}
#[test]
fn an_image_never_stored_is_absent() {
let (cat, cache, _dir, ids) = fixture(1);
assert!(!cache.holds_original(cat.connection(), ids[0]));
assert_eq!(cache.load(cat.connection(), ids[0], 0).unwrap(), None);
}
#[test]
fn eviction_takes_the_least_recently_used_first() {
let (cat, cache, _dir, ids) = fixture(3);
// 400 each against a 1000 budget: storing the third puts it 200 over.
let bytes = vec![0u8; 400];
cache.store(cat.connection(), ids[0], "a.CR2", &bytes, false, 10).unwrap();
cache.store(cat.connection(), ids[1], "b.CR2", &bytes, false, 20).unwrap();
cache.store(cat.connection(), ids[2], "c.CR2", &bytes, false, 30).unwrap();
// Touch the oldest so it is no longer the least recently used.
cache.load(cat.connection(), ids[0], 40).unwrap();
assert_eq!(cache.enforce(cat.connection()).unwrap(), 1);
// ids[1] was the stalest by the time eviction ran.
assert!(!cache.holds_original(cat.connection(), ids[1]));
assert!(cache.holds_original(cat.connection(), ids[0]));
assert!(cache.holds_original(cat.connection(), ids[2]));
}
#[test]
fn a_pinned_original_is_never_evicted() {
// The guarantee the whole feature rests on: a pinned trip must still
// be there after a day of browsing pushes the cache over its cap.
let (cat, cache, _dir, ids) = fixture(3);
let bytes = vec![0u8; 800];
cache.store(cat.connection(), ids[0], "a.CR2", &bytes, true, 10).unwrap();
cache.store(cat.connection(), ids[1], "b.CR2", &bytes, false, 20).unwrap();
cache.store(cat.connection(), ids[2], "c.CR2", &bytes, false, 30).unwrap();
cache.enforce(cat.connection()).unwrap();
assert!(
cache.holds_original(cat.connection(), ids[0]),
"the pinned original survives even though it is the oldest"
);
}
#[test]
fn pinned_bytes_do_not_count_against_the_budget() {
// Otherwise a large pin starves the passive cache into evicting
// everything, and browsing becomes uncacheable the moment a trip is
// pinned.
let (cat, cache, _dir, ids) = fixture(2);
cache
.store(cat.connection(), ids[0], "a.CR2", &vec![0u8; 5000], true, 10)
.unwrap();
cache
.store(cat.connection(), ids[1], "b.CR2", &vec![0u8; 500], false, 20)
.unwrap();
// Pinned use is far past the 1000 budget, but the passive 500 fits.
assert_eq!(cache.enforce(cat.connection()).unwrap(), 0);
assert!(cache.holds_original(cat.connection(), ids[1]));
let usage = cache.usage(cat.connection()).unwrap();
assert_eq!(usage.pinned_bytes, 5000);
assert_eq!(usage.passive_bytes, 500);
}
#[test]
fn an_unlimited_budget_evicts_nothing() {
let (cat, cache, _dir, ids) = fixture_with(2, Budget::unlimited());
for (i, id) in ids.iter().enumerate() {
cache
.store(cat.connection(), *id, "a.CR2", &vec![0u8; 100_000], false, i as i64)
.unwrap();
}
assert_eq!(cache.enforce(cat.connection()).unwrap(), 0);
}
#[test]
fn pinning_records_intent_without_bytes() {
// A pin is not a download: it says what should be here, and something
// with a network makes it so.
let (cat, cache, _dir, ids) = fixture(2);
cache.pin(cat.connection(), &ids).unwrap();
assert!(!cache.holds_original(cat.connection(), ids[0]));
assert_eq!(cache.pending_pins(cat.connection()).unwrap(), ids);
assert_eq!(cache.usage(cat.connection()).unwrap().pinned_bytes, 0);
}
#[test]
fn a_downloaded_pin_stops_being_pending() {
let (cat, cache, _dir, ids) = fixture(2);
cache.pin(cat.connection(), &ids).unwrap();
cache
.store(cat.connection(), ids[0], "a.CR2", b"bytes", true, 10)
.unwrap();
assert_eq!(cache.pending_pins(cat.connection()).unwrap(), vec![ids[1]]);
}
#[test]
fn pinning_an_already_cached_image_keeps_its_bytes() {
// Re-downloading something already on disk because the user pinned it
// would be the most visible possible waste.
let (cat, cache, _dir, ids) = fixture(1);
cache
.store(cat.connection(), ids[0], "a.CR2", b"raw bytes", false, 10)
.unwrap();
cache.pin(cat.connection(), &ids).unwrap();
assert!(cache.pending_pins(cat.connection()).unwrap().is_empty());
assert_eq!(
cache.load(cat.connection(), ids[0], 20).unwrap().as_deref(),
Some(&b"raw bytes"[..])
);
assert_eq!(cache.usage(cat.connection()).unwrap().pinned_bytes, 9);
}
#[test]
fn opening_a_pinned_image_does_not_unpin_it() {
// The develop path stores passively. If that overwrote `pinned`, then
// simply *looking at* a pinned photograph would silently make it
// evictable — the pin would decay through use.
let (cat, cache, _dir, ids) = fixture(1);
cache.pin(cat.connection(), &ids).unwrap();
cache
.store(cat.connection(), ids[0], "a.CR2", b"bytes", false, 10)
.unwrap();
let entries = cache.entries(cat.connection()).unwrap();
assert!(entries[0].pinned, "still pinned after a passive store");
}
#[test]
fn unpinning_keeps_the_bytes_but_makes_them_evictable() {
let (cat, cache, _dir, ids) = fixture(2);
cache
.store(cat.connection(), ids[0], "a.CR2", &vec![0u8; 800], true, 10)
.unwrap();
cache.unpin(cat.connection(), &ids[0..1]).unwrap();
// Still here — unpinning withdraws a guarantee, it does not delete.
assert!(cache.holds_original(cat.connection(), ids[0]));
// But now it is a candidate.
cache
.store(cat.connection(), ids[1], "b.CR2", &vec![0u8; 800], false, 20)
.unwrap();
assert_eq!(cache.enforce(cat.connection()).unwrap(), 1);
assert!(!cache.holds_original(cat.connection(), ids[0]));
}
#[test]
fn a_missing_file_is_forgotten_rather_than_reported_present() {
// A user clearing the cache directory by hand must not leave every
// image claiming to be local while every open fails.
let (cat, cache, dir, ids) = fixture_with(1, Budget::bytes(1000));
cache
.store(cat.connection(), ids[0], "a.CR2", b"bytes", false, 10)
.unwrap();
for entry in std::fs::read_dir(&dir).unwrap() {
std::fs::remove_file(entry.unwrap().path()).unwrap();
}
assert_eq!(cache.load(cat.connection(), ids[0], 20).unwrap(), None);
assert!(!cache.holds_original(cat.connection(), ids[0]));
}
#[test]
fn storing_the_same_image_twice_is_one_entry() {
let (cat, cache, _dir, ids) = fixture(1);
cache.store(cat.connection(), ids[0], "a.CR2", b"first", false, 10).unwrap();
cache.store(cat.connection(), ids[0], "a.CR2", b"second try", false, 20).unwrap();
let usage = cache.usage(cat.connection()).unwrap();
assert_eq!(usage.passive_count, 1);
assert_eq!(usage.passive_bytes, 10, "the later size, not the sum");
assert_eq!(
cache.load(cat.connection(), ids[0], 30).unwrap().as_deref(),
Some(&b"second try"[..])
);
}
#[test]
fn two_images_with_the_same_filename_do_not_collide() {
// `Photos/IMG_0001.CR2` and `Trips/IMG_0001.CR2` are different
// photographs; a cache keyed on the filename would serve one for the
// other, which is the worst failure this cache could have.
let (cat, cache, _dir, ids) = fixture(2);
cache
.store(cat.connection(), ids[0], "Photos/IMG_0001.CR2", b"first", false, 10)
.unwrap();
cache
.store(cat.connection(), ids[1], "Trips/IMG_0001.CR2", b"second", false, 20)
.unwrap();
assert_eq!(
cache.load(cat.connection(), ids[0], 30).unwrap().as_deref(),
Some(&b"first"[..])
);
assert_eq!(
cache.load(cat.connection(), ids[1], 30).unwrap().as_deref(),
Some(&b"second"[..])
);
}
#[test]
fn eviction_stops_once_the_budget_is_met() {
// Evicting everything on a small overage would throw away a working
// set to reclaim a few bytes.
let (cat, cache, _dir, ids) = fixture(3);
for (i, id) in ids.iter().enumerate() {
cache
.store(cat.connection(), *id, "a.CR2", &vec![0u8; 400], false, i as i64)
.unwrap();
}
// 1200 held against 1000: dropping one 400-byte entry suffices.
assert_eq!(cache.enforce(cat.connection()).unwrap(), 1);
assert_eq!(cache.usage(cat.connection()).unwrap().passive_count, 2);
}
#[test]
fn an_extensionless_source_still_gets_a_path() {
let (cat, cache, _dir, ids) = fixture(1);
cache
.store(cat.connection(), ids[0], "Photos/no-extension", b"bytes", false, 10)
.unwrap();
assert_eq!(
cache.load(cat.connection(), ids[0], 20).unwrap().as_deref(),
Some(&b"bytes"[..])
);
}
}
core/dr-catalog/src/lib.rs
+2
View File
@@ -30,6 +30,7 @@ use std::path::Path;
use dr_types::{Availability, ImageId};
use rusqlite::Connection;
pub mod cache;
pub mod collections;
pub mod error;
pub mod jobs;
@@ -41,6 +42,7 @@ pub mod schema;
pub mod sync;
pub mod trash;
pub use cache::{Budget, Cache, DEFAULT_BUDGET_BYTES};
pub use collections::{Collection, CollectionKind, TreeRow};
pub use error::CatalogError;
pub use jobs::{Job, JobKind, Priority};
core/dr-catalog/src/schema.rs
+80 -1
View File
@@ -15,7 +15,7 @@ use rusqlite::Connection;
use crate::error::CatalogError;
/// Schema version this build writes and understands.
pub const SCHEMA_VERSION: i64 = 4;
pub const SCHEMA_VERSION: i64 = 5;
/// Apply migrations up to [`SCHEMA_VERSION`].
///
@@ -60,6 +60,12 @@ pub fn migrate(conn: &Connection) -> Result<i64, CatalogError> {
tx.pragma_update(None, "user_version", 4)?;
tx.commit()?;
}
if from < 5 {
let tx = conn.unchecked_transaction()?;
tx.execute_batch(V5)?;
tx.pragma_update(None, "user_version", 5)?;
tx.commit()?;
}
Ok(from)
}
@@ -220,6 +226,34 @@ fn stem_of(path: &str) -> &str {
}
}
const V5: &str = r#"
-- TRACES: FR-NC-6a | FR-CAT-9 | NFR-RES-4
-- Offline availability: what is kept, why it is kept, and where it lives.
--
-- `pinned` separates a promise from a convenience, and the distinction has to
-- be a *column* rather than something inferred from `pinned_by_rule`. A pin is
-- the user saying "this collection comes with me"; a passively cached original
-- is the app noticing they opened something. Only the second is evictable, so
-- the eviction query has to be able to ask the question directly — and it has
-- to keep answering correctly for an image whose pinning rule was since
-- deleted, which `pinned_by_rule` alone cannot do because it is
-- ON DELETE SET NULL.
ALTER TABLE image_cache ADD COLUMN pinned INTEGER NOT NULL DEFAULT 0;
-- Where the cached original actually is, relative to the cache directory.
-- Relative rather than absolute: the library moves between machines and
-- between an app sandbox and a user directory, and an absolute path baked in
-- at download time would break on every one of those.
ALTER TABLE image_cache ADD COLUMN path TEXT;
-- Eviction reads exactly this: unpinned rows, oldest use first. Partial on
-- `pinned = 0` because pinned rows are never candidates and including them
-- would make the index proportional to the whole library rather than to the
-- passive cache.
CREATE INDEX image_cache_evictable ON image_cache(last_used)
WHERE pinned = 0;
"#;
const V4: &str = r#"
-- TRACES: FR-CAT-15
-- Soft delete. A trashed image is a real file that has been *moved* to a trash
@@ -596,6 +630,51 @@ mod tests {
assert_eq!(backfilled(&c, "shadowed_by"), 1);
}
#[test]
fn a_v4_catalog_gains_the_pinning_columns() {
// TRACES: FR-NC-6a
// An existing library must not have to be rescanned to gain offline
// pinning. The rows are already there; only the columns are new.
let c = mem();
c.execute_batch(V1).unwrap();
c.execute_batch(V2).unwrap();
c.execute_batch(V3).unwrap();
c.execute_batch(V4).unwrap();
c.pragma_update(None, "user_version", 4).unwrap();
c.execute(
"INSERT INTO roots(id, kind, label) VALUES (1, 'remote', 'lib')",
[],
)
.unwrap();
c.execute(
"INSERT INTO images(id, root_id, source_ref, added_at)
VALUES (7, 1, 'IMG_7.CR2', 0)",
[],
)
.unwrap();
// A cache row written before pinning existed.
c.execute(
"INSERT INTO image_cache(image_id, tier_actual, bytes) VALUES (7, 2, 100)",
[],
)
.unwrap();
assert_eq!(migrate(&c).unwrap(), 4, "migrated from v4");
// The pre-existing row survives, and defaults to unpinned — the safe
// direction, since claiming a pin nobody made would exempt it from
// eviction for ever.
let (pinned, bytes): (i64, i64) = c
.query_row(
"SELECT pinned, bytes FROM image_cache WHERE image_id = 7",
[],
|r| Ok((r.get(0)?, r.get(1)?)),
)
.unwrap();
assert_eq!(pinned, 0);
assert_eq!(bytes, 100, "the existing row is untouched");
}
#[test]
fn stems_ignore_directories_containing_dots() {
assert_eq!(stem_of("2026.08/IMG_1.CR2"), "IMG_1");
core/dr-sync-nextcloud/src/lib.rs
+18
View File
@@ -510,6 +510,24 @@ pub fn http_client(user_agent: &str) -> Result<reqwest::Client, RemoteError> {
install_crypto_provider();
reqwest::Client::builder()
.user_agent(user_agent.to_string())
// Verify against the roots compiled into the binary rather than the
// platform store — D7's escape hatch, and what makes TLS work on
// Android at all.
//
// Without this, reqwest builds a `rustls_platform_verifier::Verifier`,
// which reads Android's trust store over JNI and panics during the
// handshake unless Java initialised it first. The panic lands inside a
// tokio task, so tokio swallows it: the worker thread simply stops, the
// channel closes, and the UI reports a failure with no error and no log
// line to explain it.
//
// `tls_certs_only` is the flag reqwest branches on to skip the platform
// verifier entirely (see its ClientBuilder TLS setup); the webpki-roots
// feature supplies the roots it then uses. Spike S3 revisits this to
// honour user-installed and enterprise CAs, which bundled roots cannot.
// Empty: the flag is what matters, and the roots come from the
// webpki-roots feature rather than from certificates passed here.
.tls_certs_only(std::iter::empty())
.build()
.map_err(|e| RemoteError::Network(e.to_string()))
}
core/dr-types/Cargo.toml
+6
View File
@@ -11,3 +11,9 @@ thiserror.workspace = true
# sync'd cache rules, so the predicate language has to serialise. Derive-only:
# no serde machinery leaks into the rest of `core/`.
serde.workspace = true
# Only the tests parse JSON here. Writing `settings.json` is `dr-ui`'s job, so
# a serialiser in this crate's dependencies would be paid for by every crate in
# `core/` to serve one test module.
[dev-dependencies]
serde_json.workspace = true
core/dr-types/src/lib.rs
+5
View File
@@ -9,8 +9,13 @@ use std::fmt;
use std::ops::Range;
pub mod selector;
pub mod settings;
pub use selector::{ColourLabel, DateSelector, FlagState, Selector, Tier};
pub use settings::{
CacheSettings, CollisionPolicy, ColourSpace, ExportFormat, ExportSettings, OutputSharpening,
Settings, SizingMode,
};
/// Identifies a granted library location — a directory on Linux, a persisted
/// document tree on Android.
core/dr-types/src/selector.rs
-1
View File
@@ -232,7 +232,6 @@ mod tests {
assert_eq!(found, vec![CollectionId(1), CollectionId(2)]);
}
#[test]
#[test]
fn stored_tiers_round_trip() {
// These integers are on disk. A reordering of the variants that broke
core/dr-types/src/settings.rs
+602
View File
@@ -0,0 +1,602 @@
//! TRACES: FR-NC-6a | FR-EXP-1 | FR-EXP-2 | FR-EXP-3 | FR-EXP-6 | FR-PLAT-LIN-1
//! Device preferences: how much disk to spend, and what an export defaults to.
//!
//! # Why these live beside the session and not in the catalog
//!
//! The catalog is per-*library* and its index is shared between machines
//! (FR-NC-6): a finished scan is sync'd so a second device inherits it rather
//! than repeating hours of range fetches. Everything in this file is per-*
//! device* instead, and putting it in the catalog would carry it across that
//! sync to somewhere it is wrong.
//!
//! A cache ceiling is the clearest case. "Keep 8 GB of originals" is a
//! statement about *this* disk; following the library to a phone it becomes a
//! promise that device cannot keep. Export defaults are the same kind of fact
//! one step removed — the destination folder is a local path, and a template
//! naming a drive that exists on the desktop resolves to nothing on the
//! laptop.
//!
//! So this is a sibling of `sessions.json` in the platform config directory,
//! written by the same read-modify-rename discipline (see `SettingsStore` in
//! `dr-ui`). The split is deliberate rather than incidental: signing out must
//! not discard a cache budget, and clearing preferences must not revoke a
//! credential.
//!
//! # Why defaults are a `Default` impl and not a config file
//!
//! Every field here answers correctly with no configuration at all. A first
//! run has no settings file, and that is not an error state to be repaired —
//! it is the common case. Reading an absent or corrupt file therefore yields
//! [`Settings::default`], and every field carries `#[serde(default)]` so a
//! file written by an older build is missing fields rather than unparseable.
//!
//! # What is deliberately *not* here
//!
//! Export **presets** (FR-EXP-5) are named, plural, and applicable several at
//! once to produce multiple outputs per image. That is a library of documents,
//! not a preference, and it does not belong in a single-valued settings
//! record. What is here is the *default* an export dialogue opens on and a
//! new preset inherits — one value per field, which is what a settings page
//! can present and what FR-EXP-1..3 call configurable.
use serde::{Deserialize, Serialize};
/// Everything the settings page edits.
///
/// Grouped by the section it appears under rather than flattened, so a new
/// field lands in one obvious place and the JSON stays readable to anyone who
/// opens it by hand.
#[derive(Debug, Clone, Default, PartialEq, Serialize, Deserialize)]
#[serde(default)]
pub struct Settings {
pub cache: CacheSettings,
pub export: ExportSettings,
}
// ---------------------------------------------------------------------------
// Cache
// ---------------------------------------------------------------------------
/// How much of this device's disk the app may spend.
///
/// Two ceilings, because there are two populations with different guarantees
/// (see `dr_catalog::cache`): passively cached originals are a convenience and
/// are evicted least-recently-used, while pinned originals are a promise and
/// are never evicted. Only the first is bounded here.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
#[serde(default)]
pub struct CacheSettings {
/// Ceiling for passively cached originals, in bytes. `None` is unlimited.
///
/// Optional rather than a sentinel `0`: "unlimited" and "keep nothing" are
/// both meaningful settings and a zero cannot mean both. A user with a
/// large disk who would rather never re-download picks the first; a user
/// on a small SSD who only ever works online picks the second.
pub original_budget_bytes: Option<u64>,
/// Ceiling for the thumbnail and preview shards, in bytes. `None` is
/// unlimited.
///
/// Separate from the originals budget because the two fail differently.
/// Evicting a thumbnail costs one small re-fetch and the grid heals as it
/// scrolls; evicting an original costs a full RAW transfer before an image
/// can be opened at all. A shared budget would let a day of browsing
/// originals evict the thumbnails that make the library navigable, which
/// is the more valuable of the two per byte.
pub thumbnail_budget_bytes: Option<u64>,
/// Whether opening an image in develop keeps its original on disk.
///
/// On by default: the bytes were transferred anyway, so keeping them costs
/// no bandwidth and saves the whole transfer next time. Off is for metered
/// or small-disk devices, where the user would rather re-fetch than store.
pub keep_opened_originals: bool,
}
/// 8 GB of passively cached originals — roughly 250 full-frame RAWs.
///
/// Larger than `dr_catalog::cache::DEFAULT_BUDGET_BYTES`, and deliberately:
/// that constant is the floor a `Cache` falls back to when nobody has said
/// otherwise, whereas this is what a desktop install should actually run with.
/// A culling session covers hundreds of frames, and a cache that holds a
/// tenth of one re-downloads images the user is still moving between.
pub const DEFAULT_ORIGINAL_BUDGET_BYTES: u64 = 8 * 1024 * 1024 * 1024;
/// 2 GB of thumbnails and previews.
///
/// Sized against the library rather than the working set, because the grid can
/// scroll anywhere: this holds the derived pyramid for a library in the tens
/// of thousands of images, which is the point of having it at all.
pub const DEFAULT_THUMBNAIL_BUDGET_BYTES: u64 = 2 * 1024 * 1024 * 1024;
impl Default for CacheSettings {
fn default() -> Self {
Self {
original_budget_bytes: Some(DEFAULT_ORIGINAL_BUDGET_BYTES),
thumbnail_budget_bytes: Some(DEFAULT_THUMBNAIL_BUDGET_BYTES),
keep_opened_originals: true,
}
}
}
// ---------------------------------------------------------------------------
// Export
// ---------------------------------------------------------------------------
/// What an export starts from before the user changes anything.
///
/// These are *defaults*, not a preset (FR-EXP-5) — see the module docs. Each
/// field mirrors a requirement rather than an encoder flag, so the export
/// implementation can change without this record changing meaning.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
#[serde(default)]
pub struct ExportSettings {
pub format: ExportFormat,
/// Encoder quality, 1..=100. Ignored by lossless formats.
///
/// Clamped on read rather than trusted: this is hand-editable JSON, and a
/// quality of 0 or 900 reaching an encoder is a panic or a corrupt file.
pub quality: u8,
pub colour_space: ColourSpace,
pub sizing: SizingMode,
/// Whether a requested size larger than the source is honoured.
///
/// Off by default, as FR-EXP-3 requires. Where disabled, an oversized
/// request exports at source size rather than failing — a batch must not
/// abort because one frame was smaller than the target.
pub allow_upscaling: bool,
pub sharpening: OutputSharpening,
/// Filename template (FR-EXP-6). Tokens are resolved by the exporter.
pub filename_template: String,
/// What to do when the output filename already exists.
pub collision: CollisionPolicy,
/// Whether GPS and other identifying metadata is stripped (FR-EXP-8).
///
/// Stripping is *on* by default, which is the one place here that departs
/// from "preserve what the camera recorded". An export is usually the copy
/// that leaves the machine, and a home address embedded in a photograph
/// shared publicly is not recoverable once published. The reverse mistake —
/// a user who wanted coordinates and has to re-export — costs a minute.
pub strip_location: bool,
/// Destination folder. Empty means "ask each time".
///
/// Empty rather than a guessed `~/Pictures`: a silent default destination
/// is how exports end up somewhere the user never looks, and this is the
/// one field where the app genuinely does not know the answer.
pub destination: String,
}
impl Default for ExportSettings {
fn default() -> Self {
Self {
format: ExportFormat::Jpeg,
// 90 rather than 100: past roughly this point JPEG spends bytes
// without a visible return, and the default should be the one a
// photographer would not need to change for a web export.
quality: 90,
colour_space: ColourSpace::Srgb,
sizing: SizingMode::Original,
allow_upscaling: false,
sharpening: OutputSharpening::Screen,
filename_template: "{name}".to_string(),
collision: CollisionPolicy::Increment,
strip_location: true,
destination: String::new(),
}
}
}
/// Output container and codec (FR-EXP-1).
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum ExportFormat {
Jpeg,
Png,
/// 8-bit TIFF.
Tiff8,
/// 16-bit TIFF, for work continuing in another editor.
Tiff16,
Avif,
JpegXl,
}
impl ExportFormat {
/// Every format, in the order the settings page lists them.
///
/// Ordered by how often a photographer reaches for each rather than
/// alphabetically: JPEG is most exports, the TIFFs are the archival pair,
/// and the modern codecs sit last because support is still uneven.
pub const ALL: [Self; 6] = [
Self::Jpeg,
Self::Png,
Self::Tiff8,
Self::Tiff16,
Self::Avif,
Self::JpegXl,
];
pub fn label(self) -> &'static str {
match self {
Self::Jpeg => "JPEG",
Self::Png => "PNG",
Self::Tiff8 => "TIFF 8-bit",
Self::Tiff16 => "TIFF 16-bit",
Self::Avif => "AVIF",
Self::JpegXl => "JPEG XL",
}
}
pub fn extension(self) -> &'static str {
match self {
Self::Jpeg => "jpg",
Self::Png => "png",
Self::Tiff8 | Self::Tiff16 => "tif",
Self::Avif => "avif",
Self::JpegXl => "jxl",
}
}
/// Whether the quality setting means anything for this format.
///
/// Drives whether the page *disables* the quality control rather than
/// hiding it: a control that vanishes when PNG is chosen reads as a bug,
/// where a greyed one explains itself.
pub fn is_lossy(self) -> bool {
matches!(self, Self::Jpeg | Self::Avif | Self::JpegXl)
}
}
/// Output colour space, with its ICC profile embedded on export (FR-EXP-2).
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum ColourSpace {
Srgb,
DisplayP3,
AdobeRgb,
ProPhoto,
}
impl ColourSpace {
pub const ALL: [Self; 4] = [Self::Srgb, Self::DisplayP3, Self::AdobeRgb, Self::ProPhoto];
pub fn label(self) -> &'static str {
match self {
Self::Srgb => "sRGB",
Self::DisplayP3 => "Display P3",
Self::AdobeRgb => "Adobe RGB",
Self::ProPhoto => "ProPhoto",
}
}
}
/// How output dimensions are decided (FR-EXP-3).
///
/// Only the modes a *default* can sensibly carry are here. The full table in
/// FR-EXP-3 includes print dimensions at a DPI and a target megapixel count;
/// those need units and a resolution alongside the number, which is an export
/// dialogue's job. A default that opened on "300mm at 240dpi" would be a
/// setting almost no library wants every time.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum SizingMode {
/// Full source resolution after the crop.
Original,
/// Pixels on the longer dimension; aspect preserved.
LongEdge(u32),
/// Pixels on the shorter dimension; aspect preserved.
ShortEdge(u32),
/// A factor of the source, as a percentage.
Percentage(u32),
}
impl SizingMode {
/// The mode's name, without its value.
pub fn label(self) -> &'static str {
match self {
Self::Original => "Original",
Self::LongEdge(_) => "Long edge",
Self::ShortEdge(_) => "Short edge",
Self::Percentage(_) => "Percentage",
}
}
/// The number this mode carries, if it takes one.
pub fn value(self) -> Option<u32> {
match self {
Self::Original => None,
Self::LongEdge(n) | Self::ShortEdge(n) | Self::Percentage(n) => Some(n),
}
}
/// The same mode carrying `value`, or unchanged where it takes none.
///
/// Lets the page edit the number without re-deciding the variant, which is
/// what a field beside a mode selector needs.
pub fn with_value(self, value: u32) -> Self {
match self {
Self::Original => Self::Original,
Self::LongEdge(_) => Self::LongEdge(value),
Self::ShortEdge(_) => Self::ShortEdge(value),
Self::Percentage(_) => Self::Percentage(value),
}
}
/// The modes in page order, each with a usable starting value.
///
/// Switching to a sized mode has to land on *something*, and these are
/// values a user would plausibly keep: 2048px is a common web long edge,
/// and 100% is the identity, so choosing "Percentage" changes nothing
/// until a number is typed.
pub const CHOICES: [Self; 4] = [
Self::Original,
Self::LongEdge(2048),
Self::ShortEdge(1600),
Self::Percentage(100),
];
/// Whether two modes are the same variant, ignoring their values.
///
/// What the page's selection test needs: `LongEdge(2048)` and
/// `LongEdge(900)` are one choice showing different numbers, and equality
/// would light neither when the user had typed their own.
pub fn same_mode(self, other: Self) -> bool {
std::mem::discriminant(&self) == std::mem::discriminant(&other)
}
}
/// Output sharpening, scaled by the resize factor (FR-EXP-4).
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum OutputSharpening {
None,
Screen,
MattePaper,
GlossyPaper,
}
impl OutputSharpening {
pub const ALL: [Self; 4] = [Self::None, Self::Screen, Self::MattePaper, Self::GlossyPaper];
pub fn label(self) -> &'static str {
match self {
Self::None => "None",
Self::Screen => "Screen",
Self::MattePaper => "Matte paper",
Self::GlossyPaper => "Glossy paper",
}
}
}
/// What happens when the output filename is taken (FR-EXP-6).
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum CollisionPolicy {
/// Replace the existing file.
Overwrite,
/// Leave the existing file and export nothing for that image.
Skip,
/// Append a counter: `name-1.jpg`, `name-2.jpg`.
Increment,
}
impl CollisionPolicy {
pub const ALL: [Self; 3] = [Self::Increment, Self::Skip, Self::Overwrite];
pub fn label(self) -> &'static str {
match self {
Self::Overwrite => "Overwrite",
Self::Skip => "Skip",
Self::Increment => "Auto-increment",
}
}
}
// ---------------------------------------------------------------------------
// Validation
// ---------------------------------------------------------------------------
impl Settings {
/// Force hand-edited or older values into a usable range.
///
/// Called after every read, because this file is plain JSON in a config
/// directory and a user is entitled to edit it. Clamping rather than
/// rejecting: a nonsensical quality should not stop the app from starting,
/// and refusing to load would lose every *other* setting in the file over
/// one bad field.
pub fn sanitise(&mut self) {
self.export.quality = self.export.quality.clamp(1, 100);
// A zero-pixel or zero-percent export produces no image. Nudged to the
// smallest thing that does, rather than back to the default: the user
// clearly wanted "small", and silently restoring 2048 would ignore
// that.
self.export.sizing = match self.export.sizing {
SizingMode::LongEdge(0) => SizingMode::LongEdge(1),
SizingMode::ShortEdge(0) => SizingMode::ShortEdge(1),
SizingMode::Percentage(0) => SizingMode::Percentage(1),
other => other,
};
// An empty template names every output the same thing, so every export
// after the first collides. `{name}` is the identity and what the
// default already is.
if self.export.filename_template.trim().is_empty() {
self.export.filename_template = "{name}".to_string();
}
}
}
/// How a byte budget is shown and typed.
///
/// Gigabytes, one decimal place, because that is the unit a disk is discussed
/// in and the settings page has to round-trip whatever it displays: showing
/// "8.4 GB" and storing something that redisplays as "8.3 GB" makes the field
/// look like it is losing the edit.
pub mod budget {
const BYTES_PER_GB: f64 = (1024 * 1024 * 1024) as f64;
/// Bytes as a gigabyte figure, rounded to one decimal.
pub fn to_gb(bytes: u64) -> f64 {
((bytes as f64 / BYTES_PER_GB) * 10.0).round() / 10.0
}
/// A gigabyte figure as bytes, or `None` where it is not a positive number.
///
/// Rejects rather than clamps, because this parses what a user typed: a
/// stray keystroke should leave the previous budget in place, where
/// clamping to a minimum would silently shrink a cache to nothing.
pub fn from_gb(text: &str) -> Option<u64> {
let gb: f64 = text.trim().trim_end_matches(|c: char| {
c.is_ascii_alphabetic() || c.is_whitespace()
}).parse().ok()?;
if !gb.is_finite() || gb <= 0.0 {
return None;
}
Some((gb * BYTES_PER_GB) as u64)
}
/// A budget as the page shows it, unlimited included.
pub fn label(bytes: Option<u64>) -> String {
match bytes {
None => "Unlimited".to_string(),
Some(b) => format!("{:.1} GB", to_gb(b)),
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn defaults_round_trip_through_json() {
let settings = Settings::default();
let json = serde_json::to_string(&settings).unwrap();
assert_eq!(
serde_json::from_str::<Settings>(&json).unwrap(),
settings
);
}
#[test]
fn an_empty_object_is_the_defaults() {
// A first run has no file, and a file written by an older build is
// missing whatever was added since. Neither is an error.
assert_eq!(
serde_json::from_str::<Settings>("{}").unwrap(),
Settings::default()
);
}
#[test]
fn a_partial_file_keeps_the_defaults_for_absent_fields() {
let json = r#"{"export": {"quality": 75}}"#;
let parsed: Settings = serde_json::from_str(json).unwrap();
assert_eq!(parsed.export.quality, 75);
assert_eq!(parsed.export.format, ExportFormat::Jpeg);
assert_eq!(parsed.cache, CacheSettings::default());
}
#[test]
fn upscaling_is_off_by_default() {
// FR-EXP-3 requires an explicit opt-in.
assert!(!ExportSettings::default().allow_upscaling);
}
#[test]
fn location_metadata_is_stripped_by_default() {
assert!(ExportSettings::default().strip_location);
}
#[test]
fn sanitise_clamps_an_out_of_range_quality() {
let mut s = Settings::default();
s.export.quality = 200;
s.sanitise();
assert_eq!(s.export.quality, 100);
s.export.quality = 0;
s.sanitise();
assert_eq!(s.export.quality, 1);
}
#[test]
fn sanitise_rescues_a_zero_dimension() {
let mut s = Settings::default();
s.export.sizing = SizingMode::LongEdge(0);
s.sanitise();
assert_eq!(s.export.sizing, SizingMode::LongEdge(1));
}
#[test]
fn sanitise_restores_an_empty_template() {
// Otherwise every export after the first collides on one filename.
let mut s = Settings::default();
s.export.filename_template = " ".to_string();
s.sanitise();
assert_eq!(s.export.filename_template, "{name}");
}
#[test]
fn quality_is_ignored_by_lossless_formats() {
assert!(!ExportFormat::Png.is_lossy());
assert!(!ExportFormat::Tiff16.is_lossy());
assert!(ExportFormat::Jpeg.is_lossy());
}
#[test]
fn a_sizing_mode_keeps_its_variant_when_the_value_changes() {
let mode = SizingMode::LongEdge(2048);
assert_eq!(mode.with_value(900), SizingMode::LongEdge(900));
assert!(mode.same_mode(SizingMode::LongEdge(900)));
assert!(!mode.same_mode(SizingMode::ShortEdge(2048)));
}
#[test]
fn original_sizing_takes_no_value() {
assert_eq!(SizingMode::Original.value(), None);
// Setting one on a mode that has none must not invent a variant.
assert_eq!(SizingMode::Original.with_value(500), SizingMode::Original);
}
#[test]
fn a_budget_round_trips_through_the_field_it_is_shown_in() {
// The page shows one decimal place and parses back what it showed. If
// these disagreed, an untouched field would appear to change on save.
let bytes = 8 * 1024 * 1024 * 1024;
let shown = budget::label(Some(bytes));
assert_eq!(shown, "8.0 GB");
assert_eq!(budget::from_gb(&shown), Some(bytes));
}
#[test]
fn a_budget_accepts_a_typed_unit_or_none() {
let gb = 1024 * 1024 * 1024;
assert_eq!(budget::from_gb("4"), Some(4 * gb));
assert_eq!(budget::from_gb("4 GB"), Some(4 * gb));
assert_eq!(budget::from_gb(" 4gb "), Some(4 * gb));
}
#[test]
fn a_meaningless_budget_is_rejected_rather_than_clamped() {
// The caller keeps the previous value: clamping a typo to a minimum
// would silently shrink the cache to nothing.
assert_eq!(budget::from_gb(""), None);
assert_eq!(budget::from_gb("lots"), None);
assert_eq!(budget::from_gb("-2"), None);
assert_eq!(budget::from_gb("0"), None);
}
#[test]
fn unlimited_is_shown_as_a_word_not_a_number() {
assert_eq!(budget::label(None), "Unlimited");
}
}