Keep originals on this device, by pin and by use
Build and test / Desktop (Linux) (push) Failing after 1s
Build and test / Android (aarch64) (push) Failing after 0s
Build and test / Layer separation (push) Failing after 1s
Traceability / Requirement traces (push) Failing after 2s

Fills in `image_cache`, which the previous commit's "On this device" filter
read but nothing wrote. Also carries in-flight work that shared these files:
the Android TLS root store, the settings page, and a regenerated
traceability report.

# Two populations, deliberately separate

An original is kept here for one of two reasons, and conflating them produces
the exact failure the feature exists to prevent.

**Pinned** originals were asked for. Pinning a collection before a trip is a
promise, so pinned rows are never evicted and never counted against the
budget — a cap that could silently delete a pinned trip would make pinning
worthless, because it could not be relied on without checking.

**Passively cached** originals are a side effect of working: develop already
downloads the whole file, so keeping it costs no bandwidth and saves the
entire transfer next time. This population is what the budget bounds, evicted
least-recently-used, because it otherwise grows until a day of culling fills
a disk.

Sharing one budget would let a large pin starve the passive cache, or let
browsing evict a pin. They are separate.

# What was built

`dr_catalog::cache` owns the bookkeeping — held tier, size, last use, pinned
— and writes the bytes; deciding to download stays with the caller, which is
what keeps a crate with no network out of the network's business. Files are
written to a temporary and renamed, so a dropped connection cannot leave a
truncated file recorded as a complete original. They are named by image id,
not filename: `Photos/IMG_0001.CR2` and `Trips/IMG_0001.CR2` are different
photographs, and a flat cache keyed on the name would serve one for the other.

`spawn_full_fetch` became read-through. A hit is a disk read; a miss stores
what it downloads and enforces the budget. A cache that cannot be opened is a
miss, not a failure to open the photograph.

Pinning writes intent — `tier_desired` — without downloading, so the button
responds immediately, and `spawn_pin_fetch` fills it in sequentially
afterwards. Sequential because these are tens of megabytes each: the lanes
that make the thumbnail sweep fast buy little against one connection's
bandwidth and cost a great deal of memory. A pin interrupted by a lost
connection resumes from where it stopped.

Schema v5 adds `pinned` and `path`. `pinned` is a column rather than something
inferred from `pinned_by_rule`, which is ON DELETE SET NULL and so cannot
answer for an image whose rule was deleted. A v4 catalog migrates in place;
existing rows default to unpinned, the safe direction.

The budget and "keep opened originals" come from the settings page rather than
a constant, and are applied at startup rather than only on change — a cache
capped at 2 GB last session would otherwise spend this one filling to the
default. Turning off keeping leaves what is already cached readable: those
bytes are paid for, and refusing them would re-download images sitting right
there, including pinned ones.

Also removes a doubled `#[test]` introduced in the previous commit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-11 21:12:01 +02:00
co-authored by Claude Opus 5
parent cd75e5a4c6
commit fa12afed18
22 changed files with 4032 additions and 86 deletions
+80 -1
View File
@@ -15,7 +15,7 @@ use rusqlite::Connection;
use crate::error::CatalogError;
/// Schema version this build writes and understands.
pub const SCHEMA_VERSION: i64 = 4;
pub const SCHEMA_VERSION: i64 = 5;
/// Apply migrations up to [`SCHEMA_VERSION`].
///
@@ -60,6 +60,12 @@ pub fn migrate(conn: &Connection) -> Result<i64, CatalogError> {
tx.pragma_update(None, "user_version", 4)?;
tx.commit()?;
}
if from < 5 {
let tx = conn.unchecked_transaction()?;
tx.execute_batch(V5)?;
tx.pragma_update(None, "user_version", 5)?;
tx.commit()?;
}
Ok(from)
}
@@ -220,6 +226,34 @@ fn stem_of(path: &str) -> &str {
}
}
const V5: &str = r#"
-- TRACES: FR-NC-6a | FR-CAT-9 | NFR-RES-4
-- Offline availability: what is kept, why it is kept, and where it lives.
--
-- `pinned` separates a promise from a convenience, and the distinction has to
-- be a *column* rather than something inferred from `pinned_by_rule`. A pin is
-- the user saying "this collection comes with me"; a passively cached original
-- is the app noticing they opened something. Only the second is evictable, so
-- the eviction query has to be able to ask the question directly — and it has
-- to keep answering correctly for an image whose pinning rule was since
-- deleted, which `pinned_by_rule` alone cannot do because it is
-- ON DELETE SET NULL.
ALTER TABLE image_cache ADD COLUMN pinned INTEGER NOT NULL DEFAULT 0;
-- Where the cached original actually is, relative to the cache directory.
-- Relative rather than absolute: the library moves between machines and
-- between an app sandbox and a user directory, and an absolute path baked in
-- at download time would break on every one of those.
ALTER TABLE image_cache ADD COLUMN path TEXT;
-- Eviction reads exactly this: unpinned rows, oldest use first. Partial on
-- `pinned = 0` because pinned rows are never candidates and including them
-- would make the index proportional to the whole library rather than to the
-- passive cache.
CREATE INDEX image_cache_evictable ON image_cache(last_used)
WHERE pinned = 0;
"#;
const V4: &str = r#"
-- TRACES: FR-CAT-15
-- Soft delete. A trashed image is a real file that has been *moved* to a trash
@@ -596,6 +630,51 @@ mod tests {
assert_eq!(backfilled(&c, "shadowed_by"), 1);
}
#[test]
fn a_v4_catalog_gains_the_pinning_columns() {
// TRACES: FR-NC-6a
// An existing library must not have to be rescanned to gain offline
// pinning. The rows are already there; only the columns are new.
let c = mem();
c.execute_batch(V1).unwrap();
c.execute_batch(V2).unwrap();
c.execute_batch(V3).unwrap();
c.execute_batch(V4).unwrap();
c.pragma_update(None, "user_version", 4).unwrap();
c.execute(
"INSERT INTO roots(id, kind, label) VALUES (1, 'remote', 'lib')",
[],
)
.unwrap();
c.execute(
"INSERT INTO images(id, root_id, source_ref, added_at)
VALUES (7, 1, 'IMG_7.CR2', 0)",
[],
)
.unwrap();
// A cache row written before pinning existed.
c.execute(
"INSERT INTO image_cache(image_id, tier_actual, bytes) VALUES (7, 2, 100)",
[],
)
.unwrap();
assert_eq!(migrate(&c).unwrap(), 4, "migrated from v4");
// The pre-existing row survives, and defaults to unpinned — the safe
// direction, since claiming a pin nobody made would exempt it from
// eviction for ever.
let (pinned, bytes): (i64, i64) = c
.query_row(
"SELECT pinned, bytes FROM image_cache WHERE image_id = 7",
[],
|r| Ok((r.get(0)?, r.get(1)?)),
)
.unwrap();
assert_eq!(pinned, 0);
assert_eq!(bytes, 100, "the existing row is untouched");
}
#[test]
fn stems_ignore_directories_containing_dots() {
assert_eq!(stem_of("2026.08/IMG_1.CR2"), "IMG_1");