Keep originals on this device, by pin and by use
Fills in `image_cache`, which the previous commit's "On this device" filter read but nothing wrote. Also carries in-flight work that shared these files: the Android TLS root store, the settings page, and a regenerated traceability report. # Two populations, deliberately separate An original is kept here for one of two reasons, and conflating them produces the exact failure the feature exists to prevent. **Pinned** originals were asked for. Pinning a collection before a trip is a promise, so pinned rows are never evicted and never counted against the budget — a cap that could silently delete a pinned trip would make pinning worthless, because it could not be relied on without checking. **Passively cached** originals are a side effect of working: develop already downloads the whole file, so keeping it costs no bandwidth and saves the entire transfer next time. This population is what the budget bounds, evicted least-recently-used, because it otherwise grows until a day of culling fills a disk. Sharing one budget would let a large pin starve the passive cache, or let browsing evict a pin. They are separate. # What was built `dr_catalog::cache` owns the bookkeeping — held tier, size, last use, pinned — and writes the bytes; deciding to download stays with the caller, which is what keeps a crate with no network out of the network's business. Files are written to a temporary and renamed, so a dropped connection cannot leave a truncated file recorded as a complete original. They are named by image id, not filename: `Photos/IMG_0001.CR2` and `Trips/IMG_0001.CR2` are different photographs, and a flat cache keyed on the name would serve one for the other. `spawn_full_fetch` became read-through. A hit is a disk read; a miss stores what it downloads and enforces the budget. A cache that cannot be opened is a miss, not a failure to open the photograph. Pinning writes intent — `tier_desired` — without downloading, so the button responds immediately, and `spawn_pin_fetch` fills it in sequentially afterwards. Sequential because these are tens of megabytes each: the lanes that make the thumbnail sweep fast buy little against one connection's bandwidth and cost a great deal of memory. A pin interrupted by a lost connection resumes from where it stopped. Schema v5 adds `pinned` and `path`. `pinned` is a column rather than something inferred from `pinned_by_rule`, which is ON DELETE SET NULL and so cannot answer for an image whose rule was deleted. A v4 catalog migrates in place; existing rows default to unpinned, the safe direction. The budget and "keep opened originals" come from the settings page rather than a constant, and are applied at startup rather than only on change — a cache capped at 2 GB last session would otherwise spend this one filling to the default. Turning off keeping leaves what is already cached readable: those bytes are paid for, and refusing them would re-download images sitting right there, including pinned ones. Also removes a doubled `#[test]` introduced in the previous commit. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -15,7 +15,7 @@ use rusqlite::Connection;
|
||||
use crate::error::CatalogError;
|
||||
|
||||
/// Schema version this build writes and understands.
|
||||
pub const SCHEMA_VERSION: i64 = 4;
|
||||
pub const SCHEMA_VERSION: i64 = 5;
|
||||
|
||||
/// Apply migrations up to [`SCHEMA_VERSION`].
|
||||
///
|
||||
@@ -60,6 +60,12 @@ pub fn migrate(conn: &Connection) -> Result<i64, CatalogError> {
|
||||
tx.pragma_update(None, "user_version", 4)?;
|
||||
tx.commit()?;
|
||||
}
|
||||
if from < 5 {
|
||||
let tx = conn.unchecked_transaction()?;
|
||||
tx.execute_batch(V5)?;
|
||||
tx.pragma_update(None, "user_version", 5)?;
|
||||
tx.commit()?;
|
||||
}
|
||||
|
||||
Ok(from)
|
||||
}
|
||||
@@ -220,6 +226,34 @@ fn stem_of(path: &str) -> &str {
|
||||
}
|
||||
}
|
||||
|
||||
const V5: &str = r#"
|
||||
-- TRACES: FR-NC-6a | FR-CAT-9 | NFR-RES-4
|
||||
-- Offline availability: what is kept, why it is kept, and where it lives.
|
||||
--
|
||||
-- `pinned` separates a promise from a convenience, and the distinction has to
|
||||
-- be a *column* rather than something inferred from `pinned_by_rule`. A pin is
|
||||
-- the user saying "this collection comes with me"; a passively cached original
|
||||
-- is the app noticing they opened something. Only the second is evictable, so
|
||||
-- the eviction query has to be able to ask the question directly — and it has
|
||||
-- to keep answering correctly for an image whose pinning rule was since
|
||||
-- deleted, which `pinned_by_rule` alone cannot do because it is
|
||||
-- ON DELETE SET NULL.
|
||||
ALTER TABLE image_cache ADD COLUMN pinned INTEGER NOT NULL DEFAULT 0;
|
||||
|
||||
-- Where the cached original actually is, relative to the cache directory.
|
||||
-- Relative rather than absolute: the library moves between machines and
|
||||
-- between an app sandbox and a user directory, and an absolute path baked in
|
||||
-- at download time would break on every one of those.
|
||||
ALTER TABLE image_cache ADD COLUMN path TEXT;
|
||||
|
||||
-- Eviction reads exactly this: unpinned rows, oldest use first. Partial on
|
||||
-- `pinned = 0` because pinned rows are never candidates and including them
|
||||
-- would make the index proportional to the whole library rather than to the
|
||||
-- passive cache.
|
||||
CREATE INDEX image_cache_evictable ON image_cache(last_used)
|
||||
WHERE pinned = 0;
|
||||
"#;
|
||||
|
||||
const V4: &str = r#"
|
||||
-- TRACES: FR-CAT-15
|
||||
-- Soft delete. A trashed image is a real file that has been *moved* to a trash
|
||||
@@ -596,6 +630,51 @@ mod tests {
|
||||
assert_eq!(backfilled(&c, "shadowed_by"), 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_v4_catalog_gains_the_pinning_columns() {
|
||||
// TRACES: FR-NC-6a
|
||||
// An existing library must not have to be rescanned to gain offline
|
||||
// pinning. The rows are already there; only the columns are new.
|
||||
let c = mem();
|
||||
c.execute_batch(V1).unwrap();
|
||||
c.execute_batch(V2).unwrap();
|
||||
c.execute_batch(V3).unwrap();
|
||||
c.execute_batch(V4).unwrap();
|
||||
c.pragma_update(None, "user_version", 4).unwrap();
|
||||
c.execute(
|
||||
"INSERT INTO roots(id, kind, label) VALUES (1, 'remote', 'lib')",
|
||||
[],
|
||||
)
|
||||
.unwrap();
|
||||
c.execute(
|
||||
"INSERT INTO images(id, root_id, source_ref, added_at)
|
||||
VALUES (7, 1, 'IMG_7.CR2', 0)",
|
||||
[],
|
||||
)
|
||||
.unwrap();
|
||||
// A cache row written before pinning existed.
|
||||
c.execute(
|
||||
"INSERT INTO image_cache(image_id, tier_actual, bytes) VALUES (7, 2, 100)",
|
||||
[],
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
assert_eq!(migrate(&c).unwrap(), 4, "migrated from v4");
|
||||
|
||||
// The pre-existing row survives, and defaults to unpinned — the safe
|
||||
// direction, since claiming a pin nobody made would exempt it from
|
||||
// eviction for ever.
|
||||
let (pinned, bytes): (i64, i64) = c
|
||||
.query_row(
|
||||
"SELECT pinned, bytes FROM image_cache WHERE image_id = 7",
|
||||
[],
|
||||
|r| Ok((r.get(0)?, r.get(1)?)),
|
||||
)
|
||||
.unwrap();
|
||||
assert_eq!(pinned, 0);
|
||||
assert_eq!(bytes, 100, "the existing row is untouched");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn stems_ignore_directories_containing_dots() {
|
||||
assert_eq!(stem_of("2026.08/IMG_1.CR2"), "IMG_1");
|
||||
|
||||
Reference in New Issue
Block a user