Keep originals on this device, by pin and by use
Build and test / Desktop (Linux) (push) Failing after 1s
Build and test / Android (aarch64) (push) Failing after 0s
Build and test / Layer separation (push) Failing after 1s
Traceability / Requirement traces (push) Failing after 2s

Fills in `image_cache`, which the previous commit's "On this device" filter
read but nothing wrote. Also carries in-flight work that shared these files:
the Android TLS root store, the settings page, and a regenerated
traceability report.

# Two populations, deliberately separate

An original is kept here for one of two reasons, and conflating them produces
the exact failure the feature exists to prevent.

**Pinned** originals were asked for. Pinning a collection before a trip is a
promise, so pinned rows are never evicted and never counted against the
budget — a cap that could silently delete a pinned trip would make pinning
worthless, because it could not be relied on without checking.

**Passively cached** originals are a side effect of working: develop already
downloads the whole file, so keeping it costs no bandwidth and saves the
entire transfer next time. This population is what the budget bounds, evicted
least-recently-used, because it otherwise grows until a day of culling fills
a disk.

Sharing one budget would let a large pin starve the passive cache, or let
browsing evict a pin. They are separate.

# What was built

`dr_catalog::cache` owns the bookkeeping — held tier, size, last use, pinned
— and writes the bytes; deciding to download stays with the caller, which is
what keeps a crate with no network out of the network's business. Files are
written to a temporary and renamed, so a dropped connection cannot leave a
truncated file recorded as a complete original. They are named by image id,
not filename: `Photos/IMG_0001.CR2` and `Trips/IMG_0001.CR2` are different
photographs, and a flat cache keyed on the name would serve one for the other.

`spawn_full_fetch` became read-through. A hit is a disk read; a miss stores
what it downloads and enforces the budget. A cache that cannot be opened is a
miss, not a failure to open the photograph.

Pinning writes intent — `tier_desired` — without downloading, so the button
responds immediately, and `spawn_pin_fetch` fills it in sequentially
afterwards. Sequential because these are tens of megabytes each: the lanes
that make the thumbnail sweep fast buy little against one connection's
bandwidth and cost a great deal of memory. A pin interrupted by a lost
connection resumes from where it stopped.

Schema v5 adds `pinned` and `path`. `pinned` is a column rather than something
inferred from `pinned_by_rule`, which is ON DELETE SET NULL and so cannot
answer for an image whose rule was deleted. A v4 catalog migrates in place;
existing rows default to unpinned, the safe direction.

The budget and "keep opened originals" come from the settings page rather than
a constant, and are applied at startup rather than only on change — a cache
capped at 2 GB last session would otherwise spend this one filling to the
default. Turning off keeping leaves what is already cached readable: those
bytes are paid for, and refusing them would re-download images sitting right
there, including pinned ones.

Also removes a doubled `#[test]` introduced in the previous commit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-11 21:12:01 +02:00
co-authored by Claude Opus 5
parent cd75e5a4c6
commit fa12afed18
22 changed files with 4032 additions and 86 deletions
+101 -1
View File
@@ -20,6 +20,8 @@ mod develop;
mod labels;
mod library;
mod library_ui;
mod settings_store;
mod settings_ui;
mod trash;
#[cfg(live_style)]
mod live_style;
@@ -540,6 +542,50 @@ pub fn run(paths: Vec<PathBuf>) -> Result<()> {
}
}
// Settings: cache ceilings and export defaults, in their own config file.
//
// Wired independently of every view above. It reads no library and holds no
// session, so it has nothing to be sequenced against — which is the reason
// it is a page reachable from anywhere rather than a panel inside one view.
{
let settings = settings_ui::SettingsController::new();
// What the cache actually holds, so the ceiling above it is a figure
// the user can judge rather than an abstract one.
settings.set_usage_label(describe_cache_usage(&library));
// Rendered once up front so the page is correct the first time it is
// opened, rather than on the second open after a callback has run.
settings_ui::render(&window, &settings);
// Apply what is on disk before anything can use it. Without this the
// controller's defaults stand until the user happens to open the
// settings page and change something — so a cache deliberately capped
// at 2 GB last session would spend this one filling to the default.
{
let stored = settings.snapshot();
library.set_cache_budget(stored.cache.original_budget_bytes);
library.set_keep_opened_originals(stored.cache.keep_opened_originals);
}
let lib = library.clone();
let ctl = settings.clone();
let weak = window.as_weak();
settings_ui::wire(&window, settings.clone(), move |s| {
// A ceiling that moved has to be applied to what is already on
// disk, or lowering it would only affect future downloads and the
// cache would sit over budget indefinitely.
lib.set_cache_budget(s.cache.original_budget_bytes);
lib.set_keep_opened_originals(s.cache.keep_opened_originals);
// Lowering the ceiling evicts, so the figure beside it has just
// changed — leaving the old one would show the cache still over a
// limit that was enforced a moment ago.
ctl.set_usage_label(describe_cache_usage(&lib));
if let Some(w) = weak.upgrade() {
settings_ui::render(&w, &ctl);
}
});
}
// The device is shared by demosaic and the adjust pass. Without one the
// app still browses through the preview path, just without develop.
let gpu = match pollster::block_on(dr_gpu::GpuContext::new_headless()) {
@@ -827,10 +873,20 @@ pub fn run(paths: Vec<PathBuf>) -> Result<()> {
return;
};
// TRACES: FR-NC-6a
// The cache is consulted first, so a second open of the same
// photograph is a disk read rather than a second download of tens
// of megabytes — and so a session's worth of images stays
// openable when the connection goes.
let cache = library.cache_context(&path);
if cache.is_none() {
log::debug!("no originals cache for {path}; fetching every time");
}
log::info!("fetching {path} for develop");
w.set_load_error("Downloading…".into());
let rx = library::spawn_full_fetch(creds, user_id, path.clone());
let rx = library::spawn_full_fetch(creds, user_id, path.clone(), cache);
// Polled on the UI thread rather than joined: a join would freeze
// the window for the length of the download.
@@ -1222,6 +1278,50 @@ pub fn run(paths: Vec<PathBuf>) -> Result<()> {
Ok(())
}
/// TRACES: FR-NC-6a
/// What the originals cache is holding, for the settings page.
///
/// Pinned and passive are reported separately because they answer different
/// questions: the passive figure is what the budget above it governs, while
/// the pinned figure is disk the user asked for and no ceiling will reclaim.
/// One combined number would make the budget look wrong whenever a large
/// collection was pinned.
fn describe_cache_usage(library: &Rc<library_ui::LibraryController>) -> String {
let Some(cache) = library.cache() else {
return String::new();
};
let catalog = library.catalog();
let borrow = catalog.borrow();
let Some(catalog) = borrow.as_ref() else {
return String::new();
};
let Ok(usage) = cache.usage(catalog.connection()) else {
return String::new();
};
let gb = |b: u64| b as f64 / 1_073_741_824.0;
match (usage.passive_count, usage.pinned_count) {
(0, 0) => "Nothing cached yet".to_string(),
(_, 0) => format!(
"{:.1} GB cached ({} images)",
gb(usage.passive_bytes),
usage.passive_count
),
(0, _) => format!(
"{:.1} GB pinned ({} images)",
gb(usage.pinned_bytes),
usage.pinned_count
),
_ => format!(
"{:.1} GB cached ({} images) · {:.1} GB pinned ({} images)",
gb(usage.passive_bytes),
usage.passive_count,
gb(usage.pinned_bytes),
usage.pinned_count
),
}
}
fn describe_camera(m: &Metadata) -> String {
match (&m.make, &m.model) {
(Some(make), Some(model)) => {