dtourolle/DarkRoom
1
0
Fork 0
Code Issues 39 Pull Requests Actions Packages Projects Releases 20 Wiki Activity

Run the library from local data when the server is unreachable

Browse Source 
Also carries in-flight work that shared these files: the zoom structure-key
fix in the adjust pipeline, nearest-neighbour filtering past 1:1, the
timeline scrub marker correction, the 423-Locked retry in the metadata
sweep, and the thumbnail size-class migration.

# Offline mode (FR-CAT-9)

The app previously assumed the server was reachable and treated its absence
as a series of unrelated per-operation failures. A launch without a
connection produced an empty grid, even with a complete catalog on disk and
every thumbnail already in the shards.

Reachability is now inferred from traffic the app was already making, rather
than probed for. `RemoteError::indicates_offline` draws the line that makes
this possible: a dead connection is offline, a 403 or a 500 is not — the
server answered, so blanking the library over one forbidden file would be a
worse error than the one being reported. `Reachability` turns those outcomes
into a state, so a library browsing happily never issues a probe at all.

Going offline takes one failure, because the user is already experiencing it.
Coming back requires evidence — a completed scan or a fetched thumbnail —
with a capped exponential backoff behind the manual retry, so twelve sweep
lanes failing together do not schedule twelve immediate probes.

What keeps working: the catalog opens even when the scan that normally
provides it failed, so the grid fills from the last successful scan.
Thumbnails come from the shards. Rating, flagging and collecting are catalog
writes that never touched the network. What stops is opening an original that
was never stored locally, and it now says so in those words instead of
reporting "network error: connection refused" over a photograph.

Work that is pure network is refused rather than left to fail slowly: the
metadata sweep, derived sync, and sidecar writes. The sweep would otherwise
spend a timeout per image across the whole library while the progress bar
implied something was happening. Deferring sidecars is a real gap rather than
a hidden one — a rating made offline reaches its sidecar only when that image
is judged again while connected — and it is recorded as such at the call site.

# The "On this device" filter

A chip beside the rating filters, narrowing the grid to images whose original
is held locally. It composes with the rating terms rather than replacing them,
so "five-star frames I can actually edit on this train" is one filter. The
predicate is SQL, like the rating terms and for the same reason: the count in
the header has to agree with the cells drawn.

It reads `image_cache.tier_actual`, which nothing writes yet — the next
commit fills it. Until then the chip honestly reports zero.

`Tier` gains an explicit on-disk encoding. The variants are ordered by
generosity and the derived `Ord` invites reordering them, which would
silently reinterpret every cached row; the round-trip test is what holds the
two in agreement.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
dtourolle
2026-08-11 20:36:50 +02:00
co-authored by Claude Opus 5
parent f6a100863e
commit cd75e5a4c6
18 changed files with 2604 additions and 109 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ui/dr-ui/src/library_ui.rs
+362 -22
View File
@@ -62,6 +62,19 @@ pub struct LibraryController {
needs_metadata: RefCell<Vec<bool>>,
/// Where in the catalog the current window starts. Scrubbing moves this.
offset: RefCell<usize>,
/// The first visible ordinal, kept so returning from the develop view lands
/// where the user left rather than at the top.
///
/// Recorded when the grid is left, not read back when it is re-entered:
/// `show-library` gates an `if` in the markup, so the whole grid subtree is
/// destroyed and rebuilt, and the rebuilt `Flickable` reports a scroll to
/// row 0 before anything can restore the old position. Capturing at the
/// moment of departure is the only value the reset cannot overwrite.
///
/// Separate from `offset`, which is the *loaded window*'s start and sits a
/// quarter-window above the view. Restoring that as a viewport position
/// would land the user consistently short of where they were.
resume_at: std::cell::Cell<usize>,
/// How many cells to load, derived from what the viewport can show.
///
/// A fixed count is wrong in both directions: too small on a maximised 4K
@@ -136,6 +149,21 @@ pub struct LibraryController {
/// that already holds borrows of other fields, and a `u64` needs no borrow
/// tracking.
generation: std::cell::Cell<u64>,
/// TRACES: FR-CAT-9
/// Whether the server is reachable, inferred from what the workers saw.
///
/// Lives on the controller rather than in a global because it is scoped to
/// one open library: signing into a different account starts a fresh
/// judgement, and carrying the old one over would report a server down
/// that was never contacted.
reachability: RefCell<dr_sync::Reachability>,
/// Narrow the grid to images whose original is stored locally.
///
/// A `Cell` beside `filter` rather than a field inside it: the rating
/// filter compiles to a SQL predicate over `versions`, while this one is a
/// predicate over the cache, and folding two different joins into one type
/// would put the cache schema inside a rating concept.
local_only: std::cell::Cell<bool>,
}
impl LibraryController {
@@ -148,6 +176,7 @@ impl LibraryController {
image_ids: RefCell::new(Vec::new()),
needs_metadata: RefCell::new(Vec::new()),
offset: RefCell::new(0),
resume_at: std::cell::Cell::new(0),
window: RefCell::new(INITIAL_WINDOW),
requested: RefCell::new(Default::default()),
scan_timer: RefCell::new(None),
@@ -164,9 +193,33 @@ impl LibraryController {
filter: RefCell::new(library::RatingFilter::default()),
sidecar_timer: RefCell::new(None),
generation: std::cell::Cell::new(0),
reachability: RefCell::new(dr_sync::Reachability::new()),
local_only: std::cell::Cell::new(false),
})
}
/// TRACES: FR-CAT-9
/// Whether the app currently believes the server is unreachable.
pub fn is_offline(&self) -> bool {
self.reachability.borrow().is_offline()
}
/// Whether the grid is narrowed to locally-stored originals.
pub fn local_only(&self) -> bool {
self.local_only.get()
}
/// Toggle the local-only filter, resetting the window.
///
/// The offset is cleared for the same reason a scope change clears it: the
/// filter changes which images exist as far as the grid is concerned, so a
/// position counted against the old set names a different photograph.
pub fn set_local_only(&self, on: bool) {
self.local_only.set(on);
*self.offset.borrow_mut() = 0;
self.requested.borrow_mut().clear();
}
/// The catalog handle, for [`crate::collections_ui`] to edit through.
pub fn catalog(&self) -> Rc<RefCell<Option<Catalog>>> {
self.catalog.clone()
@@ -378,6 +431,18 @@ fn drain_scan(
);
w.set_library_scanning(false);
// A completed scan is the strongest possible evidence
// the server is reachable, so it clears offline mode
// without needing a probe of its own.
if ctl
.reachability
.borrow_mut()
.mark_reachable(std::time::Instant::now())
{
log::info!("back online");
}
refresh_offline(&w, ctl);
// An incremental rescan lists almost nothing, so
// reporting the listed count would read as "0 images"
// on a library that is simply up to date.
@@ -413,10 +478,28 @@ fn drain_scan(
stop(&ctl.scan_timer);
return;
}
ScanMessage::Failed(e) => {
log::warn!("scan failed: {e}");
ScanMessage::Failed { message, offline } => {
log::warn!("scan failed: {message}");
w.set_library_scanning(false);
w.set_library_error(e.into());
if offline {
// Not an error state. The catalog from the last
// successful scan is still on disk and still
// accurate for everything already indexed, so the
// grid keeps working — it simply cannot learn about
// anything added on the server since.
ctl.reachability
.borrow_mut()
.mark_unreachable(message, std::time::Instant::now());
refresh_offline(&w, ctl);
// Show whatever the catalog holds. Without this the
// grid stays empty on a launch that began offline,
// which is precisely the case offline mode exists
// for.
open_catalog_for_offline(&w, ctl, &catalog_path, &coll_ctl);
} else {
w.set_library_error(message.into());
}
stop(&ctl.scan_timer);
return;
}
@@ -428,6 +511,119 @@ fn drain_scan(
*ctl.scan_timer.borrow_mut() = Some(timer);
}
/// Start a scan against the configured library.
///
/// Shared by the rescan button and by the offline banner's retry, which are
/// the same operation: a scan is the only request that both proves the server
/// is reachable and brings the catalog up to date. Keeping them one function
/// is what stops "retry" from quietly becoming a weaker probe than "rescan".
fn start_rescan(
window: &AppWindow,
ctl: &Rc<LibraryController>,
coll_ctl: &Rc<crate::collections_ui::CollectionsController>,
) {
let Some((creds, session, filter)) = ctl.session.borrow().clone() else {
return;
};
window.set_library_scanning(true);
window.set_library_error(slint::SharedString::new());
window.set_library_status("Rescanning…".into());
let path = library::catalog_path(&session.server, &session.user_id);
let rx = library::spawn_scan(
creds,
session.user_id.clone(),
session.root.clone(),
filter,
path.clone(),
);
drain_scan(window.as_weak(), ctl.clone(), coll_ctl.clone(), rx, path);
}
/// TRACES: FR-CAT-9
/// Paint the connectivity state into the window.
///
/// Called wherever reachability may have moved, rather than by the state
/// itself: `Reachability` is in `dr-sync` and knows nothing about a window,
/// which is what keeps it testable without a display server.
fn refresh_offline(window: &AppWindow, ctl: &Rc<LibraryController>) {
let reach = ctl.reachability.borrow();
let offline = reach.is_offline();
window.set_library_offline(offline);
window.set_library_offline_reason(reach.reason().unwrap_or_default().into());
window.set_library_offline_since(
reach
.offline_for(std::time::Instant::now())
.map(describe_duration)
.unwrap_or_default()
.into(),
);
// A stale scan error under an offline banner reports one problem twice.
if offline {
window.set_library_error("".into());
}
}
/// A coarse "how long ago", for the offline banner.
///
/// Deliberately imprecise: the user wants to know whether this just happened
/// or has been true for a while, and a live-counting seconds display would
/// draw the eye to a number that changes without meaning anything.
fn describe_duration(d: std::time::Duration) -> String {
let secs = d.as_secs();
if secs < 60 {
"just now".to_string()
} else if secs < 3600 {
format!("{}m ago", secs / 60)
} else {
format!("{}h ago", secs / 3600)
}
}
/// TRACES: FR-CAT-9
/// Show the catalog after a scan that could not reach the server.
///
/// The whole point of offline mode: a scan is how the grid normally gets its
/// catalog handle, so a failed one previously left the library empty even
/// though a complete catalog was sitting on disk from the last successful run.
/// The images are all still there, their thumbnails are in the shards, and
/// rating and collecting them are local writes.
///
/// The sweep is deliberately **not** started — it exists to fetch headers over
/// the network, so offline it would do nothing but fail once per image.
fn open_catalog_for_offline(
window: &AppWindow,
ctl: &Rc<LibraryController>,
catalog_path: &std::path::Path,
coll_ctl: &Rc<crate::collections_ui::CollectionsController>,
) {
if ctl.catalog.borrow().is_some() {
// Already open — a rescan that failed, rather than a launch that
// began offline. The grid is showing the catalog already.
load_window(window, ctl);
return;
}
match Catalog::open(catalog_path) {
Ok(cat) => {
crate::collections_ui::refresh_tree(window, coll_ctl, &cat);
*ctl.catalog.borrow_mut() = Some(cat);
load_window(window, ctl);
}
Err(e) => {
// No catalog and no server. This is the one genuinely empty case:
// a first run that never reached the server has nothing indexed.
log::warn!("offline with no local catalog: {e}");
window.set_library_error(
"Offline, and this library has not been scanned on this device yet.".into(),
);
}
}
}
/// Fill the model from the catalog and start fetching thumbnails.
///
/// Reads the window starting at the controller's current offset, which the
@@ -636,6 +832,13 @@ fn refresh_rating_counts(window: &AppWindow, catalog: &Catalog) {
let counts = dr_catalog::rating::rating_histogram(catalog.connection()).unwrap_or_default();
let as_i32: Vec<i32> = counts.iter().map(|n| *n as i32).collect();
window.set_library_rating_counts(slint::ModelRc::new(slint::VecModel::from(as_i32)));
// TRACES: FR-CAT-9
// How many originals are actually here, for the "On this device" chip.
// Shown for the same reason the star counts are: a filter that silently
// empties the grid reads as broken, and this one will legitimately be zero
// on a library nothing has been downloaded from yet.
window.set_library_local_count(library::local_original_count(catalog).unwrap_or(0) as i32);
}
/// Apply a judgement to a set of images: catalog first, then sidecars.
@@ -784,6 +987,25 @@ fn start_sidecar_writes(
if writes.is_empty() {
return;
}
// TRACES: FR-CAT-9
// Offline, this would stall on a timeout per sidecar, on the very
// keystroke path a cull is built for speed on (FR-CULL-1). The judgement
// itself is safe either way — `apply_judgement` has already committed it
// to the catalog, which is what the grid reads and what survives a
// restart.
//
// What is deferred is the sidecar, and with it the guarantee that the
// judgement survives a *catalog rebuild* (ARCH §6.12). There is no queue
// behind this yet, so a rating made offline is written to its sidecar only
// when that image is judged again while connected. That is a real gap
// rather than a hidden one: it trades a durability property that already
// depends on the network for a cull that stays responsive without it.
if ctl.is_offline() {
log::debug!("offline: skipping {} sidecar write(s)", writes.len());
return;
}
let Some((creds, session, _)) = ctl.session.borrow().clone() else {
return;
};
@@ -1013,6 +1235,22 @@ fn drain_thumbnails(
// carry no embedded preview.
ThumbnailMessage::Ready(t) => {
w.set_library_thumbs_done(w.get_library_thumbs_done() + 1);
// Bytes arrived, so the server is reachable. This is
// what clears the banner when a connection returns
// while the user is simply scrolling, without waiting
// for a probe or a manual retry.
if ctl_cb
.reachability
.borrow_mut()
.mark_reachable(std::time::Instant::now())
{
log::info!("back online");
refresh_offline(&w, &ctl_cb);
// The sweep was refused while offline, so nothing
// else would ever restart it — the grid would stay
// partially dated until the next launch.
start_sweep(&w, &ctl_cb);
}
if let Some(mut row) = model.row_data(t.row) {
row.thumbnail = to_slint_image(t.width, t.height, &t.rgba);
row.has_thumb = true;
@@ -1027,6 +1265,29 @@ fn drain_thumbnails(
model.set_row_data(row, r);
}
}
// TRACES: FR-CAT-9
ThumbnailMessage::Offline { reason } => {
log::info!("thumbnails stopped: {reason}");
ctl_cb
.reachability
.borrow_mut()
.mark_unreachable(reason, std::time::Instant::now());
refresh_offline(&w, &ctl_cb);
// The batch is over, so the progress counter must not
// be left showing a partial fetch that will never
// finish — it would spin in the header for ever.
w.set_library_thumbs_total(0);
w.set_library_thumbs_done(0);
// Cells left without pixels stay placeholders rather
// than being marked unavailable: the images are fine,
// and a reconnect should fill them in. Marking them
// would persist a verdict about the *file* from an
// event about the *connection*.
stop(&ctl_cb.thumb_timer);
return;
}
}
}
},
@@ -1078,6 +1339,16 @@ fn start_derived_sync(window: &AppWindow, ctl: &Rc<LibraryController>) {
return;
}
// TRACES: FR-CAT-9
// Nothing to push to and nothing to take. Attempting it would upload
// shards into a timeout and light the "Syncing…" indicator over work that
// cannot start; the shards are unchanged on disk and go out on the next
// sync once the server is back.
if ctl.is_offline() {
log::debug!("offline: skipping derived sync");
return;
}
let catalog_path = library::catalog_path(&session.server, &session.user_id);
let scratch = catalog_path
.parent()
@@ -1178,6 +1449,17 @@ fn start_sweep(window: &AppWindow, ctl: &Rc<LibraryController>) {
return;
};
// TRACES: FR-CAT-9
// The sweep is nothing but network reads — one header fetch per undated
// image, across the whole library. Offline it would spend a timeout on
// every one of them, running for hours to learn nothing, while the
// progress bar implied work was happening. It resumes on reconnect, and
// the images it has already dated stay dated.
if ctl.is_offline() {
log::debug!("offline: not starting the metadata sweep");
return;
}
let rx = library::spawn_sweep(
creds,
session.user_id.clone(),
@@ -1678,6 +1960,19 @@ where
let Some(w) = weak.upgrade() else { return };
let first_visible = first_visible.max(0) as usize;
// Remember where the view is, so leaving for the develop view and
// coming back returns here. Latched on every event rather than read
// at departure: by the time the grid is hidden its scroll position
// is only in the Flickable, which is about to be destroyed.
//
// Only while the grid is the visible page. Building and tearing
// down the subtree moves the viewport through 0, and accepting that
// would erase the position on the way out — the first return would
// work and every one after it would land at the top.
if w.get_show_library() {
ctl.resume_at.set(first_visible);
}
// Move the timeline marker with the view. Scrolling the grid is a
// way of moving through time just as scrubbing is, and a marker
// that only ever moved on a scrub sat still while the photographs
@@ -1857,12 +2152,39 @@ where
}
// Develop → grid.
//
// Returns to where the user left rather than to the top. `show-library`
// gates an `if` in the markup, so the grid is rebuilt from nothing and its
// Flickable starts at row 0; the position has to be replayed explicitly.
{
let weak = window.as_weak();
let ctl = ctl.clone();
window.on_back_to_library(move || {
if let Some(w) = weak.upgrade() {
w.set_show_library(true);
let Some(w) = weak.upgrade() else { return };
let resume = ctl.resume_at.get();
if resume > 0 {
// Through the same channel a scrub uses, and for the same
// reason: the viewport and the loaded window both have to move,
// or the cells are drawn thousands of rows from where the view
// sits.
//
// Centred like `on_library_scrolled` does it, so scrolling up
// from the restored position has loaded rows above it.
let window_size = *ctl.window.borrow();
*ctl.offset.borrow_mut() = resume.saturating_sub(window_size / 4);
ctl.requested.borrow_mut().clear();
load_window(&w, &ctl);
// Before the grid is shown, not after: the markup gates it on
// an `if`, and the rebuilt Flickable reads `scroll-to` in its
// `init`. Setting these afterwards would leave that init to run
// against the previous position.
w.set_library_scroll_to(resume as i32);
w.set_library_scroll_token(w.get_library_scroll_token() + 1);
}
w.set_show_library(true);
});
}
@@ -1872,23 +2194,7 @@ where
let coll_ctl = coll_ctl.clone();
window.on_library_rescan(move || {
let Some(w) = weak.upgrade() else { return };
let Some((creds, session, filter)) = ctl.session.borrow().clone() else {
return;
};
w.set_library_scanning(true);
w.set_library_error(slint::SharedString::new());
w.set_library_status("Rescanning…".into());
let path = library::catalog_path(&session.server, &session.user_id);
let rx = library::spawn_scan(
creds,
session.user_id.clone(),
session.root.clone(),
filter,
path.clone(),
);
drain_scan(w.as_weak(), ctl.clone(), coll_ctl.clone(), rx, path);
start_rescan(&w, &ctl, &coll_ctl);
});
}
@@ -2000,6 +2306,40 @@ where
refilter(&w, &ctl);
});
}
// TRACES: FR-CAT-9
// "On this device" — the images openable without a server. Composes with
// the rating terms rather than replacing them: "five-star frames I can
// actually edit on this train" is one filter, not a mode.
{
let weak = window.as_weak();
let ctl = ctl.clone();
window.on_library_toggle_local_only(move || {
let Some(w) = weak.upgrade() else { return };
let on = !ctl.local_only();
ctl.set_local_only(on);
ctl.filter.borrow_mut().local_only = on;
w.set_library_local_only(on);
refilter(&w, &ctl);
});
}
// TRACES: FR-CAT-9
// Retry now, rather than waiting out the backoff. A user who has just
// reconnected their wifi knows something the backoff does not.
{
let weak = window.as_weak();
let ctl = ctl.clone();
let coll_ctl = coll_ctl.clone();
window.on_library_retry_connection(move || {
let Some(w) = weak.upgrade() else { return };
log::info!("retrying the connection at the user's request");
// A rescan is the probe: it is the same request the scan worker
// makes, so a success proves reachability and repopulates the
// catalog in one pass rather than proving it twice.
start_rescan(&w, &ctl, &coll_ctl);
});
}
}
/// Reload the grid after the filter changed.