dtourolle/DarkRoom
1
0
Fork 0
Code Issues 39 Pull Requests Actions Packages Projects Releases 20 Wiki Activity

Thumbnail the whole library on request, and send the shards up

Browse Source 
The grid fetches a preview only for cells that are actually browsed, which is
the right posture over a link that must not be saturated to show one screen
(FR-NC-3). The cost is that the thumbnail store ends up holding the fraction of
the library someone happened to scroll past — and that store is the one derived
thing worth syncing, since a second device that downloads the shards gets a
full grid without touching a single RAW. So the complete set is worth an hour
of range fetches paid once, deliberately, on a machine that can afford it.

That is what this adds: a pass over every visible image with an `oc:fileid`,
launched from the settings page and reported in the activity register like
every other background job. The work list is what the store lacks rather than a
flag in the catalog, so it is resumable by construction and safe to press
twice.

Lane-parallel like the metadata sweep, but it does what that sweep declined to.
The store is `&mut` and cannot cross lanes, which is why dating the library
skips thumbnails entirely; here the lanes fetch, decode and *encode*, and only
the ~20 KB result crosses back to the one thread that owns the store and writes
the chunk. The parallelism is real and the single-writer rule is not bent.

Grid class only. The large class is ~860 MB of shards against ~200 MB on the
reference library, paid by every device that syncs them; a photograph looked at
closely still gets its large thumbnail from the interactive path.

Dates come free — the header a preview needs is the header EXIF lives in — and
the pass ends by pushing the shards to the server. A filled store that never
leaves this device would be most of the cost for none of the point.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
dtourolle
2026-08-22 19:34:59 +02:00
co-authored by Claude Opus 5
parent 28325af448
commit d91f1ec277
3 changed files with 681 additions and 42 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ui/dr-ui/src/library_ui.rs
+176
View File
@@ -112,6 +112,11 @@ pub struct LibraryController {
thumb_timer: RefCell<Option<slint::Timer>>,
/// The whole-library sweep, which outlives any one grid window.
sweep_timer: RefCell<Option<slint::Timer>>,
/// TRACES: FR-CAT-3
/// Drains the whole-library thumbnail pass. Held for the same reason as
/// the others, and read as well as written: it is what stops a second
/// press of the button starting a pass alongside the first.
thumb_sweep_timer: RefCell<Option<slint::Timer>>,
/// Pushing shards and the catalog to the server.
sync_timer: RefCell<Option<slint::Timer>>,
/// Kept so a rescan can run without going back through the launch screen.
@@ -278,6 +283,7 @@ impl LibraryController {
scan_timer: RefCell::new(None),
thumb_timer: RefCell::new(None),
sweep_timer: RefCell::new(None),
thumb_sweep_timer: RefCell::new(None),
sync_timer: RefCell::new(None),
session: RefCell::new(None),
scope: RefCell::new(None),
@@ -681,6 +687,7 @@ pub fn open(
*ctl.session.borrow_mut() = Some((creds.clone(), session.clone(), filter.clone()));
window.set_show_library(true);
window.set_library_open(true);
window.set_library_scanning(true);
window.set_library_error(slint::SharedString::new());
window.set_library_status("Starting…".into());
@@ -3165,6 +3172,162 @@ fn start_sweep(window: &AppWindow, ctl: &Rc<LibraryController>) {
*ctl.sweep_timer.borrow_mut() = Some(timer);
}
/// TRACES: FR-CAT-3 | FR-NC-3 | FR-NC-7
/// Thumbnail every photograph in the library, then hand the shards over.
///
/// # Why it is asked for rather than assumed
///
/// The grid's own fetching is demand-driven on purpose: a remote library is
/// browsed over a connection that must not be saturated to show one screen
/// (FR-NC-3). But that leaves the thumbnail store holding only what has been
/// looked at, and the store is the one derived thing worth syncing — a second
/// device that downloads the shards gets a full grid without touching a RAW.
/// So the complete set is worth an hour of transfers *once*, on a machine
/// plugged in, at a moment the user chose. That is this.
///
/// The sync at the end is not a separate courtesy: a filled store that never
/// leaves this device is most of the cost for none of the point.
fn start_thumbnail_sweep(window: &AppWindow, ctl: &Rc<LibraryController>) {
let Some((creds, session, _)) = ctl.session.borrow().clone() else {
return;
};
// Already running. A second pass would re-fetch everything the first is
// part way through — the work list is built from what the store lacks, and
// the first pass has not filled it yet.
if ctl.thumb_sweep_timer.borrow().is_some() && window.get_library_thumbnailing() {
return;
}
// TRACES: FR-CAT-9
// Every image here is a range fetch. Offline the pass would spend a
// timeout per photograph and store nothing, so it is refused rather than
// started — and said so, because this one was pressed deliberately and a
// button that silently does nothing is worse than one that declines.
if ctl.is_offline() {
log::debug!("offline: not starting the thumbnail pass");
window.set_library_status("Offline — thumbnailing needs the server.".into());
return;
}
window.set_library_thumbnailing(true);
let rx = library::spawn_thumbnail_sweep(
creds,
session.user_id.clone(),
library::catalog_path(&session.server, &session.user_id),
library::thumbs_dir(&session.server, &session.user_id),
);
let timer = slint::Timer::default();
let weak = window.as_weak();
let ctl_cb = ctl.clone();
let job = ctl.activity.begin(
crate::activity::Kind::Thumbnails,
"Thumbnailing the library",
);
// The total arrives in the first message rather than up front — counting
// it means asking the store about every image — and the progress messages
// after it carry only cumulative counts, so it is kept here rather than
// re-derived.
let total = std::cell::Cell::new(0usize);
timer.start(
slint::TimerMode::Repeated,
// As slow as the metadata sweep's drain, and for the same reason: this
// runs for tens of minutes and a chunk lands every few seconds.
std::time::Duration::from_millis(400),
move || {
let Some(w) = weak.upgrade() else { return };
loop {
let msg = match rx.try_recv() {
Ok(m) => m,
Err(std::sync::mpsc::TryRecvError::Empty) => return,
Err(std::sync::mpsc::TryRecvError::Disconnected) => {
w.set_library_thumbnailing(false);
job.fail("stopped without finishing");
stop(&ctl_cb.thumb_sweep_timer);
return;
}
};
match msg {
library::ThumbSweepMessage::Total(n) => {
total.set(n);
job.total(n);
w.set_library_status(format!("Thumbnailing {n} photograph(s)…").into());
}
library::ThumbSweepMessage::Progress { done, stored } => {
job.progress(done, total.get());
// Dates arrive on the same headers, so the histogram
// grows as this runs — the visible sign it is working
// on a page that is not the grid.
if stored > 0 {
let borrow = ctl_cb.catalog.borrow();
if let Some(catalog) = borrow.as_ref() {
refresh_timeline(&w, catalog, &ctl_cb);
}
}
}
library::ThumbSweepMessage::Finished {
stored,
failed,
offline,
} => {
log::info!(
"thumbnail pass finished: {stored} stored, {failed} without a preview"
);
w.set_library_thumbnailing(false);
stop(&ctl_cb.thumb_sweep_timer);
let summary = if offline {
format!("{stored} stored · stopped, server unreachable")
} else if failed > 0 {
format!("{stored} stored · {failed} with no usable preview")
} else {
format!("{stored} stored")
};
w.set_library_status(summary.clone().into());
// Finished rather than failed even when the server
// dropped: the pass is resumable and what it did build
// is stored, and a red row would invite the user to
// redo work that is already done.
job.finish(summary);
{
let borrow = ctl_cb.catalog.borrow();
if let Some(catalog) = borrow.as_ref() {
refresh_timeline(&w, catalog, &ctl_cb);
}
}
// Cells that showed a placeholder have pixels now, and
// the grid asks the store once per photograph — so
// without this the library the pass just thumbnailed
// stays blank until something else reloads the window.
if w.get_show_library() {
ctl_cb.requested.borrow_mut().clear();
schedule_reload(&w, &ctl_cb);
}
// The point of the pass: the shards go to the server so
// every other device inherits them. Skipped when the
// connection is already gone — the sync would only
// discover the same thing more slowly.
if !offline {
start_derived_sync(&w, &ctl_cb);
}
return;
}
}
}
},
);
*ctl.thumb_sweep_timer.borrow_mut() = Some(timer);
}
/// Rebuild the timeline histogram from the catalog.
///
/// Bucket size follows the span of the library, the way darktable's does:
@@ -3783,6 +3946,19 @@ pub fn wire<F>(
});
}
// TRACES: FR-CAT-3 | FR-NC-3
// Thumbnail the whole library, from the settings page. It ends in a sync
// of its own, so this is the long version of the button above.
{
let weak = window.as_weak();
let ctl = ctl.clone();
window.on_library_thumbnail_all(move || {
if let Some(w) = weak.upgrade() {
start_thumbnail_sweep(&w, &ctl);
}
});
}
// A column-count change moves which cells begin a row, and month headings
// sit on row-leading cells.
//