dtourolle/DarkRoom
1
0
Fork 0
Code Issues 39 Pull Requests Actions Packages Projects Releases 20 Wiki Activity

Thumbnail the whole library on request, and send the shards up

Browse Source 
The grid fetches a preview only for cells that are actually browsed, which is
the right posture over a link that must not be saturated to show one screen
(FR-NC-3). The cost is that the thumbnail store ends up holding the fraction of
the library someone happened to scroll past — and that store is the one derived
thing worth syncing, since a second device that downloads the shards gets a
full grid without touching a single RAW. So the complete set is worth an hour
of range fetches paid once, deliberately, on a machine that can afford it.

That is what this adds: a pass over every visible image with an `oc:fileid`,
launched from the settings page and reported in the activity register like
every other background job. The work list is what the store lacks rather than a
flag in the catalog, so it is resumable by construction and safe to press
twice.

Lane-parallel like the metadata sweep, but it does what that sweep declined to.
The store is `&mut` and cannot cross lanes, which is why dating the library
skips thumbnails entirely; here the lanes fetch, decode and *encode*, and only
the ~20 KB result crosses back to the one thread that owns the store and writes
the chunk. The parallelism is real and the single-writer rule is not bent.

Grid class only. The large class is ~860 MB of shards against ~200 MB on the
reference library, paid by every device that syncs them; a photograph looked at
closely still gets its large thumbnail from the interactive path.

Dates come free — the header a preview needs is the header EXIF lives in — and
the pass ends by pushing the shards to the server. A filled store that never
leaves this device would be most of the cost for none of the point.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
dtourolle
2026-08-22 19:34:59 +02:00
co-authored by Claude Opus 5
parent 28325af448
commit d91f1ec277
3 changed files with 681 additions and 42 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ui/dr-ui/src/library.rs
+465 -42
View File
@@ -1845,34 +1845,78 @@ pub fn spawn_thumbnails(
rx
}
/// Fetch a preview in two stages: header, then the exact preview range.
///
/// This is what FR-NC-3 specifies, and the single-stage version it replaces
/// was wrong in a way that looked like corruption: fetching a fixed prefix cut
/// the embedded JPEG partway through, and decoders render a truncated JPEG as
/// the top fraction of the frame rather than reporting an error.
async fn fetch_one(
backend: &NextcloudBackend,
store: Option<&mut ThumbStore>,
req: &ThumbnailRequest,
found_metadata: &mut Vec<MetadataFound>,
) -> ThumbnailMessage {
let id = RemoteId::Path(RemotePath::new(&req.path));
let fail = |reason: String| ThumbnailMessage::Unavailable {
row: req.row,
reason,
let preview = match fetch_preview(backend, req, found_metadata).await {
PreviewOutcome::Ready(p) => p,
PreviewOutcome::Unavailable(reason) => {
return ThumbnailMessage::Unavailable {
row: req.row,
reason,
}
}
PreviewOutcome::Offline(reason) => return ThumbnailMessage::Offline { reason },
};
// Persist for next time, and for every other client that syncs the shard.
// A store failure is logged and dropped: the pixels are already in hand,
// and refusing to display them because they could not be cached would be
// the wrong trade.
if let (Some(store), Some(file_id)) = (store, req.file_id) {
if let Some(thumb) = encode_preview(file_id, &preview) {
store_thumbnail(store, file_id, req.thumb_size, &thumb);
}
}
ThumbnailMessage::Ready(Box::new(ThumbnailReady {
row: req.row,
width: preview.width,
height: preview.height,
rgba: preview.rgba,
from_cache: false,
}))
}
/// What one fetch produced.
///
/// Separate from [`ThumbnailMessage`] because not every caller has a grid row
/// to report against or a store to write through. The whole-library pass
/// ([`spawn_thumbnail_sweep`]) fetches on several lanes at once and stores the
/// results on the one thread that owns the store, so it needs the pixels
/// *before* anything is written or addressed to a cell.
enum PreviewOutcome {
Ready(dr_decode::Preview),
/// This image has no usable preview. The batch continues past it.
Unavailable(String),
/// The server is unreachable, so nothing after this would succeed either.
Offline(String),
}
/// Fetch a preview in two stages: header, then the exact preview range.
///
/// This is what FR-NC-3 specifies, and the single-stage version it replaces
/// was wrong in a way that looked like corruption: fetching a fixed prefix cut
/// the embedded JPEG partway through, and decoders render a truncated JPEG as
/// the top fraction of the frame rather than reporting an error.
async fn fetch_preview(
backend: &NextcloudBackend,
req: &ThumbnailRequest,
found_metadata: &mut Vec<MetadataFound>,
) -> PreviewOutcome {
let id = RemoteId::Path(RemotePath::new(&req.path));
// A connection failure is not this image's verdict. Reported as such so
// the caller can stop the batch rather than marking sixty cells
// individually unpreviewable over one dropped connection — a state the
// grid would then keep until something forced a reload.
let classify = |e: dr_sync::RemoteError| {
if e.indicates_offline() {
ThumbnailMessage::Offline {
reason: e.to_string(),
}
PreviewOutcome::Offline(e.to_string())
} else {
fail(e.to_string())
PreviewOutcome::Unavailable(e.to_string())
}
};
@@ -1905,10 +1949,13 @@ async fn fetch_one(
let Some(loc) = dr_decode::locate_preview(&header, req.size) else {
// No locatable preview. Declining beats fetching the whole file:
// that is the 370 GB path FR-NC-3 exists to avoid.
return fail("no locatable embedded preview".into());
return PreviewOutcome::Unavailable("no locatable embedded preview".into());
};
if loc.len() > MAX_PREVIEW_BYTES {
return fail(format!("preview is {} bytes, too large", loc.len()));
return PreviewOutcome::Unavailable(format!(
"preview is {} bytes, too large",
loc.len()
));
}
// Stage two: exactly the preview's bytes.
@@ -1922,13 +1969,13 @@ async fn fetch_one(
// partial frame, so without this the broken result reaches the cache and
// the screen looking like a corrupt file.
if !dr_decode::is_complete_jpeg(&bytes) {
return fail("preview bytes are incomplete".into());
return PreviewOutcome::Unavailable("preview bytes are incomplete".into());
}
// Decode on the worker, never the UI thread.
let mut preview = match dr_decode::decode_jpeg(&bytes) {
Ok(p) => p,
Err(e) => return fail(e.to_string()),
Err(e) => return PreviewOutcome::Unavailable(e.to_string()),
};
preview.downscale_to(req.thumb_size.edge());
// Turn it the right way up before it is measured, cached or shown. An
@@ -1940,33 +1987,48 @@ async fn fetch_one(
// rather than the full preview's.
preview.apply_orientation(orientation);
// Persist for next time, and for every other client that syncs the shard.
// A store failure is logged and dropped: the pixels are already in hand,
// and refusing to display them because they could not be cached would be
// the wrong trade.
if let (Some(store), Some(file_id)) = (store, req.file_id) {
match dr_thumbs::encode_rgba(preview.width, preview.height, &preview.rgba) {
Ok(encoded) => {
let thumb = dr_thumbs::Thumbnail {
width: preview.width,
height: preview.height,
bytes: encoded,
};
if let Err(e) = store.put(file_id, req.thumb_size, &thumb) {
log::debug!("storing thumbnail {file_id}: {e}");
}
}
Err(e) => log::debug!("encoding thumbnail {file_id}: {e}"),
PreviewOutcome::Ready(preview)
}
/// Compress a decoded preview to what the store holds.
///
/// Split from the write so the whole-library pass can do it on the lane that
/// fetched the image: encoding is the one part of storing a thumbnail that
/// costs CPU rather than the store's lock, and it turns a 256 KB RGBA buffer
/// into ~20 KB before the chunk is handed to the single thread that owns the
/// store.
fn encode_preview(file_id: u64, preview: &dr_decode::Preview) -> Option<dr_thumbs::Thumbnail> {
match dr_thumbs::encode_rgba(preview.width, preview.height, &preview.rgba) {
Ok(bytes) => Some(dr_thumbs::Thumbnail {
width: preview.width,
height: preview.height,
bytes,
}),
Err(e) => {
log::debug!("encoding thumbnail {file_id}: {e}");
None
}
}
}
ThumbnailMessage::Ready(Box::new(ThumbnailReady {
row: req.row,
width: preview.width,
height: preview.height,
rgba: preview.rgba,
from_cache: false,
}))
/// Put a thumbnail in the store, logging rather than failing.
///
/// A store failure costs a re-fetch next time and nothing else — the pixels
/// are already in hand, and the caller has something to show or count either
/// way (ARCH §6.12: the store is derived, never authoritative).
fn store_thumbnail(
store: &mut ThumbStore,
file_id: u64,
size: dr_thumbs::ThumbSize,
thumb: &dr_thumbs::Thumbnail,
) -> bool {
match store.put(file_id, size, thumb) {
Ok(_) => true,
Err(e) => {
log::debug!("storing thumbnail {file_id}: {e}");
false
}
}
}
/// Parse EXIF out of a header and record it.
@@ -2463,6 +2525,291 @@ fn flush_sweep(catalog: &Catalog, found: &mut Vec<MetadataFound>) {
found.clear();
}
/// TRACES: FR-CAT-3 | FR-NC-3 | NFR-RES-4
/// The class the whole-library pass fills.
///
/// Grid only, deliberately. The large class is four times the transfer for a
/// detail only a zoomed cell or the loupe asks for — on the reference library
/// that is ~200 MB of shards against ~860 MB, paid by *every* device that
/// syncs them (see [`dr_thumbs::ThumbSize`]). A photograph actually looked at
/// closely still gets its large thumbnail from the interactive path.
const SWEEP_THUMB_SIZE: dr_thumbs::ThumbSize = dr_thumbs::ThumbSize::Grid;
/// Progress from the whole-library thumbnail pass.
#[derive(Debug)]
pub enum ThumbSweepMessage {
/// How many images still lack a thumbnail, counted once at the start.
Total(usize),
/// Another chunk finished. Carries cumulative counts.
Progress { done: usize, stored: usize },
Finished {
stored: usize,
failed: usize,
/// Stopped early because the server stopped answering. The pass is
/// resumable, so this is "come back later", not a failure.
offline: bool,
},
}
/// TRACES: FR-CAT-3 | FR-NC-3 | FR-NC-7
/// Thumbnail **every** image in the library, not just the ones browsed.
///
/// # Why this exists next to the grid's own fetching
///
/// The interactive path fills cells as they are scrolled past, which is the
/// right posture for a remote library (FR-NC-3) and the wrong one for handing
/// the result to a second device: a tablet that syncs the shards inherits only
/// the fraction of the library its sibling happened to look at. This is the
/// deliberate, user-launched version of the same work — an hour of range
/// fetches paid once, on the machine that can afford it, so every other client
/// gets a full grid for the cost of a few hundred MB (`derived_sync`).
///
/// # Shape, and why it borrows the metadata sweep's
///
/// Chunked and lane-parallel exactly as [`spawn_sweep`] is, for the same
/// reason: each image is ~0.6 s of round-trip latency and almost no
/// bandwidth, so the sequential version spends its life waiting. What differs
/// is the store — it is `&mut` and cannot be shared across lanes, which is why
/// the metadata sweep skips thumbnails entirely. Here the lanes fetch, decode
/// and *encode*, and only the ~20 KB result crosses back to this thread, which
/// owns the store and writes the chunk in one go. So the parallelism is real
/// and the single-writer rule is never bent.
///
/// Dates arrive free: the header a preview needs is the header EXIF lives in,
/// so an image this pass reaches is dated on the same fetch rather than
/// costing a second one.
///
/// Resumable by construction — the work list is what the store does not have,
/// so a kill costs the chunk in flight and nothing more. An image with no
/// locatable preview is retried on a later run; it is one header fetch, and
/// the alternative is a second piece of state that has to be invalidated when
/// a file is replaced.
pub fn spawn_thumbnail_sweep(
creds: AppCredentials,
user_id: String,
catalog_path: PathBuf,
store_dir: PathBuf,
) -> Receiver<ThumbSweepMessage> {
let (tx, rx) = std::sync::mpsc::channel();
std::thread::spawn(move || {
let finish_empty = |tx: &Sender<ThumbSweepMessage>| {
let _ = tx.send(ThumbSweepMessage::Finished {
stored: 0,
failed: 0,
offline: false,
});
};
let catalog = match Catalog::open(&catalog_path) {
Ok(c) => c,
Err(e) => {
log::warn!(
"thumbnail sweep: cannot open catalog at {}: {e}",
catalog_path.display()
);
finish_empty(&tx);
return;
}
};
// Unlike the grid's fetch, which carries on without a store and simply
// shows what it downloaded, a store that will not open ends this: the
// pass exists to fill it, and running an hour of transfers with
// nowhere to put them would be worse than not starting.
let mut store = match ThumbStore::open(&store_dir) {
Ok(s) => s,
Err(e) => {
log::warn!(
"thumbnail sweep: cannot open the thumbnail store at {}: {e}",
store_dir.display()
);
finish_empty(&tx);
return;
}
};
let wanted = match thumbnails_outstanding(&catalog, &store) {
Ok(w) => w,
Err(e) => {
log::warn!("thumbnail sweep: {e}");
finish_empty(&tx);
return;
}
};
let total = wanted.len();
if total == 0 {
log::info!("thumbnail sweep: every image already has a thumbnail");
finish_empty(&tx);
return;
}
log::info!("thumbnail sweep: {total} image(s) need a thumbnail");
if tx.send(ThumbSweepMessage::Total(total)).is_err() {
return;
}
let rt = match crate::net_runtime::build() {
Ok(rt) => rt,
Err(e) => {
log::warn!("thumbnail sweep: no runtime: {e}");
finish_empty(&tx);
return;
}
};
rt.block_on(async {
let backend = match NextcloudBackend::new(&creds, &user_id) {
Ok(b) => b,
Err(e) => {
log::warn!("thumbnail sweep: {e}");
finish_empty(&tx);
return;
}
};
let (mut done, mut stored, mut failed) = (0usize, 0usize, 0usize);
let mut offline = false;
let mut found = Vec::new();
for chunk in wanted.chunks(SWEEP_CHUNK) {
// Each lane owns a disjoint slice and its own output, so
// nothing is shared and no lock is needed. The store is not
// touched here — see the note on the function.
let lanes: Vec<Vec<&ThumbnailRequest>> = (0..SWEEP_LANES)
.map(|lane| chunk.iter().skip(lane).step_by(SWEEP_LANES).collect())
.collect();
let results = futures_join_all(lanes.into_iter().map(|lane| {
let backend = &backend;
async move {
let mut made: Vec<(u64, dr_thumbs::Thumbnail)> = Vec::new();
let mut found = Vec::new();
let mut attempted = 0usize;
let mut failed = 0usize;
let mut offline = false;
for req in lane {
// Enforced by the query, which joins `remote`: an
// image with no file id has nothing to key the
// store on and is not a candidate.
let Some(file_id) = req.file_id else { continue };
attempted += 1;
match fetch_preview(backend, req, &mut found).await {
PreviewOutcome::Ready(preview) => {
match encode_preview(file_id, &preview) {
Some(thumb) => made.push((file_id, thumb)),
None => failed += 1,
}
}
PreviewOutcome::Unavailable(reason) => {
log::debug!("thumbnail sweep: {}: {reason}", req.path);
failed += 1;
}
// Nothing after this would reach the server
// either, so the lane stops rather than
// spending a timeout per remaining image.
PreviewOutcome::Offline(reason) => {
log::info!("thumbnail sweep: server unreachable: {reason}");
attempted -= 1;
offline = true;
break;
}
}
}
(made, found, attempted, failed, offline)
}
}))
.await;
for (made, lane_found, attempted, lane_failed, lane_offline) in results {
done += attempted;
failed += lane_failed;
offline |= lane_offline;
found.extend(lane_found);
for (file_id, thumb) in made {
if store_thumbnail(&mut store, file_id, SWEEP_THUMB_SIZE, &thumb) {
stored += 1;
} else {
failed += 1;
}
}
}
// Committed per chunk rather than at the end, so a kill keeps
// every date read so far — the same bargain the metadata sweep
// makes, and for the same reason.
flush_sweep(&catalog, &mut found);
if tx
.send(ThumbSweepMessage::Progress { done, stored })
.is_err()
{
return;
}
if offline {
break;
}
}
flush_sweep(&catalog, &mut found);
log::info!("thumbnail sweep: {stored} stored, {failed} without a usable preview");
let _ = tx.send(ThumbSweepMessage::Finished {
stored,
failed,
offline,
});
});
});
rx
}
/// Every visible image on the server that the store has no grid thumbnail for.
///
/// Joined against `remote` rather than left-joined: the store is keyed on
/// Nextcloud's `oc:fileid` (FR-NC-5), so an image the scan recorded without
/// one cannot be stored and is not work this pass can do.
///
/// The whole list is built up front rather than re-queried per chunk, unlike
/// the metadata sweep: "does the store have this" is answered by the store's
/// index, which this thread is also the one writing, so a stale list is not a
/// risk the way a concurrently-dating grid made it one there.
fn thumbnails_outstanding(
catalog: &Catalog,
store: &ThumbStore,
) -> Result<Vec<ThumbnailRequest>, dr_catalog::CatalogError> {
let mut stmt = catalog.connection().prepare(&format!(
"SELECT i.id, i.source_ref, r.file_id, i.file_size, i.metadata_state
FROM images i
JOIN remote r ON r.image_id = i.id
WHERE r.file_id IS NOT NULL AND {VISIBLE}
ORDER BY i.id"
))?;
let rows = stmt
.query_map([], |r| {
let file_id = r.get::<_, Option<i64>>(2)?.map(|v| v as u64);
Ok(ThumbnailRequest {
thumb_size: SWEEP_THUMB_SIZE,
// No grid cell is waiting on this, so nothing consumes the row.
row: 0,
image_id: r.get(0)?,
path: r.get(1)?,
file_id,
size: r.get::<_, Option<i64>>(3)?.unwrap_or(0) as u64,
// The header this fetch reads is the one EXIF lives in, so an
// undated image is dated on the way past for nothing.
needs_metadata: r.get::<_, i64>(4)? < 2,
})
})?
.filter_map(Result::ok)
.filter(|req| {
req.file_id
.is_some_and(|id| !store.contains(id, SWEEP_THUMB_SIZE))
})
.collect();
Ok(rows)
}
/// Where an account's thumbnail shards live.
///
/// Beside the catalog rather than in the cache directory: these sync to the
@@ -2965,6 +3312,82 @@ mod tests {
}
}
#[test]
fn the_thumbnail_pass_asks_only_for_what_is_missing() {
// The work list is the whole point of the pass being resumable and of
// it being safe to press twice: it is derived from what the store
// lacks, not from a flag in the catalog. Three things it must respect
// — a thumbnail already stored, a trashed or shadowed image, and an
// image with no `oc:fileid`, which the store cannot key on at all.
let catalog = Catalog::in_memory().unwrap();
let c = catalog.connection();
c.execute(
"INSERT INTO roots(id, kind, label) VALUES (1, 'remote', 'lib')",
[],
)
.unwrap();
// (id, name, file_id, shadowed_by, trashed_at)
for (id, name, file_id, shadow, trashed) in [
(1i64, "a.CR2", Some(11i64), None, None),
(2, "b.CR2", Some(22), None, None),
(3, "b.JPG", Some(33), Some(2i64), None),
(4, "c.CR2", Some(44), None, Some(1000i64)),
// Scanned without a file id: nothing to key the store on.
(5, "d.CR2", None, None, None),
] {
c.execute(
"INSERT INTO images(id, root_id, source_ref, shadowed_by, trashed_at, added_at)
VALUES (?1, 1, ?2, ?3, ?4, 0)",
rusqlite::params![id, name, shadow, trashed],
)
.unwrap();
if let Some(file_id) = file_id {
c.execute(
"INSERT INTO remote(image_id, file_id) VALUES (?1, ?2)",
rusqlite::params![id, file_id],
)
.unwrap();
}
}
let dir = std::env::temp_dir().join(format!("dr-thumb-sweep-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
let mut store = ThumbStore::open(&dir).unwrap();
let all = thumbnails_outstanding(&catalog, &store).unwrap();
let names: Vec<&str> = all.iter().map(|r| r.path.as_str()).collect();
assert_eq!(
names,
vec!["a.CR2", "b.CR2"],
"shadowed, trashed and file-id-less images are not work this pass can do"
);
// Store one, and it drops out — this is what stops a second run
// re-fetching an hour of previews.
store
.put(
11,
SWEEP_THUMB_SIZE,
&dr_thumbs::Thumbnail {
width: 4,
height: 4,
bytes: vec![0xFF, 0xD8, 0xFF, 0xD9],
},
)
.unwrap();
let rest = thumbnails_outstanding(&catalog, &store).unwrap();
let names: Vec<&str> = rest.iter().map(|r| r.path.as_str()).collect();
assert_eq!(names, vec!["b.CR2"]);
// The large class is a different key, so filling the grid class does
// not make the pass think the library is done at another size.
assert!(!store.contains(11, dr_thumbs::ThumbSize::Large));
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn catalog_paths_separate_accounts() {
// Two accounts on one machine must not share an index, or one
ui/dr-ui/src/library_ui.rs
+176
View File
@@ -112,6 +112,11 @@ pub struct LibraryController {
thumb_timer: RefCell<Option<slint::Timer>>,
/// The whole-library sweep, which outlives any one grid window.
sweep_timer: RefCell<Option<slint::Timer>>,
/// TRACES: FR-CAT-3
/// Drains the whole-library thumbnail pass. Held for the same reason as
/// the others, and read as well as written: it is what stops a second
/// press of the button starting a pass alongside the first.
thumb_sweep_timer: RefCell<Option<slint::Timer>>,
/// Pushing shards and the catalog to the server.
sync_timer: RefCell<Option<slint::Timer>>,
/// Kept so a rescan can run without going back through the launch screen.
@@ -278,6 +283,7 @@ impl LibraryController {
scan_timer: RefCell::new(None),
thumb_timer: RefCell::new(None),
sweep_timer: RefCell::new(None),
thumb_sweep_timer: RefCell::new(None),
sync_timer: RefCell::new(None),
session: RefCell::new(None),
scope: RefCell::new(None),
@@ -681,6 +687,7 @@ pub fn open(
*ctl.session.borrow_mut() = Some((creds.clone(), session.clone(), filter.clone()));
window.set_show_library(true);
window.set_library_open(true);
window.set_library_scanning(true);
window.set_library_error(slint::SharedString::new());
window.set_library_status("Starting…".into());
@@ -3165,6 +3172,162 @@ fn start_sweep(window: &AppWindow, ctl: &Rc<LibraryController>) {
*ctl.sweep_timer.borrow_mut() = Some(timer);
}
/// TRACES: FR-CAT-3 | FR-NC-3 | FR-NC-7
/// Thumbnail every photograph in the library, then hand the shards over.
///
/// # Why it is asked for rather than assumed
///
/// The grid's own fetching is demand-driven on purpose: a remote library is
/// browsed over a connection that must not be saturated to show one screen
/// (FR-NC-3). But that leaves the thumbnail store holding only what has been
/// looked at, and the store is the one derived thing worth syncing — a second
/// device that downloads the shards gets a full grid without touching a RAW.
/// So the complete set is worth an hour of transfers *once*, on a machine
/// plugged in, at a moment the user chose. That is this.
///
/// The sync at the end is not a separate courtesy: a filled store that never
/// leaves this device is most of the cost for none of the point.
fn start_thumbnail_sweep(window: &AppWindow, ctl: &Rc<LibraryController>) {
let Some((creds, session, _)) = ctl.session.borrow().clone() else {
return;
};
// Already running. A second pass would re-fetch everything the first is
// part way through — the work list is built from what the store lacks, and
// the first pass has not filled it yet.
if ctl.thumb_sweep_timer.borrow().is_some() && window.get_library_thumbnailing() {
return;
}
// TRACES: FR-CAT-9
// Every image here is a range fetch. Offline the pass would spend a
// timeout per photograph and store nothing, so it is refused rather than
// started — and said so, because this one was pressed deliberately and a
// button that silently does nothing is worse than one that declines.
if ctl.is_offline() {
log::debug!("offline: not starting the thumbnail pass");
window.set_library_status("Offline — thumbnailing needs the server.".into());
return;
}
window.set_library_thumbnailing(true);
let rx = library::spawn_thumbnail_sweep(
creds,
session.user_id.clone(),
library::catalog_path(&session.server, &session.user_id),
library::thumbs_dir(&session.server, &session.user_id),
);
let timer = slint::Timer::default();
let weak = window.as_weak();
let ctl_cb = ctl.clone();
let job = ctl.activity.begin(
crate::activity::Kind::Thumbnails,
"Thumbnailing the library",
);
// The total arrives in the first message rather than up front — counting
// it means asking the store about every image — and the progress messages
// after it carry only cumulative counts, so it is kept here rather than
// re-derived.
let total = std::cell::Cell::new(0usize);
timer.start(
slint::TimerMode::Repeated,
// As slow as the metadata sweep's drain, and for the same reason: this
// runs for tens of minutes and a chunk lands every few seconds.
std::time::Duration::from_millis(400),
move || {
let Some(w) = weak.upgrade() else { return };
loop {
let msg = match rx.try_recv() {
Ok(m) => m,
Err(std::sync::mpsc::TryRecvError::Empty) => return,
Err(std::sync::mpsc::TryRecvError::Disconnected) => {
w.set_library_thumbnailing(false);
job.fail("stopped without finishing");
stop(&ctl_cb.thumb_sweep_timer);
return;
}
};
match msg {
library::ThumbSweepMessage::Total(n) => {
total.set(n);
job.total(n);
w.set_library_status(format!("Thumbnailing {n} photograph(s)…").into());
}
library::ThumbSweepMessage::Progress { done, stored } => {
job.progress(done, total.get());
// Dates arrive on the same headers, so the histogram
// grows as this runs — the visible sign it is working
// on a page that is not the grid.
if stored > 0 {
let borrow = ctl_cb.catalog.borrow();
if let Some(catalog) = borrow.as_ref() {
refresh_timeline(&w, catalog, &ctl_cb);
}
}
}
library::ThumbSweepMessage::Finished {
stored,
failed,
offline,
} => {
log::info!(
"thumbnail pass finished: {stored} stored, {failed} without a preview"
);
w.set_library_thumbnailing(false);
stop(&ctl_cb.thumb_sweep_timer);
let summary = if offline {
format!("{stored} stored · stopped, server unreachable")
} else if failed > 0 {
format!("{stored} stored · {failed} with no usable preview")
} else {
format!("{stored} stored")
};
w.set_library_status(summary.clone().into());
// Finished rather than failed even when the server
// dropped: the pass is resumable and what it did build
// is stored, and a red row would invite the user to
// redo work that is already done.
job.finish(summary);
{
let borrow = ctl_cb.catalog.borrow();
if let Some(catalog) = borrow.as_ref() {
refresh_timeline(&w, catalog, &ctl_cb);
}
}
// Cells that showed a placeholder have pixels now, and
// the grid asks the store once per photograph — so
// without this the library the pass just thumbnailed
// stays blank until something else reloads the window.
if w.get_show_library() {
ctl_cb.requested.borrow_mut().clear();
schedule_reload(&w, &ctl_cb);
}
// The point of the pass: the shards go to the server so
// every other device inherits them. Skipped when the
// connection is already gone — the sync would only
// discover the same thing more slowly.
if !offline {
start_derived_sync(&w, &ctl_cb);
}
return;
}
}
}
},
);
*ctl.thumb_sweep_timer.borrow_mut() = Some(timer);
}
/// Rebuild the timeline histogram from the catalog.
///
/// Bucket size follows the span of the library, the way darktable's does:
@@ -3783,6 +3946,19 @@ pub fn wire<F>(
});
}
// TRACES: FR-CAT-3 | FR-NC-3
// Thumbnail the whole library, from the settings page. It ends in a sync
// of its own, so this is the long version of the button above.
{
let weak = window.as_weak();
let ctl = ctl.clone();
window.on_library_thumbnail_all(move || {
if let Some(w) = weak.upgrade() {
start_thumbnail_sweep(&w, &ctl);
}
});
}
// A column-count change moves which cells begin a row, and month headings
// sit on row-leading cells.
//