Fetch the photographs around the open one ahead of the step to them
Benchmarks / CPU and I/O (per commit) (push) Successful in 3m59s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / android-image (push) Canceled after 0s
🐳 Android image / Build and push (push) Canceled after 0s
Build and test / Android (aarch64) (push) Canceled after 0s
Build and test / windows-image (push) Canceled after 0s
🐳 Windows image / Build and push (push) Canceled after 0s
Build and test / Windows (x86_64, cross) (push) Canceled after 0s
Build and test / Layer separation (push) Canceled after 0s
Build and test / Desktop (Linux) (push) Canceled after 16m26s
Traceability / Requirement traces (push) Canceled after 0s
Benchmarks / CPU and I/O (per commit) (push) Successful in 3m59s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / android-image (push) Canceled after 0s
🐳 Android image / Build and push (push) Canceled after 0s
Build and test / Android (aarch64) (push) Canceled after 0s
Build and test / windows-image (push) Canceled after 0s
🐳 Windows image / Build and push (push) Canceled after 0s
Build and test / Windows (x86_64, cross) (push) Canceled after 0s
Build and test / Layer separation (push) Canceled after 0s
Build and test / Desktop (Linux) (push) Canceled after 16m26s
Traceability / Requirement traces (push) Canceled after 0s
Walking the photo roll was one download per frame: every step showed "Downloading…" over an empty canvas while tens of megabytes came down, and moving between a pair of near-identical frames paid that a dozen times. Now, once the opened photograph has landed, the ones around it are fetched into the originals cache while it is being looked at, so the next step is a disk read. A single worker serves the latest wish only, closest first and working outwards — next, previous, next-but-one, previous-but-one… — one file at a time. Each open replaces the wish, so a fast walk never leaves a trail of stale downloads competing with the one being waited on. A process-wide in-flight registry makes a click on a photograph that is still being fetched ahead wait for that transfer and read it from disk, rather than start a second download of the same file. How far each side is a setting under STORAGE — Off, 2, 5, 10 or 20, defaulting to 5 — and it is moot while "keep originals after opening" is off, since a fetch the cache would discard on arrival is transfer for nothing. Nothing is fetched ahead while offline. The transfers show in the activity list while they run and are removed when they end.
This commit is contained in:
@@ -508,6 +508,28 @@ pub struct CacheSettings {
|
|||||||
/// no bandwidth and saves the whole transfer next time. Off is for metered
|
/// no bandwidth and saves the whole transfer next time. Off is for metered
|
||||||
/// or small-disk devices, where the user would rather re-fetch than store.
|
/// or small-disk devices, where the user would rather re-fetch than store.
|
||||||
pub keep_opened_originals: bool,
|
pub keep_opened_originals: bool,
|
||||||
|
|
||||||
|
/// How many photographs either side of the open one are fetched into the
|
||||||
|
/// cache ahead of being asked for, so a step along the roll is a disk
|
||||||
|
/// read rather than a download. Zero fetches nothing ahead.
|
||||||
|
///
|
||||||
|
/// Each side, so the total is twice this; and closest first, working
|
||||||
|
/// outwards, so a small number still covers the step most likely to be
|
||||||
|
/// taken next. Bounded by [`Self::AHEAD_CHOICES`] because every unit is a
|
||||||
|
/// whole RAW file: 10 each side is a few hundred megabytes per open,
|
||||||
|
/// which a wired desktop shrugs at and a phone on a hotel connection does
|
||||||
|
/// not. Moot while `keep_opened_originals` is off — a fetch the cache
|
||||||
|
/// would discard on arrival is not made.
|
||||||
|
pub fetch_ahead: u32,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl CacheSettings {
|
||||||
|
/// The look-ahead depths the settings page offers, each side.
|
||||||
|
///
|
||||||
|
/// Snapped to, not clamped, for the reason `LibrarySettings::BAR_CHOICES`
|
||||||
|
/// is: the page lights the chip that matches, and a number it does not
|
||||||
|
/// offer would leave every chip dark.
|
||||||
|
pub const AHEAD_CHOICES: [u32; 5] = [0, 2, 5, 10, 20];
|
||||||
}
|
}
|
||||||
|
|
||||||
/// 8 GB of passively cached originals — roughly 250 full-frame RAWs.
|
/// 8 GB of passively cached originals — roughly 250 full-frame RAWs.
|
||||||
@@ -532,6 +554,10 @@ impl Default for CacheSettings {
|
|||||||
original_budget_bytes: Some(DEFAULT_ORIGINAL_BUDGET_BYTES),
|
original_budget_bytes: Some(DEFAULT_ORIGINAL_BUDGET_BYTES),
|
||||||
thumbnail_budget_bytes: Some(DEFAULT_THUMBNAIL_BUDGET_BYTES),
|
thumbnail_budget_bytes: Some(DEFAULT_THUMBNAIL_BUDGET_BYTES),
|
||||||
keep_opened_originals: true,
|
keep_opened_originals: true,
|
||||||
|
// Enough that the next few steps in either direction are already
|
||||||
|
// here by the time they are taken, without a click on one frame
|
||||||
|
// committing a phone to half a gigabyte.
|
||||||
|
fetch_ahead: 5,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1175,6 +1201,15 @@ impl Settings {
|
|||||||
.unwrap_or(LibrarySettings::default().timeline_bars);
|
.unwrap_or(LibrarySettings::default().timeline_bars);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// The same snap, for the same page.
|
||||||
|
if !CacheSettings::AHEAD_CHOICES.contains(&self.cache.fetch_ahead) {
|
||||||
|
let wanted = self.cache.fetch_ahead;
|
||||||
|
self.cache.fetch_ahead = CacheSettings::AHEAD_CHOICES
|
||||||
|
.into_iter()
|
||||||
|
.min_by_key(|n| n.abs_diff(wanted))
|
||||||
|
.unwrap_or(CacheSettings::default().fetch_ahead);
|
||||||
|
}
|
||||||
|
|
||||||
// A zero-pixel or zero-percent export produces no image. Nudged to the
|
// A zero-pixel or zero-percent export produces no image. Nudged to the
|
||||||
// smallest thing that does, rather than back to the default: the user
|
// smallest thing that does, rather than back to the default: the user
|
||||||
// clearly wanted "small", and silently restoring 2048 would ignore
|
// clearly wanted "small", and silently restoring 2048 would ignore
|
||||||
@@ -1410,6 +1445,10 @@ mod tests {
|
|||||||
Some(DEFAULT_THUMBNAIL_BUDGET_BYTES)
|
Some(DEFAULT_THUMBNAIL_BUDGET_BYTES)
|
||||||
);
|
);
|
||||||
assert!(cache.keep_opened_originals);
|
assert!(cache.keep_opened_originals);
|
||||||
|
assert!(
|
||||||
|
CacheSettings::AHEAD_CHOICES.contains(&cache.fetch_ahead),
|
||||||
|
"the default look-ahead must be one the page can light"
|
||||||
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
@@ -1740,6 +1779,24 @@ mod tests {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn sanitise_snaps_a_hand_edited_look_ahead_to_an_offered_one() {
|
||||||
|
let mut s = Settings::default();
|
||||||
|
s.cache.fetch_ahead = 7;
|
||||||
|
s.sanitise();
|
||||||
|
assert_eq!(s.cache.fetch_ahead, 5);
|
||||||
|
|
||||||
|
s.cache.fetch_ahead = 100;
|
||||||
|
s.sanitise();
|
||||||
|
assert_eq!(s.cache.fetch_ahead, 20);
|
||||||
|
|
||||||
|
for n in CacheSettings::AHEAD_CHOICES {
|
||||||
|
s.cache.fetch_ahead = n;
|
||||||
|
s.sanitise();
|
||||||
|
assert_eq!(s.cache.fetch_ahead, n, "an offered depth is left alone");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn sanitise_rescues_a_zero_dimension() {
|
fn sanitise_rescues_a_zero_dimension() {
|
||||||
let mut s = Settings::default();
|
let mut s = Settings::default();
|
||||||
|
|||||||
+11
-11
@@ -25,7 +25,7 @@ Sampling a neutral is the first move of the tonal pass — every colour judgemen
|
|||||||
|
|
||||||
Anchored on the fingers' midpoint, and on the pointer, so the gesture reads as magnifying the picture rather than sliding it about. Double-tap is the way to an exact 1:1; this is the way to everything in between.
|
Anchored on the fingers' midpoint, and on the pointer, so the gesture reads as magnifying the picture rather than sliding it about. Double-tap is the way to an exact 1:1; this is the way to everything in between.
|
||||||
|
|
||||||
<sub>`ui/dr-ui/ui/app.slint:1974`</sub>
|
<sub>`ui/dr-ui/ui/app.slint:1980`</sub>
|
||||||
|
|
||||||
### Move a magnified photograph about
|
### Move a magnified photograph about
|
||||||
|
|
||||||
@@ -34,7 +34,7 @@ Anchored on the fingers' midpoint, and on the pointer, so the gesture reads as m
|
|||||||
|
|
||||||
Only once there is something outside the viewport to reach, which is why the cursor becomes a hand exactly then. The view is clamped to the frame: panning past the edge would show undefined area beside the photograph, and that reads as a rendering fault rather than as the end of the picture.
|
Only once there is something outside the viewport to reach, which is why the cursor becomes a hand exactly then. The view is clamped to the frame: panning past the edge would show undefined area beside the photograph, and that reads as a rendering fault rather than as the end of the picture.
|
||||||
|
|
||||||
<sub>`ui/dr-ui/ui/app.slint:2065`</sub>
|
<sub>`ui/dr-ui/ui/app.slint:2071`</sub>
|
||||||
|
|
||||||
### Paint a mask by hand
|
### Paint a mask by hand
|
||||||
|
|
||||||
@@ -43,7 +43,7 @@ Only once there is something outside the viewport to reach, which is why the cur
|
|||||||
|
|
||||||
A model's mask stops inside a shoulder and leaks into the hair, and no single edge control fixes two errors that go opposite ways. The whole stroke is one step in the history, so taking a mark back costs one press however long it took to make.
|
A model's mask stops inside a shoulder and leaks into the hair, and no single edge control fixes two errors that go opposite ways. The whole stroke is one step in the history, so taking a mark back costs one press however long it took to make.
|
||||||
|
|
||||||
<sub>`ui/dr-ui/ui/app.slint:2152`</sub>
|
<sub>`ui/dr-ui/ui/app.slint:2158`</sub>
|
||||||
|
|
||||||
### Take back the last change
|
### Take back the last change
|
||||||
|
|
||||||
@@ -53,7 +53,7 @@ A model's mask stops inside a shoulder and leaks into the hair, and no single ed
|
|||||||
|
|
||||||
A whole drag is one step, so undo takes back a decision rather than a frame of a gesture. The list is there because arriving six steps back costs what arriving from one does.
|
A whole drag is one step, so undo takes back a decision rather than a frame of a gesture. The list is there because arriving six steps back costs what arriving from one does.
|
||||||
|
|
||||||
<sub>`ui/dr-ui/ui/app.slint:2373`</sub>
|
<sub>`ui/dr-ui/ui/app.slint:2379`</sub>
|
||||||
|
|
||||||
### Do it again after taking it back
|
### Do it again after taking it back
|
||||||
|
|
||||||
@@ -61,7 +61,7 @@ A whole drag is one step, so undo takes back a decision rather than a frame of a
|
|||||||
- **Pointer** — Click it, or press Redo in the History header
|
- **Pointer** — Click it, or press Redo in the History header
|
||||||
- **Keyboard** — Ctrl+Shift+Z
|
- **Keyboard** — Ctrl+Shift+Z
|
||||||
|
|
||||||
<sub>`ui/dr-ui/ui/app.slint:2386`</sub>
|
<sub>`ui/dr-ui/ui/app.slint:2392`</sub>
|
||||||
|
|
||||||
### Copy the settings from this photograph
|
### Copy the settings from this photograph
|
||||||
|
|
||||||
@@ -71,7 +71,7 @@ A whole drag is one step, so undo takes back a decision rather than a frame of a
|
|||||||
|
|
||||||
The panel is the copy that has to work: a tablet has no modifier key to hold and no menu bar to hang the action from. The shortcut is an accelerator for a control that is on screen either way.
|
The panel is the copy that has to work: a tablet has no modifier key to hold and no menu bar to hang the action from. The shortcut is an accelerator for a control that is on screen either way.
|
||||||
|
|
||||||
<sub>`ui/dr-ui/ui/app.slint:2419`</sub>
|
<sub>`ui/dr-ui/ui/app.slint:2425`</sub>
|
||||||
|
|
||||||
### Paste the settings onto this photograph
|
### Paste the settings onto this photograph
|
||||||
|
|
||||||
@@ -81,7 +81,7 @@ The panel is the copy that has to work: a tablet has no modifier key to hold and
|
|||||||
|
|
||||||
The button names what would be pasted — "3 adjustments", and whether the crop is coming with it — which the shortcut cannot say. Both paste the same scope.
|
The button names what would be pasted — "3 adjustments", and whether the crop is coming with it — which the shortcut cannot say. Both paste the same scope.
|
||||||
|
|
||||||
<sub>`ui/dr-ui/ui/app.slint:2431`</sub>
|
<sub>`ui/dr-ui/ui/app.slint:2437`</sub>
|
||||||
|
|
||||||
### Change which group of adjustments is on screen
|
### Change which group of adjustments is on screen
|
||||||
|
|
||||||
@@ -91,7 +91,7 @@ The button names what would be pasted — "3 adjustments", and whether the crop
|
|||||||
|
|
||||||
The groups are whatever the operation set declares itself to be about, so there are as many as the pipeline has and no key can be assigned to one of them by name. Stepping is the binding that survives a node being added.
|
The groups are whatever the operation set declares itself to be about, so there are as many as the pipeline has and no key can be assigned to one of them by name. Stepping is the binding that survives a node being added.
|
||||||
|
|
||||||
<sub>`ui/dr-ui/ui/app.slint:2459`</sub>
|
<sub>`ui/dr-ui/ui/app.slint:2465`</sub>
|
||||||
|
|
||||||
### Look at the photograph at 1:1
|
### Look at the photograph at 1:1
|
||||||
|
|
||||||
@@ -101,7 +101,7 @@ The groups are whatever the operation set declares itself to be about, so there
|
|||||||
|
|
||||||
Noise reduction and capture sharpening are judgements about single pixels, and a fitted view averages several of the file's into each one on screen — so the frame looks softer than it is and the correction goes too far. The point and the magnification survive opening the next photograph, which is what makes checking the same eye across forty portraits forty keystrokes rather than forty pans.
|
Noise reduction and capture sharpening are judgements about single pixels, and a fitted view averages several of the file's into each one on screen — so the frame looks softer than it is and the correction goes too far. The point and the magnification survive opening the next photograph, which is what makes checking the same eye across forty portraits forty keystrokes rather than forty pans.
|
||||||
|
|
||||||
<sub>`ui/dr-ui/ui/app.slint:2494`</sub>
|
<sub>`ui/dr-ui/ui/app.slint:2500`</sub>
|
||||||
|
|
||||||
### Move to the next or previous photograph
|
### Move to the next or previous photograph
|
||||||
|
|
||||||
@@ -111,7 +111,7 @@ Noise reduction and capture sharpening are judgements about single pixels, and a
|
|||||||
|
|
||||||
The edit on screen is saved on the way out, so stepping through a folder is as much a departure as going back to the grid and loses nothing.
|
The edit on screen is saved on the way out, so stepping through a folder is as much a departure as going back to the grid and loses nothing.
|
||||||
|
|
||||||
<sub>`ui/dr-ui/ui/app.slint:2546`</sub>
|
<sub>`ui/dr-ui/ui/app.slint:2552`</sub>
|
||||||
|
|
||||||
### See the photograph before you edited it
|
### See the photograph before you edited it
|
||||||
|
|
||||||
@@ -121,7 +121,7 @@ The edit on screen is saved on the way out, so stepping through a folder is as m
|
|||||||
|
|
||||||
Held rather than toggled, and no split screen: a split halves the working image on the tablet the column was sized for, and the comparison photographers describe making is a flick back and forth. It takes no history step, so checking whether a frame is overcooked costs nothing to undo afterwards.
|
Held rather than toggled, and no split screen: a split halves the working image on the tablet the column was sized for, and the comparison photographers describe making is a flick back and forth. It takes no history step, so checking whether a frame is overcooked costs nothing to undo afterwards.
|
||||||
|
|
||||||
<sub>`ui/dr-ui/ui/app.slint:2670`</sub>
|
<sub>`ui/dr-ui/ui/app.slint:2676`</sub>
|
||||||
|
|
||||||
### Put one control back to its default
|
### Put one control back to its default
|
||||||
|
|
||||||
|
|||||||
+64
-64
File diff suppressed because one or more lines are too long
@@ -1315,6 +1315,12 @@ pub fn run(paths: Vec<PathBuf>) -> Result<()> {
|
|||||||
// develop side reads `paths` to rebuild its browsing list when an image is
|
// develop side reads `paths` to rebuild its browsing list when an image is
|
||||||
// opened from the grid.
|
// opened from the grid.
|
||||||
let library = library_ui::LibraryController::new(activity.clone());
|
let library = library_ui::LibraryController::new(activity.clone());
|
||||||
|
// TRACES: FR-NC-6a | FR-UI-4
|
||||||
|
// The photographs either side of the open one, fetched into the cache
|
||||||
|
// while it is being looked at, so a step along the roll is a disk read.
|
||||||
|
// Fed from the develop open below; its transfers are shown here.
|
||||||
|
let prefetch = Rc::new(library::Prefetcher::new());
|
||||||
|
show_prefetches(&prefetch, &activity);
|
||||||
// Hoisted out of the launch block below because the settings clipboard
|
// Hoisted out of the launch block below because the settings clipboard
|
||||||
// needs it too: a paste onto "the selection" reads the selection from
|
// needs it too: a paste onto "the selection" reads the selection from
|
||||||
// here, and that wiring happens once the develop session exists.
|
// here, and that wiring happens once the develop session exists.
|
||||||
@@ -1597,6 +1603,7 @@ pub fn run(paths: Vec<PathBuf>) -> Result<()> {
|
|||||||
let stored = settings.snapshot();
|
let stored = settings.snapshot();
|
||||||
library.set_cache_budget(stored.cache.original_budget_bytes);
|
library.set_cache_budget(stored.cache.original_budget_bytes);
|
||||||
library.set_keep_opened_originals(stored.cache.keep_opened_originals);
|
library.set_keep_opened_originals(stored.cache.keep_opened_originals);
|
||||||
|
library.set_fetch_ahead(stored.cache.fetch_ahead);
|
||||||
library.set_write_xmp_sidecars(stored.library.write_xmp_sidecars);
|
library.set_write_xmp_sidecars(stored.library.write_xmp_sidecars);
|
||||||
library.set_timeline_bars(stored.library.timeline_bars);
|
library.set_timeline_bars(stored.library.timeline_bars);
|
||||||
library.set_face_model_id(stored.faces.detector.model_id());
|
library.set_face_model_id(stored.faces.detector.model_id());
|
||||||
@@ -1729,6 +1736,7 @@ pub fn run(paths: Vec<PathBuf>) -> Result<()> {
|
|||||||
// cache would sit over budget indefinitely.
|
// cache would sit over budget indefinitely.
|
||||||
lib.set_cache_budget(s.cache.original_budget_bytes);
|
lib.set_cache_budget(s.cache.original_budget_bytes);
|
||||||
lib.set_keep_opened_originals(s.cache.keep_opened_originals);
|
lib.set_keep_opened_originals(s.cache.keep_opened_originals);
|
||||||
|
lib.set_fetch_ahead(s.cache.fetch_ahead);
|
||||||
lib.set_write_xmp_sidecars(s.library.write_xmp_sidecars);
|
lib.set_write_xmp_sidecars(s.library.write_xmp_sidecars);
|
||||||
if let Some(w) = weak.upgrade() {
|
if let Some(w) = weak.upgrade() {
|
||||||
refresh_export_label(&w, &ctl);
|
refresh_export_label(&w, &ctl);
|
||||||
@@ -2359,6 +2367,7 @@ pub fn run(paths: Vec<PathBuf>) -> Result<()> {
|
|||||||
let open_image = open_image.clone();
|
let open_image = open_image.clone();
|
||||||
let viewport = viewport.clone();
|
let viewport = viewport.clone();
|
||||||
let inspection = inspection.clone();
|
let inspection = inspection.clone();
|
||||||
|
let prefetch = prefetch.clone();
|
||||||
*open_from_library.borrow_mut() = Some(Rc::new(move |path: String| {
|
*open_from_library.borrow_mut() = Some(Rc::new(move |path: String| {
|
||||||
let Some(w) = weak.upgrade() else { return };
|
let Some(w) = weak.upgrade() else { return };
|
||||||
|
|
||||||
@@ -2447,6 +2456,7 @@ pub fn run(paths: Vec<PathBuf>) -> Result<()> {
|
|||||||
// Cloned for the timer closure: the outer callback is an `Fn` and
|
// Cloned for the timer closure: the outer callback is an `Fn` and
|
||||||
// may run again for the next photograph.
|
// may run again for the next photograph.
|
||||||
let library = library.clone();
|
let library = library.clone();
|
||||||
|
let prefetch = prefetch.clone();
|
||||||
let path = path.clone();
|
let path = path.clone();
|
||||||
let viewport = viewport.clone();
|
let viewport = viewport.clone();
|
||||||
let inspection = inspection.clone();
|
let inspection = inspection.clone();
|
||||||
@@ -2484,6 +2494,14 @@ pub fn run(paths: Vec<PathBuf>) -> Result<()> {
|
|||||||
job.finish(activity::describe_bytes(bytes.len() as u64));
|
job.finish(activity::describe_bytes(bytes.len() as u64));
|
||||||
log::info!("{name}: {} bytes fetched", bytes.len());
|
log::info!("{name}: {} bytes fetched", bytes.len());
|
||||||
|
|
||||||
|
// TRACES: FR-NC-6a | FR-UI-4
|
||||||
|
// Now, and not when the click happened: started earlier
|
||||||
|
// they would share the connection with the download the
|
||||||
|
// user is watching, and the photograph on screen is the
|
||||||
|
// one that matters. Before the decode below, so the
|
||||||
|
// transfers are under way while the UI thread is busy.
|
||||||
|
prefetch_neighbours(&library, &prefetch, &path);
|
||||||
|
|
||||||
match load_bytes(gpu.as_ref(), &bytes) {
|
match load_bytes(gpu.as_ref(), &bytes) {
|
||||||
Ok(l) => {
|
Ok(l) => {
|
||||||
w.set_load_error("".into());
|
w.set_load_error("".into());
|
||||||
@@ -3773,6 +3791,86 @@ pub fn run(paths: Vec<PathBuf>) -> Result<()> {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/// TRACES: FR-NC-6a
|
/// TRACES: FR-NC-6a
|
||||||
|
/// TRACES: FR-NC-6a | FR-UI-4
|
||||||
|
/// Ask for the photographs around `path` — as many each side as the settings
|
||||||
|
/// say — to be fetched into the cache.
|
||||||
|
///
|
||||||
|
/// Every gate the prefetcher documents is applied here, because this is the
|
||||||
|
/// side that can see the settings and the network: nothing while offline, and
|
||||||
|
/// nothing into a cache that would not keep the bytes. A neighbour the cache
|
||||||
|
/// cannot place — a row that has scrolled out of the window — is left out
|
||||||
|
/// rather than fetched to nowhere.
|
||||||
|
///
|
||||||
|
/// Called on every landing, including one with no neighbours: an empty wish
|
||||||
|
/// cancels whatever the previous photograph asked for, which is right — those
|
||||||
|
/// were *its* neighbours.
|
||||||
|
fn prefetch_neighbours(
|
||||||
|
library: &Rc<library_ui::LibraryController>,
|
||||||
|
prefetch: &library::Prefetcher,
|
||||||
|
path: &str,
|
||||||
|
) {
|
||||||
|
let Some(conn) = library.credentials() else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
let jobs: Vec<library::PrefetchJob> = if library.is_offline() {
|
||||||
|
Vec::new()
|
||||||
|
} else {
|
||||||
|
library
|
||||||
|
.neighbours_of(path, library.fetch_ahead())
|
||||||
|
.into_iter()
|
||||||
|
.filter_map(|path| {
|
||||||
|
let cache = library.cache_context(&path).filter(|c| c.store)?;
|
||||||
|
Some(library::PrefetchJob { path, cache })
|
||||||
|
})
|
||||||
|
.collect()
|
||||||
|
};
|
||||||
|
if !jobs.is_empty() {
|
||||||
|
log::debug!("fetching {} neighbour(s) of {path} ahead", jobs.len());
|
||||||
|
}
|
||||||
|
prefetch.want(conn, jobs);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// TRACES: FR-NC-6a
|
||||||
|
/// Show the prefetcher's transfers in the activity list while they run.
|
||||||
|
///
|
||||||
|
/// A row each, like the download the user is waiting on, so a transfer that
|
||||||
|
/// is using the connection is never invisible. Removed rather than kept when
|
||||||
|
/// it ends, on the same grounds as a thumbnail batch: routine work the user
|
||||||
|
/// never asked for by name would push a failed transfer out of the list. A
|
||||||
|
/// prefetch that fails says nothing here — the click that wants the
|
||||||
|
/// photograph will try again and report in its own row.
|
||||||
|
fn show_prefetches(prefetch: &Rc<library::Prefetcher>, activity: &Rc<activity::ActivityLog>) {
|
||||||
|
let prefetch = prefetch.clone();
|
||||||
|
let activity = activity.clone();
|
||||||
|
let mut live: std::collections::HashMap<String, activity::Activity> = Default::default();
|
||||||
|
// Held by its own closure for the life of the window, like every other
|
||||||
|
// drain here.
|
||||||
|
let timer = Rc::new(slint::Timer::default());
|
||||||
|
let held = timer.clone();
|
||||||
|
timer.start(
|
||||||
|
slint::TimerMode::Repeated,
|
||||||
|
std::time::Duration::from_millis(200),
|
||||||
|
move || {
|
||||||
|
let _keep = &held;
|
||||||
|
for event in prefetch.poll() {
|
||||||
|
match event {
|
||||||
|
library::PrefetchEvent::Started(path) => {
|
||||||
|
let name = path.rsplit('/').next().unwrap_or(&path);
|
||||||
|
let job = activity
|
||||||
|
.begin(activity::Kind::Download, format!("Fetching {name} ahead"));
|
||||||
|
live.insert(path, job);
|
||||||
|
}
|
||||||
|
library::PrefetchEvent::Ended(path) => {
|
||||||
|
if let Some(job) = live.remove(&path) {
|
||||||
|
job.finish_quietly();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
/// What the originals cache is holding, for the settings page.
|
/// What the originals cache is holding, for the settings page.
|
||||||
///
|
///
|
||||||
/// Pinned and passive are reported separately because they answer different
|
/// Pinned and passive are reported separately because they answer different
|
||||||
|
|||||||
+409
-77
@@ -2424,26 +2424,6 @@ pub struct CacheContext {
|
|||||||
pub store: bool,
|
pub store: bool,
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Fetch one file in full, for opening it in develop.
|
|
||||||
///
|
|
||||||
/// Deliberately *not* the preview path. Browsing fetches a range and decodes
|
|
||||||
/// an embedded JPEG (FR-NC-3); develop needs every byte, because demosaic
|
|
||||||
/// needs every photosite. On a RAW file that is tens of megabytes, which is
|
|
||||||
/// why this is a click-triggered download and not something the grid does.
|
|
||||||
///
|
|
||||||
/// # Read-through
|
|
||||||
///
|
|
||||||
/// With a `cache`, this checks disk before the network and stores what it
|
|
||||||
/// downloads. That is what makes opening the same photograph twice cost one
|
|
||||||
/// transfer, and what leaves a working session's images openable offline
|
|
||||||
/// without anyone having pinned anything.
|
|
||||||
///
|
|
||||||
/// A cache miss is not an error and a cache failure is not fatal: both fall
|
|
||||||
/// through to the network, which is exactly the behaviour that existed before
|
|
||||||
/// the cache did.
|
|
||||||
///
|
|
||||||
/// Returns the bytes on a channel rather than blocking: the download runs on
|
|
||||||
/// its own thread and the UI stays live, exactly as thumbnail fetching does.
|
|
||||||
/// TRACES: FR-CAT-8 | FR-DEV-6
|
/// TRACES: FR-CAT-8 | FR-DEV-6
|
||||||
/// Fetch and parse the sidecar beside one image.
|
/// Fetch and parse the sidecar beside one image.
|
||||||
///
|
///
|
||||||
@@ -2570,80 +2550,356 @@ pub fn spawn_sidecar_fetch(
|
|||||||
rx
|
rx
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Fetch one file in full, for opening it in develop.
|
||||||
|
///
|
||||||
|
/// Deliberately *not* the preview path. Browsing fetches a range and decodes
|
||||||
|
/// an embedded JPEG (FR-NC-3); develop needs every byte, because demosaic
|
||||||
|
/// needs every photosite. On a RAW file that is tens of megabytes, which is
|
||||||
|
/// why this is a click-triggered download and not something the grid does.
|
||||||
|
///
|
||||||
|
/// # Read-through
|
||||||
|
///
|
||||||
|
/// With a `cache`, this checks disk before the network and stores what it
|
||||||
|
/// downloads. That is what makes opening the same photograph twice cost one
|
||||||
|
/// transfer, and what leaves a working session's images openable offline
|
||||||
|
/// without anyone having pinned anything.
|
||||||
|
///
|
||||||
|
/// A cache miss is not an error and a cache failure is not fatal: both fall
|
||||||
|
/// through to the network, which is exactly the behaviour that existed before
|
||||||
|
/// the cache did.
|
||||||
|
///
|
||||||
|
/// Returns the bytes on a channel rather than blocking: the download runs on
|
||||||
|
/// its own thread and the UI stays live, exactly as thumbnail fetching does.
|
||||||
|
/// The work itself is [`fetch_original`], which is also what the
|
||||||
|
/// [`Prefetcher`] runs — one transfer path, so a photograph fetched ahead is
|
||||||
|
/// stored exactly as one fetched on a click.
|
||||||
pub fn spawn_full_fetch(
|
pub fn spawn_full_fetch(
|
||||||
conn: Connection,
|
conn: Connection,
|
||||||
path: String,
|
path: String,
|
||||||
cache: Option<CacheContext>,
|
cache: Option<CacheContext>,
|
||||||
) -> Receiver<Result<Vec<u8>, FetchFailure>> {
|
) -> Receiver<Result<Vec<u8>, FetchFailure>> {
|
||||||
let (tx, rx) = std::sync::mpsc::channel();
|
let (tx, rx) = std::sync::mpsc::channel();
|
||||||
|
|
||||||
std::thread::spawn(move || {
|
std::thread::spawn(move || {
|
||||||
// Opened on this thread: `rusqlite::Connection` is not `Send`, and the
|
let _ = tx.send(fetch_original(conn, &path, cache.as_ref()));
|
||||||
// UI thread's handle cannot be borrowed across the spawn.
|
});
|
||||||
let cached = cache.as_ref().and_then(|c| {
|
rx
|
||||||
let store = dr_catalog::Cache::open(&c.dir, c.budget).ok()?;
|
}
|
||||||
let conn = Catalog::open(&c.catalog_path).ok()?;
|
|
||||||
Some((store, conn))
|
|
||||||
});
|
|
||||||
|
|
||||||
if let (Some(c), Some((store, conn))) = (cache.as_ref(), cached.as_ref()) {
|
/// TRACES: FR-NC-6a
|
||||||
match store.load(conn.connection(), c.image, now_secs()) {
|
/// Every original transfer that is under way right now, by remote path.
|
||||||
Ok(Some(bytes)) => {
|
///
|
||||||
log::info!("{path}: {} bytes from the local cache", bytes.len());
|
/// **One file, one transfer.** The [`Prefetcher`] fetches the photographs
|
||||||
let _ = tx.send(Ok(bytes));
|
/// beside the open one before they are asked for, and the whole point is that
|
||||||
return;
|
/// the user then asks for one of them — often while it is still coming down.
|
||||||
}
|
/// Without this the click would miss the cache, start a second download of
|
||||||
Ok(None) => {}
|
/// the same file, and the two would halve each other's bandwidth for the rest
|
||||||
// A cache that cannot be read is a cache miss, not a failure
|
/// of the transfer. With it, the click finds the path claimed, waits for the
|
||||||
// to open the photograph.
|
/// prefetch to store its bytes, and reads them from disk.
|
||||||
Err(e) => log::debug!("cache lookup for {path}: {e}"),
|
///
|
||||||
|
/// Process-wide rather than passed in, because the property it enforces is
|
||||||
|
/// process-wide: there is no caller for whom two concurrent downloads of one
|
||||||
|
/// file is the right answer. Keyed on the path rather than the image id
|
||||||
|
/// because that is the one name every caller has.
|
||||||
|
static IN_FLIGHT: std::sync::LazyLock<InFlight> = std::sync::LazyLock::new(InFlight::default);
|
||||||
|
|
||||||
|
#[derive(Default)]
|
||||||
|
struct InFlight {
|
||||||
|
busy: std::sync::Mutex<std::collections::HashSet<String>>,
|
||||||
|
freed: std::sync::Condvar,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl InFlight {
|
||||||
|
/// Take `path` for this thread, or wait for whoever holds it.
|
||||||
|
///
|
||||||
|
/// `Some` is a claim, released when the guard drops. `None` means another
|
||||||
|
/// thread held the path and has now let it go — so the caller's cache
|
||||||
|
/// check is worth repeating, because that thread has very probably just
|
||||||
|
/// stored what the caller was about to download.
|
||||||
|
fn claim(&self, path: &str) -> Option<InFlightGuard<'_>> {
|
||||||
|
let mut busy = self.busy.lock().unwrap_or_else(|e| e.into_inner());
|
||||||
|
if busy.insert(path.to_string()) {
|
||||||
|
return Some(InFlightGuard {
|
||||||
|
of: self,
|
||||||
|
path: path.to_string(),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
while busy.contains(path) {
|
||||||
|
busy = self.freed.wait(busy).unwrap_or_else(|e| e.into_inner());
|
||||||
|
}
|
||||||
|
None
|
||||||
|
}
|
||||||
|
|
||||||
|
fn release(&self, path: &str) {
|
||||||
|
self.busy
|
||||||
|
.lock()
|
||||||
|
.unwrap_or_else(|e| e.into_inner())
|
||||||
|
.remove(path);
|
||||||
|
self.freed.notify_all();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A claim on a path, dropped on every exit from the fetch — a failed
|
||||||
|
/// download must free the path too, or the click waiting on it never wakes.
|
||||||
|
struct InFlightGuard<'a> {
|
||||||
|
of: &'a InFlight,
|
||||||
|
path: String,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Drop for InFlightGuard<'_> {
|
||||||
|
fn drop(&mut self) {
|
||||||
|
self.of.release(&self.path);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The blocking body of [`spawn_full_fetch`]: cache, then in-flight registry,
|
||||||
|
/// then network, storing what it downloads when the cache says to.
|
||||||
|
fn fetch_original(
|
||||||
|
conn: Connection,
|
||||||
|
path: &str,
|
||||||
|
cache: Option<&CacheContext>,
|
||||||
|
) -> Result<Vec<u8>, FetchFailure> {
|
||||||
|
// Opened on this thread: `rusqlite::Connection` is not `Send`, and the
|
||||||
|
// UI thread's handle cannot be borrowed across the spawn.
|
||||||
|
let cached = cache.and_then(|c| {
|
||||||
|
let store = dr_catalog::Cache::open(&c.dir, c.budget).ok()?;
|
||||||
|
let conn = Catalog::open(&c.catalog_path).ok()?;
|
||||||
|
Some((store, conn))
|
||||||
|
});
|
||||||
|
|
||||||
|
let from_cache = || {
|
||||||
|
let (c, (store, conn)) = (cache?, cached.as_ref()?);
|
||||||
|
match store.load(conn.connection(), c.image, now_secs()) {
|
||||||
|
Ok(Some(bytes)) => {
|
||||||
|
log::info!("{path}: {} bytes from the local cache", bytes.len());
|
||||||
|
Some(bytes)
|
||||||
|
}
|
||||||
|
Ok(None) => None,
|
||||||
|
// A cache that cannot be read is a cache miss, not a failure
|
||||||
|
// to open the photograph.
|
||||||
|
Err(e) => {
|
||||||
|
log::debug!("cache lookup for {path}: {e}");
|
||||||
|
None
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
// Miss, claim, and if the claim had to wait, look again: the thread that
|
||||||
|
// held the path has finished with it, and what it fetched is on disk.
|
||||||
|
let _claim = loop {
|
||||||
|
if let Some(bytes) = from_cache() {
|
||||||
|
return Ok(bytes);
|
||||||
|
}
|
||||||
|
if let Some(claim) = IN_FLIGHT.claim(path) {
|
||||||
|
break claim;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
let rt = crate::net_runtime::build().map_err(FetchFailure::local)?;
|
||||||
|
|
||||||
|
rt.block_on(async {
|
||||||
|
let backend = crate::remote::connect(&conn).map_err(FetchFailure::local)?;
|
||||||
|
|
||||||
|
let id = RemoteId::Path(RemotePath::new(path));
|
||||||
|
let bytes = backend.get(&id, None).await?;
|
||||||
|
|
||||||
|
// Store before returning, so the bytes are on disk by the time the
|
||||||
|
// image is on screen. Doing it after would leave a window where
|
||||||
|
// closing the app immediately lost the download.
|
||||||
|
if let (Some(c), Some((store, conn))) = (cache.filter(|c| c.store), cached.as_ref()) {
|
||||||
|
// `pinned: false` — this is the passive population. A pin is
|
||||||
|
// something the user asks for explicitly; opening an image is
|
||||||
|
// not that, and treating it as one would make the pinned set
|
||||||
|
// grow silently and never be evicted.
|
||||||
|
if let Err(e) = store.store(conn.connection(), c.image, path, &bytes, false, now_secs())
|
||||||
|
{
|
||||||
|
log::debug!("caching {path}: {e}");
|
||||||
|
} else if let Err(e) = store.enforce(conn.connection()) {
|
||||||
|
log::debug!("enforcing the cache budget: {e}");
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
let rt = match crate::net_runtime::build() {
|
Ok(bytes)
|
||||||
Ok(rt) => rt,
|
})
|
||||||
Err(e) => {
|
}
|
||||||
let _ = tx.send(Err(FetchFailure::local(e)));
|
|
||||||
return;
|
/// TRACES: FR-NC-6a | FR-UI-4
|
||||||
|
/// One original to fetch ahead of its being asked for.
|
||||||
|
pub struct PrefetchJob {
|
||||||
|
pub path: String,
|
||||||
|
pub cache: CacheContext,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What the [`Prefetcher`] is doing, for the activity list.
|
||||||
|
pub enum PrefetchEvent {
|
||||||
|
/// A transfer has started for this path.
|
||||||
|
Started(String),
|
||||||
|
/// And has ended — stored, or not; either way the row can go.
|
||||||
|
Ended(String),
|
||||||
|
}
|
||||||
|
|
||||||
|
/// TRACES: FR-NC-6a | FR-UI-4
|
||||||
|
/// Fetches the photographs beside the open one into the cache, ahead of the
|
||||||
|
/// step that asks for them.
|
||||||
|
///
|
||||||
|
/// **Why this exists.** Walking the photo roll is one click per frame, and
|
||||||
|
/// without this every click is a download of tens of megabytes with a
|
||||||
|
/// "Downloading…" line over an empty canvas. A photographer moving between a
|
||||||
|
/// pair of near-identical frames does that a dozen times. Fetching the two
|
||||||
|
/// neighbours while the current photograph is being looked at turns the next
|
||||||
|
/// step into a disk read, which is what makes stepping feel like stepping.
|
||||||
|
///
|
||||||
|
/// **One worker, one wish.** A single thread serves the *latest* request and
|
||||||
|
/// nothing older. Each open replaces the previous wish outright, so a fast
|
||||||
|
/// walk along the roll does not leave a trail of stale downloads competing
|
||||||
|
/// with the one the user is actually waiting on; a job already under way is
|
||||||
|
/// finished rather than abandoned, because the bytes are mostly here. Jobs
|
||||||
|
/// run in the order given — next before previous, since that is the way a
|
||||||
|
/// roll is mostly walked — and one at a time, so two neighbours never halve
|
||||||
|
/// each other's bandwidth.
|
||||||
|
///
|
||||||
|
/// **What it never does.** It never fetches into a cache that would not keep
|
||||||
|
/// the bytes: the caller only hands it jobs whose cache stores, because a
|
||||||
|
/// prefetch that is discarded on arrival is pure transfer for nothing — and
|
||||||
|
/// "keep opened originals" being off is the user saying this device is
|
||||||
|
/// metered or small (FR-NC-6). It never starts while offline, for the same
|
||||||
|
/// reason. And it fetches only the immediate neighbours: originals are
|
||||||
|
/// "explicit pin or on-demand open only", and ±1 is as far as "on demand"
|
||||||
|
/// honestly stretches.
|
||||||
|
pub struct Prefetcher {
|
||||||
|
shared: std::sync::Arc<PrefetchShared>,
|
||||||
|
events: Receiver<PrefetchEvent>,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Default)]
|
||||||
|
struct PrefetchShared {
|
||||||
|
wanted: std::sync::Mutex<Wanted>,
|
||||||
|
changed: std::sync::Condvar,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The latest wish, and a generation so the worker can tell it has been
|
||||||
|
/// replaced mid-list.
|
||||||
|
#[derive(Default)]
|
||||||
|
struct Wanted {
|
||||||
|
generation: u64,
|
||||||
|
conn: Option<Connection>,
|
||||||
|
jobs: Vec<PrefetchJob>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Wanted {
|
||||||
|
/// Replace whatever was wanted with `jobs`.
|
||||||
|
fn replace(&mut self, conn: Connection, jobs: Vec<PrefetchJob>) {
|
||||||
|
self.generation += 1;
|
||||||
|
self.conn = Some(conn);
|
||||||
|
self.jobs = jobs;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whether a wish taken at `generation` is still the current one.
|
||||||
|
fn is_current(&self, generation: u64) -> bool {
|
||||||
|
self.generation == generation
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Default for Prefetcher {
|
||||||
|
fn default() -> Self {
|
||||||
|
Self::new()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Prefetcher {
|
||||||
|
/// Start the worker. It sleeps until the first [`Self::want`].
|
||||||
|
pub fn new() -> Self {
|
||||||
|
let shared = std::sync::Arc::new(PrefetchShared::default());
|
||||||
|
let (tx, events) = std::sync::mpsc::channel();
|
||||||
|
let worker = shared.clone();
|
||||||
|
std::thread::Builder::new()
|
||||||
|
.name("prefetch".into())
|
||||||
|
.spawn(move || serve_prefetches(&worker, &tx))
|
||||||
|
.expect("spawning the prefetch worker");
|
||||||
|
Self { shared, events }
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Fetch these, in this order, instead of whatever was asked for before.
|
||||||
|
///
|
||||||
|
/// An empty list is a valid wish: it cancels the rest of the previous
|
||||||
|
/// one, and is what a photograph with no neighbours in the window asks.
|
||||||
|
pub fn want(&self, conn: Connection, jobs: Vec<PrefetchJob>) {
|
||||||
|
self.shared
|
||||||
|
.wanted
|
||||||
|
.lock()
|
||||||
|
.unwrap_or_else(|e| e.into_inner())
|
||||||
|
.replace(conn, jobs);
|
||||||
|
self.shared.changed.notify_one();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Everything the worker has reported since the last poll.
|
||||||
|
pub fn poll(&self) -> Vec<PrefetchEvent> {
|
||||||
|
std::iter::from_fn(|| self.events.try_recv().ok()).collect()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The worker: take the current wish, serve it job by job, stop the moment it
|
||||||
|
/// is superseded, sleep until the next one.
|
||||||
|
fn serve_prefetches(shared: &PrefetchShared, events: &Sender<PrefetchEvent>) {
|
||||||
|
loop {
|
||||||
|
let (generation, conn, jobs) = {
|
||||||
|
let mut wanted = shared.wanted.lock().unwrap_or_else(|e| e.into_inner());
|
||||||
|
while wanted.jobs.is_empty() {
|
||||||
|
wanted = shared
|
||||||
|
.changed
|
||||||
|
.wait(wanted)
|
||||||
|
.unwrap_or_else(|e| e.into_inner());
|
||||||
}
|
}
|
||||||
|
let jobs = std::mem::take(&mut wanted.jobs);
|
||||||
|
let Some(conn) = wanted.conn.clone() else {
|
||||||
|
continue;
|
||||||
|
};
|
||||||
|
(wanted.generation, conn, jobs)
|
||||||
};
|
};
|
||||||
|
|
||||||
rt.block_on(async {
|
for job in jobs {
|
||||||
let backend = match crate::remote::connect(&conn) {
|
let current = shared
|
||||||
Ok(b) => b,
|
.wanted
|
||||||
Err(e) => {
|
.lock()
|
||||||
let _ = tx.send(Err(FetchFailure::local(e)));
|
.unwrap_or_else(|e| e.into_inner())
|
||||||
return;
|
.is_current(generation);
|
||||||
}
|
if !current {
|
||||||
};
|
break;
|
||||||
|
}
|
||||||
let id = RemoteId::Path(RemotePath::new(&path));
|
if holds_original(&job.cache) {
|
||||||
let got = backend.get(&id, None).await.map_err(FetchFailure::from);
|
continue;
|
||||||
|
|
||||||
// Store before sending, so the bytes are on disk by the time the
|
|
||||||
// image is on screen. Doing it after would leave a window where
|
|
||||||
// closing the app immediately lost the download.
|
|
||||||
if let (Ok(bytes), Some(c), Some((store, conn))) =
|
|
||||||
(&got, cache.as_ref().filter(|c| c.store), cached.as_ref())
|
|
||||||
{
|
|
||||||
// `pinned: false` — this is the passive population. A pin is
|
|
||||||
// something the user asks for explicitly; opening an image is
|
|
||||||
// not that, and treating it as one would make the pinned set
|
|
||||||
// grow silently and never be evicted.
|
|
||||||
if let Err(e) =
|
|
||||||
store.store(conn.connection(), c.image, &path, bytes, false, now_secs())
|
|
||||||
{
|
|
||||||
log::debug!("caching {path}: {e}");
|
|
||||||
} else if let Err(e) = store.enforce(conn.connection()) {
|
|
||||||
log::debug!("enforcing the cache budget: {e}");
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
let _ = tx.send(got);
|
// A closed channel means the window is gone: nothing to fetch
|
||||||
});
|
// for any more.
|
||||||
});
|
if events
|
||||||
|
.send(PrefetchEvent::Started(job.path.clone()))
|
||||||
|
.is_err()
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
match fetch_original(conn.clone(), &job.path, Some(&job.cache)) {
|
||||||
|
Ok(bytes) => log::info!("{}: {} bytes fetched ahead", job.path, bytes.len()),
|
||||||
|
// Not a failure anyone needs to hear about now: the click
|
||||||
|
// that wants this photograph will try again and say so.
|
||||||
|
Err(e) => log::debug!("fetching {} ahead: {}", job.path, e.message),
|
||||||
|
}
|
||||||
|
if events.send(PrefetchEvent::Ended(job.path)).is_err() {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
rx
|
/// Whether the cache already has this original — a row check, not a read,
|
||||||
|
/// so asking costs nothing and touches no `last_used`.
|
||||||
|
fn holds_original(cache: &CacheContext) -> bool {
|
||||||
|
let Ok(store) = dr_catalog::Cache::open(&cache.dir, cache.budget) else {
|
||||||
|
return false;
|
||||||
|
};
|
||||||
|
let Ok(catalog) = Catalog::open(&cache.catalog_path) else {
|
||||||
|
return false;
|
||||||
|
};
|
||||||
|
store.holds_original(catalog.connection(), cache.image)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Serve thumbnails for a set of rows: store first, network second.
|
/// Serve thumbnails for a set of rows: store first, network second.
|
||||||
@@ -7864,3 +8120,79 @@ mod amending_across_devices {
|
|||||||
assert_eq!(out.versions["for-print"].rating, 5);
|
assert_eq!(out.versions["for-print"].rating, 5);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod fetching_ahead {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
/// The whole reason the registry exists: a click on a photograph that is
|
||||||
|
/// being fetched ahead waits for that transfer rather than starting its
|
||||||
|
/// own, and is told so — `None` — so it looks in the cache again.
|
||||||
|
#[test]
|
||||||
|
fn a_second_claim_waits_for_the_first_to_be_released() {
|
||||||
|
let registry = std::sync::Arc::new(InFlight::default());
|
||||||
|
let first = registry.claim("shoot/one.CR2");
|
||||||
|
assert!(first.is_some(), "an unclaimed path is claimed outright");
|
||||||
|
|
||||||
|
let (tx, rx) = std::sync::mpsc::channel();
|
||||||
|
let waiter = {
|
||||||
|
let registry = registry.clone();
|
||||||
|
std::thread::spawn(move || {
|
||||||
|
tx.send(()).unwrap();
|
||||||
|
registry.claim("shoot/one.CR2").is_some()
|
||||||
|
})
|
||||||
|
};
|
||||||
|
rx.recv().unwrap();
|
||||||
|
// The waiter is blocked on the first claim. Not provable without a
|
||||||
|
// sleep, but a release that reaches it proves the wait ended there.
|
||||||
|
assert!(
|
||||||
|
!waiter.is_finished(),
|
||||||
|
"the second claim must not return while the first is held"
|
||||||
|
);
|
||||||
|
|
||||||
|
drop(first);
|
||||||
|
let claimed = waiter.join().unwrap();
|
||||||
|
assert!(
|
||||||
|
!claimed,
|
||||||
|
"after waiting, the caller is told to recheck the cache"
|
||||||
|
);
|
||||||
|
|
||||||
|
assert!(
|
||||||
|
registry.claim("shoot/one.CR2").is_some(),
|
||||||
|
"and once nobody holds the path it can be claimed again"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Different photographs never wait on each other.
|
||||||
|
#[test]
|
||||||
|
fn distinct_paths_are_claimed_independently() {
|
||||||
|
let registry = InFlight::default();
|
||||||
|
let _a = registry.claim("a.CR2");
|
||||||
|
assert!(registry.claim("b.CR2").is_some());
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A newer wish supersedes an older one mid-list; the worker checks this
|
||||||
|
/// between jobs, and it is what keeps a fast walk along the roll from
|
||||||
|
/// queueing every neighbour it passed.
|
||||||
|
#[test]
|
||||||
|
fn a_new_wish_supersedes_the_one_being_served() {
|
||||||
|
let mut wanted = Wanted::default();
|
||||||
|
let conn = test_connection();
|
||||||
|
wanted.replace(conn.clone(), Vec::new());
|
||||||
|
let taken = wanted.generation;
|
||||||
|
assert!(wanted.is_current(taken));
|
||||||
|
|
||||||
|
wanted.replace(conn, Vec::new());
|
||||||
|
assert!(
|
||||||
|
!wanted.is_current(taken),
|
||||||
|
"the list taken before the replacement is stale"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn test_connection() -> Connection {
|
||||||
|
Connection::new(
|
||||||
|
Account::new("nextcloud", "https://cloud.example").with_login("d", "d"),
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -384,6 +384,11 @@ pub struct LibraryController {
|
|||||||
/// small budget still keeps a working set. A metered or small-disk device
|
/// small budget still keeps a working set. A metered or small-disk device
|
||||||
/// wants the first.
|
/// wants the first.
|
||||||
keep_opened: std::cell::Cell<bool>,
|
keep_opened: std::cell::Cell<bool>,
|
||||||
|
/// TRACES: FR-NC-6a | FR-UI-4
|
||||||
|
/// How many photographs each side of the open one are fetched ahead.
|
||||||
|
/// Mirrors `CacheSettings::fetch_ahead`, held beside the flag that
|
||||||
|
/// gates it.
|
||||||
|
fetch_ahead: std::cell::Cell<u32>,
|
||||||
/// TRACES: FR-CAT-13 | NFR-R4
|
/// TRACES: FR-CAT-13 | NFR-R4
|
||||||
/// Whether judgements and keywords also go to the `.xmp` beside the
|
/// Whether judgements and keywords also go to the `.xmp` beside the
|
||||||
/// original. Mirrors `LibrarySettings::write_xmp_sidecars`.
|
/// original. Mirrors `LibrarySettings::write_xmp_sidecars`.
|
||||||
@@ -474,6 +479,7 @@ impl LibraryController {
|
|||||||
keep_opened: std::cell::Cell::new(
|
keep_opened: std::cell::Cell::new(
|
||||||
dr_types::CacheSettings::default().keep_opened_originals,
|
dr_types::CacheSettings::default().keep_opened_originals,
|
||||||
),
|
),
|
||||||
|
fetch_ahead: std::cell::Cell::new(dr_types::CacheSettings::default().fetch_ahead),
|
||||||
write_xmp: std::cell::Cell::new(
|
write_xmp: std::cell::Cell::new(
|
||||||
dr_types::LibrarySettings::default().write_xmp_sidecars,
|
dr_types::LibrarySettings::default().write_xmp_sidecars,
|
||||||
),
|
),
|
||||||
@@ -516,6 +522,16 @@ impl LibraryController {
|
|||||||
self.keep_opened.set(keep);
|
self.keep_opened.set(keep);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// TRACES: FR-NC-6a | FR-UI-4
|
||||||
|
/// How far along the roll, each side, the next open fetches ahead.
|
||||||
|
pub fn set_fetch_ahead(&self, depth: u32) {
|
||||||
|
self.fetch_ahead.set(depth);
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn fetch_ahead(&self) -> u32 {
|
||||||
|
self.fetch_ahead.get()
|
||||||
|
}
|
||||||
|
|
||||||
/// TRACES: FR-CAT-13 | NFR-R4
|
/// TRACES: FR-CAT-13 | NFR-R4
|
||||||
pub fn set_write_xmp_sidecars(&self, on: bool) {
|
pub fn set_write_xmp_sidecars(&self, on: bool) {
|
||||||
self.write_xmp.set(on);
|
self.write_xmp.set(on);
|
||||||
@@ -583,6 +599,32 @@ impl LibraryController {
|
|||||||
.map(|id| dr_types::ImageId(*id as u64))
|
.map(|id| dr_types::ImageId(*id as u64))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// TRACES: FR-NC-6a | FR-UI-4
|
||||||
|
/// The photographs within `depth` of `path` on the roll, closest first
|
||||||
|
/// and working outwards: next, previous, next-but-one, previous-but-one…
|
||||||
|
///
|
||||||
|
/// That order is the point. The prefetcher serves the list one at a time
|
||||||
|
/// and drops the rest the moment the user moves, so whatever depth is
|
||||||
|
/// set, the frame most likely to be stepped to is the first thing
|
||||||
|
/// fetched — and the step most likely is one forward, which is why next
|
||||||
|
/// leads previous at every distance.
|
||||||
|
///
|
||||||
|
/// In the roll's order, which is the loaded window's — the same rows a
|
||||||
|
/// roll pick names — so what gets fetched ahead is exactly what a step
|
||||||
|
/// lands on, filter and scope included. The window's edges cut the list
|
||||||
|
/// short; a photograph not in the window has no neighbours, and a
|
||||||
|
/// prefetch of nothing is the right answer for it.
|
||||||
|
pub fn neighbours_of(&self, path: &str, depth: u32) -> Vec<String> {
|
||||||
|
let paths = self.paths.borrow();
|
||||||
|
let Some(at) = paths.iter().position(|p| p == path) else {
|
||||||
|
return Vec::new();
|
||||||
|
};
|
||||||
|
(1..=depth as usize)
|
||||||
|
.flat_map(|d| [at.checked_add(d), at.checked_sub(d)])
|
||||||
|
.filter_map(|i| paths.get(i?).cloned())
|
||||||
|
.collect()
|
||||||
|
}
|
||||||
|
|
||||||
/// TRACES: FR-NC-8 | FR-DEV-6
|
/// TRACES: FR-NC-8 | FR-DEV-6
|
||||||
/// The default version's uuid for an image, by its remote path.
|
/// The default version's uuid for an image, by its remote path.
|
||||||
///
|
///
|
||||||
@@ -7919,6 +7961,39 @@ mod tests {
|
|||||||
assert_eq!(seen, vec![0, 1, 2, 3]);
|
assert_eq!(seen, vec![0, 1, 2, 3]);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The prefetch order is the walking order: closest first, next before
|
||||||
|
/// previous at every distance, and the window's edges cut it short.
|
||||||
|
#[test]
|
||||||
|
fn neighbours_are_listed_closest_first_working_outwards() {
|
||||||
|
let ctl = LibraryController::new(crate::activity::ActivityLog::new());
|
||||||
|
*ctl.paths.borrow_mut() = ["a", "b", "c", "d", "e", "f"].map(String::from).to_vec();
|
||||||
|
|
||||||
|
assert_eq!(ctl.neighbours_of("c", 2), vec!["d", "b", "e", "a"]);
|
||||||
|
assert_eq!(
|
||||||
|
ctl.neighbours_of("c", 10),
|
||||||
|
vec!["d", "b", "e", "a", "f"],
|
||||||
|
"past an edge the other side keeps going"
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
ctl.neighbours_of("a", 1),
|
||||||
|
vec!["b"],
|
||||||
|
"nothing before the first"
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
ctl.neighbours_of("f", 1),
|
||||||
|
vec!["e"],
|
||||||
|
"nothing after the last"
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
ctl.neighbours_of("c", 0).is_empty(),
|
||||||
|
"zero fetches nothing ahead"
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
ctl.neighbours_of("z", 3).is_empty(),
|
||||||
|
"a photograph outside the window has no neighbours to fetch"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
/// A controller holding a catalog of four named images.
|
/// A controller holding a catalog of four named images.
|
||||||
fn with_catalog() -> Rc<LibraryController> {
|
fn with_catalog() -> Rc<LibraryController> {
|
||||||
let ctl = LibraryController::new(crate::activity::ActivityLog::new());
|
let ctl = LibraryController::new(crate::activity::ActivityLog::new());
|
||||||
|
|||||||
@@ -26,8 +26,8 @@ use std::rc::Rc;
|
|||||||
|
|
||||||
use dr_types::settings::budget;
|
use dr_types::settings::budget;
|
||||||
use dr_types::{
|
use dr_types::{
|
||||||
CollisionPolicy, ColourSpace, ExportFormat, ExportTarget, FaceDetector, GroupNavigation,
|
CacheSettings, CollisionPolicy, ColourSpace, ExportFormat, ExportTarget, FaceDetector,
|
||||||
LibrarySettings, OutputSharpening, ScreenSize, Settings, SizingMode,
|
GroupNavigation, LibrarySettings, OutputSharpening, ScreenSize, Settings, SizingMode,
|
||||||
};
|
};
|
||||||
use slint::ComponentHandle;
|
use slint::ComponentHandle;
|
||||||
|
|
||||||
@@ -154,6 +154,24 @@ pub fn render(window: &AppWindow, controller: &SettingsController) {
|
|||||||
window.set_settings_thumbnail_budget(budget::label(s.cache.thumbnail_budget_bytes).into());
|
window.set_settings_thumbnail_budget(budget::label(s.cache.thumbnail_budget_bytes).into());
|
||||||
window.set_settings_thumbnail_unlimited(s.cache.thumbnail_budget_bytes.is_none());
|
window.set_settings_thumbnail_unlimited(s.cache.thumbnail_budget_bytes.is_none());
|
||||||
window.set_settings_keep_opened(s.cache.keep_opened_originals);
|
window.set_settings_keep_opened(s.cache.keep_opened_originals);
|
||||||
|
// TRACES: FR-NC-6a | FR-UI-4
|
||||||
|
// Labelled from the record's own list, as the bar counts below are.
|
||||||
|
window.set_settings_fetch_ahead_labels(slint::ModelRc::new(slint::VecModel::from(
|
||||||
|
CacheSettings::AHEAD_CHOICES
|
||||||
|
.iter()
|
||||||
|
.map(|n| {
|
||||||
|
slint::SharedString::from(if *n == 0 {
|
||||||
|
"Off".to_string()
|
||||||
|
} else {
|
||||||
|
n.to_string()
|
||||||
|
})
|
||||||
|
})
|
||||||
|
.collect::<Vec<_>>(),
|
||||||
|
)));
|
||||||
|
window.set_settings_fetch_ahead_selected(index_of(
|
||||||
|
&CacheSettings::AHEAD_CHOICES,
|
||||||
|
&s.cache.fetch_ahead,
|
||||||
|
));
|
||||||
window.set_settings_write_xmp(s.library.write_xmp_sidecars);
|
window.set_settings_write_xmp(s.library.write_xmp_sidecars);
|
||||||
|
|
||||||
// --- develop -------------------------------------------------------
|
// --- develop -------------------------------------------------------
|
||||||
@@ -502,6 +520,22 @@ pub fn wire<F, G>(
|
|||||||
render(&w, &ctl);
|
render(&w, &ctl);
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
{
|
||||||
|
// TRACES: FR-NC-6a | FR-UI-4
|
||||||
|
// Notified like a budget change: the depth is read by the library
|
||||||
|
// controller at the next open, not by this page.
|
||||||
|
let weak = window.as_weak();
|
||||||
|
let ctl = controller.clone();
|
||||||
|
let notify = on_budget_changed.clone();
|
||||||
|
window.on_settings_fetch_ahead_picked(move |i| {
|
||||||
|
let Some(w) = weak.upgrade() else { return };
|
||||||
|
if let Some(n) = CacheSettings::AHEAD_CHOICES.get(i as usize).copied() {
|
||||||
|
ctl.edit(|s| s.cache.fetch_ahead = n);
|
||||||
|
}
|
||||||
|
notify(&ctl.snapshot());
|
||||||
|
render(&w, &ctl);
|
||||||
|
});
|
||||||
|
}
|
||||||
{
|
{
|
||||||
// TRACES: FR-CAT-13 | NFR-R4
|
// TRACES: FR-CAT-13 | NFR-R4
|
||||||
let weak = window.as_weak();
|
let weak = window.as_weak();
|
||||||
|
|||||||
@@ -835,6 +835,9 @@ export component AppWindow inherits Window {
|
|||||||
in property <string> settings-thumbnail-budget: "";
|
in property <string> settings-thumbnail-budget: "";
|
||||||
in property <bool> settings-thumbnail-unlimited: false;
|
in property <bool> settings-thumbnail-unlimited: false;
|
||||||
in property <bool> settings-keep-opened: true;
|
in property <bool> settings-keep-opened: true;
|
||||||
|
in property <[string]> settings-fetch-ahead-labels;
|
||||||
|
in property <int> settings-fetch-ahead-selected: 0;
|
||||||
|
callback settings-fetch-ahead-picked(int);
|
||||||
in property <string> settings-cache-usage: "";
|
in property <string> settings-cache-usage: "";
|
||||||
|
|
||||||
callback settings-original-budget-changed(string);
|
callback settings-original-budget-changed(string);
|
||||||
@@ -1280,6 +1283,9 @@ in property <bool> panel-visible: true;
|
|||||||
thumbnail-budget: root.settings-thumbnail-budget;
|
thumbnail-budget: root.settings-thumbnail-budget;
|
||||||
thumbnail-unlimited: root.settings-thumbnail-unlimited;
|
thumbnail-unlimited: root.settings-thumbnail-unlimited;
|
||||||
keep-opened: root.settings-keep-opened;
|
keep-opened: root.settings-keep-opened;
|
||||||
|
fetch-ahead-labels: root.settings-fetch-ahead-labels;
|
||||||
|
fetch-ahead-selected: root.settings-fetch-ahead-selected;
|
||||||
|
fetch-ahead-picked(i) => { root.settings-fetch-ahead-picked(i); }
|
||||||
copy-scope-kinds: root.copy-scope-kinds;
|
copy-scope-kinds: root.copy-scope-kinds;
|
||||||
copy-scope-empty: root.copy-scope-empty;
|
copy-scope-empty: root.copy-scope-empty;
|
||||||
copy-scope-toggled(name) => { root.copy-scope-toggled(name); }
|
copy-scope-toggled(name) => { root.copy-scope-toggled(name); }
|
||||||
|
|||||||
@@ -105,6 +105,11 @@ export component SettingsPage inherits Rectangle {
|
|||||||
in-out property <string> thumbnail-budget;
|
in-out property <string> thumbnail-budget;
|
||||||
in property <bool> thumbnail-unlimited: false;
|
in property <bool> thumbnail-unlimited: false;
|
||||||
in property <bool> keep-opened: true;
|
in property <bool> keep-opened: true;
|
||||||
|
/// TRACES: FR-NC-6a | FR-UI-4
|
||||||
|
/// How many photographs each side of the open one are fetched ahead.
|
||||||
|
in property <[string]> fetch-ahead-labels;
|
||||||
|
in property <int> fetch-ahead-selected: 0;
|
||||||
|
callback fetch-ahead-picked(int);
|
||||||
/// TRACES: FR-DEV-6
|
/// TRACES: FR-DEV-6
|
||||||
/// Which kinds of edit a copy or preset carries. Supersedes the boolean
|
/// Which kinds of edit a copy or preset carries. Supersedes the boolean
|
||||||
/// above, which named only the one kind anybody wanted to exclude.
|
/// above, which named only the one kind anybody wanted to exclude.
|
||||||
@@ -592,6 +597,20 @@ export component SettingsPage inherits Rectangle {
|
|||||||
checked: root.keep-opened;
|
checked: root.keep-opened;
|
||||||
toggled(on) => { root.keep-opened-toggled(on); }
|
toggled(on) => { root.keep-opened-toggled(on); }
|
||||||
}
|
}
|
||||||
|
|
||||||
|
Rectangle { height: Theme.gap-sm; }
|
||||||
|
|
||||||
|
// TRACES: FR-NC-6a | FR-UI-4
|
||||||
|
Segmented {
|
||||||
|
label: "Fetch ahead";
|
||||||
|
hint: "photographs each side of the open one, "
|
||||||
|
+ "downloaded while you look at it so the "
|
||||||
|
+ "next step is instant — closest first";
|
||||||
|
enabled: root.keep-opened;
|
||||||
|
options: root.fetch-ahead-labels;
|
||||||
|
selected: root.fetch-ahead-selected;
|
||||||
|
picked(i) => { root.fetch-ahead-picked(i); }
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user