Merge the batch export, and settle the seven-branch merge
Resolves the last of the parallel work. Two conflicts worth recording, because both were semantic rather than textual: `render_for_export` gained a colour space on master while the batch branch was rewriting the single-image export path around it. Kept both: the batch request supersedes the synchronous path, and the space still has to be chosen at render time because the conversion happens in the shader before the clip to 0..1. `render_open_frame` takes it as an argument rather than reaching for a controller it does not hold. The map-wait moved into `readback::await_mapping` on one branch while another was editing the constant it used, so `READBACK_POLL_LIMIT` survived the merge with no callers. Removed rather than left for clippy to find later. 1164 tests pass, clippy clean, fmt clean. Traceability 53.0% -> 54.3%. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5
+949
-2
@@ -27,13 +27,45 @@
|
||||
//! promise and never do. An export waiting to upload is a promise — the user
|
||||
//! was told the export succeeded — and sweeping it away to reclaim disk would
|
||||
//! destroy work that no longer exists anywhere else.
|
||||
//!
|
||||
//! # The batch (FR-EXP-7)
|
||||
//!
|
||||
//! The second half of this file is a worker that exports a whole selection.
|
||||
//! It runs on a thread of its own and reaches the interface through the same
|
||||
//! two mechanisms everything else here does: an `mpsc` channel drained by a
|
||||
//! Slint timer, and a row in [`crate::activity`].
|
||||
//!
|
||||
//! **Why the worker opens its own sessions.** A [`crate::DevelopSession`] owns a GPU
|
||||
//! `AdjustPass`, and the one the interface is holding is the one the canvas
|
||||
//! renders from — handing it to a worker would mean the develop view could not
|
||||
//! draw while a batch ran, which is the freeze this exists to remove. A
|
||||
//! `GpuContext` is an `Arc` pair over a device and queue and is cheap to
|
||||
//! clone, so the worker takes a clone and opens each photograph for itself.
|
||||
//! The cost is a `Demosaicer` and an `AdjustPass` per image rather than one
|
||||
//! for the run; against a full-resolution decode, render and encode it is
|
||||
//! small, and it keeps this file out of the pipeline that `develop` owns.
|
||||
//!
|
||||
//! **The open image is the exception.** Its edit lives in the interface's
|
||||
//! session and may not have reached a sidecar yet, so a worker that re-opened
|
||||
//! the file for itself would export the *saved* version rather than the one on
|
||||
//! screen. That one frame is therefore rendered by the caller and handed over
|
||||
//! as [`Source::Rendered`]; everything after the render still moves off the UI
|
||||
//! thread.
|
||||
|
||||
use std::collections::HashSet;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::rc::Rc;
|
||||
use std::sync::atomic::{AtomicBool, Ordering};
|
||||
use std::sync::mpsc::{Receiver, RecvTimeoutError, Sender};
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
|
||||
use dr_export::Encoded;
|
||||
use dr_export::{Encoded, NameContext};
|
||||
use dr_sync::{RemoteBackend, RemotePath};
|
||||
use dr_sync_nextcloud::{AppCredentials, NextcloudBackend};
|
||||
use dr_types::ExportTarget;
|
||||
use dr_types::{ExportSettings, ExportTarget};
|
||||
|
||||
use crate::AppWindow;
|
||||
|
||||
/// Where exports wait for a server that is not there yet.
|
||||
///
|
||||
@@ -371,6 +403,619 @@ pub fn spawn_upload(
|
||||
rx
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Batch export (FR-EXP-7)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// TRACES: NFR-ARCH-3
|
||||
/// The stop flag a running batch reads.
|
||||
///
|
||||
/// One bit rather than a channel. Cancellation is read thousands of times more
|
||||
/// often than it is written, and a message would only be seen if the worker
|
||||
/// happened to be listening at the moment the button was pressed rather than at
|
||||
/// the next point it is safe to stop.
|
||||
///
|
||||
/// `Relaxed` throughout: the flag guards no other data, so there is nothing for
|
||||
/// an acquire/release pair to publish. The only ordering that matters is that
|
||||
/// the write eventually becomes visible, which every ordering guarantees.
|
||||
#[derive(Clone, Default)]
|
||||
pub struct Cancel(Arc<AtomicBool>);
|
||||
|
||||
impl Cancel {
|
||||
pub fn cancel(&self) {
|
||||
self.0.store(true, Ordering::Relaxed);
|
||||
}
|
||||
|
||||
pub fn is_cancelled(&self) -> bool {
|
||||
self.0.load(Ordering::Relaxed)
|
||||
}
|
||||
}
|
||||
|
||||
/// How long the worker blocks on another worker before rereading the flag.
|
||||
///
|
||||
/// TRACES: NFR-ARCH-3
|
||||
/// This is the cancellation bound for everything the batch spends its time
|
||||
/// *waiting* on — a download of tens of megabytes would otherwise hold a
|
||||
/// cancelled batch open until the transfer finished. It is not the bound for
|
||||
/// the frame being rendered and encoded when the button is pressed: that has no
|
||||
/// interior stopping point, so the true worst case is one image, and on a 24 MP
|
||||
/// frame that is well past NFR-ARCH-3's 100 ms target. Closing that gap needs
|
||||
/// the render itself to become interruptible (NFR-ARCH-2's scheduler), not a
|
||||
/// finer poll here.
|
||||
const CANCEL_POLL: Duration = Duration::from_millis(100);
|
||||
|
||||
/// How often the drain looks at what the worker has said.
|
||||
const DRAIN_INTERVAL: Duration = Duration::from_millis(120);
|
||||
|
||||
/// Where one image's pixels come from.
|
||||
pub enum Source {
|
||||
/// A photograph in the library, fetched and rendered by the worker.
|
||||
///
|
||||
/// Carries its own cache context because that is assembled from the catalog
|
||||
/// and the session, and a worker thread can reach neither.
|
||||
Library {
|
||||
path: String,
|
||||
cache: Option<crate::library::CacheContext>,
|
||||
},
|
||||
/// A frame the caller has already rendered — the image open in develop.
|
||||
///
|
||||
/// See the module docs for why this one cannot be left to the worker.
|
||||
Rendered {
|
||||
stem: String,
|
||||
frame: dr_export::Frame,
|
||||
},
|
||||
}
|
||||
|
||||
impl Source {
|
||||
/// What to call this image in a progress row.
|
||||
fn describe(&self) -> String {
|
||||
match self {
|
||||
Source::Library { path, .. } => path.rsplit('/').next().unwrap_or(path).to_string(),
|
||||
Source::Rendered { stem, .. } => stem.clone(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Everything a batch needs, assembled on the UI thread.
|
||||
///
|
||||
/// Assembled there and not in the worker because most of it is only reachable
|
||||
/// from a `RefCell` the interface owns: the session, the settings record, the
|
||||
/// catalog. Passing the finished answers means the worker borrows nothing.
|
||||
pub struct BatchRequest {
|
||||
pub sources: Vec<Source>,
|
||||
/// Credentials for the account the library is open on. `None` where no
|
||||
/// library is open, which is fine for a [`Source::Rendered`] and fatal for
|
||||
/// anything that has to be fetched.
|
||||
pub creds: Option<(AppCredentials, String)>,
|
||||
pub settings: ExportSettings,
|
||||
pub outbox: PathBuf,
|
||||
pub sidecar_cache: PathBuf,
|
||||
pub offline: bool,
|
||||
/// `None` on a build with no adapter, where a library image cannot be
|
||||
/// rendered at all — reported per image rather than refused up front, so
|
||||
/// the reason lands in the same place every other failure does.
|
||||
pub gpu: Option<dr_gpu::GpuContext>,
|
||||
}
|
||||
|
||||
/// TRACES: NFR-ARCH-4
|
||||
/// Why one image of a batch produced no file.
|
||||
///
|
||||
/// One variant per stage. "Export failed" repeated forty times is not a report
|
||||
/// anybody can act on: a destination folder that cannot be written is one fix,
|
||||
/// a body rawler cannot decode is another, and a server that went away is a
|
||||
/// third — and only the first is worth stopping the batch to correct.
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum ItemError {
|
||||
#[error("could not be fetched: {0}")]
|
||||
Fetch(String),
|
||||
|
||||
#[error("could not be opened for editing: {0}")]
|
||||
Open(String),
|
||||
|
||||
#[error("could not be rendered: {0}")]
|
||||
Render(String),
|
||||
|
||||
/// The name was taken and the collision policy is Skip. Not really a
|
||||
/// failure — the user asked for exactly this — but it is still an image
|
||||
/// that produced no file, and a batch reporting "40 exported" when it wrote
|
||||
/// 31 would be lying.
|
||||
#[error("a file of that name is already there, and the collision setting is Skip")]
|
||||
NameTaken,
|
||||
|
||||
#[error(transparent)]
|
||||
Encode(#[from] dr_export::ExportError),
|
||||
|
||||
#[error("could not be written: {0}")]
|
||||
Place(String),
|
||||
}
|
||||
|
||||
/// What the worker says as it goes.
|
||||
#[derive(Debug)]
|
||||
pub enum BatchMessage {
|
||||
/// Work has begun on one image; `done` counts the ones behind it.
|
||||
Started {
|
||||
done: usize,
|
||||
total: usize,
|
||||
name: String,
|
||||
},
|
||||
/// One image is finished, for better or worse.
|
||||
Item {
|
||||
name: String,
|
||||
outcome: Result<Placed, ItemError>,
|
||||
},
|
||||
Finished {
|
||||
exported: usize,
|
||||
failed: usize,
|
||||
cancelled: bool,
|
||||
},
|
||||
}
|
||||
|
||||
/// Export a selection on a thread of its own.
|
||||
pub fn spawn_batch(request: BatchRequest, cancel: Cancel) -> Receiver<BatchMessage> {
|
||||
let (tx, rx) = std::sync::mpsc::channel();
|
||||
std::thread::spawn(move || run(request, &cancel, &tx));
|
||||
rx
|
||||
}
|
||||
|
||||
/// The batch itself, one image at a time.
|
||||
///
|
||||
/// Sequential rather than parallel, which FR-EXP-7's "uses all available cores"
|
||||
/// does not yet get. One reason and one excuse: the reason is that a
|
||||
/// full-resolution frame is tens of megabytes and four in flight is a
|
||||
/// straightforward way to exhaust a tablet; the excuse is that the GPU is
|
||||
/// shared with the interface, and the render is where the time goes.
|
||||
fn run(mut request: BatchRequest, cancel: &Cancel, tx: &Sender<BatchMessage>) {
|
||||
let sources = std::mem::take(&mut request.sources);
|
||||
let total = sources.len();
|
||||
|
||||
// Names this run has already written. See [`resolve_batch_name`] for why
|
||||
// the destination alone is not enough to keep two exports apart.
|
||||
let mut issued: HashSet<String> = HashSet::new();
|
||||
let (mut exported, mut failed) = (0usize, 0usize);
|
||||
|
||||
for (i, source) in sources.into_iter().enumerate() {
|
||||
if cancel.is_cancelled() {
|
||||
break;
|
||||
}
|
||||
|
||||
let name = source.describe();
|
||||
let _ = tx.send(BatchMessage::Started {
|
||||
done: i,
|
||||
total,
|
||||
name: name.clone(),
|
||||
});
|
||||
|
||||
// `None` is cancellation mid-image, which is not an outcome for this
|
||||
// photograph: nothing failed, the user simply stopped asking.
|
||||
let Some(outcome) = export_one(&request, source, i as u32 + 1, &mut issued, cancel) else {
|
||||
break;
|
||||
};
|
||||
|
||||
// Counted, reported, and then on to the next one. A failure here must
|
||||
// not end the run — the whole point of exporting three hundred frames
|
||||
// unattended is that the one unreadable file is a line in a report
|
||||
// rather than an evening lost (FR-EXP-7).
|
||||
match &outcome {
|
||||
Ok(_) => exported += 1,
|
||||
Err(_) => failed += 1,
|
||||
}
|
||||
let _ = tx.send(BatchMessage::Item { name, outcome });
|
||||
}
|
||||
|
||||
let _ = tx.send(BatchMessage::Finished {
|
||||
exported,
|
||||
failed,
|
||||
cancelled: cancel.is_cancelled(),
|
||||
});
|
||||
}
|
||||
|
||||
/// One image, start to finish. `None` where the run was cancelled part way.
|
||||
fn export_one(
|
||||
request: &BatchRequest,
|
||||
source: Source,
|
||||
sequence: u32,
|
||||
issued: &mut HashSet<String>,
|
||||
cancel: &Cancel,
|
||||
) -> Option<Result<Placed, ItemError>> {
|
||||
let (stem, date, frame) = match source {
|
||||
Source::Rendered { stem, frame } => (stem, String::new(), frame),
|
||||
Source::Library { path, cache } => {
|
||||
match render_from_library(request, &path, cache, cancel)? {
|
||||
Ok(rendered) => rendered,
|
||||
Err(e) => return Some(Err(e)),
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
Some(place_frame(request, &stem, &date, sequence, &frame, issued))
|
||||
}
|
||||
|
||||
/// Fetch a photograph, apply its stored edit, and render it at full size.
|
||||
fn render_from_library(
|
||||
request: &BatchRequest,
|
||||
path: &str,
|
||||
cache: Option<crate::library::CacheContext>,
|
||||
cancel: &Cancel,
|
||||
) -> Option<Result<(String, String, dr_export::Frame), ItemError>> {
|
||||
let Some((creds, user_id)) = request.creds.clone() else {
|
||||
return Some(Err(ItemError::Fetch("no library is open".into())));
|
||||
};
|
||||
let Some(gpu) = request.gpu.as_ref() else {
|
||||
return Some(Err(ItemError::Open(
|
||||
"this build found no GPU adapter".into(),
|
||||
)));
|
||||
};
|
||||
|
||||
// Both started before either is waited on, exactly as opening an image in
|
||||
// develop does: the sidecar is three orders of magnitude smaller than the
|
||||
// RAW, so it costs nothing to have in hand by the time there is a session
|
||||
// to apply it to.
|
||||
let sidecar_rx = crate::library::spawn_sidecar_fetch(
|
||||
creds.clone(),
|
||||
user_id.clone(),
|
||||
path.to_string(),
|
||||
request.sidecar_cache.clone(),
|
||||
request.offline,
|
||||
);
|
||||
let bytes_rx = crate::library::spawn_full_fetch(creds, user_id, path.to_string(), cache);
|
||||
|
||||
let bytes = match wait_for(&bytes_rx, cancel) {
|
||||
Waited::Got(Ok(bytes)) => bytes,
|
||||
Waited::Got(Err(e)) => return Some(Err(ItemError::Fetch(e.message))),
|
||||
Waited::Cancelled => return None,
|
||||
Waited::Silent => {
|
||||
return Some(Err(ItemError::Fetch(
|
||||
"the download ended without answering".into(),
|
||||
)))
|
||||
}
|
||||
};
|
||||
|
||||
let sidecar = match wait_for(&sidecar_rx, cancel) {
|
||||
Waited::Got(sidecar) => sidecar,
|
||||
Waited::Cancelled => return None,
|
||||
// An unedited photograph has no sidecar and a fetch that died looks
|
||||
// identical from here. Exporting at defaults is what opening it would
|
||||
// do, and refusing the image over a missing edit it may never have had
|
||||
// would fail the common case.
|
||||
Waited::Silent => None,
|
||||
};
|
||||
|
||||
let meta = dr_decode::metadata(&bytes).unwrap_or_default();
|
||||
let orientation = meta.orientation.unwrap_or_default();
|
||||
// `{date}` is the capture date, not today's: a template naming exports by
|
||||
// when the shutter fired is the reason the token exists.
|
||||
let date = meta
|
||||
.captured_at
|
||||
.map(crate::library_ui::format_date)
|
||||
.unwrap_or_default();
|
||||
|
||||
let mut session = match crate::open_session(gpu, &bytes, orientation) {
|
||||
Ok(session) => session,
|
||||
Err(e) => return Some(Err(ItemError::Open(e))),
|
||||
};
|
||||
|
||||
// TRACES: FR-CAT-8
|
||||
// The stored edit. FR-EXP-9 is about resolution; this is the other half of
|
||||
// exporting what the user actually has, and a batch that skipped it would
|
||||
// write out three hundred unedited frames without saying so.
|
||||
if let Some(version) = sidecar
|
||||
.as_ref()
|
||||
.and_then(dr_pipeline::Sidecar::default_version)
|
||||
{
|
||||
session.apply_version(version);
|
||||
}
|
||||
|
||||
// The last cheap place to stop. Everything past here is a full-resolution
|
||||
// render and an encode with no interior stopping point — see [`CANCEL_POLL`].
|
||||
if cancel.is_cancelled() {
|
||||
return None;
|
||||
}
|
||||
|
||||
let frame = match session.render_for_export(request.settings.colour_space) {
|
||||
Ok(frame) => frame,
|
||||
Err(e) => return Some(Err(ItemError::Render(e))),
|
||||
};
|
||||
|
||||
let stem = Path::new(path)
|
||||
.file_stem()
|
||||
.map(|s| s.to_string_lossy().into_owned())
|
||||
.unwrap_or_else(|| "export".into());
|
||||
|
||||
Some(Ok((stem, date, frame)))
|
||||
}
|
||||
|
||||
/// Name, encode and write one rendered frame.
|
||||
///
|
||||
/// The tail every source shares, whoever rendered it.
|
||||
fn place_frame(
|
||||
request: &BatchRequest,
|
||||
stem: &str,
|
||||
date: &str,
|
||||
sequence: u32,
|
||||
frame: &dr_export::Frame,
|
||||
issued: &mut HashSet<String>,
|
||||
) -> Result<Placed, ItemError> {
|
||||
// The size is resolved before the name because `{dimensions}` is one of the
|
||||
// tokens a template can carry.
|
||||
let (width, height) = dr_export::target_size(
|
||||
frame.width,
|
||||
frame.height,
|
||||
request.settings.sizing,
|
||||
request.settings.allow_upscaling,
|
||||
);
|
||||
|
||||
let ctx = NameContext {
|
||||
source_stem: stem,
|
||||
sequence,
|
||||
date,
|
||||
width,
|
||||
height,
|
||||
preset: "",
|
||||
};
|
||||
|
||||
let name = resolve_batch_name(&request.settings, &ctx, issued).ok_or(ItemError::NameTaken)?;
|
||||
let encoded = dr_export::export(frame, &request.settings, name)?;
|
||||
|
||||
place(
|
||||
&encoded,
|
||||
request.settings.target,
|
||||
&request.settings.destination,
|
||||
&request.outbox,
|
||||
)
|
||||
.map_err(ItemError::Place)
|
||||
}
|
||||
|
||||
/// The name this export takes, avoiding both what was in the destination and
|
||||
/// what this run has already written.
|
||||
///
|
||||
/// Those are two different collisions and they deserve two different answers.
|
||||
/// [`dr_types::CollisionPolicy`] is the user's answer to "a file of this name
|
||||
/// was already there", and Overwrite is a perfectly reasonable one. It is not
|
||||
/// an answer to "the frame I exported four seconds ago was also called this":
|
||||
/// two photographs of the same stem in different folders are ordinary in a
|
||||
/// library, and a batch that quietly handed back fewer files than images —
|
||||
/// having destroyed its own output — is not something anybody asked for. So a
|
||||
/// name this run issued is always stepped past, whatever the policy says about
|
||||
/// the folder.
|
||||
fn resolve_batch_name(
|
||||
settings: &ExportSettings,
|
||||
ctx: &NameContext<'_>,
|
||||
issued: &mut HashSet<String>,
|
||||
) -> Option<String> {
|
||||
let dir = PathBuf::from(&settings.destination);
|
||||
let taken = |name: &str| -> bool {
|
||||
match settings.target {
|
||||
dr_types::ExportTarget::Device => dir.join(name).exists(),
|
||||
// A queued export cannot see the server, and may never be able to.
|
||||
// Names are kept apart in the outbox instead — see [`stage`].
|
||||
dr_types::ExportTarget::Remote => false,
|
||||
}
|
||||
};
|
||||
|
||||
let name = dr_export::resolve_name(
|
||||
&settings.filename_template,
|
||||
ctx,
|
||||
settings.format,
|
||||
settings.collision,
|
||||
&|name| taken(name) || issued.contains(name),
|
||||
)?;
|
||||
|
||||
// Only reachable under Overwrite, which hands back the taken name by
|
||||
// design. Skip and Increment have already been through the closure above.
|
||||
let name = if issued.contains(&name) {
|
||||
step_past(&name, &|candidate| {
|
||||
taken(candidate) || issued.contains(candidate)
|
||||
})?
|
||||
} else {
|
||||
name
|
||||
};
|
||||
|
||||
issued.insert(name.clone());
|
||||
Some(name)
|
||||
}
|
||||
|
||||
/// `photo.jpg` → `photo-1.jpg`, and on until the name is free.
|
||||
fn step_past(name: &str, taken: &dyn Fn(&str) -> bool) -> Option<String> {
|
||||
let path = Path::new(name);
|
||||
let stem = path
|
||||
.file_stem()
|
||||
.map(|s| s.to_string_lossy().into_owned())
|
||||
.unwrap_or_else(|| "export".into());
|
||||
let ext = path
|
||||
.extension()
|
||||
.map(|s| s.to_string_lossy().into_owned())
|
||||
.unwrap_or_default();
|
||||
|
||||
// Bounded for the same reason `resolve_name`'s own search is: a destination
|
||||
// that reports every name as taken has to fail rather than spin.
|
||||
(1..10_000)
|
||||
.map(|n| {
|
||||
if ext.is_empty() {
|
||||
format!("{stem}-{n}")
|
||||
} else {
|
||||
format!("{stem}-{n}.{ext}")
|
||||
}
|
||||
})
|
||||
.find(|candidate| !taken(candidate))
|
||||
}
|
||||
|
||||
/// What waiting on another worker produced.
|
||||
///
|
||||
/// Three answers rather than an `Option`, because "the user cancelled" and "the
|
||||
/// worker died without a word" have the same shape and want opposite treatment:
|
||||
/// one ends the batch quietly, the other is a failure belonging to one image.
|
||||
enum Waited<T> {
|
||||
Got(T),
|
||||
Cancelled,
|
||||
Silent,
|
||||
}
|
||||
|
||||
/// Block on another worker's answer without going deaf to cancellation.
|
||||
fn wait_for<T>(rx: &Receiver<T>, cancel: &Cancel) -> Waited<T> {
|
||||
loop {
|
||||
match rx.recv_timeout(CANCEL_POLL) {
|
||||
Ok(value) => return Waited::Got(value),
|
||||
Err(RecvTimeoutError::Timeout) => {
|
||||
if cancel.is_cancelled() {
|
||||
return Waited::Cancelled;
|
||||
}
|
||||
}
|
||||
Err(RecvTimeoutError::Disconnected) => return Waited::Silent,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Which status line a batch writes its commentary to.
|
||||
///
|
||||
/// One drain serves both entry points rather than two near-identical timers, so
|
||||
/// it has to be told where the words go: develop has its own line beside the
|
||||
/// export button, and the grid has the header's.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum Reporting {
|
||||
Develop,
|
||||
Library,
|
||||
}
|
||||
|
||||
/// TRACES: FR-EXP-7 | NFR-P9
|
||||
/// Drain a batch's progress on the UI thread.
|
||||
///
|
||||
/// `slot` holds the timer, and dropping what was there before is what stops a
|
||||
/// superseded batch's drain — and, through [`crate::activity::Activity`]'s
|
||||
/// `Drop`, takes its row with it.
|
||||
pub fn drain_batch(
|
||||
weak: slint::Weak<AppWindow>,
|
||||
activity: &Rc<crate::activity::ActivityLog>,
|
||||
slot: &Rc<std::cell::RefCell<Option<slint::Timer>>>,
|
||||
rx: Receiver<BatchMessage>,
|
||||
total: usize,
|
||||
reporting: Reporting,
|
||||
) {
|
||||
let job = activity.begin(
|
||||
crate::activity::Kind::Export,
|
||||
if total == 1 {
|
||||
"Exporting".to_string()
|
||||
} else {
|
||||
format!("Exporting {total} images")
|
||||
},
|
||||
);
|
||||
job.total(total);
|
||||
|
||||
// The first failure, kept for the summary. Only the first: a run where
|
||||
// every image failed for the same reason should say that reason once, and
|
||||
// the rest are in the log.
|
||||
let mut first_failure: Option<String> = None;
|
||||
|
||||
let timer = slint::Timer::default();
|
||||
let held = slot.clone();
|
||||
|
||||
timer.start(slint::TimerMode::Repeated, DRAIN_INTERVAL, move || {
|
||||
let Some(w) = weak.upgrade() else { return };
|
||||
|
||||
loop {
|
||||
let message = match rx.try_recv() {
|
||||
Ok(m) => m,
|
||||
Err(std::sync::mpsc::TryRecvError::Empty) => return,
|
||||
Err(std::sync::mpsc::TryRecvError::Disconnected) => {
|
||||
// A worker that died without a word must not leave the
|
||||
// button saying "Cancel export" for the rest of the session.
|
||||
job.fail("ended unexpectedly");
|
||||
settle(&w, reporting, "Export ended unexpectedly");
|
||||
stop_timer(&held);
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
match message {
|
||||
BatchMessage::Started { done, total, name } => {
|
||||
job.progress(done, total);
|
||||
job.detail(name.clone());
|
||||
report(
|
||||
&w,
|
||||
reporting,
|
||||
&format!("Exporting {name} ({}/{total})", done + 1),
|
||||
);
|
||||
}
|
||||
BatchMessage::Item { name, outcome } => match outcome {
|
||||
Ok(placed) => log::info!("{name}: {}", placed.describe()),
|
||||
Err(e) => {
|
||||
// Every failure is logged, not only the first: the
|
||||
// summary is a sentence and this is the record of which
|
||||
// photographs it is about (NFR-ARCH-4).
|
||||
log::warn!("exporting {name}: {e}");
|
||||
first_failure.get_or_insert_with(|| format!("{name}: {e}"));
|
||||
}
|
||||
},
|
||||
BatchMessage::Finished {
|
||||
exported,
|
||||
failed,
|
||||
cancelled,
|
||||
} => {
|
||||
let (text, is_failure) =
|
||||
summarise(exported, failed, cancelled, first_failure.as_deref());
|
||||
if is_failure {
|
||||
job.fail(text.clone());
|
||||
} else {
|
||||
job.finish(text.clone());
|
||||
}
|
||||
settle(&w, reporting, &text);
|
||||
stop_timer(&held);
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
*slot.borrow_mut() = Some(timer);
|
||||
}
|
||||
|
||||
/// How a finished batch reads, and whether it counts as a failure.
|
||||
///
|
||||
/// A cancelled run is never a failure however little it exported: the user
|
||||
/// stopped it, and a red row telling them so is the application arguing. A run
|
||||
/// with even one failure is, and stays in the list until it is cleared — that
|
||||
/// is the row somebody came looking for.
|
||||
fn summarise(
|
||||
exported: usize,
|
||||
failed: usize,
|
||||
cancelled: bool,
|
||||
first_failure: Option<&str>,
|
||||
) -> (String, bool) {
|
||||
if cancelled {
|
||||
return (format!("Cancelled after {exported}"), false);
|
||||
}
|
||||
if failed == 0 {
|
||||
return (format!("Exported {exported}"), false);
|
||||
}
|
||||
let detail = first_failure.unwrap_or("see the log");
|
||||
(
|
||||
format!("Exported {exported}, {failed} failed — {detail}"),
|
||||
true,
|
||||
)
|
||||
}
|
||||
|
||||
/// Say what the batch is doing on the line the caller came from.
|
||||
fn report(window: &AppWindow, reporting: Reporting, text: &str) {
|
||||
match reporting {
|
||||
Reporting::Develop => window.set_export_status(text.into()),
|
||||
Reporting::Library => window.set_library_status(text.into()),
|
||||
}
|
||||
}
|
||||
|
||||
/// Report, and put the buttons back.
|
||||
fn settle(window: &AppWindow, reporting: Reporting, text: &str) {
|
||||
report(window, reporting, text);
|
||||
match reporting {
|
||||
Reporting::Develop => window.set_export_busy(false),
|
||||
Reporting::Library => window.set_library_exporting(false),
|
||||
}
|
||||
}
|
||||
|
||||
fn stop_timer(slot: &Rc<std::cell::RefCell<Option<slint::Timer>>>) {
|
||||
if let Some(timer) = slot.borrow().as_ref() {
|
||||
timer.stop();
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
@@ -596,4 +1241,306 @@ mod tests {
|
||||
.describe()
|
||||
.contains("Exported"));
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// The batch (FR-EXP-7)
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
/// Settings that write PNGs into `dir`, so a test can look at what landed.
|
||||
fn to_folder(dir: &Path) -> ExportSettings {
|
||||
ExportSettings {
|
||||
format: dr_types::ExportFormat::Png,
|
||||
target: ExportTarget::Device,
|
||||
destination: dir.to_string_lossy().into_owned(),
|
||||
..Default::default()
|
||||
}
|
||||
}
|
||||
|
||||
fn request(settings: ExportSettings, sources: Vec<Source>) -> BatchRequest {
|
||||
BatchRequest {
|
||||
sources,
|
||||
creds: None,
|
||||
settings,
|
||||
outbox: std::env::temp_dir().join("dr-batch-test-outbox"),
|
||||
sidecar_cache: std::env::temp_dir().join("dr-batch-test-sidecars"),
|
||||
offline: true,
|
||||
gpu: None,
|
||||
}
|
||||
}
|
||||
|
||||
/// A frame of flat pixels — enough for the encoder, cheap for a test.
|
||||
fn frame(width: u32, height: u32) -> dr_export::Frame {
|
||||
dr_export::Frame::new(width, height, vec![128; (width * height * 4) as usize])
|
||||
.expect("well-formed")
|
||||
}
|
||||
|
||||
fn drive(request: BatchRequest, cancel: Cancel) -> Vec<BatchMessage> {
|
||||
let (tx, rx) = std::sync::mpsc::channel();
|
||||
run(request, &cancel, &tx);
|
||||
drop(tx);
|
||||
rx.into_iter().collect()
|
||||
}
|
||||
|
||||
fn finished(messages: &[BatchMessage]) -> (usize, usize, bool) {
|
||||
messages
|
||||
.iter()
|
||||
.find_map(|m| match m {
|
||||
BatchMessage::Finished {
|
||||
exported,
|
||||
failed,
|
||||
cancelled,
|
||||
} => Some((*exported, *failed, *cancelled)),
|
||||
_ => None,
|
||||
})
|
||||
.expect("a batch always says it has finished")
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_rendered_frame_reaches_the_destination_folder() {
|
||||
// The whole tail of the batch — name, size, sharpen, encode, write —
|
||||
// with no GPU and no network, which is what makes it testable at all.
|
||||
let dir = tmp();
|
||||
let out = dir.join("exports");
|
||||
let messages = drive(
|
||||
request(
|
||||
to_folder(&out),
|
||||
vec![Source::Rendered {
|
||||
stem: "IMG_0001".into(),
|
||||
frame: frame(16, 12),
|
||||
}],
|
||||
),
|
||||
Cancel::default(),
|
||||
);
|
||||
|
||||
assert_eq!(finished(&messages), (1, 0, false));
|
||||
assert!(out.join("IMG_0001.png").exists());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn two_images_of_one_name_both_survive_the_batch() {
|
||||
// Two folders in a library holding an IMG_0001 each is ordinary, and a
|
||||
// batch that wrote one file for two photographs would destroy work
|
||||
// without saying anything. Overwrite is set deliberately: it is the
|
||||
// user's answer about the *folder*, not about this run's own output.
|
||||
let dir = tmp();
|
||||
let out = dir.join("exports");
|
||||
let mut settings = to_folder(&out);
|
||||
settings.collision = dr_types::CollisionPolicy::Overwrite;
|
||||
|
||||
let messages = drive(
|
||||
request(
|
||||
settings,
|
||||
vec![
|
||||
Source::Rendered {
|
||||
stem: "IMG_0001".into(),
|
||||
frame: frame(16, 12),
|
||||
},
|
||||
Source::Rendered {
|
||||
stem: "IMG_0001".into(),
|
||||
frame: frame(16, 12),
|
||||
},
|
||||
],
|
||||
),
|
||||
Cancel::default(),
|
||||
);
|
||||
|
||||
assert_eq!(finished(&messages), (2, 0, false));
|
||||
assert!(out.join("IMG_0001.png").exists());
|
||||
assert!(out.join("IMG_0001-1.png").exists());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_name_that_was_already_there_still_obeys_the_collision_setting() {
|
||||
// The other half of the rule above: a file that existed *before* the
|
||||
// batch is exactly what the policy is about, and Overwrite must still
|
||||
// mean overwrite or the setting would do nothing.
|
||||
let dir = tmp();
|
||||
let out = dir.join("exports");
|
||||
std::fs::create_dir_all(&out).expect("temp dir");
|
||||
std::fs::write(out.join("IMG_0001.png"), b"older").expect("seed");
|
||||
|
||||
let mut settings = to_folder(&out);
|
||||
settings.collision = dr_types::CollisionPolicy::Overwrite;
|
||||
|
||||
let mut issued = HashSet::new();
|
||||
let name = resolve_batch_name(
|
||||
&settings,
|
||||
&NameContext {
|
||||
source_stem: "IMG_0001",
|
||||
sequence: 1,
|
||||
..Default::default()
|
||||
},
|
||||
&mut issued,
|
||||
);
|
||||
assert_eq!(name.as_deref(), Some("IMG_0001.png"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_skipped_name_is_reported_rather_than_silently_dropped() {
|
||||
// Skip is a legitimate answer, but the image still produced no file —
|
||||
// and a batch claiming to have exported it would be lying.
|
||||
let dir = tmp();
|
||||
let out = dir.join("exports");
|
||||
std::fs::create_dir_all(&out).expect("temp dir");
|
||||
std::fs::write(out.join("IMG_0001.png"), b"older").expect("seed");
|
||||
|
||||
let mut settings = to_folder(&out);
|
||||
settings.collision = dr_types::CollisionPolicy::Skip;
|
||||
|
||||
let messages = drive(
|
||||
request(
|
||||
settings,
|
||||
vec![Source::Rendered {
|
||||
stem: "IMG_0001".into(),
|
||||
frame: frame(8, 8),
|
||||
}],
|
||||
),
|
||||
Cancel::default(),
|
||||
);
|
||||
|
||||
assert_eq!(finished(&messages), (0, 1, false));
|
||||
assert!(matches!(
|
||||
messages.iter().find_map(|m| match m {
|
||||
BatchMessage::Item { outcome, .. } => Some(outcome),
|
||||
_ => None,
|
||||
}),
|
||||
Some(Err(ItemError::NameTaken))
|
||||
));
|
||||
assert_eq!(std::fs::read(out.join("IMG_0001.png")).unwrap(), b"older");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn one_failure_does_not_abandon_the_rest_of_the_batch() {
|
||||
// FR-EXP-7's central promise. The first source cannot be fetched — no
|
||||
// library is open — and the two after it must still be attempted and
|
||||
// still be counted.
|
||||
let dir = tmp();
|
||||
let out = dir.join("exports");
|
||||
let messages = drive(
|
||||
request(
|
||||
to_folder(&out),
|
||||
vec![
|
||||
Source::Library {
|
||||
path: "Photos/broken.CR2".into(),
|
||||
cache: None,
|
||||
},
|
||||
Source::Rendered {
|
||||
stem: "good-a".into(),
|
||||
frame: frame(8, 8),
|
||||
},
|
||||
Source::Rendered {
|
||||
stem: "good-b".into(),
|
||||
frame: frame(8, 8),
|
||||
},
|
||||
],
|
||||
),
|
||||
Cancel::default(),
|
||||
);
|
||||
|
||||
assert_eq!(finished(&messages), (2, 1, false));
|
||||
assert!(out.join("good-a.png").exists());
|
||||
assert!(out.join("good-b.png").exists());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn every_image_gets_its_own_outcome() {
|
||||
// The report is per image, not one verdict for the run: two failures
|
||||
// for two different reasons have to arrive as two messages, or a
|
||||
// three-hundred-frame batch is unreportable (NFR-ARCH-4).
|
||||
let dir = tmp();
|
||||
let messages = drive(
|
||||
request(
|
||||
to_folder(&dir.join("exports")),
|
||||
vec![
|
||||
Source::Library {
|
||||
path: "Photos/a.CR2".into(),
|
||||
cache: None,
|
||||
},
|
||||
Source::Rendered {
|
||||
stem: "b".into(),
|
||||
frame: frame(8, 8),
|
||||
},
|
||||
],
|
||||
),
|
||||
Cancel::default(),
|
||||
);
|
||||
|
||||
let outcomes: Vec<&Result<Placed, ItemError>> = messages
|
||||
.iter()
|
||||
.filter_map(|m| match m {
|
||||
BatchMessage::Item { outcome, .. } => Some(outcome),
|
||||
_ => None,
|
||||
})
|
||||
.collect();
|
||||
assert_eq!(outcomes.len(), 2);
|
||||
assert!(outcomes[0].is_err());
|
||||
assert!(outcomes[1].is_ok());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_cancelled_batch_stops_and_writes_nothing_more() {
|
||||
// TRACES: NFR-ARCH-3
|
||||
// Cancelled before it began, which is the strongest form of the
|
||||
// property: not one file, and the run still reports itself finished
|
||||
// rather than leaving the interface waiting for a message.
|
||||
let dir = tmp();
|
||||
let out = dir.join("exports");
|
||||
let cancel = Cancel::default();
|
||||
cancel.cancel();
|
||||
|
||||
let messages = drive(
|
||||
request(
|
||||
to_folder(&out),
|
||||
vec![Source::Rendered {
|
||||
stem: "IMG_0001".into(),
|
||||
frame: frame(8, 8),
|
||||
}],
|
||||
),
|
||||
cancel,
|
||||
);
|
||||
|
||||
assert_eq!(finished(&messages), (0, 0, true));
|
||||
assert!(!out.join("IMG_0001.png").exists());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_cancelled_wait_gives_up_instead_of_blocking_for_ever() {
|
||||
// TRACES: NFR-ARCH-3
|
||||
// The bound on cancelling a batch that is waiting on a download. With
|
||||
// a plain `recv` this test would hang, which is precisely the bug.
|
||||
let (tx, rx) = std::sync::mpsc::channel::<u8>();
|
||||
let cancel = Cancel::default();
|
||||
cancel.cancel();
|
||||
|
||||
assert!(matches!(wait_for(&rx, &cancel), Waited::Cancelled));
|
||||
drop(tx);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_worker_that_dies_is_not_mistaken_for_a_cancellation() {
|
||||
// The two look identical from a channel and mean opposite things: one
|
||||
// ends the batch, the other fails one image and moves on.
|
||||
let (tx, rx) = std::sync::mpsc::channel::<u8>();
|
||||
drop(tx);
|
||||
assert!(matches!(wait_for(&rx, &Cancel::default()), Waited::Silent));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_cancelled_run_is_not_reported_as_a_failure() {
|
||||
// The user stopped it. A red row saying so is the application arguing
|
||||
// with something it was told to do.
|
||||
let (text, failed) = summarise(5, 0, true, None);
|
||||
assert!(!failed, "{text}");
|
||||
assert!(text.contains('5'), "{text}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_run_with_a_failure_keeps_its_row_and_names_one() {
|
||||
// This is the row somebody comes to the activity list to find, so it
|
||||
// has to survive being trimmed — and it has to say which photograph.
|
||||
let (text, failed) = summarise(38, 2, false, Some("IMG_0007.CR2: could not be rendered"));
|
||||
assert!(failed);
|
||||
assert!(text.contains("IMG_0007.CR2"), "{text}");
|
||||
assert!(text.contains("38"), "{text}");
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user