Provenance in the sidecar, a launch hook for the page, and where it stands
derived_from and merge are top-level sidecar fields (FR-MRG-6): one line per source in order, and how the composite was made. A build that predates them keeps the lines as unknown and writes them back. The job writes the sidecar beside the composite and stages it with its own record when the composite goes through the outbox. DARKROOM_START_MERGE=a.CR2,b.CR2 lands on the merge page at startup with the job running on local files, on the model of DARKROOM_START_IDENTITY, for looking at the page where synthetic clicks do not reach it. The fetch and the start are shared with the grid's button. panorama.md §11 records what exists, the fixture's figures, and the six things still open, auto-crop first.
This commit is contained in:
@@ -110,6 +110,21 @@ pub struct Sidecar {
|
||||
///
|
||||
/// Preserved so a older build round-trips a newer file without loss.
|
||||
unknown_blocks: Vec<String>,
|
||||
/// TRACES: FR-MRG-6
|
||||
/// Provenance: the sources a composite was merged from, in order, as
|
||||
/// the library names them. Empty for a photograph the camera took.
|
||||
///
|
||||
/// Top-level rather than per version because it is a fact about the
|
||||
/// file, not about an edit — every version of a panorama is a version
|
||||
/// of the same twelve frames. Written as one `derived_from = …` line per
|
||||
/// source before the first version block, where a build that predates
|
||||
/// the field keeps the lines as unknown and writes them back untouched.
|
||||
pub derived_from: Vec<String>,
|
||||
/// TRACES: FR-MRG-6
|
||||
/// How the composite was made: `panorama cylindrical 49.7mm 12 frames`.
|
||||
/// Free text for a panel; the parameters that matter to a re-merge are
|
||||
/// the sources and the projection, and both are legible in it.
|
||||
pub merge: Option<String>,
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-3f
|
||||
@@ -823,6 +838,13 @@ impl Sidecar {
|
||||
/// caller may compare content to decide whether an upload is needed.
|
||||
pub fn to_text(&self) -> String {
|
||||
let mut out = format!("drsc {FORMAT_VERSION}\n");
|
||||
// TRACES: FR-MRG-6
|
||||
for source in &self.derived_from {
|
||||
let _ = writeln!(out, "derived_from = {source}");
|
||||
}
|
||||
if let Some(merge) = &self.merge {
|
||||
let _ = writeln!(out, "merge = {merge}");
|
||||
}
|
||||
for block in &self.unknown_blocks {
|
||||
let _ = writeln!(out, "{block}");
|
||||
}
|
||||
@@ -1008,7 +1030,12 @@ impl Sidecar {
|
||||
}
|
||||
|
||||
let Some(version) = current.as_mut() else {
|
||||
sidecar.unknown_blocks.push(line.to_string());
|
||||
// TRACES: FR-MRG-6
|
||||
match key {
|
||||
"derived_from" => sidecar.derived_from.push(value.to_string()),
|
||||
"merge" => sidecar.merge = Some(value.to_string()),
|
||||
_ => sidecar.unknown_blocks.push(line.to_string()),
|
||||
}
|
||||
continue;
|
||||
};
|
||||
|
||||
@@ -2209,6 +2236,20 @@ mod tests {
|
||||
/// the claim is about *those bytes*: a sidecar generated by this build
|
||||
/// would agree with this build by construction, and would go on agreeing
|
||||
/// with it through a rename that broke every file on disk.
|
||||
#[test]
|
||||
fn provenance_round_trips_at_the_top_level() {
|
||||
// TRACES: FR-MRG-6
|
||||
let text = "drsc 1\nderived_from = 2025/_MG_8320.CR2\nderived_from = 2025/_MG_8321.CR2\n\
|
||||
merge = panorama cylindrical 49.7mm 2 frames\n\n[version u1]\nname = Default\n\
|
||||
revision = 1\nmodified = 0\n";
|
||||
let sidecar = Sidecar::parse(text).expect("parses");
|
||||
assert_eq!(sidecar.derived_from, vec!["2025/_MG_8320.CR2", "2025/_MG_8321.CR2"]);
|
||||
assert_eq!(sidecar.merge.as_deref(), Some("panorama cylindrical 49.7mm 2 frames"));
|
||||
let out = sidecar.to_text();
|
||||
assert!(out.starts_with("drsc 1\nderived_from = 2025/_MG_8320.CR2\n"));
|
||||
assert_eq!(Sidecar::parse(&out).expect("re-parses"), sidecar);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_sidecar_from_before_the_channel_curves_still_names_the_master() {
|
||||
let text = "drsc 1\n\n[version u1]\nname = Default\nrevision = 4\nmodified = 9\n\
|
||||
|
||||
@@ -298,3 +298,49 @@ carrying the first source's EXIF in a sub-IFD as `dr-export` already does.
|
||||
5. Gain, seams, multi-band.
|
||||
6. The container, the catalog entry, provenance, the history entry.
|
||||
7. Tablet: NFR-MRG-1's figure, and FR-MRG-9's ceiling.
|
||||
|
||||
## 11. Where it stands — 2026-09-19, end of the first day
|
||||
|
||||
Built, on branch `merge/panorama`, in the order §10 gave:
|
||||
|
||||
| Piece | Where | State |
|
||||
|---|---|---|
|
||||
| Geometry: keypoints, matching, homography, focal, bundle adjustment, projections | `core/dr-pano` | Done; 33 tests without a model; the fixture aligns in 4.5 s |
|
||||
| XFeat at two shapes under tract | `models/keypoints`, `dr_pano::xfeat` | Done; 300 ms/frame desktop, 400 ms tablet |
|
||||
| The camera-space tap | `OutputMode::CameraLinear`, `AdjustPass::render_camera_linear` | Done, `rgba32float`, tiles by view rect |
|
||||
| Linear DNG writer, streamed | `dr_export::write_linear_dng` | Done; rawler reads it back |
|
||||
| A three-sample `RawImage` re-entering the pipeline | `dr-decode`, `DemosaicedImage::from_linear_rgb16` | Done |
|
||||
| Warp, accumulate, resolve, chunk by chunk | `dr_gpu::MergePass`, `merge.wgsl` | Done; feathered blend, scalar gain |
|
||||
| The job: load, proxies, align, gains, confirm, merge, provenance | `dr_ui::merge` | Done; `examples/merge.rs` drives it headless |
|
||||
| The page: table, preview, projection, Merge/Stop/Back; the grid's button | `merge.slint`, `merge_ui.rs` | Done; `DARKROOM_START_MERGE=a.CR2,b.CR2` lands on it |
|
||||
| Placement beside the sources through the outbox, rescan | `merge_ui.rs` | Done, untested against a server |
|
||||
|
||||
**Measured on the fixture (desktop, 12 × 20 MP, Intel adapter):** proxies
|
||||
and keypoints 4 s, alignment 4.5–12.6 s (load-sensitive: the matcher is
|
||||
every core), the merge **26 s for a 22 993 × 5 980 composite** in twelve
|
||||
bands of 2048 × 512 chunks, 45 s all told, an 825 MB DNG. NFR-MRG-1's 60 s
|
||||
holds on the desktop with room; the tablet's figure is still S15.4's open
|
||||
half.
|
||||
|
||||
**Open, in the order they matter:**
|
||||
|
||||
1. **Auto-crop (FR-MRG-4).** The merge returns a coverage mask per band and
|
||||
the file carries the black border. The largest inscribed rectangle over
|
||||
the coverage, then the DNG's `DefaultCropOrigin`/`DefaultCropSize`, so
|
||||
nothing is thrown away and the develop view opens on the picture.
|
||||
2. **Seams and the pyramid** (§10 step 5). The feather hides exposure and
|
||||
small misalignment; parallax on the near slope will show as a soft
|
||||
double edge at 1:1.
|
||||
3. **Vignetting in the tap.** The lens profile's distortion is applied
|
||||
before the fetch; its vignetting is an operation and is not. Frame edges
|
||||
are darker than their centres by the lens's falloff, and the feather
|
||||
averages them into the overlaps.
|
||||
4. **The tablet:** memory (twelve 40 MB sensor buffers on the CPU, one
|
||||
demosaiced frame at a time on the GPU), the figure, and FR-MRG-9's
|
||||
ceiling.
|
||||
5. **Horizon and drag-to-correct (FR-MRG-4, the proposed 4a).** The
|
||||
alignment failed on nothing in the fixture; the interaction waits for a
|
||||
set it fails on.
|
||||
6. **`derived_from` names sources by file name**, not content hash: the
|
||||
catalog's `content_hash` is null for most images most of the time. The
|
||||
hash can join it when the catalog has one.
|
||||
|
||||
+19
-19
File diff suppressed because one or more lines are too long
@@ -460,6 +460,38 @@ fn run_inner(
|
||||
return Err(e);
|
||||
}
|
||||
log::info!("merge: {}×{} written to {} in {:?}", out_w, out_h, out_path.display(), t.elapsed());
|
||||
|
||||
// TRACES: FR-MRG-6
|
||||
// Provenance, beside the composite: what it was merged from, in order,
|
||||
// and how. Its own file and its own outbox record, so the drain places
|
||||
// it as it places the composite; a sidecar that failed to write is
|
||||
// reported but does not un-write the composite.
|
||||
let mut sidecar = dr_pipeline::Sidecar::new();
|
||||
sidecar.derived_from = request.frames.iter().map(|f| f.name.clone()).collect();
|
||||
sidecar.merge = Some(format!(
|
||||
"panorama {} {:.1}mm {} frames",
|
||||
match projection {
|
||||
Projection::Perspective => "perspective",
|
||||
Projection::Cylindrical => "cylindrical",
|
||||
Projection::Spherical => "spherical",
|
||||
},
|
||||
focal_full * 36.0 / full_long,
|
||||
frames.len()
|
||||
));
|
||||
let sidecar_path = out_path.with_extension(dr_pipeline::sidecar::EXTENSION);
|
||||
if let Err(e) = std::fs::write(&sidecar_path, sidecar.to_text()) {
|
||||
log::warn!("merge: the sidecar could not be written: {e}");
|
||||
} else if let MergeDestination::Outbox { remote_dir, .. } = &request.destination {
|
||||
let record = crate::export::destination_record(&sidecar_path);
|
||||
let name = sidecar_path
|
||||
.file_name()
|
||||
.map(|n| n.to_string_lossy().into_owned())
|
||||
.unwrap_or_default();
|
||||
if let Err(e) = std::fs::write(&record, format!("{remote_dir}\n{name}\n")) {
|
||||
log::warn!("merge: the sidecar's record could not be written: {e}");
|
||||
}
|
||||
}
|
||||
|
||||
Ok(Some(MergeEvent::Done {
|
||||
path: out_path,
|
||||
staged,
|
||||
|
||||
+143
-87
@@ -79,6 +79,7 @@ pub fn wire<S, C, F>(
|
||||
F: Fn(&AppWindow) + 'static,
|
||||
{
|
||||
let on_done = Rc::new(on_done);
|
||||
let gpu_for_start = gpu.clone();
|
||||
|
||||
// --- start -----------------------------------------------------------
|
||||
{
|
||||
@@ -139,97 +140,84 @@ pub fn wire<S, C, F>(
|
||||
})
|
||||
.collect();
|
||||
|
||||
let (tx, rx) = std::sync::mpsc::channel();
|
||||
let (decide, decision) = std::sync::mpsc::channel();
|
||||
let cancel = Cancel::default();
|
||||
let activity = ctl.activity.begin(Kind::Download, "Panorama");
|
||||
activity.total(sources.len());
|
||||
|
||||
// The worker: fetch every original, then run the job. Fetching
|
||||
// here rather than in the job keeps the job free of the library
|
||||
// (it takes bytes), and sequential rather than all at once so
|
||||
// twelve downloads do not halve each other's bandwidth.
|
||||
{
|
||||
let conn = context.conn.clone();
|
||||
let cancel = cancel.clone();
|
||||
let tx = tx.clone();
|
||||
let names = names.clone();
|
||||
std::thread::spawn(move || {
|
||||
let mut frames = Vec::with_capacity(sources.len());
|
||||
for (i, (path, cache)) in sources.into_iter().enumerate() {
|
||||
if cancel.is_cancelled() {
|
||||
let _ = tx.send(MergeEvent::Cancelled);
|
||||
return;
|
||||
}
|
||||
let _ = tx.send(MergeEvent::Progress {
|
||||
stage: "Fetching",
|
||||
done: i,
|
||||
total: names.len(),
|
||||
});
|
||||
let rx = crate::library::spawn_full_fetch(conn.clone(), path.clone(), cache);
|
||||
let bytes = match rx.recv() {
|
||||
Ok(Ok(bytes)) => bytes,
|
||||
Ok(Err(e)) => {
|
||||
let _ = tx.send(MergeEvent::Failed(format!("{}: {}", names[i], e.message)));
|
||||
return;
|
||||
}
|
||||
Err(_) => {
|
||||
let _ = tx.send(MergeEvent::Failed(format!(
|
||||
"{}: the download ended without answering",
|
||||
names[i]
|
||||
)));
|
||||
return;
|
||||
}
|
||||
};
|
||||
frames.push(MergeInput {
|
||||
name: names[i].clone(),
|
||||
bytes: std::sync::Arc::new(bytes),
|
||||
});
|
||||
// The fetch, on the job's own thread before the job: every
|
||||
// original comes down in turn — sequential so twelve downloads
|
||||
// do not halve each other's bandwidth — and the job then takes
|
||||
// bytes, which keeps it free of the library.
|
||||
let conn = context.conn.clone();
|
||||
let fetch_names = names.clone();
|
||||
let fetch = move |tx: &Sender<MergeEvent>, cancel: &Cancel| -> Option<Vec<MergeInput>> {
|
||||
let mut frames = Vec::with_capacity(sources.len());
|
||||
for (i, (path, cache)) in sources.into_iter().enumerate() {
|
||||
if cancel.is_cancelled() {
|
||||
let _ = tx.send(MergeEvent::Cancelled);
|
||||
return None;
|
||||
}
|
||||
let request = MergeRequest::new(frames, destination);
|
||||
merge::run(gpu, request, tx, decision, cancel);
|
||||
});
|
||||
}
|
||||
|
||||
*ctl.job.borrow_mut() = Some(Job {
|
||||
rx,
|
||||
decide,
|
||||
cancel,
|
||||
names,
|
||||
activity,
|
||||
});
|
||||
*ctl.report.borrow_mut() = None;
|
||||
ctl.projection.set(0);
|
||||
|
||||
// The page, from a clean slate.
|
||||
w.set_merge_stage("Fetching".into());
|
||||
w.set_merge_running(true);
|
||||
w.set_merge_fraction(0.0);
|
||||
w.set_merge_status("".into());
|
||||
w.set_merge_error("".into());
|
||||
w.set_merge_aligned(false);
|
||||
w.set_merge_complete(false);
|
||||
w.set_merge_summary("".into());
|
||||
w.set_merge_frames(slint::ModelRc::new(slint::VecModel::from(Vec::<MergeFrameRow>::new())));
|
||||
w.set_merge_preview(slint::Image::default());
|
||||
w.set_merge_projection_selected(0);
|
||||
w.set_merge_done(false);
|
||||
w.set_show_merge(true);
|
||||
|
||||
let timer = slint::Timer::default();
|
||||
{
|
||||
let weak = w.as_weak();
|
||||
let ctl = ctl.clone();
|
||||
let on_done = on_done.clone();
|
||||
timer.start(slint::TimerMode::Repeated, DRAIN_INTERVAL, move || {
|
||||
let Some(w) = weak.upgrade() else { return };
|
||||
drain(&w, &ctl, &on_done);
|
||||
});
|
||||
}
|
||||
*ctl.timer.borrow_mut() = Some(timer);
|
||||
let _ = tx.send(MergeEvent::Progress {
|
||||
stage: "Fetching",
|
||||
done: i,
|
||||
total: fetch_names.len(),
|
||||
});
|
||||
let rx = crate::library::spawn_full_fetch(conn.clone(), path.clone(), cache);
|
||||
let bytes = match rx.recv() {
|
||||
Ok(Ok(bytes)) => bytes,
|
||||
Ok(Err(e)) => {
|
||||
let _ = tx.send(MergeEvent::Failed(format!("{}: {}", fetch_names[i], e.message)));
|
||||
return None;
|
||||
}
|
||||
Err(_) => {
|
||||
let _ = tx.send(MergeEvent::Failed(format!(
|
||||
"{}: the download ended without answering",
|
||||
fetch_names[i]
|
||||
)));
|
||||
return None;
|
||||
}
|
||||
};
|
||||
frames.push(MergeInput {
|
||||
name: fetch_names[i].clone(),
|
||||
bytes: std::sync::Arc::new(bytes),
|
||||
});
|
||||
}
|
||||
Some(frames)
|
||||
};
|
||||
start(&w, &ctl, gpu, names, destination, fetch, &on_done);
|
||||
});
|
||||
}
|
||||
|
||||
// A way to land on the page at startup with local files, for looking
|
||||
// at it without a mouse: `DARKROOM_START_MERGE=a.CR2,b.CR2`. The same
|
||||
// job, the same page; only the fetch is a read. Off unless set.
|
||||
if let Some(list) = std::env::var_os("DARKROOM_START_MERGE") {
|
||||
let paths: Vec<std::path::PathBuf> = list
|
||||
.to_string_lossy()
|
||||
.split(',')
|
||||
.filter(|p| !p.is_empty())
|
||||
.map(std::path::PathBuf::from)
|
||||
.collect();
|
||||
if let (Some(gpu), Some(first)) = (gpu_for_start.clone(), paths.first()) {
|
||||
let names: Vec<String> = paths
|
||||
.iter()
|
||||
.map(|p| p.file_name().map(|n| n.to_string_lossy().into_owned()).unwrap_or_default())
|
||||
.collect();
|
||||
let dir = first.parent().map(std::path::Path::to_path_buf).unwrap_or_default();
|
||||
let read_paths = paths.clone();
|
||||
let fetch = move |tx: &Sender<MergeEvent>, _: &Cancel| -> Option<Vec<MergeInput>> {
|
||||
let mut frames = Vec::new();
|
||||
for p in &read_paths {
|
||||
match MergeInput::read(p) {
|
||||
Ok(f) => frames.push(f),
|
||||
Err(e) => {
|
||||
let _ = tx.send(MergeEvent::Failed(e));
|
||||
return None;
|
||||
}
|
||||
}
|
||||
}
|
||||
Some(frames)
|
||||
};
|
||||
start(window, &ctl, gpu, names, MergeDestination::Local(dir), fetch, &on_done);
|
||||
}
|
||||
}
|
||||
|
||||
// --- the decision ----------------------------------------------------
|
||||
{
|
||||
let weak = window.as_weak();
|
||||
@@ -292,6 +280,74 @@ pub fn wire<S, C, F>(
|
||||
}
|
||||
}
|
||||
|
||||
/// Start a job: `fetch` runs first on the job's thread and hands back the
|
||||
/// frames (or reports why not and returns `None`); the job follows on the
|
||||
/// same thread. The page opens clean, and a timer drains the events.
|
||||
fn start<Fetch>(
|
||||
window: &AppWindow,
|
||||
ctl: &Rc<MergeController>,
|
||||
gpu: dr_gpu::GpuContext,
|
||||
names: Vec<String>,
|
||||
destination: MergeDestination,
|
||||
fetch: Fetch,
|
||||
on_done: &Rc<impl Fn(&AppWindow) + 'static>,
|
||||
) where
|
||||
Fetch: FnOnce(&Sender<MergeEvent>, &Cancel) -> Option<Vec<MergeInput>> + Send + 'static,
|
||||
{
|
||||
let (tx, rx) = std::sync::mpsc::channel();
|
||||
let (decide, decision) = std::sync::mpsc::channel();
|
||||
let cancel = Cancel::default();
|
||||
let activity = ctl.activity.begin(Kind::Download, "Panorama");
|
||||
activity.total(names.len());
|
||||
|
||||
{
|
||||
let cancel = cancel.clone();
|
||||
let tx = tx.clone();
|
||||
std::thread::spawn(move || {
|
||||
let Some(frames) = fetch(&tx, &cancel) else { return };
|
||||
let request = MergeRequest::new(frames, destination);
|
||||
merge::run(gpu, request, tx, decision, cancel);
|
||||
});
|
||||
}
|
||||
|
||||
*ctl.job.borrow_mut() = Some(Job {
|
||||
rx,
|
||||
decide,
|
||||
cancel,
|
||||
names,
|
||||
activity,
|
||||
});
|
||||
*ctl.report.borrow_mut() = None;
|
||||
ctl.projection.set(0);
|
||||
|
||||
// The page, from a clean slate.
|
||||
window.set_merge_stage("Fetching".into());
|
||||
window.set_merge_running(true);
|
||||
window.set_merge_fraction(0.0);
|
||||
window.set_merge_status("".into());
|
||||
window.set_merge_error("".into());
|
||||
window.set_merge_aligned(false);
|
||||
window.set_merge_complete(false);
|
||||
window.set_merge_summary("".into());
|
||||
window.set_merge_frames(slint::ModelRc::new(slint::VecModel::from(Vec::<MergeFrameRow>::new())));
|
||||
window.set_merge_preview(slint::Image::default());
|
||||
window.set_merge_projection_selected(0);
|
||||
window.set_merge_done(false);
|
||||
window.set_show_merge(true);
|
||||
|
||||
let timer = slint::Timer::default();
|
||||
{
|
||||
let weak = window.as_weak();
|
||||
let ctl = ctl.clone();
|
||||
let on_done = on_done.clone();
|
||||
timer.start(slint::TimerMode::Repeated, DRAIN_INTERVAL, move || {
|
||||
let Some(w) = weak.upgrade() else { return };
|
||||
drain(&w, &ctl, &on_done);
|
||||
});
|
||||
}
|
||||
*ctl.timer.borrow_mut() = Some(timer);
|
||||
}
|
||||
|
||||
/// Take everything the job has said and reflect it on the page.
|
||||
fn drain(window: &AppWindow, ctl: &Rc<MergeController>, on_done: &Rc<impl Fn(&AppWindow)>) {
|
||||
let events = {
|
||||
|
||||
Reference in New Issue
Block a user