Compare commits

...
11 Commits
Author SHA1 Message Date
dtourolle 5ae742816d Release 0.19.1
Benchmarks / CPU and I/O (per commit) (push) Successful in 5m29s
Benchmarks / Frame budget (on demand) (push) Skipped
Traceability / Requirement traces (push) Successful in 1m8s
Build and test / Android (aarch64) (push) Successful in 30m34s
Build and test / android-image (push) Successful in 3s
🐳 Android image / Build and push (push) Successful in 3s
Build and test / Desktop (Linux) (push) Successful in 48m59s
Build and test / windows-image (push) Successful in 2s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / Layer separation (push) Successful in 48s
Build and test / Windows (x86_64, cross) (push) Successful in 35m51s
Build and test / Publish the release (push) Successful in 51s
2026-09-28 19:58:04 -04:00
dtourolle 8d66fa9be5 Show the composite in the grid in the manual, and say how it gets there
The panorama scene now ends in the grid, on the composite in its wide
cell beside the twelve frames it was made from, with the thumbnail the
merge made: the picture is taken straight after the merge, before any
rescan could have found the file. The manual's library section says how
a panorama's cell is sized and packed, and its panorama section that the
composite is in the grid the moment it is written. panorama.md §9 records
the race that kept it out, the catalogue-then-register path that
replaced it, how the thumbnail is developed, and the layout's classes.
2026-09-28 19:57:37 -04:00
dtourolle 34630ff752 Name the folder lookup the merge page is handed, for clippy 2026-09-28 19:57:37 -04:00
dtourolle 3b7d7129ff Learn a photograph's shape from its header
A panorama the merge did not make, a stitch from another program or a
phone's sweep, never had a width or height in the catalog, so it could
never be given a wide cell. The header read that dates a photograph now
records its size as it is seen, orientation applied, alongside the date.
2026-09-28 19:57:37 -04:00
dtourolle 6050a8e703 Size a panorama's cell and thumbnail by class: two, three or four columns
One lookup, natural_span, maps a photograph's aspect to the columns its
cell spans, and the same number names its thumbnail class, Wide2, Wide3
or Wide4, 512 pixels of long edge per column, so a 4:1 panorama is as
sharp across four columns as a frame is in one. The boundaries are
where the two neighbouring cells would leave the same share of
themselves undrawn, sqrt(s(s+1)): 2.45 and 3.46, with the first at 1.9
so a 3:2 frame stays a frame. A grid too narrow for the class falls
back to the widest that fits, the tablet gives the whole row, and a
cell asks for the class it is actually drawn at: its span, but never
more than its own class. The merge renders each wide class up to the
composite's own, which covers every fallback.
2026-09-28 19:57:37 -04:00
dtourolle ae4e1a0f07 Give a panorama a wide cell in the grid
A 4:1 composite drawn in one square cell is a strip a few pixels high.
A photograph about twice as wide as it is tall (1.9 and up) now spans
two columns, three from 2.9, with a thumbnail class of its own whose
long edge is sized for that width; on the tablet, or where the columns
are too few to put it beside anything, it takes the whole row.

Rows are computed in one place, library_ui::layout. The grid is a
lattice of slots: each cell is drawn at the slot Rust gives it, and a
wide cell that would not fit in what is left of a row starts the next
one, leaving the gap empty so the grid still reads in capture order.
The scrollbar spans the slots, a scroll reports a slot that the layout
turns back into an ordinal, and scrubs, restores and the cursor go
through the same conversion. Up and down step by rows through the
layout rather than by a row's worth of ordinals; left and right, a
shift-click's run, burst folding and the timeline are ordinal-based
and unchanged.

The window's own read carries each photograph's w and h, so the cells
know their shape with no query per cell. Where the wide ones sit in
the whole list is one query, run when what the grid lists changes or
when a window finds the layout out of date, and a library with no
panorama answers it from a partial index created on first use
(images_wide), not a schema bump. The merge makes the wide thumbnail
for a wide composite along with the others.
2026-09-28 19:57:37 -04:00
dtourolle ce5b7d72e3 Thumbnail a composite during the merge, as develop first shows it
The DNG a merge writes has no embedded preview, and an embedded preview
is all the grid's thumbnail path reads, so a composite stood in the grid
as a blank cell until it was opened. Reading an 800 MB file back to make
one would cost what the merge already has in hand.

The bands are box-reduced as they are written, after the border fill,
into a copy 4096 pixels long. That copy is written as a linear DNG in
memory with the composite's own profile, header and crop, and opened
through open_session, the function develop opens every file with: the
same decode, the default graph and view transform, the as-shot white
balance and the conversion to the display's space. The grid and large
thumbnails are rendered from that session, staged beside the payload
before the rename releases it to a drain, and put in the store under the
file id once the upload has learned it. Until then the grid draws them
from memory, so the cell is not blank while the file is uploading.

A test develops a synthetic composite both ways, the whole file as
develop opens it and the merge's reduced copy, and holds the thumbnail's
mean, 95th and 99.5th luma percentiles to within 3-4 levels of develop's
render; the naive balanced-and-gamma picture the merge's preview draws
misses by 13.
2026-09-28 19:57:37 -04:00
dtourolle 98a67393d9 Catalogue a merged panorama the moment it is written
A finished merge drained the outbox and started a rescan beside it. The
scan raced the upload: on a folder library the 800 MB copy was still
running when the folder was listed, on Nextcloud the upload takes
minutes, and either way the listing lacked the composite, recorded the
folder's validator, and nothing looked again until the next sync pass.

The job now reports what the catalog needs (the name, the size of the
picture it opens on, the capture time it wrote into the DNG) and the
library writes the row at once, in one transaction, keyed where the scan
will list the file; a composite with no time of its own takes its
sources' earliest. The grid reloads and shows it beside its sources.

The upload then gives the row what only the server knows: after sending
a file the catalog already has a row for, it lists the folder once, takes
the file id the server assigned, and records it (and, once the merge
makes them, the thumbnails waiting beside the payload) under that id.
Every drain now rescans when something landed in the library, after the
upload rather than beside it. A second merge of the same frames is no
longer named over the first: the name is checked against the catalog's
names in that folder, which is all that knows it once the outbox is empty.
2026-09-28 19:57:37 -04:00
dtourolle 37136f7377 Let one outbox drain run at a time
A finished merge drained the outbox and the sync pass that followed
drained it again beside it: each read the 800 MB composite into memory
and sent it, and the later one found its record cleared underneath it
and logged the file as missing. Drains now take a lock and the one that
waited finds the queue empty.
2026-09-28 19:57:37 -04:00
dtourolle e2e2181469 Write the composite under a .part name until it is whole
The merge writes its outbox record before the DNG, which takes minutes,
and a drain that ran in the meantime took whatever lay at the record's
name: a sync pass that fired mid-merge uploaded the first part of the
composite and cleared the record. The file is now written as x.dng.part
and renamed into place once the last strip is in; the drain skips a
record whose payload does not exist yet.
2026-09-28 19:57:37 -04:00
dtourolle ad27369cdc Draft the design for learned denoise (FR-DEV-3g)
A design draft for the learned stage outstanding.md §3 lists as missing:
a joint demosaic-and-denoise network that runs on the mosaic, in the
slot architecture.md §5.2 reserves for it, with its result kept as a
cache rather than a new file in the library. It covers the model and
tiling, synthetic training pairs from the library and a per-body noise
calibration, evaluation on real pairs, the Amount control, speed on the
tablet, X-Trans, and the decisions still open. Nothing in it is built;
figures marked "estimate" wait for the measurements that replace them.
2026-09-28 07:31:35 -04:00
37 changed files with 3510 additions and 335 deletions
Generated
+25 -25
View File
@@ -1265,7 +1265,7 @@ checksum = "f27ae1dd37df86211c42e150270f82743308803d90a6f6e6651cd730d5e1732f"
[[package]] [[package]]
name = "darkroom-android" name = "darkroom-android"
version = "0.19.0" version = "0.19.1"
dependencies = [ dependencies = [
"android_logger", "android_logger",
"dr-plat", "dr-plat",
@@ -1278,7 +1278,7 @@ dependencies = [
[[package]] [[package]]
name = "darkroom-desktop" name = "darkroom-desktop"
version = "0.19.0" version = "0.19.1"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"dr-plat", "dr-plat",
@@ -1454,7 +1454,7 @@ checksum = "d8b14ccef22fc6f5a8f4d7d768562a182c04ce9a3b3157b91390b52ddfdf1a76"
[[package]] [[package]]
name = "dr-bench" name = "dr-bench"
version = "0.19.0" version = "0.19.1"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"dr-catalog", "dr-catalog",
@@ -1471,7 +1471,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-catalog" name = "dr-catalog"
version = "0.19.0" version = "0.19.1"
dependencies = [ dependencies = [
"dr-face", "dr-face",
"dr-plat", "dr-plat",
@@ -1486,7 +1486,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-decode" name = "dr-decode"
version = "0.19.0" version = "0.19.1"
dependencies = [ dependencies = [
"dr-types", "dr-types",
"env_logger", "env_logger",
@@ -1500,7 +1500,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-export" name = "dr-export"
version = "0.19.0" version = "0.19.1"
dependencies = [ dependencies = [
"dr-decode", "dr-decode",
"dr-gpu", "dr-gpu",
@@ -1519,7 +1519,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-face" name = "dr-face"
version = "0.19.0" version = "0.19.1"
dependencies = [ dependencies = [
"dr-inference-engine", "dr-inference-engine",
"env_logger", "env_logger",
@@ -1532,7 +1532,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-film" name = "dr-film"
version = "0.19.0" version = "0.19.1"
dependencies = [ dependencies = [
"log", "log",
"serde", "serde",
@@ -1541,7 +1541,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-gpu" name = "dr-gpu"
version = "0.19.0" version = "0.19.1"
dependencies = [ dependencies = [
"bytemuck", "bytemuck",
"dr-decode", "dr-decode",
@@ -1559,7 +1559,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-inference-engine" name = "dr-inference-engine"
version = "0.19.0" version = "0.19.1"
dependencies = [ dependencies = [
"env_logger", "env_logger",
"libloading", "libloading",
@@ -1574,7 +1574,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-ingest" name = "dr-ingest"
version = "0.19.0" version = "0.19.1"
dependencies = [ dependencies = [
"dr-plat", "dr-plat",
"dr-types", "dr-types",
@@ -1586,7 +1586,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-lens" name = "dr-lens"
version = "0.19.0" version = "0.19.1"
dependencies = [ dependencies = [
"lensfun", "lensfun",
"log", "log",
@@ -1594,7 +1594,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-pano" name = "dr-pano"
version = "0.19.0" version = "0.19.1"
dependencies = [ dependencies = [
"dr-decode", "dr-decode",
"dr-inference-engine", "dr-inference-engine",
@@ -1608,7 +1608,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-pipeline" name = "dr-pipeline"
version = "0.19.0" version = "0.19.1"
dependencies = [ dependencies = [
"dr-types", "dr-types",
"log", "log",
@@ -1617,7 +1617,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-plat" name = "dr-plat"
version = "0.19.0" version = "0.19.1"
dependencies = [ dependencies = [
"android-native-keyring-store", "android-native-keyring-store",
"dr-types", "dr-types",
@@ -1633,7 +1633,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-preset-xmp" name = "dr-preset-xmp"
version = "0.19.0" version = "0.19.1"
dependencies = [ dependencies = [
"dr-pipeline", "dr-pipeline",
"log", "log",
@@ -1643,7 +1643,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-segment" name = "dr-segment"
version = "0.19.0" version = "0.19.1"
dependencies = [ dependencies = [
"dr-inference-engine", "dr-inference-engine",
"env_logger", "env_logger",
@@ -1656,7 +1656,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-sync" name = "dr-sync"
version = "0.19.0" version = "0.19.1"
dependencies = [ dependencies = [
"async-trait", "async-trait",
"dr-plat", "dr-plat",
@@ -1670,7 +1670,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-sync-folder" name = "dr-sync-folder"
version = "0.19.0" version = "0.19.1"
dependencies = [ dependencies = [
"async-trait", "async-trait",
"dr-sync", "dr-sync",
@@ -1682,7 +1682,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-sync-nextcloud" name = "dr-sync-nextcloud"
version = "0.19.0" version = "0.19.1"
dependencies = [ dependencies = [
"async-trait", "async-trait",
"dr-decode", "dr-decode",
@@ -1704,7 +1704,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-thumbs" name = "dr-thumbs"
version = "0.19.0" version = "0.19.1"
dependencies = [ dependencies = [
"dr-types", "dr-types",
"jpeg-encoder", "jpeg-encoder",
@@ -1716,7 +1716,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-types" name = "dr-types"
version = "0.19.0" version = "0.19.1"
dependencies = [ dependencies = [
"serde", "serde",
"serde_json", "serde_json",
@@ -1725,7 +1725,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-ui" name = "dr-ui"
version = "0.19.0" version = "0.19.1"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"async-trait", "async-trait",
@@ -1773,7 +1773,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-xmp" name = "dr-xmp"
version = "0.19.0" version = "0.19.1"
dependencies = [ dependencies = [
"dr-types", "dr-types",
"log", "log",
@@ -7107,7 +7107,7 @@ checksum = "8df9b6e13f2d32c91b9bd719c00d1958837bc7dec474d94952798cc8e69eeec3"
[[package]] [[package]]
name = "traceability" name = "traceability"
version = "0.19.0" version = "0.19.1"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"proc-macro2", "proc-macro2",
+1 -1
View File
@@ -32,7 +32,7 @@ members = [
exclude = ["third_party"] exclude = ["third_party"]
[workspace.package] [workspace.package]
version = "0.19.0" version = "0.19.1"
edition = "2021" edition = "2021"
rust-version = "1.92" rust-version = "1.92"
license = "GPL-3.0-or-later" license = "GPL-3.0-or-later"
+1 -1
View File
@@ -104,7 +104,7 @@ controls, its place in the chain and its tests.
## Where it stands ## Where it stands
**0.19.0**, twenty-nine tagged releases in. 193 numbered requirements in **0.19.1**, thirty tagged releases in. 193 numbered requirements in
scope, 85% of them claimed by code and [traced to it](docs/dev/traceability.md); scope, 85% of them claimed by code and [traced to it](docs/dev/traceability.md);
the rest are written down rather than merely absent. the rest are written down rather than merely absent.
+51 -4
View File
@@ -72,17 +72,54 @@ pub enum ThumbSize {
/// Zoomed cells, the loupe, and the filmstrip. ~45 KB each, fetched only /// Zoomed cells, the loupe, and the filmstrip. ~45 KB each, fetched only
/// where something actually asks for that detail. /// where something actually asks for that detail.
Large = 1, Large = 1,
/// TRACES: FR-MRG-6
/// A panorama's cell two columns wide, at the height of one: long edge
/// sized for the width rather than for a square, since the grid class
/// of a 4:1 panorama is 256×64 — a smear across the cells. The wide
/// classes are made only for photographs that wide, so they cost a
/// library nothing else.
Wide2 = 2,
/// Three columns.
Wide3 = 3,
/// Four columns: the widest class.
Wide4 = 4,
} }
/// The most columns a wide class spans.
pub const WIDEST_SPAN: usize = 4;
impl ThumbSize { impl ThumbSize {
/// Long edge in pixels. /// Long edge in pixels. A wide class is 512 per column it spans, which
/// keeps its short edge near the large class's for the aspect that
/// class is chosen for — sharp at the largest cells on a 2x display.
pub fn edge(self) -> u32 { pub fn edge(self) -> u32 {
match self { match self {
ThumbSize::Grid => 256, ThumbSize::Grid => 256,
ThumbSize::Large => 1024, ThumbSize::Large => 1024,
ThumbSize::Wide2 => 1024,
ThumbSize::Wide3 => 1536,
ThumbSize::Wide4 => 2048,
} }
} }
/// The wide class for a cell `span` columns wide: `None` for one
/// column, and the widest class for anything past it.
pub fn wide(span: usize) -> Option<Self> {
match span {
0 | 1 => None,
2 => Some(ThumbSize::Wide2),
3 => Some(ThumbSize::Wide3),
_ => Some(ThumbSize::Wide4),
}
}
/// The class for a cell `span` columns wide whose columns are drawn at
/// `pixels`: a wide class for any cell wider than one, whatever the
/// zoom, since its height is a column's and its width is not.
pub fn for_span(span: usize, pixels: u32) -> Self {
Self::wide(span).unwrap_or_else(|| Self::for_cell(pixels))
}
/// The smallest class that can fill a cell of this size without visibly /// The smallest class that can fill a cell of this size without visibly
/// softening. /// softening.
/// ///
@@ -96,12 +133,22 @@ impl ThumbSize {
} }
} }
fn from_i64(v: i64) -> Self { /// The class a stored discriminant names, or `None` for one this build
/// does not know.
pub fn from_stored(v: i64) -> Option<Self> {
match v { match v {
1 => ThumbSize::Large, 0 => Some(ThumbSize::Grid),
_ => ThumbSize::Grid, 1 => Some(ThumbSize::Large),
2 => Some(ThumbSize::Wide2),
3 => Some(ThumbSize::Wide3),
4 => Some(ThumbSize::Wide4),
_ => None,
} }
} }
fn from_i64(v: i64) -> Self {
Self::from_stored(v).unwrap_or(ThumbSize::Grid)
}
} }
/// Long edge of a grid thumbnail. /// Long edge of a grid thumbnail.
+358
View File
@@ -0,0 +1,358 @@
# Learned denoise — joint demosaic and denoise on the mosaic
Design for **FR-DEV-3g** ([requirements.md](requirements.md)), the learned stage
[outstanding.md §3](outstanding.md) says is missing. Draft of 2026-09-27: nothing here is built,
and every figure marked *estimate* is waiting for the measurement that replaces it.
---
## 1. What we are matching
Lightroom's Denoise (April 2023, Eric Chan's "Denoise demystified") is the reference, and three
facts about it set the shape of this design:
- **It runs on the mosaic.** The network takes Bayer or X-Trans photosites before any demosaic
and emits full RGB: denoise and demosaic are one learned step. It descends from Adobe's 2019
learned demosaic (Raw Details). A photograph that is already demosaiced is not eligible.
- **It is run once, not per frame.** The result is written as a new linear DNG beside the
original, and every later edit reads that file. The amount is chosen once, from a preview crop.
- **It is trained on synthetic pairs.** Clean raws with sensor-modelled noise added, not
photographed pairs.
The reason the mosaic is the right place is physical: before the demosaic, noise is independent
per photosite with a known distribution (shot plus read). After it, the interpolation has
correlated that noise into colour blotches many photosites across, which classical noise reduction
cannot separate from texture. The same step removes demosaic artefacts
— maze, zipper, false colour, X-Trans worms (FR-RAW-5).
We match the first and third facts and not the second: our result is a cache, not a file in the
library (§7).
## 2. Where it sits
[architecture.md §5.2](architecture.md) already reserves the slot. The learned stage **replaces
the demosaic box** when it is on; nothing else in the chain moves.
```
RawImage ─► hot/dead photosites ─► black/white levels ─► ┬─ demosaic (classical) ─┬─► camera profile ─► …
└─ learned demosaic+NR ──┘
(cached, §7)
```
- **In:** the repaired, normalised mosaic, from the same buffer `Demosaicer::run` reads. The hot
pixel pass stays in front: an outlier of 50σ is outside anything the noise model generates, and
a network shown one invents a structure around it.
- **Out:** linear camera RGB, f16, full resolution — exactly the texture the classical demosaic
produces, so the camera profile, the raw histogram and every operation below it are unchanged.
- **Off by default, per photograph.** The classical path stays the default and the fallback; the
stage's absence degrades gracefully, as FR-DEV-3g requires.
## 3. The model
### 3.1 The 12×12 → 4×4 question
The proposal: a network that reads a 12×12 window of photosites and predicts the RGB of the
central 4×4, slid across the frame in steps of four.
**The output half is right. The input half is too small by a factor of five or more.**
*What is right about it.* Predicting a block aligned to the colour-filter period keeps the phase
fixed: every prediction sees the same arrangement of red, green and blue around it, so the network
never has to work out where it stands in the pattern. It also makes tiling trivial and exact.
Both properties are kept below — as the head of the network and as the tiling contract (§3.4).
*What is wrong with it.* A denoiser can only average away noise it can see around the pixel, and
at high ISO it needs to see a long way:
- The Canon 6D at ISO 6400 (clip ≈ 1,200 e⁻, read noise ≈ 2 e⁻ — *estimate*, §5 measures it) has a
mid-tone of ~150 e⁻, shot SNR ≈ 12, and a shadow three stops down of ~19 e⁻, SNR ≈ 4.
- A shadow that looks clean wants SNR ≈ 40: a factor of 10, which is ~100 independent same-colour
samples in a flat area. Red and blue are a quarter of the photosites, so that is ~400
photosites: a **20×20 window just for a flat shadow**, 40×40 two stops further down.
- A 12×12 window holds 36 red photosites. Averaged perfectly, that is a factor of 6 on red and
blue in a flat area, and less everywhere there is structure.
- Chroma blotches are low-frequency noise — 16 to 64 photosites across. A window smaller than the
blotch cannot tell it from a colour change.
Demosaic alone is content with 12×12: good classical demosaics read 5×5 to 9×9. So the proposal is
a good demosaic network and a weak denoiser — which is a useful ablation (experiment E1, §6.3).
*What it costs.* Adjacent 12×12 windows with a 4×4 output overlap nine-fold, so a network
evaluated per window recomputes each photosite's features nine times. A convolutional network is
the same computation with that work shared: it is "predict the central block from its
neighbourhood" evaluated everywhere at once.
### 3.2 The shape
```
mosaic (H×W) ──space-to-depth 2×2──► 4 ch @ H/2 × W/2 ┐
noise map σ(x) ─space-to-depth 2×2──► 4 ch @ H/2 × W/2 ┴► U-Net ─► 12 ch @ H/2 × W/2 ─depth-to-space─► RGB @ H×W
(2×2 block × RGB per position)
```
- **Packing.** Bayer is packed 2×2 into four channels at half resolution, so every input position
is one whole quad and every output position is the 2×2 block of RGB it covers — the proposal's
head, at the Bayer period. (A 4×4 packing with a 48-channel head is the same thing at a coarser
stride and is a free parameter.)
- **Phase unification.** Every body's pattern is cropped by a row or a column to RGGB before
packing, and the output is un-cropped. Flips are only used for augmentation in the CFA-preserving
form (Liu et al., "Bayer pattern unification and augmentation", 2019).
- **Body.** A U-Net with four downsamplings and NAFNet blocks (Chen et al., 2022; MIT). The
receptive field at the raw scale is several hundred photosites, which covers §3.1's worst case
with room.
- **Two sizes.** **M** (widths 32-64-128-256, ~6 M parameters, ~60 GMAC per raw megapixel —
*estimate*) is the desktop model and the one trained first. **S** (widths 16-32-64-128, fewer
bottleneck blocks, ~1 M parameters, ~12 GMAC/MP) is distilled from M for the tablet (§8).
### 3.3 Conditioning on the noise
The network is told how noisy each photosite is, rather than learning one model per ISO:
- A per-photosite standard-deviation map, `σ(x) = √(K·x + σ_r²)` from the body's gain `K` and read
noise `σ_r` at that ISO, packed alongside the mosaic (FFDNet's arrangement, Zhang et al., 2018).
- **This is what makes it camera-general.** A body it was never trained on only has to supply
`K` and `σ_r`. Three sources, in order of preference: a calibration table for the body (§5); the
DNG `NoiseProfile` tag, which Adobe's converter writes; a blind estimate from the photograph's
own flat regions (Foi et al., 2008), which always exists.
- **It is also the Amount control.** Scaling the map up tells the network there is more noise than
there is and it smooths harder; scaling it down preserves more grain. Changing the amount re-runs
inference (§7.2), which is why it is set on a preview crop, as Lightroom does.
The alternative — PMRID's k-sigma transform, which maps every ISO onto one noise level — is
simpler and gives no Amount control. It is the fallback if conditioning underperforms.
### 3.4 Tiling
A 20 MP frame does not go through a network in one piece on either device. Inference tiles the
mosaic into 512×512 input tiles with a 64-photosite halo on every side and keeps the central
384×384 of each output: the proposal's "12 in, 4 out", scaled up. Halo and tile sizes must be
multiples of 2 (the CFA phase) and of 16 (four downsamplings at half resolution), so the seams
land at identical positions in every tile's own coordinates.
This is inference-local tiling and does not depend on FR-DSP-2's render-path tiling, which stays
under the challenge [outstanding.md §4](outstanding.md) records.
## 4. Training data
### 4.1 What the library holds
From the reference catalog, 2026-09-27: 17,255 catalogued RAWs (9,345 DNG, 7,910 CR2), **all but
seven from one body, the Canon EOS 6D** (RGGB Bayer, 5472×3648, AA filter), 166 shooting days from
2015 to 2026.
| ISO | Frames | Use |
|---|---|---|
| ≤ 200 | 5,065 | Clean sources for synthetic pairs |
| 201–1600 | 7,807 | Low-noise end of the eval set |
| 1601–6400 | 3,379 | Real-noise eval set; noise-model check (§5.3) |
| > 6400 | 562 | The hard cases, by eye |
There are **no X-Trans raws**, which matters for §9. The catalog does not hold shutter speed, so
selection needs the files' EXIF. Whether the DNGs are mosaic (converted CR2) or linear must be
checked before they are counted as sources: a linear DNG has no photosites to learn from.
### 4.2 How a training pair is made
1. **Clean source.** A base-ISO 6D frame, black-subtracted and normalised.
2. **Full-colour truth by binning.** Each plane is resampled by half a photosite so the four
planes share a centre, then every 2×2 quad becomes one RGB pixel (R, mean of the two G, B):
a true full-colour image at 2736×1824 with no interpolation in it. This is the only way to have
ground truth for the demosaic half.
3. **Re-mosaic.** That RGB image is sampled back into an RGGB mosaic. (It can equally be sampled
into X-Trans, §9.)
4. **Darken and add noise.** Scale the signal by `1/g` for a target ISO `100·g`, then add noise
from the calibrated model at that ISO (§5): Poisson shot, Tukey-lambda read noise, row noise
and quantisation — the ELD model (Wei et al., CVPR 2020). The input is this mosaic; the target
is the clean RGB at the same scale.
5. **Augment.** Random blur (Gaussian, σ 0–0.7 px) before re-mosaicking, because a binned image is
sharper per pixel than the AA-filtered sensor the model will see; exposure jitter; white-balance
gains within the body's range; CFA-preserving flips.
**Why the target's own noise is tolerable.** A base-ISO frame is not noise-free, and binning only
halves the green noise; red and blue keep theirs. But darkening by `g` scales signal and target
noise together, while the added shot noise grows as `√g`. At ISO 3200 the input is ≈ 5.7× noisier
than its target, at ISO 800 only ≈ 2.8×. L1 against a noisy target converges on the median, which
is unbiased for symmetric noise. The low-ISO end is the one at risk of learning to keep grain: if
it does, bin 4×4 instead (red and blue noise halved, 1368×912 per source) for those samples.
**Why not the native mosaic as the target.** That trains denoise alone, with base-ISO noise baked
into the answer ("noisier2noise") and no demosaic truth at all.
### 4.3 How much
The limit is scene diversity, not pixel count; every source yields an effectively unlimited
number of pairs through random crops, ISO and noise draws.
| Figure | Value | Reasoning |
|---|---|---|
| Sources, train | **3,000** | 5,065 base-ISO frames, less bursts (perceptual-hash dedup), heavy clipping, motion blur and linear DNGs. For scale: ELD reaches state of the art trained on ~230 scenes; SID has ~5,000 pairs of ~400 scenes |
| Sources, validation | 200 | Split by shooting day, not by frame, so no scene is on both sides |
| Pixels | ~15 Gpx of RGB truth | 3,000 × 5 MP after binning |
| Crops per step | 8–16 × 256×256 photosites | Fits a 6 GB RTX 3050 at fp16 with M |
| Stored | ~20 GB | 24 random 512×512 crops per source, uint16, zstd. Keeping whole CR2s would be ~75 GB |
| Training | 200–400 k steps, one to two nights per run on the 3050 — *estimate*; expect three to five runs | |
Stratify the selection: across all 166 days, and deliberately include faces and hair (the library
has 19k detected faces, and skin is where over-smoothing shows first), foliage, fabric, text, and
any base-ISO tripod night work.
### 4.4 Reading raws the same way in training and in the app
The training data must be decoded by **the same decoder the app uses**. rawpy (LibRaw) and
`dr_decode::Rawler` can disagree on black level, white level, active area and therefore CFA phase,
and a network trained on one pattern phase and run on another produces colour moiré everywhere.
A `dr-decode` example that dumps the mosaic and its metadata as `.npy` is the only source the
training repo reads — not rawpy, as `darkroom-infill`'s `develop-raws.py` does.
## 5. The noise model and its calibration
### 5.1 What is measured
Per ISO: gain `K` (DN per electron), read-noise distribution (Gaussian σ and Tukey-λ shape),
row-noise σ, black-level offset and any fixed pattern. Canon's third-stop ISOs on bodies of the
6D's generation are digital gains of the full stops, so noise does not scale smoothly between
them; **every third stop is calibrated**, not interpolated.
### 5.2 The capture (one hour, once per body)
- **Darks.** Lens cap on, viewfinder covered, manual. Five frames at 1/4000 s and five at 1/30 s
at every third stop from ISO 100 to 25600. They give read noise, row noise and the black-level
pattern; the two shutter speeds confirm dark current is negligible.
- **Flats.** An evenly lit white wall, defocused, at every full stop: pairs at six exposure levels
from 1/64 of clip to 3/4 of it. The variance of each pair's difference against their mean is
the photon transfer curve, whose slope is `K`.
### 5.3 The check
Fit the same `(K, σ_r)` blindly from flat regions of the library's 3,379 ISO 1601–6400 frames
(§3.3's third source). If it disagrees with the calibration by more than ~10%, one of them is
wrong — and it tells us how far the blind estimate can be trusted for bodies with no calibration.
## 6. Evaluation
### 6.1 Real pairs (the test set)
Synthetic validation says whether the model learned the synthetic problem; only photographed pairs
say whether it learned the real one. On a tripod, with remote release and mirror lock-up, manual
focus and white balance: **12 scenes** — low-light interior, a night street, fabric, foliage, fine
text, a colour chart if one is to hand, and a still subject with skin and hair. At each, four
ISO 100 frames at a long exposure (averaged: the reference), then ISO 1600, 3200, 6400, 12800 and
25600 at the same aperture with the shutter shortened by the ISO ratio. A per-channel linear fit
against the reference absorbs residual exposure mismatch (ELD's protocol).
Plus 100 real library frames above ISO 3200 with no reference, judged by eye side by side.
### 6.2 Baseline and metrics
The baseline is today's path: the classical demosaic plus `ops/noise_reduction.rs` tuned by hand
per ISO on the validation set. If a Lightroom or DxO trial is to hand, their output on the same
twelve scenes is the ceiling, for our comparison only.
Metrics, measured after a fixed tone curve (the camera profile and an sRGB curve) and not in linear
light, where the highlights would dominate: PSNR and SSIM per ISO; chroma bias on flat patches,
because denoisers desaturate; a slanted-edge MTF for detail; and maze or zipper artefacts on the
resolution target at ISO 100.
### 6.3 Experiments that answer design questions
| | Question | Runs |
|---|---|---|
| E1 | How much context does denoise need? (§3.1) | Same data, receptive field 12, 36, 100, 300+ photosites; PSNR per ISO against it |
| E2 | Noise-map conditioning or k-sigma? (§3.3) | M both ways |
| E3 | Bin 2×2 or 4×4 for truth? (§4.2) | Compare at ISO 400–800, where it matters |
| E4 | Is the blind noise estimate good enough? (§5.3) | Inference with calibrated vs blind maps on the real pairs |
### 6.4 Acceptance
- On the real pairs, ≥ 3 dB over the baseline at ISO 6400, and **no ISO at which it is worse**,
ISO 100 included — at base ISO it has to be at least as good a demosaic as the classical one.
- Mean chroma error on flat patches under ΔE 1.
- No maze, zipper or false colour on the resolution target that the classical demosaic does not
also show.
- A 20 MP frame in ≤ 3 s on the laptop's GPU and ≤ 30 s on its CPU (§8).
## 7. In the application
### 7.1 A cache, not a new file
Lightroom writes a DNG into the library. We do not: the library is synced, a 20 MP linear RGB file
is ~120 MB, and a derived file inside a synced tree is exactly what
[storage.md](storage.md) refuses. Instead:
- The sidecar records the intent — denoise on, amount, model id — as the rest of the edit is
recorded, so it syncs and another device reproduces it.
- The result is a local cache entry: f16 linear camera RGB, zstd, keyed on
`(file identity, decoder version, model id, amount, noise source)`. ~60–80 MB per frame
(*estimate*), LRU under a budget (default 5 GB, §10).
- On open, the classical demosaic shows at once and the learned result swaps in when it is ready,
with progress over the canvas — the same pattern as a photograph that is only on the server.
- Export needs the result and computes it if the cache has lost it.
### 7.2 The Amount control
A Denoise toggle and one Amount slider in develop. Moving the slider runs inference on the
**visible viewport only** (~1 MP, a fraction of a second — *estimate*) so the photographer judges
on the real result; releasing it queues the whole frame. There is no per-frame blend between the
two paths: blending the classical output back in re-adds the noise the network removed.
### 7.3 Runtime
Through `dr-inference-engine`, as the other models run ([inference.md](inference.md)): TensorRT or
CUDA fp16 on the laptop, MIGraphX on the desktop, ORT CPU everywhere, QNN on the tablet. Work is
scheduled in the `Background` class so a slider never waits on it (architecture §5.3).
## 8. Speed and the tablet
M at ~60 GMAC/MP is ~1.2 TMAC for a 20 MP frame (*estimate*). On the RTX 3050 at fp16 that is
about a second; on 20 CPU threads, tens of seconds.
The tablet's Hexagon is fast — scrfd_10g's ~10 GFLOP in 3.2 ms, [inference.md §1.1](inference.md) —
but **accepts int8 only**, and int8 is hostile to this task: a 14-bit signal quantised to 256
levels loses the shadow steps the model exists to recover. Two ways round it, to be measured in
this order:
1. **Predict the residual, not the image.** S emits the correction to a cheap bilinear demosaic
computed in float outside the graph. The residual spans a few σ, which 256 levels resolve; the
addition happens in float. With a variance-stabilising transform (Anscombe) on the input.
2. **16-bit activations** (QNN's A16W8), if the partition log shows the HTP running them.
If neither holds S's quality within 0.5 dB of fp32 on the real pairs, **v1 is desktop-only** and the
tablet shows the classical path. The sidecar still records the intent, so a desktop can render the
learned result for a photograph edited on the tablet.
## 9. X-Trans
The requirements tie this stage to FR-RAW-5, and the library has no Fuji raws. What we can do
without a Fuji body:
- **Training does not need one.** §4.2 step 3 samples the binned RGB truth into any pattern.
X-Trans packs 6×6 into 36 channels at a sixth of the resolution, with a 108-channel head: the
same design at the X-Trans period. It is a separate model.
- **Noise does.** A calibration capture (§5.2) or, failing that, the blind estimate — plus the
DNG `NoiseProfile` of converted Fuji files.
- **The test set does.** raw.pixls.us has CC0 samples per body but no tripod ISO ladders. A few
hours with a borrowed X-Trans body and the §6.1 protocol is the honest version; without it,
X-Trans ships marked experimental.
## 10. Plan and open decisions
| Phase | Work | Output |
|---|---|---|
| P0 | Calibration capture; the `dr-decode` dump example; source selection and crop store | Noise tables, ~20 GB of crops, the 12-scene test set |
| P1 | M on Bayer; eval harness; E1–E4 | A model that passes §6.4 on the laptop |
| P2 | The stage in `dr-gpu`, cache, sidecar field, develop controls, export | A photograph denoised in the app |
| P3 | S distilled; int8 and the residual head on the tablet | Tablet in or out of v1 (§8) |
| P4 | X-Trans model | Experimental unless a body is borrowed |
Training lives in a sibling repo, `darkroom-denoise`, next to `darkroom-infill` and reusing its
hydration tools. The weights are trained from scratch on the author's own photographs with an
MIT architecture, so this model adds no third-party licence to D13.
**Decisions wanted before P1:**
1. Bin 2×2 or 4×4 for the truth, or both (E3 answers it, but the crop store is built once).
2. Cache budget and location.
3. Whether the tablet is in v1's scope or explicitly deferred behind §8's measurement.
4. Whether a Lightroom or DxO comparison is available for §6.2.
5. A borrowed X-Trans body, or X-Trans experimental in v1.
+47
View File
@@ -294,6 +294,53 @@ carrying the first source's EXIF in a sub-IFD as `dr-export` already does.
file is written and catalogued, beside its sources, with the merge as the file is written and catalogued, beside its sources, with the merge as the
first entry in its history. first entry in its history.
**How it gets there (FR-MRG-6, 2026-09-28).** A rescan fired as the merge
finished raced the upload it followed — the 800 MB copy into a folder library
was still running when the folder was listed, and a Nextcloud upload takes
minutes — so the listing lacked the composite, recorded the folder's
validator, and the grid did not show it until the next sync pass. Now:
- *Catalogued by the merge.* `MergeEvent::Done` carries a `Composite` — the
name it will have, the size of the picture it opens on (the crop, or the
whole when filled), the capture time written into the DNG (the mean of the
frames'; the sources' earliest where none has one), the body, and its
thumbnails. `library::catalogue_composite` writes the row in one
transaction, keyed on `(root_id, source_ref)` exactly as the scan will list
the file, at `metadata_state = 2`, and the grid reloads. The name is chosen
against the catalog's names in that folder (`names_in_folder`), since the
upload replaces whatever is at its name.
- *The server's half after the upload.* Once a file the catalog already has
a row for is sent, the drain lists its folder once, records the file id the
server assigned (`record_uploaded`) and puts the merge's thumbnails in the
store under it; then the grid rescans. A scan that ran before the upload
leaves the row alone, and the one after it updates it in place.
- *Thumbnails from the merge.* The bands are box-reduced as they are written,
after the fill, to a copy 4096 pixels long (`merge_thumbs::Reduced`). That
copy is written as a linear DNG in memory with the composite's own profile,
header and crop and opened through `open_session` — develop's first open:
the D19 pipeline, the default view transform and tone mapping, the as-shot
balance and the working-space-to-display conversion. The grid, large and
wide classes are rendered from that session, staged in the outbox as
`x.dng.thumbs` before the rename releases the payload, and drawn from
memory until the upload has a file id to store them under. A test develops
a synthetic composite both ways and holds the mean, 95th and 99.5th luma
percentiles within 3–4 levels; the naive balanced-and-gamma picture misses
by 13. Older composites, which have no staged thumbnails, are thumbnailed
the ordinary way.
- *A wide cell.* `library_ui::layout` places the grid as a lattice of slots.
`natural_span` maps aspect to 2, 3 or 4 columns (from 1.9, 2.45 and 3.46 —
√(s(s+1)) is where two neighbouring classes leave the same share of their
cell empty), capped at the columns there are and the whole row on the
tablet, and the same number names the thumbnail class (`Wide2`–`Wide4`, 512
pixels of long edge per column). A wide cell that does not fit in the rest
of a row starts the next; nothing later moves into the gap, so ordinals —
the arrows, a shift-click's run, the timeline, burst folding — are
untouched, and up/down step by rows through the layout. The window's own
read carries `w` and `h`; where the wide ones sit in the whole list is one
query, run when the list changes, and a library with no panorama answers it
from the partial index `images_wide`, created on first use rather than by a
schema bump.
## 10. Order of work ## 10. Order of work
1. **S15**, all four, before anything else. (1) and (2) are a day each and 1. **S15**, all four, before anything else. (1) and (2) are a day each and
File diff suppressed because one or more lines are too long
+48 -48
View File
@@ -39,7 +39,7 @@ The list is longer than it is tall, so a way to walk it that cannot be lost to t
Anchored on the fingers' midpoint, and on the pointer, so the gesture reads as magnifying the picture rather than sliding it about. Double-tap is the way to an exact 1:1; this is the way to everything in between. Past 1:1 the pixels are shown as they are, square and unsmoothed; below it, filtered. Anchored on the fingers' midpoint, and on the pointer, so the gesture reads as magnifying the picture rather than sliding it about. Double-tap is the way to an exact 1:1; this is the way to everything in between. Past 1:1 the pixels are shown as they are, square and unsmoothed; below it, filtered.
<sub>`ui/dr-ui/ui/app.slint:1977`</sub> <sub>`ui/dr-ui/ui/app.slint:1982`</sub>
### Move a magnified photograph about ### Move a magnified photograph about
@@ -50,7 +50,7 @@ Anchored on the fingers' midpoint, and on the pointer, so the gesture reads as m
Only once there is something outside the viewport to reach, which is why the cursor becomes a hand exactly then. The view is clamped to the frame: panning past the edge would show undefined area beside the photograph, and that reads as a rendering fault rather than as the end of the picture. Only once there is something outside the viewport to reach, which is why the cursor becomes a hand exactly then. The view is clamped to the frame: panning past the edge would show undefined area beside the photograph, and that reads as a rendering fault rather than as the end of the picture.
<sub>`ui/dr-ui/ui/app.slint:2073`</sub> <sub>`ui/dr-ui/ui/app.slint:2078`</sub>
### Paint a mask by hand ### Paint a mask by hand
@@ -60,7 +60,7 @@ Only once there is something outside the viewport to reach, which is why the cur
A model's mask stops inside a shoulder and leaks into the hair, and no single edge control fixes two errors that go opposite ways. The whole stroke is one step in the history, so taking a mark back costs one press however long it took to make. A model's mask stops inside a shoulder and leaks into the hair, and no single edge control fixes two errors that go opposite ways. The whole stroke is one step in the history, so taking a mark back costs one press however long it took to make.
<sub>`ui/dr-ui/ui/app.slint:2164`</sub> <sub>`ui/dr-ui/ui/app.slint:2169`</sub>
### Open this list ### Open this list
@@ -70,7 +70,7 @@ A model's mask stops inside a shoulder and leaks into the hair, and no single ed
Most of the keys are develop's, and a reference that could only be opened from the grid had to be looked up before opening the photograph they were wanted for. Most of the keys are develop's, and a reference that could only be opened from the grid had to be looked up before opening the photograph they were wanted for.
<sub>`ui/dr-ui/ui/app.slint:2390`</sub> <sub>`ui/dr-ui/ui/app.slint:2395`</sub>
### Take back the last change ### Take back the last change
@@ -81,7 +81,7 @@ Most of the keys are develop's, and a reference that could only be opened from t
A whole drag is one step, so undo takes back a decision rather than a frame of a gesture. The list is there because arriving six steps back costs what arriving from one does. A whole drag is one step, so undo takes back a decision rather than a frame of a gesture. The list is there because arriving six steps back costs what arriving from one does.
<sub>`ui/dr-ui/ui/app.slint:2420`</sub> <sub>`ui/dr-ui/ui/app.slint:2425`</sub>
### Do it again after taking it back ### Do it again after taking it back
@@ -90,7 +90,7 @@ A whole drag is one step, so undo takes back a decision rather than a frame of a
- **Keyboard** — `Ctrl+Shift+Z`, or `Ctrl+Y` - **Keyboard** — `Ctrl+Shift+Z`, or `Ctrl+Y`
- **See it** — [in the manual](manual/README.md#history-snapshots-presets) - **See it** — [in the manual](manual/README.md#history-snapshots-presets)
<sub>`ui/dr-ui/ui/app.slint:2434`</sub> <sub>`ui/dr-ui/ui/app.slint:2439`</sub>
### Remove a repair ### Remove a repair
@@ -98,7 +98,7 @@ A whole drag is one step, so undo takes back a decision rather than a frame of a
- **Pointer** — Click it, then Delete Repair - **Pointer** — Click it, then Delete Repair
- **Keyboard** — `Delete` or `Backspace`, while repairing - **Keyboard** — `Delete` or `Backspace`, while repairing
<sub>`ui/dr-ui/ui/app.slint:2454`</sub> <sub>`ui/dr-ui/ui/app.slint:2459`</sub>
### Copy the settings from this photograph ### Copy the settings from this photograph
@@ -109,7 +109,7 @@ A whole drag is one step, so undo takes back a decision rather than a frame of a
The button is the copy that has to work: a tablet has no modifier key to hold and no menu bar to hang the action from. The shortcut is an accelerator for a control that is on screen either way. The button is the copy that has to work: a tablet has no modifier key to hold and no menu bar to hang the action from. The shortcut is an accelerator for a control that is on screen either way.
<sub>`ui/dr-ui/ui/app.slint:2473`</sub> <sub>`ui/dr-ui/ui/app.slint:2478`</sub>
### Paste the settings onto this photograph ### Paste the settings onto this photograph
@@ -120,7 +120,7 @@ The button is the copy that has to work: a tablet has no modifier key to hold an
The button names what would be pasted — "3 adjustments", and whether the crop is coming with it — which the shortcut cannot say. Both paste the same scope. The button names what would be pasted — "3 adjustments", and whether the crop is coming with it — which the shortcut cannot say. Both paste the same scope.
<sub>`ui/dr-ui/ui/app.slint:2486`</sub> <sub>`ui/dr-ui/ui/app.slint:2491`</sub>
### Choose which kinds of edit a copy carries ### Choose which kinds of edit a copy carries
@@ -131,7 +131,7 @@ The button names what would be pasted — "3 adjustments", and whether the crop
Lightroom's Copy Settings. Pasting a look across a shoot usually means leaving each frame's crop and rotation alone, and that is a choice to make at the moment of copying. Lightroom's Copy Settings. Pasting a look across a shoot usually means leaving each frame's crop and rotation alone, and that is a choice to make at the moment of copying.
<sub>`ui/dr-ui/ui/app.slint:2504`</sub> <sub>`ui/dr-ui/ui/app.slint:2509`</sub>
### Export this photograph as the last one was ### Export this photograph as the last one was
@@ -142,7 +142,7 @@ Lightroom's Copy Settings. Pasting a look across a shoot usually means leaving e
Every export runs on the defaults in Settings, so "as the last one was" is what the button already does. The chord is Lightroom's and darktable's, kept so hands that learned it there need not learn it again. Every export runs on the defaults in Settings, so "as the last one was" is what the button already does. The chord is Lightroom's and darktable's, kept so hands that learned it there need not learn it again.
<sub>`ui/dr-ui/ui/app.slint:2529`</sub> <sub>`ui/dr-ui/ui/app.slint:2534`</sub>
### Choose how to export, then export ### Choose how to export, then export
@@ -153,7 +153,7 @@ Every export runs on the defaults in Settings, so "as the last one was" is what
The export sheet is the export defaults alone with an Export button. What is chosen there is kept, so it is also what the next Ctrl+Shift+E uses. The export sheet is the export defaults alone with an Export button. What is chosen there is kept, so it is also what the next Ctrl+Shift+E uses.
<sub>`ui/dr-ui/ui/app.slint:2542`</sub> <sub>`ui/dr-ui/ui/app.slint:2547`</sub>
### Keep a crop that leaves a mask outside ### Keep a crop that leaves a mask outside
@@ -161,7 +161,7 @@ The export sheet is the export defaults alone with an Export button. What is cho
- **Pointer** — Press "Keep crop" on the notice, or "Undo crop" to take it back - **Pointer** — Press "Keep crop" on the notice, or "Undo crop" to take it back
- **Keyboard** — `Enter` keeps it; `Ctrl+Z` takes the crop back, like any other step - **Keyboard** — `Enter` keeps it; `Ctrl+Z` takes the crop back, like any other step
<sub>`ui/dr-ui/ui/app.slint:2607`</sub> <sub>`ui/dr-ui/ui/app.slint:2612`</sub>
### Go back to the grid ### Go back to the grid
@@ -171,7 +171,7 @@ The export sheet is the export defaults alone with an Export button. What is cho
Lightroom's key for the grid. Escape gets there too, but a step at a time — out of a mode, then out of a zoom — where this goes straight back. Lightroom's key for the grid. Escape gets there too, but a step at a time — out of a mode, then out of a zoom — where this goes straight back.
<sub>`ui/dr-ui/ui/app.slint:2624`</sub> <sub>`ui/dr-ui/ui/app.slint:2629`</sub>
### Nudge the control last moved ### Nudge the control last moved
@@ -181,7 +181,7 @@ Lightroom's key for the grid. Escape gets there too, but a step at a time — ou
Lightroom's keys for the selected slider. There is no focus ring on a slider here, so "selected" is the last one moved — the same control `R` puts back — which covers the framing sliders, perspective included, as well as the adjustments. Lightroom's keys for the selected slider. There is no focus ring on a slider here, so "selected" is the last one moved — the same control `R` puts back — which covers the framing sliders, perspective included, as well as the adjustments.
<sub>`ui/dr-ui/ui/app.slint:2653`</sub> <sub>`ui/dr-ui/ui/app.slint:2658`</sub>
### Change which group of adjustments is on screen ### Change which group of adjustments is on screen
@@ -192,7 +192,7 @@ Lightroom's keys for the selected slider. There is no focus ring on a slider her
The groups are whatever the operation set declares itself to be about, so there are as many as the pipeline has and no key can be assigned to one of them by name. Stepping is the binding that survives a node being added. The groups are whatever the operation set declares itself to be about, so there are as many as the pipeline has and no key can be assigned to one of them by name. Stepping is the binding that survives a node being added.
<sub>`ui/dr-ui/ui/app.slint:2681`</sub> <sub>`ui/dr-ui/ui/app.slint:2686`</sub>
### Look at the photograph at 1:1 ### Look at the photograph at 1:1
@@ -203,7 +203,7 @@ The groups are whatever the operation set declares itself to be about, so there
Noise reduction and capture sharpening are judgements about single pixels, and a fitted view averages several of the file's into each one on screen — so the frame looks softer than it is and the correction goes too far. The point and the magnification survive opening the next photograph, which is what makes checking the same eye across forty portraits forty keystrokes rather than forty pans. From 1:1 on the photograph is drawn as its own pixels, each a hard-edged square, rather than smoothed into a blur. Noise reduction and capture sharpening are judgements about single pixels, and a fitted view averages several of the file's into each one on screen — so the frame looks softer than it is and the correction goes too far. The point and the magnification survive opening the next photograph, which is what makes checking the same eye across forty portraits forty keystrokes rather than forty pans. From 1:1 on the photograph is drawn as its own pixels, each a hard-edged square, rather than smoothed into a blur.
<sub>`ui/dr-ui/ui/app.slint:2717`</sub> <sub>`ui/dr-ui/ui/app.slint:2722`</sub>
### Rate this photograph ### Rate this photograph
@@ -211,7 +211,7 @@ Noise reduction and capture sharpening are judgements about single pixels, and a
- **Pointer** — Click a star in the top bar - **Pointer** — Click a star in the top bar
- **Keyboard** — `0`–`5` - **Keyboard** — `0`–`5`
<sub>`ui/dr-ui/ui/app.slint:2774`</sub> <sub>`ui/dr-ui/ui/app.slint:2779`</sub>
### Pick or reject this photograph ### Pick or reject this photograph
@@ -221,7 +221,7 @@ Noise reduction and capture sharpening are judgements about single pixels, and a
The grid's keys, on the photograph that is open (FR-UI-5, 2026-09-19). Judging here does not move on to the next frame: that belongs to culling, and in develop the photograph in front of you is the one being worked on. The grid's keys, on the photograph that is open (FR-UI-5, 2026-09-19). Judging here does not move on to the next frame: that belongs to culling, and in develop the photograph in front of you is the one being worked on.
<sub>`ui/dr-ui/ui/app.slint:2780`</sub> <sub>`ui/dr-ui/ui/app.slint:2785`</sub>
### Give this photograph a colour label ### Give this photograph a colour label
@@ -232,7 +232,7 @@ The grid's keys, on the photograph that is open (FR-UI-5, 2026-09-19). Judging h
The grid's keys, on the photograph that is open, so labelling while stepping through a folder is one hand's work. The bar names the label in words beside its mark. The grid's keys, on the photograph that is open, so labelling while stepping through a folder is one hand's work. The bar names the label in words beside its mark.
<sub>`ui/dr-ui/ui/app.slint:2810`</sub> <sub>`ui/dr-ui/ui/app.slint:2815`</sub>
### Move to the next or previous photograph ### Move to the next or previous photograph
@@ -243,7 +243,7 @@ The grid's keys, on the photograph that is open, so labelling while stepping thr
The edit on screen is saved on the way out, so stepping through a folder is as much a departure as going back to the grid and loses nothing. A and D as well as the arrows, so the left hand steps along the roll while the right stays on the mouse. The edit on screen is saved on the way out, so stepping through a folder is as much a departure as going back to the grid and loses nothing. A and D as well as the arrows, so the left hand steps along the roll while the right stays on the mouse.
<sub>`ui/dr-ui/ui/app.slint:2835`</sub> <sub>`ui/dr-ui/ui/app.slint:2840`</sub>
### See the photograph before you edited it ### See the photograph before you edited it
@@ -254,7 +254,7 @@ The edit on screen is saved on the way out, so stepping through a folder is as m
Held rather than toggled, and no split screen: a split halves the working image on the tablet the column was sized for, and the comparison photographers describe making is a flick back and forth. It takes no history step, so checking whether a frame is overcooked costs nothing to undo afterwards. Held rather than toggled, and no split screen: a split halves the working image on the tablet the column was sized for, and the comparison photographers describe making is a flick back and forth. It takes no history step, so checking whether a frame is overcooked costs nothing to undo afterwards.
<sub>`ui/dr-ui/ui/app.slint:2965`</sub> <sub>`ui/dr-ui/ui/app.slint:2970`</sub>
### Put one control back to its default ### Put one control back to its default
@@ -521,7 +521,7 @@ The right match confidence is a property of your library, not of the model. "Wha
Touch has no ctrl, so without a mode there is no way to select a second photograph — the first tap would open it. The hold is the fast way in and the button is the one that can be found. Touch has no ctrl, so without a mode there is no way to select a second photograph — the first tap would open it. The hold is the fast way in and the button is the one that can be found.
<sub>`ui/dr-ui/ui/library.slint:1706`</sub> <sub>`ui/dr-ui/ui/library.slint:1732`</sub>
### Add or remove one photograph ### Add or remove one photograph
@@ -531,7 +531,7 @@ Touch has no ctrl, so without a mode there is no way to select a second photogra
While selecting, a tap never opens. That is the whole point of the mode: one meaning per gesture at a time. Press Done to get tap-to-open back. While selecting, a tap never opens. That is the whole point of the mode: one meaning per gesture at a time. Press Done to get tap-to-open back.
<sub>`ui/dr-ui/ui/library.slint:1716`</sub> <sub>`ui/dr-ui/ui/library.slint:1742`</sub>
### Leave selecting ### Leave selecting
@@ -540,7 +540,7 @@ While selecting, a tap never opens. That is the whole point of the mode: one mea
- **Keyboard** — `Escape`, or `Back`; an open sheet closes first - **Keyboard** — `Escape`, or `Back`; an open sheet closes first
- **See it** — [in the manual](manual/README.md#selecting-several) - **See it** — [in the manual](manual/README.md#selecting-several)
<sub>`ui/dr-ui/ui/library.slint:1725`</sub> <sub>`ui/dr-ui/ui/library.slint:1751`</sub>
### Pick a photograph up to drag it ### Pick a photograph up to drag it
@@ -550,7 +550,7 @@ While selecting, a tap never opens. That is the whole point of the mode: one mea
A finger on a photograph might be starting a scroll, and for the first half-second the grid assumes it is. Holding says otherwise, and the ring is the grid saying it heard — from there the drag cannot be lost to a scroll. A mouse never waits: the cursor is precise enough that a sideways drag is unambiguous from the first pixel. A finger on a photograph might be starting a scroll, and for the first half-second the grid assumes it is. Holding says otherwise, and the ring is the grid saying it heard — from there the drag cannot be lost to a scroll. A mouse never waits: the cursor is precise enough that a sideways drag is unambiguous from the first pixel.
<sub>`ui/dr-ui/ui/library.slint:1756`</sub> <sub>`ui/dr-ui/ui/library.slint:1782`</sub>
### Select a range ### Select a range
@@ -561,7 +561,7 @@ A finger on a photograph might be starting a scroll, and for the first half-seco
This replaced a double tap, which had no visible state and could take forty photographs by accident. The run is resolved by the catalog rather than by what is on screen, so the grid can scroll between the two taps — the ranges that hurt on a tablet are longer than a screenful, which is exactly where a finger sweep runs out. This replaced a double tap, which had no visible state and could take forty photographs by accident. The run is resolved by the catalog rather than by what is on screen, so the grid can scroll between the two taps — the ranges that hurt on a tablet are longer than a screenful, which is exactly where a finger sweep runs out.
<sub>`ui/dr-ui/ui/library.slint:1822`</sub> <sub>`ui/dr-ui/ui/library.slint:1848`</sub>
### Take the blinks out of a burst ### Take the blinks out of a burst
@@ -571,7 +571,7 @@ This replaced a double tap, which had no visible state and could take forty phot
Face indexing reads each face's eyes. The chip drops frames where the chosen people are caught blinking, and leaves sunglasses and eyes it could not read alone. Face indexing reads each face's eyes. The chip drops frames where the chosen people are caught blinking, and leaves sunglasses and eyes it could not read alone.
<sub>`ui/dr-ui/ui/library.slint:2540`</sub> <sub>`ui/dr-ui/ui/library.slint:2573`</sub>
### Find photographs with two people in them ### Find photographs with two people in them
@@ -581,7 +581,7 @@ Face indexing reads each face's eyes. The chip drops frames where the chosen peo
"Any of them" is a union and "all of them" is an intersection. The tray is where both terms and the choice between them live, because a filter belongs on the filter bar. "Any of them" is a union and "all of them" is an intersection. The tray is where both terms and the choice between them live, because a filter belongs on the filter bar.
<sub>`ui/dr-ui/ui/library.slint:2570`</sub> <sub>`ui/dr-ui/ui/library.slint:2603`</sub>
### Show only photographs with one colour label ### Show only photographs with one colour label
@@ -591,7 +591,7 @@ Face indexing reads each face's eyes. The chip drops frames where the chosen peo
Each chip is the label's mark and its name, so the one you want is found by reading it; tap the lit chip again to show every label. Each chip is the label's mark and its name, so the one you want is found by reading it; tap the lit chip again to show every label.
<sub>`ui/dr-ui/ui/library.slint:2694`</sub> <sub>`ui/dr-ui/ui/library.slint:2727`</sub>
### Export the selection as the last export was ### Export the selection as the last export was
@@ -602,7 +602,7 @@ Each chip is the label's mark and its name, so the one you want is found by read
Lightroom's and darktable's chords. Every export runs on the saved defaults, so the plain chord opens them beside an Export button and the shifted one skips straight to exporting. Lightroom's and darktable's chords. Every export runs on the saved defaults, so the plain chord opens them beside an Export button and the shifted one skips straight to exporting.
<sub>`ui/dr-ui/ui/library.slint:3164`</sub> <sub>`ui/dr-ui/ui/library.slint:3197`</sub>
### Paste copied settings onto the selection ### Paste copied settings onto the selection
@@ -611,7 +611,7 @@ Lightroom's and darktable's chords. Every export runs on the saved defaults, so
- **Keyboard** — `Ctrl+V` - **Keyboard** — `Ctrl+V`
- **See it** — [in the manual](manual/README.md#copying-settings) - **See it** — [in the manual](manual/README.md#copying-settings)
<sub>`ui/dr-ui/ui/library.slint:3188`</sub> <sub>`ui/dr-ui/ui/library.slint:3221`</sub>
### Keyword the selection ### Keyword the selection
@@ -621,7 +621,7 @@ Lightroom's and darktable's chords. Every export runs on the saved defaults, so
Lightroom's keywording chord. The sheet opens with its field ready for typing, so the keys that judge in the grid are out of the way until it closes. Lightroom's keywording chord. The sheet opens with its field ready for typing, so the keys that judge in the grid are out of the way until it closes.
<sub>`ui/dr-ui/ui/library.slint:3217`</sub> <sub>`ui/dr-ui/ui/library.slint:3250`</sub>
### Show only photographs with some number of stars ### Show only photographs with some number of stars
@@ -632,7 +632,7 @@ Lightroom's keywording chord. The sheet opens with its field ready for typing, s
The chips say "this many or more". A range with a ceiling — the twos and threes still to be decided — is the keyboard's alone, and the bar says so in words while it holds. The chips say "this many or more". A range with a ceiling — the twos and threes still to be decided — is the keyboard's alone, and the bar says so in words while it holds.
<sub>`ui/dr-ui/ui/library.slint:3251`</sub> <sub>`ui/dr-ui/ui/library.slint:3284`</sub>
### Give photographs a colour label ### Give photographs a colour label
@@ -643,7 +643,7 @@ The chips say "this many or more". A range with a ceiling — the twos and three
Lightroom's keys, so hands that learned them there need not learn them again. Purple has no key there either, and is on the bar. Every mark carries its label's initial, so the label is read without telling the colours apart. Lightroom's keys, so hands that learned them there need not learn them again. Purple has no key there either, and is on the bar. Every mark carries its label's initial, so the label is read without telling the colours apart.
<sub>`ui/dr-ui/ui/library.slint:3301`</sub> <sub>`ui/dr-ui/ui/library.slint:3334`</sub>
### Pick or reject a photograph ### Pick or reject a photograph
@@ -653,7 +653,7 @@ Lightroom's keys, so hands that learned them there need not learn them again. Pu
The keys every culling tool uses, so muscle memory built elsewhere works here. The keys every culling tool uses, so muscle memory built elsewhere works here.
<sub>`ui/dr-ui/ui/library.slint:3325`</sub> <sub>`ui/dr-ui/ui/library.slint:3358`</sub>
### Move photographs to the trash ### Move photographs to the trash
@@ -663,7 +663,7 @@ The keys every culling tool uses, so muscle memory built elsewhere works here.
The bin acts on one photograph, so a stray click cannot trash a selection; the key acts on the selection because that is what every file manager's Delete does. Both are undone from the trash view. The bin acts on one photograph, so a stray click cannot trash a selection; the key acts on the selection because that is what every file manager's Delete does. Both are undone from the trash view.
<sub>`ui/dr-ui/ui/library.slint:3352`</sub> <sub>`ui/dr-ui/ui/library.slint:3385`</sub>
### Open this list ### Open this list
@@ -671,7 +671,7 @@ The bin acts on one photograph, so a stray click cannot trash a selection; the k
- **Pointer** — Press Help in the header, and Done to put it away - **Pointer** — Press Help in the header, and Done to put it away
- **Keyboard** — `F1`, and `Escape` to put it away - **Keyboard** — `F1`, and `Escape` to put it away
<sub>`ui/dr-ui/ui/library.slint:3377`</sub> <sub>`ui/dr-ui/ui/library.slint:3410`</sub>
### Rename the collection the grid is showing ### Rename the collection the grid is showing
@@ -679,7 +679,7 @@ The bin acts on one photograph, so a stray click cannot trash a selection; the k
- **Pointer** — Double-click it in the sidebar - **Pointer** — Double-click it in the sidebar
- **Keyboard** — `F2` - **Keyboard** — `F2`
<sub>`ui/dr-ui/ui/library.slint:3385`</sub> <sub>`ui/dr-ui/ui/library.slint:3418`</sub>
### Move through the grid ### Move through the grid
@@ -689,7 +689,7 @@ The bin acts on one photograph, so a stray click cannot trash a selection; the k
The cursor selects what it lands on, so walking and judging are one hand's work. The cursor selects what it lands on, so walking and judging are one hand's work.
<sub>`ui/dr-ui/ui/library.slint:3405`</sub> <sub>`ui/dr-ui/ui/library.slint:3438`</sub>
### Resize the thumbnails ### Resize the thumbnails
@@ -700,7 +700,7 @@ The cursor selects what it lands on, so walking and judging are one hand's work.
There is no wheel on a tablet, so without the pinch the cell size could only be changed by a control a finger cannot reach. There is no wheel on a tablet, so without the pinch the cell size could only be changed by a control a finger cannot reach.
<sub>`ui/dr-ui/ui/library.slint:3536`</sub> <sub>`ui/dr-ui/ui/library.slint:3567`</sub>
### File photographs in a collection ### File photographs in a collection
@@ -710,7 +710,7 @@ There is no wheel on a tablet, so without the pinch the cell size could only be
The selection is what the drag carries, which is why selecting several is worth the mode: forty photographs file in one gesture. The selection is what the drag carries, which is why selecting several is worth the mode: forty photographs file in one gesture.
<sub>`ui/dr-ui/ui/library.slint:3735`</sub> <sub>`ui/dr-ui/ui/library.slint:3766`</sub>
### Open a photograph ### Open a photograph
@@ -721,7 +721,7 @@ The selection is what the drag carries, which is why selecting several is worth
A tap opens; a tap that *moved* does not. Travel is what separates a deliberate tap from a hand brushing past, and it is the only thing that does: the two are the same length. An earlier version required the finger to dwell 120 ms instead, and that rejected ordinary taps — a real tap is often quicker than a brush. A tap opens; a tap that *moved* does not. Travel is what separates a deliberate tap from a hand brushing past, and it is the only thing that does: the two are the same length. An earlier version required the finger to dwell 120 ms instead, and that rejected ordinary taps — a real tap is often quicker than a brush.
<sub>`ui/dr-ui/ui/library.slint:4040`</sub> <sub>`ui/dr-ui/ui/library.slint:4073`</sub>
### Rate a photograph without opening it ### Rate a photograph without opening it
@@ -732,7 +732,7 @@ A tap opens; a tap that *moved* does not. Travel is what separates a deliberate
A star has to take the press without it also reaching the cell, or every rating throws the user into develop. A star has to take the press without it also reaching the cell, or every rating throws the user into develop.
<sub>`ui/dr-ui/ui/library.slint:4163`</sub> <sub>`ui/dr-ui/ui/library.slint:4196`</sub>
### Choose the frame a folded burst shows ### Choose the frame a folded burst shows
@@ -742,7 +742,7 @@ A star has to take the press without it also reaching the cell, or every rating
A folded burst draws its earliest frame, which is a fact about the clock and not a judgement about the photograph — nothing in this application ranks a frame (FR-CULL-5). But the point of a burst is that one of the twelve is better than the other eleven, and the photographer is the only one who knows which. So the choice is offered on the frames themselves, while they are open and side by side, which is the one moment the alternatives are on screen to be compared. A folded burst draws its earliest frame, which is a fact about the clock and not a judgement about the photograph — nothing in this application ranks a frame (FR-CULL-5). But the point of a burst is that one of the twelve is better than the other eleven, and the photographer is the only one who knows which. So the choice is offered on the frames themselves, while they are open and side by side, which is the one moment the alternatives are on screen to be compared.
<sub>`ui/dr-ui/ui/library.slint:4296`</sub> <sub>`ui/dr-ui/ui/library.slint:4329`</sub>
### Drop the selection but keep selecting ### Drop the selection but keep selecting
@@ -753,7 +753,7 @@ A folded burst draws its earliest frame, which is a fact about the clock and not
Distinct from Done, which leaves the mode entirely. Clearing keeps it, so the next selection can start straight away. Distinct from Done, which leaves the mode entirely. Clearing keeps it, so the next selection can start straight away.
<sub>`ui/dr-ui/ui/library.slint:4987`</sub> <sub>`ui/dr-ui/ui/library.slint:5022`</sub>
### Select everything the grid is showing ### Select everything the grid is showing
@@ -764,7 +764,7 @@ Distinct from Done, which leaves the mode entirely. Clearing keeps it, so the ne
A scoped grid of two hundred frames is two hundred taps otherwise, and "all of them, except those three" is a far more common shape than the taps it took to say it. A scoped grid of two hundred frames is two hundred taps otherwise, and "all of them, except those three" is a far more common shape than the taps it took to say it.
<sub>`ui/dr-ui/ui/library.slint:5006`</sub> <sub>`ui/dr-ui/ui/library.slint:5041`</sub>
### Take photographs out of a collection ### Take photographs out of a collection
@@ -774,7 +774,7 @@ A scoped grid of two hundred frames is two hundred taps otherwise, and "all of t
The badge on a cell says a photograph is filed in three collections and never which. This is the sheet that names them, and the only way out of one the grid is not currently scoped to. The badge on a cell says a photograph is filed in three collections and never which. This is the sheet that names them, and the only way out of one the grid is not currently scoped to.
<sub>`ui/dr-ui/ui/library.slint:5187`</sub> <sub>`ui/dr-ui/ui/library.slint:5222`</sub>
## Settings ## Settings
+17 -2
View File
@@ -75,6 +75,14 @@ past the window. On a tablet they scroll by flick alone, and a thin line at
the right-hand edge shows where the view is while it moves, fading once it the right-hand edge shows where the view is while it moves, fading once it
stops; it is only a picture, and a flick that starts on it scrolls the list. stops; it is only a picture, and a flick that starts on it scrolls the list.
A panorama gets a wider cell: about twice as wide as it is tall and it spans
two columns, then three, then four for the widest — whichever leaves the least
of the cell empty — with a thumbnail made for that width. One that would not
fit in what is left of a row starts the next, so the grid still reads in the
order the photographs were taken; the arrows walk it in that order, and up and
down go to whatever is above or below. On a tablet, or with too few columns to
put it beside anything, a panorama takes the whole row.
`Help` in the header, or `F1`, opens the controls and shortcuts: every key and `Help` in the header, or `F1`, opens the controls and shortcuts: every key and
gesture, screen by screen, with `See it` beside those this page shows, and gesture, screen by screen, with `See it` beside those this page shows, and
`Manual` to open this page. In develop it is the `?` beside `Settings`. `Manual` to open this page. In develop it is the `?` beside `Settings`.
@@ -345,8 +353,13 @@ outlined where it landed — twelve hand-held portrait frames across an alpine
valley, here. Change the projection (a 150° sweep on a flat perspective is valley, here. Change the projection (a 150° sweep on a flat perspective is
what the middle of the film shows, and why cylindrical is suggested), ask for what the middle of the film shows, and why cylindrical is suggested), ask for
the border to be filled rather than cropped, then `Merge`. The composite is the border to be filled rather than cropped, then `Merge`. The composite is
written beside its sources as a DNG and appears in the grid with the merge written beside its sources as a DNG and is in the grid the moment it is
as the first step in its history. written — in a wide cell beside its frames, placed by when they were taken —
with the merge as the first step in its history. Its thumbnail is made during
the merge, from the finished picture, as develop will show it when you open
it; on a server library it is in the grid while the file is still uploading.
A second merge of the same frames is named `-pano-2`, never written over the
first.
Each frame has a box in the `Frames` list. Untick one to leave it out, and Each frame has a box in the `Frames` list. Untick one to leave it out, and
the rest are aligned again at once, without reading the frames again; tick the rest are aligned again at once, without reading the frames again; tick
@@ -367,6 +380,8 @@ goes for a panorama Lightroom stitched and saved as a DNG.
![The same, with the ragged border filled by the model rather than cropped away](media/panorama-filled.png) ![The same, with the ragged border filled by the model rather than cropped away](media/panorama-filled.png)
![The composite in the grid straight after the merge, spanning four columns beside the twelve frames it was made from](media/panorama-in-grid.png)
## Export ## Export
`Export` in the develop header, or `Export N` from a selection. Format, `Export` in the develop header, or `Export N` from a selection. Format,
+15 -2
View File
@@ -194,6 +194,13 @@ the sidebar, the develop column and Settings have one too whenever they run
past the window. On a tablet they scroll by flick alone, and a thin line at past the window. On a tablet they scroll by flick alone, and a thin line at
the right-hand edge shows where the view is while it moves, fading once it the right-hand edge shows where the view is while it moves, fading once it
stops; it is only a picture, and a flick that starts on it scrolls the list.</p> stops; it is only a picture, and a flick that starts on it scrolls the list.</p>
<p>A panorama gets a wider cell: about twice as wide as it is tall and it spans
two columns, then three, then four for the widest — whichever leaves the least
of the cell empty — with a thumbnail made for that width. One that would not
fit in what is left of a row starts the next, so the grid still reads in the
order the photographs were taken; the arrows walk it in that order, and up and
down go to whatever is above or below. On a tablet, or with too few columns to
put it beside anything, a panorama takes the whole row.</p>
<p><code>Help</code> in the header, or <code>F1</code>, opens the controls and shortcuts: every key and <p><code>Help</code> in the header, or <code>F1</code>, opens the controls and shortcuts: every key and
gesture, screen by screen, with <code>See it</code> beside those this page shows, and gesture, screen by screen, with <code>See it</code> beside those this page shows, and
<code>Manual</code> to open this page. In develop it is the <code>?</code> beside <code>Settings</code>.</p> <code>Manual</code> to open this page. In develop it is the <code>?</code> beside <code>Settings</code>.</p>
@@ -389,8 +396,13 @@ outlined where it landed — twelve hand-held portrait frames across an alpine
valley, here. Change the projection (a 150° sweep on a flat perspective is valley, here. Change the projection (a 150° sweep on a flat perspective is
what the middle of the film shows, and why cylindrical is suggested), ask for what the middle of the film shows, and why cylindrical is suggested), ask for
the border to be filled rather than cropped, then <code>Merge</code>. The composite is the border to be filled rather than cropped, then <code>Merge</code>. The composite is
written beside its sources as a DNG and appears in the grid with the merge written beside its sources as a DNG and is in the grid the moment it is
as the first step in its history.</p> written — in a wide cell beside its frames, placed by when they were taken —
with the merge as the first step in its history. Its thumbnail is made during
the merge, from the finished picture, as develop will show it when you open
it; on a server library it is in the grid while the file is still uploading.
A second merge of the same frames is named <code>-pano-2</code>, never written over the
first.</p>
<p>Each frame has a box in the <code>Frames</code> list. Untick one to leave it out, and <p>Each frame has a box in the <code>Frames</code> list. Untick one to leave it out, and
the rest are aligned again at once, without reading the frames again; tick the rest are aligned again at once, without reading the frames again; tick
it to bring it back. A frame that cannot be placed is named there with why, it to bring it back. A frame that cannot be placed is named there with why,
@@ -404,6 +416,7 @@ goes for a panorama Lightroom stitched and saved as a DNG.</p>
<figure><img loading="lazy" src="media/panorama.gif" alt="Twelve frames aligned, the first left out and brought back, the projections tried, and the border filled"><figcaption>Twelve frames aligned, the first left out and brought back, the projections tried, and the border filled</figcaption></figure> <figure><img loading="lazy" src="media/panorama.gif" alt="Twelve frames aligned, the first left out and brought back, the projections tried, and the border filled"><figcaption>Twelve frames aligned, the first left out and brought back, the projections tried, and the border filled</figcaption></figure>
<figure><img loading="lazy" src="media/panorama-aligned.png" alt="The alignment on a cylinder, each frame outlined where it landed"><figcaption>The alignment on a cylinder, each frame outlined where it landed</figcaption></figure> <figure><img loading="lazy" src="media/panorama-aligned.png" alt="The alignment on a cylinder, each frame outlined where it landed"><figcaption>The alignment on a cylinder, each frame outlined where it landed</figcaption></figure>
<figure><img loading="lazy" src="media/panorama-filled.png" alt="The same, with the ragged border filled by the model rather than cropped away"><figcaption>The same, with the ragged border filled by the model rather than cropped away</figcaption></figure> <figure><img loading="lazy" src="media/panorama-filled.png" alt="The same, with the ragged border filled by the model rather than cropped away"><figcaption>The same, with the ragged border filled by the model rather than cropped away</figcaption></figure>
<figure><img loading="lazy" src="media/panorama-in-grid.png" alt="The composite in the grid straight after the merge, spanning four columns beside the twelve frames it was made from"><figcaption>The composite in the grid straight after the merge, spanning four columns beside the twelve frames it was made from</figcaption></figure>
<h2 id="export">Export</h2> <h2 id="export">Export</h2>
<p><code>Export</code> in the develop header, or <code>Export N</code> from a selection. Format, <p><code>Export</code> in the develop header, or <code>Export N</code> from a selection. Format,
size, colour space, sharpening and naming are in Settings, and apply to every size, colour space, sharpening and naming are in Settings, and apply to every
Binary file not shown.
Binary file not shown.
+1 -1
View File
@@ -4,7 +4,7 @@
# makes `makepkg -si` in this directory install what you are actually working # makes `makepkg -si` in this directory install what you are actually working
# on. Swap `source` for a tagged tarball when there is something to release. # on. Swap `source` for a tagged tarball when there is something to release.
pkgname=darkroom pkgname=darkroom
pkgver=0.19.0 pkgver=0.19.1
# Back to 1 with the version: a new pkgver is a new archive name, so there is # Back to 1 with the version: a new pkgver is a new archive name, so there is
# nothing for makepkg to reuse and nothing for a release number to disambiguate. # nothing for makepkg to reuse and nothing for a release number to disambiguate.
pkgrel=1 pkgrel=1
+11 -2
View File
@@ -1314,9 +1314,10 @@ def join(name, parts):
os.remove(f'{OUT}/{p}.mp4') os.remove(f'{OUT}/{p}.mp4')
@scene(media=['panorama.gif', 'panorama-aligned.png', 'panorama-filled.png'], @scene(media=['panorama.gif', 'panorama-aligned.png', 'panorama-filled.png', 'panorama-in-grid.png'],
sources=COMMON + ['ui/dr-ui/ui/merge.slint', 'ui/dr-ui/src/merge_ui.rs', 'ui/dr-ui/src/merge.rs', sources=COMMON + ['ui/dr-ui/ui/merge.slint', 'ui/dr-ui/src/merge_ui.rs', 'ui/dr-ui/src/merge.rs',
'core/dr-pano/src/**']) 'ui/dr-ui/src/merge_thumbs.rs', 'ui/dr-ui/src/library_ui/layout.rs',
'ui/dr-ui/ui/library.slint', 'core/dr-pano/src/**'])
def panorama(): def panorama():
select_pano() select_pano()
rec('panorama-0') rec('panorama-0')
@@ -1358,6 +1359,14 @@ def panorama():
wait_for_new(DEMO_LIBRARY, '.dng', started, 600) if DEMO_LIBRARY else pause(60) wait_for_new(DEMO_LIBRARY, '.dng', started, 600) if DEMO_LIBRARY else pause(60)
pause(5) pause(5)
at_library() at_library()
# The composite in the grid beside its frames, in its wide cell, with the
# thumbnail the merge made: catalogued when it was written, not when a
# scan got round to it.
grid_top()
dr.wait_for(f'{PANO_FIRST}-pano', 120)
show_cell(f'{PANO_FIRST}-pano')
pause(2)
shot('panorama-in-grid')
# --- settings --------------------------------------------------------------- # --- settings ---------------------------------------------------------------
+1 -1
View File
@@ -315,7 +315,7 @@ fn wire_export(window: &AppWindow, w: &DevelopWiring) {
total, total,
to, to,
move |written| { move |written| {
drain_outbox(&library_for_drain); drain_outbox(&library_for_drain, weak.clone());
// What the album now holds, and which photograph // What the album now holds, and which photograph
// each file came from. // each file came from.
if let (Some(albums), Some(w)) = (albums.as_ref(), weak.upgrade()) { if let (Some(albums), Some(w)) = (albums.as_ref(), weak.upgrade()) {
+589 -22
View File
@@ -146,6 +146,20 @@ impl Pending {
} }
} }
/// TRACES: FR-MRG-6
/// Where a file staged for `remote_dir` of the library at `root`, called
/// `name`, will be once it is uploaded — spelled as the scan will list it,
/// because the catalog row written ahead of the scan is keyed on it.
pub(crate) fn staged_remote_path(root: &str, remote_dir: &str, name: &str) -> RemotePath {
Pending {
local: PathBuf::new(),
remote_dir: remote_dir.to_string(),
account: false,
name: name.to_string(),
}
.remote_path(root)
}
/// Where an export was put, for the interface to report. /// Where an export was put, for the interface to report.
#[derive(Debug, Clone, PartialEq, Eq)] #[derive(Debug, Clone, PartialEq, Eq)]
pub enum Placed { pub enum Placed {
@@ -332,6 +346,75 @@ pub(crate) fn destination_record(payload: &Path) -> PathBuf {
)) ))
} }
/// TRACES: FR-MRG-6
/// Where the thumbnails made with a staged payload wait for its upload:
/// `name.ext.thumbs` beside it.
///
/// Not a record: [`pending`] reads only `.dest` files, so this never goes to
/// the server. The upload reads it once the server has named the file
/// ([`take_thumbnails`]) and puts its contents in the thumbnail store under
/// that name; until then there is no key to store them under.
pub(crate) fn thumbnails_record(payload: &Path) -> PathBuf {
let mut name = payload.as_os_str().to_owned();
name.push(".thumbs");
PathBuf::from(name)
}
/// The magic line a thumbnails record starts with.
const THUMBS_MAGIC: &[u8] = b"darkroom-thumbs 1\n";
/// Write the thumbnails for a staged payload. A size-class byte and three
/// little-endian `u32`s — width, height, length — before each JPEG.
pub(crate) fn write_thumbnails(
payload: &Path,
thumbnails: &[(dr_thumbs::ThumbSize, dr_thumbs::Thumbnail)],
) -> std::io::Result<()> {
let mut out = THUMBS_MAGIC.to_vec();
for (size, t) in thumbnails {
out.push(*size as i64 as u8);
for v in [t.width, t.height, t.bytes.len() as u32] {
out.extend_from_slice(&v.to_le_bytes());
}
out.extend_from_slice(&t.bytes);
}
std::fs::write(thumbnails_record(payload), out)
}
/// Read and remove the thumbnails waiting beside a payload. Nothing, for a
/// payload that has none or a record that does not parse — a composite whose
/// thumbnails are lost is thumbnailed the ordinary way.
pub(crate) fn take_thumbnails(payload: &Path) -> Vec<(dr_thumbs::ThumbSize, dr_thumbs::Thumbnail)> {
let path = thumbnails_record(payload);
let Ok(bytes) = std::fs::read(&path) else {
return Vec::new();
};
let _ = std::fs::remove_file(&path);
parse_thumbnails(&bytes).unwrap_or_default()
}
fn parse_thumbnails(bytes: &[u8]) -> Option<Vec<(dr_thumbs::ThumbSize, dr_thumbs::Thumbnail)>> {
let mut rest = bytes.strip_prefix(THUMBS_MAGIC)?;
let mut out = Vec::new();
while let Some((&class, tail)) = rest.split_first() {
let word = |at: usize| -> Option<u32> {
Some(u32::from_le_bytes(tail.get(at..at + 4)?.try_into().ok()?))
};
let (width, height, len) = (word(0)?, word(4)?, word(8)? as usize);
let data = tail.get(12..12 + len)?;
let size = dr_thumbs::ThumbSize::from_stored(i64::from(class))?;
out.push((
size,
dr_thumbs::Thumbnail {
width,
height,
bytes: data.to_vec(),
},
));
rest = &tail[12 + len..];
}
Some(out)
}
/// Everything currently waiting in the outbox. /// Everything currently waiting in the outbox.
/// ///
/// A payload with no record is skipped rather than guessed at — see the write /// A payload with no record is skipped rather than guessed at — see the write
@@ -381,11 +464,12 @@ pub fn pending_count(outbox: &Path) -> usize {
pending(outbox).len() pending(outbox).len()
} }
/// Remove an entry and its record, once it is safely on the server. /// Remove an entry and its records, once it is safely on the server.
fn clear(entry: &Pending) { fn clear(entry: &Pending) {
let record = PathBuf::from(format!("{}.dest", entry.local.display())); let record = PathBuf::from(format!("{}.dest", entry.local.display()));
let _ = std::fs::remove_file(&entry.local); let _ = std::fs::remove_file(&entry.local);
let _ = std::fs::remove_file(&record); let _ = std::fs::remove_file(&record);
let _ = std::fs::remove_file(thumbnails_record(&entry.local));
} }
/// Progress from the upload worker. /// Progress from the upload worker.
@@ -393,13 +477,173 @@ fn clear(entry: &Pending) {
pub enum UploadMessage { pub enum UploadMessage {
Status(String), Status(String),
/// Uploaded, still pending, and the first error if there was one. /// Uploaded, still pending, and the first error if there was one.
/// `landed` counts the uploads that went into the library itself rather
/// than an album outside it: those are what a scan has something to
/// find.
Finished { Finished {
uploaded: usize, uploaded: usize,
remaining: usize, remaining: usize,
error: Option<String>, error: Option<String>,
landed: usize,
}, },
} }
/// Log a drain as it goes and, once files have landed in the library, ask
/// the grid to rescan — the scan that finds them, run after the upload
/// rather than beside it (FR-MRG-6).
///
/// On a thread of its own, since the drain reports only when it finishes and
/// that can be minutes for a composite; the window is reached through the
/// event loop.
pub(crate) fn watch_upload(rx: Receiver<UploadMessage>, window: slint::Weak<AppWindow>) {
executors::spawn(Executor::Io, "upload-log", move || {
while let Ok(msg) = rx.recv() {
match msg {
UploadMessage::Status(s) => log::info!("export: {s}"),
UploadMessage::Finished {
uploaded,
remaining,
error,
landed,
} => {
log::info!("export: {uploaded} uploaded, {remaining} still queued");
if let Some(e) = error {
log::warn!("export upload stopped: {e}");
}
if landed > 0 {
let _ = window.upgrade_in_event_loop(|w| {
w.global::<Library>().invoke_library_rescan();
});
}
}
}
}
});
}
/// Where a library's catalog and thumbnail store are, for the upload to
/// record what the server made of a file.
pub(crate) struct LibraryFiles {
pub catalog: PathBuf,
pub thumbs: PathBuf,
}
impl LibraryFiles {
fn of(account: &Account) -> Self {
LibraryFiles {
catalog: crate::library::catalog_path(account),
thumbs: crate::library::thumbs_dir(account),
}
}
}
/// What became of one outbox entry.
#[derive(Debug, PartialEq, Eq)]
enum Sent {
Uploaded,
/// Its payload had gone; the record went with it.
Gone,
}
/// Send one outbox entry, and clear it once it is on the server.
async fn send(
backend: &dyn dr_sync::RemoteBackend,
library: &LibraryFiles,
root: &str,
entry: &Pending,
) -> Result<Sent, String> {
let Ok(bytes) = std::fs::read(&entry.local) else {
// The payload vanished under us. Drop the record too; retrying
// forever against a file that is gone helps nobody.
clear(entry);
return Ok(Sent::Gone);
};
// The folder may not exist — this is the first export into it — and
// `create_dir` treats "already there" as success, so it is unconditional
// rather than guarded by a check that would cost a request every time.
backend
.create_dir(&entry.remote_folder(root))
.await
.map_err(|e| e.to_string())?;
backend
.put(&entry.remote_path(root), bytes, None)
.await
.map_err(|e| e.to_string())?;
let thumbnails = take_thumbnails(&entry.local);
register_upload(backend, library, root, entry, thumbnails).await;
clear(entry);
Ok(Sent::Uploaded)
}
/// TRACES: FR-MRG-6
/// Tell the catalog what the server made of a file it has just been given.
///
/// Only for a file the catalog already has a row for — a composite the merge
/// catalogued before its upload. Anything else is the scan's to find, and
/// this spends no request on it. The listing is one request for the folder,
/// and it is the only way to learn the id a server assigns on upload: the
/// thumbnail store keys on it, and the thumbnails made during the merge wait
/// beside the payload until it is known.
async fn register_upload(
backend: &dyn dr_sync::RemoteBackend,
library: &LibraryFiles,
root: &str,
entry: &Pending,
thumbnails: Vec<(dr_thumbs::ThumbSize, dr_thumbs::Thumbnail)>,
) {
if entry.account {
return;
}
let path = entry.remote_path(root);
let catalog = match dr_catalog::Catalog::open(&library.catalog) {
Ok(c) => c,
Err(e) => {
log::warn!("upload: opening the catalog to record {}: {e}", entry.name);
return;
}
};
let Some(image) = crate::library::image_at(&catalog, root, path.as_str()) else {
return;
};
let listed = match backend.list(&entry.remote_folder(root), None).await {
Ok(entries) => entries.into_iter().find(|e| e.path == path),
Err(e) => {
log::warn!("upload: listing {} after sending it: {e}", entry.name);
None
}
};
let Some(listed) = listed else {
log::warn!("upload: {} is not listed where it was sent", entry.name);
return;
};
if let Err(e) = crate::library::record_uploaded(&catalog, image, &listed) {
log::warn!("upload: recording {}: {e}", entry.name);
}
let dr_sync::RemoteId::Stable(file_id) = listed.id else {
return;
};
if thumbnails.is_empty() {
return;
}
match dr_thumbs::ThumbStore::open(&library.thumbs) {
Ok(mut store) => {
for (size, thumb) in &thumbnails {
crate::library::store_thumbnail(&mut store, file_id, *size, thumb);
}
log::info!(
"upload: {} thumbnail(s) of {} stored under file {file_id}",
thumbnails.len(),
entry.name
);
}
Err(e) => log::warn!("upload: opening the thumbnail store: {e}"),
}
}
/// Held by whichever drain is sending the outbox. See [`spawn_upload`].
static OUTBOX_DRAIN: std::sync::Mutex<()> = std::sync::Mutex::new(());
/// Drain the outbox to the server. /// Drain the outbox to the server.
/// ///
/// Its own thread with its own runtime, like every other network path here — /// Its own thread with its own runtime, like every other network path here —
@@ -416,6 +660,16 @@ pub fn spawn_upload(
let (tx, rx) = std::sync::mpsc::channel(); let (tx, rx) = std::sync::mpsc::channel();
executors::spawn(Executor::Network, "upload", move || { executors::spawn(Executor::Network, "upload", move || {
// One drain at a time. Three things start one — a sync pass, an
// export, a finished merge — and they used to run side by side over
// the same queue: each read the same 800 MB composite into memory and
// sent it, and the one that came second found its record cleared
// under it and reported the file missing. Waiting here costs nothing:
// the drain that holds the lock is already sending everything queued,
// and the one that waited finds the queue empty and finishes.
let _draining = OUTBOX_DRAIN
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
let rt = match crate::net_runtime::build() { let rt = match crate::net_runtime::build() {
Ok(rt) => rt, Ok(rt) => rt,
Err(e) => { Err(e) => {
@@ -423,6 +677,7 @@ pub fn spawn_upload(
uploaded: 0, uploaded: 0,
remaining: pending_count(&outbox), remaining: pending_count(&outbox),
error: Some(e.to_string()), error: Some(e.to_string()),
landed: 0,
}); });
return; return;
} }
@@ -436,14 +691,17 @@ pub fn spawn_upload(
uploaded: 0, uploaded: 0,
remaining: pending_count(&outbox), remaining: pending_count(&outbox),
error: Some(e.to_string()), error: Some(e.to_string()),
landed: 0,
}); });
return; return;
} }
}; };
let library = LibraryFiles::of(&conn.account);
let queue = pending(&outbox); let queue = pending(&outbox);
let total = queue.len(); let total = queue.len();
let mut uploaded = 0; let mut uploaded = 0;
let mut landed = 0;
let mut error = None; let mut error = None;
for (i, entry) in queue.iter().enumerate() { for (i, entry) in queue.iter().enumerate() {
@@ -453,30 +711,16 @@ pub fn spawn_upload(
i + 1 i + 1
))); )));
let Ok(bytes) = std::fs::read(&entry.local) else { match send(&*backend, &library, &root, entry).await {
// The payload vanished under us. Drop the record too; Ok(Sent::Gone) => {}
// retrying forever against a file that is gone helps Ok(Sent::Uploaded) => {
// nobody.
clear(entry);
continue;
};
// The folder may not exist — this is the first export into it
// — and `create_dir` treats "already there" as success, so it
// is unconditional rather than guarded by a check that would
// cost a request every time.
if let Err(e) = backend.create_dir(&entry.remote_folder(&root)).await {
error = Some(e.to_string());
break;
}
match backend.put(&entry.remote_path(&root), bytes, None).await {
Ok(_) => {
clear(entry);
uploaded += 1; uploaded += 1;
if !entry.account {
landed += 1;
}
} }
Err(e) => { Err(e) => {
error = Some(e.to_string()); error = Some(e);
break; break;
} }
} }
@@ -486,6 +730,7 @@ pub fn spawn_upload(
uploaded, uploaded,
remaining: pending_count(&outbox), remaining: pending_count(&outbox),
error, error,
landed,
}); });
}); });
}); });
@@ -1282,6 +1527,328 @@ fn stop_timer(slot: &Rc<std::cell::RefCell<Option<slint::Timer>>>) {
mod tests { mod tests {
use super::*; use super::*;
/// A server that names a file when it is given one, as Nextcloud does:
/// the id exists only once the upload has landed, and a listing is the
/// way to learn it.
#[derive(Default)]
struct Assigning {
files: std::sync::Mutex<std::collections::BTreeMap<String, (u64, Vec<u8>)>>,
lists: std::sync::atomic::AtomicUsize,
caps: std::sync::OnceLock<dr_sync::Capabilities>,
}
impl Assigning {
fn listing(&self, dir: &str) -> Vec<dr_sync::RemoteEntry> {
self.files
.lock()
.unwrap()
.iter()
.filter(|(p, _)| p.rsplit_once('/').map_or("", |(d, _)| d) == dir)
.map(|(p, (id, body))| dr_sync::RemoteEntry {
id: dr_sync::RemoteId::Stable(*id),
path: RemotePath::new(p.clone()),
kind: dr_sync::EntryKind::File,
validator: dr_sync::Validator::new(format!("etag-{id}")),
size: body.len() as u64,
modified: None,
has_preview: false,
materialised: true,
})
.collect()
}
}
#[async_trait::async_trait]
impl dr_sync::RemoteBackend for Assigning {
fn capabilities(&self) -> &dr_sync::Capabilities {
self.caps.get_or_init(dr_sync::Capabilities::minimal)
}
fn name(&self) -> &str {
"assigning"
}
async fn list(
&self,
dir: &RemotePath,
_since: Option<&dr_sync::Validator>,
) -> Result<Vec<dr_sync::RemoteEntry>, dr_sync::RemoteError> {
self.lists.fetch_add(1, Ordering::SeqCst);
Ok(self.listing(dir.as_str()))
}
async fn dir_validator(
&self,
_dir: &RemotePath,
) -> Result<dr_sync::Validator, dr_sync::RemoteError> {
Err(dr_sync::RemoteError::Unsupported("test"))
}
async fn delta(
&self,
_c: &dr_sync::Cursor,
) -> Result<(Vec<dr_sync::RemoteChange>, dr_sync::Cursor), dr_sync::RemoteError> {
Err(dr_sync::RemoteError::Unsupported("test"))
}
async fn get(
&self,
_id: &dr_sync::RemoteId,
_r: Option<std::ops::Range<u64>>,
) -> Result<Vec<u8>, dr_sync::RemoteError> {
Err(dr_sync::RemoteError::Unsupported("test"))
}
async fn put(
&self,
path: &RemotePath,
body: Vec<u8>,
_pc: Option<dr_sync::Precondition>,
) -> Result<dr_sync::Validator, dr_sync::RemoteError> {
let mut files = self.files.lock().unwrap();
let id = files
.get(path.as_str())
.map(|(id, _)| *id)
.unwrap_or(40_000 + files.len() as u64);
files.insert(path.as_str().to_string(), (id, body));
Ok(dr_sync::Validator::new(format!("etag-{id}")))
}
async fn delete(
&self,
_id: &dr_sync::RemoteId,
_pc: Option<dr_sync::Precondition>,
) -> Result<(), dr_sync::RemoteError> {
Ok(())
}
async fn move_to(
&self,
_f: &dr_sync::RemoteId,
_t: &RemotePath,
) -> Result<(), dr_sync::RemoteError> {
Ok(())
}
async fn create_dir(&self, _p: &RemotePath) -> Result<(), dr_sync::RemoteError> {
Ok(())
}
}
/// A library of one sweep, its catalog on disk, a store beside it, and
/// an outbox holding the composite the merge staged: the row written
/// ahead of the scan, the payload, its record, and its thumbnails.
struct Staged {
dir: PathBuf,
library: LibraryFiles,
entry: Pending,
image: i64,
}
fn staged(name: &str, root: &str) -> Staged {
let dir =
std::env::temp_dir().join(format!("dr-composite-upload-{name}-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(dir.join("outbox")).unwrap();
let library = LibraryFiles {
catalog: dir.join("catalog.sqlite"),
thumbs: dir.join("thumbs"),
};
let at = |n: &str| {
if root.is_empty() {
format!("Alps/{n}")
} else {
format!("{root}/Alps/{n}")
}
};
let catalog = dr_catalog::Catalog::open(&library.catalog).unwrap();
crate::library::test_support::scanned_listing(
&catalog,
root,
vec![crate::library::test_support::entry(
&at("_MG_8320.CR2"),
1,
30,
)],
);
let image = crate::library::catalogue_composite(
&catalog,
root,
&crate::library::CompositeRow {
source_ref: staged_remote_path(root, "Alps", "_MG_8320-pano.dng")
.as_str()
.to_string(),
width: 4000,
height: 1000,
captured_at: Some(1_000),
captured_offset: None,
camera: None,
lens: None,
iso: None,
file_size: 5,
sources: vec![at("_MG_8320.CR2")],
},
)
.unwrap();
// The scan that ran while the upload was still going: the folder is
// listed without the composite, and the row is still there after it.
crate::library::test_support::scanned_listing(
&catalog,
root,
vec![crate::library::test_support::entry(
&at("_MG_8320.CR2"),
1,
30,
)],
);
let local = dir.join("outbox").join("_MG_8320-pano.dng");
std::fs::write(&local, b"pano!").unwrap();
std::fs::write(destination_record(&local), "Alps\n_MG_8320-pano.dng\n").unwrap();
let jpeg = dr_thumbs::encode_rgba(4, 1, &[200u8; 16]).unwrap();
write_thumbnails(
&local,
&[
(
dr_thumbs::ThumbSize::Grid,
dr_thumbs::Thumbnail {
width: 4,
height: 1,
bytes: jpeg.clone(),
},
),
(
dr_thumbs::ThumbSize::Large,
dr_thumbs::Thumbnail {
width: 4,
height: 1,
bytes: jpeg,
},
),
],
)
.unwrap();
let entry = pending(&dir.join("outbox")).pop().unwrap();
Staged {
dir,
library,
entry,
image,
}
}
fn file_id_of(library: &LibraryFiles, image: i64) -> Option<i64> {
dr_catalog::Catalog::open(&library.catalog)
.unwrap()
.connection()
.query_row(
"SELECT file_id FROM remote WHERE image_id = ?1",
[image],
|r| r.get(0),
)
.ok()
}
#[tokio::test]
async fn an_upload_that_lands_after_the_scan_gives_the_row_its_server_id() {
let s = staged("nextcloud", "PhotosRaw");
let server = Assigning::default();
assert_eq!(file_id_of(&s.library, s.image), None);
let sent = send(&server, &s.library, "PhotosRaw", &s.entry)
.await
.unwrap();
assert_eq!(sent, Sent::Uploaded);
// The id the server assigned, on the row the merge wrote, and the
// merge's thumbnails in the store under it.
let listed = server.listing("PhotosRaw/Alps");
let dr_sync::RemoteId::Stable(id) = listed[0].id else {
panic!("the server names its files");
};
assert_eq!(file_id_of(&s.library, s.image), Some(id as i64));
let store = dr_thumbs::ThumbStore::open(&s.library.thumbs).unwrap();
assert!(store.contains(id, dr_thumbs::ThumbSize::Grid));
assert!(store.contains(id, dr_thumbs::ThumbSize::Large));
// Nothing left in the outbox: not the payload, not its records.
assert_eq!(
std::fs::read_dir(s.dir.join("outbox")).unwrap().count(),
0,
"the outbox is empty"
);
// The scan that follows the upload finds the row already there.
let catalog = dr_catalog::Catalog::open(&s.library.catalog).unwrap();
crate::library::test_support::scanned_listing(&catalog, "PhotosRaw", listed);
let n: i64 = catalog
.connection()
.query_row(
"SELECT count(*) FROM images WHERE source_ref LIKE '%pano%'",
[],
|r| r.get(0),
)
.unwrap();
assert_eq!(n, 1);
let _ = std::fs::remove_dir_all(&s.dir);
}
#[tokio::test]
async fn a_folder_library_learns_the_id_the_folder_gives_its_path() {
let s = staged("folder", "");
let library_dir = s.dir.join("library");
std::fs::create_dir_all(library_dir.join("Alps")).unwrap();
let folder = dr_sync_folder::FolderBackend::new(&library_dir).unwrap();
send(&folder, &s.library, "", &s.entry).await.unwrap();
assert_eq!(
std::fs::read(library_dir.join("Alps/_MG_8320-pano.dng")).unwrap(),
b"pano!"
);
use dr_sync::RemoteBackend as _;
let listed = folder
.list(&RemotePath::new("Alps"), None)
.await
.unwrap()
.into_iter()
.find(|e| e.path.name() == "_MG_8320-pano.dng")
.unwrap();
let dr_sync::RemoteId::Stable(id) = listed.id else {
panic!("a folder names a file by its path");
};
assert_eq!(file_id_of(&s.library, s.image), Some(id as i64));
let store = dr_thumbs::ThumbStore::open(&s.library.thumbs).unwrap();
assert!(store.contains(id, dr_thumbs::ThumbSize::Grid));
let _ = std::fs::remove_dir_all(&s.dir);
}
#[tokio::test]
async fn an_ordinary_export_costs_no_listing() {
let s = staged("export", "PhotosRaw");
let other = s.dir.join("outbox").join("print.jpg");
std::fs::write(&other, b"jpeg").unwrap();
std::fs::write(destination_record(&other), "Alps\nprint.jpg\n").unwrap();
let entry = pending(&s.dir.join("outbox"))
.into_iter()
.find(|p| p.name == "print.jpg")
.unwrap();
let server = Assigning::default();
send(&server, &s.library, "PhotosRaw", &entry)
.await
.unwrap();
assert_eq!(server.lists.load(Ordering::SeqCst), 0);
let _ = std::fs::remove_dir_all(&s.dir);
}
#[test]
fn thumbnails_survive_the_outbox_and_a_damaged_record_is_nothing() {
let dir = tmp();
let payload = dir.join("x.dng");
let t = dr_thumbs::Thumbnail {
width: 3,
height: 2,
bytes: vec![1, 2, 3, 4, 5],
};
write_thumbnails(&payload, &[(dr_thumbs::ThumbSize::Large, t.clone())]).unwrap();
assert_eq!(
take_thumbnails(&payload),
vec![(dr_thumbs::ThumbSize::Large, t)]
);
assert!(!thumbnails_record(&payload).exists(), "taken, not copied");
std::fs::write(thumbnails_record(&payload), b"darkroom-thumbs 1\n\x00\x01").unwrap();
assert!(take_thumbnails(&payload).is_empty());
}
fn encoded(name: &str, bytes: &[u8]) -> Encoded { fn encoded(name: &str, bytes: &[u8]) -> Encoded {
Encoded { Encoded {
name: name.to_string(), name: name.to_string(),
+23 -24
View File
@@ -59,6 +59,7 @@ pub mod manual;
mod masks_ui; mod masks_ui;
pub mod memory; pub mod memory;
pub mod merge; pub mod merge;
mod merge_thumbs;
mod merge_ui; mod merge_ui;
mod net_runtime; mod net_runtime;
mod peaking; mod peaking;
@@ -731,7 +732,7 @@ fn batch_request(
/// the first is what makes an upload feel immediate, and the second is what /// the first is what makes an upload feel immediate, and the second is what
/// eventually delivers the exports made while the train was in a tunnel. /// eventually delivers the exports made while the train was in a tunnel.
/// Running it twice over an empty outbox costs a directory listing. /// Running it twice over an empty outbox costs a directory listing.
fn drain_outbox(library: &Rc<library_ui::LibraryController>) { fn drain_outbox(library: &Rc<library_ui::LibraryController>, window: slint::Weak<AppWindow>) {
// Offline is not a failure worth reporting here — the entries stay // Offline is not a failure worth reporting here — the entries stay
// staged and the next pass takes them. // staged and the next pass takes them.
if library.is_offline() { if library.is_offline() {
@@ -747,23 +748,7 @@ fn drain_outbox(library: &Rc<library_ui::LibraryController>) {
let root = conn.account.root.clone(); let root = conn.account.root.clone();
let rx = export::spawn_upload(conn, root, outbox); let rx = export::spawn_upload(conn, root, outbox);
executors::spawn(executors::Executor::Io, "upload-log", move || { export::watch_upload(rx, window);
while let Ok(msg) = rx.recv() {
match msg {
export::UploadMessage::Status(s) => log::info!("export: {s}"),
export::UploadMessage::Finished {
uploaded,
remaining,
error,
} => {
log::info!("export: {uploaded} uploaded, {remaining} still queued");
if let Some(e) = error {
log::warn!("export upload stopped: {e}");
}
}
}
}
});
} }
/// What the export button should say, given where an export would go. /// What the export button should say, given where an export would go.
@@ -1901,17 +1886,31 @@ fn wire_import_and_merge(
move || library_for_sources.export_sources(&collections_for_sources.selected()), move || library_for_sources.export_sources(&collections_for_sources.selected()),
move || { move || {
let conn = library_for_context.session()?; let conn = library_for_context.session()?;
let catalog = library_for_context.catalog();
let root = conn.account.root.clone();
Some(merge_ui::Context { Some(merge_ui::Context {
outbox: export::outbox_dir(&conn.account), outbox: export::outbox_dir(&conn.account),
conn, conn,
names_in: Box::new(move |folder| {
catalog
.borrow()
.as_ref()
.map(|c| library::names_in_folder(c, &root, folder))
.unwrap_or_default()
}),
}) })
}, },
move |w| { move |w, placed| {
// Staged beside its sources: upload it now rather than // TRACES: FR-MRG-6
// on the next sync pass, then look for it, exactly as an // In the grid now, from what the merge knows: a scan
// import does. // started here raced the upload and did not find it.
drain_outbox(&library_for_done); if let Some(placed) = placed {
w.global::<Library>().invoke_library_rescan(); library_ui::catalogue_composite(w, &library_for_done, placed);
}
// Upload it now rather than on the next sync pass; the
// scan that follows the upload gives the row the file id
// the server assigned.
drain_outbox(&library_for_done, w.as_weak());
}, },
); );
} }
+185 -2
View File
@@ -39,7 +39,7 @@ pub(super) fn uncollapsed(image: &str) -> String {
} }
/// One grid cell's data, read from the catalog. /// One grid cell's data, read from the catalog.
#[derive(Debug, Clone, PartialEq, Eq)] #[derive(Debug, Clone, PartialEq)]
pub struct LibraryCell { pub struct LibraryCell {
pub image_id: i64, pub image_id: i64,
pub name: String, pub name: String,
@@ -53,6 +53,10 @@ pub struct LibraryCell {
pub metadata_state: u8, pub metadata_state: u8,
/// UTC seconds, once EXIF has been read. /// UTC seconds, once EXIF has been read.
pub captured_at: Option<i64>, pub captured_at: Option<i64>,
/// TRACES: FR-MRG-6
/// Width over height, where the catalog knows both: what decides whether
/// the cell spans columns. Read with the window, never per cell.
pub aspect: Option<f32>,
} }
/// Read a window of cells out of the catalog. /// Read a window of cells out of the catalog.
@@ -346,6 +350,94 @@ pub fn ordinal_of_path(
} }
} }
/// TRACES: FR-MRG-6
/// Which rows of the catalog are wide enough to span columns, spelled as
/// [`crate::library_ui`]'s layout counts them: `w / h >= 1.9`, in integers
/// so the index below can be matched term for term. Change one, change the
/// other — `the_wide_predicate_is_the_layouts` holds them together.
pub(super) const WIDE: &str = "w * 10 >= h * 19";
/// Make sure the partial index of wide photographs exists.
///
/// Created on first use rather than by a migration: it holds only the
/// panoramas, it changes nothing an older build reads, and a schema bump
/// would make every older peer refuse this catalog's snapshot. Keyed on the
/// grid's ordering, so the planner can answer "is there any?" from it alone.
fn ensure_wide_index(catalog: &Catalog) -> Result<(), dr_catalog::CatalogError> {
catalog.connection().execute_batch(&format!(
"CREATE INDEX IF NOT EXISTS images_wide
ON images(captured_at, source_ref) WHERE {WIDE}"
))?;
Ok(())
}
/// TRACES: FR-MRG-6
/// The photographs in the grid's current list that are wide enough to span
/// columns: each one's ordinal and aspect, in order.
///
/// What the layout places them from. Answered from the small side first: a
/// library with no panorama in it — the reference library, and every one
/// until a merge writes a composite — is told so by one probe of the
/// partial index, and nothing else runs. Otherwise one pass over the list,
/// numbering it through the ordering the cells are read with, exactly as
/// [`ordinal_of_path`] does and for its reason; it runs when what the grid
/// lists changes, not per scroll.
pub fn wide_ordinals(
catalog: &Catalog,
scope: Option<Scope>,
filter: &RatingFilter,
trash: bool,
) -> Result<Vec<(usize, f32)>, dr_catalog::CatalogError> {
ensure_wide_index(catalog)?;
let any: bool = catalog.connection().query_row(
&format!("SELECT EXISTS(SELECT 1 FROM images WHERE {WIDE})"),
[],
|r| r.get(0),
)?;
if !any {
return Ok(Vec::new());
}
let (inner, params) = if trash {
(
format!(
"SELECT i.w AS w, i.h AS h,
row_number() OVER ({TRASH_ORDER}) - 1 AS ord
FROM images i
WHERE {TRASHED}"
),
Vec::new(),
)
} else {
let (clause, scope_params) = scope_clause(catalog, scope)?;
let rated = filter.sql();
let folded = uncollapsed("i");
let (order, mut params) = grid_order_for(catalog, scope);
// The window's parameters first — see `ordinal_of_path`.
params.extend(scope_params);
(
format!(
"SELECT i.w AS w, i.h AS h,
row_number() OVER ({order}) - 1 AS ord
FROM images i
WHERE {VISIBLE}{rated}{folded}{clause}"
),
params,
)
};
let mut stmt = catalog.connection().prepare(&format!(
"SELECT ord, w, h FROM ({inner}) WHERE {WIDE} ORDER BY ord"
))?;
let rows = stmt
.query_map(rusqlite::params_from_iter(params.iter()), |r| {
Ok((
r.get::<_, i64>(0)?.max(0) as usize,
r.get::<_, i64>(1)? as f32 / r.get::<_, i64>(2)?.max(1) as f32,
))
})?
.collect::<Result<Vec<_>, _>>()?;
Ok(rows)
}
/// The columns every windowed read selects, in the order [`row_to_cell`] reads /// The columns every windowed read selects, in the order [`row_to_cell`] reads
/// them. /// them.
/// ///
@@ -353,7 +445,7 @@ pub fn ordinal_of_path(
/// the one column that is *not* here stays conspicuous. See /// the one column that is *not* here stays conspicuous. See
/// [`attach_file_ids`] for why the server's file id is fetched separately. /// [`attach_file_ids`] for why the server's file id is fetched separately.
pub(super) const CELL_COLUMNS: &str = pub(super) const CELL_COLUMNS: &str =
"i.id, i.source_ref, i.file_size, i.metadata_state, i.captured_at"; "i.id, i.source_ref, i.file_size, i.metadata_state, i.captured_at, i.w, i.h";
/// Shared row mapping, so the scoped and unscoped queries cannot drift. /// Shared row mapping, so the scoped and unscoped queries cannot drift.
/// ///
@@ -368,6 +460,10 @@ pub(super) fn row_to_cell(r: &rusqlite::Row) -> rusqlite::Result<LibraryCell> {
size: r.get::<_, Option<i64>>(2)?.unwrap_or(0) as u64, size: r.get::<_, Option<i64>>(2)?.unwrap_or(0) as u64,
metadata_state: r.get::<_, i64>(3)? as u8, metadata_state: r.get::<_, i64>(3)? as u8,
captured_at: r.get(4)?, captured_at: r.get(4)?,
aspect: match (r.get::<_, Option<i64>>(5)?, r.get::<_, Option<i64>>(6)?) {
(Some(w), Some(h)) if w > 0 && h > 0 => Some(w as f32 / h as f32),
_ => None,
},
}) })
} }
@@ -944,6 +1040,93 @@ mod tests {
// --- the trash view (FR-CAT-15) --------------------------------------- // --- the trash view (FR-CAT-15) ---------------------------------------
/// Mark one image trashed, at a given instant. /// Mark one image trashed, at a given instant.
fn shaped(catalog: &Catalog, path: &str, w: i64, h: i64) {
let n = catalog
.connection()
.execute(
"UPDATE images SET w = ?2, h = ?3 WHERE source_ref = ?1",
rusqlite::params![path, w, h],
)
.unwrap();
assert_eq!(n, 1, "fixture should have shaped exactly {path}");
}
#[test]
fn the_wide_photographs_are_found_at_their_ordinals() {
let catalog = with_images(20);
assert!(
wide_ordinals(&catalog, None, &RatingFilter::default(), false)
.unwrap()
.is_empty()
);
shaped(&catalog, "PhotosRaw/img005.CR2", 4000, 1000);
shaped(&catalog, "PhotosRaw/img012.CR2", 3000, 1400);
shaped(&catalog, "PhotosRaw/img013.CR2", 3000, 2000);
let wide = wide_ordinals(&catalog, None, &RatingFilter::default(), false).unwrap();
assert_eq!(
wide.iter().map(|(o, _)| *o).collect::<Vec<_>>(),
vec![5, 12],
"{wide:?}"
);
assert!((wide[0].1 - 4.0).abs() < 1e-6);
// The ordinals are the grid's: a frame trashed before them moves
// them up, and in the trash only the trash counts.
trash_at(&catalog, "PhotosRaw/img003.CR2", 1);
trash_at(&catalog, "PhotosRaw/img012.CR2", 2);
let wide = wide_ordinals(&catalog, None, &RatingFilter::default(), false).unwrap();
assert_eq!(wide.iter().map(|(o, _)| *o).collect::<Vec<_>>(), vec![4]);
let trashed = wide_ordinals(&catalog, None, &RatingFilter::default(), true).unwrap();
// Newest deletion first: img012 then img003.
assert_eq!(trashed.iter().map(|(o, _)| *o).collect::<Vec<_>>(), vec![0]);
}
#[test]
fn a_library_with_no_panorama_is_told_so_by_the_index() {
let catalog = with_images(20);
wide_ordinals(&catalog, None, &RatingFilter::default(), false).unwrap();
let plan: Vec<String> = catalog
.connection()
.prepare(&format!(
"EXPLAIN QUERY PLAN SELECT EXISTS(SELECT 1 FROM images WHERE {WIDE})"
))
.unwrap()
.query_map([], |r| r.get::<_, String>(3))
.unwrap()
.flatten()
.collect();
let plan = plan.join(" | ");
assert!(plan.contains("images_wide"), "{plan}");
}
#[test]
fn the_wide_predicate_is_the_layouts() {
use crate::library_ui::layout::natural_span;
let catalog = with_images(1);
for (w, h) in [
(19, 10),
(189, 100),
(190, 100),
(191, 100),
(6000, 4000),
(4000, 6000),
(22_993, 5_980),
(1_899, 1_000),
(1_900, 1_000),
] {
let sql: bool = catalog
.connection()
.query_row(
&format!("SELECT {}", WIDE.replace('w', "?1").replace('h', "?2")),
rusqlite::params![w, h],
|r| r.get(0),
)
.unwrap();
let rust = natural_span(Some(w as f32 / h as f32)) > 1;
assert_eq!(sql, rust, "{w}×{h}");
}
}
fn trash_at(catalog: &Catalog, path: &str, when: i64) { fn trash_at(catalog: &Catalog, path: &str, when: i64) {
let n = catalog let n = catalog
.connection() .connection()
+461
View File
@@ -0,0 +1,461 @@
//! TRACES: FR-MRG-6
//! A merged panorama in the catalog the moment it is written, rather than
//! whenever a scan next happens to find it.
//!
//! # Why the merge catalogues its own file
//!
//! The composite reaches the library through the outbox, like an export,
//! and the grid used to learn of it the way it learns of anything: a rescan,
//! fired as the merge finished. That scan raced the upload it followed. On a
//! folder library the copy of an 800 MB file was still running when the
//! folder was listed; on Nextcloud the upload takes minutes. Either way the
//! listing did not have the file, the folder's validator was recorded as it
//! stood, and nothing looked again until the next sync pass — the panorama
//! the photographer had just watched being made was not in the grid.
//!
//! So the job hands the library what it knows — the name the file will have,
//! the size of the picture, when it was taken — and the row is written at
//! once, in one transaction. The upload later supplies what only the server
//! knows, its file id, through [`record_uploaded`]; the scan that follows
//! finds a row already there and updates it in place, on the same
//! `(root_id, source_ref)` key it would have inserted under.
use dr_catalog::Catalog;
use super::scan::now_secs;
/// What the catalog is told about a composite before any scan has seen it.
#[derive(Debug, Clone, PartialEq)]
pub struct CompositeRow {
/// Where it will be in the library, spelled as the scan spells it.
pub source_ref: String,
/// The picture it opens on — the crop, where the merge cropped — upright.
pub width: u32,
pub height: u32,
/// When it was taken: the middle of its sweep, as the DNG says. `None`
/// where no frame carried a time, and then the earliest of `sources` as
/// the catalog has them.
pub captured_at: Option<i64>,
pub captured_offset: Option<i32>,
pub camera: Option<String>,
pub lens: Option<String>,
pub iso: Option<u32>,
pub file_size: u64,
/// The frames it was merged from, as source refs.
pub sources: Vec<String>,
}
/// Write the composite's row, and return its image id.
///
/// One transaction: the root, the capture time the sources lend it if it has
/// none of its own, and the row. At `metadata_state = 2`, since everything a
/// header read would add is already here — the sweep has no reason to fetch
/// the tail of an 800 MB file for a date the merge wrote into it.
///
/// A row already under the name is brought up to date rather than
/// duplicated. The merge picks a name the catalog does not hold, so that is
/// a safety net: the file at that name is the one just written.
pub fn catalogue_composite(
catalog: &Catalog,
root: &str,
row: &CompositeRow,
) -> Result<i64, dr_catalog::CatalogError> {
let conn = catalog.connection();
let tx = conn.unchecked_transaction()?;
tx.execute(
"INSERT INTO roots(kind, label, last_seen) VALUES ('remote', ?1, ?2)
ON CONFLICT DO NOTHING",
rusqlite::params![root, now_secs()],
)?;
let root_id: i64 = tx.query_row(
"SELECT id FROM roots WHERE label = ?1 AND kind = 'remote'",
[root],
|r| r.get(0),
)?;
// The folder, if a scan has recorded it; the sources are in it, so it
// almost always has. `None` is what the scan itself writes for a file at
// the root, and the next scan fills it either way.
let folder_id: Option<i64> = match row.source_ref.rsplit_once('/') {
Some((parent, _)) => tx
.query_row(
"SELECT id FROM folders WHERE root_id = ?1 AND path = ?2",
rusqlite::params![root_id, parent],
|r| r.get(0),
)
.ok(),
None => None,
};
// The sources' earliest, where the composite carries no time. One
// aggregate over the handful of frames, by their key.
let captured_at = match row.captured_at {
Some(t) => Some(t),
None if row.sources.is_empty() => None,
None => {
let placeholders = std::iter::repeat_n("?", row.sources.len())
.collect::<Vec<_>>()
.join(",");
let mut params: Vec<rusqlite::types::Value> =
vec![rusqlite::types::Value::Integer(root_id)];
params.extend(
row.sources
.iter()
.map(|s| rusqlite::types::Value::Text(s.clone())),
);
tx.query_row(
&format!(
"SELECT min(captured_at) FROM images
WHERE root_id = ? AND source_ref IN ({placeholders})"
),
rusqlite::params_from_iter(params.iter()),
|r| r.get::<_, Option<i64>>(0),
)?
}
};
let format = row
.source_ref
.rsplit_once('.')
.map(|(_, e)| e.to_ascii_lowercase());
tx.execute(
"INSERT INTO images(root_id, folder_id, source_ref, format, w, h,
captured_at, captured_offset, camera, lens, iso,
file_size, availability, metadata_state, added_at)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, 0, 2, ?13)
ON CONFLICT(root_id, source_ref) DO UPDATE SET
folder_id = coalesce(excluded.folder_id, images.folder_id),
w = excluded.w,
h = excluded.h,
captured_at = coalesce(excluded.captured_at, images.captured_at),
captured_offset = coalesce(excluded.captured_offset, images.captured_offset),
camera = coalesce(excluded.camera, images.camera),
lens = coalesce(excluded.lens, images.lens),
iso = coalesce(excluded.iso, images.iso),
file_size = excluded.file_size,
metadata_state = 2",
rusqlite::params![
root_id,
folder_id,
row.source_ref,
format,
row.width as i64,
row.height as i64,
captured_at,
row.captured_offset,
row.camera,
row.lens,
row.iso.map(i64::from),
row.file_size as i64,
now_secs(),
],
)?;
let image_id: i64 = tx.query_row(
"SELECT id FROM images WHERE root_id = ?1 AND source_ref = ?2",
rusqlite::params![root_id, row.source_ref],
|r| r.get(0),
)?;
tx.commit()?;
Ok(image_id)
}
/// The image a library path names, if the catalog has it. What the upload
/// asks before it spends a listing on a file: a file the catalog has no row
/// for is the scan's to find, and only a row written ahead of the scan is
/// waiting for its server identity.
pub fn image_at(catalog: &Catalog, root: &str, source_ref: &str) -> Option<i64> {
catalog
.connection()
.query_row(
"SELECT i.id FROM images i JOIN roots r ON r.id = i.root_id
WHERE r.label = ?1 AND r.kind = 'remote' AND i.source_ref = ?2",
rusqlite::params![root, source_ref],
|r| r.get(0),
)
.ok()
}
/// The names the catalog holds directly in `folder` of the library at
/// `root`, whatever their state — what a new file there must not be called.
///
/// A range over the `(root_id, source_ref)` key, `/` to `0` being the byte
/// after it, so a folder of a thousand costs a seek and a thousand index
/// entries, not the library.
pub fn names_in_folder(
catalog: &Catalog,
root: &str,
folder: &str,
) -> std::collections::HashSet<String> {
let folder = folder.trim_matches('/');
let (from, to) = if folder.is_empty() {
(String::new(), "\u{10ffff}".to_string())
} else {
(format!("{folder}/"), format!("{folder}0"))
};
let Ok(mut stmt) = catalog.connection().prepare(
"SELECT i.source_ref FROM images i JOIN roots r ON r.id = i.root_id
WHERE r.label = ?1 AND r.kind = 'remote'
AND i.source_ref >= ?2 AND i.source_ref < ?3",
) else {
return Default::default();
};
let Ok(rows) = stmt.query_map(rusqlite::params![root, from, to], |r| r.get::<_, String>(0))
else {
return Default::default();
};
rows.flatten()
.filter_map(|p| {
let name = p.get(from.len()..)?;
(!name.contains('/')).then(|| name.to_string())
})
.collect()
}
/// Give a catalogued file the identity the server assigned it on upload:
/// its file id, which the thumbnail store keys on, and its validator.
///
/// The same row the scan's `remote` upsert writes, so whichever of the two
/// runs second agrees with the first.
pub fn record_uploaded(
catalog: &Catalog,
image_id: i64,
entry: &dr_sync::RemoteEntry,
) -> Result<(), dr_catalog::CatalogError> {
let conn = catalog.connection();
let tx = conn.unchecked_transaction()?;
if let dr_sync::RemoteId::Stable(file_id) = entry.id {
tx.execute(
"INSERT INTO remote(image_id, file_id, etag, remote_path)
VALUES (?1, ?2, ?3, ?4)
ON CONFLICT(image_id) DO UPDATE SET
file_id = excluded.file_id, etag = excluded.etag,
remote_path = excluded.remote_path",
rusqlite::params![
image_id,
file_id as i64,
entry.validator.as_str(),
entry.path.as_str()
],
)?;
}
tx.execute(
"UPDATE images SET file_size = ?2 WHERE id = ?1 AND file_size IS NOT ?2",
rusqlite::params![image_id, entry.size as i64],
)?;
tx.commit()?;
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
use crate::library::read_cells;
use crate::library::scan::persist;
use crate::library::test_support::entry;
use dr_sync::RemotePath;
/// Twelve frames of a sweep a second apart, scanned and dated.
fn a_sweep() -> Catalog {
let catalog = Catalog::in_memory().unwrap();
let images = (0..12)
.map(|i| {
entry(
&format!("Alps/_MG_{:04}.CR2", 8320 + i),
100 + i,
30_000_000,
)
})
.chain(std::iter::once(entry("Alps/later.CR2", 200, 30_000_000)))
.collect();
persist(
&catalog,
"",
&dr_sync::ScanResult {
images,
directories: vec![(RemotePath::new("Alps"), dr_sync::Validator::new("e1"))],
progress: Default::default(),
sidecars: Vec::new(),
},
)
.unwrap();
for i in 0..12 {
catalog
.connection()
.execute(
"UPDATE images SET captured_at = ?2, metadata_state = 2
WHERE source_ref = ?1",
rusqlite::params![format!("Alps/_MG_{:04}.CR2", 8320 + i), 1_000 + i],
)
.unwrap();
}
catalog
.connection()
.execute(
"UPDATE images SET captured_at = 5000 WHERE source_ref = 'Alps/later.CR2'",
[],
)
.unwrap();
catalog
}
fn composite(captured_at: Option<i64>) -> CompositeRow {
CompositeRow {
source_ref: "Alps/_MG_8320-pano.dng".into(),
width: 22_000,
height: 5_600,
captured_at,
captured_offset: None,
camera: Some("Canon EOS 6D".into()),
lens: None,
iso: Some(100),
file_size: 800_000_000,
sources: (0..12)
.map(|i| format!("Alps/_MG_{:04}.CR2", 8320 + i))
.collect(),
}
}
fn names(catalog: &Catalog) -> Vec<String> {
read_cells(catalog, 0, 100)
.unwrap()
.into_iter()
.map(|c| c.name)
.collect()
}
#[test]
fn a_composite_sits_among_its_sources_the_moment_it_is_written() {
let catalog = a_sweep();
catalogue_composite(&catalog, "", &composite(Some(1_005))).unwrap();
let names = names(&catalog);
let at = names.iter().position(|n| n == "_MG_8320-pano.dng").unwrap();
// Dated at the middle of the sweep, so it is among the frames and not
// after the photograph taken an hour later.
assert!(at > 0 && at < 12, "{names:?}");
assert_eq!(names.last().unwrap(), "later.CR2");
}
#[test]
fn a_composite_with_no_time_takes_its_earliest_source() {
let catalog = a_sweep();
let id = catalogue_composite(&catalog, "", &composite(None)).unwrap();
let (t, w, h, state): (Option<i64>, i64, i64, i64) = catalog
.connection()
.query_row(
"SELECT captured_at, w, h, metadata_state FROM images WHERE id = ?1",
[id],
|r| Ok((r.get(0)?, r.get(1)?, r.get(2)?, r.get(3)?)),
)
.unwrap();
assert_eq!(t, Some(1_000));
assert_eq!((w, h, state), (22_000, 5_600, 2));
}
#[test]
fn a_scan_that_ran_before_the_upload_does_not_lose_the_row() {
// Nextcloud: the merge finishes, the row is written, and a sync pass
// lists the folder before the upload has finished — the file is not
// on the server yet.
let catalog = a_sweep();
let id = catalogue_composite(&catalog, "", &composite(Some(1_005))).unwrap();
persist(
&catalog,
"",
&dr_sync::ScanResult {
images: vec![entry("Alps/later.CR2", 200, 30_000_000)],
directories: vec![(RemotePath::new("Alps"), dr_sync::Validator::new("e2"))],
progress: Default::default(),
sidecars: Vec::new(),
},
)
.unwrap();
assert!(names(&catalog).contains(&"_MG_8320-pano.dng".to_string()));
// The upload lands and the server names it; then the scan finds it.
let listed = entry("Alps/_MG_8320-pano.dng", 9_001, 812_345_678);
record_uploaded(&catalog, id, &listed).unwrap();
persist(
&catalog,
"",
&dr_sync::ScanResult {
images: vec![listed],
directories: vec![(RemotePath::new("Alps"), dr_sync::Validator::new("e3"))],
progress: Default::default(),
sidecars: Vec::new(),
},
)
.unwrap();
let rows: Vec<(i64, Option<i64>, i64, Option<i64>)> = catalog
.connection()
.prepare(
"SELECT i.id, r.file_id, i.file_size, i.captured_at FROM images i
LEFT JOIN remote r ON r.image_id = i.id
WHERE i.source_ref = 'Alps/_MG_8320-pano.dng'",
)
.unwrap()
.query_map([], |r| Ok((r.get(0)?, r.get(1)?, r.get(2)?, r.get(3)?)))
.unwrap()
.map(Result::unwrap)
.collect();
assert_eq!(rows, vec![(id, Some(9_001), 812_345_678, Some(1_005))]);
let cell = read_cells(&catalog, 0, 100)
.unwrap()
.into_iter()
.find(|c| c.image_id == id)
.unwrap();
assert_eq!(cell.file_id, Some(9_001));
}
#[test]
fn merging_again_under_the_same_name_updates_the_one_row() {
let catalog = a_sweep();
let first = catalogue_composite(&catalog, "", &composite(Some(1_005))).unwrap();
let mut again = composite(Some(1_006));
again.width = 21_000;
let second = catalogue_composite(&catalog, "", &again).unwrap();
assert_eq!(first, second);
let w: i64 = catalog
.connection()
.query_row("SELECT w FROM images WHERE id = ?1", [first], |r| r.get(0))
.unwrap();
assert_eq!(w, 21_000);
}
#[test]
fn the_names_in_a_folder_are_its_own_and_not_its_subfolders() {
let catalog = a_sweep();
catalogue_composite(&catalog, "", &composite(Some(1_005))).unwrap();
persist(
&catalog,
"",
&dr_sync::ScanResult {
images: vec![
entry("Alps/2019/deep.CR2", 300, 1),
entry("Alpsine/near.CR2", 301, 1),
],
directories: Vec::new(),
progress: Default::default(),
sidecars: Vec::new(),
},
)
.unwrap();
let names = names_in_folder(&catalog, "", "Alps");
assert!(names.contains("_MG_8320-pano.dng"));
assert!(names.contains("_MG_8320.CR2"));
assert!(!names
.iter()
.any(|n| n.contains("deep") || n.contains("near")));
assert_eq!(names.len(), 14);
}
#[test]
fn only_a_row_the_merge_wrote_is_found_for_the_upload() {
let catalog = a_sweep();
assert!(image_at(&catalog, "", "Alps/_MG_8320-pano.dng").is_none());
let id = catalogue_composite(&catalog, "", &composite(Some(1_005))).unwrap();
assert_eq!(image_at(&catalog, "", "Alps/_MG_8320-pano.dng"), Some(id));
assert!(image_at(&catalog, "elsewhere", "Alps/_MG_8320-pano.dng").is_none());
}
}
+17
View File
@@ -28,6 +28,7 @@
//! `library` needs to change. //! `library` needs to change.
mod cells; mod cells;
mod composite;
mod filters; mod filters;
mod paths; mod paths;
mod scan; mod scan;
@@ -39,6 +40,7 @@ mod thumbnails_gen;
mod xmp; mod xmp;
pub use cells::*; pub use cells::*;
pub use composite::*;
pub use filters::*; pub use filters::*;
pub use paths::*; pub use paths::*;
pub use scan::*; pub use scan::*;
@@ -84,6 +86,21 @@ pub(crate) mod test_support {
} }
} }
/// Write a scan's listing of `images` under `root`, as a scan would.
pub fn scanned_listing(catalog: &Catalog, root: &str, images: Vec<RemoteEntry>) {
persist(
catalog,
root,
&dr_sync::ScanResult {
images,
directories: Vec::new(),
progress: Default::default(),
sidecars: Vec::new(),
},
)
.unwrap();
}
/// A catalog with `n` images, ready to file into collections. /// A catalog with `n` images, ready to file into collections.
pub fn with_images(n: usize) -> Catalog { pub fn with_images(n: usize) -> Catalog {
let catalog = Catalog::in_memory().unwrap(); let catalog = Catalog::in_memory().unwrap();
+37 -1
View File
@@ -28,6 +28,23 @@ pub struct MetadataFound {
pub camera: Option<String>, pub camera: Option<String>,
pub lens: Option<String>, pub lens: Option<String>,
pub iso: Option<u32>, pub iso: Option<u32>,
/// TRACES: FR-MRG-6
/// Width and height as the photograph is seen — orientation applied —
/// where the header says: what gives a panorama from elsewhere, a
/// stitch from another program or a phone's sweep, its wide cell.
pub size: Option<(u32, u32)>,
}
impl MetadataFound {
/// The upright size a header describes.
pub fn upright_size(md: &dr_decode::Metadata) -> Option<(u32, u32)> {
let (w, h) = (md.width?, md.height?);
if w == 0 || h == 0 {
return None;
}
let turned = md.orientation.is_some_and(|o| o.quarter_turns % 2 == 1);
Some(if turned { (h, w) } else { (w, h) })
}
} }
/// Write a batch of dates and tell the UI, draining `found`. /// Write a batch of dates and tell the UI, draining `found`.
@@ -140,7 +157,9 @@ pub fn write_metadata(
camera = coalesce(?4, camera), camera = coalesce(?4, camera),
lens = coalesce(?5, lens), lens = coalesce(?5, lens),
iso = coalesce(?6, iso), iso = coalesce(?6, iso),
metadata_state = max(metadata_state, ?7) metadata_state = max(metadata_state, ?7),
w = coalesce(?8, w),
h = coalesce(?9, h)
WHERE id = ?1", WHERE id = ?1",
rusqlite::params![ rusqlite::params![
m.image_id, m.image_id,
@@ -150,6 +169,8 @@ pub fn write_metadata(
m.lens, m.lens,
m.iso, m.iso,
state, state,
m.size.map(|s| s.0),
m.size.map(|s| s.1),
], ],
)?; )?;
if state == 2 { if state == 2 {
@@ -404,6 +425,7 @@ pub fn spawn_sweep(conn: Connection, catalog_path: PathBuf) -> Receiver<SweepMes
camera: None, camera: None,
lens: None, lens: None,
iso: None, iso: None,
size: None,
}), }),
); );
@@ -874,6 +896,20 @@ pub(super) fn thumbnails_outstanding(
mod tests { mod tests {
use super::*; use super::*;
#[test]
fn a_header_gives_the_size_the_photograph_is_seen_at() {
let mut md = dr_decode::Metadata {
width: Some(9000),
height: Some(3000),
..Default::default()
};
assert_eq!(MetadataFound::upright_size(&md), Some((9000, 3000)));
md.orientation = Some(dr_types::Orientation::from_exif(6));
assert_eq!(MetadataFound::upright_size(&md), Some((3000, 9000)));
md.width = None;
assert_eq!(MetadataFound::upright_size(&md), None);
}
#[test] #[test]
fn join_all_preserves_order_regardless_of_completion() { fn join_all_preserves_order_regardless_of_completion() {
// The ordering guarantee is what lets a caller pair results back to // The ordering guarantee is what lets a caller pair results back to
+1
View File
@@ -517,6 +517,7 @@ pub(super) async fn collect_metadata(
captured_at: md.captured_at, captured_at: md.captured_at,
captured_offset: md.captured_offset, captured_offset: md.captured_offset,
camera: camera_label(md.make.as_deref(), md.model.as_deref()), camera: camera_label(md.make.as_deref(), md.model.as_deref()),
size: MetadataFound::upright_size(&md),
lens: md.lens.map(|l| l.trim().to_string()), lens: md.lens.map(|l| l.trim().to_string()),
iso: md.iso, iso: md.iso,
}); });
+22
View File
@@ -113,6 +113,25 @@ pub struct LibraryController {
/// two states apart — without it a carried thumbnail either blocks the /// two states apart — without it a carried thumbnail either blocks the
/// sharper fetch forever or is re-fetched on every scroll. /// sharper fetch forever or is re-fetched on every scroll.
pub(super) thumb_class: RefCell<Vec<Option<dr_thumbs::ThumbSize>>>, pub(super) thumb_class: RefCell<Vec<Option<dr_thumbs::ThumbSize>>>,
/// TRACES: FR-MRG-6
/// Thumbnails a merge made for a composite it has just catalogued, by
/// image, decoded and at each class it made.
///
/// The grid's own path keys the store on the server's file id, and a
/// composite has none until its upload lands — minutes, on Nextcloud. The
/// merge's pixels are drawn from here until then, so the cell is not
/// blank in the meantime; after, the store holds the same ones.
pub(super) merged_thumbs:
RefCell<std::collections::HashMap<i64, Vec<(dr_thumbs::ThumbSize, slint::Image)>>>,
/// TRACES: FR-MRG-6
/// Where each photograph of the current list sits in the grid — see
/// [`super::layout`]. Rebuilt on every window load from `wide` and the
/// column count, which is arithmetic over a handful of panoramas.
pub(super) layout: RefCell<super::layout::Layout>,
/// The wide photographs of the current list, `(ordinal, natural span)`,
/// or `None` when they have to be read again: what the list is changed,
/// or a window found them out of date.
pub(super) wide: RefCell<Option<Vec<(usize, usize)>>>,
/// Where in the catalog the current window starts. Scrubbing moves this. /// Where in the catalog the current window starts. Scrubbing moves this.
pub(super) offset: RefCell<usize>, pub(super) offset: RefCell<usize>,
/// The first visible ordinal, kept so returning from the develop view lands /// The first visible ordinal, kept so returning from the develop view lands
@@ -449,6 +468,9 @@ impl LibraryController {
needs_metadata: RefCell::new(Vec::new()), needs_metadata: RefCell::new(Vec::new()),
captured_at: RefCell::new(Vec::new()), captured_at: RefCell::new(Vec::new()),
thumb_class: RefCell::new(Vec::new()), thumb_class: RefCell::new(Vec::new()),
merged_thumbs: RefCell::new(Default::default()),
layout: RefCell::new(super::layout::Layout::uniform(1, 0)),
wide: RefCell::new(None),
offset: RefCell::new(0), offset: RefCell::new(0),
resume_at: std::cell::Cell::new(0), resume_at: std::cell::Cell::new(0),
roll_open: std::cell::Cell::new(None), roll_open: std::cell::Cell::new(None),
+31 -7
View File
@@ -814,19 +814,20 @@ pub(super) fn refilter(window: &AppWindow, ctl: &Rc<LibraryController>) {
/// lands in: a scrolled window would otherwise show no date at all until /// lands in: a scrolled window would otherwise show no date at all until
/// the next month began. /// the next month began.
pub(super) fn period_headings( pub(super) fn period_headings(
months: impl Iterator<Item = Option<(i64, i64)>>, months: impl Iterator<Item = (Option<(i64, i64)>, usize)>,
offset: usize,
columns: usize, columns: usize,
) -> Vec<String> { ) -> Vec<String> {
let columns = columns.max(1); let columns = columns.max(1);
let mut announced: Option<(i64, i64)> = None; let mut announced: Option<(i64, i64)> = None;
months months
.enumerate() .enumerate()
.map(|(i, month)| { .map(|(i, (month, slot))| {
let Some((y, m)) = month else { let Some((y, m)) = month else {
return String::new(); return String::new();
}; };
let begins_row = (i + offset).is_multiple_of(columns); // By slot, not ordinal: a panorama's cell pushes what follows it
// along (FR-MRG-6).
let begins_row = slot.is_multiple_of(columns);
if i == 0 || (begins_row && announced != Some((y, m))) { if i == 0 || (begins_row && announced != Some((y, m))) {
announced = Some((y, m)); announced = Some((y, m));
format!("{} {y}", month_name(m)) format!("{} {y}", month_name(m))
@@ -863,6 +864,29 @@ fn show_range(window: &AppWindow, ctl: &Rc<LibraryController>) {
mod tests { mod tests {
use super::*; use super::*;
/// Every cell one slot, the window starting at `offset`.
fn at_slots<const N: usize>(
months: [Option<(i64, i64)>; N],
offset: usize,
) -> impl Iterator<Item = (Option<(i64, i64)>, usize)> {
months
.into_iter()
.enumerate()
.map(move |(i, m)| (m, offset + i))
}
#[test]
fn a_month_is_announced_on_the_row_a_panorama_pushed_it_to() {
// Four columns; a panorama spans the first three slots of row two,
// and the frame after it sits in the fourth. February's first frame
// is pushed to the start of row three, and is announced there.
let jan = Some((2026, 1));
let feb = Some((2026, 2));
let cells = [(jan, 0), (jan, 1), (jan, 2), (jan, 4), (jan, 7), (feb, 8)];
let h = period_headings(cells.into_iter(), 4);
assert_eq!(h, ["January 2026", "", "", "", "", "February 2026"]);
}
#[test] #[test]
fn a_month_that_begins_mid_row_is_still_announced() { fn a_month_that_begins_mid_row_is_still_announced() {
// Four columns; August starts in the second column of the second // Four columns; August starts in the second column of the second
@@ -877,7 +901,7 @@ mod tests {
aug, aug, oct, oct, // aug, aug, oct, oct, //
oct, oct,
]; ];
let h = period_headings(months.into_iter(), 0, 4); let h = period_headings(at_slots(months, 0), 4);
assert_eq!(h[0], "March 2024"); assert_eq!(h[0], "March 2024");
assert_eq!(h[4], "", "still March at the top of row two"); assert_eq!(h[4], "", "still March at the top of row two");
assert_eq!(h[5], "", "never mid-row"); assert_eq!(h[5], "", "never mid-row");
@@ -892,12 +916,12 @@ mod tests {
let feb = Some((2026, 2)); let feb = Some((2026, 2));
// Offset 2 into a 4-column grid: the window's first cell is in the // Offset 2 into a 4-column grid: the window's first cell is in the
// third column, and the row after it begins a new month. // third column, and the row after it begins a new month.
let h = period_headings([jan, jan, feb, feb, feb].into_iter(), 2, 4); let h = period_headings(at_slots([jan, jan, feb, feb, feb], 2), 4);
assert_eq!(h[0], "January 2026"); assert_eq!(h[0], "January 2026");
assert_eq!(h[2], "February 2026", "cell 2 begins a row at offset 2"); assert_eq!(h[2], "February 2026", "cell 2 begins a row at offset 2");
assert_eq!(h[3], ""); assert_eq!(h[3], "");
// Undated cells carry nothing and announce nothing. // Undated cells carry nothing and announce nothing.
let h = period_headings([None, jan, None, None, jan].into_iter(), 0, 4); let h = period_headings(at_slots([None, jan, None, None, jan], 0), 4);
assert_eq!(h[0], ""); assert_eq!(h[0], "");
assert_eq!(h[4], "January 2026"); assert_eq!(h[4], "January 2026");
} }
+41 -5
View File
@@ -27,7 +27,7 @@ use super::timeline::{
capture_time_at, catalog_span, note_place, resume_position, wire_timeline, write_place, capture_time_at, catalog_span, note_place, resume_position, wire_timeline, write_place,
zoomed_span, zoomed_span,
}; };
use super::window::{bring_window_to, load_window, mark_open, window_move}; use super::window::{bring_window_to, load_window, mark_open, show_cursor, window_move};
/// Walk the keyboard cursor through the library — the arrow keys. /// Walk the keyboard cursor through the library — the arrow keys.
/// ///
@@ -109,7 +109,7 @@ fn place_cursor(
} }
crate::collections_ui::select_row(window, coll, &ids, offset, row, false, extend); crate::collections_ui::select_row(window, coll, &ids, offset, row, false, extend);
window.global::<Library>().set_library_cursor(at as i32); show_cursor(window, ctl, at);
} }
/// Where a step of `delta` along the roll lands, as a library ordinal, from /// Where a step of `delta` along the roll lands, as a library ordinal, from
@@ -419,6 +419,33 @@ fn wire_grid_cursor_and_zoom(
}); });
} }
// TRACES: FR-MRG-6
// Up and down, by rows: with a panorama's cell in the grid a row is not
// a fixed number of photographs, so the layout says what is under the
// cursor that many rows on, and the step is the distance to it.
{
let weak = window.as_weak();
let ctl = ctl.clone();
let coll = coll_ctl.clone();
window
.global::<Library>()
.on_library_move_cursor_rows(move |rows, extend| {
let Some(w) = weak.upgrade() else { return };
let delta = match coll.cursor() {
Some(from) => {
let to = ctl.layout.borrow().step_rows(from, rows as isize);
to as i32 - from as i32
}
// The first press takes hold of the grid, whichever key.
None => rows,
};
if delta == 0 && coll.cursor().is_some() {
return;
}
move_cursor(&w, &ctl, &coll, delta, extend);
});
}
{ {
let weak = window.as_weak(); let weak = window.as_weak();
let ctl = ctl.clone(); let ctl = ctl.clone();
@@ -635,9 +662,18 @@ fn wire_grid_sync_and_load(window: &AppWindow, ctl: &Rc<LibraryController>) {
let ctl = ctl.clone(); let ctl = ctl.clone();
window window
.global::<Library>() .global::<Library>()
.on_library_scrolled(move |first_visible| { .on_library_scrolled(move |first_slot| {
let Some(w) = weak.upgrade() else { return }; let Some(w) = weak.upgrade() else { return };
let first_visible = first_visible.max(0) as usize; // The grid reports the first slot of its first visible row;
// the photograph there is what everything below means
// (FR-MRG-6).
let first_slot = first_slot.max(0) as usize;
let total = w.global::<Library>().get_library_total().max(0) as usize;
let first_visible = ctl
.layout
.borrow()
.ordinal_at(first_slot)
.min(total.saturating_sub(1));
// **A report from a grid that is not on screen is not a scroll.** // **A report from a grid that is not on screen is not a scroll.**
// //
@@ -693,7 +729,7 @@ fn wire_grid_sync_and_load(window: &AppWindow, ctl: &Rc<LibraryController>) {
// hand. Without this they each rebuilt against whatever `scroll-to` // hand. Without this they each rebuilt against whatever `scroll-to`
// was last *set* to — a stale scrub, or zero — and landed there. // was last *set* to — a stale scrub, or zero — and landed there.
w.global::<Library>() w.global::<Library>()
.set_library_scroll_to(first_visible as i32); .set_library_scroll_to(first_slot as i32);
// Move the timeline marker with the view. Scrolling the grid is a // Move the timeline marker with the view. Scrolling the grid is a
// way of moving through time just as scrubbing is, and a marker // way of moving through time just as scrubbing is, and a marker
+439
View File
@@ -0,0 +1,439 @@
//! TRACES: FR-MRG-6 | FR-CAT-4
//! Where each photograph sits in the grid: the one place rows are computed.
//!
//! # Slots
//!
//! The grid is a lattice of `columns` slots per row, and until panoramas had
//! cells of their own a photograph's slot was its ordinal — row
//! `ordinal / columns`, column `ordinal % columns`, spelled out in half a
//! dozen places in the markup and in Rust. A photograph at least about twice
//! as wide as it is tall now takes two, three or four slots side by side —
//! the class [`natural_span`] picks from its aspect, which also picks its
//! thumbnail's size class — and a slot is no longer an ordinal: every cell after a wide one
//! is pushed along, and a wide cell that would not fit in what is left of a
//! row starts the next one and leaves the rest of its row empty. Reading
//! order is kept — nothing later is moved up into the gap — so the arrows,
//! a shift-click's run and a scrub all still mean what they meant.
//!
//! Everything that turns an ordinal into a place on screen, or a place on
//! screen into an ordinal, asks [`Layout`]; the markup draws each cell at
//! the slot Rust gives it and sizes the scrollbar from [`Layout::total_slots`].
//!
//! # What it costs
//!
//! Where a wide photograph sits depends on every wide photograph before it,
//! so the layout is built from the ordinals of all of them in the current
//! view, not from the loaded window. Those are few — a library has a handful
//! of panoramas among thousands of frames — and they are read in one query,
//! and only when what the grid lists changes (`LibraryFacts`), or when the
//! window finds them out of date. The window itself says how wide each of
//! its own photographs is: the aspect comes with the cells' own read.
//!
//! # On the tablet, and with few columns
//!
//! A wide photograph takes the whole row there: on a touch-first device
//! always, and anywhere the columns are too few to put it beside anything.
/// The aspect from which a photograph takes more than one slot, and the
/// classes above it: `SPAN_FROM[k]` is where a span of `k + 2` begins.
///
/// # Why these
///
/// A cell `s` slots wide is about `s:1`, and the thumbnail is fitted inside
/// it, so a photograph of aspect `a` between `s` and `s + 1` either fills
/// the width of an `s` cell and leaves `s / a` of it drawn, or the height of
/// an `s + 1` cell and leaves `a / (s + 1)`. The two are equal at
/// `a = √(s(s+1))`: 2.45 between two and three, 3.46 between three and four.
/// The first step is the exception, at 1.9 rather than √2: a 3:2 frame is a
/// photograph, not a panorama, and a 2:1 crop a pixel short of 2 still is.
pub const SPAN_FROM: [f32; 3] = [1.9, 2.45, 3.46];
/// TRACES: FR-MRG-6
/// How many slots a photograph of this aspect would like, before the columns
/// have their say: 1, or 2, 3 or 4 — the one lookup both the packing and
/// the thumbnail's size class (`ThumbSize::wide`) are chosen from.
pub fn natural_span(aspect: Option<f32>) -> usize {
let Some(a) = aspect else {
return 1;
};
1 + SPAN_FROM.iter().take_while(|from| a >= **from).count()
}
/// How many slots it gets in a grid of `columns`: its own, or the whole row
/// where `full_width` says so or the columns are too few to put it beside
/// anything.
pub fn span_in(natural: usize, columns: usize, full_width: bool) -> usize {
let columns = columns.max(1);
if natural <= 1 {
1
} else if full_width || columns <= natural {
columns
} else {
natural
}
}
/// The grid's placement of the current view.
#[derive(Debug, Clone, PartialEq)]
pub struct Layout {
columns: usize,
total: usize,
/// Each wide photograph: its ordinal, the slot it starts at, how many
/// it takes, and how many it would like. Ascending in the first two.
anchors: Vec<(usize, usize, usize, usize)>,
}
impl Layout {
/// Every photograph one slot: the grid as it was.
pub fn uniform(columns: usize, total: usize) -> Self {
Layout {
columns: columns.max(1),
total,
anchors: Vec::new(),
}
}
/// Pack a view of `total` photographs whose wide ones are `wide` —
/// `(ordinal, natural span)`, in any order — into rows of `columns`.
pub fn pack(columns: usize, total: usize, full_width: bool, wide: &[(usize, usize)]) -> Self {
let columns = columns.max(1);
let mut wide: Vec<(usize, usize, usize)> = wide
.iter()
.filter(|(ordinal, _)| *ordinal < total)
.map(|&(ordinal, natural)| (ordinal, span_in(natural, columns, full_width), natural))
.filter(|(_, span, _)| *span > 1)
.collect();
wide.sort_unstable();
wide.dedup_by_key(|(ordinal, _, _)| *ordinal);
let mut anchors = Vec::with_capacity(wide.len());
// The slot and ordinal just past the last wide cell placed.
let (mut next_slot, mut next_ordinal) = (0usize, 0usize);
for (ordinal, span, natural) in wide {
let mut slot = next_slot + (ordinal - next_ordinal);
if slot % columns + span > columns {
slot = slot.div_ceil(columns) * columns;
}
anchors.push((ordinal, slot, span, natural));
next_slot = slot + span;
next_ordinal = ordinal + 1;
}
Layout {
columns,
total,
anchors,
}
}
/// Whether any photograph takes more than one slot.
#[cfg(test)]
pub fn is_uniform(&self) -> bool {
self.anchors.is_empty()
}
/// The last wide cell at or before `ordinal`.
fn anchor_before(&self, ordinal: usize) -> Option<&(usize, usize, usize, usize)> {
let at = self.anchors.partition_point(|(o, _, _, _)| *o <= ordinal);
at.checked_sub(1).map(|i| &self.anchors[i])
}
/// The slot `ordinal` starts at.
pub fn slot_of(&self, ordinal: usize) -> usize {
match self.anchor_before(ordinal) {
None => ordinal,
Some(&(o, slot, _, _)) if o == ordinal => slot,
Some(&(o, slot, span, _)) => slot + span + (ordinal - o - 1),
}
}
/// How many slots `ordinal` takes.
pub fn span_of(&self, ordinal: usize) -> usize {
match self.anchor_before(ordinal) {
Some(&(o, _, span, _)) if o == ordinal => span,
_ => 1,
}
}
/// The columns' worth of thumbnail `ordinal`'s cell draws: its span,
/// but no more than its own class — a 2:1 panorama given the whole row
/// on the tablet is fitted to the row's height, and is drawn two
/// columns wide in it.
pub fn class_span(&self, ordinal: usize) -> usize {
match self.anchor_before(ordinal) {
Some(&(o, _, span, natural)) if o == ordinal => span.min(natural),
_ => 1,
}
}
/// The slot just past the last photograph: what the scrollbar spans.
pub fn total_slots(&self) -> usize {
self.slot_of(self.total)
}
/// The first photograph whose cell ends after `slot` — the one a view
/// whose first row begins at `slot` shows first. `total` past the end.
pub fn ordinal_at(&self, slot: usize) -> usize {
// `slot_of(n) + span_of(n)` rises with `n`, so the answer is where
// it first passes `slot`.
let (mut lo, mut hi) = (0usize, self.total);
while lo < hi {
let mid = lo + (hi - lo) / 2;
if self.slot_of(mid) + self.span_of(mid) > slot {
hi = mid;
} else {
lo = mid + 1;
}
}
lo
}
/// The photograph a step of `rows` rows from `ordinal` lands on: the
/// one under the same column, or where that falls in the gap a wide cell
/// left at the end of a row, the last one in that row. Clamped to the
/// view.
pub fn step_rows(&self, ordinal: usize, rows: isize) -> usize {
if self.total == 0 {
return 0;
}
let rows_total = self.total_slots().div_ceil(self.columns) as isize;
let from = self.slot_of(ordinal.min(self.total - 1));
let row = (from / self.columns) as isize + rows;
if row < 0 {
return 0;
}
if row >= rows_total {
return self.total - 1;
}
let target = row as usize * self.columns + from % self.columns;
// The last photograph starting at or before the target slot.
let after = self.ordinal_starting_after(target);
after.saturating_sub(1).min(self.total - 1)
}
/// The first photograph that starts after `slot`.
fn ordinal_starting_after(&self, slot: usize) -> usize {
let (mut lo, mut hi) = (0usize, self.total);
while lo < hi {
let mid = lo + (hi - lo) / 2;
if self.slot_of(mid) > slot {
hi = mid;
} else {
lo = mid + 1;
}
}
lo
}
/// Whether the window starting at `offset` agrees with this layout
/// about which of its photographs are wide — `natural[i]` being the
/// natural span of the window's row `i`, as its own read found it. A
/// layout built before a reorder, a merge or a change of aspect does
/// not, and is read again.
pub fn agrees_with(&self, offset: usize, natural: &[usize], full_width: bool) -> bool {
natural
.iter()
.enumerate()
.all(|(i, &n)| self.span_of(offset + i) == span_in(n, self.columns, full_width))
}
}
#[cfg(test)]
mod tests {
use super::*;
/// The grid drawn as text, one row per line: `.` a photograph, a letter
/// for each slot of a wide one, `_` an empty slot.
fn draw(layout: &Layout) -> Vec<String> {
let mut slots = vec!['_'; layout.total_slots()];
let mut wide = b'A';
for n in 0..layout.total {
let (s, span) = (layout.slot_of(n), layout.span_of(n));
for slot in &mut slots[s..s + span] {
assert_eq!(*slot, '_', "photograph {n} overlaps another at slot {s}");
*slot = if span > 1 { wide as char } else { '.' };
}
if span > 1 {
wide += 1;
}
}
slots
.chunks(layout.columns)
.map(|r| r.iter().collect())
.collect()
}
#[test]
fn a_grid_of_ordinary_photographs_is_the_grid_it_always_was() {
let l = Layout::pack(4, 10, false, &[]);
assert!(l.is_uniform());
assert_eq!(l, Layout::uniform(4, 10));
assert_eq!(draw(&l), ["....", "....", ".."]);
for n in 0..10 {
assert_eq!(l.slot_of(n), n);
}
assert_eq!(l.ordinal_at(8), 8);
}
#[test]
fn a_wide_photograph_takes_two_slots_and_the_rest_move_along() {
// Photograph 2 is a 2:1 panorama, 5 a 4:1.
let l = Layout::pack(5, 10, false, &[(2, 2), (5, 3)]);
assert_eq!(draw(&l), ["..AA.", ".BBB.", "..."]);
assert_eq!(l.slot_of(3), 4);
assert_eq!(l.slot_of(5), 6);
assert_eq!(l.span_of(5), 3);
assert_eq!(l.total_slots(), 13);
}
#[test]
fn a_wide_photograph_that_does_not_fit_starts_the_next_row() {
// Three ordinary frames fill three of four columns; the panorama
// after them needs two, so it opens the next row and the fourth
// column of the first stays empty. Nothing later is moved up into
// it: the grid still reads in capture order.
let l = Layout::pack(4, 8, false, &[(3, 3)]);
assert_eq!(draw(&l), ["..._", "AAA.", "..."]);
assert_eq!(l.slot_of(3), 4);
assert_eq!(l.slot_of(4), 7);
}
#[test]
fn with_too_few_columns_a_wide_photograph_takes_the_whole_row() {
let l = Layout::pack(3, 5, false, &[(1, 3)]);
assert_eq!(draw(&l), [".__", "AAA", "..."]);
let l = Layout::pack(2, 4, false, &[(1, 2)]);
assert_eq!(draw(&l), ["._", "AA", ".."]);
// One column: nothing to span.
let l = Layout::pack(1, 3, false, &[(1, 3)]);
assert!(l.is_uniform());
}
#[test]
fn on_the_tablet_a_wide_photograph_takes_the_whole_row() {
let l = Layout::pack(5, 8, true, &[(2, 2)]);
assert_eq!(draw(&l), ["..___", "AAAAA", "....."]);
}
#[test]
fn at_each_class_boundary_the_cell_left_empty_is_the_smaller() {
// At `a = √(s(s+1))` an `s` cell and an `s + 1` one leave the same
// share undrawn; either side, the class chosen leaves less.
let drawn = |a: f32, s: usize| (s as f32 / a).min(a / s as f32);
for (k, from) in SPAN_FROM.iter().enumerate().skip(1) {
let s = k + 1;
for a in [from - 0.05, from + 0.05] {
let chosen = natural_span(Some(a));
let other = if chosen == s { s + 1 } else { s };
assert!(
drawn(a, chosen) >= drawn(a, other),
"{a}: {chosen} draws {} and {other} {}",
drawn(a, chosen),
drawn(a, other)
);
}
}
}
#[test]
fn a_row_of_every_class_packs_in_reading_order() {
// Six columns: a 2:1, a frame, a 3:1, then a 4:1 that does not fit
// beside them, a frame, a 5:1 that is still four, and a 2:1 beside
// it.
let aspects = [2.0, 1.5, 3.0, 4.0, 1.5, 5.0, 2.0, 1.5];
let wide: Vec<(usize, usize)> = aspects
.iter()
.enumerate()
.map(|(n, a)| (n, natural_span(Some(*a))))
.filter(|(_, s)| *s > 1)
.collect();
let l = Layout::pack(6, aspects.len(), false, &wide);
assert_eq!(draw(&l), ["AA.BBB", "CCCC._", "DDDDEE", "."]);
assert_eq!(l.class_span(3), 4);
assert_eq!(l.class_span(5), 4);
// With four columns the 4:1s take whole rows and nothing else moves
// out of order.
let l = Layout::pack(4, aspects.len(), false, &wide);
assert_eq!(draw(&l), ["AA._", "BBB_", "CCCC", ".___", "DDDD", "EE."]);
// On the tablet every wide one takes the row, and its thumbnail is
// still its own class.
let l = Layout::pack(6, aspects.len(), true, &wide);
assert_eq!(l.span_of(0), 6);
assert_eq!(l.class_span(0), 2);
}
#[test]
fn wide_photographs_side_by_side_and_back_to_back() {
let l = Layout::pack(5, 7, false, &[(0, 2), (1, 2), (2, 2), (3, 3)]);
assert_eq!(draw(&l), ["AABB_", "CCDDD", "..."]);
}
#[test]
fn a_first_visible_slot_names_the_photograph_there() {
let l = Layout::pack(4, 8, false, &[(3, 3)]);
// Row 1 begins at slot 4, the panorama.
assert_eq!(l.ordinal_at(4), 3);
// The empty slot at the end of row 0 belongs to nothing: the next
// photograph is the panorama.
assert_eq!(l.ordinal_at(3), 3);
// Slot 5 is inside the panorama.
assert_eq!(l.ordinal_at(5), 3);
assert_eq!(l.ordinal_at(8), 5);
// Past the end.
assert_eq!(l.ordinal_at(40), 8);
// And every ordinal is found again at its own slot.
for n in 0..8 {
assert_eq!(l.ordinal_at(l.slot_of(n)), n);
}
}
#[test]
fn up_and_down_move_by_rows_not_by_a_row_of_ordinals() {
// ..._
// AAA.
// ....
let l = Layout::pack(4, 8, false, &[(3, 3)]);
// Down from the second frame lands in the panorama under it.
assert_eq!(l.step_rows(1, 1), 3);
// Down from the panorama, to the frame under its first column.
assert_eq!(l.step_rows(3, 1), 5);
// Up from the frame beside the panorama: the gap above it is empty,
// so the last frame of that row.
assert_eq!(l.step_rows(4, -1), 2);
// Up from under the panorama's middle, into the panorama.
assert_eq!(l.step_rows(6, -1), 3);
// Clamped at both ends.
assert_eq!(l.step_rows(1, -3), 0);
assert_eq!(l.step_rows(1, 9), 7);
// A page is several rows at once.
assert_eq!(l.step_rows(0, 2), 5);
}
#[test]
fn a_window_that_disagrees_with_the_layout_is_caught() {
let l = Layout::pack(4, 8, false, &[(3, 3)]);
assert!(l.agrees_with(2, &[1, 3, 1], false));
// The panorama has moved to ordinal 4: the window sees it there.
assert!(!l.agrees_with(2, &[1, 1, 3], false));
// A photograph that became wide.
assert!(!l.agrees_with(0, &[2], false));
}
#[test]
fn spans_by_aspect() {
assert_eq!(natural_span(None), 1);
assert_eq!(natural_span(Some(1.5)), 1);
// The class boundaries, each side.
assert_eq!(natural_span(Some(1.89)), 1);
assert_eq!(natural_span(Some(1.9)), 2);
assert_eq!(natural_span(Some(2.44)), 2);
assert_eq!(natural_span(Some(2.45)), 3);
assert_eq!(natural_span(Some(3.45)), 3);
assert_eq!(natural_span(Some(3.46)), 4);
assert_eq!(natural_span(Some(9.0)), 4, "four is the widest");
assert_eq!(span_in(2, 6, false), 2);
assert_eq!(span_in(3, 3, false), 3);
assert_eq!(span_in(3, 2, false), 2);
assert_eq!(span_in(2, 6, true), 6);
assert_eq!(span_in(1, 6, true), 1);
}
}
+2 -1
View File
@@ -39,6 +39,7 @@
mod controller; mod controller;
mod filter_bar; mod filter_bar;
mod grid; mod grid;
pub(crate) mod layout;
mod offline; mod offline;
mod open; mod open;
mod ratings_keywords; mod ratings_keywords;
@@ -48,8 +49,8 @@ mod window;
pub use controller::LibraryController; pub use controller::LibraryController;
pub use grid::wire; pub use grid::wire;
pub use open::{catalogue_composite, open, reload};
pub(crate) use open::{forget_catalog, show_catalog_now, start_rescan}; pub(crate) use open::{forget_catalog, show_catalog_now, start_rescan};
pub use open::{open, reload};
pub use ratings_keywords::paste_settings_to_selection; pub use ratings_keywords::paste_settings_to_selection;
pub(crate) use ratings_keywords::{save_judgements, start_sidecar_writes}; pub(crate) use ratings_keywords::{save_judgements, start_sidecar_writes};
pub use timeline::format_date; pub use timeline::format_date;
+65 -4
View File
@@ -26,7 +26,7 @@ use super::offline::refresh_offline;
use super::ratings_keywords::refresh_xmp_conflicts; use super::ratings_keywords::refresh_xmp_conflicts;
use super::sync::{start_derived_sync, start_sweep}; use super::sync::{start_derived_sync, start_sweep};
use super::timeline::{apply_place, note_place}; use super::timeline::{apply_place, note_place};
use super::window::{load_window, window_start}; use super::window::{load_window, seek_to, window_start};
/// Reload the grid for the current scope and offset. /// Reload the grid for the current scope and offset.
/// ///
@@ -41,6 +41,66 @@ pub fn reload(window: &AppWindow, ctl: &Rc<LibraryController>) {
note_place(window, ctl); note_place(window, ctl);
} }
/// TRACES: FR-MRG-6
/// Put a composite the merge has just staged into the catalog and the grid,
/// before its upload has finished and before any scan could find it.
///
/// See `library::composite` for why this does not wait for the scan. The
/// row is keyed where the scan will list the file, so the scan after the
/// upload updates it rather than adding a second.
pub fn catalogue_composite(
window: &AppWindow,
ctl: &Rc<LibraryController>,
placed: &crate::merge_ui::Placed,
) {
let Some((conn, _)) = ctl.session.borrow().clone() else {
return;
};
let root = conn.account.root.clone();
let c = &placed.composite;
let source_ref = crate::export::staged_remote_path(&root, &placed.remote_dir, &c.name);
let row = library::CompositeRow {
source_ref: source_ref.as_str().to_string(),
width: c.width,
height: c.height,
captured_at: c.captured_at,
captured_offset: c.captured_offset,
camera: c.camera.clone(),
lens: c.lens.clone(),
iso: c.iso,
file_size: c.file_size,
sources: placed.sources.clone(),
};
let written = {
let borrow = ctl.catalog.borrow();
let Some(catalog) = borrow.as_ref() else {
return;
};
library::catalogue_composite(catalog, &root, &row)
};
match written {
Ok(image) => {
log::info!("merge: {} catalogued as image {image}", row.source_ref);
let decoded: Vec<(dr_thumbs::ThumbSize, slint::Image)> = c
.thumbnails
.iter()
.filter_map(|(class, t)| {
let (w, h, rgba) = dr_thumbs::decode_rgba(&t.bytes).ok()?;
Some((*class, super::window::to_slint_image(w, h, &rgba)))
})
.collect();
if !decoded.is_empty() {
ctl.merged_thumbs.borrow_mut().insert(image, decoded);
}
// Asked for afresh: a row under this name may have been drawn,
// and answered, before this merge wrote over it.
ctl.requested.borrow_mut().retain(|(id, _)| *id != image);
reload(window, ctl);
}
Err(e) => log::warn!("merge: cataloguing {}: {e}", row.source_ref),
}
}
/// Open a library: show the grid, start a scan, then fill in thumbnails. /// Open a library: show the grid, start a scan, then fill in thumbnails.
/// ///
/// Called from the launch screen's "Open library" button — the callback that /// Called from the launch screen's "Open library" button — the callback that
@@ -68,6 +128,9 @@ pub fn open(
let filter = account.format_filter(); let filter = account.format_filter();
*ctl.session.borrow_mut() = Some((conn.clone(), filter.clone())); *ctl.session.borrow_mut() = Some((conn.clone(), filter.clone()));
// Image ids are one catalog's; another library's would name other
// photographs.
ctl.merged_thumbs.borrow_mut().clear();
// TRACES: FR-UI-8 // TRACES: FR-UI-8
// Where this library's position is kept, and what this device signs it // Where this library's position is kept, and what this device signs it
@@ -713,9 +776,7 @@ pub(super) fn schedule_reload(window: &AppWindow, ctl: &Rc<LibraryController>) {
let total = w.global::<Library>().get_library_total().max(0) as usize; let total = w.global::<Library>().get_library_total().max(0) as usize;
*ctl_cb.offset.borrow_mut() = window_start(anchor, window_size, total); *ctl_cb.offset.borrow_mut() = window_start(anchor, window_size, total);
load_window(&w, &ctl_cb); load_window(&w, &ctl_cb);
w.global::<Library>().set_library_scroll_to(anchor as i32); seek_to(&w, &ctl_cb, anchor);
w.global::<Library>()
.set_library_scroll_token(w.global::<Library>().get_library_scroll_token() + 1);
}); });
// Replacing the slot drops the previous timer, which is what makes this // Replacing the slot drops the previous timer, which is what makes this
+1 -17
View File
@@ -112,23 +112,7 @@ pub(super) fn start_derived_sync(window: &AppWindow, ctl: &Rc<LibraryController>
let outbox = crate::export::outbox_dir(&conn.account); let outbox = crate::export::outbox_dir(&conn.account);
if crate::export::pending_count(&outbox) > 0 { if crate::export::pending_count(&outbox) > 0 {
let rx = crate::export::spawn_upload(conn.clone(), conn.account.root.clone(), outbox); let rx = crate::export::spawn_upload(conn.clone(), conn.account.root.clone(), outbox);
executors::spawn(Executor::Io, "upload-log", move || { crate::export::watch_upload(rx, window.as_weak());
while let Ok(msg) = rx.recv() {
match msg {
crate::export::UploadMessage::Status(s) => log::info!("export: {s}"),
crate::export::UploadMessage::Finished {
uploaded,
remaining,
error,
} => {
log::info!("export: {uploaded} uploaded, {remaining} still queued");
if let Some(e) = error {
log::warn!("export upload stopped: {e}");
}
}
}
}
});
} }
} }
+7 -20
View File
@@ -18,7 +18,7 @@ use crate::{AppWindow, Library, TimelineBar, View};
use super::controller::LibraryController; use super::controller::LibraryController;
use super::filter_bar::{push_people_chips, refilter}; use super::filter_bar::{push_people_chips, refilter};
use super::window::{load_window, mark_open, window_start}; use super::window::{load_window, mark_open, seek_to, show_cursor, window_start};
/// Move the timeline's zoom by whole levels. /// Move the timeline's zoom by whole levels.
/// ///
@@ -501,13 +501,10 @@ fn scrub_to(window: &AppWindow, ctl: &Rc<LibraryController>, when: i64) {
// Move the viewport as well as the window. Cells are drawn at their // Move the viewport as well as the window. Cells are drawn at their
// absolute place in the library, so loading rows around image 15,000 while // absolute place in the library, so loading rows around image 15,000 while
// the viewport sits at row 0 shows an empty grid until the user scrolls. // the viewport sits at row 0 shows an empty grid until the user scrolls.
window
.global::<Library>()
.set_library_scroll_to(position as i32);
window
.global::<Library>()
.set_library_scroll_token(window.global::<Library>().get_library_scroll_token() + 1);
load_window(window, ctl); load_window(window, ctl);
// After the load, which is what places the cells: the grid scrolls by
// slot, and a panorama before `position` moves its slot along.
seek_to(window, ctl, position);
} }
/// TRACES: FR-CAT-7 /// TRACES: FR-CAT-7
@@ -543,12 +540,7 @@ pub fn restore_position(window: &AppWindow, ctl: &Rc<LibraryController>) {
// `resume_at` is what the grid last reported as its first visible image, // `resume_at` is what the grid last reported as its first visible image,
// which is the thing the photographer is actually looking at. // which is the thing the photographer is actually looking at.
let anchor = ctl.resume_at.get().min(total - 1); let anchor = ctl.resume_at.get().min(total - 1);
window seek_to(window, ctl, anchor);
.global::<Library>()
.set_library_scroll_to(anchor as i32);
window
.global::<Library>()
.set_library_scroll_token(window.global::<Library>().get_library_scroll_token() + 1);
} }
/// Format a bucket start for the histogram's hover label. /// Format a bucket start for the histogram's hover label.
@@ -998,16 +990,11 @@ pub(super) fn resume_position(
// Before the grid is shown, not after: the markup gates it on an `if`, and // Before the grid is shown, not after: the markup gates it on an `if`, and
// the rebuilt Flickable reads `scroll-to` in its `init`. Setting these // the rebuilt Flickable reads `scroll-to` in its `init`. Setting these
// afterwards would leave that init to run against the previous position. // afterwards would leave that init to run against the previous position.
window seek_to(window, ctl, anchor);
.global::<Library>()
.set_library_scroll_to(anchor as i32);
window
.global::<Library>()
.set_library_scroll_token(window.global::<Library>().get_library_scroll_token() + 1);
if let Some(at) = focus { if let Some(at) = focus {
coll.set_cursor(Some(at)); coll.set_cursor(Some(at));
window.global::<Library>().set_library_cursor(at as i32); show_cursor(window, ctl, at);
} }
// The view has moved, so the capture-time marker has to move with it. // The view has moved, so the capture-time marker has to move with it.
+168 -19
View File
@@ -230,6 +230,31 @@ pub(super) fn bring_window_to(
.filter(|row| *row < ctl.paths.borrow().len()) .filter(|row| *row < ctl.paths.borrow().len())
} }
/// TRACES: FR-MRG-6
/// Ask the grid to show ordinal `at` in its first row.
///
/// The grid scrolls by slots, which a panorama's cell makes different from
/// ordinals; this is the one conversion, so no caller spells `scroll-to`
/// in the wrong unit.
pub(super) fn seek_to(window: &AppWindow, ctl: &LibraryController, at: usize) {
let slot = ctl.layout.borrow().slot_of(at);
window
.global::<Library>()
.set_library_scroll_to(slot as i32);
window
.global::<Library>()
.set_library_scroll_token(window.global::<Library>().get_library_scroll_token() + 1);
}
/// Put the keyboard cursor on ordinal `at`, telling the grid where its cell
/// is so its row can be kept in view.
pub(super) fn show_cursor(window: &AppWindow, ctl: &LibraryController, at: usize) {
window.global::<Library>().set_library_cursor(at as i32);
window
.global::<Library>()
.set_library_cursor_slot(ctl.layout.borrow().slot_of(at) as i32);
}
/// Record that the photograph at `row` of the loaded window is the one now /// Record that the photograph at `row` of the loaded window is the one now
/// open in develop, and mark it on the roll. /// open in develop, and mark it on the roll.
pub(super) fn mark_open(window: &AppWindow, ctl: &LibraryController, row: usize) { pub(super) fn mark_open(window: &AppWindow, ctl: &LibraryController, row: usize) {
@@ -327,6 +352,10 @@ pub(super) fn load_window(window: &AppWindow, ctl: &Rc<LibraryController>) {
}; };
let describes_something_new = ctl.library_facts.borrow().as_ref() != Some(&facts); let describes_something_new = ctl.library_facts.borrow().as_ref() != Some(&facts);
*ctl.library_facts.borrow_mut() = Some(facts); *ctl.library_facts.borrow_mut() = Some(facts);
// A different list has its panoramas in different places.
if describes_something_new {
*ctl.wide.borrow_mut() = None;
}
// TRACES: FR-NC-6a // TRACES: FR-NC-6a
// Whether the newly scoped collection is already pinned. Read here rather // Whether the newly scoped collection is already pinned. Read here rather
@@ -444,18 +473,37 @@ pub(super) fn load_window(window: &AppWindow, ctl: &Rc<LibraryController>) {
.into(), .into(),
); );
// TRACES: FR-MRG-6
// Where each cell goes: the layout of the whole list, checked against
// what this window says about its own photographs' shapes and read again
// if they disagree — a reorder, a merge, a composite's size learned.
let columns = window.global::<Library>().get_library_columns().max(1) as usize;
let layout = place_cells(
catalog, ctl, &cells, offset, total, columns, scope, &filter, trash,
);
window
.global::<Library>()
.set_library_total_slots(layout.total_slots() as i32);
let cursor = window.global::<Library>().get_library_cursor();
if cursor >= 0 {
window
.global::<Library>()
.set_library_cursor_slot(layout.slot_of(cursor as usize) as i32);
}
// Month headings. The grid is ordered by capture time, so without these a // Month headings. The grid is ordered by capture time, so without these a
// wall of thumbnails gives no sense of *when* you are looking — the // wall of thumbnails gives no sense of *when* you are looking — the
// sidebar says it, but only if you consult it. // sidebar says it, but only if you consult it.
let columns = window.global::<Library>().get_library_columns().max(1) as usize;
let headings = period_headings( let headings = period_headings(
cells.iter().map(|c| { cells.iter().enumerate().map(|(i, c)| {
c.captured_at.map(|t| { (
let (y, m, _, _) = civil_from_unix(t); c.captured_at.map(|t| {
(y, m) let (y, m, _, _) = civil_from_unix(t);
}) (y, m)
}),
layout.slot_of(offset + i),
)
}), }),
offset,
columns, columns,
); );
@@ -467,12 +515,51 @@ pub(super) fn load_window(window: &AppWindow, ctl: &Rc<LibraryController>) {
hold_thumbnails(&previous, &ids, &classes) hold_thumbnails(&previous, &ids, &classes)
}; };
// A composite a merge has just catalogued is drawn from the merge's own
// thumbnails until the store can serve it (FR-MRG-6), and counts as
// served at that class so no fetch is spent on a file still uploading.
let cell_pixels = window.global::<Library>().get_library_cell_size().max(1.0) as u32;
let mut held = held;
{
let merged = ctl.merged_thumbs.borrow();
for (i, c) in cells.iter().enumerate() {
let Some(made) = merged.get(&c.image_id) else {
continue;
};
let wanted = dr_thumbs::ThumbSize::for_span(layout.class_span(offset + i), cell_pixels);
if held
.get(&c.image_id)
.is_some_and(|h| h.class == Some(wanted))
{
continue;
}
let pick = made
.iter()
.find(|(class, _)| *class == wanted)
.or_else(|| made.iter().max_by_key(|(class, _)| class.edge()));
if let Some((class, image)) = pick {
held.insert(
c.image_id,
Held {
thumbnail: image.clone(),
has_thumb: true,
unavailable: false,
class: Some(*class),
},
);
}
}
}
let rows: Vec<LibraryCell> = cells let rows: Vec<LibraryCell> = cells
.iter() .iter()
.zip(headings) .zip(headings)
.map(|(c, heading)| { .enumerate()
.map(|(i, (c, heading))| {
let carried = held.get(&c.image_id); let carried = held.get(&c.image_id);
LibraryCell { LibraryCell {
slot: layout.slot_of(offset + i) as i32,
span: layout.span_of(offset + i) as i32,
period_heading: heading.into(), period_heading: heading.into(),
// A freshly loaded window has no drag in flight. // A freshly loaded window has no drag in flight.
lifted: false, lifted: false,
@@ -574,6 +661,55 @@ pub(super) fn load_window(window: &AppWindow, ctl: &Rc<LibraryController>) {
request_thumbnails(window, ctl); request_thumbnails(window, ctl);
} }
/// TRACES: FR-MRG-6
/// The layout of the current list, made to agree with the window just read.
///
/// The panoramas' ordinals are kept between loads — a scroll does not move
/// them — and read again only when they are unknown or when this window
/// finds a photograph wide that the layout does not, or the other way round.
/// That check is over the window's own rows, in memory; the read it can
/// trigger is one query ([`library::wide_ordinals`]).
#[allow(clippy::too_many_arguments)]
fn place_cells(
catalog: &dr_catalog::Catalog,
ctl: &LibraryController,
cells: &[library::LibraryCell],
offset: usize,
total: usize,
columns: usize,
scope: Option<library::Scope>,
filter: &library::RatingFilter,
trash: bool,
) -> super::layout::Layout {
use super::layout::{natural_span, Layout};
let full_width = dr_plat::is_touch_first();
let natural: Vec<usize> = cells.iter().map(|c| natural_span(c.aspect)).collect();
let read = || -> Vec<(usize, usize)> {
match library::wide_ordinals(catalog, scope, filter, trash) {
Ok(wide) => wide
.into_iter()
.map(|(ordinal, aspect)| (ordinal, natural_span(Some(aspect))))
.collect(),
Err(e) => {
log::warn!("reading the grid's panoramas: {e}");
Vec::new()
}
}
};
if ctl.wide.borrow().is_none() {
*ctl.wide.borrow_mut() = Some(read());
}
let pack = |wide: &[(usize, usize)]| Layout::pack(columns, total, full_width, wide);
let mut layout = pack(ctl.wide.borrow().as_deref().unwrap_or_default());
if !layout.agrees_with(offset, &natural, full_width) {
let wide = read();
layout = pack(&wide);
*ctl.wide.borrow_mut() = Some(wide);
}
*ctl.layout.borrow_mut() = layout.clone();
layout
}
/// Where a row of the loaded window sits in the fetch queue. /// Where a row of the loaded window sits in the fetch queue.
/// ///
/// On screen first, top to bottom; then the rows below the view, nearest /// On screen first, top to bottom; then the rows below the view, nearest
@@ -611,11 +747,17 @@ fn request_thumbnails(window: &AppWindow, ctl: &Rc<LibraryController>) {
return; return;
}; };
// The drawn cell size decides which class to ask for. Chosen once for the // The drawn cell size decides which class to ask for, and a panorama's
// batch rather than per row, and carried through to the drain so a cell it // cell asks for the wide one (FR-MRG-6). Carried through to the drain per
// fills can record what it is now showing. // row, so a cell it fills can record what it is now showing.
let cell_pixels = window.global::<Library>().get_library_cell_size().max(1.0) as u32; let cell_pixels = window.global::<Library>().get_library_cell_size().max(1.0) as u32;
let class = dr_thumbs::ThumbSize::for_cell(cell_pixels); let offset = *ctl.offset.borrow();
let classes: Vec<dr_thumbs::ThumbSize> = {
let layout = ctl.layout.borrow();
(0..ctl.paths.borrow().len())
.map(|i| dr_thumbs::ThumbSize::for_span(layout.class_span(offset + i), cell_pixels))
.collect()
};
let mut wanted: Vec<library::ThumbnailRequest> = { let mut wanted: Vec<library::ThumbnailRequest> = {
let paths = ctl.paths.borrow(); let paths = ctl.paths.borrow();
@@ -631,7 +773,10 @@ fn request_thumbnails(window: &AppWindow, ctl: &Rc<LibraryController>) {
let image_id = *image_ids.get(i)?; let image_id = *image_ids.get(i)?;
// A zoomed grid asks for detail a 256px thumbnail cannot give, // A zoomed grid asks for detail a 256px thumbnail cannot give,
// and a wall of small cells does not pay for it. // and a wall of small cells does not pay for it.
let thumb_size = class; let thumb_size = classes
.get(i)
.copied()
.unwrap_or(dr_thumbs::ThumbSize::Grid);
// Keyed on the photograph, so scrolling back over a cell that // Keyed on the photograph, so scrolling back over a cell that
// has already been served does not ask for it again. // has already been served does not ask for it again.
if !requested.insert((image_id, thumb_size)) { if !requested.insert((image_id, thumb_size)) {
@@ -660,7 +805,7 @@ fn request_thumbnails(window: &AppWindow, ctl: &Rc<LibraryController>) {
// the model they were built against; only the order they are asked for in // the model they were built against; only the order they are asked for in
// changes. // changes.
{ {
let first_on_screen = ctl.resume_at.get().saturating_sub(*ctl.offset.borrow()); let first_on_screen = ctl.resume_at.get().saturating_sub(offset);
let on_screen = ctl.viewport_cells.get().max(1); let on_screen = ctl.viewport_cells.get().max(1);
wanted.sort_by_key(|r| fetch_rank(r.row, first_on_screen, on_screen)); wanted.sort_by_key(|r| fetch_rank(r.row, first_on_screen, on_screen));
} }
@@ -672,7 +817,7 @@ fn request_thumbnails(window: &AppWindow, ctl: &Rc<LibraryController>) {
library::thumbs_dir(&conn.account), library::thumbs_dir(&conn.account),
library::catalog_path(&conn.account), library::catalog_path(&conn.account),
); );
drain_thumbnails(window.as_weak(), ctl.clone(), rx, requested, class); drain_thumbnails(window.as_weak(), ctl.clone(), rx, requested, classes);
} }
/// TRACES: FR-CAT-9 | FR-DEV-6 /// TRACES: FR-CAT-9 | FR-DEV-6
@@ -798,7 +943,7 @@ fn drain_thumbnails(
ctl: Rc<LibraryController>, ctl: Rc<LibraryController>,
rx: Receiver<ThumbnailMessage>, rx: Receiver<ThumbnailMessage>,
requested: usize, requested: usize,
class: dr_thumbs::ThumbSize, classes: Vec<dr_thumbs::ThumbSize>,
) { ) {
let timer = slint::Timer::default(); let timer = slint::Timer::default();
let ctl_cb = ctl.clone(); let ctl_cb = ctl.clone();
@@ -940,7 +1085,9 @@ fn drain_thumbnails(
model.set_row_data(t.row, row); model.set_row_data(t.row, row);
// What this cell is now showing, so the next reload // What this cell is now showing, so the next reload
// can carry it over and know not to ask again. // can carry it over and know not to ask again.
record_class(&ctl_cb, t.row, class); if let Some(class) = classes.get(t.row) {
record_class(&ctl_cb, t.row, *class);
}
} }
} }
ThumbnailMessage::Unavailable { row, reason } => { ThumbnailMessage::Unavailable { row, reason } => {
@@ -952,7 +1099,9 @@ fn drain_thumbnails(
// A verdict is worth carrying too: "no preview" is // A verdict is worth carrying too: "no preview" is
// an answer about the file, and re-asking it on // an answer about the file, and re-asking it on
// every scroll is a fetch that will fail again. // every scroll is a fetch that will fail again.
record_class(&ctl_cb, row, class); if let Some(class) = classes.get(row) {
record_class(&ctl_cb, row, *class);
}
} }
} }
// TRACES: FR-CAT-9 // TRACES: FR-CAT-9
@@ -989,7 +1138,7 @@ fn drain_thumbnails(
/// This is a CPU copy, which is acceptable here and not in the develop path: /// This is a CPU copy, which is acceptable here and not in the develop path:
/// a 256px thumbnail is 256 KB and happens once per image, where the canvas /// a 256px thumbnail is 256 KB and happens once per image, where the canvas
/// would pay per frame (ARCH §6.1). /// would pay per frame (ARCH §6.1).
fn to_slint_image(width: u32, height: u32, rgba: &[u8]) -> slint::Image { pub(super) fn to_slint_image(width: u32, height: u32, rgba: &[u8]) -> slint::Image {
let mut buf = slint::SharedPixelBuffer::<slint::Rgba8Pixel>::new(width, height); let mut buf = slint::SharedPixelBuffer::<slint::Rgba8Pixel>::new(width, height);
let expected = (width as usize) * (height as usize) * 4; let expected = (width as usize) * (height as usize) * 4;
let src = &rgba[..expected.min(rgba.len())]; let src = &rgba[..expected.min(rgba.len())];
+191 -12
View File
@@ -87,6 +87,11 @@ pub enum MergeDestination {
outbox: PathBuf, outbox: PathBuf,
/// The sources' folder, relative to the library root. /// The sources' folder, relative to the library root.
remote_dir: String, remote_dir: String,
/// The names the library already holds in that folder, as the
/// catalog has them. The upload replaces whatever is at its name, so
/// a second merge of the same frames must not be called what the
/// first one was.
taken: std::collections::HashSet<String>,
}, },
} }
@@ -228,18 +233,45 @@ pub enum MergeEvent {
/// for the photographer to confirm, and what names a failure. /// for the photographer to confirm, and what names a failure.
Aligned(AlignmentReport), Aligned(AlignmentReport),
/// The composite is written: on the device at `path`, or staged in the /// The composite is written: on the device at `path`, or staged in the
/// outbox for the drain to upload, in which case `staged` is true and /// outbox for the drain to upload, in which case `staged` is true.
/// the library learns of it when the folder is next scanned. /// `composite` is what the library needs to show it before the upload
/// has finished and a scan has found it (FR-MRG-6).
Done { Done {
path: PathBuf, path: PathBuf,
staged: bool, staged: bool,
width: u32, width: u32,
height: u32, height: u32,
composite: Box<Composite>,
}, },
Failed(String), Failed(String),
Cancelled, Cancelled,
} }
/// TRACES: FR-MRG-6
/// A finished composite, described for the catalog: everything a scan and a
/// header read would have found, known here without either.
#[derive(Debug, Clone, PartialEq)]
pub struct Composite {
/// Its name where it goes — in the library folder for a staged merge.
pub name: String,
/// The picture it opens on: the crop where the border was cropped, the
/// whole composite where it was filled.
pub width: u32,
pub height: u32,
/// The middle of the sweep, as written into the DNG; `None` where no
/// frame carried a time.
pub captured_at: Option<i64>,
pub captured_offset: Option<i32>,
pub camera: Option<String>,
pub lens: Option<String>,
pub iso: Option<u32>,
pub file_size: u64,
/// Made from the final image as develop would first show it
/// ([`crate::merge_thumbs`]); empty if that failed, and the grid then
/// thumbnails it the ordinary way.
pub thumbnails: Vec<(dr_thumbs::ThumbSize, dr_thumbs::Thumbnail)>,
}
impl MergeEvent { impl MergeEvent {
/// Whether this is the job's last word: after one of these the worker /// Whether this is the job's last word: after one of these the worker
/// has nothing more to say, so its channel closing is expected. /// has nothing more to say, so its channel closing is expected.
@@ -598,10 +630,14 @@ fn run_inner(
// 5. Merge, into a DNG beside the first frame. // 5. Merge, into a DNG beside the first frame.
let name = format!("{}-pano.dng", stem(Path::new(&first_name))); let name = format!("{}-pano.dng", stem(Path::new(&first_name)));
let (out_path, staged) = match &request.destination { let (out_path, staged) = match &request.destination {
MergeDestination::Local(dir) => (unused_name(dir, &name), false), MergeDestination::Local(dir) => (unused_name(dir, &name, &Default::default()), false),
MergeDestination::Outbox { outbox, remote_dir } => { MergeDestination::Outbox {
outbox,
remote_dir,
taken,
} => {
std::fs::create_dir_all(outbox).map_err(|e| format!("{}: {e}", outbox.display()))?; std::fs::create_dir_all(outbox).map_err(|e| format!("{}: {e}", outbox.display()))?;
let path = unused_name(outbox, &name); let path = unused_name(outbox, &name, taken);
// The record first here, unlike an export: the payload is // The record first here, unlike an export: the payload is
// written over minutes and a record naming a half-written file // written over minutes and a record naming a half-written file
// is worse than a payload with no record, so the record is // is worse than a payload with no record, so the record is
@@ -617,10 +653,20 @@ fn run_inner(
(path, true) (path, true)
} }
}; };
// Written under a name of its own and renamed into place once the last
// strip is in. The record above names `out_path`, and a drain that runs
// while the merge is still writing — a sync pass fires one whenever the
// sweep finishes — takes whatever is at that name: it uploaded the first
// few hundred megabytes of a composite, cleared the record, and the file
// on the server stayed truncated for good. `pending` skips a record whose
// payload does not exist yet, so the rename is what releases it.
let part_path = part_name(&out_path);
let cleanup = |path: &Path| { let cleanup = |path: &Path| {
let _ = std::fs::remove_file(path); let _ = std::fs::remove_file(path);
let _ = std::fs::remove_file(part_name(path));
if staged { if staged {
let _ = std::fs::remove_file(crate::export::destination_record(path)); let _ = std::fs::remove_file(crate::export::destination_record(path));
let _ = std::fs::remove_file(crate::export::thumbnails_record(path));
} }
}; };
let (out_w, out_h) = (report_size(&bounds).0, report_size(&bounds).1); let (out_w, out_h) = (report_size(&bounds).0, report_size(&bounds).1);
@@ -630,6 +676,8 @@ fn run_inner(
let profile = dng_profile(first, white_level); let profile = dng_profile(first, white_level);
let balance = profile.as_shot_neutral.map(|n| 1.0 / n.max(1e-3)); let balance = profile.as_shot_neutral.map(|n| 1.0 / n.max(1e-3));
let (_, mut carried) = crate::export::header_for_file(&first.meta); let (_, mut carried) = crate::export::header_for_file(&first.meta);
// The thumbnails' copy of the profile; the writer takes the original.
let thumb_profile = profile.clone();
// The composite is dated at the middle of its sweep — the mean of the // The composite is dated at the middle of its sweep — the mean of the
// frames' capture times — so it sorts among the frames it was made // frames' capture times — so it sorts among the frames it was made
// from, not at the first of them and not at the moment of the merge. // from, not at the first of them and not at the moment of the merge.
@@ -640,6 +688,17 @@ fn run_inner(
carried.captured_at = Some((sum / stamps.len() as i128) as i64); carried.captured_at = Some((sum / stamps.len() as i128) as i64);
} }
// What the catalog is told about the file (FR-MRG-6), taken before the
// header moves into the writer.
let thumb_header = carried.clone();
let described = (
carried.captured_at,
carried.captured_offset,
crate::library::camera_label(carried.make.as_deref(), carried.model.as_deref()),
carried.lens.as_ref().map(|l| l.trim().to_string()),
carried.iso,
);
let output = MergeOutput { let output = MergeOutput {
projection, projection,
scale: focal_full, scale: focal_full,
@@ -798,7 +857,7 @@ fn run_inner(
let inscribed_for_writer = inscribed.clone(); let inscribed_for_writer = inscribed.clone();
let fill_for_writer = fill_cam.is_some(); let fill_for_writer = fill_cam.is_some();
let file = let file =
std::fs::File::create(&out_path).map_err(|e| format!("{}: {e}", out_path.display()))?; std::fs::File::create(&part_path).map_err(|e| format!("{}: {e}", part_path.display()))?;
let writer = executors::spawn(Executor::Io, "dng-write", move || -> Result<(), String> { let writer = executors::spawn(Executor::Io, "dng-write", move || -> Result<(), String> {
let mut file = std::io::BufWriter::new(file); let mut file = std::io::BufWriter::new(file);
dr_export::write_linear_dng( dr_export::write_linear_dng(
@@ -830,6 +889,9 @@ fn run_inner(
let total_bands = out_h.div_ceil(rows_per_strip) as usize; let total_bands = out_h.div_ceil(rows_per_strip) as usize;
let mut bands_done = 0usize; let mut bands_done = 0usize;
// The final image, reduced as it goes by, for the thumbnails (FR-MRG-6).
let mut reduced =
crate::merge_thumbs::Reduced::new(out_w, out_h, crate::merge_thumbs::SOURCE_EDGE);
progress("Merging", 0, total_bands); progress("Merging", 0, total_bands);
let merged = pass.merge( let merged = pass.merge(
&mut adjust, &mut adjust,
@@ -884,6 +946,7 @@ fn run_inner(
} }
} }
} }
reduced.push(band.first_row, band.rows, &rgb);
band_tx band_tx
.send(rgb) .send(rgb)
.map_err(|_| dr_gpu::GpuError::Readback("the writer stopped".into())) .map_err(|_| dr_gpu::GpuError::Readback("the writer stopped".into()))
@@ -910,6 +973,52 @@ fn run_inner(
cleanup(&out_path); cleanup(&out_path);
return Err(e); return Err(e);
} }
// The rectangle the file opens on, as the writer recorded it.
let picture = if fill_cam.is_some() {
None
} else {
inscribed
.lock()
.ok()
.map(|i| clamp_crop(i.best(), out_w, out_h))
.filter(|r| r.width > 0 && r.height > 0)
};
// TRACES: FR-MRG-6
// The thumbnails, from the final image, developed as develop will first
// show the file. Before the rename, and staged beside the payload before
// it: the rename is what lets a drain take the file, and the drain is
// what puts these in the store once the server has named it.
// The merge's GPU memory back before develop asks for its own.
drop(resident);
drop(pass);
progress("Making thumbnails", 0, 1);
let (pw, ph) = picture.map_or((out_w, out_h), |r| (r.width, r.height));
let thumbnails = match crate::merge_thumbs::render(
ctx,
reduced,
&thumb_profile,
&thumb_header,
picture,
request.decoder,
&crate::merge_thumbs::classes(pw, ph),
) {
Ok(t) => t,
Err(e) => {
log::warn!("merge: the thumbnails could not be made: {e}");
Vec::new()
}
};
if staged && !thumbnails.is_empty() {
if let Err(e) = crate::export::write_thumbnails(&out_path, &thumbnails) {
log::warn!("merge: the thumbnails could not be staged: {e}");
}
}
if let Err(e) = std::fs::rename(&part_path, &out_path) {
cleanup(&out_path);
return Err(format!("{}: {e}", out_path.display()));
}
log::info!( log::info!(
"merge: {}×{} written to {} in {:?}", "merge: {}×{} written to {} in {:?}",
out_w, out_w,
@@ -958,14 +1067,44 @@ fn run_inner(
} }
} }
let (captured_at, captured_offset, camera, lens, iso) = described;
let composite = Composite {
name: out_path
.file_name()
.map(|n| n.to_string_lossy().into_owned())
.unwrap_or_default(),
width: pw,
height: ph,
captured_at,
captured_offset,
camera,
lens,
iso,
file_size: std::fs::metadata(&out_path).map(|m| m.len()).unwrap_or(0),
thumbnails,
};
Ok(Some(MergeEvent::Done { Ok(Some(MergeEvent::Done {
path: out_path, path: out_path,
staged, staged,
width: out_w, width: out_w,
height: out_h, height: out_h,
composite: Box::new(composite),
})) }))
} }
/// The crop as `write_linear_dng` writes it: inside the frame.
fn clamp_crop(r: dr_export::Rect, width: u32, height: u32) -> dr_export::Rect {
let x = r.x.min(width.saturating_sub(1));
let y = r.y.min(height.saturating_sub(1));
dr_export::Rect {
x,
y,
width: r.width.min(width - x),
height: r.height.min(height - y),
}
}
/// The proxy the detector reads: the frame through the camera-space tap at /// The proxy the detector reads: the frame through the camera-space tap at
/// proxy size, upright, as gamma-encoded grey. /// proxy size, upright, as gamma-encoded grey.
/// ///
@@ -1471,17 +1610,29 @@ fn stem(p: &Path) -> String {
.unwrap_or_else(|| p.display().to_string()) .unwrap_or_else(|| p.display().to_string())
} }
/// Where the composite is written until it is whole: `x.dng.part` for
/// `x.dng`.
fn part_name(path: &Path) -> PathBuf {
let mut name = path.as_os_str().to_owned();
name.push(".part");
PathBuf::from(name)
}
/// `name` in `dir`, numbered if that name is taken: a merge never /// `name` in `dir`, numbered if that name is taken: a merge never
/// overwrites (FR-MRG-3). /// overwrites (FR-MRG-3). `name` in `dir`, or `name-2`, `name-3`… — the first that neither `dir`
fn unused_name(dir: &Path, name: &str) -> PathBuf { /// nor `taken` (the names already at the destination) holds.
let mut candidate = dir.join(name); fn unused_name(dir: &Path, name: &str, taken: &std::collections::HashSet<String>) -> PathBuf {
let mut candidate = name.to_string();
let base = stem(Path::new(name)); let base = stem(Path::new(name));
let mut n = 2; let mut n = 2;
while candidate.exists() { while dir.join(&candidate).exists()
candidate = dir.join(format!("{base}-{n}.dng")); || part_name(&dir.join(&candidate)).exists()
|| taken.contains(&candidate)
{
candidate = format!("{base}-{n}.dng");
n += 1; n += 1;
} }
candidate dir.join(candidate)
} }
/// Drain everything a job has said so far. /// Drain everything a job has said so far.
@@ -1568,6 +1719,34 @@ mod tests {
); );
} }
#[test]
fn a_second_merge_is_not_named_over_the_first_in_the_library() {
// The outbox is empty once the first has uploaded, so only the
// catalog knows the name is taken — and the upload replaces
// whatever is at its name.
let dir = std::env::temp_dir().join(format!("dr-merge-names-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
let none = std::collections::HashSet::new();
assert_eq!(
unused_name(&dir, "a-pano.dng", &none),
dir.join("a-pano.dng")
);
let taken: std::collections::HashSet<String> =
["a-pano.dng".to_string(), "a-pano-2.dng".to_string()].into();
assert_eq!(
unused_name(&dir, "a-pano.dng", &taken),
dir.join("a-pano-3.dng")
);
// And one still being written in the outbox holds its name too.
std::fs::write(part_name(&dir.join("a-pano.dng")), b"").unwrap();
assert_eq!(
unused_name(&dir, "a-pano.dng", &none),
dir.join("a-pano-2.dng")
);
let _ = std::fs::remove_dir_all(&dir);
}
#[test] #[test]
fn a_worker_that_hangs_up_mid_job_is_reported_as_gone() { fn a_worker_that_hangs_up_mid_job_is_reported_as_gone() {
// The failure the page could not see: a panic drops the sender with // The failure the page could not see: a panic drops the sender with
+429
View File
@@ -0,0 +1,429 @@
//! TRACES: FR-MRG-6
//! The composite's thumbnails, made by the merge from its own final image.
//!
//! # Why the merge makes them
//!
//! A linear DNG as the merge writes it carries no embedded preview, and an
//! embedded preview is all the grid's ordinary thumbnail path reads — so a
//! composite sat in the grid as a blank cell until someone opened it. Making
//! the thumbnail afterwards means reading back a file of 800 MB; the merge
//! has every pixel of it in hand as the bands go by.
//!
//! # From the final image, through develop
//!
//! The bands are box-reduced as they are written — after the border fill,
//! so the thumbnail shows what the file holds — into a copy
//! [`SOURCE_EDGE`] pixels long. That copy is written as a linear DNG in
//! memory with the composite's own profile, header and crop, and opened
//! exactly as develop opens a file on first sight (`crate::open_session`):
//! the same decode, the same default graph with its default view transform
//! and tone mapping, the file's as-shot white balance and the conversion
//! from the working space to the display's. The thumbnail is therefore the
//! photograph the photographer will see on opening it, not a second,
//! plausible conversion — the merge's own quick look ([`crate::merge`]'s
//! `Look`) is exactly the naive linear-to-gamma picture this is not.
//!
//! The reduction is develop's too, in effect: a composite past one texture
//! is developed from a box-reduced copy (`DevelopSession::open_owned`), and
//! this is the same box at a coarser step.
use std::io::Cursor;
use dr_thumbs::{ThumbSize, Thumbnail};
/// The long edge of the reduced copy the thumbnails are developed from:
/// twice the largest thumbnail, so the develop render resamples down rather
/// than up, and small enough to cost the merge a few tens of megabytes.
pub const SOURCE_EDGE: u32 = 4096;
/// The composite box-reduced as its bands go by.
pub struct Reduced {
width: u32,
height: u32,
factor: u32,
/// Per reduced pixel, per channel: the sum of the full-resolution
/// samples that fall in its box.
sums: Vec<u32>,
}
impl Reduced {
/// A reduction of a `width × height` composite to at most `long_edge`
/// pixels on its longer side, by a whole factor.
pub fn new(width: u32, height: u32, long_edge: u32) -> Self {
let factor = width.max(height).div_ceil(long_edge.max(1)).max(1);
let (w, h) = (width.div_ceil(factor), height.div_ceil(factor));
Reduced {
width,
height,
factor,
sums: vec![0; w as usize * h as usize * 3],
}
}
/// The reduced size.
pub fn size(&self) -> (u32, u32) {
(
self.width.div_ceil(self.factor),
self.height.div_ceil(self.factor),
)
}
/// The whole factor each side is divided by.
pub fn factor(&self) -> u32 {
self.factor
}
/// Fold in `rows` rows of the composite starting at `first_row`,
/// interleaved RGB at the full width.
pub fn push(&mut self, first_row: u32, rows: u32, rgb: &[u16]) {
let (rw, _) = self.size();
let (w, f) = (self.width as usize, self.factor as usize);
for r in 0..rows as usize {
let y = first_row as usize + r;
if y >= self.height as usize {
break;
}
let row = &rgb[r * w * 3..(r + 1) * w * 3];
let out = (y / f) * rw as usize * 3;
for (x, px) in row.chunks_exact(3).enumerate() {
let o = out + (x / f) * 3;
self.sums[o] += u32::from(px[0]);
self.sums[o + 1] += u32::from(px[1]);
self.sums[o + 2] += u32::from(px[2]);
}
}
}
/// The averages, as the DNG's `u16` samples. A box cut short by the
/// edge of the composite is the mean of what it holds.
pub fn finish(self) -> (u32, u32, Vec<u16>) {
let (rw, rh) = self.size();
let f = self.factor;
let mut out = vec![0u16; self.sums.len()];
for y in 0..rh {
let bh = f.min(self.height - y * f);
for x in 0..rw {
let bw = f.min(self.width - x * f);
let n = bh * bw;
let i = (y * rw + x) as usize * 3;
for c in 0..3 {
out[i + c] = ((self.sums[i + c] + n / 2) / n) as u16;
}
}
}
(rw, rh, out)
}
}
/// Which size classes a composite of this shape is thumbnailed at: the two
/// every photograph has, and the wide classes its cell can be drawn at —
/// its own, from the one lookup the grid packs by, and each narrower one a
/// grid with too few columns falls back to.
pub fn classes(width: u32, height: u32) -> Vec<ThumbSize> {
let aspect = width as f32 / height.max(1) as f32;
let natural = crate::library_ui::layout::natural_span(Some(aspect));
let mut out = vec![ThumbSize::Grid, ThumbSize::Large];
out.extend((2..=natural).filter_map(ThumbSize::wide));
out
}
/// Develop the reduced composite as develop would open the file, and render
/// it at each of `classes`.
///
/// `crop` is the rectangle the file opens on, in the composite's pixels;
/// it is carried to the reduced copy so the thumbnail is of the picture
/// and not of the border around it.
pub fn render(
ctx: &dr_gpu::GpuContext,
reduced: Reduced,
profile: &dr_export::DngProfile,
header: &dr_export::SourceMetadata,
crop: Option<dr_export::Rect>,
decoder: &dyn dr_decode::Decoder,
classes: &[ThumbSize],
) -> Result<Vec<(ThumbSize, Thumbnail)>, String> {
let f = reduced.factor();
let (w, h, samples) = reduced.finish();
let crop = crop.map(|r| {
let x = (r.x / f).min(w.saturating_sub(1));
let y = (r.y / f).min(h.saturating_sub(1));
dr_export::Rect {
x,
y,
width: (r.width / f).clamp(1, w - x),
height: (r.height / f).clamp(1, h - y),
}
});
let mut file = Cursor::new(Vec::new());
let mut once = Some(samples);
dr_export::write_linear_dng(
&mut file,
w,
h,
h,
profile,
Some(header),
|_, buf| {
buf.extend_from_slice(&once.take().unwrap_or_default());
Ok(())
},
|| crop,
)
.map_err(|e| e.to_string())?;
let bytes = file.into_inner();
// The file as develop reads it on first open: its header, then the
// pixels, through the one function every develop session starts from.
let meta = decoder.metadata(&bytes).map_err(|e| e.to_string())?;
let mut session = crate::open_session(ctx, decoder, &bytes, &meta)?;
let mut out = Vec::with_capacity(classes.len());
for &class in classes {
let (tw, th, rgba) = session.render_thumbnail(class.edge())?;
let jpeg = dr_thumbs::encode_rgba(tw, th, &rgba).map_err(|e| e.to_string())?;
out.push((
class,
Thumbnail {
width: tw,
height: th,
bytes: jpeg,
},
));
}
Ok(out)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_panorama_is_thumbnailed_for_its_wide_cell_and_a_squat_merge_is_not() {
use ThumbSize::*;
assert_eq!(
classes(22_000, 5_600),
[Grid, Large, Wide2, Wide3, Wide4],
"a 3.9:1 panorama spans four"
);
assert_eq!(classes(8_000, 4_000), [Grid, Large, Wide2]);
assert_eq!(classes(9_000, 6_000), [Grid, Large]);
}
#[test]
fn a_reduction_averages_its_boxes_and_the_ragged_edge() {
// 5 × 3 by 2: boxes of 2 × 2, then 1 × 2 at the right, 2 × 1 and
// 1 × 1 along the bottom.
let mut r = Reduced::new(5, 3, 3);
assert_eq!(r.factor(), 2);
assert_eq!(r.size(), (3, 2));
let px = |x: u16, y: u16| [x * 10 + y, 0, 1000];
let rows: Vec<u16> = (0..3)
.flat_map(|y| (0..5).flat_map(move |x| px(x, y)))
.collect();
// In two bands, as the merge hands them over.
r.push(0, 2, &rows[..5 * 2 * 3]);
r.push(2, 1, &rows[5 * 2 * 3..]);
let (w, h, out) = r.finish();
assert_eq!((w, h), (3, 2));
let at = |x: usize, y: usize| out[(y * 3 + x) * 3];
// (0,0),(1,0),(0,1),(1,1) → 0, 10, 1, 11: mean 5.5, rounded.
assert_eq!(at(0, 0), 6);
// The right-hand column is one pixel wide: (4,0) and (4,1).
assert_eq!(at(2, 0), 41, "40 and 41, rounded");
// The bottom-right is a single pixel.
assert_eq!(at(2, 1), 42);
assert!(out.chunks_exact(3).all(|p| p[2] == 1000));
}
/// A synthetic composite in camera space: a sky that brightens to a
/// clipped band, green ground with texture, and a blown sun.
fn composite(width: u32, height: u32, white: u32) -> Vec<u16> {
let mut out = Vec::with_capacity((width * height * 3) as usize);
for y in 0..height {
for x in 0..width {
let fy = y as f32 / height as f32;
let fx = x as f32 / width as f32;
let tex = ((x / 7 + y / 5) % 2) as f32 * 0.03;
let cam = if fy < 0.45 {
// Sky: blue-ish in camera space, brighter toward the top
// and toward the right, clipping in the top corner.
let b = (0.25 + 0.9 * (1.0 - fy / 0.45) * fx).min(1.2);
[b * 0.45, b * 0.8, b]
} else {
let g = 0.08 + 0.25 * fx + tex;
[g * 0.6, g, g * 0.4]
};
let sun = ((fx - 0.8).powi(2) + (fy - 0.2).powi(2)).sqrt() < 0.04;
let cam = if sun { [1.0, 1.0, 1.0] } else { cam };
for c in cam {
out.push(((c * white as f32).round() as u32).min(white) as u16);
}
}
}
out
}
fn profile(white: u32) -> dr_export::DngProfile {
// A Canon 6D's D65 matrix and as-shot neutral, as the fixture's
// composite carries them.
dr_export::DngProfile {
unique_model: "Canon EOS 6D".into(),
calibrations: vec![(
21,
[
[0.7034, -0.0804, -0.1014],
[-0.4420, 1.2564, 0.2058],
[-0.0851, 0.1994, 0.5758],
],
)],
as_shot_neutral: [0.4985, 1.0, 0.6505],
white_level: white,
}
}
/// Luma, in 0..255, for each pixel of an RGBA buffer.
fn luma(rgba: &[u8]) -> Vec<f32> {
rgba.chunks_exact(4)
.map(|p| 0.2126 * p[0] as f32 + 0.7152 * p[1] as f32 + 0.0722 * p[2] as f32)
.collect()
}
fn stats(mut l: Vec<f32>) -> (f32, f32, f32) {
let mean = l.iter().sum::<f32>() / l.len() as f32;
l.sort_by(f32::total_cmp);
let at = |q: f32| l[((l.len() - 1) as f32 * q).round() as usize];
(mean, at(0.95), at(0.995))
}
/// FR-MRG-6's acceptance: the thumbnail made during the merge is the
/// picture develop shows on first opening the composite — its overall
/// level and its highlights — and not a naive conversion of the linear
/// data, which comes out a stop and more away on both.
#[test]
fn the_thumbnail_is_what_develop_shows_on_first_open() {
let Some(ctx) = crate::develop::test_support::headless() else {
eprintln!("no GPU; skipping");
return;
};
// Past `SOURCE_EDGE`, so the thumbnails come from a reduced copy.
let (width, height, white) = (4600u32, 1160u32, 13_023u32);
let pixels = composite(width, height, white);
let profile = profile(white);
let header = dr_export::SourceMetadata {
make: Some("Canon".into()),
model: Some("Canon EOS 6D".into()),
captured_at: Some(1_600_000_000),
..Default::default()
};
let crop = dr_export::Rect {
x: 40,
y: 30,
width: 4500,
height: 1080,
};
// Develop's side: the whole composite written as the merge writes
// it, opened as develop opens it, rendered for the grid.
let rows_per_strip = 256u32;
let mut file = Cursor::new(Vec::new());
dr_export::write_linear_dng(
&mut file,
width,
height,
rows_per_strip,
&profile,
Some(&header),
|k, buf| {
let from = k * (rows_per_strip * width * 3) as usize;
let to = (from + (rows_per_strip * width * 3) as usize).min(pixels.len());
buf.extend_from_slice(&pixels[from..to]);
Ok(())
},
|| Some(crop),
)
.unwrap();
let bytes = file.into_inner();
let decoder = dr_decode::default();
let meta = decoder.metadata(&bytes).unwrap();
let mut developed = crate::open_session(&ctx, decoder, &bytes, &meta).unwrap();
// The merge's side: the same bands, reduced as they pass.
let mut reduced = Reduced::new(width, height, SOURCE_EDGE);
assert_eq!(reduced.factor(), 2);
for first in (0..height).step_by(rows_per_strip as usize) {
let rows = rows_per_strip.min(height - first);
let from = (first * width * 3) as usize;
reduced.push(
first,
rows,
&pixels[from..from + (rows * width * 3) as usize],
);
}
let thumbs = render(
&ctx,
reduced,
&profile,
&header,
Some(crop),
decoder,
&classes(crop.width, crop.height),
)
.unwrap();
// The naive picture: the linear camera values balanced and gamma
// encoded, with no matrix, no curve and no highlight handling — what
// the merge's preview draws.
let naive = {
let mut l = Vec::new();
for y in (crop.y..crop.y + crop.height).step_by(4) {
for x in (crop.x..crop.x + crop.width).step_by(4) {
let i = ((y * width + x) * 3) as usize;
let v = |c: usize| {
let lin = pixels[i + c] as f32 / white as f32 / profile.as_shot_neutral[c];
lin.clamp(0.0, 1.0).powf(1.0 / 2.2) * 255.0
};
l.push(0.2126 * v(0) + 0.7152 * v(1) + 0.0722 * v(2));
}
}
stats(l)
};
for (class, thumb) in thumbs {
let (tw, th, ours) = dr_thumbs::decode_rgba(&thumb.bytes).unwrap();
let (dw, dh, theirs) = developed.render_thumbnail(class.edge()).unwrap();
// Through the store's encoding too: both are JPEGs in the store,
// and it is the encoding, not the development, that rounds a
// blown sun off to 247.
let theirs = {
let jpeg = dr_thumbs::encode_rgba(dw, dh, &theirs).unwrap();
dr_thumbs::decode_rgba(&jpeg).unwrap().2
};
// The picture, not the border: the crop's shape at the class's
// size, give or take the reduction's rounding.
assert!(
(tw as i64 - dw as i64).abs() <= 2 && (th as i64 - dh as i64).abs() <= 2,
"{class:?}: {tw}×{th} against develop's {dw}×{dh}"
);
let (m0, p95_0, p995_0) = stats(luma(&ours));
let (m1, p95_1, p995_1) = stats(luma(&theirs));
eprintln!(
"{class:?}: mean {m0:.1}/{m1:.1}, p95 {p95_0:.1}/{p95_1:.1}, \
p99.5 {p995_0:.1}/{p995_1:.1}; naive {naive:?}"
);
assert!((m0 - m1).abs() <= 3.0, "{class:?} mean {m0} vs {m1}");
// And the tolerance means something: the conversion this
// replaces is nowhere near it.
assert!(
(naive.0 - m1).abs() > 12.0 || (naive.1 - p95_1).abs() > 12.0,
"a naive conversion ({naive:?}) would pass as well"
);
assert!(
(p95_0 - p95_1).abs() <= 4.0,
"{class:?} p95 {p95_0} vs {p95_1}"
);
assert!(
(p995_0 - p995_1).abs() <= 4.0,
"{class:?} p99.5 {p995_0} vs {p995_1}"
);
}
}
}
+79 -13
View File
@@ -8,9 +8,10 @@
//! opens the page; a timer drains the job's events into the page's //! opens the page; a timer drains the job's events into the page's
//! properties. When the alignment arrives the page shows it and waits. //! properties. When the alignment arrives the page shows it and waits.
//! A frame's box leaves it out or brings it back, and the job aligns again //! A frame's box leaves it out or brings it back, and the job aligns again
//! over the rest. "Merge" sends the decision; "Stop" or "Back" cancels. When the file is //! over the rest. "Merge" sends the decision; "Stop" or "Back" cancels. When
//! staged, the outbox drains and the library rescans, and the composite //! the file is staged the composite is catalogued at once and appears in the
//! appears in the grid beside its sources. //! grid beside its sources; the outbox drains, and the scan that follows the
//! upload gives it the identity the server assigned (FR-MRG-6).
use crate::executors::{self, Executor}; use crate::executors::{self, Executor};
use std::cell::{Cell, RefCell}; use std::cell::{Cell, RefCell};
@@ -36,6 +37,29 @@ struct Job {
cancel: Cancel, cancel: Cancel,
names: Vec<String>, names: Vec<String>,
activity: Activity, activity: Activity,
/// Where in the library the composite goes, for a merge of library
/// frames; `None` for one written to a folder on the device.
library: Option<Destined>,
}
/// A library merge's destination, kept for the moment it finishes.
#[derive(Debug, Clone)]
struct Destined {
remote_dir: String,
sources: Vec<String>,
}
/// TRACES: FR-MRG-6
/// A composite that has been staged for the library: where it goes, what it
/// was made from, and what the job knows about it — enough for the library
/// to catalogue it before the upload has finished.
#[derive(Debug, Clone)]
pub struct Placed {
/// The sources' folder, relative to the library root.
pub remote_dir: String,
/// The frames, as library paths.
pub sources: Vec<String>,
pub composite: merge::Composite,
} }
pub struct MergeController { pub struct MergeController {
@@ -73,16 +97,22 @@ impl MergeController {
} }
} }
/// The names a library folder holds, by folder.
pub type NamesIn = Box<dyn Fn(&str) -> std::collections::HashSet<String>>;
/// What the page needs from the library to start: the account for the /// What the page needs from the library to start: the account for the
/// fetch, and where the outbox is. /// fetch, where the outbox is, and the names a library folder already
/// holds, so the composite is not named over one of them.
pub struct Context { pub struct Context {
pub conn: dr_sync::Connection, pub conn: dr_sync::Connection,
pub outbox: std::path::PathBuf, pub outbox: std::path::PathBuf,
pub names_in: NamesIn,
} }
/// Wire the page. `sources` yields the selection as fetchable library /// Wire the page. `sources` yields the selection as fetchable library
/// sources; `context` the account; `on_done` runs when a composite has /// sources; `context` the account; `on_done` runs when a composite has
/// been staged, so the caller can drain the outbox and rescan. /// been written, with where it went in the library when it was staged for
/// one, so the caller can catalogue it and drain the outbox.
pub fn wire<S, C, F>( pub fn wire<S, C, F>(
window: &AppWindow, window: &AppWindow,
ctl: Rc<MergeController>, ctl: Rc<MergeController>,
@@ -93,7 +123,7 @@ pub fn wire<S, C, F>(
) where ) where
S: Fn() -> Vec<crate::export::Source> + 'static, S: Fn() -> Vec<crate::export::Source> + 'static,
C: Fn() -> Option<Context> + 'static, C: Fn() -> Option<Context> + 'static,
F: Fn(&AppWindow) + 'static, F: Fn(&AppWindow, Option<&Placed>) + 'static,
{ {
wire_start(window, &ctl, gpu, sources, context, on_done); wire_start(window, &ctl, gpu, sources, context, on_done);
wire_decision(window, &ctl); wire_decision(window, &ctl);
@@ -111,7 +141,7 @@ fn wire_start<S, C, F>(
) where ) where
S: Fn() -> Vec<crate::export::Source> + 'static, S: Fn() -> Vec<crate::export::Source> + 'static,
C: Fn() -> Option<Context> + 'static, C: Fn() -> Option<Context> + 'static,
F: Fn(&AppWindow) + 'static, F: Fn(&AppWindow, Option<&Placed>) + 'static,
{ {
let on_done = Rc::new(on_done); let on_done = Rc::new(on_done);
let gpu_for_start = gpu.clone(); let gpu_for_start = gpu.clone();
@@ -165,10 +195,15 @@ fn wire_start<S, C, F>(
let remote_dir = first_dir let remote_dir = first_dir
.strip_prefix(&root) .strip_prefix(&root)
.map(|s| s.trim_start_matches('/').to_string()) .map(|s| s.trim_start_matches('/').to_string())
.unwrap_or(first_dir); .unwrap_or_else(|| first_dir.clone());
let destination = MergeDestination::Outbox { let destination = MergeDestination::Outbox {
outbox: context.outbox, outbox: context.outbox,
remote_dir: remote_dir.clone(),
taken: (context.names_in)(&first_dir),
};
let destined = Destined {
remote_dir, remote_dir,
sources: sources.iter().map(|(p, _)| p.clone()).collect(),
}; };
let names: Vec<String> = sources let names: Vec<String> = sources
@@ -226,7 +261,16 @@ fn wire_start<S, C, F>(
} }
Some(frames) Some(frames)
}; };
start(&w, &ctl, gpu, names, destination, fetch, &on_done); start(
&w,
&ctl,
gpu,
names,
destination,
Some(destined),
fetch,
&on_done,
);
}); });
} }
@@ -273,6 +317,7 @@ fn wire_start<S, C, F>(
gpu, gpu,
names, names,
MergeDestination::Local(dir), MergeDestination::Local(dir),
None,
fetch, fetch,
&on_done, &on_done,
); );
@@ -459,14 +504,18 @@ fn wire_stop_and_leave(window: &AppWindow, ctl: &Rc<MergeController>) {
/// Start a job: `fetch` runs first on the job's thread and hands back the /// Start a job: `fetch` runs first on the job's thread and hands back the
/// frames (or reports why not and returns `None`); the job follows on the /// frames (or reports why not and returns `None`); the job follows on the
/// same thread. The page opens clean, and a timer drains the events. /// same thread. The page opens clean, and a timer drains the events.
// Each argument is a different part of the job: where it runs, what it is
// called, where it goes, how its frames arrive, and who is told.
#[allow(clippy::too_many_arguments)]
fn start<Fetch>( fn start<Fetch>(
window: &AppWindow, window: &AppWindow,
ctl: &Rc<MergeController>, ctl: &Rc<MergeController>,
gpu: dr_gpu::GpuContext, gpu: dr_gpu::GpuContext,
names: Vec<String>, names: Vec<String>,
destination: MergeDestination, destination: MergeDestination,
library: Option<Destined>,
fetch: Fetch, fetch: Fetch,
on_done: &Rc<impl Fn(&AppWindow) + 'static>, on_done: &Rc<impl Fn(&AppWindow, Option<&Placed>) + 'static>,
) where ) where
Fetch: FnOnce(&Sender<MergeEvent>, &Cancel) -> Option<Vec<MergeInput>> + Send + 'static, Fetch: FnOnce(&Sender<MergeEvent>, &Cancel) -> Option<Vec<MergeInput>> + Send + 'static,
{ {
@@ -495,6 +544,7 @@ fn start<Fetch>(
cancel, cancel,
names, names,
activity, activity,
library,
}); });
*ctl.report.borrow_mut() = None; *ctl.report.borrow_mut() = None;
ctl.projection.set(0); ctl.projection.set(0);
@@ -547,7 +597,11 @@ fn chip_projection(i: i32) -> Option<dr_pano::Projection> {
} }
/// Take everything the job has said and reflect it on the page. /// Take everything the job has said and reflect it on the page.
fn drain(window: &AppWindow, ctl: &Rc<MergeController>, on_done: &Rc<impl Fn(&AppWindow)>) { fn drain(
window: &AppWindow,
ctl: &Rc<MergeController>,
on_done: &Rc<impl Fn(&AppWindow, Option<&Placed>)>,
) {
let (events, gone) = { let (events, gone) = {
let job = ctl.job.borrow(); let job = ctl.job.borrow();
let Some(job) = job.as_ref() else { return }; let Some(job) = job.as_ref() else { return };
@@ -606,6 +660,7 @@ fn drain(window: &AppWindow, ctl: &Rc<MergeController>, on_done: &Rc<impl Fn(&Ap
staged, staged,
width, width,
height, height,
composite,
} => { } => {
let name = path let name = path
.file_name() .file_name()
@@ -616,7 +671,7 @@ fn drain(window: &AppWindow, ctl: &Rc<MergeController>, on_done: &Rc<impl Fn(&Ap
window.set_merge_done(true); window.set_merge_done(true);
window.set_merge_status( window.set_merge_status(
if staged { if staged {
format!("{name} ({width}×{height}) is queued to go beside its sources.") format!("{name} ({width}×{height}) is in the library, beside its sources.")
} else { } else {
format!("{name} ({width}×{height}) written to {}.", path.display()) format!("{name} ({width}×{height}) written to {}.", path.display())
} }
@@ -626,7 +681,18 @@ fn drain(window: &AppWindow, ctl: &Rc<MergeController>, on_done: &Rc<impl Fn(&Ap
job.activity.finish(format!("{name} merged")); job.activity.finish(format!("{name} merged"));
} }
finished = true; finished = true;
on_done(window); let placed = ctl
.job
.borrow()
.as_ref()
.and_then(|j| j.library.clone())
.filter(|_| staged)
.map(|d| Placed {
remote_dir: d.remote_dir,
sources: d.sources,
composite: *composite,
});
on_done(window, placed.as_ref());
} }
MergeEvent::Failed(e) => { MergeEvent::Failed(e) => {
window.set_merge_running(false); window.set_merge_running(false);
+2
View File
@@ -712,6 +712,7 @@ fn metadata(
captured_at: md.captured_at, captured_at: md.captured_at,
captured_offset: md.captured_offset, captured_offset: md.captured_offset,
camera: crate::library::camera_label(md.make.as_deref(), md.model.as_deref()), camera: crate::library::camera_label(md.make.as_deref(), md.model.as_deref()),
size: MetadataFound::upright_size(&md),
lens: md.lens.map(|l| l.trim().to_string()), lens: md.lens.map(|l| l.trim().to_string()),
iso: md.iso, iso: md.iso,
}, },
@@ -724,6 +725,7 @@ fn metadata(
camera: None, camera: None,
lens: None, lens: None,
iso: None, iso: None,
size: None,
} }
} }
}; };
+5
View File
@@ -1463,6 +1463,7 @@ in property <bool> panel-visible: true;
expanded: root.expanded; expanded: root.expanded;
cells: Library.library-cells; cells: Library.library-cells;
total: Library.library-total; total: Library.library-total;
total-slots: Library.library-total-slots;
scanning: Library.library-scanning; scanning: Library.library-scanning;
opening: Library.library-opening; opening: Library.library-opening;
scan-status: Library.library-status; scan-status: Library.library-status;
@@ -1584,9 +1585,13 @@ in property <bool> panel-visible: true;
assign-keyword(word) => { Library.library-assign-keyword(word); } assign-keyword(word) => { Library.library-assign-keyword(word); }
unassign-keyword(word) => { Library.library-unassign-keyword(word); } unassign-keyword(word) => { Library.library-unassign-keyword(word); }
cursor: Library.library-cursor; cursor: Library.library-cursor;
cursor-slot: Library.library-cursor-slot;
move-cursor(delta, extend) => { move-cursor(delta, extend) => {
Library.library-move-cursor(delta, extend); Library.library-move-cursor(delta, extend);
} }
move-cursor-rows(rows, extend) => {
Library.library-move-cursor-rows(rows, extend);
}
open-cursor() => { Library.library-open-cursor(); } open-cursor() => { Library.library-open-cursor(); }
drag-image: Library.library-drag-image; drag-image: Library.library-drag-image;
drag-payload() => { return Library.library-drag-payload(); } drag-payload() => { return Library.library-drag-payload(); }
+52 -17
View File
@@ -483,6 +483,13 @@ export struct LibraryCell {
// the one cell on screen is the representative by construction, and a mark // the one cell on screen is the representative by construction, and a mark
// saying so would be telling the user what they can already see. // saying so would be telling the user what they can already see.
burst-representative: bool, burst-representative: bool,
// TRACES: FR-MRG-6
// Where the cell sits in the grid's lattice of slots — row
// `slot / columns`, column `slot % columns` — and how many slots it
// takes: one, or two, three or four for a panorama, or the whole row. Rust's
// layout (`library_ui::layout`) is the one place these are computed.
slot: int,
span: int,
} }
// TRACES: FR-CAT-3 | FR-CAT-4 | FR-CAT-5 | FR-CAT-6 | FR-CAT-7 | FR-CULL-4 | FR-CULL-5 // TRACES: FR-CAT-3 | FR-CAT-4 | FR-CAT-5 | FR-CAT-6 | FR-CAT-7 | FR-CULL-4 | FR-CULL-5
@@ -535,8 +542,14 @@ export global Library {
callback library-pinch-cells(float); callback library-pinch-cells(float);
/// A pinch on the grid has begun, so the press that opened it was not one. /// A pinch on the grid has begun, so the press that opened it was not one.
callback library-pinch-started(); callback library-pinch-started();
/// The slot the grid's first row should show — a slot, not an ordinal:
/// see `LibraryCell.slot`.
in property <int> library-scroll-to: 0; in property <int> library-scroll-to: 0;
in property <int> library-scroll-token: 0; in property <int> library-scroll-token: 0;
/// TRACES: FR-MRG-6
/// The slots the whole list takes, which a panorama's cell makes more
/// than its count of photographs: what the scrollbar spans.
in property <int> library-total-slots: 0;
/// Which row of the loaded window is the photograph currently open in /// Which row of the loaded window is the photograph currently open in
/// develop, so the roll can mark it. `-1` when it is not in the window. /// develop, so the roll can mark it. `-1` when it is not in the window.
/// Rust re-finds it on every reload, so it follows the photograph and not /// Rust re-finds it on every reload, so it follows the photograph and not
@@ -559,6 +572,7 @@ export global Library {
/// what the roll happens to hold. /// what the roll happens to hold.
callback library-roll-step(int); callback library-roll-step(int);
callback library-columns-changed(); callback library-columns-changed();
/// The first visible row's first slot.
callback library-scrolled(int); callback library-scrolled(int);
/// How many cells the grid's viewport shows at once. Rust sizes the /// How many cells the grid's viewport shows at once. Rust sizes the
/// loaded window from it — see `library_ui::window_move`. /// loaded window from it — see `library_ui::window_move`.
@@ -616,7 +630,15 @@ export global Library {
// ordinal. Rust owns it — clamping it needs the library's length, and // ordinal. Rust owns it — clamping it needs the library's length, and
// moving it may have to swap the loaded window underneath. // moving it may have to swap the loaded window underneath.
in property <int> library-cursor: -1; in property <int> library-cursor: -1;
/// The slot the cursor's photograph starts at, for keeping its row in
/// view.
in property <int> library-cursor-slot: -1;
callback library-move-cursor(int, bool); callback library-move-cursor(int, bool);
/// TRACES: FR-MRG-6
/// Move the cursor by rows, which with a panorama's cell in the way is
/// not a fixed number of photographs: Rust asks the layout what is under
/// the cursor that many rows on.
callback library-move-cursor-rows(int, bool);
callback library-open-cursor(); callback library-open-cursor();
/// TRACES: FR-CAT-5 | FR-CAT-6 /// TRACES: FR-CAT-5 | FR-CAT-6
/// Keywording the grid's selection. The catalog has been searchable by /// Keywording the grid's selection. The catalog has been searchable by
@@ -1461,6 +1483,10 @@ export component LibraryGrid inherits Rectangle {
in property <[LibraryCell]> cells; in property <[LibraryCell]> cells;
in property <int> total: 0; in property <int> total: 0;
/// TRACES: FR-MRG-6
/// The slots the whole list takes: `total` where every cell is one slot,
/// more where a panorama spans several or a row ends short of one.
in property <int> total-slots: 0;
in property <bool> scanning: false; in property <bool> scanning: false;
/// The catalog on this device is being opened and checked before the scan /// The catalog on this device is being opened and checked before the scan
/// starts. Distinct from `scanning` for the reason the empty state below /// starts. Distinct from `scanning` for the reason the empty state below
@@ -2036,6 +2062,9 @@ export component LibraryGrid inherits Rectangle {
// and the first arrow press picks up where the view already is rather than // and the first arrow press picks up where the view already is rather than
// teleporting to image zero. // teleporting to image zero.
in property <int> cursor: -1; in property <int> cursor: -1;
/// The slot the cursor's photograph starts at; its row is what is kept
/// in view.
in property <int> cursor-slot: -1;
/// Move the cursor by a number of images; the flag extends the selection /// Move the cursor by a number of images; the flag extends the selection
/// from the anchor instead of replacing it. /// from the anchor instead of replacing it.
/// ///
@@ -2043,6 +2072,10 @@ export component LibraryGrid inherits Rectangle {
/// longer than the library, which Rust clamps — so this file needs to know /// longer than the library, which Rust clamps — so this file needs to know
/// neither how many images there are nor where the loaded window starts. /// neither how many images there are nor where the loaded window starts.
callback move-cursor(int, bool); callback move-cursor(int, bool);
/// Move the cursor by rows. Up and down are not a row's worth of
/// photographs once a panorama spans columns, so Rust finds the one
/// under the cursor that many rows on.
callback move-cursor-rows(int, bool);
/// Open the image under the cursor. `Return`, and the reason the arrows /// Open the image under the cursor. `Return`, and the reason the arrows
/// are worth having: a cull is walk, judge, open, back, without the hand /// are worth having: a cull is walk, judge, open, back, without the hand
/// ever leaving the keyboard. /// ever leaving the keyboard.
@@ -2205,7 +2238,6 @@ export component LibraryGrid inherits Rectangle {
// start-up. Every month then began a row and was announced wherever // start-up. Every month then began a row and was announced wherever
// its first cell fell, mid-row included. // its first cell fell, mid-row included.
init => { root.columns-changed(root.columns); } init => { root.columns-changed(root.columns); }
property <int> row-count: ceil(root.cells.length / max(1, columns));
// --- while a pinch is happening, and just after ----------------------- // --- while a pinch is happening, and just after -----------------------
// //
@@ -2276,7 +2308,8 @@ export component LibraryGrid inherits Rectangle {
property <int> viewport-cells: root.columns * (root.visible-rows + 1); property <int> viewport-cells: root.columns * (root.visible-rows + 1);
changed viewport-cells => { root.viewport-cells-changed(root.viewport-cells); } changed viewport-cells => { root.viewport-cells-changed(root.viewport-cells); }
/// Rows the *whole library* occupies, which is what the scrollbar spans. /// Rows the *whole library* occupies, which is what the scrollbar spans.
property <int> total-rows: ceil(root.total / max(1, columns)); /// Counted in slots, since a panorama's cell takes several.
property <int> total-rows: ceil(max(root.total, root.total-slots) / max(1, columns));
background: Theme.ground; background: Theme.ground;
@@ -3437,21 +3470,19 @@ export component LibraryGrid inherits Rectangle {
return accept; return accept;
} }
if (Keys.chord(event) == "Up" || Keys.chord(event) == "Shift+Up") { if (Keys.chord(event) == "Up" || Keys.chord(event) == "Shift+Up") {
root.move-cursor(-root.columns, event.modifiers.shift); root.move-cursor-rows(-1, event.modifiers.shift);
return accept; return accept;
} }
if (Keys.chord(event) == "Down" || Keys.chord(event) == "Shift+Down") { if (Keys.chord(event) == "Down" || Keys.chord(event) == "Shift+Down") {
root.move-cursor(root.columns, event.modifiers.shift); root.move-cursor-rows(1, event.modifiers.shift);
return accept; return accept;
} }
if (Keys.chord(event) == "PageUp" || Keys.chord(event) == "Shift+PageUp") { if (Keys.chord(event) == "PageUp" || Keys.chord(event) == "Shift+PageUp") {
root.move-cursor(-root.columns * root.visible-rows, root.move-cursor-rows(-root.visible-rows, event.modifiers.shift);
event.modifiers.shift);
return accept; return accept;
} }
if (Keys.chord(event) == "PageDown" || Keys.chord(event) == "Shift+PageDown") { if (Keys.chord(event) == "PageDown" || Keys.chord(event) == "Shift+PageDown") {
root.move-cursor(root.columns * root.visible-rows, root.move-cursor-rows(root.visible-rows, event.modifiers.shift);
event.modifiers.shift);
return accept; return accept;
} }
// A step longer than the library, clamped at the far end. // A step longer than the library, clamped at the far end.
@@ -3603,7 +3634,7 @@ export component LibraryGrid inherits Rectangle {
// on screen moves nothing at all, and one that has just left // on screen moves nothing at all, and one that has just left
// brings in exactly its own row. // brings in exactly its own row.
property <length> pitch: root.cell-size + Theme.gap; property <length> pitch: root.cell-size + Theme.gap;
property <int> cursor-row: floor(root.cursor / max(1, root.columns)); property <int> cursor-row: floor(max(0, root.cursor-slot) / max(1, root.columns));
changed cursor-row => { self.reveal(); } changed cursor-row => { self.reveal(); }
function reveal() { function reveal() {
@@ -3722,7 +3753,7 @@ export component LibraryGrid inherits Rectangle {
// Sits in the gap above its row, so it labels the row // Sits in the gap above its row, so it labels the row
// rather than displacing it. // rather than displacing it.
y: Theme.gap y: Theme.gap
+ floor((i + root.offset) / root.columns) * (root.cell-size + Theme.gap) + floor(cell.slot / root.columns) * (root.cell-size + Theme.gap)
- 15px; - 15px;
width: parent.width - 2 * Theme.gap; width: parent.width - 2 * Theme.gap;
text: cell.period-heading; text: cell.period-heading;
@@ -3756,10 +3787,12 @@ export component LibraryGrid inherits Rectangle {
// Cells are positioned at their **absolute** place in the // Cells are positioned at their **absolute** place in the
// library, not their index in the loaded window: the window // library, not their index in the loaded window: the window
// starts at `offset`, so a cell drawn at window-index 0 belongs // starts at `offset`, so a cell drawn at window-index 0 belongs
// wherever `offset` sits in the full grid. // wherever `offset` sits in the full grid. That place is its
x: Theme.gap + mod(i + root.offset, root.columns) * (root.cell-size + Theme.gap); // slot, which Rust computes (FR-MRG-6): a panorama before it
y: Theme.gap + floor((i + root.offset) / root.columns) * (root.cell-size + Theme.gap); // pushes it along, and a panorama itself spans `span` slots.
width: root.cell-size; x: Theme.gap + mod(cell.slot, root.columns) * (root.cell-size + Theme.gap);
y: Theme.gap + floor(cell.slot / root.columns) * (root.cell-size + Theme.gap);
width: max(1, cell.span) * root.cell-size + (max(1, cell.span) - 1) * Theme.gap;
height: root.cell-size; height: root.cell-size;
// Copy, not move: dropping into a collection files the // Copy, not move: dropping into a collection files the
@@ -4433,12 +4466,14 @@ export component LibraryGrid inherits Rectangle {
visible: root.held-row >= 0; visible: root.held-row >= 0;
x: Theme.gap x: Theme.gap
+ mod(root.held-row + root.offset, root.columns) * self.pitch + mod(root.cells[root.held-row].slot, root.columns) * self.pitch
- self.reach; - self.reach;
y: Theme.gap y: Theme.gap
+ floor((root.held-row + root.offset) / root.columns) * self.pitch + floor(root.cells[root.held-row].slot / root.columns) * self.pitch
- self.reach; - self.reach;
width: root.cell-size + 2 * self.reach; property <int> span: max(1, root.cells[root.held-row].span);
width: self.span * root.cell-size + (self.span - 1) * Theme.gap
+ 2 * self.reach;
height: root.cell-size + 2 * self.reach; height: root.cell-size + 2 * self.reach;
animate x, y, width, height { duration: 120ms; easing: ease-out; } animate x, y, width, height { duration: 120ms; easing: ease-out; }