dtourolle
a8043e6827
With Skia drawing pre-rotated on wgpu's Vulkan swapchain, Android no longer needs to draw with Skia over OpenGL, which was the only reason the develop view read its frame back through memory (TD-1). So `unstable-wgpu-29` moves back to the common slint dependency. The android-activity backend then builds `SkiaRenderer::default_wgpu_29`, and `shared_gpu` loses its Android arm. The one wgpu device is handed to Slint through `BackendSelector::require_wgpu_29` on both platforms. `slint::android::init_with_event_listener` runs before `dr_ui::run`, so the selector reaches the Android adapter before its window exists. `renderer-femtovg-wgpu` stays desktop-only, since Android has no FemtoVG. The two `#[cfg(target_os = "android")]` readbacks in `develop::render` (the frame through `export_pixels` and the focus overlay through `read_overlay`) are gone. `read_overlay` stays for the tests that check what the overlay marks. Built for arm64 and release-signed. Not yet run on the tablet.
1001 lines
40 KiB
Rust
1001 lines
40 KiB
Rust
//! TRACES: FR-CULL-3 | NFR-P14
|
|
//! Focus peaking — saying what is sharp, so nobody has to zoom in to find out.
|
|
//!
|
|
//! # What this is for
|
|
//!
|
|
//! FR-CULL-3 lists three raw-truth overlays and gives the reason for this one
|
|
//! plainly: peaking "removes the largest single source of culling latency from
|
|
//! the critical path". Checking focus by zooming to 1:1 costs a render, a pan
|
|
//! to the subject's eye, and a zoom back, per frame, on a folder of three
|
|
//! thousand. D11 keeps 1:1 available for certainty; this is what makes reaching
|
|
//! for it the exception.
|
|
//!
|
|
//! It is *raw*-truth for the same reason the histogram beside it is: what this
|
|
//! measures is the frame the develop pipeline rendered from sensor data
|
|
//! through the demosaic, not the camera's embedded JPEG. A JPEG has already
|
|
//! been sharpened by the body, at a strength and radius nobody outside the
|
|
//! manufacturer knows, and peaking on one measures that sharpening at least as
|
|
//! much as it measures the lens.
|
|
//!
|
|
//! # Why a layer, and not a tint in the picture
|
|
//!
|
|
//! The first shape tried was the obvious one — read the display frame, write
|
|
//! the same frame with marked pixels replaced, hand *that* to the compositor.
|
|
//! It is wrong, and `app.slint` had already written down why, on the region
|
|
//! overlay it composites over the canvas: a diagnostic "must not reach the
|
|
//! histogram, an export, or the texture the develop pass hands the
|
|
//! compositor".
|
|
//!
|
|
//! All three would have happened. `HistogramPass` counts whatever texture the
|
|
//! session last rendered, so a red mark on every in-focus edge would have
|
|
//! arrived in the histogram as a red spike and in the clipping figure as blown
|
|
//! highlights. So this pass writes its **own** texture, transparent everywhere
|
|
//! except where something is in focus, and the interface lays it over the
|
|
//! canvas. The photograph is untouched by construction rather than by care.
|
|
//!
|
|
//! # Why it stays on the GPU
|
|
//!
|
|
//! Per-pixel work over the whole frame, every settled frame, is exactly what
|
|
//! ARCH §6.1 exists about: the measurement on this project puts a 4K readback
|
|
//! at 7.4 ms against a 0.28 ms compute pass, 96% of it transfer. The marks are
|
|
//! produced where the pixels already are and handed to the compositor as a
|
|
//! texture, and nothing in this file can read a pixel back on the display path
|
|
//! — see [`FocusPeakPass::read_overlay`] for the one transfer that exists and
|
|
//! who is allowed to call it.
|
|
//!
|
|
//! # Two textures, alternating
|
|
//!
|
|
//! For the reason [`crate::AdjustPass`] keeps two: Slint decides whether to
|
|
//! repaint by comparing the image property against its previous value, and two
|
|
//! images wrapping the same `wgpu::Texture` compare equal. A pass that always
|
|
//! wrote one texture would compute a new overlay every frame and never once be
|
|
//! asked to show it.
|
|
//!
|
|
//! # What it costs
|
|
//!
|
|
//! One dispatch, nine texture loads per pixel, no readback and no
|
|
//! reallocation at a steady viewport size. NFR-P14 allows 100 ms after the
|
|
//! preview on desktop and 150 ms on Android; this is two orders of magnitude
|
|
//! inside that, and the assertion in `overlay_is_ready_well_inside_the_budget`
|
|
//! is what keeps the claim honest rather than remembered.
|
|
|
|
use wgpu::util::DeviceExt as _;
|
|
|
|
use crate::readback::await_mapping;
|
|
use crate::{GpuContext, GpuError};
|
|
|
|
/// How much local contrast counts as focus.
|
|
///
|
|
/// Three steps rather than a slider, because the number underneath is not one
|
|
/// a photographer can reason about and the choice being made is coarse: *this
|
|
/// frame is noisy, mark less* or *this subject is low-contrast, mark more*.
|
|
/// Every camera that offers peaking offers it this way.
|
|
#[derive(Copy, Clone, Debug, Default, PartialEq, Eq, Hash)]
|
|
pub enum PeakSensitivity {
|
|
/// The high-ISO setting. Marks only edges nothing but focus explains.
|
|
Low,
|
|
#[default]
|
|
Medium,
|
|
/// For a low-contrast subject — fur, fabric, distant foliage — at the
|
|
/// price of marking noise as well.
|
|
High,
|
|
}
|
|
|
|
impl PeakSensitivity {
|
|
/// The luma difference, 0..1, at which a pixel is called in focus.
|
|
///
|
|
/// **Where these three numbers come from.** A hard one-pixel step of
|
|
/// height *D* produces a response of `0.375 D` (see the shader), so a
|
|
/// threshold *t* marks any sharp edge whose contrast exceeds `t / 0.375`.
|
|
/// At `Medium` that is 0.107 in luma — about 27 code values — which is a
|
|
/// perfectly ordinary edge and well above what a photograph's own texture
|
|
/// produces by accident.
|
|
///
|
|
/// **And what bounds them from below is noise, not taste.** Sensor noise
|
|
/// surviving into an 8-bit render is a few code values; call it a standard
|
|
/// deviation of 0.008. The response subtracts a mean of eight independent
|
|
/// neighbours from one sample, so its standard deviation is
|
|
/// `0.008 * sqrt(1 + 1/8)` = 0.0085. `Medium` sits 4.7 of those out, which
|
|
/// a normal tail puts at roughly one pixel in a million; `High` sits 2.4
|
|
/// out, which is about one pixel in a hundred — visible as a dusting on a
|
|
/// noisy frame, which is the trade the setting is named for. `Low` is 9.4
|
|
/// out and will not mark noise at all.
|
|
///
|
|
/// Noise reduction, if the edit has any, has already run by the time this
|
|
/// pass sees the frame, so those figures are the pessimistic end.
|
|
pub fn threshold(self) -> f32 {
|
|
match self {
|
|
PeakSensitivity::Low => 0.080,
|
|
PeakSensitivity::Medium => 0.040,
|
|
PeakSensitivity::High => 0.020,
|
|
}
|
|
}
|
|
}
|
|
|
|
/// What colour the marks are drawn in.
|
|
///
|
|
/// A choice rather than a constant, and the reason is NFR-A11Y-3's: status
|
|
/// must not rest on hue alone. An overlay's information *is* positional — it
|
|
/// says where, not what — so the requirement is not violated by there being a
|
|
/// colour; it would be violated by there being only one, because a red mark on
|
|
/// a red jersey conveys nothing, and a photographer with a red-green
|
|
/// deficiency looking at foliage is in the same position permanently.
|
|
///
|
|
/// Four fully saturated choices, no mixtures. A desaturated mark has to
|
|
/// compete with the photograph for the same colours, which is the one thing a
|
|
/// mark must not do.
|
|
#[derive(Copy, Clone, Debug, Default, PartialEq, Eq, Hash)]
|
|
pub enum PeakColour {
|
|
/// The convention, and what most cameras do.
|
|
#[default]
|
|
Red,
|
|
/// For a red or warm subject, and the most visible of the four on a dark
|
|
/// frame.
|
|
Yellow,
|
|
/// For a warm photograph as a whole, and the choice that survives a
|
|
/// red-green deficiency.
|
|
Cyan,
|
|
/// For a cool or green photograph — the hue photographs contain least.
|
|
Magenta,
|
|
}
|
|
|
|
impl PeakColour {
|
|
/// The mark, as the encoded triple the overlay is written in.
|
|
pub fn rgb(self) -> [f32; 3] {
|
|
match self {
|
|
PeakColour::Red => [1.0, 0.0, 0.0],
|
|
PeakColour::Yellow => [1.0, 1.0, 0.0],
|
|
PeakColour::Cyan => [0.0, 1.0, 1.0],
|
|
PeakColour::Magenta => [1.0, 0.0, 1.0],
|
|
}
|
|
}
|
|
}
|
|
|
|
/// TRACES: FR-CULL-3
|
|
/// Everything the photographer chose about the overlay.
|
|
///
|
|
/// One value rather than two arguments, because the two travel together
|
|
/// everywhere — into the session, into the sidecar-free view state, back out
|
|
/// to the panel — and a pair that is always passed together is a type.
|
|
#[derive(Copy, Clone, Debug, Default, PartialEq, Eq, Hash)]
|
|
pub struct FocusPeaking {
|
|
pub sensitivity: PeakSensitivity,
|
|
pub colour: PeakColour,
|
|
}
|
|
|
|
/// The dispatch's view of the frame. Padded to std140's 16 bytes before the
|
|
/// marker, exactly as the shader's `Params` declares it.
|
|
#[repr(C)]
|
|
#[derive(Copy, Clone, bytemuck::Pod, bytemuck::Zeroable)]
|
|
struct Params {
|
|
width: u32,
|
|
height: u32,
|
|
threshold: f32,
|
|
pad_0: u32,
|
|
marker: [f32; 4],
|
|
}
|
|
|
|
/// One overlay texture.
|
|
struct Layer {
|
|
texture: wgpu::Texture,
|
|
view: wgpu::TextureView,
|
|
width: u32,
|
|
height: u32,
|
|
}
|
|
|
|
/// TRACES: FR-CULL-3 | NFR-P14
|
|
/// Produces the focus-peaking overlay for a rendered frame.
|
|
pub struct FocusPeakPass {
|
|
ctx: GpuContext,
|
|
pipeline: wgpu::ComputePipeline,
|
|
bind_group_layout: wgpu::BindGroupLayout,
|
|
params: wgpu::Buffer,
|
|
/// Alternating overlay textures — see the module documentation for why
|
|
/// there are two rather than one.
|
|
layers: [Option<Layer>; 2],
|
|
current: usize,
|
|
/// Textures allocated since this pass was created. Exists to be asserted
|
|
/// on: an overlay reallocated per frame instead of per resize costs a
|
|
/// great deal of bandwidth and looks identical in the picture, which is
|
|
/// the shape of regression only a counter can see.
|
|
allocations: usize,
|
|
dispatches: usize,
|
|
}
|
|
|
|
impl FocusPeakPass {
|
|
/// The overlay's format.
|
|
///
|
|
/// The same `Rgba8Unorm` [`crate::AdjustPass`] writes, and for the same
|
|
/// non-negotiable reason: it is one of the two formats Slint's texture
|
|
/// import accepts. The alpha channel is what carries the overlay, so the
|
|
/// eight bits it has are seven more than this needs.
|
|
pub const FORMAT: wgpu::TextureFormat = wgpu::TextureFormat::Rgba8Unorm;
|
|
|
|
pub fn new(ctx: &GpuContext) -> Result<Self, GpuError> {
|
|
// A validation error here is a bug in the shader beside this file
|
|
// rather than anything a user did, so it is caught in an error scope
|
|
// and returned — wgpu's default handler panics.
|
|
let scope = ctx.device.push_error_scope(wgpu::ErrorFilter::Validation);
|
|
|
|
let module = ctx
|
|
.device
|
|
.create_shader_module(wgpu::ShaderModuleDescriptor {
|
|
label: Some("focus-peak"),
|
|
source: wgpu::ShaderSource::Wgsl(include_str!("shaders/focus_peak.wgsl").into()),
|
|
});
|
|
|
|
let bind_group_layout =
|
|
ctx.device
|
|
.create_bind_group_layout(&wgpu::BindGroupLayoutDescriptor {
|
|
label: Some("focus-peak-bgl"),
|
|
entries: &[
|
|
// The rendered frame, read with `textureLoad` — the
|
|
// same texture the compositor is showing, so what is
|
|
// measured is what is on screen.
|
|
wgpu::BindGroupLayoutEntry {
|
|
binding: 0,
|
|
visibility: wgpu::ShaderStages::COMPUTE,
|
|
ty: wgpu::BindingType::Texture {
|
|
sample_type: wgpu::TextureSampleType::Float { filterable: true },
|
|
view_dimension: wgpu::TextureViewDimension::D2,
|
|
multisampled: false,
|
|
},
|
|
count: None,
|
|
},
|
|
wgpu::BindGroupLayoutEntry {
|
|
binding: 1,
|
|
visibility: wgpu::ShaderStages::COMPUTE,
|
|
ty: wgpu::BindingType::Buffer {
|
|
ty: wgpu::BufferBindingType::Uniform,
|
|
has_dynamic_offset: false,
|
|
min_binding_size: None,
|
|
},
|
|
count: None,
|
|
},
|
|
wgpu::BindGroupLayoutEntry {
|
|
binding: 2,
|
|
visibility: wgpu::ShaderStages::COMPUTE,
|
|
ty: wgpu::BindingType::StorageTexture {
|
|
access: wgpu::StorageTextureAccess::WriteOnly,
|
|
format: Self::FORMAT,
|
|
view_dimension: wgpu::TextureViewDimension::D2,
|
|
},
|
|
count: None,
|
|
},
|
|
],
|
|
});
|
|
|
|
let layout = ctx
|
|
.device
|
|
.create_pipeline_layout(&wgpu::PipelineLayoutDescriptor {
|
|
label: Some("focus-peak-layout"),
|
|
bind_group_layouts: &[Some(&bind_group_layout)],
|
|
immediate_size: 0,
|
|
});
|
|
|
|
let pipeline = ctx
|
|
.device
|
|
.create_compute_pipeline(&wgpu::ComputePipelineDescriptor {
|
|
label: Some("focus-peak-pipeline"),
|
|
layout: Some(&layout),
|
|
module: &module,
|
|
entry_point: Some("main"),
|
|
compilation_options: Default::default(),
|
|
cache: None,
|
|
});
|
|
|
|
if let Some(err) = pollster::block_on(scope.pop()) {
|
|
return Err(GpuError::ShaderCompilation(err.to_string()));
|
|
}
|
|
|
|
let params = ctx
|
|
.device
|
|
.create_buffer_init(&wgpu::util::BufferInitDescriptor {
|
|
label: Some("focus-peak-params"),
|
|
contents: bytemuck::bytes_of(&Params {
|
|
width: 0,
|
|
height: 0,
|
|
threshold: PeakSensitivity::default().threshold(),
|
|
pad_0: 0,
|
|
marker: [0.0; 4],
|
|
}),
|
|
usage: wgpu::BufferUsages::UNIFORM | wgpu::BufferUsages::COPY_DST,
|
|
});
|
|
|
|
Ok(Self {
|
|
ctx: ctx.clone(),
|
|
pipeline,
|
|
bind_group_layout,
|
|
params,
|
|
layers: [None, None],
|
|
current: 0,
|
|
allocations: 0,
|
|
dispatches: 0,
|
|
})
|
|
}
|
|
|
|
/// TRACES: FR-CULL-3 | NFR-P14
|
|
/// Mark the in-focus regions of `frame`, returning the overlay to lay
|
|
/// over it.
|
|
///
|
|
/// `frame` must carry `TEXTURE_BINDING`, which [`crate::AdjustPass`]'s
|
|
/// output does because the compositor samples it.
|
|
///
|
|
/// **Give this a settled frame, not a draft one.** The measure is the
|
|
/// energy in the top octave of what it is handed, so it is a statement
|
|
/// about a particular sampling grid: at half resolution — which is what a
|
|
/// draft frame is rendered at — a defocused edge spanning four pixels
|
|
/// spans two, which is the signature of a sharp one. Peaking a draft frame
|
|
/// would mark the out-of-focus background of every photograph, briefly,
|
|
/// during every drag. The interface runs this where it runs the histogram,
|
|
/// on the settled frame, and for the same class of reason.
|
|
pub fn render(
|
|
&mut self,
|
|
frame: &wgpu::Texture,
|
|
settings: FocusPeaking,
|
|
) -> Result<&wgpu::Texture, GpuError> {
|
|
// No zero-size guard: wgpu will not create a texture with a zero
|
|
// extent, so a frame that exists has at least one pixel in it.
|
|
let (width, height) = (frame.width(), frame.height());
|
|
self.ensure_layer(width, height);
|
|
|
|
let rgb = settings.colour.rgb();
|
|
self.ctx.queue.write_buffer(
|
|
&self.params,
|
|
0,
|
|
bytemuck::bytes_of(&Params {
|
|
width,
|
|
height,
|
|
threshold: settings.sensitivity.threshold(),
|
|
pad_0: 0,
|
|
marker: [rgb[0], rgb[1], rgb[2], 1.0],
|
|
}),
|
|
);
|
|
|
|
let layer = self.layers[self.current]
|
|
.as_ref()
|
|
.expect("ensure_layer just built it");
|
|
let source = frame.create_view(&Default::default());
|
|
let bind_group = self
|
|
.ctx
|
|
.device
|
|
.create_bind_group(&wgpu::BindGroupDescriptor {
|
|
label: Some("focus-peak-bg"),
|
|
layout: &self.bind_group_layout,
|
|
entries: &[
|
|
wgpu::BindGroupEntry {
|
|
binding: 0,
|
|
resource: wgpu::BindingResource::TextureView(&source),
|
|
},
|
|
wgpu::BindGroupEntry {
|
|
binding: 1,
|
|
resource: self.params.as_entire_binding(),
|
|
},
|
|
wgpu::BindGroupEntry {
|
|
binding: 2,
|
|
resource: wgpu::BindingResource::TextureView(&layer.view),
|
|
},
|
|
],
|
|
});
|
|
|
|
let mut enc = self
|
|
.ctx
|
|
.device
|
|
.create_command_encoder(&wgpu::CommandEncoderDescriptor {
|
|
label: Some("focus-peak-encoder"),
|
|
});
|
|
{
|
|
let mut pass = enc.begin_compute_pass(&wgpu::ComputePassDescriptor {
|
|
label: Some("focus-peak-pass"),
|
|
timestamp_writes: None,
|
|
});
|
|
pass.set_pipeline(&self.pipeline);
|
|
pass.set_bind_group(0, &bind_group, &[]);
|
|
pass.dispatch_workgroups(width.div_ceil(8), height.div_ceil(8), 1);
|
|
}
|
|
// Submitted on its own queue entry after the render that produced
|
|
// `frame`. Submission order is the whole of the synchronisation, as it
|
|
// is between the fused pass and the detail chain: one queue, and this
|
|
// reads what that wrote.
|
|
self.ctx.queue.submit(Some(enc.finish()));
|
|
self.dispatches += 1;
|
|
|
|
Ok(&self.layers[self.current]
|
|
.as_ref()
|
|
.expect("just written")
|
|
.texture)
|
|
}
|
|
|
|
/// The overlay the last [`Self::render`] wrote, if there has been one.
|
|
pub fn overlay(&self) -> Option<&wgpu::Texture> {
|
|
self.layers[self.current].as_ref().map(|l| &l.texture)
|
|
}
|
|
|
|
/// Forget the overlay, so nothing stale is composited.
|
|
///
|
|
/// Called when the photograph changes and when peaking is switched off.
|
|
/// The alternative — leaving the last overlay resident and merely not
|
|
/// drawing it — is one interface bug away from laying one photograph's
|
|
/// focus marks over another's, which is the single worst thing an
|
|
/// instrument like this can do.
|
|
pub fn clear(&mut self) {
|
|
self.layers = [None, None];
|
|
self.current = 0;
|
|
}
|
|
|
|
/// Overlay textures allocated since this pass was created.
|
|
///
|
|
/// For tests. A steady viewport must not move this number, and the only
|
|
/// evidence of that is a counter — a per-frame reallocation renders
|
|
/// identically to a cached one.
|
|
pub fn allocations(&self) -> usize {
|
|
self.allocations
|
|
}
|
|
|
|
/// Dispatches encoded since this pass was created.
|
|
///
|
|
/// The other half of [`Self::allocations`]: together they say that eight
|
|
/// frames cost eight dispatches and two textures, which is the shape a
|
|
/// steady viewport is supposed to have.
|
|
pub fn dispatches(&self) -> usize {
|
|
self.dispatches
|
|
}
|
|
|
|
/// Make sure the current slot holds a texture of this size.
|
|
///
|
|
/// Rotates first, so consecutive frames land in different textures — see
|
|
/// the module documentation. A size change drops both, because neither
|
|
/// fits any more and a stale one of the wrong size would be composited
|
|
/// stretched over the new frame.
|
|
fn ensure_layer(&mut self, width: u32, height: u32) {
|
|
self.current ^= 1;
|
|
let fits = self.layers[self.current]
|
|
.as_ref()
|
|
.is_some_and(|l| l.width == width && l.height == height);
|
|
if fits {
|
|
return;
|
|
}
|
|
let texture = self.ctx.device.create_texture(&wgpu::TextureDescriptor {
|
|
label: Some("focus-peak-overlay"),
|
|
size: wgpu::Extent3d {
|
|
width,
|
|
height,
|
|
depth_or_array_layers: 1,
|
|
},
|
|
mip_level_count: 1,
|
|
sample_count: 1,
|
|
dimension: wgpu::TextureDimension::D2,
|
|
format: Self::FORMAT,
|
|
// STORAGE_BINDING to be written by the dispatch and
|
|
// TEXTURE_BINDING to be sampled by the compositor.
|
|
// RENDER_ATTACHMENT is not used by anything here and is required
|
|
// anyway: Slint rejects an imported texture without it. COPY_SRC
|
|
// is for `read_overlay` and the tests that call it.
|
|
usage: wgpu::TextureUsages::STORAGE_BINDING
|
|
| wgpu::TextureUsages::TEXTURE_BINDING
|
|
| wgpu::TextureUsages::RENDER_ATTACHMENT
|
|
| wgpu::TextureUsages::COPY_SRC,
|
|
view_formats: &[],
|
|
});
|
|
let view = texture.create_view(&Default::default());
|
|
self.layers[self.current] = Some(Layer {
|
|
texture,
|
|
view,
|
|
width,
|
|
height,
|
|
});
|
|
self.allocations += 1;
|
|
}
|
|
|
|
/// TRACES: AC-8
|
|
/// Copy the overlay to the CPU, as RGBA8 rows with no padding.
|
|
///
|
|
/// **The tests below are the only caller, and never the display path.** An
|
|
/// overlay is a claim about which pixels are sharp, and there is no way to
|
|
/// check that claim without looking at the pixels. On screen, on desktop
|
|
/// and Android alike, the overlay reaches the compositor as a texture and
|
|
/// ARCH §6.1 holds. (Android read it back here until TD-1 was paid off.)
|
|
pub fn read_overlay(&self) -> Result<(Vec<u8>, u32, u32), GpuError> {
|
|
let Some(layer) = self.layers[self.current].as_ref() else {
|
|
return Err(GpuError::Readback("no overlay has been rendered".into()));
|
|
};
|
|
let (w, h) = (layer.width, layer.height);
|
|
|
|
let unpadded = w * 4;
|
|
let align = wgpu::COPY_BYTES_PER_ROW_ALIGNMENT;
|
|
let padded = unpadded.div_ceil(align) * align;
|
|
|
|
let buf = self.ctx.device.create_buffer(&wgpu::BufferDescriptor {
|
|
label: Some("focus-peak-readback"),
|
|
size: (padded * h) as u64,
|
|
usage: wgpu::BufferUsages::COPY_DST | wgpu::BufferUsages::MAP_READ,
|
|
mapped_at_creation: false,
|
|
});
|
|
|
|
let mut enc = self.ctx.device.create_command_encoder(&Default::default());
|
|
enc.copy_texture_to_buffer(
|
|
wgpu::TexelCopyTextureInfo {
|
|
texture: &layer.texture,
|
|
mip_level: 0,
|
|
origin: wgpu::Origin3d::ZERO,
|
|
aspect: wgpu::TextureAspect::All,
|
|
},
|
|
wgpu::TexelCopyBufferInfo {
|
|
buffer: &buf,
|
|
layout: wgpu::TexelCopyBufferLayout {
|
|
offset: 0,
|
|
bytes_per_row: Some(padded),
|
|
rows_per_image: Some(h),
|
|
},
|
|
},
|
|
wgpu::Extent3d {
|
|
width: w,
|
|
height: h,
|
|
depth_or_array_layers: 1,
|
|
},
|
|
);
|
|
self.ctx.queue.submit(Some(enc.finish()));
|
|
|
|
let slice = buf.slice(..);
|
|
let (tx, rx) = std::sync::mpsc::channel();
|
|
slice.map_async(wgpu::MapMode::Read, move |r| {
|
|
let _ = tx.send(r);
|
|
});
|
|
await_mapping(&self.ctx, &rx)?;
|
|
|
|
let data = slice.get_mapped_range();
|
|
let mut out = Vec::with_capacity((unpadded * h) as usize);
|
|
for row in 0..h {
|
|
let start = (row * padded) as usize;
|
|
out.extend_from_slice(&data[start..start + unpadded as usize]);
|
|
}
|
|
drop(data);
|
|
buf.unmap();
|
|
Ok((out, w, h))
|
|
}
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
fn ctx() -> Option<GpuContext> {
|
|
match pollster::block_on(GpuContext::new_headless()) {
|
|
Ok(c) => Some(c),
|
|
Err(e) => {
|
|
eprintln!("skipping: no GPU adapter ({e})");
|
|
None
|
|
}
|
|
}
|
|
}
|
|
|
|
/// Upload a greyscale frame the pass can read, the way `AdjustPass` hands
|
|
/// its output over.
|
|
fn frame(ctx: &GpuContext, grey: &[f32], width: u32, height: u32) -> wgpu::Texture {
|
|
let rgba: Vec<u8> = grey
|
|
.iter()
|
|
.flat_map(|v| {
|
|
let c = (v.clamp(0.0, 1.0) * 255.0).round() as u8;
|
|
[c, c, c, 255]
|
|
})
|
|
.collect();
|
|
let tex = ctx.device.create_texture(&wgpu::TextureDescriptor {
|
|
label: Some("focus-peak-test-frame"),
|
|
size: wgpu::Extent3d {
|
|
width,
|
|
height,
|
|
depth_or_array_layers: 1,
|
|
},
|
|
mip_level_count: 1,
|
|
sample_count: 1,
|
|
dimension: wgpu::TextureDimension::D2,
|
|
format: wgpu::TextureFormat::Rgba8Unorm,
|
|
usage: wgpu::TextureUsages::TEXTURE_BINDING | wgpu::TextureUsages::COPY_DST,
|
|
view_formats: &[],
|
|
});
|
|
ctx.queue.write_texture(
|
|
wgpu::TexelCopyTextureInfo {
|
|
texture: &tex,
|
|
mip_level: 0,
|
|
origin: wgpu::Origin3d::ZERO,
|
|
aspect: wgpu::TextureAspect::All,
|
|
},
|
|
&rgba,
|
|
wgpu::TexelCopyBufferLayout {
|
|
offset: 0,
|
|
bytes_per_row: Some(width * 4),
|
|
rows_per_image: Some(height),
|
|
},
|
|
wgpu::Extent3d {
|
|
width,
|
|
height,
|
|
depth_or_array_layers: 1,
|
|
},
|
|
);
|
|
ctx.queue.submit(std::iter::empty());
|
|
tex
|
|
}
|
|
|
|
/// A vertical step from `dark` to `bright`, blurred with a Gaussian of
|
|
/// this `sigma` in pixels. `sigma == 0.0` is the hard edge.
|
|
///
|
|
/// The profile is evaluated analytically rather than by convolving a
|
|
/// sampled image, so "the same edge, out of focus" is exactly that and not
|
|
/// a second thing the test would also have to trust.
|
|
fn step(width: u32, height: u32, dark: f32, bright: f32, sigma: f32) -> Vec<f32> {
|
|
let edge = width as f32 / 2.0 - 0.5;
|
|
(0..height)
|
|
.flat_map(|_| {
|
|
(0..width).map(move |x| {
|
|
let d = x as f32 - edge;
|
|
let t = if sigma <= 0.0 {
|
|
if d < 0.0 {
|
|
0.0
|
|
} else {
|
|
1.0
|
|
}
|
|
} else {
|
|
// The error function, which is what a Gaussian blur of
|
|
// a step is, via a tanh approximation good to ~1e-4 —
|
|
// ample, since the assertions below are about orders
|
|
// of magnitude rather than about the fourth digit.
|
|
0.5 * (1.0 + (1.202_7 * (d / sigma)).tanh())
|
|
};
|
|
dark + t * (bright - dark)
|
|
})
|
|
})
|
|
.collect()
|
|
}
|
|
|
|
/// How many pixels of an overlay carry a mark, and what colour they are.
|
|
fn marks(pixels: &[u8]) -> (usize, Vec<[u8; 3]>) {
|
|
let mut count = 0;
|
|
let mut colours: Vec<[u8; 3]> = Vec::new();
|
|
for px in pixels.chunks_exact(4) {
|
|
if px[3] != 0 {
|
|
count += 1;
|
|
let c = [px[0], px[1], px[2]];
|
|
if !colours.contains(&c) {
|
|
colours.push(c);
|
|
}
|
|
}
|
|
}
|
|
(count, colours)
|
|
}
|
|
|
|
fn peak(
|
|
ctx: &GpuContext,
|
|
pass: &mut FocusPeakPass,
|
|
grey: &[f32],
|
|
w: u32,
|
|
h: u32,
|
|
settings: FocusPeaking,
|
|
) -> Vec<u8> {
|
|
let tex = frame(ctx, grey, w, h);
|
|
pass.render(&tex, settings).expect("peak");
|
|
pass.read_overlay().expect("readback").0
|
|
}
|
|
|
|
#[test]
|
|
fn the_three_sensitivities_are_ordered_and_none_of_them_is_zero() {
|
|
// Arithmetic, so no device. A threshold of zero marks every pixel of
|
|
// every photograph — including a flat sky, where the response is
|
|
// whatever the last bit of the encoder rounded to — and an overlay
|
|
// that covers the frame says nothing at all.
|
|
let (low, med, high) = (
|
|
PeakSensitivity::Low.threshold(),
|
|
PeakSensitivity::Medium.threshold(),
|
|
PeakSensitivity::High.threshold(),
|
|
);
|
|
assert!(high > 0.0, "a zero threshold marks everything");
|
|
assert!(
|
|
low > med && med > high,
|
|
"more sensitive must mean a lower bar: {low} {med} {high}"
|
|
);
|
|
assert_eq!(PeakSensitivity::default(), PeakSensitivity::Medium);
|
|
}
|
|
|
|
#[test]
|
|
fn every_colour_is_fully_saturated_and_distinct() {
|
|
// The overlay competes with the photograph for attention, and a
|
|
// desaturated mark loses. Each choice must also be a different colour
|
|
// from the others — two entries that render the same would be a menu
|
|
// that lies.
|
|
let mut seen: Vec<[f32; 3]> = Vec::new();
|
|
for c in [
|
|
PeakColour::Red,
|
|
PeakColour::Yellow,
|
|
PeakColour::Cyan,
|
|
PeakColour::Magenta,
|
|
] {
|
|
let rgb = c.rgb();
|
|
assert!(
|
|
rgb.iter().all(|v| *v == 0.0 || *v == 1.0),
|
|
"{c:?} is not a saturated primary: {rgb:?}"
|
|
);
|
|
assert!(rgb.iter().any(|v| *v > 0.0), "{c:?} is black");
|
|
assert!(!seen.contains(&rgb), "{c:?} duplicates another choice");
|
|
seen.push(rgb);
|
|
}
|
|
}
|
|
|
|
#[test]
|
|
fn a_flat_frame_is_marked_nowhere() {
|
|
// The floor. An overlay that marks an empty sky is one a photographer
|
|
// stops believing within a frame, and there is nothing subtle about
|
|
// the failure — it would be the whole picture.
|
|
let Some(ctx) = ctx() else { return };
|
|
let mut pass = FocusPeakPass::new(&ctx).expect("pass");
|
|
|
|
let flat = vec![0.45f32; 64 * 64];
|
|
let px = peak(&ctx, &mut pass, &flat, 64, 64, FocusPeaking::default());
|
|
assert_eq!(marks(&px).0, 0, "flat grey produced marks");
|
|
}
|
|
|
|
#[test]
|
|
fn a_sharp_edge_is_marked_and_the_same_edge_defocused_is_not() {
|
|
// **The claim the whole overlay rests on**, and the one a gradient
|
|
// detector would fail: these two frames have identical contrast and
|
|
// differ only in how many pixels the transition is spread over. If
|
|
// both are marked, the overlay is an edge detector wearing focus
|
|
// peaking's name and it will light up every out-of-focus background
|
|
// ever photographed.
|
|
let Some(ctx) = ctx() else { return };
|
|
let mut pass = FocusPeakPass::new(&ctx).expect("pass");
|
|
let (w, h) = (64u32, 16u32);
|
|
|
|
let sharp = step(w, h, 0.25, 0.75, 0.0);
|
|
let soft = step(w, h, 0.25, 0.75, 2.0);
|
|
|
|
let marked_sharp = marks(&peak(
|
|
&ctx,
|
|
&mut pass,
|
|
&sharp,
|
|
w,
|
|
h,
|
|
FocusPeaking::default(),
|
|
))
|
|
.0;
|
|
let marked_soft = marks(&peak(&ctx, &mut pass, &soft, w, h, FocusPeaking::default())).0;
|
|
|
|
// Two columns of the sharp edge respond — the pixel each side of the
|
|
// transition — so a full-height edge marks 2 per row.
|
|
assert_eq!(
|
|
marked_sharp,
|
|
(2 * h) as usize,
|
|
"a hard edge should mark the column each side of it, on every row"
|
|
);
|
|
assert_eq!(
|
|
marked_soft, 0,
|
|
"the same edge at sigma 2 is out of focus and must not be marked"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn sensitivity_decides_how_faint_an_edge_still_counts() {
|
|
// The setting doing what its name says, from both sides. A hard edge
|
|
// of 0.08 contrast responds at `0.375 * 0.08` = 0.031 — above `High`'s
|
|
// bar of 0.020 and below `Medium`'s of 0.040 — so the same photograph
|
|
// must be marked at one setting and clean at the other. Without this,
|
|
// three menu entries that all behaved identically would pass every
|
|
// other test in this file.
|
|
let Some(ctx) = ctx() else { return };
|
|
let mut pass = FocusPeakPass::new(&ctx).expect("pass");
|
|
let (w, h) = (64u32, 8u32);
|
|
let faint = step(w, h, 0.46, 0.54, 0.0);
|
|
|
|
let at = |pass: &mut FocusPeakPass, sensitivity| {
|
|
marks(&peak(
|
|
&ctx,
|
|
pass,
|
|
&faint,
|
|
w,
|
|
h,
|
|
FocusPeaking {
|
|
sensitivity,
|
|
..Default::default()
|
|
},
|
|
))
|
|
.0
|
|
};
|
|
|
|
assert!(
|
|
at(&mut pass, PeakSensitivity::High) > 0,
|
|
"a faint but sharp edge is exactly what High is for"
|
|
);
|
|
assert_eq!(
|
|
at(&mut pass, PeakSensitivity::Medium),
|
|
0,
|
|
"Medium should hold its bar above an 0.08 edge"
|
|
);
|
|
assert_eq!(at(&mut pass, PeakSensitivity::Low), 0);
|
|
}
|
|
|
|
#[test]
|
|
fn the_mark_is_the_colour_that_was_asked_for_and_the_rest_is_transparent() {
|
|
// Both halves matter. The colour, because a menu that quietly draws
|
|
// red whatever is chosen is worse than not offering the choice; and
|
|
// the transparency, because the overlay is composited over the
|
|
// photograph and an alpha of even 1/255 across the frame is a veil
|
|
// over every judgement made through it.
|
|
let Some(ctx) = ctx() else { return };
|
|
let mut pass = FocusPeakPass::new(&ctx).expect("pass");
|
|
let (w, h) = (32u32, 8u32);
|
|
let sharp = step(w, h, 0.2, 0.8, 0.0);
|
|
|
|
for colour in [
|
|
PeakColour::Red,
|
|
PeakColour::Yellow,
|
|
PeakColour::Cyan,
|
|
PeakColour::Magenta,
|
|
] {
|
|
let px = peak(
|
|
&ctx,
|
|
&mut pass,
|
|
&sharp,
|
|
w,
|
|
h,
|
|
FocusPeaking {
|
|
colour,
|
|
..Default::default()
|
|
},
|
|
);
|
|
let (count, colours) = marks(&px);
|
|
assert!(count > 0, "{colour:?} marked nothing");
|
|
let expected: [u8; 3] = colour.rgb().map(|v| (v * 255.0).round() as u8);
|
|
assert_eq!(colours, vec![expected], "{colour:?} drew the wrong ink");
|
|
|
|
// Everything unmarked is fully transparent *and* black, which is
|
|
// what makes the layer correct whether the compositor treats it as
|
|
// premultiplied or not.
|
|
for texel in px.chunks_exact(4) {
|
|
if texel[3] == 0 {
|
|
assert!(
|
|
texel[..3].iter().all(|c| *c == 0),
|
|
"an unmarked texel carried colour under a zero alpha"
|
|
);
|
|
} else {
|
|
assert_eq!(texel[3], 255, "a mark was drawn part-way transparent");
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
#[test]
|
|
fn the_edge_of_the_frame_is_not_marked_by_its_own_edge() {
|
|
// The clamp in `neighbour`. Wrapping instead would fold the right-hand
|
|
// column of the picture into the left-hand one's neighbourhood, and a
|
|
// photograph whose two sides differ — which is most of them — would be
|
|
// marked down both borders regardless of focus.
|
|
let Some(ctx) = ctx() else { return };
|
|
let mut pass = FocusPeakPass::new(&ctx).expect("pass");
|
|
let (w, h) = (33u32, 17u32);
|
|
|
|
// Dark on the left, bright on the right, with the transition a long
|
|
// way from either border.
|
|
let split = step(w, h, 0.1, 0.9, 0.0);
|
|
let px = peak(&ctx, &mut pass, &split, w, h, FocusPeaking::default());
|
|
|
|
for y in 0..h {
|
|
for x in [0u32, w - 1] {
|
|
let a = px[(((y * w + x) * 4) + 3) as usize];
|
|
assert_eq!(a, 0, "the frame border at ({x}, {y}) was marked");
|
|
}
|
|
}
|
|
// A size that is not a multiple of the 8x8 workgroup, so the edge
|
|
// groups run off the image: every texel must still have been written.
|
|
assert_eq!(px.len(), (w * h * 4) as usize);
|
|
}
|
|
|
|
#[test]
|
|
fn consecutive_overlays_are_different_textures() {
|
|
// Slint compares the image property by value to decide whether to
|
|
// repaint, so two frames wrapping one texture compare equal and the
|
|
// second is never drawn. `AdjustPass` keeps two targets for this
|
|
// reason; the overlay beside it has to do the same or it freezes on
|
|
// whatever it first showed.
|
|
let Some(ctx) = ctx() else { return };
|
|
let mut pass = FocusPeakPass::new(&ctx).expect("pass");
|
|
let grey = vec![0.5f32; 16 * 16];
|
|
|
|
let tex = frame(&ctx, &grey, 16, 16);
|
|
let first =
|
|
std::ptr::from_ref(pass.render(&tex, FocusPeaking::default()).expect("first")) as usize;
|
|
let second = std::ptr::from_ref(pass.render(&tex, FocusPeaking::default()).expect("second"))
|
|
as usize;
|
|
assert_ne!(
|
|
first, second,
|
|
"two consecutive overlays landed in the same texture"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn a_steady_viewport_allocates_nothing_after_the_first_two_frames() {
|
|
// Reallocating a viewport-sized texture per frame costs a great deal
|
|
// of bandwidth and renders identically, which is why this is a counter
|
|
// and not an assertion about the picture. Two allocations, not one:
|
|
// the pair alternates, so the second frame builds the other slot.
|
|
let Some(ctx) = ctx() else { return };
|
|
let mut pass = FocusPeakPass::new(&ctx).expect("pass");
|
|
let grey = vec![0.5f32; 64 * 64];
|
|
let tex = frame(&ctx, &grey, 64, 64);
|
|
|
|
for _ in 0..8 {
|
|
pass.render(&tex, FocusPeaking::default()).expect("render");
|
|
}
|
|
assert_eq!(pass.dispatches(), 8);
|
|
assert_eq!(
|
|
pass.allocations(),
|
|
2,
|
|
"the overlay was reallocated per frame"
|
|
);
|
|
|
|
// A resize is the one thing that must reallocate: a stale overlay of
|
|
// the wrong size would be composited stretched over the new frame.
|
|
let small = vec![0.5f32; 32 * 32];
|
|
let smaller = frame(&ctx, &small, 32, 32);
|
|
pass.render(&smaller, FocusPeaking::default())
|
|
.expect("render");
|
|
assert_eq!(pass.allocations(), 3);
|
|
}
|
|
|
|
#[test]
|
|
fn a_cleared_pass_has_no_overlay_to_composite() {
|
|
// Switching peaking off, or opening a different photograph, must leave
|
|
// nothing behind. One interface bug away from laying one frame's focus
|
|
// marks over another's, which is the worst thing an instrument can do:
|
|
// be confidently about the wrong picture.
|
|
let Some(ctx) = ctx() else { return };
|
|
let mut pass = FocusPeakPass::new(&ctx).expect("pass");
|
|
let grey = vec![0.5f32; 16 * 16];
|
|
let tex = frame(&ctx, &grey, 16, 16);
|
|
pass.render(&tex, FocusPeaking::default()).expect("render");
|
|
assert!(pass.overlay().is_some());
|
|
|
|
pass.clear();
|
|
assert!(pass.overlay().is_none());
|
|
assert!(
|
|
pass.read_overlay().is_err(),
|
|
"a cleared pass must not hand out a stale overlay"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn the_overlay_is_ready_well_inside_its_budget() {
|
|
// NFR-P14: ready within 100 ms of the preview on desktop, 150 ms on
|
|
// Android. The bound asserted here is 50 ms at 4K, which is half the
|
|
// desktop budget and a third of Android's, and is still around two
|
|
// orders of magnitude above what the dispatch actually costs — chosen
|
|
// so that an unrelated machine under load does not fail the suite,
|
|
// while a change that made this a multi-pass or readback-bound
|
|
// operation could not possibly stay under it.
|
|
//
|
|
// The readback is deliberately outside the timed region. It is a
|
|
// property of this test rather than of the overlay, and at 4K it is
|
|
// the 7 ms transfer ARCH §6.1 exists to keep off the frame path.
|
|
let Some(ctx) = ctx() else { return };
|
|
let mut pass = FocusPeakPass::new(&ctx).expect("pass");
|
|
let (w, h) = (3840u32, 2160u32);
|
|
let tex = frame(&ctx, &step(w, h, 0.2, 0.8, 0.0), w, h);
|
|
|
|
// One frame to allocate the layer and warm the pipeline; a resize is
|
|
// not what the budget is about.
|
|
pass.render(&tex, FocusPeaking::default()).expect("warm");
|
|
ctx.device
|
|
.poll(wgpu::PollType::wait_indefinitely())
|
|
.expect("idle");
|
|
|
|
let start = std::time::Instant::now();
|
|
pass.render(&tex, FocusPeaking::default()).expect("render");
|
|
ctx.device
|
|
.poll(wgpu::PollType::wait_indefinitely())
|
|
.expect("idle");
|
|
let elapsed = start.elapsed();
|
|
|
|
assert!(
|
|
elapsed.as_millis() < 50,
|
|
"the overlay took {elapsed:?} at {w}x{h}; NFR-P14 allows 100 ms"
|
|
);
|
|
}
|
|
}
|