dtourolle/DarkRoom
1
0
Fork 0
Code Issues 39 Pull Requests Actions Packages Projects Releases 20 Wiki Activity

Correct converging verticals with a keystone in framing

Browse Source 
There was no perspective transform anywhere in the pipeline: framing
offered a ±45° straighten, quarter turns and flips, and a building shot
looking up kept its leaning walls.

Framing gains a vertical and a horizontal keystone (-100..100). They are
parameters of framing rather than a new stage, so they carry its Compose
attribute, persist in the sidecar under framing, and are withheld from a
default paste exactly as the crop is. In the prologue the keystone runs
after the crop and the straightening and before the stored orientation
and the lens warp, so "vertical" is the photograph's displayed height and
the lens still sees its whole frame.

The map takes the output frame onto a trapezoid inside the source, built
as a homography from four corners and uploaded as three columns in the
framing uniform block (which grows from two vec4s to five). A keystone on
its own therefore never exposes an empty corner and leaves any crop valid.
Combined with a straightening angle the empty area is a pulled-back
quadrilateral the closed-form inscribed rectangle cannot describe, so
max_inscribed_crop searches for the largest centred rectangle whose
corners all have a source pixel behind them. source_at and output_at
apply the same map, so masks, gradients and spot handles follow it.
This commit is contained in:
dtourolle
2026-09-24 22:13:11 -04:00
parent ff89a4fa21
commit 5a500118ae
6 changed files with 786 additions and 36 deletions
Download Patch File Download Diff File Expand all files Collapse all files
core/dr-pipeline/src/framing.rs
+681 -9
View File
@@ -27,6 +27,33 @@
//! chain exactly the space it documents: normalised, centred, `r == 1` at the
//! corner. Neither stage needs to know the other exists.
//!
//! # Perspective sits inside framing (FR-DEV-20)
//!
//! A keystone correction is composition too — straightening converging
//! verticals reframes the photograph — so it is a step *of* framing rather
//! than a stage beside it, and inherits framing's `Compose` attribute, its
//! place in the sidecar and its exclusion from a default paste. Expanded, the
//! chain reads:
//!
//! ```text
//! output pixel → crop → straighten → perspective → orientation → warp (lens) → sample
//! ```
//!
//! After the straightening, because the angle is a nudge applied to the
//! corrected picture: the verticals are made parallel and *then* the whole is
//! levelled. Before the stored orientation, because "vertical" means vertical
//! in the photograph as it is shown — a portrait frame the camera stored on
//! its side must converge along its displayed height, not along the sensor's
//! rows. And before the lens warp, which still sees the whole frame it
//! corrects, for the reason given above.
//!
//! The correction maps the output frame onto a trapezoid **inside** the
//! source rather than pulling the source edges in. So a keystone on its own
//! never exposes an empty corner, and the crop the user drew is still valid
//! after it; only in combination with a straightening angle does the
//! inscribed crop have anything to account for — see
//! [`Framing::max_inscribed_crop`].
//!
//! # Why sampling changes with the angle
//!
//! At 90° steps and flips, output pixels land exactly on source pixels, so
@@ -58,6 +85,13 @@ pub const CROP_X: ParamId = ParamId("crop_x");
pub const CROP_Y: ParamId = ParamId("crop_y");
pub const CROP_W: ParamId = ParamId("crop_w");
pub const CROP_H: ParamId = ParamId("crop_h");
/// TRACES: FR-DEV-20
/// Vertical keystone. Positive spreads the top of the frame — the correction
/// for a building photographed looking up, whose verticals lean together.
pub const KEYSTONE_V: ParamId = ParamId("keystone_v");
/// TRACES: FR-DEV-20
/// Horizontal keystone. Positive spreads the right-hand side of the frame.
pub const KEYSTONE_H: ParamId = ParamId("keystone_h");
/// Widest straightening the control offers, in degrees either way.
///
@@ -66,12 +100,28 @@ pub const CROP_H: ParamId = ParamId("crop_h");
/// the edits actually are.
pub const MAX_STRAIGHTEN: f32 = 45.0;
/// The keystone sliders' travel either way.
///
/// A plain amount rather than degrees of tilt: the angle a camera was tilted
/// by depends on a focal length the correction does not know, and a number
/// that claimed to be one would be wrong for every lens but one.
pub const MAX_KEYSTONE: f32 = 100.0;
/// How far a full keystone narrows the far edge of the frame, as a fraction
/// of its width: at `MAX_KEYSTONE` the source trapezoid's short side is half
/// its long one.
///
/// Enough for a tall building from its own pavement, and short of the point
/// where the stretched edge is so magnified that the correction reads as a
/// fault of its own.
const KEYSTONE_REACH: f64 = 0.5;
/// The parameters the framing widget owns — every one of them.
///
/// In the order the widget expects: the rect first, then the angle it is
/// straightened by, then the exact reorientations.
static FRAMING_PARAMS: [ParamId; 8] = [
CROP_X, CROP_Y, CROP_W, CROP_H, ANGLE, ROTATION, FLIP_H, FLIP_V,
static FRAMING_PARAMS: [ParamId; 10] = [
CROP_X, CROP_Y, CROP_W, CROP_H, ANGLE, KEYSTONE_V, KEYSTONE_H, ROTATION, FLIP_H, FLIP_V,
];
static DESCRIPTOR: LazyLock<Arc<OpDescriptor>> = LazyLock::new(|| {
@@ -117,6 +167,30 @@ static DESCRIPTOR: LazyLock<Arc<OpDescriptor>> = LazyLock::new(|| {
ParamDescriptor::fraction("crop_y", "param.crop_y", 0.0),
ParamDescriptor::fraction("crop_w", "param.crop_w", 1.0),
ParamDescriptor::fraction("crop_h", "param.crop_h", 1.0),
// TRACES: FR-DEV-20
// Perspective. Last so every sidecar written before these existed
// reads exactly as it did: a missing parameter is its default, and
// the default is no correction.
ParamDescriptor::scalar(
"keystone_v",
"param.keystone_v",
-MAX_KEYSTONE,
MAX_KEYSTONE,
0.0,
Unit::None,
Scale::Linear,
0,
),
ParamDescriptor::scalar(
"keystone_h",
"param.keystone_h",
-MAX_KEYSTONE,
MAX_KEYSTONE,
0.0,
Unit::None,
Scale::Linear,
0,
),
],
})
});
@@ -311,6 +385,86 @@ fn finite(v: f32, fallback: f32) -> f32 {
}
}
/// TRACES: FR-DEV-20
/// A plane projective map, row-major, acting on `(x, y, 1)`.
///
/// Held in `f64` because it is built by solving for four corners and then
/// inverted for [`Framing::output_at`]; the shader gets `f32` copies of the
/// forward map only.
#[derive(Debug, Clone, Copy, PartialEq)]
struct Homography([[f64; 3]; 3]);
impl Homography {
/// The map taking the square `[-0.5, 0.5]²` onto the quadrilateral whose
/// corners are `q`, listed top-left, top-right, bottom-right, bottom-left.
///
/// Heckbert's closed form for the unit square, composed with the shift
/// from the centred square onto it.
fn square_to_quad(q: [(f64, f64); 4]) -> Self {
let [(x0, y0), (x1, y1), (x2, y2), (x3, y3)] = q;
let (dx1, dx2, dx3) = (x1 - x2, x3 - x2, x0 - x1 + x2 - x3);
let (dy1, dy2, dy3) = (y1 - y2, y3 - y2, y0 - y1 + y2 - y3);
let den = dx1 * dy2 - dx2 * dy1;
let (g, h) = if den.abs() < 1e-12 {
(0.0, 0.0)
} else {
((dx3 * dy2 - dx2 * dy3) / den, (dx1 * dy3 - dx3 * dy1) / den)
};
// Unit square (u, v) -> quad.
let unit = [
[x1 - x0 + g * x1, x3 - x0 + h * x3, x0],
[y1 - y0 + g * y1, y3 - y0 + h * y3, y0],
[g, h, 1.0],
];
// Centred square -> unit square is `u = x + 0.5`, so fold the shift
// into the constant column.
let mut m = unit;
for row in &mut m {
row[2] += 0.5 * (row[0] + row[1]);
}
Self(m)
}
/// Where `(x, y)` lands, or `None` past the line the map sends to
/// infinity — a point with no image, which the caller treats as outside
/// the source.
fn apply(&self, (x, y): (f64, f64)) -> Option<(f64, f64)> {
let m = &self.0;
let w = m[2][0] * x + m[2][1] * y + m[2][2];
if w <= 1e-9 {
return None;
}
Some((
(m[0][0] * x + m[0][1] * y + m[0][2]) / w,
(m[1][0] * x + m[1][1] * y + m[1][2]) / w,
))
}
/// The inverse map, by the adjugate. Scale is irrelevant to a projective
/// map, so the determinant is only divided out to keep `w` positive and
/// near one — which is what [`Self::apply`]'s horizon test relies on.
fn inverse(&self) -> Self {
let m = &self.0;
let c = |r0: usize, c0: usize, r1: usize, c1: usize| {
m[r0][c0] * m[r1][c1] - m[r0][c1] * m[r1][c0]
};
let adj = [
[c(1, 1, 2, 2), -c(0, 1, 2, 2), c(0, 1, 1, 2)],
[-c(1, 0, 2, 2), c(0, 0, 2, 2), -c(0, 0, 1, 2)],
[c(1, 0, 2, 1), -c(0, 0, 2, 1), c(0, 0, 1, 1)],
];
let det = m[0][0] * adj[0][0] + m[0][1] * adj[1][0] + m[0][2] * adj[2][0];
let det = if det.abs() < 1e-12 { 1.0 } else { det };
let mut inv = adj;
for row in &mut inv {
for v in row.iter_mut() {
*v /= det;
}
}
Self(inv)
}
}
/// TRACES: FR-DEV-3 | FR-DEV-3d
/// Crop, straighten, rotation and flips for one image.
///
@@ -325,6 +479,10 @@ pub struct Framing {
quarter_turns: u8,
flip_h: bool,
flip_v: bool,
/// TRACES: FR-DEV-20
/// Vertical and horizontal keystone, each `-MAX_KEYSTONE..=MAX_KEYSTONE`.
keystone_v: f32,
keystone_h: f32,
/// TRACES: FR-DEV-3h
/// How the file's pixels were stored, from its EXIF orientation.
///
@@ -376,6 +534,8 @@ impl Default for Framing {
quarter_turns: 0,
flip_h: false,
flip_v: false,
keystone_v: 0.0,
keystone_h: 0.0,
baseline: dr_types::Orientation::NORMAL,
crop: CropRect::default(),
view: CropRect::default(),
@@ -396,7 +556,7 @@ impl Framing {
/// How framing would like to be presented.
///
/// **This is what stops a frontend having to name this stage.** Rendered
/// generically these eight parameters are eight bad controls: four crop
/// generically these ten parameters are ten bad controls: four crop
/// edges the photographer would have to type coordinates into, a "rotate"
/// slider running 0..3, and two switches. Every one of them is a worse
/// control than the gesture it stands for — a crop is dragged on the
@@ -407,10 +567,10 @@ impl Framing {
/// a second frontend would have had to learn the same special case, and
/// nothing in the capability output said why. Now the preference is
/// declared, the demand says what the widget needs, and a frontend that
/// cannot meet it falls back to the eight sliders — tedious, but complete,
/// cannot meet it falls back to the ten sliders — tedious, but complete,
/// which is the guarantee the whole hint mechanism rests on.
///
/// The widget owns **all eight** parameters rather than only the rect: a
/// The widget owns **all ten** parameters rather than only the rect: a
/// frontend that takes this on is taking on the whole framing control
/// surface, and leaving rotation and the flips behind would scatter them
/// into the generated panel underneath a crop control that already exists.
@@ -476,6 +636,52 @@ impl Framing {
(self.flip_h, self.flip_v)
}
/// TRACES: FR-DEV-20
/// The vertical and horizontal keystone, as the sliders show them.
pub fn keystone(&self) -> (f32, f32) {
(self.keystone_v, self.keystone_h)
}
/// Whether a perspective correction is applied at all.
pub fn has_keystone(&self) -> bool {
self.keystone_v != 0.0 || self.keystone_h != 0.0
}
/// TRACES: FR-DEV-20
/// The perspective map, from the straightened output frame to the upright
/// source frame, both measured as the centred square `[-0.5, 0.5]²`.
///
/// **Measured in fractions of the frame, not in the aspect-scaled space
/// the rest of the prologue works in**, so the map is the same for every
/// frame shape and the uniforms need no image size. The prologue divides
/// `p.x` by the frame's aspect on the way in and multiplies it back on
/// the way out.
///
/// The output frame's corners go to a trapezoid inside the source: a
/// positive vertical keystone brings the top corners in, so the top of
/// the source is spread across the full width of the output and lines
/// that converged upward come out parallel. Nothing is ever mapped from
/// outside the source, which is why a keystone alone needs no crop.
fn keystone_map(&self) -> Option<Homography> {
if !self.has_keystone() {
return None;
}
let amount = |v: f32| f64::from(v / MAX_KEYSTONE).clamp(-1.0, 1.0) * KEYSTONE_REACH;
let (tv, th) = (amount(self.keystone_v), amount(self.keystone_h));
// How much of each edge survives: the top and bottom rows' widths,
// the left and right columns' heights.
let top = 1.0 - tv.max(0.0);
let bottom = 1.0 + tv.min(0.0);
let right = 1.0 - th.max(0.0);
let left = 1.0 + th.min(0.0);
Some(Homography::square_to_quad([
(-0.5 * top, -0.5 * left),
(0.5 * top, -0.5 * right),
(0.5 * bottom, 0.5 * right),
(-0.5 * bottom, 0.5 * left),
]))
}
/// Add quarter turns, wrapping. The rotate-left/right buttons.
pub fn rotate_quarters(&mut self, turns: i32) {
self.quarter_turns = (i32::from(self.quarter_turns) + turns).rem_euclid(4) as u8;
@@ -544,6 +750,7 @@ impl Framing {
pub fn is_active(&self) -> bool {
let (turns, flip_h, flip_v) = self.effective();
self.angle != 0.0
|| self.has_keystone()
// Effective, not the user's: a file stored sideways needs the
// prologue emitted even on an untouched image, or it renders
// through the identity map and lies on its side.
@@ -572,6 +779,7 @@ impl Framing {
/// edited, the file was merely read correctly.
pub fn edits_image(&self) -> bool {
self.angle != 0.0
|| self.has_keystone()
|| self.quarter_turns != 0
|| self.flip_h
|| self.flip_v
@@ -591,7 +799,9 @@ impl Framing {
/// warp being active forces interpolation regardless, which is the
/// composer's call to make rather than this stage's.
pub fn needs_interpolation(&self) -> bool {
self.angle != 0.0
// A keystone stretches the frame by a different amount at every row,
// so it lands between pixels everywhere but on its centre line.
self.angle != 0.0 || self.has_keystone()
}
pub fn set_param(&mut self, id: ParamId, value: f32) {
@@ -629,6 +839,8 @@ impl Framing {
}
.normalised()
}
KEYSTONE_V => self.keystone_v = finite(value, 0.0).clamp(-MAX_KEYSTONE, MAX_KEYSTONE),
KEYSTONE_H => self.keystone_h = finite(value, 0.0).clamp(-MAX_KEYSTONE, MAX_KEYSTONE),
_ => log::warn!("framing: unknown parameter {id}"),
}
}
@@ -643,6 +855,8 @@ impl Framing {
CROP_Y => self.crop.y,
CROP_W => self.crop.width,
CROP_H => self.crop.height,
KEYSTONE_V => self.keystone_v,
KEYSTONE_H => self.keystone_h,
_ => 0.0,
}
}
@@ -707,8 +921,22 @@ impl Framing {
///
/// The standard largest-inscribed-rectangle result for a rotated
/// rectangle of the same aspect ratio.
///
/// TRACES: FR-DEV-20
/// **With a keystone the closed form no longer applies**: the area with a
/// source pixel behind it is the source rectangle pulled back through the
/// perspective map and then turned, a quadrilateral no textbook result
/// describes. That case is searched instead — see
/// [`Self::inscribed_by_search`]. A keystone alone never needs a crop, so
/// the search returns the whole frame for it, exactly.
pub fn max_inscribed_crop(&self, width: u32, height: u32) -> CropRect {
if self.angle == 0.0 || width == 0 || height == 0 {
if width == 0 || height == 0 {
return CropRect::default();
}
if self.has_keystone() {
return self.inscribed_by_search(width, height);
}
if self.angle == 0.0 {
return CropRect::default();
}
@@ -750,6 +978,123 @@ impl Framing {
.normalised()
}
/// TRACES: FR-DEV-20
/// The largest centred crop with a source pixel behind every point, found
/// by search rather than by formula.
///
/// The area that has a source pixel behind it is convex — the source
/// rectangle pulled back through a projective map whose horizon lies
/// outside it, then turned — and a rectangle lies inside a convex region
/// exactly when its four corners do. For a given width the tallest
/// rectangle that fits is therefore found by bisection, and the area
/// `width × tallest(width)` is unimodal in the width (a positive concave
/// function times a line), so a golden-section search finds the best
/// width. Every rectangle returned has been tested corner by corner, so
/// the answer errs inside, never outside.
///
/// A few hundred corner tests, on a gesture's release, is nothing next to
/// the render that follows it.
fn inscribed_by_search(&self, width: u32, height: u32) -> CropRect {
let (w, h) = if self.swaps_axes() {
(f64::from(height), f64::from(width))
} else {
(f64::from(width), f64::from(height))
};
let fa = w / h;
let rad = f64::from(self.angle).to_radians();
let (sn, cs) = (rad.sin(), rad.cos());
let map = self.keystone_map();
// Whether the output point `(fx, fy)`, in fractions of the frame from
// its centre, has a source pixel behind it. The prologue's steps, in
// its order, stopping short of the turns: those are a permutation of
// the frame and cannot move a point across its edge.
let defined = |fx: f64, fy: f64| {
let p = (fx * fa, fy);
let q = (p.0 * cs - p.1 * sn, p.0 * sn + p.1 * cs);
let n = (q.0 / fa, q.1);
let src = match &map {
Some(m) => m.apply(n),
None => Some(n),
};
const EDGE: f64 = 0.5 + 1e-9;
src.is_some_and(|(x, y)| x.abs() <= EDGE && y.abs() <= EDGE)
};
let fits = |hw: f64, hh: f64| {
[(-1.0, -1.0), (1.0, -1.0), (1.0, 1.0), (-1.0, 1.0)]
.iter()
.all(|(sx, sy)| defined(sx * hw, sy * hh))
};
if fits(0.5, 0.5) {
return CropRect::default();
}
// Half the tallest height that fits at half-width `hw`.
let tallest = |hw: f64| {
if fits(hw, 0.5) {
return 0.5;
}
if !fits(hw, 0.0) {
return 0.0;
}
let (mut lo, mut hi) = (0.0, 0.5);
for _ in 0..40 {
let mid = 0.5 * (lo + hi);
if fits(hw, mid) {
lo = mid;
} else {
hi = mid;
}
}
lo
};
let area = |hw: f64| hw * tallest(hw);
let ratio = (5.0_f64.sqrt() - 1.0) * 0.5;
let (mut a, mut b) = (0.0, 0.5);
let mut c = b - ratio * (b - a);
let mut d = a + ratio * (b - a);
let (mut fc, mut fd) = (area(c), area(d));
for _ in 0..48 {
if fc < fd {
a = c;
c = d;
fc = fd;
d = a + ratio * (b - a);
fd = area(d);
} else {
b = d;
d = c;
fd = fc;
c = b - ratio * (b - a);
fc = area(c);
}
}
let hw = 0.5 * (a + b);
let hh = tallest(hw);
// Tested at `hw` as returned, so a width that the search's last step
// nudged past the boundary cannot come back with a height that no
// longer fits it.
let (hw, hh) = if hh > 0.0 && fits(hw, hh) {
(hw, hh)
} else {
(c.min(d), tallest(c.min(d)))
};
let fw = (2.0 * hw) as f32;
let fh = (2.0 * hh) as f32;
let fw = fw.clamp(CropRect::MIN_EXTENT, 1.0);
let fh = fh.clamp(CropRect::MIN_EXTENT, 1.0);
CropRect {
x: (1.0 - fw) * 0.5,
y: (1.0 - fh) * 0.5,
width: fw,
height: fh,
}
.normalised()
}
/// TRACES: FR-DEV-3
/// Where an output point comes from in the source, both in normalised
/// `0..1` coordinates.
@@ -782,6 +1127,15 @@ impl Framing {
p = (p.0 * c - p.1 * s, p.0 * s + p.1 * c);
}
if let Some(m) = self.keystone_map() {
p = match m.apply((f64::from(p.0 / fx), f64::from(p.1))) {
Some((x, y)) => (x as f32 * fx, y as f32),
// Beyond the map's horizon: no source point at all, reported
// as one far outside the frame rather than as a NaN.
None => (1e6, 1e6),
};
}
let (turns, flip_h, flip_v) = self.effective();
p = match turns {
1 => (p.1 * ax, -p.0 / fx),
@@ -824,6 +1178,13 @@ impl Framing {
_ => p,
};
if let Some(m) = self.keystone_map() {
p = match m.inverse().apply((f64::from(p.0 / fx), f64::from(p.1))) {
Some((x, y)) => (x as f32 * fx, y as f32),
None => (1e6, 1e6),
};
}
if self.angle != 0.0 {
let rad = -self.angle * PI / 180.0;
let (s, c) = (rad.sin(), rad.cos());
@@ -865,6 +1226,19 @@ impl Framing {
// identical whether or not the user is zoomed in, and costs no extra
// uniform slot.
let rect = self.visible_rect();
// The perspective map by columns, so the prologue can apply it as
// three multiply-adds. The identity when there is none: the slots
// exist either way and the prologue does not read them.
let m = self
.keystone_map()
.unwrap_or(Homography([
[1.0, 0.0, 0.0],
[0.0, 1.0, 0.0],
[0.0, 0.0, 1.0],
]))
.0;
let col = |c: usize| [m[0][c] as f32, m[1][c] as f32, m[2][c] as f32, 0.0];
let [c0, c1, c2] = [col(0), col(1), col(2)];
[
rect.x,
rect.y,
@@ -874,6 +1248,18 @@ impl Framing {
rad.cos(),
0.0,
0.0,
c0[0],
c0[1],
c0[2],
c0[3],
c1[0],
c1[1],
c1[2],
c1[3],
c2[0],
c2[1],
c2[2],
c2[3],
]
}
@@ -972,6 +1358,28 @@ impl Framing {
);
}
if self.has_keystone() {
// TRACES: FR-DEV-20
// After the straightening and before the turns, so the keystone
// acts on the photograph as it is shown. The map is measured in
// fractions of the frame (see `Framing::keystone_map`), hence the
// aspect divided out and put back. A point past the map's horizon
// has no source at all and is sent far outside it, where the
// sampler's bounds test renders it void.
s.push_str(
"
// Perspective: the straightened frame onto a trapezoid of the source.
let key_n = vec2<f32>(p.x / frame_aspect.x, p.y);
let key_h = u.keystone_c0.xyz * key_n.x + u.keystone_c1.xyz * key_n.y + u.keystone_c2.xyz;
p = select(
vec2<f32>(1.0e6),
vec2<f32>(key_h.x / key_h.z * frame_aspect.x, key_h.y / key_h.z),
key_h.z > 1.0e-6,
);
",
);
}
// The user's turns and mirrors composed with the file's stored
// orientation. One permutation covers both, so honouring the EXIF tag
// adds no per-pixel work over an untagged file.
@@ -1040,13 +1448,15 @@ impl Framing {
| u64::from(flip_v) << 3
| u64::from(turns) << 4
| u64::from(self.is_active()) << 6
| u64::from(self.has_keystone()) << 7
}
}
/// Floats the framing block occupies in the generated uniform struct.
///
/// Two `vec4`s: the crop rect, and the angle's sin/cos with padding.
pub const FRAMING_UNIFORM_FIELDS: usize = 8;
/// Five `vec4`s: the crop rect, the angle's sin/cos with padding, and the
/// perspective map's three columns, each padded.
pub const FRAMING_UNIFORM_FIELDS: usize = 20;
#[cfg(test)]
mod tests {
@@ -2025,6 +2435,8 @@ mod tests {
(CROP_Y, 0.2),
(CROP_W, 0.5),
(CROP_H, 0.4),
(KEYSTONE_V, 35.0),
(KEYSTONE_H, -20.0),
] {
f.set_param(id, v);
assert_eq!(f.param(id), v, "{id} did not round-trip");
@@ -2241,6 +2653,8 @@ mod tests {
f.rotate_quarters(turns);
f.set_param(FLIP_H, 1.0);
f.set_param(FLIP_V, 1.0);
f.set_param(KEYSTONE_V, 60.0);
f.set_param(KEYSTONE_H, -25.0);
for out in [(0.0, 0.0), (0.5, 0.5), (0.2, 0.9), (0.95, 0.05)] {
let src = f.source_at(out, SRC.0, SRC.1);
@@ -2316,6 +2730,27 @@ mod tests {
"the three-turn permutation moved; `source_at` must move with it"
);
// The perspective step: the map applied in fractions of the frame,
// which is what `source_at` divides the aspect out for.
let mut f = Framing::new();
f.set_param(KEYSTONE_V, 40.0);
let prologue = f.wgsl_prologue();
assert!(
prologue.contains("let key_n = vec2<f32>(p.x / frame_aspect.x, p.y);")
&& prologue.contains("key_h.x / key_h.z * frame_aspect.x"),
"the perspective step moved; `source_at` must move with it"
);
// After the straightening and before the turns, as `source_at` has it.
let mut f = Framing::new();
f.set_param(KEYSTONE_V, 40.0);
f.set_param(ANGLE, 3.0);
f.rotate_quarters(1);
let prologue = f.wgsl_prologue();
let straighten = prologue.find("// Straighten").unwrap();
let keystone = prologue.find("// Perspective").unwrap();
let turn = prologue.find("90° clockwise").unwrap();
assert!(straighten < keystone && keystone < turn, "{prologue}");
// And the sampler's last step, which lives in `operation.rs` and is
// the half of the map this file does not emit.
assert!(
@@ -2323,4 +2758,241 @@ mod tests {
"the sampler's return to texture coordinates moved"
);
}
// ---- perspective (FR-DEV-20) -----------------------------------------
/// A grid over the whole output frame, edges included.
fn grid() -> impl Iterator<Item = (f32, f32)> {
(0..=10).flat_map(|j| (0..=10).map(move |i| (i as f32 / 10.0, j as f32 / 10.0)))
}
fn inside(p: (f32, f32)) -> bool {
(-1e-4..=1.0 + 1e-4).contains(&p.0) && (-1e-4..=1.0 + 1e-4).contains(&p.1)
}
#[test]
fn a_keystone_is_an_edit_and_a_resample() {
let mut f = Framing::new();
let neutral = f.structure_key();
f.set_param(KEYSTONE_V, 30.0);
assert!(f.is_active());
assert!(f.edits_image(), "a keystone must light the modified dot");
assert!(f.needs_interpolation());
assert_ne!(f.structure_key(), neutral);
assert!(f.wgsl_prologue().contains("u.keystone_c0"));
// Neither the output size nor the crop moves: the frame is reshaped
// inside itself, so what the user cropped stays cropped.
assert_eq!(f.output_size(6000, 4000), (6000, 4000));
assert!(f.crop().is_full());
}
#[test]
fn the_keystone_magnitude_does_not_reach_the_structure_key() {
// Dragging the slider is a uniform upload, never a shader build.
let mut f = Framing::new();
f.set_param(KEYSTONE_V, 10.0);
let key = f.structure_key();
for (v, h) in [(80.0, 0.0), (-45.0, 30.0), (1.0, -100.0)] {
f.set_param(KEYSTONE_V, v);
f.set_param(KEYSTONE_H, h);
assert_eq!(f.structure_key(), key, "{v}/{h} forced a recompile");
}
}
#[test]
fn the_keystone_is_clamped_to_its_travel() {
let mut f = Framing::new();
f.set_param(KEYSTONE_V, 1e9);
f.set_param(KEYSTONE_H, f32::NAN);
assert_eq!(f.keystone(), (MAX_KEYSTONE, 0.0));
assert!(f.uniforms().iter().all(|v| v.is_finite()));
}
#[test]
fn a_keystone_alone_never_reaches_outside_the_source() {
// The design decision the crop relies on: the output frame is mapped
// onto a trapezoid *inside* the source, so no corner goes empty and
// a crop drawn before the keystone is still a crop of the picture.
for (v, h) in [
(100.0, 0.0),
(-100.0, 0.0),
(0.0, 100.0),
(0.0, -100.0),
(100.0, 100.0),
(-100.0, 100.0),
(37.0, -64.0),
] {
for turns in 0..4 {
let mut f = Framing::new();
f.rotate_quarters(turns);
f.set_param(KEYSTONE_V, v);
f.set_param(KEYSTONE_H, h);
for out in grid() {
let src = f.source_at(out, SRC.0, SRC.1);
assert!(inside(src), "{v}/{h}, {turns} turn(s): {out:?} -> {src:?}");
}
assert!(f.max_inscribed_crop(SRC.0, SRC.1).is_full());
}
}
}
#[test]
fn a_vertical_keystone_makes_upward_converging_lines_parallel() {
// What the control is for. Output columns are straight verticals;
// with a positive keystone each must come from a straight source line
// that leans in toward the centre as it rises — the shape a building
// has when photographed looking up.
let mut f = Framing::new();
f.set_param(KEYSTONE_V, 60.0);
for x in [0.1f32, 0.3, 0.7, 0.9] {
let bottom = f.source_at((x, 1.0), SRC.0, SRC.1);
let middle = f.source_at((x, 0.5), SRC.0, SRC.1);
let top = f.source_at((x, 0.0), SRC.0, SRC.1);
// Straight: the middle sits on the line through the two ends.
let cross = (top.0 - bottom.0) * (middle.1 - bottom.1)
- (top.1 - bottom.1) * (middle.0 - bottom.0);
assert!(cross.abs() < 1e-4, "column {x} is not a straight line");
// Leaning in: the top is nearer the centre than the bottom.
assert!(
(top.0 - 0.5).abs() < (bottom.0 - 0.5).abs(),
"column {x}: top {top:?} is not inside bottom {bottom:?}"
);
}
// The bottom row is left where it was; the top row is the one spread.
close(
f.source_at((0.0, 1.0), SRC.0, SRC.1),
(0.0, 1.0),
"bottom-left",
);
close(
f.source_at((0.0, 0.0), SRC.0, SRC.1),
(0.15, 0.0),
"top-left",
);
}
#[test]
fn a_horizontal_keystone_spreads_the_right_hand_side() {
let mut f = Framing::new();
f.set_param(KEYSTONE_H, 100.0);
// The right-hand column comes from half the source's height.
close(
f.source_at((1.0, 0.0), SRC.0, SRC.1),
(1.0, 0.25),
"top-right",
);
close(
f.source_at((1.0, 1.0), SRC.0, SRC.1),
(1.0, 0.75),
"bottom-right",
);
close(
f.source_at((0.0, 0.0), SRC.0, SRC.1),
(0.0, 0.0),
"top-left",
);
}
#[test]
fn the_keystone_acts_on_the_frame_as_shown() {
// A portrait frame the camera stored on its side: "vertical" is the
// frame's displayed height, so the same keystone must move the same
// *displayed* points whatever the file's stored orientation.
let mut upright = Framing::new();
upright.set_param(KEYSTONE_V, 50.0);
let mut sideways = Framing::new();
sideways.set_baseline(dr_types::Orientation::from_exif(6));
sideways.set_param(KEYSTONE_V, 50.0);
// Compare in the displayed frame: map the sideways result back
// through the orientation alone.
let mut turn_only = Framing::new();
turn_only.set_baseline(dr_types::Orientation::from_exif(6));
for out in grid() {
let a = upright.source_at(out, SRC.1, SRC.0);
let b = turn_only.output_at(sideways.source_at(out, SRC.0, SRC.1), SRC.0, SRC.1);
close(a, b, "the keystone turned with the file");
}
}
#[test]
fn the_inscribed_crop_accounts_for_the_keystone() {
// Straightening a keystoned frame: the empty area is no longer the
// rotated rectangle's, and the crop must avoid the area that is.
for (angle, v, h) in [
(5.0f32, 50.0f32, 0.0f32),
(-8.0, -70.0, 20.0),
(12.0, 100.0, 100.0),
(2.0, 0.0, -40.0),
] {
for turns in [0, 1] {
let mut f = Framing::new();
f.rotate_quarters(turns);
f.set_param(ANGLE, angle);
f.set_param(KEYSTONE_V, v);
f.set_param(KEYSTONE_H, h);
let c = f.max_inscribed_crop(SRC.0, SRC.1);
assert!(
c.width > 0.3 && c.height > 0.3 && !c.is_full(),
"{angle}°/{v}/{h}: {c:?}"
);
assert!(
((c.x + c.width * 0.5) - 0.5).abs() < 1e-4
&& ((c.y + c.height * 0.5) - 0.5).abs() < 1e-4,
"{c:?} is not centred"
);
// Every point of it, edges included, has a source pixel.
f.set_crop(c);
for out in grid() {
let src = f.source_at(out, SRC.0, SRC.1);
assert!(inside(src), "{angle}°/{v}/{h}: {out:?} -> {src:?}");
}
}
}
}
#[test]
fn the_inscribed_crop_with_a_keystone_is_not_needlessly_small() {
// The search must find the best rectangle, not merely a safe one. A
// tenth larger in either direction has to reach outside the source.
let mut f = Framing::new();
f.set_param(ANGLE, 6.0);
f.set_param(KEYSTONE_V, 60.0);
let c = f.max_inscribed_crop(SRC.0, SRC.1);
for (gw, gh) in [(1.1, 1.0), (1.0, 1.1)] {
let mut g = f;
let (w, h) = (c.width * gw, c.height * gh);
g.set_crop(CropRect {
x: 0.5 - w * 0.5,
y: 0.5 - h * 0.5,
width: w,
height: h,
});
let spills = [(0.0, 0.0), (1.0, 0.0), (1.0, 1.0), (0.0, 1.0)]
.into_iter()
.any(|out| !inside(g.source_at(out, SRC.0, SRC.1)));
// Either the grown rect spills, or it could not grow at all
// because the crop was already at the frame's edge on that axis.
assert!(
spills
|| (gw > 1.0 && c.width >= 1.0 - 1e-4)
|| (gh > 1.0 && c.height >= 1.0 - 1e-4),
"{c:?} grown by {gw}x{gh} still fits"
);
}
}
#[test]
fn reset_clears_the_keystone() {
let mut f = Framing::new();
f.set_param(KEYSTONE_V, 30.0);
f.set_param(KEYSTONE_H, -30.0);
f.reset();
assert_eq!(f.keystone(), (0.0, 0.0));
assert!(!f.is_active());
}
}
core/dr-pipeline/src/operation.rs
+5 -1
View File
@@ -801,7 +801,11 @@ fn compose_inner(
\x20 // angle as sin/cos — a trig call per pixel would recompute a\n\
\x20 // value that is constant across the dispatch.\n\
\x20 crop_rect: vec4<f32>,\n\
\x20 framing_angle: vec4<f32>,\n",
\x20 framing_angle: vec4<f32>,\n\
\x20 // The perspective map (FR-DEV-20) by columns, `.w` unused.\n\
\x20 keystone_c0: vec4<f32>,\n\
\x20 keystone_c1: vec4<f32>,\n\
\x20 keystone_c2: vec4<f32>,\n",
);
uniform_values.extend_from_slice(&framing.uniforms());
core/dr-pipeline/src/preset.rs
+10
View File
@@ -725,6 +725,7 @@ mod tests {
height: 0.5,
});
g.set_param(framing::ID, framing::ANGLE, -2.0);
g.set_param(framing::ID, framing::KEYSTONE_V, 40.0);
g
}
@@ -820,6 +821,14 @@ mod tests {
Some(0.0),
"the source's straightening must not travel on this scope"
);
// TRACES: FR-DEV-20
// Perspective is composition on the same terms as the crop: it is
// withheld by the default scope, not pasted over the target's frame.
assert_eq!(
target.param(framing::ID, framing::KEYSTONE_V),
Some(0.0),
"the source's keystone must not travel on this scope"
);
}
#[test]
@@ -829,6 +838,7 @@ mod tests {
preset.apply(&mut target, Scope::everything());
assert_eq!(target.param(framing::ID, framing::ANGLE), Some(-2.0));
assert_eq!(target.param(framing::ID, framing::KEYSTONE_V), Some(40.0));
assert!(
(target.crop().width - 0.5).abs() < 1e-5,
"{:?}",
core/dr-pipeline/src/sidecar.rs
+8
View File
@@ -2174,6 +2174,8 @@ mod tests {
height: 0.6,
});
g.set_param(framing::ID, framing::ANGLE, -1.5);
g.set_param(framing::ID, framing::KEYSTONE_V, 42.0);
g.set_param(framing::ID, framing::KEYSTONE_H, -17.0);
g.rotate_quarters(1);
let mut sidecar = Sidecar::new();
@@ -2190,6 +2192,12 @@ mod tests {
assert_eq!(restored.crop(), g.crop());
assert_eq!(restored.param(framing::ID, framing::ANGLE), Some(-1.5));
assert_eq!(restored.param(framing::ID, framing::ROTATION), Some(1.0));
// TRACES: FR-DEV-20
assert_eq!(restored.param(framing::ID, framing::KEYSTONE_V), Some(42.0));
assert_eq!(
restored.param(framing::ID, framing::KEYSTONE_H),
Some(-17.0)
);
}
#[test]