dtourolleandClaude Opus 5
954246b969
Two CI gates have failed on every push since 0.13.4 and both are fixed here. The traceability gate rejected `FR-INF-1` as an orphan: settings.slint and dr-ui tag it, but the four register entries inference.md §12 wrote were never carried into requirements.md, which is the only file the extractor reads. §3.12 and §4.10 now hold FR-INF-1..3 and NFR-INF-1 verbatim, with the acceptance milestones pointed back at inference.md. The matrix is regenerated (188 defined, 155 covered) and the README's "where it stands" line, which the 0.13.6 release commit skipped, says 0.13.6 and the new figures. `cargo fmt --check` failed on the `fill_border` call in dr-pano's padding test, which is the first thing the Desktop job runs after installing the toolchain and why it failed within a minute. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
918 lines
31 KiB
Rust
918 lines
31 KiB
Rust
//! TRACES: FR-MRG-4
|
||
//! Filling a composite's uncovered border, tile by tile, with an inpainter.
|
||
//!
|
||
//! A merged panorama has a ragged border where no frame reached. FR-MRG-4
|
||
//! crops it by default; this fills it instead, when the photographer asks,
|
||
//! with pixels a model invents from the picture around them. Everything
|
||
//! here is the geometry of that — which tiles to run, what context to hand
|
||
//! the model, how to put its answers back — and none of it is the model:
|
||
//! that is the [`Inpainter`] trait, with MI-GAN behind it in `migan.rs`
|
||
//! and a fake in the tests.
|
||
//!
|
||
//! # Context across the edge
|
||
//!
|
||
//! An inpainting model is trained on holes *inside* pictures. A panorama's
|
||
//! border is a hole at the picture's *edge*: real content on one side,
|
||
//! nothing on the other, and a model given that invents a structure along
|
||
//! the open side — streaks of road in the sky, on the first try
|
||
//! (2026-09-19). So the known content is mirrored across the coverage
|
||
//! edge, column by column for the top and bottom bands and row by row for
|
||
//! the sides, into the hole and into a padding ring around the picture,
|
||
//! and the ring is presented as *known*. The model then interpolates
|
||
//! between real content and its mirror rather than extrapolating into
|
||
//! nothing. The ring is cut off at the end.
|
||
//!
|
||
//! # Structure from far away, texture from near
|
||
//!
|
||
//! One tiled pass at the working resolution was not enough: a 512-px tile
|
||
//! straddling the coverage edge sees a few hundred pixels of real content
|
||
//! on one side and invents the rest from that, two neighbouring tiles
|
||
//! invent differently, and the seams and the merge's own fringe leak into
|
||
//! the fill. [`fill_border`] therefore runs in two stages. A **coarse**
|
||
//! pass at a quarter of the size, where the whole border and hundreds of
|
||
//! pixels of real context sit inside a handful of tiles, decides the
|
||
//! structure — where the slope goes, where the sky stays sky. Then
|
||
//! **fine** passes regenerate the hole in bands from the real edge
|
||
//! outward: each band is the only unknown, with real content (or the band
|
||
//! before, freshly textured) on its near side and the coarse fill,
|
||
//! upsampled, on its far side — blurry, but the right structure — so the
|
||
//! model generates texture and a transition, never a large hole from
|
||
//! nothing.
|
||
//!
|
||
//! Tiles overlap by a third and are blended under a raised-cosine window,
|
||
//! so the seams between tiles do not show; the model's answer replaces
|
||
//! only the pixels that were unknown, and the picture itself is untouched.
|
||
|
||
use crate::PanoError;
|
||
|
||
/// A model that fills a square hole from its surroundings.
|
||
pub trait Inpainter {
|
||
/// The square tile it takes, in pixels.
|
||
fn tile(&self) -> usize;
|
||
|
||
/// Fill one tile. `rgb` is `tile × tile × 3`, row-major, 0..1, with the
|
||
/// unknown pixels' values meaningless; `known` is `tile × tile`. The
|
||
/// result is `tile × tile × 3`, 0..1, of which only the unknown pixels
|
||
/// are read.
|
||
fn fill(&mut self, rgb: &[f32], known: &[bool]) -> Result<Vec<f32>, PanoError>;
|
||
}
|
||
|
||
/// What a caller hears from [`fill_border`]: progress, for a page's bar,
|
||
/// and — for whoever is looking at why a fill went wrong — each stage's
|
||
/// picture as it lands. A plain `FnMut(usize, usize)` is an observer that
|
||
/// hears only the progress.
|
||
pub trait Observer {
|
||
/// `(done, total)` tiles, the total an estimate until the last band.
|
||
fn progress(&mut self, done: usize, total: usize);
|
||
/// A stage's result, `width × height × 3`: `coarse` (at the coarse
|
||
/// size), `band-N` after each fine band, `feathered` at the end.
|
||
fn stage(&mut self, _name: &str, _rgb: &[f32], _width: usize, _height: usize) {}
|
||
}
|
||
|
||
impl<F: FnMut(usize, usize)> Observer for F {
|
||
fn progress(&mut self, done: usize, total: usize) {
|
||
self(done, total)
|
||
}
|
||
}
|
||
|
||
/// How far the picture is extended with mirrored content before tiling.
|
||
/// Half a tile: enough that a hole at the edge sits well inside a tile.
|
||
pub const RING: usize = 256;
|
||
|
||
/// The fill's knobs, in pixels of the working image. The defaults are
|
||
/// what the fixture panorama looked best with on 2026-09-19; the merge
|
||
/// page exposes every one of them while the fill is experimental, so a
|
||
/// bad corner can be worked on from the picture rather than the code.
|
||
#[derive(Debug, Clone, Copy, PartialEq)]
|
||
pub struct Params {
|
||
/// The coarse pass's reduction: 1 skips it.
|
||
pub coarse: usize,
|
||
/// The fine passes' band width.
|
||
pub band: usize,
|
||
/// How deep into the picture the mirrored context reaches, or **zero
|
||
/// for no mirrored context at all**: the void is then shown to the
|
||
/// model as it is — reaching the picture's edge with nothing beyond,
|
||
/// and, beyond the band being filled, still unknown. That is what the
|
||
/// shipped model was trained on (a fine-tune of MI-GAN on voids cut
|
||
/// from photographs the way a cylindrical merge cuts them, see
|
||
/// `docs/panorama.md` §14); a ring would give it a fold to continue.
|
||
///
|
||
/// Non-zero is the stock model's crutch: a plain reflection of a deep
|
||
/// hole pulls in whatever is that far from the edge — a ridge, a peak —
|
||
/// and the model, told that is what lies beyond, paints it upside down.
|
||
/// Folding the reflection within this band keeps the ring looking like
|
||
/// the edge it continues and nothing further away.
|
||
pub mirror_depth: usize,
|
||
/// How far inside the real edge the fill also regenerates, the two
|
||
/// blended by distance. A hard cut between real pixels and invented
|
||
/// ones is a line whatever the fill's quality; blended over this many
|
||
/// pixels it is not. Zero is the hard cut.
|
||
pub feather: usize,
|
||
/// The step between tiles, at most the tile; two thirds of it usual.
|
||
pub stride: usize,
|
||
}
|
||
|
||
impl Default for Params {
|
||
fn default() -> Self {
|
||
Params {
|
||
coarse: 1,
|
||
band: 192,
|
||
mirror_depth: 0,
|
||
feather: 24,
|
||
stride: 384,
|
||
}
|
||
}
|
||
}
|
||
|
||
/// Fill the unknown pixels of `rgb` (`width × height × 3`, 0..1) in place:
|
||
/// the coarse pass, then the fine bands, then the seam feathered over
|
||
/// `feather` pixels inside the real edge. Returns the tiles run.
|
||
///
|
||
/// `known` is `width × height`. `observer` hears the progress and, if it
|
||
/// cares, each stage.
|
||
pub fn fill_border(
|
||
rgb: &mut [f32],
|
||
width: usize,
|
||
height: usize,
|
||
known: &[bool],
|
||
model: &mut dyn Inpainter,
|
||
params: Params,
|
||
observer: &mut dyn Observer,
|
||
) -> Result<usize, PanoError> {
|
||
let Params {
|
||
coarse: q,
|
||
band,
|
||
mirror_depth,
|
||
feather,
|
||
stride,
|
||
} = params;
|
||
let q = q.max(1);
|
||
let band = band.max(8);
|
||
// No ring: the void beyond the band stays unknown, as in the model's
|
||
// training; with a ring the far side is the coarse fill, presented as
|
||
// known, which the stock model needed to see something there.
|
||
let open = mirror_depth == 0;
|
||
if width == 0 || height == 0 || rgb.len() != width * height * 3 || known.len() != width * height
|
||
{
|
||
return Err(PanoError::Input("fill: buffer sizes disagree".into()));
|
||
}
|
||
if known.iter().all(|&k| k) {
|
||
return Ok(0);
|
||
}
|
||
// The fill regenerates a margin inside the real edge too, and the
|
||
// result is blended with the real pixels across it at the end.
|
||
let real = rgb.to_vec();
|
||
let outer = known.to_vec();
|
||
let mut inner = known.to_vec();
|
||
erode(&mut inner, width, height, feather);
|
||
let known = &inner[..];
|
||
let mut done = 0usize;
|
||
|
||
// Coarse: a fraction of the size, unknown where any pixel of the cell was.
|
||
let (cw, ch) = ((width / q).max(1), (height / q).max(1));
|
||
let mut coarse = vec![0.0f32; cw * ch * 3];
|
||
let mut cknown = vec![true; cw * ch];
|
||
for y in 0..ch {
|
||
for x in 0..cw {
|
||
let mut sum = [0.0f32; 3];
|
||
let mut n = 0.0f32;
|
||
let mut all_known = true;
|
||
for dy in 0..q {
|
||
for dx in 0..q {
|
||
let (sx, sy) = ((x * q + dx).min(width - 1), (y * q + dy).min(height - 1));
|
||
let i = sy * width + sx;
|
||
all_known &= known[i];
|
||
for c in 0..3 {
|
||
sum[c] += rgb[i * 3 + c];
|
||
}
|
||
n += 1.0;
|
||
}
|
||
}
|
||
for c in 0..3 {
|
||
coarse[(y * cw + x) * 3 + c] = sum[c] / n;
|
||
}
|
||
cknown[y * cw + x] = all_known;
|
||
}
|
||
}
|
||
let estimate = |tiles: usize| tiles * 4;
|
||
done += fill_once(
|
||
&mut coarse,
|
||
cw,
|
||
ch,
|
||
&cknown,
|
||
model,
|
||
stride,
|
||
mirror_depth,
|
||
|n, t| observer.progress(n, estimate(t)),
|
||
)?;
|
||
observer.stage("coarse", &coarse, cw, ch);
|
||
|
||
// The hole starts as the coarse structure, upsampled.
|
||
for y in 0..height {
|
||
for x in 0..width {
|
||
let i = y * width + x;
|
||
if known[i] {
|
||
continue;
|
||
}
|
||
let fx = ((x as f32 + 0.5) / q as f32 - 0.5).clamp(0.0, (cw - 1) as f32);
|
||
let fy = ((y as f32 + 0.5) / q as f32 - 0.5).clamp(0.0, (ch - 1) as f32);
|
||
let (x0, y0) = (fx as usize, fy as usize);
|
||
let (x1, y1) = ((x0 + 1).min(cw - 1), (y0 + 1).min(ch - 1));
|
||
let (tx, ty) = (fx - x0 as f32, fy - y0 as f32);
|
||
for c in 0..3 {
|
||
let at = |xx: usize, yy: usize| coarse[(yy * cw + xx) * 3 + c];
|
||
rgb[i * 3 + c] = (at(x0, y0) * (1.0 - tx) + at(x1, y0) * tx) * (1.0 - ty)
|
||
+ (at(x0, y1) * (1.0 - tx) + at(x1, y1) * tx) * ty;
|
||
}
|
||
}
|
||
}
|
||
|
||
// Fine, in bands from the edge outward.
|
||
let dist = distance_to_known(known, width, height);
|
||
let mut band_known = vec![true; width * height];
|
||
let mut b = 0usize;
|
||
loop {
|
||
let lo = (b * band).saturating_sub(band / 2) as f32;
|
||
let hi = ((b + 1) * band) as f32;
|
||
let mut any = false;
|
||
for i in 0..width * height {
|
||
let in_band = !known[i] && dist[i] > lo && dist[i] <= hi;
|
||
band_known[i] = if open {
|
||
known[i] || dist[i] <= lo
|
||
} else {
|
||
!in_band
|
||
};
|
||
any |= in_band;
|
||
}
|
||
if !any {
|
||
break;
|
||
}
|
||
let before = done;
|
||
done += fill_once(
|
||
rgb,
|
||
width,
|
||
height,
|
||
&band_known,
|
||
model,
|
||
stride,
|
||
mirror_depth,
|
||
|n, t| observer.progress(before + n, before + estimate(t)),
|
||
)?;
|
||
observer.stage(&format!("band-{b}"), rgb, width, height);
|
||
b += 1;
|
||
}
|
||
|
||
// The seam: across the margin, real on the inside, invented on the
|
||
// outside, a smooth ramp between by distance from the true hole.
|
||
if feather > 0 {
|
||
let to_hole =
|
||
distance_to_known(&outer.iter().map(|k| !k).collect::<Vec<_>>(), width, height);
|
||
for i in 0..width * height {
|
||
if !outer[i] || known[i] {
|
||
continue;
|
||
}
|
||
// In the margin: outer says known, inner says not.
|
||
let t = (to_hole[i] / feather as f32).clamp(0.0, 1.0);
|
||
let t = t * t * (3.0 - 2.0 * t);
|
||
for c in 0..3 {
|
||
rgb[i * 3 + c] = rgb[i * 3 + c] * (1.0 - t) + real[i * 3 + c] * t;
|
||
}
|
||
}
|
||
}
|
||
observer.stage("feathered", rgb, width, height);
|
||
observer.progress(done, done);
|
||
Ok(done)
|
||
}
|
||
|
||
/// One tiled pass: every unknown pixel regenerated from the tiles that
|
||
/// touch it, the rest kept. Returns the tiles run.
|
||
#[allow(clippy::too_many_arguments)]
|
||
fn fill_once(
|
||
rgb: &mut [f32],
|
||
width: usize,
|
||
height: usize,
|
||
known: &[bool],
|
||
model: &mut dyn Inpainter,
|
||
stride: usize,
|
||
mirror_depth: usize,
|
||
mut progress: impl FnMut(usize, usize),
|
||
) -> Result<usize, PanoError> {
|
||
let t = model.tile();
|
||
if t == 0 || known.iter().all(|&k| k) {
|
||
return Ok(0);
|
||
}
|
||
|
||
// The padded canvas with mirrored context, and the hole within it.
|
||
let ctx = MirroredContext::build(rgb, width, height, known, mirror_depth, t);
|
||
let (pw, ph) = (ctx.width, ctx.height);
|
||
|
||
// Tiles that touch the hole, on a grid that reaches both far edges.
|
||
let starts = |n: usize| -> Vec<usize> {
|
||
if n <= t {
|
||
return vec![0];
|
||
}
|
||
let mut v: Vec<usize> = (0..=n - t).step_by(stride.clamp(1, t)).collect();
|
||
if *v.last().unwrap_or(&0) != n - t {
|
||
v.push(n - t);
|
||
}
|
||
v
|
||
};
|
||
let ys = starts(ph);
|
||
let xs = starts(pw);
|
||
let mut tiles = Vec::new();
|
||
for &y in &ys {
|
||
for &x in &xs {
|
||
if y + t > ph || x + t > pw {
|
||
continue;
|
||
}
|
||
let touches =
|
||
(y..y + t).any(|yy| ctx.hole[yy * pw + x..yy * pw + x + t].iter().any(|&h| h));
|
||
if touches {
|
||
tiles.push((x, y));
|
||
}
|
||
}
|
||
}
|
||
|
||
// Raised-cosine window, so overlapping tiles blend.
|
||
let hann: Vec<f32> = (0..t)
|
||
.map(|i| {
|
||
let s = ((i as f32 + 1.0) / (t as f32 + 1.0) * std::f32::consts::PI).sin();
|
||
s * s + 1e-3
|
||
})
|
||
.collect();
|
||
|
||
let mut acc = vec![0.0f32; pw * ph * 3];
|
||
let mut wsum = vec![0.0f32; pw * ph];
|
||
let mut tile_rgb = vec![0.0f32; t * t * 3];
|
||
let mut tile_known = vec![false; t * t];
|
||
let total = tiles.len();
|
||
for (n, &(x, y)) in tiles.iter().enumerate() {
|
||
progress(n, total);
|
||
for r in 0..t {
|
||
let src = ((y + r) * pw + x) * 3;
|
||
tile_rgb[r * t * 3..(r + 1) * t * 3].copy_from_slice(&ctx.rgb[src..src + t * 3]);
|
||
let ks = (y + r) * pw + x;
|
||
for c in 0..t {
|
||
tile_known[r * t + c] = !ctx.hole[ks + c];
|
||
}
|
||
}
|
||
let out = model.fill(&tile_rgb, &tile_known)?;
|
||
if out.len() != t * t * 3 {
|
||
return Err(PanoError::Model(format!(
|
||
"the inpainter returned {} values for a {t}×{t} tile",
|
||
out.len()
|
||
)));
|
||
}
|
||
for r in 0..t {
|
||
for c in 0..t {
|
||
let w = hann[r] * hann[c];
|
||
let p = (y + r) * pw + (x + c);
|
||
for ch in 0..3 {
|
||
acc[p * 3 + ch] += out[(r * t + c) * 3 + ch] * w;
|
||
}
|
||
wsum[p] += w;
|
||
}
|
||
}
|
||
}
|
||
progress(total, total);
|
||
|
||
// Back into the picture: only the unknown pixels change.
|
||
for yy in 0..height {
|
||
for xx in 0..width {
|
||
let i = yy * width + xx;
|
||
if known[i] {
|
||
continue;
|
||
}
|
||
let p = (yy + ctx.ring) * pw + (xx + ctx.ring);
|
||
if wsum[p] > 0.0 {
|
||
for ch in 0..3 {
|
||
rgb[i * 3 + ch] = (acc[p * 3 + ch] / wsum[p]).clamp(0.0, 1.0);
|
||
}
|
||
}
|
||
}
|
||
}
|
||
Ok(total)
|
||
}
|
||
|
||
/// Shrink `known` by `iterations` pixels on every side, in place.
|
||
///
|
||
/// The merge's coverage edge carries a fringe — the last partly-covered
|
||
/// pixels of a frame, and whatever the renderer did at the boundary — and
|
||
/// a fill that stops exactly at the coverage bit leaves it as a dark line
|
||
/// along the seam. Eight pixels at a quarter of the composite's resolution
|
||
/// was what it took on the fixture.
|
||
pub fn erode(known: &mut [bool], width: usize, height: usize, iterations: usize) {
|
||
let mut next = known.to_vec();
|
||
for _ in 0..iterations {
|
||
for y in 0..height {
|
||
for x in 0..width {
|
||
let i = y * width + x;
|
||
if !known[i] {
|
||
continue;
|
||
}
|
||
let edge = x == 0
|
||
|| y == 0
|
||
|| x + 1 == width
|
||
|| y + 1 == height
|
||
|| !known[i - 1]
|
||
|| !known[i + 1]
|
||
|| !known[i - width]
|
||
|| !known[i + width];
|
||
next[i] = !edge;
|
||
}
|
||
}
|
||
known.copy_from_slice(&next);
|
||
}
|
||
}
|
||
|
||
/// Distance from each pixel to the nearest known one, by two chamfer
|
||
/// sweeps — within a few percent of Euclidean, and enough to cut bands.
|
||
fn distance_to_known(known: &[bool], width: usize, height: usize) -> Vec<f32> {
|
||
let inf = (width + height) as f32;
|
||
let mut d: Vec<f32> = known.iter().map(|&k| if k { 0.0 } else { inf }).collect();
|
||
let (a, b) = (1.0f32, std::f32::consts::SQRT_2);
|
||
for y in 0..height {
|
||
for x in 0..width {
|
||
let i = y * width + x;
|
||
let mut v = d[i];
|
||
if x > 0 {
|
||
v = v.min(d[i - 1] + a);
|
||
}
|
||
if y > 0 {
|
||
v = v.min(d[i - width] + a);
|
||
if x > 0 {
|
||
v = v.min(d[i - width - 1] + b);
|
||
}
|
||
if x + 1 < width {
|
||
v = v.min(d[i - width + 1] + b);
|
||
}
|
||
}
|
||
d[i] = v;
|
||
}
|
||
}
|
||
for y in (0..height).rev() {
|
||
for x in (0..width).rev() {
|
||
let i = y * width + x;
|
||
let mut v = d[i];
|
||
if x + 1 < width {
|
||
v = v.min(d[i + 1] + a);
|
||
}
|
||
if y + 1 < height {
|
||
v = v.min(d[i + width] + a);
|
||
if x + 1 < width {
|
||
v = v.min(d[i + width + 1] + b);
|
||
}
|
||
if x > 0 {
|
||
v = v.min(d[i + width - 1] + b);
|
||
}
|
||
}
|
||
d[i] = v;
|
||
}
|
||
}
|
||
d
|
||
}
|
||
|
||
/// Distance beyond the edge to distance inside it, folded within `depth`
|
||
/// ([`Params::mirror_depth`]): a triangle wave, so the band is read
|
||
/// forward and back rather than clamped to one row.
|
||
fn fold(d: usize, depth: usize) -> usize {
|
||
let period = 2 * depth;
|
||
let r = d % period;
|
||
if r <= depth {
|
||
r
|
||
} else {
|
||
period - r
|
||
}
|
||
}
|
||
|
||
/// The picture on a canvas `RING` wider on every side, with the hole and
|
||
/// the ring filled by mirroring the known content across the coverage
|
||
/// edge — the nearest `depth` of it, folded — and the hole, the
|
||
/// original unknown and nothing else, marked.
|
||
struct MirroredContext {
|
||
width: usize,
|
||
height: usize,
|
||
/// The padding on every side: `RING` with mirrored context, 0 without.
|
||
ring: usize,
|
||
rgb: Vec<f32>,
|
||
hole: Vec<bool>,
|
||
}
|
||
|
||
impl MirroredContext {
|
||
fn build(
|
||
rgb: &[f32],
|
||
width: usize,
|
||
height: usize,
|
||
known: &[bool],
|
||
depth: usize,
|
||
tile: usize,
|
||
) -> Self {
|
||
if depth == 0 {
|
||
// Open: the picture as it is, the hole as it is. What the hole
|
||
// holds does not matter — the model masks it out. A picture
|
||
// smaller than a tile (the merge page's preview) sits at the
|
||
// origin of a tile-sized canvas whose rest is hole: still the
|
||
// void as it is, and the only way a tile fits at all.
|
||
let (pw, ph) = (width.max(tile), height.max(tile));
|
||
let mut canvas = vec![0.0f32; pw * ph * 3];
|
||
let mut hole = vec![true; pw * ph];
|
||
for y in 0..height {
|
||
canvas[y * pw * 3..(y * pw + width) * 3]
|
||
.copy_from_slice(&rgb[y * width * 3..(y + 1) * width * 3]);
|
||
for x in 0..width {
|
||
hole[y * pw + x] = !known[y * width + x];
|
||
}
|
||
}
|
||
return MirroredContext {
|
||
width: pw,
|
||
height: ph,
|
||
ring: 0,
|
||
rgb: canvas,
|
||
hole,
|
||
};
|
||
}
|
||
let fold = |d: usize| fold(d, depth);
|
||
let (pw, ph) = (width + 2 * RING, height + 2 * RING);
|
||
let mut canvas = vec![0.0f32; pw * ph * 3];
|
||
let mut kn = vec![false; pw * ph];
|
||
let mut hole = vec![false; pw * ph];
|
||
for y in 0..height {
|
||
for x in 0..width {
|
||
let i = y * width + x;
|
||
let p = (y + RING) * pw + (x + RING);
|
||
canvas[p * 3..p * 3 + 3].copy_from_slice(&rgb[i * 3..i * 3 + 3]);
|
||
kn[p] = known[i];
|
||
hole[p] = !known[i];
|
||
}
|
||
}
|
||
|
||
// Per column: mirror across the first and last known row.
|
||
for x in 0..pw {
|
||
let first = (0..ph).find(|&y| kn[y * pw + x]);
|
||
let Some(first) = first else { continue };
|
||
let last = (0..ph).rev().find(|&y| kn[y * pw + x]).unwrap_or(first);
|
||
for y in 0..first {
|
||
let m = (first + fold(first - y)).min(last);
|
||
let (d, s) = ((y * pw + x) * 3, (m * pw + x) * 3);
|
||
canvas.copy_within(s..s + 3, d);
|
||
}
|
||
for y in last + 1..ph {
|
||
let m = last.saturating_sub(fold(y - last)).max(first);
|
||
let (d, s) = ((y * pw + x) * 3, (m * pw + x) * 3);
|
||
canvas.copy_within(s..s + 3, d);
|
||
}
|
||
}
|
||
// Per row, for the sides, over what is there now.
|
||
for y in 0..ph {
|
||
let first = (0..pw).find(|&x| kn[y * pw + x]);
|
||
let Some(first) = first else { continue };
|
||
let last = (0..pw).rev().find(|&x| kn[y * pw + x]).unwrap_or(first);
|
||
for x in 0..first {
|
||
let m = (first + fold(first - x)).min(last);
|
||
let (d, s) = ((y * pw + x) * 3, (y * pw + m) * 3);
|
||
canvas.copy_within(s..s + 3, d);
|
||
}
|
||
for x in last + 1..pw {
|
||
let m = last.saturating_sub(fold(x - last)).max(first);
|
||
let (d, s) = ((y * pw + x) * 3, (y * pw + m) * 3);
|
||
canvas.copy_within(s..s + 3, d);
|
||
}
|
||
}
|
||
MirroredContext {
|
||
width: pw,
|
||
height: ph,
|
||
ring: RING,
|
||
rgb: canvas,
|
||
hole,
|
||
}
|
||
}
|
||
}
|
||
|
||
#[cfg(test)]
|
||
mod tests {
|
||
use super::*;
|
||
|
||
/// The tests' small pictures: a 48-px stride, a given feather.
|
||
fn test_params(feather: usize) -> Params {
|
||
Params {
|
||
stride: 48,
|
||
feather,
|
||
..Params::default()
|
||
}
|
||
}
|
||
|
||
/// Paints every unknown pixel a fixed grey and copies the known ones,
|
||
/// and remembers what it was shown.
|
||
struct Flat {
|
||
tile: usize,
|
||
seen: Vec<(Vec<f32>, Vec<bool>)>,
|
||
}
|
||
|
||
impl Inpainter for Flat {
|
||
fn tile(&self) -> usize {
|
||
self.tile
|
||
}
|
||
fn fill(&mut self, rgb: &[f32], known: &[bool]) -> Result<Vec<f32>, PanoError> {
|
||
self.seen.push((rgb.to_vec(), known.to_vec()));
|
||
Ok(rgb
|
||
.chunks_exact(3)
|
||
.zip(known)
|
||
.flat_map(|(p, &k)| if k { [p[0], p[1], p[2]] } else { [0.5; 3] })
|
||
.collect())
|
||
}
|
||
}
|
||
|
||
fn picture(w: usize, h: usize, border: usize) -> (Vec<f32>, Vec<bool>) {
|
||
let mut rgb = vec![0.0; w * h * 3];
|
||
let mut known = vec![false; w * h];
|
||
for y in 0..h {
|
||
for x in 0..w {
|
||
let i = y * w + x;
|
||
if y >= border && y < h - border {
|
||
known[i] = true;
|
||
rgb[i * 3] = x as f32 / w as f32;
|
||
rgb[i * 3 + 1] = y as f32 / h as f32;
|
||
rgb[i * 3 + 2] = 0.25;
|
||
}
|
||
}
|
||
}
|
||
(rgb, known)
|
||
}
|
||
|
||
#[test]
|
||
fn unknown_pixels_take_the_model_and_known_ones_do_not_move() {
|
||
let (mut rgb, known) = picture(300, 200, 20);
|
||
let before = rgb.clone();
|
||
let mut model = Flat {
|
||
tile: 64,
|
||
seen: Vec::new(),
|
||
};
|
||
let tiles = fill_border(
|
||
&mut rgb,
|
||
300,
|
||
200,
|
||
&known,
|
||
&mut model,
|
||
test_params(0),
|
||
&mut |_, _| {},
|
||
)
|
||
.unwrap();
|
||
assert!(tiles > 0);
|
||
for i in 0..300 * 200 {
|
||
if known[i] {
|
||
assert_eq!(&rgb[i * 3..i * 3 + 3], &before[i * 3..i * 3 + 3]);
|
||
} else {
|
||
for c in 0..3 {
|
||
assert!((rgb[i * 3 + c] - 0.5).abs() < 1e-4, "pixel {i}");
|
||
}
|
||
}
|
||
}
|
||
}
|
||
|
||
#[test]
|
||
fn the_model_is_shown_mirrored_context_not_black() {
|
||
let (mut rgb, known) = picture(300, 200, 20);
|
||
let mut model = Flat {
|
||
tile: 64,
|
||
seen: Vec::new(),
|
||
};
|
||
fill_border(
|
||
&mut rgb,
|
||
300,
|
||
200,
|
||
&known,
|
||
&mut model,
|
||
Params {
|
||
mirror_depth: 48,
|
||
..test_params(0)
|
||
},
|
||
&mut |_, _| {},
|
||
)
|
||
.unwrap();
|
||
for (tile_rgb, tile_known) in &model.seen {
|
||
let known_non_black = tile_rgb
|
||
.chunks_exact(3)
|
||
.zip(tile_known)
|
||
.filter(|(_, &k)| k)
|
||
.any(|(p, _)| p.iter().any(|v| *v > 0.0));
|
||
assert!(known_non_black);
|
||
}
|
||
}
|
||
|
||
#[test]
|
||
fn an_open_void_reaches_the_tile_edge_and_stays_unknown_beyond_the_band() {
|
||
// A 150-tall hole above and below; bands of 96. With no ring the
|
||
// first band's tiles sit at the picture's edge, so a tile's top
|
||
// row is unknown, and the rows deeper than the band are unknown
|
||
// too — not "known" coarse fill — exactly as the model was trained.
|
||
let (mut rgb, known) = picture(200, 500, 150);
|
||
let mut model = Flat {
|
||
tile: 64,
|
||
seen: Vec::new(),
|
||
};
|
||
fill_border(
|
||
&mut rgb,
|
||
200,
|
||
500,
|
||
&known,
|
||
&mut model,
|
||
Params {
|
||
band: 96,
|
||
..test_params(0)
|
||
},
|
||
&mut |_, _| {},
|
||
)
|
||
.unwrap();
|
||
// The first pass's tile at the picture's top edge is unknown
|
||
// through and through: the hole is 150 deep, the tile 64, and
|
||
// nothing beyond the band was presented as known. With a ring, or
|
||
// with the far side shown as coarse fill, no tile is ever all hole.
|
||
assert!(model.seen.iter().any(|(_, k)| k.iter().all(|&v| !v)));
|
||
for i in 0..200 * 500 {
|
||
if !known[i] {
|
||
assert!((rgb[i * 3] - 0.5).abs() < 1e-4, "pixel {i}");
|
||
}
|
||
}
|
||
}
|
||
|
||
#[test]
|
||
fn a_picture_smaller_than_the_tile_is_still_filled_when_the_void_is_open() {
|
||
// The merge page's preview is 1600 wide and a few hundred tall —
|
||
// shorter than a 512 tile. With no ring the canvas is padded to a
|
||
// tile, the padding hole, and the border is still filled.
|
||
let (mut rgb, known) = picture(300, 40, 8);
|
||
let mut model = Flat {
|
||
tile: 64,
|
||
seen: Vec::new(),
|
||
};
|
||
let tiles = fill_border(
|
||
&mut rgb,
|
||
300,
|
||
40,
|
||
&known,
|
||
&mut model,
|
||
test_params(0),
|
||
&mut |_, _| {},
|
||
)
|
||
.unwrap();
|
||
assert!(tiles > 0, "no tile fitted a picture shorter than the tile");
|
||
for i in 0..300 * 40 {
|
||
if !known[i] {
|
||
assert!((rgb[i * 3] - 0.5).abs() < 1e-4, "pixel {i}");
|
||
}
|
||
}
|
||
// And the model saw the padding as hole, never as black content.
|
||
for (_, k) in &model.seen {
|
||
assert_eq!(k.len(), 64 * 64);
|
||
}
|
||
}
|
||
|
||
#[test]
|
||
fn the_fine_passes_run_in_bands_after_the_coarse_one() {
|
||
// A 150-tall hole above and below a picture: the coarse pass sees
|
||
// it at a quarter; the fine passes need two bands of BAND pixels.
|
||
let (mut rgb, known) = picture(200, 500, 150);
|
||
let mut model = Flat {
|
||
tile: 64,
|
||
seen: Vec::new(),
|
||
};
|
||
fill_border(
|
||
&mut rgb,
|
||
200,
|
||
500,
|
||
&known,
|
||
&mut model,
|
||
Params {
|
||
coarse: 4,
|
||
band: 96,
|
||
mirror_depth: 48,
|
||
..test_params(0)
|
||
},
|
||
&mut |_, _| {},
|
||
)
|
||
.unwrap();
|
||
assert!(model.seen.len() > 4);
|
||
// Every unknown pixel was reached.
|
||
for i in 0..200 * 500 {
|
||
if !known[i] {
|
||
assert!((rgb[i * 3] - 0.5).abs() < 1e-4, "pixel {i}");
|
||
}
|
||
}
|
||
}
|
||
|
||
#[test]
|
||
fn the_seam_ramps_from_real_to_invented_across_the_feather() {
|
||
let (mut rgb, known) = picture(300, 200, 20);
|
||
let before = rgb.clone();
|
||
let mut model = Flat {
|
||
tile: 64,
|
||
seen: Vec::new(),
|
||
};
|
||
fill_border(
|
||
&mut rgb,
|
||
300,
|
||
200,
|
||
&known,
|
||
&mut model,
|
||
test_params(8),
|
||
&mut |_, _| {},
|
||
)
|
||
.unwrap();
|
||
// Row 20 is the real edge; the margin runs to row 27. At the edge
|
||
// the value is the model's grey, eight rows in it is the picture's.
|
||
let at = |y: usize| rgb[(y * 300 + 150) * 3 + 2];
|
||
assert!((at(20) - 0.5).abs() < 0.05, "{}", at(20));
|
||
assert!((at(29) - before[(29 * 300 + 150) * 3 + 2]).abs() < 1e-4);
|
||
let (lo, hi) = (at(20).min(at(29)), at(20).max(at(29)));
|
||
assert!(
|
||
at(23) > lo + 0.02 && at(23) < hi - 0.02,
|
||
"{} between {lo} and {hi}",
|
||
at(23)
|
||
);
|
||
// The hole itself is the model's.
|
||
assert!((at(5) - 0.5).abs() < 1e-4);
|
||
}
|
||
|
||
#[test]
|
||
fn erosion_shrinks_the_known_region_from_every_edge() {
|
||
let (_, mut known) = picture(20, 20, 4);
|
||
erode(&mut known, 20, 20, 2);
|
||
assert!(known[8 * 20 + 10]);
|
||
assert!(!known[5 * 20 + 10]);
|
||
assert!(!known[8 * 20 + 1]);
|
||
}
|
||
|
||
#[test]
|
||
fn distance_counts_pixels_from_the_known_region() {
|
||
let (_, known) = picture(20, 20, 4);
|
||
let d = distance_to_known(&known, 20, 20);
|
||
assert_eq!(d[4 * 20 + 10], 0.0);
|
||
assert!((d[3 * 20 + 10] - 1.0).abs() < 1e-6);
|
||
assert!((d[10] - 4.0).abs() < 1e-6);
|
||
}
|
||
|
||
#[test]
|
||
fn a_fully_covered_picture_runs_nothing() {
|
||
let (mut rgb, known) = picture(100, 100, 0);
|
||
let mut model = Flat {
|
||
tile: 64,
|
||
seen: Vec::new(),
|
||
};
|
||
assert_eq!(
|
||
fill_border(
|
||
&mut rgb,
|
||
100,
|
||
100,
|
||
&known,
|
||
&mut model,
|
||
test_params(0),
|
||
&mut |_, _| {}
|
||
)
|
||
.unwrap(),
|
||
0
|
||
);
|
||
}
|
||
|
||
#[test]
|
||
fn the_context_mirrors_the_top_rows_upward() {
|
||
let (rgb, known) = picture(40, 30, 5);
|
||
let ctx = MirroredContext::build(&rgb, 40, 30, &known, 48, 64);
|
||
let x = RING + 10;
|
||
let first = RING + 5;
|
||
for k in 1..=4 {
|
||
let above = ((first - k) * ctx.width + x) * 3;
|
||
let mirror = ((first + k) * ctx.width + x) * 3;
|
||
assert_eq!(&ctx.rgb[above..above + 3], &ctx.rgb[mirror..mirror + 3]);
|
||
}
|
||
assert!(ctx.hole[(RING + 2) * ctx.width + x]);
|
||
assert!(!ctx.hole[(RING - 2) * ctx.width + x]);
|
||
}
|
||
|
||
#[test]
|
||
fn the_mirror_reaches_no_deeper_than_its_band() {
|
||
// A ridge 200 rows in must not appear in the ring: beyond the band
|
||
// the reflection folds back towards the edge rather than on into
|
||
// the picture.
|
||
let (mut rgb, known) = picture(40, 400, 5);
|
||
let ridge = 5 + 200;
|
||
for x in 0..40 {
|
||
rgb[(ridge * 40 + x) * 3..(ridge * 40 + x) * 3 + 3].copy_from_slice(&[0.9, 0.1, 0.1]);
|
||
}
|
||
let ctx = MirroredContext::build(&rgb, 40, 400, &known, 48, 64);
|
||
let x = RING + 10;
|
||
for y in 0..RING + 5 {
|
||
let p = (y * ctx.width + x) * 3;
|
||
assert!(
|
||
ctx.rgb[p] < 0.5,
|
||
"row {y} of the ring shows the ridge ({:?})",
|
||
&ctx.rgb[p..p + 3]
|
||
);
|
||
}
|
||
assert_eq!(fold(0, 48), 0);
|
||
assert_eq!(fold(48, 48), 48);
|
||
assert_eq!(fold(58, 48), 38);
|
||
assert_eq!(fold(96, 48), 0);
|
||
assert_eq!(fold(99, 48), 3);
|
||
}
|
||
}
|