Test that the category geometry means what it says
The scene tests so far checked the descriptor and the arithmetic. Neither would have noticed if `rasterise` put the sky along the bottom of the frame, because both build their own weights and never ask where those weights land. Three that do: - **Weight stays on the side it came from.** Fill the top half of the grid, read the top and bottom quarters of the image. A flipped y axis is the mistake this code is actually prone to — it is a letterbox inverse, and the numbers stay perfectly plausible when it is wrong. - **No transpose.** The vertical check alone passes under a transpose, which maps a top band onto a left band. A horizontal split is what distinguishes them, and neither test is worth much without the other. - **Coverage is a fraction.** A quarter of the cells must read 0.25. The scene tab hides a category below half a percent, so an error of a factor of the grid size would hide everything or nothing — and both look like the model failing rather than the arithmetic. `Scene::from_weights` is test-only and exists because the property under test needs weights whose correct destination is known in advance, which no real inference can provide. It uses a square window so the letterbox is the identity: any offset these find is the mapping's own rather than the padding's. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -57,6 +57,8 @@ use std::sync::Arc;
|
||||
|
||||
use ndarray::ArrayView3;
|
||||
|
||||
#[cfg(test)]
|
||||
use crate::semantic::INPUT_EDGE;
|
||||
use crate::semantic::{install_backend, Letterbox, Window};
|
||||
use crate::SegmentError;
|
||||
|
||||
@@ -353,6 +355,34 @@ impl Scene {
|
||||
}
|
||||
}
|
||||
|
||||
impl Scene {
|
||||
/// Build a scene from known weights, for tests.
|
||||
///
|
||||
/// The geometry in [`Scene::rasterise`] is a letterbox inverse, and a
|
||||
/// letterbox inverse is exactly the kind of code that produces confident,
|
||||
/// plausible, wrong answers. Testing it needs weights whose correct
|
||||
/// destination is known in advance, which a real inference can never give.
|
||||
#[cfg(test)]
|
||||
fn from_weights(names: Vec<Arc<str>>, weight: Vec<f32>, gw: usize, gh: usize) -> Self {
|
||||
// A square source, so the letterbox is the identity and any offset
|
||||
// this finds is the mapping's own rather than the padding's.
|
||||
let window = Window {
|
||||
x: 0.0,
|
||||
y: 0.0,
|
||||
w: INPUT_EDGE as f32,
|
||||
h: INPUT_EDGE as f32,
|
||||
};
|
||||
Self {
|
||||
names,
|
||||
weight,
|
||||
grid_width: gw,
|
||||
grid_height: gh,
|
||||
letterbox: Letterbox::fit(window.w, window.h),
|
||||
window,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Read `models/scene/categories.txt`, resolving class names to indices.
|
||||
///
|
||||
/// Hand-written rather than generated, unlike the `.classes.json` beside it,
|
||||
@@ -477,6 +507,98 @@ mod tests {
|
||||
assert!(cats.iter().any(|c| &*c.name == "vegetation"));
|
||||
}
|
||||
|
||||
/// Weight put in the top half of the grid must come back in the top half
|
||||
/// of the image.
|
||||
///
|
||||
/// The one property that makes a category mask worth anything: if the
|
||||
/// model says "sky up here" and `rasterise` puts it down there, every
|
||||
/// grade lands on the wrong half of the photograph and nothing about the
|
||||
/// numbers looks wrong. A vertical split is the cheapest arrangement that
|
||||
/// catches a flipped axis, and a flipped axis is the mistake this code is
|
||||
/// actually prone to.
|
||||
#[test]
|
||||
fn rasterise_keeps_weight_on_the_side_it_came_from() {
|
||||
let (gw, gh) = (8usize, 8usize);
|
||||
let mut weight = vec![0.0f32; gw * gh];
|
||||
for y in 0..gh / 2 {
|
||||
for x in 0..gw {
|
||||
weight[y * gw + x] = 1.0;
|
||||
}
|
||||
}
|
||||
let scene = Scene::from_weights(vec![Arc::from("sky")], weight, gw, gh);
|
||||
|
||||
let (w, h) = (64usize, 64usize);
|
||||
let mask = scene.rasterise(0, w, h).expect("category 0 exists");
|
||||
|
||||
let mean = |y0: usize, y1: usize| {
|
||||
let band: f32 = (y0..y1)
|
||||
.flat_map(|y| (0..w).map(move |x| (y, x)))
|
||||
.map(|(y, x)| mask[y * w + x])
|
||||
.sum();
|
||||
band / ((y1 - y0) * w) as f32
|
||||
};
|
||||
let top = mean(0, h / 4);
|
||||
let bottom = mean(3 * h / 4, h);
|
||||
assert!(
|
||||
top > 0.9,
|
||||
"the half that had the weight should keep it: {top}"
|
||||
);
|
||||
assert!(
|
||||
bottom < 0.1,
|
||||
"the half that had none should stay empty: {bottom}"
|
||||
);
|
||||
}
|
||||
|
||||
/// A horizontal split too, because a transpose passes the vertical test.
|
||||
///
|
||||
/// Swapping x and y maps a top band onto a left band, and the check above
|
||||
/// would still see the top band full. Two axes is what makes the pair
|
||||
/// meaningful; either alone is not.
|
||||
#[test]
|
||||
fn rasterise_does_not_transpose() {
|
||||
let (gw, gh) = (8usize, 8usize);
|
||||
let mut weight = vec![0.0f32; gw * gh];
|
||||
for y in 0..gh {
|
||||
for x in 0..gw / 2 {
|
||||
weight[y * gw + x] = 1.0;
|
||||
}
|
||||
}
|
||||
let scene = Scene::from_weights(vec![Arc::from("left")], weight, gw, gh);
|
||||
|
||||
let (w, h) = (64usize, 64usize);
|
||||
let mask = scene.rasterise(0, w, h).expect("category 0 exists");
|
||||
let mean = |x0: usize, x1: usize| {
|
||||
let band: f32 = (0..h)
|
||||
.flat_map(|y| (x0..x1).map(move |x| (y, x)))
|
||||
.map(|(y, x)| mask[y * w + x])
|
||||
.sum();
|
||||
band / (h * (x1 - x0)) as f32
|
||||
};
|
||||
assert!(mean(0, w / 4) > 0.9, "left stays left");
|
||||
assert!(mean(3 * w / 4, w) < 0.1, "right stays empty");
|
||||
}
|
||||
|
||||
/// Coverage is the fraction of the frame, not a count or a sum.
|
||||
///
|
||||
/// The scene tab hides a category below half a percent, so a coverage that
|
||||
/// is off by a factor of the grid size would either hide everything or
|
||||
/// hide nothing, and both look like the model failing rather than the
|
||||
/// arithmetic.
|
||||
#[test]
|
||||
fn coverage_is_a_fraction_of_the_frame() {
|
||||
let (gw, gh) = (10usize, 10usize);
|
||||
let mut weight = vec![0.0f32; gw * gh];
|
||||
for cell in weight.iter_mut().take(25) {
|
||||
*cell = 1.0;
|
||||
}
|
||||
let scene = Scene::from_weights(vec![Arc::from("quarter")], weight, gw, gh);
|
||||
let c = scene.coverage(0);
|
||||
assert!(
|
||||
(c - 0.25).abs() < 1e-6,
|
||||
"a quarter of the cells is 0.25, got {c}"
|
||||
);
|
||||
}
|
||||
|
||||
/// Softmax then group: the reported weights must never exceed one, and
|
||||
/// must equal one exactly when the categories name every class.
|
||||
#[test]
|
||||
|
||||
Reference in New Issue
Block a user