Record what the spec got wrong about the model that exists
docs/segmentation.md §4 priced arm B as costing a C dependency under the NDK and treated that as most of the difference between the arms. It is not a cost that has to be paid: `ort`'s `alternative-backend` disables its linking entirely and `ort-tract` supplies the API from tract, which is pure Rust. D13's "largest exception the policy would tolerate" turns out not to be needed, and the answer generalises to the face pipeline — so D13's runtime half is now answered and only its licensing half is open. Three findings contradict §4 outright and are recorded as F4-F6 rather than quietly designed around. There is no ADE20K-trained YOLO, so the shipped vocabulary selects subjects and not stuff — "select the sky" comes from the watershed or from nowhere. It is instance segmentation, so it partitions nothing and two people come back as two instances. And tract cannot parse a dynamic-shape export, which fixes the input at 640 square and makes tiling the only route to more semantic resolution. Arm C ships, but §8's criteria are not what decided it, and saying so matters more than claiming the process worked. §8 asked for a two- interaction margin over arm A on a traced corpus. That comparison was never run: F4 and F5 changed what the arms are, and a model that recognises subjects but has no word for sky cannot be a selection tool alone, while a watershed cannot tell a person from the wall behind them. They stopped being candidates and became complements. What is *not* done is written down as plainly: the 24-image corpus is untraced, so M1-M4 have no numbers and "this feels right" has not become one. M5 is answered on one device only, and region ids now reach the sidecar — so a cross-vendor divergence would mean a mask written on the desktop meaning something else on Android. F3 stands.
This commit is contained in:
@@ -535,4 +535,182 @@ mod tests {
|
||||
stronger.prior.strength = 0.1;
|
||||
assert_eq!(tuning_hash(&base), tuning_hash(&stronger));
|
||||
}
|
||||
|
||||
// ------------------------------------------------------------------
|
||||
// On a device, end to end
|
||||
// ------------------------------------------------------------------
|
||||
|
||||
/// A bright disc on a dark ground: one unambiguous boundary, which is the
|
||||
/// backlit-silhouette control case from docs/segmentation.md §9.
|
||||
fn disc(size: u32) -> Vec<u8> {
|
||||
let r = size as f32 * 0.28;
|
||||
let c = size as f32 / 2.0;
|
||||
let mut out = Vec::with_capacity((size * size) as usize * 4);
|
||||
for y in 0..size {
|
||||
for x in 0..size {
|
||||
let d = ((x as f32 + 0.5 - c).powi(2) + (y as f32 + 0.5 - c).powi(2)).sqrt();
|
||||
let v = if d < r { 230 } else { 30 };
|
||||
out.extend_from_slice(&[v, v, v, 255]);
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
fn context() -> Option<dr_gpu::GpuContext> {
|
||||
pollster::block_on(dr_gpu::GpuContext::new_headless()).ok()
|
||||
}
|
||||
|
||||
/// The whole arm-A chain on a real adapter: watershed, region graph, merge
|
||||
/// tree, and a click landing on the disc rather than on the ground.
|
||||
///
|
||||
/// The CPU tests above use hand-built fields, which cannot catch a
|
||||
/// watershed that produces nothing, a proxy scaled the wrong way, or a
|
||||
/// click transformed into the wrong pixel.
|
||||
#[test]
|
||||
fn a_real_segmentation_separates_the_disc_from_the_ground() {
|
||||
let Some(ctx) = context() else {
|
||||
eprintln!("no adapter; skipping");
|
||||
return;
|
||||
};
|
||||
|
||||
const SIZE: u32 = 256;
|
||||
let rgba = disc(SIZE);
|
||||
let source = dr_gpu::DemosaicedImage::from_rgba8(&ctx, &rgba, SIZE, SIZE).expect("upload");
|
||||
let rgb: Vec<f32> = rgba
|
||||
.chunks_exact(4)
|
||||
.flat_map(|p| [p[0] as f32 / 255.0, p[1] as f32 / 255.0, p[2] as f32 / 255.0])
|
||||
.collect();
|
||||
|
||||
// Arm A alone. The model has no COCO class for "grey disc", so running
|
||||
// it here would cost half a second to contribute nothing — and the
|
||||
// point of this test is the watershed half.
|
||||
let options = Options {
|
||||
semantic: false,
|
||||
..Options::default()
|
||||
};
|
||||
|
||||
let mut seg = compute(&ctx, &source, &rgb, SIZE as usize, SIZE as usize, &options)
|
||||
.expect("segmentation");
|
||||
assert!(seg.region_count() > 1, "a boundary should make regions");
|
||||
|
||||
// Coarse enough that the disc is one region rather than several.
|
||||
seg.set_level(4);
|
||||
|
||||
let centre = seg.group_at(0.5, 0.5).expect("centre is in frame");
|
||||
let corner = seg.group_at(0.04, 0.04).expect("corner is in frame");
|
||||
assert_ne!(
|
||||
centre, corner,
|
||||
"the disc and the ground must not be one group"
|
||||
);
|
||||
|
||||
// And the selection must be the disc, not the whole frame: a mask that
|
||||
// covers everything is the failure mode a "they differ" assertion on
|
||||
// its own would not catch.
|
||||
let selected = seg.regions_at(0.5, 0.5);
|
||||
assert!(!selected.is_empty());
|
||||
assert!(
|
||||
selected.len() < seg.region_count(),
|
||||
"selecting the disc should not select every region"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_overlay_matches_the_segmentation_it_describes() {
|
||||
let Some(ctx) = context() else {
|
||||
eprintln!("no adapter; skipping");
|
||||
return;
|
||||
};
|
||||
|
||||
const SIZE: u32 = 128;
|
||||
let rgba = disc(SIZE);
|
||||
let source = dr_gpu::DemosaicedImage::from_rgba8(&ctx, &rgba, SIZE, SIZE).expect("upload");
|
||||
let rgb: Vec<f32> = rgba
|
||||
.chunks_exact(4)
|
||||
.flat_map(|p| [p[0] as f32 / 255.0, p[1] as f32 / 255.0, p[2] as f32 / 255.0])
|
||||
.collect();
|
||||
|
||||
let options = Options {
|
||||
semantic: false,
|
||||
..Options::default()
|
||||
};
|
||||
let mut seg = compute(&ctx, &source, &rgb, SIZE as usize, SIZE as usize, &options)
|
||||
.expect("segmentation");
|
||||
seg.set_level(4);
|
||||
|
||||
let (pixels, w, h) = seg.overlay_rgba();
|
||||
assert_eq!(pixels.len(), (w * h) as usize * 4);
|
||||
|
||||
// The disc's centre and the ground must be drawn differently, or the
|
||||
// overlay is not showing the thing it claims to show.
|
||||
let at = |x: u32, y: u32| {
|
||||
let p = ((y * w + x) * 4) as usize;
|
||||
[pixels[p], pixels[p + 1], pixels[p + 2]]
|
||||
};
|
||||
assert_ne!(at(w / 2, h / 2), at(2, 2));
|
||||
}
|
||||
|
||||
/// The signature has to change when a mask's ids would mean something
|
||||
/// different, and this is the case that actually happens: the same image
|
||||
/// segmented at another proxy size.
|
||||
#[test]
|
||||
fn re_segmenting_at_another_size_invalidates_stored_ids() {
|
||||
let Some(ctx) = context() else {
|
||||
eprintln!("no adapter; skipping");
|
||||
return;
|
||||
};
|
||||
|
||||
const SIZE: u32 = 192;
|
||||
let rgba = disc(SIZE);
|
||||
let source = dr_gpu::DemosaicedImage::from_rgba8(&ctx, &rgba, SIZE, SIZE).expect("upload");
|
||||
let rgb: Vec<f32> = rgba
|
||||
.chunks_exact(4)
|
||||
.flat_map(|p| [p[0] as f32 / 255.0, p[1] as f32 / 255.0, p[2] as f32 / 255.0])
|
||||
.collect();
|
||||
|
||||
let mut coarse = Options {
|
||||
semantic: false,
|
||||
..Options::default()
|
||||
};
|
||||
coarse.segment.max_edge = 96;
|
||||
|
||||
let mut fine = coarse;
|
||||
fine.segment.max_edge = 192;
|
||||
|
||||
let a = compute(&ctx, &source, &rgb, SIZE as usize, SIZE as usize, &coarse).expect("a");
|
||||
let b = compute(&ctx, &source, &rgb, SIZE as usize, SIZE as usize, &fine).expect("b");
|
||||
assert_ne!(
|
||||
a.signature(),
|
||||
b.signature(),
|
||||
"ids from one proxy must not be read as ids from another"
|
||||
);
|
||||
}
|
||||
|
||||
/// Run-to-run stability on one device, which is what lets a selection be
|
||||
/// stored and reopened at all (M5).
|
||||
#[test]
|
||||
fn the_same_image_segments_identically_twice() {
|
||||
let Some(ctx) = context() else {
|
||||
eprintln!("no adapter; skipping");
|
||||
return;
|
||||
};
|
||||
|
||||
const SIZE: u32 = 128;
|
||||
let rgba = disc(SIZE);
|
||||
let source = dr_gpu::DemosaicedImage::from_rgba8(&ctx, &rgba, SIZE, SIZE).expect("upload");
|
||||
let rgb: Vec<f32> = rgba
|
||||
.chunks_exact(4)
|
||||
.flat_map(|p| [p[0] as f32 / 255.0, p[1] as f32 / 255.0, p[2] as f32 / 255.0])
|
||||
.collect();
|
||||
let options = Options {
|
||||
semantic: false,
|
||||
..Options::default()
|
||||
};
|
||||
|
||||
let a = compute(&ctx, &source, &rgb, SIZE as usize, SIZE as usize, &options).expect("a");
|
||||
let b = compute(&ctx, &source, &rgb, SIZE as usize, SIZE as usize, &options).expect("b");
|
||||
|
||||
assert_eq!(a.signature(), b.signature());
|
||||
assert_eq!(a.region_count(), b.region_count());
|
||||
assert_eq!(a.regions_at(0.5, 0.5), b.regions_at(0.5, 0.5));
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user