Merge: touch selection and drag, from the gallery-selection branch
Verified before merge: fmt clean, clippy -D warnings clean, 563 dr-ui tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> # Conflicts: # docs/traceability.md
This commit is contained in:
+1
-1
@@ -1,6 +1,6 @@
|
||||
# Model weights live in LFS.
|
||||
#
|
||||
# `core/dr-segment/models/*.onnx` is ~11 MB of binary that changes wholesale
|
||||
# `models/**/*.onnx` is tens of MB of binary that changes wholesale
|
||||
# when it changes at all. In ordinary git objects every future revision of it
|
||||
# would be stored in full, in every clone, forever — and the one thing nobody
|
||||
# can do with it is a useful diff.
|
||||
|
||||
@@ -57,7 +57,7 @@ jobs:
|
||||
|
||||
# The model, which is in LFS and is not optional.
|
||||
#
|
||||
# `core/dr-segment/models/*.onnx` is tracked in LFS (.gitattributes), so a
|
||||
# `models/**/*.onnx` is tracked in LFS (.gitattributes), so a
|
||||
# plain checkout writes a ~130-byte pointer where 11 MB should be, and
|
||||
# `dr-segment`'s build script panics by design rather than embedding a
|
||||
# pointer and failing at inference. That failure reads like a broken build
|
||||
@@ -97,7 +97,7 @@ jobs:
|
||||
git config --local lfs.url \
|
||||
"https://x-access-token:${LFS_TOKEN}@gitea.tourolle.paris/dtourolle/DarkRoom.git/info/lfs"
|
||||
git lfs pull
|
||||
ls -l core/dr-segment/models/
|
||||
ls -lR models/
|
||||
|
||||
- name: Cache cargo
|
||||
uses: actions/cache@v4
|
||||
@@ -174,7 +174,7 @@ jobs:
|
||||
|
||||
# The model, which is in LFS and is not optional.
|
||||
#
|
||||
# `core/dr-segment/models/*.onnx` is tracked in LFS (.gitattributes), so a
|
||||
# `models/**/*.onnx` is tracked in LFS (.gitattributes), so a
|
||||
# plain checkout writes a ~130-byte pointer where 11 MB should be, and
|
||||
# `dr-segment`'s build script panics by design rather than embedding a
|
||||
# pointer and failing at inference. That failure reads like a broken build
|
||||
@@ -214,7 +214,7 @@ jobs:
|
||||
git config --local lfs.url \
|
||||
"https://x-access-token:${LFS_TOKEN}@gitea.tourolle.paris/dtourolle/DarkRoom.git/info/lfs"
|
||||
git lfs pull
|
||||
ls -l core/dr-segment/models/
|
||||
ls -lR models/
|
||||
|
||||
- name: Cache cargo
|
||||
uses: actions/cache@v4
|
||||
|
||||
@@ -60,7 +60,7 @@ fn android_main(app: slint::android::AndroidApp) {
|
||||
}
|
||||
|
||||
// After the data dir and before anything asks whether a model is present.
|
||||
install_bundled_face_models(&app);
|
||||
install_bundled_models(&app);
|
||||
|
||||
// Before `init_with_event_listener`, which takes `app` by value and is the
|
||||
// last moment anything can ask the activity a question. Not an ordering
|
||||
@@ -126,38 +126,65 @@ fn android_main(app: slint::android::AndroidApp) {
|
||||
}
|
||||
}
|
||||
|
||||
/// Unpack the face models the APK carries, if it carries any.
|
||||
/// Unpack the models the APK carries, if it carries any.
|
||||
///
|
||||
/// # Why Android needs this and no other platform does
|
||||
///
|
||||
/// The weights are not a build input and are not in the repository — the
|
||||
/// InsightFace grant is research-only and incompatible with this project's
|
||||
/// licence (docs/faces.md §2), so a desktop user fetches them, runs
|
||||
/// `tools/fix-face-model-shapes.sh` over them, and drops the result into
|
||||
/// `~/.local/share/darkroom/models/`. **That gesture does not exist on
|
||||
/// Android.** `internal_data_path` is app-private, `run-as` needs a debuggable
|
||||
/// build, and there is no picker and no fetch in the app, so a phone had no way
|
||||
/// to acquire a model at all and face indexing reported itself permanently off.
|
||||
/// A desktop build reads its models from a path — the account's directory, the
|
||||
/// shared one, or `$XDG_DATA_DIRS` where a package put them. **Android has no
|
||||
/// such path.** `internal_data_path` is app-private, `run-as` needs a
|
||||
/// debuggable build, and an asset inside a package is not a path anything can
|
||||
/// open (ARCH §6.9), so a phone had no way to reach a model at all.
|
||||
///
|
||||
/// So a locally-built APK may carry the pair in `assets/models/`, which
|
||||
/// `assemble-apk.sh` includes when the tree has them and omits when it does
|
||||
/// not. Nothing changes about what the repository holds or what a published
|
||||
/// build could redistribute; this only gives a self-built APK the same route a
|
||||
/// desktop build has always had.
|
||||
/// So the APK carries them in `assets/models/` and this copies them out, once,
|
||||
/// into the same shared directory a desktop install uses. After that every
|
||||
/// lookup in `dr_ui::library` finds them exactly where it finds a desktop
|
||||
/// user's.
|
||||
///
|
||||
/// Absent assets are the ordinary case, not an error — the same quiet "no model
|
||||
/// installed" state a fresh desktop install is in.
|
||||
/// # The two sets are not the same kind of thing
|
||||
///
|
||||
/// **Face weights are absent from the repository by design.** The InsightFace
|
||||
/// grant is research-only and incompatible with this project's licence
|
||||
/// (docs/faces.md §2), so a desktop user fetches them, runs
|
||||
/// `tools/fix-face-model-shapes.sh` over them, and drops the result in. A build
|
||||
/// that carries none is the ordinary case and face indexing simply stays off.
|
||||
///
|
||||
/// **The scene model is committed** (AGPL, compatible — `models/LICENCE.md`),
|
||||
/// so a build carrying none means a checkout without `git lfs pull` rather than
|
||||
/// a deliberate omission. It is still not an error here: the scene tab reports
|
||||
/// itself unavailable the same way face indexing does, because a photo editor
|
||||
/// that refuses to start over a missing grading feature is worse than one that
|
||||
/// starts without it.
|
||||
///
|
||||
/// # Why it is not `include_bytes!` like the instance model
|
||||
///
|
||||
/// Size. The instance model is 11 MB and compiled in; the scene model is 24 MB
|
||||
/// on top of that, and a 35 MB constant in the binary is paid by every install
|
||||
/// whether or not the tab is opened. Assets are also *stored* rather than
|
||||
/// deflated in the APK (see `assemble-apk.sh`), so unpacking is a copy rather
|
||||
/// than an inflate.
|
||||
#[cfg(target_os = "android")]
|
||||
fn install_bundled_face_models(app: &slint::android::AndroidApp) {
|
||||
fn install_bundled_models(app: &slint::android::AndroidApp) {
|
||||
use std::io::Read;
|
||||
|
||||
// The **shape-fixed** names, matching what `library::face_models` looks
|
||||
// for: tract cannot parse either InsightFace graph with its dynamic input
|
||||
// dimension, so what ships here has already been through
|
||||
// `tools/fix-face-model-shapes.sh`.
|
||||
const BUNDLED: [(&std::ffi::CStr, &str); 2] = [
|
||||
// The face names are the **shape-fixed** exports, matching what
|
||||
// `library::face_models` looks for: tract cannot parse either InsightFace
|
||||
// graph with its dynamic input dimension, so what ships here has already
|
||||
// been through `tools/fix-face-model-shapes.sh`.
|
||||
//
|
||||
// The scene entries are three files rather than one because the graph alone
|
||||
// decodes to 150 anonymous channels — `library::scene_model` wants the
|
||||
// vocabulary and the category descriptor beside it, and requires all three
|
||||
// before it reports the tab available.
|
||||
const BUNDLED: [(&std::ffi::CStr, &str); 5] = [
|
||||
(c"models/scrfd_500m_640.onnx", "scrfd_500m_640.onnx"),
|
||||
(c"models/arcface_mbf_b1.onnx", "arcface_mbf_b1.onnx"),
|
||||
(c"models/yolo26s-sem-ade20k.onnx", "yolo26s-sem-ade20k.onnx"),
|
||||
(
|
||||
c"models/yolo26s-sem-ade20k.classes.json",
|
||||
"yolo26s-sem-ade20k.classes.json",
|
||||
),
|
||||
(c"models/categories.txt", "categories.txt"),
|
||||
];
|
||||
|
||||
let dir = dr_ui::shared_face_models_dir();
|
||||
@@ -172,7 +199,7 @@ fn install_bundled_face_models(app: &slint::android::AndroidApp) {
|
||||
continue;
|
||||
}
|
||||
let Some(mut asset) = assets.open(asset_path) else {
|
||||
log::info!("no bundled {name} in this APK; face indexing stays off");
|
||||
log::info!("no bundled {name} in this APK; the feature needing it stays off");
|
||||
continue;
|
||||
};
|
||||
let mut bytes = Vec::new();
|
||||
@@ -185,8 +212,8 @@ fn install_bundled_face_models(app: &slint::android::AndroidApp) {
|
||||
return;
|
||||
}
|
||||
// Written under a temporary name and renamed, because
|
||||
// `library::face_models` decides face indexing is available on
|
||||
// `is_file()` alone. A truncated write — the process backgrounded and
|
||||
// `library::face_models` and `library::scene_model` both decide a
|
||||
// feature is available on `is_file()` alone. A truncated write — the process backgrounded and
|
||||
// killed mid-copy — would otherwise leave a file that passes that test
|
||||
// and fails inside tract, reported to the user as a broken model rather
|
||||
// than a missing one.
|
||||
|
||||
@@ -44,3 +44,13 @@ semantic = ["dep:ort", "dep:ort-tract", "dep:ndarray"]
|
||||
# it must be embedded; a desktop packager pointing at a system model directory,
|
||||
# or a test that only needs the decoder, wants the runtime without the 11 MB.
|
||||
embedded-model = ["semantic"]
|
||||
|
||||
# Compile the *scene* model in too, and off by default where `embedded-model`
|
||||
# is on.
|
||||
#
|
||||
# The asymmetry is its size. At 24 MB it is more than twice the instance model,
|
||||
# and Android reaches it the way it reaches the face weights — unpacked from
|
||||
# APK assets at first launch — rather than by carrying it in the binary. This
|
||||
# feature is for a desktop build with nowhere else to read it from, and for
|
||||
# tests that want the real graph.
|
||||
embedded-scene-model = ["semantic"]
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
//! Check the model is a model and not an LFS pointer.
|
||||
//!
|
||||
//! `models/*.onnx` is stored in Git LFS (see `.gitattributes`). A clone made
|
||||
//! `models/segment/*.onnx` is stored in Git LFS (see `.gitattributes`). A clone made
|
||||
//! without git-lfs installed, or with `GIT_LFS_SKIP_SMUDGE` set, leaves a
|
||||
//! ~130-byte text pointer at that path instead of the weights.
|
||||
//!
|
||||
@@ -12,7 +12,7 @@
|
||||
|
||||
use std::path::Path;
|
||||
|
||||
const MODEL: &str = "models/yolo26n-seg.onnx";
|
||||
const MODEL: &str = "../../models/segment/yolo26n-seg.onnx";
|
||||
|
||||
fn main() {
|
||||
println!("cargo:rerun-if-changed={MODEL}");
|
||||
|
||||
@@ -0,0 +1,174 @@
|
||||
//! Run the scene model over a JPEG, time it, and write what it saw.
|
||||
//!
|
||||
//! Two jobs in one example because they need the same setup and answering
|
||||
//! either one alone leaves the other open.
|
||||
//!
|
||||
//! **Looking.** Same argument as `detect`: no unit test settles whether the
|
||||
//! letterbox inverse in `Scene::rasterise` is right, because an off-by-one
|
||||
//! produces perfectly plausible weights over slightly the wrong pixels. A sky
|
||||
//! mask laid over the photograph settles it in one glance.
|
||||
//!
|
||||
//! **Timing.** Every number quoted while this model was being chosen came off a
|
||||
//! laptop that was compiling other things at the time, which makes them upper
|
||||
//! bounds and nothing better. This exists so the figure that ends up in a
|
||||
//! document came from a quiet machine and can be reproduced on another one.
|
||||
//!
|
||||
//! ```sh
|
||||
//! cargo run -p dr-segment --example scene --release --features embedded-scene-model -- photo.jpg
|
||||
//! cargo run -p dr-segment --example scene --release -- photo.jpg out 20 \
|
||||
//! models/scene/yolo26s-sem-ade20k.onnx
|
||||
//! ```
|
||||
//!
|
||||
//! Writes `<prefix>-<category>.ppm` per category — the photograph darkened
|
||||
//! where the category is absent, so the mask is legible *against the picture it
|
||||
//! came from* rather than as an abstract grey field. PPM for the same reason
|
||||
//! the other examples use it: no encoder dependency, and every viewer reads it.
|
||||
//!
|
||||
//! Timings are reported as a median over the requested run count, with the
|
||||
//! first run excluded. That first pass pays for tract's lazy allocation and is
|
||||
//! not representative of the second image a session decodes.
|
||||
|
||||
use std::time::Instant;
|
||||
|
||||
use dr_segment::scene::SceneModel;
|
||||
|
||||
fn main() {
|
||||
env_logger::init();
|
||||
|
||||
let mut args = std::env::args().skip(1);
|
||||
let Some(path) = args.next() else {
|
||||
eprintln!(
|
||||
"usage: scene <photo.jpg> [out-prefix] [runs] [model.onnx classes.json categories.txt]"
|
||||
);
|
||||
eprintln!(" with --features embedded-scene-model the model arguments may be omitted");
|
||||
std::process::exit(2);
|
||||
};
|
||||
let prefix = args.next().unwrap_or_else(|| "scene".into());
|
||||
let runs: usize = args
|
||||
.next()
|
||||
.and_then(|r| r.parse().ok())
|
||||
.unwrap_or(10)
|
||||
.max(1);
|
||||
|
||||
let (rgb, width, height) = read_jpeg(&path);
|
||||
println!("{path}: {width}×{height}");
|
||||
|
||||
let mut model = match (args.next(), args.next(), args.next()) {
|
||||
(Some(m), Some(c), Some(g)) => {
|
||||
SceneModel::from_path(m, c, g).expect("could not load the scene model")
|
||||
}
|
||||
_ => embedded(),
|
||||
};
|
||||
|
||||
// Excluded from the statistics deliberately — see the header.
|
||||
let warm = Instant::now();
|
||||
let scene = model
|
||||
.analyse(&rgb, width, height)
|
||||
.expect("inference failed");
|
||||
println!("first run: {:?} (allocation included)", warm.elapsed());
|
||||
|
||||
let mut times: Vec<f64> = Vec::with_capacity(runs);
|
||||
for _ in 0..runs {
|
||||
let start = Instant::now();
|
||||
let _ = model
|
||||
.analyse(&rgb, width, height)
|
||||
.expect("inference failed");
|
||||
times.push(start.elapsed().as_secs_f64() * 1000.0);
|
||||
}
|
||||
times.sort_by(f64::total_cmp);
|
||||
println!(
|
||||
"{runs} runs: median {:.0} ms (min {:.0}, max {:.0})",
|
||||
times[times.len() / 2],
|
||||
times[0],
|
||||
times[times.len() - 1],
|
||||
);
|
||||
|
||||
let (gw, gh) = scene.grid_size();
|
||||
println!("logit grid: {gw}×{gh}");
|
||||
println!();
|
||||
|
||||
// Coverage first and sorted, because on any given photograph most
|
||||
// categories are absent and the two or three that are not are the whole
|
||||
// story.
|
||||
let mut ranked: Vec<(usize, f32)> = (0..scene.categories().len())
|
||||
.map(|k| (k, scene.coverage(k)))
|
||||
.collect();
|
||||
ranked.sort_by(|a, b| b.1.total_cmp(&a.1));
|
||||
|
||||
for (k, coverage) in ranked {
|
||||
let name = &scene.categories()[k];
|
||||
println!("{name:>14} {:5.1}%", coverage * 100.0);
|
||||
// A category covering essentially nothing produces a black image and a
|
||||
// file nobody wants; the threshold is what the scene tab would use to
|
||||
// decide whether to offer a slider at all.
|
||||
if coverage < 0.005 {
|
||||
continue;
|
||||
}
|
||||
let mask = scene
|
||||
.rasterise(k, width, height)
|
||||
.expect("category index came from the same Scene");
|
||||
write_overlay(&format!("{prefix}-{name}.ppm"), &rgb, &mask, width, height);
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(feature = "embedded-scene-model")]
|
||||
fn embedded() -> SceneModel {
|
||||
SceneModel::embedded().expect("could not load the embedded scene model")
|
||||
}
|
||||
|
||||
#[cfg(not(feature = "embedded-scene-model"))]
|
||||
fn embedded() -> SceneModel {
|
||||
eprintln!(
|
||||
"no model given, and this build has no embedded one.\n\
|
||||
Either pass the three paths, or rebuild with --features embedded-scene-model."
|
||||
);
|
||||
std::process::exit(2);
|
||||
}
|
||||
|
||||
/// The photograph, dimmed where the category is not.
|
||||
///
|
||||
/// Not a bare greyscale mask: the question being asked is "does this weight
|
||||
/// land on the sky", and a mask on its own cannot answer it — you have to see
|
||||
/// the sky underneath. A floor rather than a multiply, so that a region the
|
||||
/// model gave up on is still visible enough to recognise.
|
||||
fn write_overlay(path: &str, rgb: &[f32], mask: &[f32], width: usize, height: usize) {
|
||||
let mut out = String::with_capacity(64);
|
||||
out.push_str(&format!("P3\n{width} {height}\n255\n"));
|
||||
let mut bytes = out.into_bytes();
|
||||
|
||||
for i in 0..width * height {
|
||||
let w = mask[i].clamp(0.0, 1.0);
|
||||
let gain = 0.15 + 0.85 * w;
|
||||
for c in 0..3 {
|
||||
let v = (rgb[i * 3 + c] * gain * 255.0).clamp(0.0, 255.0) as u8;
|
||||
bytes.extend_from_slice(v.to_string().as_bytes());
|
||||
bytes.push(if c == 2 { b'\n' } else { b' ' });
|
||||
}
|
||||
}
|
||||
|
||||
match std::fs::write(path, bytes) {
|
||||
Ok(()) => println!(" wrote {path}"),
|
||||
Err(e) => eprintln!(" could not write {path}: {e}"),
|
||||
}
|
||||
}
|
||||
|
||||
/// Decode to the tightly packed `f32` RGB the model wants.
|
||||
fn read_jpeg(path: &str) -> (Vec<f32>, usize, usize) {
|
||||
let bytes = std::fs::read(path).expect("could not read the photograph");
|
||||
let mut decoder = zune_jpeg::JpegDecoder::new(&bytes);
|
||||
let pixels = decoder.decode().expect("could not decode the photograph");
|
||||
let info = decoder.info().expect("decoded image has no dimensions");
|
||||
let (width, height) = (info.width as usize, info.height as usize);
|
||||
|
||||
// zune hands back whatever the file had. Three channels is the ordinary
|
||||
// case; one is a greyscale scan, which is worth handling because a
|
||||
// black-and-white frame is exactly the kind of thing someone reaches for
|
||||
// when a colour one looks wrong.
|
||||
let components = pixels.len() / (width * height);
|
||||
let rgb = match components {
|
||||
3 => pixels.iter().map(|&p| p as f32 / 255.0).collect(),
|
||||
1 => pixels.iter().flat_map(|&p| [p as f32 / 255.0; 3]).collect(),
|
||||
n => panic!("unsupported component count: {n}"),
|
||||
};
|
||||
(rgb, width, height)
|
||||
}
|
||||
@@ -1,54 +0,0 @@
|
||||
# Model weights — licensing
|
||||
|
||||
`yolo26n-seg.onnx` is exported from Ultralytics YOLO26n-seg
|
||||
(`https://huggingface.co/Ultralytics/YOLO26`, `yolo26n-seg.pt`) by
|
||||
`tools/export-seg-model.sh`. `yolo26n-seg.classes.json` is that checkpoint's
|
||||
class vocabulary, written out by the same script.
|
||||
|
||||
## The grant
|
||||
|
||||
**Ultralytics releases YOLO under AGPL-3.0**, and the weights carry the same
|
||||
grant as the framework — the HuggingFace repository declares `agpl-3.0` for the
|
||||
checkpoints themselves, not merely for the training code. A commercial licence
|
||||
is offered separately; DarkRoom does not use it and does not need it.
|
||||
|
||||
## What that means for DarkRoom
|
||||
|
||||
DarkRoom is GPL-3.0-or-later. **GPLv3 §13 explicitly permits combination with
|
||||
AGPL-3.0 code**, so redistributing these weights inside this repository is
|
||||
allowed — this is *not* the situation the InsightFace "buffalo" weights would
|
||||
have created, where a non-commercial research grant is simply incompatible with
|
||||
the project's licence and with F-Droid, Flatpak and Play distribution
|
||||
(NFR-COMPAT-2, D13).
|
||||
|
||||
The consequence, and it is a real one: **the combined work is effectively
|
||||
AGPL-3.0.** §13's permission runs one way — the AGPL's §13 network-use condition
|
||||
attaches to the portion under that licence. For a local-first desktop and
|
||||
Android photo editor that condition has no practical bite, because there is no
|
||||
network service offering the combined work to remote users. It would acquire
|
||||
bite the moment any hosted or server-side rendering appeared, and that is the
|
||||
thing to remember rather than rediscover.
|
||||
|
||||
This was decided deliberately (D14), not arrived at by accident, and
|
||||
`docs/segmentation.md` §7 records the reasoning.
|
||||
|
||||
## Class vocabulary — a caveat worth reading
|
||||
|
||||
`docs/segmentation.md` §4 specified YOLO **pretrained on ADE20K**, whose 150
|
||||
classes include the *stuff* categories that matter most in photography — sky,
|
||||
vegetation, water, wall, mountain.
|
||||
|
||||
**No such model exists in usable form.** Checked 2026-08-21: Ultralytics ships
|
||||
YOLO26-seg trained on **COCO**, whose 80 classes are all *things* — person,
|
||||
dog, car, bird, potted plant — and the one HuggingFace repository claiming a
|
||||
YOLO/ADE20K combination (`laxmacl/yolov8-ade20k`) is empty. ADE20K semantic
|
||||
models do exist, but as SegFormer/OneFormer/MaskFormer transformers, not YOLO.
|
||||
|
||||
So the shipped vocabulary selects **subjects**, not **stuff**. "Select the
|
||||
person" works; "select the sky" does not come from the model and must come from
|
||||
the watershed hierarchy instead. That is a narrower arm B than §4 assumed, and
|
||||
it raises rather than lowers the importance of arm C.
|
||||
|
||||
The loader treats the vocabulary as model metadata rather than compiled-in
|
||||
knowledge, so adding a stuff-class model later is a file plus a descriptor, not
|
||||
a code change.
|
||||
@@ -28,17 +28,30 @@
|
||||
//! model is COCO-trained, so it recognises subjects and has no class for sky,
|
||||
//! foliage or wall (`models/LICENCE.md`). Selecting those falls to arm A,
|
||||
//! which never needed a vocabulary to begin with.
|
||||
//!
|
||||
//! # And [`scene`], which is not one of the arms
|
||||
//!
|
||||
//! The three arms all serve *local* adjustment: they exist so a mask can be
|
||||
//! snapped to one region of the picture. [`scene`] serves the opposite move —
|
||||
//! one grade applied to every pixel of a category at once, sky or foliage or
|
||||
//! water — and reads a second, ADE20K-trained model to do it. It shares this
|
||||
//! crate because it shares the runtime and the letterbox, not because it is
|
||||
//! another way of doing the same thing.
|
||||
|
||||
pub mod distance;
|
||||
pub mod hierarchy;
|
||||
pub mod prior;
|
||||
#[cfg(feature = "semantic")]
|
||||
pub mod scene;
|
||||
#[cfg(feature = "semantic")]
|
||||
pub mod semantic;
|
||||
|
||||
pub use distance::{signed_distance, Falloff, Morphology, Shaped};
|
||||
pub use hierarchy::{Edge, Merge, MergeTree, RegionField};
|
||||
pub use prior::{Membership, PriorOptions};
|
||||
#[cfg(feature = "semantic")]
|
||||
pub use scene::{Category, Scene, SceneModel};
|
||||
#[cfg(feature = "semantic")]
|
||||
pub use semantic::{Instance, SemanticModel, SemanticOptions, Tiling};
|
||||
|
||||
/// What can go wrong between an image and a region map.
|
||||
@@ -58,4 +71,10 @@ pub enum SegmentError {
|
||||
/// different model, or a different export of the same one.
|
||||
#[error("model output '{0}' did not have the expected shape")]
|
||||
OutputShape(&'static str),
|
||||
|
||||
/// `models/scene/categories.txt` and the model disagree, or the descriptor
|
||||
/// is malformed. Its own variant rather than a parse error because every
|
||||
/// case carries a specific sentence about what to fix.
|
||||
#[error("category descriptor: {0}")]
|
||||
CategoryDescriptor(String),
|
||||
}
|
||||
|
||||
@@ -0,0 +1,525 @@
|
||||
//! Per-category weights over the whole frame — what the scene tab grades.
|
||||
//!
|
||||
//! [`semantic`](crate::semantic) answers "what objects are in this picture, and
|
||||
//! which pixels are each one". This module answers a different question: "how
|
||||
//! much of each pixel is sky". They are not the same question and they do not
|
||||
//! want the same model.
|
||||
//!
|
||||
//! # Why a second model rather than a second reading of the first
|
||||
//!
|
||||
//! The instance model is COCO-trained, and COCO is eighty classes of *things*.
|
||||
//! There is no class for sky, none for foliage, none for water — the categories
|
||||
//! a landscape is mostly made of. That gap is recorded in `models/LICENCE.md`
|
||||
//! and it is why the scene model exists: ADE20K's 150 classes are a scene
|
||||
//! parse, *stuff* included.
|
||||
//!
|
||||
//! Going the other way is just as impossible. A semantic model merges every
|
||||
//! pixel of a class into one region, so it cannot tell three people apart, and
|
||||
//! telling three people apart is exactly what clicking a subject needs. Neither
|
||||
//! model substitutes for the other, which is why both ship.
|
||||
//!
|
||||
//! # The partition of unity, and why it is the point
|
||||
//!
|
||||
//! [`Scene::weight`] is not a mask per category that each independently says
|
||||
//! yes or no. It is a *partition*: at every pixel the listed categories plus
|
||||
//! the unlisted remainder sum to one, because they come from one softmax over
|
||||
//! all 150 channels, summed within each category.
|
||||
//!
|
||||
//! That property is what makes feathering safe. Feather a hard label map
|
||||
//! outward from sky and outward from vegetation and the boundary band belongs
|
||||
//! to both, so a `+20` on sky and a `−10` on vegetation both land there and
|
||||
//! every horizon acquires a visible seam. Feather a partition of unity and the
|
||||
//! weights still sum to one — the band gets a blend of the two grades, which is
|
||||
//! what a photographer drawing that boundary by hand would have painted.
|
||||
//!
|
||||
//! # Resolution, stated plainly
|
||||
//!
|
||||
//! The graph's logits are `[1, 150, 80, 80]`: an eighth of the input edge, and
|
||||
//! that is the real spatial resolution of everything here. The stock export
|
||||
//! ends with a `Resize` to 640×640 and an `ArgMax`, and
|
||||
//! `tools/export-seg-model.sh` cuts both — the upsample adds no information and
|
||||
//! the argmax destroys the per-class scores this module needs. [`Scene`] keeps
|
||||
//! the native grid and resamples on demand ([`Scene::rasterise`]) so that the
|
||||
//! coarseness is visible in the type rather than hidden behind an early
|
||||
//! upsample.
|
||||
//!
|
||||
//! Practically: a graduated grade over sky or water is unbothered by 80×80. A
|
||||
//! hard edge — a rooftop against sky at 100% zoom — will show it, and no
|
||||
//! feather setting invents detail the model never had.
|
||||
//!
|
||||
//! # Cost
|
||||
//!
|
||||
//! One inference per image, on the same background precompute as the instance
|
||||
//! pass and never on the frame path (ARCH §6.1). The scene tab's sliders read
|
||||
//! [`Scene`] and re-run nothing.
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use ndarray::ArrayView3;
|
||||
|
||||
use crate::semantic::{install_backend, Letterbox, Window};
|
||||
use crate::SegmentError;
|
||||
|
||||
/// Classes in the ADE20K vocabulary the scene model was trained on.
|
||||
///
|
||||
/// Checked against the graph's output rather than trusted: a re-export against
|
||||
/// a different dataset would otherwise be decoded as though its channels meant
|
||||
/// what these ones mean, which produces plausible weights for the wrong thing.
|
||||
pub const CLASSES: usize = 150;
|
||||
|
||||
/// Logit grid stride — the graph's output is this many times coarser than its
|
||||
/// input edge, giving the 80×80 grid at [`crate::semantic::INPUT_EDGE`] 640.
|
||||
const GRID_STRIDE: usize = 8;
|
||||
|
||||
/// One photographic category and the ADE20K classes it marginalises over.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Category {
|
||||
pub name: Arc<str>,
|
||||
/// Indices into the model's vocabulary. Resolved from names at load, so a
|
||||
/// descriptor cannot silently drift out of step with a re-exported model.
|
||||
pub classes: Vec<u16>,
|
||||
}
|
||||
|
||||
/// The scene model, and the categories it has been told to report.
|
||||
pub struct SceneModel {
|
||||
session: ort::session::Session,
|
||||
categories: Vec<Category>,
|
||||
}
|
||||
|
||||
/// The weights that ship in `models/scene/` (AGPL — see `models/LICENCE.md`).
|
||||
///
|
||||
/// Behind its own feature and **off by default**: this graph is 24 MB, where
|
||||
/// the instance model is 11, and Android carries it as an unpacked asset
|
||||
/// rather than inside the binary (`install_bundled_models`). A desktop build
|
||||
/// or a test that wants it compiled in opts in.
|
||||
#[cfg(feature = "embedded-scene-model")]
|
||||
const EMBEDDED_MODEL: &[u8] = include_bytes!("../../../models/scene/yolo26s-sem-ade20k.onnx");
|
||||
#[cfg(feature = "embedded-scene-model")]
|
||||
const EMBEDDED_CLASSES: &str =
|
||||
include_str!("../../../models/scene/yolo26s-sem-ade20k.classes.json");
|
||||
#[cfg(feature = "embedded-scene-model")]
|
||||
const EMBEDDED_CATEGORIES: &str = include_str!("../../../models/scene/categories.txt");
|
||||
|
||||
impl SceneModel {
|
||||
/// Load the scene model compiled into the binary.
|
||||
#[cfg(feature = "embedded-scene-model")]
|
||||
pub fn embedded() -> Result<Self, SegmentError> {
|
||||
let classes = crate::semantic::parse_classes(EMBEDDED_CLASSES);
|
||||
let categories = parse_categories(EMBEDDED_CATEGORIES, &classes)?;
|
||||
Self::from_bytes(EMBEDDED_MODEL, categories)
|
||||
}
|
||||
|
||||
/// Load from files on disk: the graph, its vocabulary, and the category
|
||||
/// descriptor that groups the vocabulary into what the scene tab shows.
|
||||
///
|
||||
/// Three paths rather than one directory because a packager may put the
|
||||
/// weights somewhere the descriptor is not, and because a caller
|
||||
/// experimenting with a different grouping should not have to move a 24 MB
|
||||
/// file to try it.
|
||||
pub fn from_path(
|
||||
model: impl AsRef<std::path::Path>,
|
||||
classes: impl AsRef<std::path::Path>,
|
||||
categories: impl AsRef<std::path::Path>,
|
||||
) -> Result<Self, SegmentError> {
|
||||
let bytes = std::fs::read(model).map_err(SegmentError::ModelRead)?;
|
||||
let classes = std::fs::read_to_string(classes).map_err(SegmentError::ModelRead)?;
|
||||
let categories = std::fs::read_to_string(categories).map_err(SegmentError::ModelRead)?;
|
||||
let classes = crate::semantic::parse_classes(&classes);
|
||||
let categories = parse_categories(&categories, &classes)?;
|
||||
Self::from_bytes(&bytes, categories)
|
||||
}
|
||||
|
||||
pub fn from_bytes(bytes: &[u8], categories: Vec<Category>) -> Result<Self, SegmentError> {
|
||||
install_backend();
|
||||
|
||||
let session = ort::session::Session::builder()
|
||||
.map_err(SegmentError::Inference)?
|
||||
.commit_from_memory(bytes)
|
||||
.map_err(SegmentError::Inference)?;
|
||||
|
||||
Ok(Self {
|
||||
session,
|
||||
categories,
|
||||
})
|
||||
}
|
||||
|
||||
pub fn categories(&self) -> &[Category] {
|
||||
&self.categories
|
||||
}
|
||||
|
||||
/// Weigh every category over one image.
|
||||
///
|
||||
/// `rgb` is tightly packed `f32` RGB in `0.0..=1.0`, row-major — the same
|
||||
/// proxy buffer the instance pass reads, so the two describe one picture.
|
||||
///
|
||||
/// One inference over the whole frame. There is no tiling counterpart to
|
||||
/// [`crate::semantic::Tiling`] here on purpose: tiling buys resolution on a
|
||||
/// small subject, and no category in the descriptor is a small subject.
|
||||
pub fn analyse(
|
||||
&mut self,
|
||||
rgb: &[f32],
|
||||
width: usize,
|
||||
height: usize,
|
||||
) -> Result<Scene, SegmentError> {
|
||||
if rgb.len() != width * height * 3 {
|
||||
return Err(SegmentError::ImageShape {
|
||||
expected: width * height * 3,
|
||||
got: rgb.len(),
|
||||
});
|
||||
}
|
||||
|
||||
// Split the borrow: `run` needs the session mutably while
|
||||
// `marginalise` needs the categories, and going through `self` for
|
||||
// both at once is what the borrow checker objects to.
|
||||
let Self {
|
||||
session,
|
||||
categories,
|
||||
} = self;
|
||||
|
||||
let window = Window {
|
||||
x: 0.0,
|
||||
y: 0.0,
|
||||
w: width as f32,
|
||||
h: height as f32,
|
||||
};
|
||||
let letterbox = Letterbox::fit(window.w, window.h);
|
||||
let input = letterbox.sample(rgb, width, height, &window);
|
||||
|
||||
let outputs = session
|
||||
.run(ort::inputs![
|
||||
ort::value::Tensor::from_array(input).map_err(SegmentError::Inference)?
|
||||
])
|
||||
.map_err(SegmentError::Inference)?;
|
||||
|
||||
let (shape, logits) = outputs[0]
|
||||
.try_extract_tensor::<f32>()
|
||||
.map_err(|_| SegmentError::OutputShape("logits"))?;
|
||||
|
||||
// `[1, 150, gh, gw]`. Checked rather than assumed: the stock export
|
||||
// ends in an ArgMax and returns `[1, 640, 640]` u8 instead, and that
|
||||
// mistake should read as "wrong model" rather than as garbled output.
|
||||
if shape.len() != 4 || shape[0] != 1 || shape[1] as usize != CLASSES {
|
||||
return Err(SegmentError::OutputShape("logits"));
|
||||
}
|
||||
let (gh, gw) = (shape[2] as usize, shape[3] as usize);
|
||||
let logits = ArrayView3::from_shape((CLASSES, gh, gw), &logits[..CLASSES * gh * gw])
|
||||
.map_err(|_| SegmentError::OutputShape("logits"))?;
|
||||
|
||||
Ok(marginalise(categories, logits, gw, gh, letterbox, window))
|
||||
}
|
||||
}
|
||||
|
||||
/// Softmax over the vocabulary, then sum within each category.
|
||||
///
|
||||
/// The summation is what makes the result a partition: softmax gives 150
|
||||
/// numbers summing to one, and grouping them cannot change that total. The
|
||||
/// remainder — every class no category claims — is simply not reported, which
|
||||
/// is why the listed weights sum to *at most* one rather than to one.
|
||||
///
|
||||
/// Free rather than a method so it can be called while the session is borrowed
|
||||
/// mutably, and so the tests can reach it without a graph.
|
||||
fn marginalise(
|
||||
categories: &[Category],
|
||||
logits: ArrayView3<f32>,
|
||||
gw: usize,
|
||||
gh: usize,
|
||||
letterbox: Letterbox,
|
||||
window: Window,
|
||||
) -> Scene {
|
||||
let cells = gw * gh;
|
||||
let mut weight = vec![0.0f32; categories.len() * cells];
|
||||
let mut probability = vec![0.0f32; CLASSES];
|
||||
|
||||
for cell in 0..cells {
|
||||
let (y, x) = (cell / gw, cell % gw);
|
||||
|
||||
// Shift by the maximum before exponentiating. The logits here are
|
||||
// small enough that the naive form would not actually overflow,
|
||||
// but a re-export with a hotter head would, and the cost is one
|
||||
// pass over 150 floats.
|
||||
let mut peak = f32::NEG_INFINITY;
|
||||
for c in 0..CLASSES {
|
||||
peak = peak.max(logits[[c, y, x]]);
|
||||
}
|
||||
let mut total = 0.0f32;
|
||||
for c in 0..CLASSES {
|
||||
let p = (logits[[c, y, x]] - peak).exp();
|
||||
probability[c] = p;
|
||||
total += p;
|
||||
}
|
||||
let norm = if total > 0.0 { 1.0 / total } else { 0.0 };
|
||||
|
||||
for (k, category) in categories.iter().enumerate() {
|
||||
let mut sum = 0.0f32;
|
||||
for &class in &category.classes {
|
||||
sum += probability[class as usize];
|
||||
}
|
||||
weight[k * cells + cell] = sum * norm;
|
||||
}
|
||||
}
|
||||
|
||||
Scene {
|
||||
names: categories.iter().map(|c| c.name.clone()).collect(),
|
||||
weight,
|
||||
grid_width: gw,
|
||||
grid_height: gh,
|
||||
letterbox,
|
||||
window,
|
||||
}
|
||||
}
|
||||
|
||||
/// One image's category weights, at the model's own resolution.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Scene {
|
||||
names: Vec<Arc<str>>,
|
||||
/// `[category][y * grid_width + x]`, each in `0.0..=1.0`, and across
|
||||
/// categories summing to at most one at every cell.
|
||||
weight: Vec<f32>,
|
||||
grid_width: usize,
|
||||
grid_height: usize,
|
||||
letterbox: Letterbox,
|
||||
window: Window,
|
||||
}
|
||||
|
||||
impl Scene {
|
||||
pub fn categories(&self) -> &[Arc<str>] {
|
||||
&self.names
|
||||
}
|
||||
|
||||
pub fn grid_size(&self) -> (usize, usize) {
|
||||
(self.grid_width, self.grid_height)
|
||||
}
|
||||
|
||||
/// One category's weights over the logit grid.
|
||||
pub fn weight(&self, category: usize) -> Option<&[f32]> {
|
||||
let cells = self.grid_width * self.grid_height;
|
||||
self.weight.get(category * cells..(category + 1) * cells)
|
||||
}
|
||||
|
||||
pub fn index_of(&self, name: &str) -> Option<usize> {
|
||||
self.names.iter().position(|n| &**n == name)
|
||||
}
|
||||
|
||||
/// How much of the frame this category covers, `0.0..=1.0`.
|
||||
///
|
||||
/// Cheap, and the scene tab needs it: a category weighing essentially
|
||||
/// nothing should not be offered a slider, because a control that does
|
||||
/// nothing when moved is worse than an absent one.
|
||||
pub fn coverage(&self, category: usize) -> f32 {
|
||||
match self.weight(category) {
|
||||
Some(w) if !w.is_empty() => w.iter().sum::<f32>() / w.len() as f32,
|
||||
_ => 0.0,
|
||||
}
|
||||
}
|
||||
|
||||
/// Resample one category to source-image resolution.
|
||||
///
|
||||
/// Bilinear over the logit grid. This does not add detail and is not meant
|
||||
/// to — see the module header on resolution — it exists because a mask has
|
||||
/// to be the size of the picture before it can weight an adjustment, and
|
||||
/// doing the resample here keeps the one correct letterbox inverse in one
|
||||
/// place.
|
||||
pub fn rasterise(&self, category: usize, width: usize, height: usize) -> Option<Vec<f32>> {
|
||||
let grid = self.weight(category)?;
|
||||
let mut out = vec![0.0f32; width * height];
|
||||
|
||||
for y in 0..height {
|
||||
for x in 0..width {
|
||||
let (gx, gy) = self.letterbox.to_grid(
|
||||
x as f32 + 0.5,
|
||||
y as f32 + 0.5,
|
||||
&self.window,
|
||||
GRID_STRIDE as f32,
|
||||
);
|
||||
// Half-cell shift: `to_grid` lands on the grid's coordinate
|
||||
// space, where a cell's *centre* is at its index plus a half.
|
||||
let (gx, gy) = (gx - 0.5, gy - 0.5);
|
||||
let x0 = gx.floor();
|
||||
let y0 = gy.floor();
|
||||
let (fx, fy) = (gx - x0, gy - y0);
|
||||
let x0 = (x0 as isize).clamp(0, self.grid_width as isize - 1) as usize;
|
||||
let y0 = (y0 as isize).clamp(0, self.grid_height as isize - 1) as usize;
|
||||
let x1 = (x0 + 1).min(self.grid_width - 1);
|
||||
let y1 = (y0 + 1).min(self.grid_height - 1);
|
||||
|
||||
let at = |gx: usize, gy: usize| grid[gy * self.grid_width + gx];
|
||||
let top = at(x0, y0) * (1.0 - fx) + at(x1, y0) * fx;
|
||||
let bot = at(x0, y1) * (1.0 - fx) + at(x1, y1) * fx;
|
||||
out[y * width + x] = top * (1.0 - fy) + bot * fy;
|
||||
}
|
||||
}
|
||||
|
||||
Some(out)
|
||||
}
|
||||
}
|
||||
|
||||
/// Read `models/scene/categories.txt`, resolving class names to indices.
|
||||
///
|
||||
/// Hand-written rather than generated, unlike the `.classes.json` beside it,
|
||||
/// which is why the format is line-oriented with comments: the *reasoning* for
|
||||
/// a grouping belongs next to the grouping, and JSON has nowhere to put it.
|
||||
pub fn parse_categories(text: &str, classes: &[Arc<str>]) -> Result<Vec<Category>, SegmentError> {
|
||||
let mut out: Vec<Category> = Vec::new();
|
||||
let mut claimed: Vec<Option<Arc<str>>> = vec![None; classes.len()];
|
||||
|
||||
for line in text.lines() {
|
||||
let line = line.split('#').next().unwrap_or("").trim();
|
||||
if line.is_empty() {
|
||||
continue;
|
||||
}
|
||||
let Some((name, members)) = line.split_once('=') else {
|
||||
return Err(SegmentError::CategoryDescriptor(format!(
|
||||
"line is not `name = class, class, ...`: {line}"
|
||||
)));
|
||||
};
|
||||
let name: Arc<str> = name.trim().into();
|
||||
|
||||
let mut indices = Vec::new();
|
||||
for member in members.split(',') {
|
||||
let member = member.trim();
|
||||
if member.is_empty() {
|
||||
continue;
|
||||
}
|
||||
let Some(index) = classes.iter().position(|c| &**c == member) else {
|
||||
return Err(SegmentError::CategoryDescriptor(format!(
|
||||
"category '{name}' names class '{member}', which this model does not have"
|
||||
)));
|
||||
};
|
||||
// Two categories sharing a class would each count its probability,
|
||||
// so the weights would exceed one where it appears and the
|
||||
// partition — the whole reason for summing after a softmax — would
|
||||
// be quietly untrue.
|
||||
if let Some(owner) = &claimed[index] {
|
||||
return Err(SegmentError::CategoryDescriptor(format!(
|
||||
"class '{member}' is claimed by both '{owner}' and '{name}'"
|
||||
)));
|
||||
}
|
||||
claimed[index] = Some(name.clone());
|
||||
indices.push(index as u16);
|
||||
}
|
||||
|
||||
if indices.is_empty() {
|
||||
return Err(SegmentError::CategoryDescriptor(format!(
|
||||
"category '{name}' lists no classes"
|
||||
)));
|
||||
}
|
||||
out.push(Category {
|
||||
name,
|
||||
classes: indices,
|
||||
});
|
||||
}
|
||||
|
||||
if out.is_empty() {
|
||||
return Err(SegmentError::CategoryDescriptor(
|
||||
"descriptor defines no categories".into(),
|
||||
));
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn vocabulary() -> Vec<Arc<str>> {
|
||||
["sky", "tree", "grass", "person", "wall"]
|
||||
.iter()
|
||||
.map(|s| Arc::from(*s))
|
||||
.collect()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn descriptor_resolves_names_to_indices() {
|
||||
let v = vocabulary();
|
||||
let cats = parse_categories("sky = sky\nvegetation = tree, grass\n", &v).unwrap();
|
||||
assert_eq!(cats.len(), 2);
|
||||
assert_eq!(&*cats[0].name, "sky");
|
||||
assert_eq!(cats[0].classes, vec![0]);
|
||||
assert_eq!(cats[1].classes, vec![1, 2]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn comments_and_blank_lines_are_ignored() {
|
||||
let v = vocabulary();
|
||||
let cats = parse_categories("# a note\n\nsky = sky # trailing\n", &v).unwrap();
|
||||
assert_eq!(cats.len(), 1);
|
||||
assert_eq!(cats[0].classes, vec![0]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_unknown_class_is_refused() {
|
||||
let v = vocabulary();
|
||||
let e = parse_categories("sky = cloud\n", &v).unwrap_err();
|
||||
assert!(format!("{e}").contains("cloud"), "{e}");
|
||||
}
|
||||
|
||||
/// The partition is the module's one load-bearing property, so the
|
||||
/// descriptor is not allowed to break it before inference even runs.
|
||||
#[test]
|
||||
fn a_class_in_two_categories_is_refused() {
|
||||
let v = vocabulary();
|
||||
let e = parse_categories("a = tree\nb = grass, tree\n", &v).unwrap_err();
|
||||
assert!(format!("{e}").contains("claimed by both"), "{e}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_shipped_descriptor_matches_the_shipped_vocabulary() {
|
||||
let classes = crate::semantic::parse_classes(include_str!(
|
||||
"../../../models/scene/yolo26s-sem-ade20k.classes.json"
|
||||
));
|
||||
assert_eq!(classes.len(), CLASSES);
|
||||
let cats = parse_categories(
|
||||
include_str!("../../../models/scene/categories.txt"),
|
||||
&classes,
|
||||
)
|
||||
.expect("shipped descriptor must load against the shipped vocabulary");
|
||||
assert!(cats.iter().any(|c| &*c.name == "sky"));
|
||||
assert!(cats.iter().any(|c| &*c.name == "vegetation"));
|
||||
}
|
||||
|
||||
/// Softmax then group: the reported weights must never exceed one, and
|
||||
/// must equal one exactly when the categories name every class.
|
||||
#[test]
|
||||
fn marginalising_preserves_the_partition() {
|
||||
let classes: Vec<Arc<str>> = vocabulary();
|
||||
let cats = parse_categories(
|
||||
"sky = sky\nvegetation = tree, grass\nrest = person, wall\n",
|
||||
&classes,
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
// Hand-rolled rather than run through a graph: this test is about the
|
||||
// arithmetic, and a model would only make it slower and less certain.
|
||||
let (gw, gh) = (2usize, 2usize);
|
||||
let mut logits = vec![0.0f32; classes.len() * gw * gh];
|
||||
for (i, v) in logits.iter_mut().enumerate() {
|
||||
*v = (i % 7) as f32 * 0.3;
|
||||
}
|
||||
let view = ArrayView3::from_shape((classes.len(), gh, gw), &logits).unwrap();
|
||||
|
||||
// `marginalise` is a method for access to `self.categories`; build the
|
||||
// smallest thing that owns them rather than a session.
|
||||
let cells = gw * gh;
|
||||
let mut weight = vec![0.0f32; cats.len() * cells];
|
||||
for cell in 0..cells {
|
||||
let (y, x) = (cell / gw, cell % gw);
|
||||
let peak = (0..classes.len()).fold(f32::NEG_INFINITY, |m, c| m.max(view[[c, y, x]]));
|
||||
let p: Vec<f32> = (0..classes.len())
|
||||
.map(|c| (view[[c, y, x]] - peak).exp())
|
||||
.collect();
|
||||
let total: f32 = p.iter().sum();
|
||||
for (k, category) in cats.iter().enumerate() {
|
||||
let s: f32 = category.classes.iter().map(|&c| p[c as usize]).sum();
|
||||
weight[k * cells + cell] = s / total;
|
||||
}
|
||||
}
|
||||
|
||||
for cell in 0..cells {
|
||||
let sum: f32 = (0..cats.len()).map(|k| weight[k * cells + cell]).sum();
|
||||
assert!(
|
||||
(sum - 1.0).abs() < 1e-5,
|
||||
"categories covering every class must sum to 1, got {sum}"
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -20,7 +20,7 @@
|
||||
//! behaviour, and it is worth being glad of rather than working around.
|
||||
//!
|
||||
//! So arm B here contributes *subjects*, and the watershed contributes
|
||||
//! everything else. See `models/LICENCE.md` for why no ADE20K variant is
|
||||
//! everything else. See `models/LICENCE.md` at the repository root for why no ADE20K variant is
|
||||
//! shipped instead.
|
||||
//!
|
||||
//! # Cost, and where it may run
|
||||
@@ -198,15 +198,15 @@ pub struct SemanticModel {
|
||||
classes: Vec<Arc<str>>,
|
||||
}
|
||||
|
||||
/// The weights that ship with this crate (`models/`, AGPL — see LICENCE.md).
|
||||
/// The weights, from the repository-root `models/segment/` (AGPL — see `models/LICENCE.md`).
|
||||
///
|
||||
/// Embedded rather than read from a path because Android hands the app no
|
||||
/// filesystem location to read from (ARCH §6.9) — the same reasoning that has
|
||||
/// the Lensfun database shipping inside its crate.
|
||||
#[cfg(feature = "embedded-model")]
|
||||
const EMBEDDED_MODEL: &[u8] = include_bytes!("../models/yolo26n-seg.onnx");
|
||||
const EMBEDDED_MODEL: &[u8] = include_bytes!("../../../models/segment/yolo26n-seg.onnx");
|
||||
#[cfg(feature = "embedded-model")]
|
||||
const EMBEDDED_CLASSES: &str = include_str!("../models/yolo26n-seg.classes.json");
|
||||
const EMBEDDED_CLASSES: &str = include_str!("../../../models/segment/yolo26n-seg.classes.json");
|
||||
|
||||
impl SemanticModel {
|
||||
/// Load the model that ships with this crate.
|
||||
@@ -446,16 +446,16 @@ fn decode(
|
||||
|
||||
/// A source-space rectangle fed through one inference.
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
struct Window {
|
||||
x: f32,
|
||||
y: f32,
|
||||
w: f32,
|
||||
h: f32,
|
||||
pub(crate) struct Window {
|
||||
pub(crate) x: f32,
|
||||
pub(crate) y: f32,
|
||||
pub(crate) w: f32,
|
||||
pub(crate) h: f32,
|
||||
}
|
||||
|
||||
/// The scale-and-pad that fits an arbitrary rectangle into the square input.
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
struct Letterbox {
|
||||
pub(crate) struct Letterbox {
|
||||
/// Input pixels per source pixel.
|
||||
scale: f32,
|
||||
pad_x: f32,
|
||||
@@ -463,7 +463,7 @@ struct Letterbox {
|
||||
}
|
||||
|
||||
impl Letterbox {
|
||||
fn fit(w: f32, h: f32) -> Self {
|
||||
pub(crate) fn fit(w: f32, h: f32) -> Self {
|
||||
let scale = (INPUT_EDGE as f32 / w).min(INPUT_EDGE as f32 / h);
|
||||
Self {
|
||||
scale,
|
||||
@@ -477,7 +477,13 @@ impl Letterbox {
|
||||
/// Bilinear, and grey (`0.5`) in the padding — the value the network sees
|
||||
/// least as an edge, where black would draw a hard border across the frame
|
||||
/// and invite a detection along it.
|
||||
fn sample(&self, rgb: &[f32], width: usize, height: usize, window: &Window) -> Array4<f32> {
|
||||
pub(crate) fn sample(
|
||||
&self,
|
||||
rgb: &[f32],
|
||||
width: usize,
|
||||
height: usize,
|
||||
window: &Window,
|
||||
) -> Array4<f32> {
|
||||
let mut input = Array4::<f32>::from_elem((1, 3, INPUT_EDGE, INPUT_EDGE), 0.5);
|
||||
|
||||
for iy in 0..INPUT_EDGE {
|
||||
@@ -519,12 +525,23 @@ impl Letterbox {
|
||||
)
|
||||
}
|
||||
|
||||
/// Source pixel to the coordinates of an output grid `stride` times
|
||||
/// coarser than the graph's input.
|
||||
///
|
||||
/// Every dense output this crate reads is some even fraction of the input
|
||||
/// edge — YOLO's mask prototypes at a quarter, the scene model's logits at
|
||||
/// an eighth — and they all sit inside the same letterboxed square, so the
|
||||
/// mapping differs only in that divisor.
|
||||
pub(crate) fn to_grid(self, sx: f32, sy: f32, w: &Window, stride: f32) -> (f32, f32) {
|
||||
(
|
||||
((sx - w.x) * self.scale + self.pad_x) / stride,
|
||||
((sy - w.y) * self.scale + self.pad_y) / stride,
|
||||
)
|
||||
}
|
||||
|
||||
/// Source pixel to prototype-grid coordinates.
|
||||
fn to_proto(self, sx: f32, sy: f32, w: &Window) -> (f32, f32) {
|
||||
(
|
||||
((sx - w.x) * self.scale + self.pad_x) / PROTO_STRIDE as f32,
|
||||
((sy - w.y) * self.scale + self.pad_y) / PROTO_STRIDE as f32,
|
||||
)
|
||||
self.to_grid(sx, sy, w, PROTO_STRIDE as f32)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -648,7 +665,7 @@ fn steps(extent: f32, edge: f32, stride: f32) -> usize {
|
||||
}
|
||||
|
||||
/// Point `ort` at tract, exactly once per process.
|
||||
fn install_backend() {
|
||||
pub(crate) fn install_backend() {
|
||||
use std::sync::Once;
|
||||
static ONCE: Once = Once::new();
|
||||
ONCE.call_once(|| {
|
||||
|
||||
@@ -255,25 +255,37 @@ fi
|
||||
cp "${SO}" "${OUT}/staging/lib/${ABI}/libdarkroom.so"
|
||||
cp "${DEX}" "${OUT}/staging/classes.dex"
|
||||
|
||||
# The face models. Android has no other route to one — app-private storage is
|
||||
# not user-reachable and the in-app fetch is unbuilt (docs/faces.md §2.2a) — so
|
||||
# The models. Android has no other route to one — app-private storage is not
|
||||
# user-reachable and the in-app fetch is unbuilt (docs/faces.md §2.2a) — so
|
||||
# they go in the APK and `android_main` unpacks them on first launch. The
|
||||
# source is `models/face/`, shared with the Arch package rather than living
|
||||
# under this one platform's directory.
|
||||
# sources are `models/face/` and `models/scene/`, shared with the Arch package
|
||||
# rather than living under this one platform's directory.
|
||||
#
|
||||
# Two directories, and they are not the same kind of thing. The face weights
|
||||
# are absent from most checkouts by design (research-only grant), so finding
|
||||
# none is ordinary. The scene model is committed, so finding none means a
|
||||
# broken checkout — but this script still only warns, because the failure it
|
||||
# would otherwise cause is at APK build time on a machine that may legitimately
|
||||
# be building the face-less variant.
|
||||
#
|
||||
# Through the staging directory rather than aapt2's `-A`: the .so and the dex
|
||||
# already go in with `zip` below, and one mechanism for "extra files in the
|
||||
# APK" is easier to follow than two.
|
||||
ASSETS="${REPO}/models/face"
|
||||
#
|
||||
# Cleared first: a previous run that died between staging and cleanup would
|
||||
# otherwise leave models in the APK that are no longer in the tree.
|
||||
rm -rf "${OUT}/staging/assets"
|
||||
if compgen -G "${ASSETS}/*.onnx" >/dev/null; then
|
||||
mkdir -p "${OUT}/staging/assets/models"
|
||||
_bundled=""
|
||||
for _dir in face scene; do
|
||||
ASSETS="${REPO}/models/${_dir}"
|
||||
compgen -G "${ASSETS}/*.onnx" >/dev/null || continue
|
||||
# An LFS pointer is ~130 bytes and looks exactly like a model to `cp`. Left
|
||||
# unchecked it reaches the device and fails inside tract, which reports a
|
||||
# broken graph rather than a clone that needs `git lfs pull`. Same guard
|
||||
# dr-segment's build script applies to yolo26n-seg.onnx, and the same
|
||||
# reason.
|
||||
# reason. Only the weights are checked: the vocabulary and the category
|
||||
# descriptor beside them are legitimately a few kilobytes.
|
||||
for m in "${ASSETS}"/*.onnx; do
|
||||
if [[ "$(stat -c%s "${m}")" -lt 100000 ]]; then
|
||||
echo "error: $(basename "${m}") is $(stat -c%s "${m}") bytes — an LFS pointer, not a model." >&2
|
||||
@@ -281,11 +293,21 @@ if compgen -G "${ASSETS}/*.onnx" >/dev/null; then
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
mkdir -p "${OUT}/staging/assets/models"
|
||||
cp "${ASSETS}"/*.onnx "${OUT}/staging/assets/models/"
|
||||
echo " assets: $(ls "${ASSETS}" | grep '\.onnx$' | tr '\n' ' ')"
|
||||
# The scene model is three files: the graph, its vocabulary, and the
|
||||
# category descriptor. All three are needed to decode anything, so they
|
||||
# travel together; README.md is documentation and stays out of the APK.
|
||||
for f in "${ASSETS}"/*; do
|
||||
case "$(basename "${f}")" in
|
||||
README.md) continue ;;
|
||||
esac
|
||||
cp "${f}" "${OUT}/staging/assets/models/"
|
||||
_bundled="${_bundled} $(basename "${f}")"
|
||||
done
|
||||
done
|
||||
if [[ -n "${_bundled}" ]]; then
|
||||
echo " assets:${_bundled}"
|
||||
else
|
||||
echo " assets: no face models found (face indexing will be off on the device)"
|
||||
echo " assets: no models found (face indexing and the scene tab will be off on the device)"
|
||||
fi
|
||||
|
||||
# -0 "" stores the .so without compression so Android can mmap it directly
|
||||
|
||||
+18
-9
@@ -5,7 +5,7 @@
|
||||
|
||||
Every entry here is extracted from the comment beside the code that implements it, so this file cannot describe a gesture the application does not have. Add one by writing a `GESTURE:` block next to the implementation; there is nowhere else to write it.
|
||||
|
||||
15 gestures, in 2 places.
|
||||
16 gestures, in 2 places.
|
||||
|
||||
## People
|
||||
|
||||
@@ -73,6 +73,15 @@ While selecting, a tap never opens. That is the whole point of the mode: one mea
|
||||
|
||||
<sub>`ui/dr-ui/ui/library.slint:1291`</sub>
|
||||
|
||||
### Pick a photograph up to drag it
|
||||
|
||||
- **Touch** — Press and hold it until a ring opens around it, then drag
|
||||
- **Pointer** — Drag it
|
||||
|
||||
A finger on a photograph might be starting a scroll, and for the first half-second the grid assumes it is. Holding says otherwise, and the ring is the grid saying it heard — from there the drag cannot be lost to a scroll. A mouse never waits: the cursor is precise enough that a sideways drag is unambiguous from the first pixel.
|
||||
|
||||
<sub>`ui/dr-ui/ui/library.slint:1321`</sub>
|
||||
|
||||
### Select a range
|
||||
|
||||
- **Touch** — While selecting, press "Select to…", then tap the last photograph of the run
|
||||
@@ -80,7 +89,7 @@ While selecting, a tap never opens. That is the whole point of the mode: one mea
|
||||
|
||||
This replaced a double tap, which had no visible state and could take forty photographs by accident. The run is resolved by the catalog rather than by what is on screen, so the grid can scroll between the two taps — the ranges that hurt on a tablet are longer than a screenful, which is exactly where a finger sweep runs out.
|
||||
|
||||
<sub>`ui/dr-ui/ui/library.slint:1352`</sub>
|
||||
<sub>`ui/dr-ui/ui/library.slint:1386`</sub>
|
||||
|
||||
### Find photographs with two people in them
|
||||
|
||||
@@ -89,7 +98,7 @@ This replaced a double tap, which had no visible state and could take forty phot
|
||||
|
||||
"Any of them" is a union and "all of them" is an intersection. The tray is where both terms and the choice between them live, because a filter belongs on the filter bar.
|
||||
|
||||
<sub>`ui/dr-ui/ui/library.slint:2093`</sub>
|
||||
<sub>`ui/dr-ui/ui/library.slint:2141`</sub>
|
||||
|
||||
### Resize the thumbnails
|
||||
|
||||
@@ -98,7 +107,7 @@ This replaced a double tap, which had no visible state and could take forty phot
|
||||
|
||||
There is no wheel on a tablet, so without the pinch the cell size could only be changed by a control a finger cannot reach.
|
||||
|
||||
<sub>`ui/dr-ui/ui/library.slint:2719`</sub>
|
||||
<sub>`ui/dr-ui/ui/library.slint:2783`</sub>
|
||||
|
||||
### File photographs in a collection
|
||||
|
||||
@@ -107,7 +116,7 @@ There is no wheel on a tablet, so without the pinch the cell size could only be
|
||||
|
||||
The selection is what the drag carries, which is why selecting several is worth the mode: forty photographs file in one gesture.
|
||||
|
||||
<sub>`ui/dr-ui/ui/library.slint:2878`</sub>
|
||||
<sub>`ui/dr-ui/ui/library.slint:2942`</sub>
|
||||
|
||||
### Open a photograph
|
||||
|
||||
@@ -116,7 +125,7 @@ The selection is what the drag carries, which is why selecting several is worth
|
||||
|
||||
A tap opens; a tap that *moved* does not. Travel is what separates a deliberate tap from a hand brushing past, and it is the only thing that does: the two are the same length. An earlier version required the finger to dwell 120 ms instead, and that rejected ordinary taps — a real tap is often quicker than a brush.
|
||||
|
||||
<sub>`ui/dr-ui/ui/library.slint:3133`</sub>
|
||||
<sub>`ui/dr-ui/ui/library.slint:3210`</sub>
|
||||
|
||||
### Rate a photograph without opening it
|
||||
|
||||
@@ -126,7 +135,7 @@ A tap opens; a tap that *moved* does not. Travel is what separates a deliberate
|
||||
|
||||
A star has to take the press without it also reaching the cell, or every rating throws the user into develop.
|
||||
|
||||
<sub>`ui/dr-ui/ui/library.slint:3245`</sub>
|
||||
<sub>`ui/dr-ui/ui/library.slint:3330`</sub>
|
||||
|
||||
### Drop the selection but keep selecting
|
||||
|
||||
@@ -135,7 +144,7 @@ A star has to take the press without it also reaching the cell, or every rating
|
||||
|
||||
Distinct from Done, which leaves the mode entirely. Clearing keeps it, so the next selection can start straight away.
|
||||
|
||||
<sub>`ui/dr-ui/ui/library.slint:3808`</sub>
|
||||
<sub>`ui/dr-ui/ui/library.slint:3938`</sub>
|
||||
|
||||
### Select everything the grid is showing
|
||||
|
||||
@@ -144,4 +153,4 @@ Distinct from Done, which leaves the mode entirely. Clearing keeps it, so the ne
|
||||
|
||||
A scoped grid of two hundred frames is two hundred taps otherwise, and "all of them, except those three" is a far more common shape than the taps it took to say it.
|
||||
|
||||
<sub>`ui/dr-ui/ui/library.slint:3825`</sub>
|
||||
<sub>`ui/dr-ui/ui/library.slint:3955`</sub>
|
||||
|
||||
@@ -435,3 +435,69 @@ dependency detail (`core/dr-segment/models/LICENCE.md`).
|
||||
generalises: `ort` + `ort-tract` gives ONNX inference in pure Rust, so the face pipeline of §3.9.1
|
||||
needs no C dependency either. The *model licensing* half of D13 is untouched — the InsightFace
|
||||
weights are still non-commercial and still unusable here.
|
||||
|
||||
---
|
||||
|
||||
## 16. The scene model — per-category grades
|
||||
|
||||
Added 2026-08-30, after §4's premise stopped being true.
|
||||
|
||||
### What changed
|
||||
|
||||
§4 specified a semantic model pretrained on ADE20K, whose 150 classes include the *stuff* categories
|
||||
photography cares about. §13 recorded that no such model existed in usable form and that arm B would
|
||||
therefore contribute subjects only, which made "select the sky" arm A's problem. Re-checked
|
||||
2026-08-30: **Ultralytics now ships a `semantic` task with ADE20K checkpoints**
|
||||
(`docs.ultralytics.com/tasks/semantic`). `yolo26s-sem-ade20k` is in `models/scene/`.
|
||||
|
||||
### It is an addition, not a correction to arm B
|
||||
|
||||
The instance model stays exactly where it was, and the reason is the one §13 already gave and was
|
||||
right about: a semantic model merges every pixel of a class into one region, so it cannot separate
|
||||
two people, and separating two people is what clicking a subject requires. Swapping arm B for this
|
||||
would regress the primary interaction to fix a secondary one.
|
||||
|
||||
So the two divide by *what the user is doing*, not by which is better:
|
||||
|
||||
| | `models/segment/` (COCO instances) | `models/scene/` (ADE20K semantics) |
|
||||
|---|---|---|
|
||||
| Question | which pixels are *that* dog | how much of this pixel is sky |
|
||||
| Granularity | per instance | per category, whole frame |
|
||||
| Drives | local adjustments, subject selection | the scene tab's per-category sliders |
|
||||
| Vocabulary | 80 things | 150 classes, stuff included |
|
||||
|
||||
### The export is truncated, and both reasons matter
|
||||
|
||||
Ultralytics ends the graph with `Resize → ArgMax → Cast`, returning a `[1, 640, 640]` u8 label map.
|
||||
`tools/export-seg-model.sh` cuts that tail and ships the classifier's `[1, 150, 80, 80]` f32 logits.
|
||||
|
||||
**Cost.** The `Resize` materialises 150 × 640 × 640 × f32 — 246 MB — and the `ArgMax` then reduces
|
||||
across the channel axis, striding 409,600 elements per comparison. Measured under load it was
|
||||
roughly four fifths of total runtime, spent on work the application discards.
|
||||
|
||||
**Softness, which is the more important one.** `ArgMax` destroys the per-class scores, and the whole
|
||||
design of the scene tab rests on keeping them. Softmax over the 150 channels, summed within each
|
||||
category, produces per-category weights that sum to one at every pixel — a partition of unity.
|
||||
Feathering that cannot double-grade a boundary. Feathering *hard labels* outward from two adjacent
|
||||
categories paints both grades into the overlap, and every horizon in the frame acquires a seam.
|
||||
|
||||
### The resolution is 80×80, and no setting changes that
|
||||
|
||||
The discarded upsample was never information. `Scene` keeps the native grid and resamples on demand,
|
||||
so the coarseness is visible in the type rather than hidden. A graduated grade over sky or water is
|
||||
untroubled by it; a rooftop against sky at 100% zoom will show it. This is the constraint most likely
|
||||
to decide whether the tab feels good, and it is not addressable by choosing a larger checkpoint —
|
||||
`yolo26n-sem` and `yolo26s-sem` have the same output grid.
|
||||
|
||||
### Licence
|
||||
|
||||
Unchanged. Same AGPL-3.0 grant as the instance model, same GPLv3 §13 permission, same consequence
|
||||
already accepted in D14 — so this needed no new licence decision, which is most of why it was cheap.
|
||||
See `models/LICENCE.md`.
|
||||
|
||||
### Measurement
|
||||
|
||||
Timings taken while this was chosen came off a laptop compiling other things and are upper bounds
|
||||
only. `cargo run -p dr-segment --example scene --release --features embedded-scene-model` reports a
|
||||
median over N runs with the first excluded; a number worth quoting should come from that, on an idle
|
||||
machine.
|
||||
|
||||
@@ -0,0 +1,71 @@
|
||||
# Model weights — licensing
|
||||
|
||||
Two Ultralytics checkpoints ship here, both exported by
|
||||
`tools/export-seg-model.sh`, each with its class vocabulary written out by the
|
||||
same script:
|
||||
|
||||
| File | Checkpoint | Trained on | Used by |
|
||||
|---|---|---|---|
|
||||
| `segment/yolo26n-seg.onnx` | `yolo26n-seg.pt` | COCO, 80 *thing* classes | local adjustments, subject selection |
|
||||
| `scene/yolo26s-sem-ade20k.onnx` | `yolo26s-sem-ade20k.pt` | ADE20K, 150 classes | the scene tab's per-category grades |
|
||||
|
||||
Both come from `https://huggingface.co/Ultralytics/YOLO26`. The face weights in
|
||||
`face/` are a separate matter with a separate grant — see `face/README.md`.
|
||||
|
||||
## The grant
|
||||
|
||||
**Ultralytics releases YOLO under AGPL-3.0**, and the weights carry the same
|
||||
grant as the framework — the HuggingFace repository declares `agpl-3.0` for the
|
||||
checkpoints themselves, not merely for the training code. A commercial licence
|
||||
is offered separately; DarkRoom does not use it and does not need it.
|
||||
|
||||
## What that means for DarkRoom
|
||||
|
||||
DarkRoom is GPL-3.0-or-later. **GPLv3 §13 explicitly permits combination with
|
||||
AGPL-3.0 code**, so redistributing these weights inside this repository is
|
||||
allowed — this is *not* the situation the InsightFace "buffalo" weights would
|
||||
have created, where a non-commercial research grant is simply incompatible with
|
||||
the project's licence and with F-Droid, Flatpak and Play distribution
|
||||
(NFR-COMPAT-2, D13).
|
||||
|
||||
The consequence, and it is a real one: **the combined work is effectively
|
||||
AGPL-3.0.** §13's permission runs one way — the AGPL's §13 network-use condition
|
||||
attaches to the portion under that licence. For a local-first desktop and
|
||||
Android photo editor that condition has no practical bite, because there is no
|
||||
network service offering the combined work to remote users. It would acquire
|
||||
bite the moment any hosted or server-side rendering appeared, and that is the
|
||||
thing to remember rather than rediscover.
|
||||
|
||||
This was decided deliberately (D14), not arrived at by accident, and
|
||||
`docs/segmentation.md` §7 records the reasoning.
|
||||
|
||||
## Class vocabulary — a caveat worth reading
|
||||
|
||||
`docs/segmentation.md` §4 specified YOLO **pretrained on ADE20K**, whose 150
|
||||
classes include the *stuff* categories that matter most in photography — sky,
|
||||
vegetation, water, wall, mountain.
|
||||
|
||||
**This was true when written and is not any more.** Checked 2026-08-21, no
|
||||
YOLO/ADE20K combination existed: Ultralytics shipped YOLO26-seg on **COCO**
|
||||
only, and the one HuggingFace repository claiming otherwise
|
||||
(`laxmacl/yolov8-ade20k`) was empty. Re-checked 2026-08-30: Ultralytics now
|
||||
ships a `semantic` task with ADE20K checkpoints
|
||||
(`https://docs.ultralytics.com/tasks/semantic`), and `yolo26s-sem-ade20k` is
|
||||
what `scene/` holds.
|
||||
|
||||
So the two vocabularies divide the work rather than compete:
|
||||
|
||||
- **`segment/`, COCO, 80 things.** Separates *instances* — clicking one of
|
||||
three people selects that person. This is what local adjustments need, and a
|
||||
semantic model cannot do it: it would return one "person" region covering all
|
||||
three.
|
||||
- **`scene/`, ADE20K, 150 classes.** Labels every pixel, including the *stuff*
|
||||
COCO has no word for — sky, vegetation, water, mountain, wall. This is what
|
||||
the scene tab's per-category grades need, and it does not care that instances
|
||||
are merged, because a per-category grade applies to the whole category.
|
||||
|
||||
Neither replaces the other. Keeping both is the deliberate choice.
|
||||
|
||||
The loader treats each vocabulary as model metadata rather than compiled-in
|
||||
knowledge, which is what made adding the second model a file plus a descriptor
|
||||
rather than a code change — as this document predicted it would be.
|
||||
@@ -0,0 +1,55 @@
|
||||
# Photographic categories, over ADE20K's 150 classes.
|
||||
#
|
||||
# The scene tab offers a slider per category, not per class: nobody wants to
|
||||
# grade "sconce" and "crt screen" separately, and ADE20K's vocabulary is a
|
||||
# scene-parsing benchmark rather than a photographer's list. This file is the
|
||||
# translation, and it is data so that changing it is not a code change.
|
||||
#
|
||||
# Format: one category per line, `name = class, class, ...`, where each class
|
||||
# is a name from the model's own `.classes.json`. Names rather than indices
|
||||
# because an index is silently wrong after a re-export and a name is loudly
|
||||
# wrong; `SceneModel::from_path` refuses a file naming a class the model does
|
||||
# not have.
|
||||
#
|
||||
# ## Why the list is short, and why "other" is not in it
|
||||
#
|
||||
# Weights come from a softmax over all 150 channels summed within each
|
||||
# category, so the categories listed here plus everything unlisted sum to 1 at
|
||||
# every pixel. That is what lets the scene tab feather two adjacent categories
|
||||
# without painting both grades into the overlap. Adding a category takes
|
||||
# weight from the unlisted remainder rather than from its neighbours, so this
|
||||
# list can grow without disturbing what is already here.
|
||||
#
|
||||
# Classes are assigned to at most one category — an overlap would break the
|
||||
# partition and double-count the shared class, so the loader rejects it.
|
||||
|
||||
# The one the whole exercise started from. ADE20K's easiest class, and the one
|
||||
# most often graded on its own in a landscape.
|
||||
sky = sky
|
||||
|
||||
# Foliage, not "green things": a lawn and a canopy take the same saturation
|
||||
# and luminance moves far more often than either takes the sky's.
|
||||
vegetation = tree, grass, plant, flower, palm, field
|
||||
|
||||
# Standing and moving water together. `swimming pool` and `fountain` are here
|
||||
# rather than under architecture because what a photographer adjusts is the
|
||||
# water, not the basin.
|
||||
water = water, sea, river, lake, waterfall, swimming pool, fountain
|
||||
|
||||
# Distant landform. Separate from `ground` because it is usually far, hazy and
|
||||
# wants dehaze and contrast where a foreground surface wants neither.
|
||||
terrain = mountain, rock, hill
|
||||
|
||||
# What the photographer is standing on, or would be. Earth and sand sit here
|
||||
# rather than with terrain for the same near/far reason.
|
||||
ground = earth, sand, land, dirt track, path, road, sidewalk, runway, floor, step, stairs, stairway
|
||||
|
||||
# Built structure. Deliberately broad: a facade, its railings and its awning
|
||||
# are one surface as far as a global grade is concerned.
|
||||
architecture = building, house, skyscraper, wall, tower, bridge, hovel, fence, column, awning, booth, canopy, pier, railing, grandstand
|
||||
|
||||
# Present for the scene tab's "expose people" move, and *not* a replacement for
|
||||
# the instance model — this is every person in the frame at once, which is the
|
||||
# right granularity for a global grade and the wrong one for selecting a
|
||||
# subject. See `models/LICENCE.md`.
|
||||
person = person
|
||||
@@ -0,0 +1,152 @@
|
||||
[
|
||||
"wall",
|
||||
"building",
|
||||
"sky",
|
||||
"floor",
|
||||
"tree",
|
||||
"ceiling",
|
||||
"road",
|
||||
"bed",
|
||||
"windowpane",
|
||||
"grass",
|
||||
"cabinet",
|
||||
"sidewalk",
|
||||
"person",
|
||||
"earth",
|
||||
"door",
|
||||
"table",
|
||||
"mountain",
|
||||
"plant",
|
||||
"curtain",
|
||||
"chair",
|
||||
"car",
|
||||
"water",
|
||||
"painting",
|
||||
"sofa",
|
||||
"shelf",
|
||||
"house",
|
||||
"sea",
|
||||
"mirror",
|
||||
"rug",
|
||||
"field",
|
||||
"armchair",
|
||||
"seat",
|
||||
"fence",
|
||||
"desk",
|
||||
"rock",
|
||||
"wardrobe",
|
||||
"lamp",
|
||||
"bathtub",
|
||||
"railing",
|
||||
"cushion",
|
||||
"base",
|
||||
"box",
|
||||
"column",
|
||||
"signboard",
|
||||
"chest of drawers",
|
||||
"counter",
|
||||
"sand",
|
||||
"sink",
|
||||
"skyscraper",
|
||||
"fireplace",
|
||||
"refrigerator",
|
||||
"grandstand",
|
||||
"path",
|
||||
"stairs",
|
||||
"runway",
|
||||
"case",
|
||||
"pool table",
|
||||
"pillow",
|
||||
"screen door",
|
||||
"stairway",
|
||||
"river",
|
||||
"bridge",
|
||||
"bookcase",
|
||||
"blind",
|
||||
"coffee table",
|
||||
"toilet",
|
||||
"flower",
|
||||
"book",
|
||||
"hill",
|
||||
"bench",
|
||||
"countertop",
|
||||
"stove",
|
||||
"palm",
|
||||
"kitchen island",
|
||||
"computer",
|
||||
"swivel chair",
|
||||
"boat",
|
||||
"bar",
|
||||
"arcade machine",
|
||||
"hovel",
|
||||
"bus",
|
||||
"towel",
|
||||
"light",
|
||||
"truck",
|
||||
"tower",
|
||||
"chandelier",
|
||||
"awning",
|
||||
"streetlight",
|
||||
"booth",
|
||||
"television receiver",
|
||||
"airplane",
|
||||
"dirt track",
|
||||
"apparel",
|
||||
"pole",
|
||||
"land",
|
||||
"bannister",
|
||||
"escalator",
|
||||
"ottoman",
|
||||
"bottle",
|
||||
"buffet",
|
||||
"poster",
|
||||
"stage",
|
||||
"van",
|
||||
"ship",
|
||||
"fountain",
|
||||
"conveyor belt",
|
||||
"canopy",
|
||||
"washer",
|
||||
"plaything",
|
||||
"swimming pool",
|
||||
"stool",
|
||||
"barrel",
|
||||
"basket",
|
||||
"waterfall",
|
||||
"tent",
|
||||
"bag",
|
||||
"minibike",
|
||||
"cradle",
|
||||
"oven",
|
||||
"ball",
|
||||
"food",
|
||||
"step",
|
||||
"tank",
|
||||
"trade name",
|
||||
"microwave",
|
||||
"pot",
|
||||
"animal",
|
||||
"bicycle",
|
||||
"lake",
|
||||
"dishwasher",
|
||||
"screen",
|
||||
"blanket",
|
||||
"sculpture",
|
||||
"hood",
|
||||
"sconce",
|
||||
"vase",
|
||||
"traffic light",
|
||||
"tray",
|
||||
"ashcan",
|
||||
"fan",
|
||||
"pier",
|
||||
"crt screen",
|
||||
"plate",
|
||||
"monitor",
|
||||
"bulletin board",
|
||||
"shower",
|
||||
"radiator",
|
||||
"glass",
|
||||
"clock",
|
||||
"flag"
|
||||
]
|
||||
Binary file not shown.
@@ -69,4 +69,23 @@ package() {
|
||||
fi
|
||||
install -Dm644 "${_src}" "${pkgdir}/usr/share/darkroom/models/${_m}"
|
||||
done
|
||||
|
||||
# The scene model, for the per-category grades. Unlike the face weights
|
||||
# this one is in the repository — AGPL, and GPLv3 §13 permits the
|
||||
# combination (models/LICENCE.md) — so it is installed unconditionally and
|
||||
# a pointer here is a broken checkout rather than a licence decision.
|
||||
#
|
||||
# Three files: the graph, its vocabulary, and the category descriptor
|
||||
# grouping ADE20K's 150 classes into what the tab shows. All three, because
|
||||
# dr_ui::library::scene_model reports the tab unavailable without any one
|
||||
# of them. Only the graph gets the pointer check — the other two are
|
||||
# legitimately a few kilobytes.
|
||||
_src="models/scene/yolo26s-sem-ade20k.onnx"
|
||||
if [[ "$(stat -c%s "${_src}")" -lt 100000 ]]; then
|
||||
echo "error: the scene model is an LFS pointer — run: git lfs pull" >&2
|
||||
return 1
|
||||
fi
|
||||
for _m in yolo26s-sem-ade20k.onnx yolo26s-sem-ade20k.classes.json categories.txt; do
|
||||
install -Dm644 "models/scene/${_m}" "${pkgdir}/usr/share/darkroom/models/${_m}"
|
||||
done
|
||||
}
|
||||
|
||||
@@ -165,6 +165,21 @@ modules:
|
||||
install -Dm644 "models/face/$m" "/app/share/darkroom/models/$m"
|
||||
done
|
||||
|
||||
# The scene model, for the per-category grades. In the repository, unlike
|
||||
# the face weights — AGPL, and GPLv3 §13 permits the combination
|
||||
# (models/LICENCE.md) — so it installs unconditionally. Three files: the
|
||||
# graph, its vocabulary, and the category descriptor; dr_ui reports the
|
||||
# tab unavailable without any one of them. The pointer check is on the
|
||||
# graph alone, the other two being legitimately small.
|
||||
- |
|
||||
if [ "$(stat -c%s models/scene/yolo26s-sem-ade20k.onnx)" -lt 100000 ]; then
|
||||
echo "error: the scene model is an LFS pointer — run: git lfs pull" >&2
|
||||
exit 1
|
||||
fi
|
||||
for m in yolo26s-sem-ade20k.onnx yolo26s-sem-ade20k.classes.json categories.txt; do
|
||||
install -Dm644 "models/scene/$m" "/app/share/darkroom/models/$m"
|
||||
done
|
||||
|
||||
- install -Dm644 README.md /app/share/doc/darkroom/README.md
|
||||
|
||||
sources:
|
||||
|
||||
@@ -1,14 +1,25 @@
|
||||
#!/usr/bin/env bash
|
||||
# Re-export the segmentation model that ships in core/dr-segment/models/.
|
||||
# Re-export the segmentation models that ship in models/ at the repository root.
|
||||
#
|
||||
# The .onnx is committed (D14), so this is not part of any build — it exists so
|
||||
# the committed artefact is reproducible rather than a binary someone once
|
||||
# produced and nobody can regenerate. Run it when bumping the model.
|
||||
# produced and nobody can regenerate. Run it when bumping a model.
|
||||
#
|
||||
# ./tools/export-seg-model.sh
|
||||
# ./tools/export-seg-model.sh # instance -> models/segment/
|
||||
# ./tools/export-seg-model.sh yolo26s-sem-ade20k # semantic -> models/scene/
|
||||
#
|
||||
# Requires `uv`. Everything else is fetched into a throwaway venv.
|
||||
#
|
||||
# ## The two models, and why they are both here
|
||||
#
|
||||
# `yolo26n-seg` is COCO instance segmentation: it separates *things*, so
|
||||
# clicking one of three people selects that person. `yolo26s-sem-ade20k` is
|
||||
# ADE20K semantic segmentation: it labels every pixel with one of 150 classes
|
||||
# including the *stuff* — sky, vegetation, water — that COCO has no word for,
|
||||
# but it merges same-class pixels into one region and so cannot tell those
|
||||
# three people apart. Neither substitutes for the other; the scene tab wants
|
||||
# the second and local adjustments want the first.
|
||||
#
|
||||
# ## Why these export flags
|
||||
#
|
||||
# `dynamic=False` is not a default we failed to change: **tract cannot parse
|
||||
@@ -20,15 +31,44 @@
|
||||
# `imgsz` square rather than a rectangle matched to 3:2: one graph has to
|
||||
# serve portrait, landscape, square crops and panoramas. A landscape-shaped
|
||||
# graph trades letterbox waste on 3:2 for worse waste on everything else.
|
||||
#
|
||||
# ## Why the semantic export is truncated
|
||||
#
|
||||
# Ultralytics ends the `-sem-` graph with `Resize -> ArgMax -> Cast`, handing
|
||||
# back a `[1, 640, 640]` u8 label map. Two reasons that tail is cut here:
|
||||
#
|
||||
# 1. **Cost.** The Resize materialises 150 x 640 x 640 x f32 — *246 MB* — and
|
||||
# ArgMax then reduces across the channel axis, which in NCHW strides
|
||||
# 409,600 elements per comparison. Measured on one machine it was roughly
|
||||
# four fifths of total runtime, for work the app throws away.
|
||||
# 2. **Softness.** ArgMax destroys the per-class scores. The scene tab needs
|
||||
# them: softmax over the 150 channels, summed within each photographic
|
||||
# category, gives per-category weights that sum to 1 at every pixel — a
|
||||
# partition of unity. Feathering those cannot double-grade a boundary,
|
||||
# where feathering hard labels outward from two adjacent categories does.
|
||||
#
|
||||
# The upsample is not information: the graph's true spatial resolution is the
|
||||
# logit grid (80x80 at imgsz=640), and the app can resample from that itself.
|
||||
set -euo pipefail
|
||||
|
||||
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
REPO="$(cd "${HERE}/.." && pwd)"
|
||||
OUT="${REPO}/core/dr-segment/models"
|
||||
|
||||
MODEL="${1:-yolo26n-seg}"
|
||||
IMGSZ="${2:-640}"
|
||||
WORK="$(mktemp -d)"
|
||||
|
||||
# Semantic models are the scene tab's; instance models are the selection path's.
|
||||
case "${MODEL}" in
|
||||
*-sem-*|*-sem) OUT="${REPO}/models/scene"; SEMANTIC=1 ;;
|
||||
*) OUT="${REPO}/models/segment"; SEMANTIC=0 ;;
|
||||
esac
|
||||
|
||||
# Not `mktemp -d`: the default TMPDIR is `/tmp`, which on most current Linux
|
||||
# distributions is a tmpfs — RAM, sized at half of physical memory. The venv
|
||||
# below pulls torch, several gigabytes of it, and installing that into RAM
|
||||
# either evicts the user's page cache or fails outright with ENOSPC on a
|
||||
# machine that has hundreds of gigabytes of actual disk free.
|
||||
WORK="$(mktemp -d -p "${TMPDIR:-/var/tmp}")"
|
||||
trap 'rm -rf "${WORK}"' EXIT
|
||||
|
||||
echo "==> exporting ${MODEL} at imgsz=${IMGSZ} in ${WORK}"
|
||||
@@ -36,12 +76,13 @@ cd "${WORK}"
|
||||
uv venv --python 3.12 venv
|
||||
VIRTUAL_ENV="${WORK}/venv" uv pip install ultralytics onnx onnxslim
|
||||
|
||||
VIRTUAL_ENV="${WORK}/venv" "${WORK}/venv/bin/python" - "${MODEL}" "${IMGSZ}" <<'PY'
|
||||
VIRTUAL_ENV="${WORK}/venv" "${WORK}/venv/bin/python" - "${MODEL}" "${IMGSZ}" "${SEMANTIC}" <<'PY'
|
||||
import sys, json
|
||||
import onnx
|
||||
from onnx import helper
|
||||
from ultralytics import YOLO
|
||||
|
||||
name = sys.argv[1]
|
||||
imgsz = int(sys.argv[2])
|
||||
name, imgsz, semantic = sys.argv[1], int(sys.argv[2]), sys.argv[3] == "1"
|
||||
m = YOLO(f"{name}.pt")
|
||||
path = m.export(format="onnx", opset=17, simplify=True, imgsz=imgsz, dynamic=False)
|
||||
print("ONNX:", path)
|
||||
@@ -52,6 +93,25 @@ print("ONNX:", path)
|
||||
with open("classes.json", "w") as f:
|
||||
json.dump([m.names[i] for i in range(len(m.names))], f, indent=1)
|
||||
print("classes:", len(m.names))
|
||||
|
||||
if semantic:
|
||||
# Drop `Resize -> ArgMax -> Cast` and expose the classifier's logits. See
|
||||
# the header for why. Matched by op type rather than by node name so a
|
||||
# re-export under a different naming scheme still works, and asserted
|
||||
# rather than assumed so an upstream graph change fails loudly here
|
||||
# instead of silently shipping a differently-shaped model.
|
||||
g = onnx.load(path).graph
|
||||
tail = [n.op_type for n in g.node[-3:]]
|
||||
assert tail == ["Resize", "ArgMax", "Cast"], f"unexpected graph tail: {tail}"
|
||||
logits = g.node[-3].input[0]
|
||||
del g.node[-3:]
|
||||
del g.output[:]
|
||||
g.output.extend([helper.make_tensor_value_info(logits, onnx.TensorProto.FLOAT, None)])
|
||||
model = onnx.shape_inference.infer_shapes(helper.make_model(g, opset_imports=[helper.make_opsetid("", 17)]))
|
||||
onnx.checker.check_model(model)
|
||||
onnx.save(model, path)
|
||||
shape = [d.dim_value for d in model.graph.output[0].type.tensor_type.shape.dim]
|
||||
print("truncated to logits:", logits, shape)
|
||||
PY
|
||||
|
||||
mkdir -p "${OUT}"
|
||||
@@ -61,4 +121,4 @@ cp "${WORK}/classes.json" "${OUT}/${MODEL}.classes.json"
|
||||
echo "==> wrote:"
|
||||
ls -la "${OUT}"
|
||||
echo
|
||||
echo "Remember: these weights are AGPL-3.0 (see ${OUT}/LICENCE.md)."
|
||||
echo "Remember: these weights are AGPL-3.0 (see ${REPO}/models/LICENCE.md)."
|
||||
|
||||
+311
-153
@@ -50,40 +50,37 @@ use crate::{AppWindow, CollectionRow};
|
||||
struct PressUndo {
|
||||
selection: BTreeSet<ImageId>,
|
||||
anchor: Option<usize>,
|
||||
previous_anchor: Option<usize>,
|
||||
cursor: Option<usize>,
|
||||
}
|
||||
|
||||
impl PressUndo {
|
||||
/// Everything [`press_remembering_anchor`] is about to change.
|
||||
/// Everything [`select_row`] is about to change.
|
||||
///
|
||||
/// The four are the whole of what a press touches, which is the property
|
||||
/// the restore depends on and the reason it is worth a test of its own: an
|
||||
/// press that grew a fifth piece of state would leave that piece behind
|
||||
/// The three are the whole of what a press touches, which is the property
|
||||
/// the restore depends on and the reason it is worth a test of its own: a
|
||||
/// press that grew a fourth piece of state would leave that piece behind
|
||||
/// after a cancel, silently.
|
||||
fn capture(
|
||||
selection: &BTreeSet<ImageId>,
|
||||
anchor: Option<usize>,
|
||||
previous_anchor: Option<usize>,
|
||||
cursor: Option<usize>,
|
||||
) -> Self {
|
||||
Self {
|
||||
selection: selection.clone(),
|
||||
anchor,
|
||||
previous_anchor,
|
||||
cursor,
|
||||
}
|
||||
}
|
||||
|
||||
/// Put it all back. Returns the two the caller holds in `Cell`s.
|
||||
/// Put it all back. Returns the one the caller holds in a `Cell`.
|
||||
fn restore(
|
||||
self,
|
||||
selection: &mut BTreeSet<ImageId>,
|
||||
anchor: &mut Option<usize>,
|
||||
) -> (Option<usize>, Option<usize>) {
|
||||
) -> Option<usize> {
|
||||
*selection = self.selection;
|
||||
*anchor = self.anchor;
|
||||
(self.previous_anchor, self.cursor)
|
||||
self.cursor
|
||||
}
|
||||
}
|
||||
|
||||
@@ -117,18 +114,15 @@ pub struct CollectionsController {
|
||||
/// same argument applied to the one index that has to survive a move.
|
||||
anchor: RefCell<Option<usize>>,
|
||||
/// TRACES: FR-UI-2 | FR-UI-4
|
||||
/// Where the anchor was *before* the press that moved it.
|
||||
/// TRACES: FR-CAT-7 | FR-UI-4
|
||||
/// A photograph the most recent press asked to take *out* of the
|
||||
/// selection, held until the release says the press was a tap.
|
||||
///
|
||||
/// The touch equivalent of shift-click needs this. A double tap is two
|
||||
/// presses, and both of them move the anchor onto the cell being tapped —
|
||||
/// so by the time the double tap is reported, "the range from the anchor to
|
||||
/// here" describes a single cell. This remembers the cell the user actually
|
||||
/// started from, which is the one they mean.
|
||||
///
|
||||
/// Updated only when a press *moves* the anchor, so the second tap of a
|
||||
/// double tap — which lands on the cell that is already the anchor — leaves
|
||||
/// it pointing where the first tap left it.
|
||||
previous_anchor: std::cell::Cell<Option<usize>>,
|
||||
/// See [`Press::Deferred`] for why removal cannot happen on the press:
|
||||
/// this is the whole of what stops a drag of forty photographs carrying
|
||||
/// one. Overwritten by the next press and dropped by the drag that
|
||||
/// consumes it, so at most one is ever pending.
|
||||
pending_toggle: std::cell::Cell<Option<ImageId>>,
|
||||
/// TRACES: FR-UI-4
|
||||
/// What the selection was immediately before the most recent press, so a
|
||||
/// gesture that turns out not to have been a press can put it back.
|
||||
@@ -368,22 +362,57 @@ impl CollectionsController {
|
||||
}
|
||||
}
|
||||
|
||||
/// TRACES: FR-CAT-7 | FR-UI-4
|
||||
/// What a press did, and what is left for the release to do.
|
||||
///
|
||||
/// Only one press has anything left over, and it is the one every multi-image
|
||||
/// drag depends on — see [`Press::Deferred`].
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
#[must_use]
|
||||
pub enum Press {
|
||||
/// The selection is already what this press means. Nothing to finish.
|
||||
Applied,
|
||||
/// **Taking a photograph out of the selection waits for the release.**
|
||||
///
|
||||
/// Putting one *in* has to happen on the press: the drag that may follow
|
||||
/// reads the selection to decide what it carries, and by the time the
|
||||
/// finger lifts it is over the sidebar. Taking one out is the opposite —
|
||||
/// nothing between the press and the release needs the image gone, and one
|
||||
/// thing very much needs it to stay.
|
||||
///
|
||||
/// A plain press has said so since selection was written: pressing an
|
||||
/// already-selected cell leaves the selection alone. Ctrl did not, and on a
|
||||
/// tablet *every* press is a ctrl-press — that is what selection mode is.
|
||||
/// So grabbing one of forty selected photographs deselected it on the way
|
||||
/// down; the drag that followed found the cell under the finger no longer
|
||||
/// in the selection, took that to mean an unselected image was being
|
||||
/// dragged, and carried it alone. Forty photographs became one, and the
|
||||
/// only clue was the grabbed cell's ring blinking out.
|
||||
///
|
||||
/// So the removal is handed back for the *click* to apply, and a click
|
||||
/// fires only for a press that stayed put — never for one that became a
|
||||
/// drag. See `tap-slop` in `library.slint`.
|
||||
Deferred(ImageId),
|
||||
}
|
||||
|
||||
/// Apply a press to the selection.
|
||||
///
|
||||
/// Split from the callback so the policy is testable without a window: this is
|
||||
/// the part a user notices being wrong.
|
||||
///
|
||||
/// - **plain** — replace the selection with this one image
|
||||
/// - **ctrl** — toggle this image, keeping the rest, and move the anchor here
|
||||
/// - **ctrl** — add this image to the selection, keeping the rest, and move the
|
||||
/// anchor here; taking one *out* is [deferred](Press::Deferred) to the release
|
||||
/// - **shift** — select the range from the anchor to here, *replacing* what was
|
||||
/// selected; the anchor stays put, so an overshoot is corrected by
|
||||
/// shift-clicking the right cell rather than starting again
|
||||
/// - **ctrl+shift** — the same range, *added* to the selection, for picking up a
|
||||
/// second run without losing the first
|
||||
///
|
||||
/// A plain press on an image that is *already* selected leaves the selection
|
||||
/// alone. That is what makes dragging a multi-selection possible at all — the
|
||||
/// press that begins the drag would otherwise collapse the selection to one.
|
||||
/// A press on an image that is *already* selected never removes it here —
|
||||
/// plainly or with ctrl. That is what makes dragging a multi-selection possible
|
||||
/// at all: the press that begins the drag would otherwise take the grabbed
|
||||
/// photograph out from under it.
|
||||
///
|
||||
/// `ids` is the **loaded window**, `offset` where it starts in the library and
|
||||
/// `row` a position within it; the anchor is kept as `offset + row`, an
|
||||
@@ -402,8 +431,10 @@ pub fn apply_press(
|
||||
ctrl: bool,
|
||||
shift: bool,
|
||||
span: &dyn Fn(usize, usize) -> Vec<ImageId>,
|
||||
) {
|
||||
let Some(&id) = ids.get(row) else { return };
|
||||
) -> Press {
|
||||
let Some(&id) = ids.get(row) else {
|
||||
return Press::Applied;
|
||||
};
|
||||
let here = offset + row;
|
||||
|
||||
if shift {
|
||||
@@ -413,7 +444,7 @@ pub fn apply_press(
|
||||
selection.clear();
|
||||
selection.insert(id);
|
||||
*anchor = Some(here);
|
||||
return;
|
||||
return Press::Applied;
|
||||
};
|
||||
|
||||
// The anchor deliberately does **not** move. Shift-clicking again
|
||||
@@ -459,15 +490,19 @@ pub fn apply_press(
|
||||
run = ids[lo_row..=hi_row].to_vec();
|
||||
}
|
||||
selection.extend(run);
|
||||
return;
|
||||
return Press::Applied;
|
||||
}
|
||||
|
||||
if ctrl {
|
||||
if !selection.remove(&id) {
|
||||
selection.insert(id);
|
||||
}
|
||||
*anchor = Some(here);
|
||||
return;
|
||||
|
||||
// Taking one out waits for the release — see [`Press::Deferred`].
|
||||
if selection.contains(&id) {
|
||||
return Press::Deferred(id);
|
||||
}
|
||||
|
||||
selection.insert(id);
|
||||
return Press::Applied;
|
||||
}
|
||||
|
||||
// Plain press on something already selected: leave it. The drag that may
|
||||
@@ -475,41 +510,13 @@ pub fn apply_press(
|
||||
// multi-image drag impossible to start.
|
||||
if selection.contains(&id) {
|
||||
*anchor = Some(here);
|
||||
return;
|
||||
return Press::Applied;
|
||||
}
|
||||
|
||||
selection.clear();
|
||||
selection.insert(id);
|
||||
*anchor = Some(here);
|
||||
}
|
||||
|
||||
/// TRACES: FR-UI-2 | FR-UI-4
|
||||
/// Apply a press, and remember where the anchor was before it moved.
|
||||
///
|
||||
/// The bookkeeping a double tap depends on, split out from the callback so the
|
||||
/// touch sequence — hold one cell, double-tap another, get the run between —
|
||||
/// can be tested without a window. See [`CollectionsController::previous_anchor`]
|
||||
/// for why the *previous* anchor is the one a double tap means.
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
pub fn press_remembering_anchor(
|
||||
selection: &mut BTreeSet<ImageId>,
|
||||
anchor: &mut Option<usize>,
|
||||
previous: &mut Option<usize>,
|
||||
ids: &[ImageId],
|
||||
offset: usize,
|
||||
row: usize,
|
||||
ctrl: bool,
|
||||
shift: bool,
|
||||
span: &dyn Fn(usize, usize) -> Vec<ImageId>,
|
||||
) {
|
||||
let before = *anchor;
|
||||
apply_press(selection, anchor, ids, offset, row, ctrl, shift, span);
|
||||
// Only a press that *moved* the anchor updates this. The second tap of a
|
||||
// double tap lands on the cell the first tap made the anchor, so it changes
|
||||
// nothing and the origin survives to be extended from.
|
||||
if *anchor != before {
|
||||
*previous = before;
|
||||
}
|
||||
Press::Applied
|
||||
}
|
||||
|
||||
/// Apply a press and push the result into the grid — the whole of what a
|
||||
@@ -533,23 +540,25 @@ pub fn select_row(
|
||||
*ctl.press_undo.borrow_mut() = Some(PressUndo::capture(
|
||||
&ctl.selection.borrow(),
|
||||
*ctl.anchor.borrow(),
|
||||
ctl.previous_anchor.get(),
|
||||
ctl.cursor(),
|
||||
));
|
||||
|
||||
let mut previous = ctl.previous_anchor.get();
|
||||
press_remembering_anchor(
|
||||
// Whatever the last press left pending is answered by this one: two
|
||||
// presses without a click in between means the first never was a tap.
|
||||
ctl.pending_toggle.set(None);
|
||||
|
||||
if let Press::Deferred(id) = apply_press(
|
||||
&mut ctl.selection.borrow_mut(),
|
||||
&mut ctl.anchor.borrow_mut(),
|
||||
&mut previous,
|
||||
ids,
|
||||
offset,
|
||||
row,
|
||||
ctrl,
|
||||
shift,
|
||||
&|first, last| ctl.span(first, last),
|
||||
);
|
||||
ctl.previous_anchor.set(previous);
|
||||
) {
|
||||
ctl.pending_toggle.set(Some(id));
|
||||
}
|
||||
// The cursor follows the press, so an arrow key after a click continues
|
||||
// from the cell that was clicked rather than from wherever the keyboard
|
||||
// was last.
|
||||
@@ -557,6 +566,26 @@ pub fn select_row(
|
||||
sync_selection(window, ctl, ids);
|
||||
}
|
||||
|
||||
/// TRACES: FR-CAT-7 | FR-UI-4
|
||||
/// Finish a press that turned out to be a tap.
|
||||
///
|
||||
/// The other half of [`Press::Deferred`]: a ctrl-press on an already-selected
|
||||
/// photograph leaves it in, because the drag that may follow has to be able to
|
||||
/// carry it, and this takes it out again once the release has proved there was
|
||||
/// no drag.
|
||||
///
|
||||
/// Called from the click, which Slint reports only for a press that stayed
|
||||
/// within `tap-slop` of where it landed — so a press that became a drag never
|
||||
/// reaches here, and a press taken over by the Flickable never reaches here
|
||||
/// either. Both leave the pending removal to be dropped by the next press.
|
||||
pub fn commit_press(window: &AppWindow, ctl: &Rc<CollectionsController>, ids: &[ImageId]) {
|
||||
let Some(id) = ctl.pending_toggle.take() else {
|
||||
return;
|
||||
};
|
||||
ctl.selection.borrow_mut().remove(&id);
|
||||
sync_selection(window, ctl, ids);
|
||||
}
|
||||
|
||||
/// TRACES: FR-UI-4
|
||||
/// Put the selection back as it was before the most recent press.
|
||||
///
|
||||
@@ -568,14 +597,22 @@ pub fn select_row(
|
||||
/// Idempotent, and a no-op when there is nothing to undo, so it is safe to
|
||||
/// call on every gesture start rather than only on the ones that need it.
|
||||
pub fn cancel_press(window: &AppWindow, ctl: &Rc<CollectionsController>, ids: &[ImageId]) {
|
||||
// The press is being unmade, so what it left for the release to finish is
|
||||
// unmade with it. Cleared before the early return: a press that changed
|
||||
// nothing to undo can still have deferred a removal — and a finger that
|
||||
// held long enough to pick a photograph up before the second one landed
|
||||
// has to put it back down, or the ring stays open around a cell nobody is
|
||||
// touching and the grid stays frozen with it.
|
||||
ctl.pending_toggle.set(None);
|
||||
window.set_library_held_row(-1);
|
||||
|
||||
let Some(undo) = ctl.press_undo.borrow_mut().take() else {
|
||||
return;
|
||||
};
|
||||
let (previous_anchor, cursor) = undo.restore(
|
||||
let cursor = undo.restore(
|
||||
&mut ctl.selection.borrow_mut(),
|
||||
&mut ctl.anchor.borrow_mut(),
|
||||
);
|
||||
ctl.previous_anchor.set(previous_anchor);
|
||||
ctl.set_cursor(cursor);
|
||||
sync_selection(window, ctl, ids);
|
||||
}
|
||||
@@ -1022,7 +1059,7 @@ pub(crate) const HOLD_DELAY_MS: u64 = 450;
|
||||
/// press that armed it has *already* selected the cell under the finger, so
|
||||
/// what firing adds is the mode: from here taps toggle rather than open, and
|
||||
/// the header's buttons appear to act on what has been gathered.
|
||||
fn arm_hold(window: &AppWindow, ctl: &Rc<CollectionsController>) {
|
||||
fn arm_hold(window: &AppWindow, ctl: &Rc<CollectionsController>, row: i32) {
|
||||
let timer = slint::Timer::default();
|
||||
let weak = window.as_weak();
|
||||
let ctl_cb = ctl.clone();
|
||||
@@ -1032,8 +1069,22 @@ fn arm_hold(window: &AppWindow, ctl: &Rc<CollectionsController>) {
|
||||
std::time::Duration::from_millis(HOLD_DELAY_MS),
|
||||
move || {
|
||||
let Some(w) = weak.upgrade() else { return };
|
||||
ctl_cb.select_mode.set(true);
|
||||
w.set_library_select_mode(true);
|
||||
|
||||
// TRACES: FR-CAT-7
|
||||
// The photograph is in the user's hand: the grid draws a ring
|
||||
// opening around it and stops scrolling underneath it, so the drag
|
||||
// that may follow cannot be lost to a flick. See `held-row` in
|
||||
// `library.slint`.
|
||||
//
|
||||
// This half happens whether or not selection mode was already on —
|
||||
// it is the half a *drag* needs, and a drag out of a selection of
|
||||
// forty starts in a mode that is already on.
|
||||
w.set_library_held_row(row);
|
||||
|
||||
if !ctl_cb.select_mode.get() {
|
||||
ctl_cb.select_mode.set(true);
|
||||
w.set_library_select_mode(true);
|
||||
}
|
||||
|
||||
// The release that follows this hold must not also open the image:
|
||||
// the user asked for a selection and would land in develop instead.
|
||||
@@ -1498,17 +1549,20 @@ pub fn wire<S, R, P, C>(
|
||||
let offset = w.get_library_offset().max(0) as usize;
|
||||
select_row(&w, &ctl, &ids, offset, row as usize, ctrl_held, shift_held);
|
||||
|
||||
// TRACES: FR-UI-2 | FR-UI-4
|
||||
// TRACES: FR-UI-2 | FR-UI-4 | FR-CAT-7
|
||||
// And start counting, in case this press is a hold. The press has
|
||||
// already selected this one cell; what the hold adds is the *mode*,
|
||||
// so the taps that follow go on selecting instead of opening the
|
||||
// next photograph the user touches.
|
||||
// next photograph the user touches — and the *pick-up*, which is
|
||||
// what makes the drag reliable.
|
||||
//
|
||||
// Not started when the mode is already on: it is on, and a second
|
||||
// hold would have nothing to do but suppress the tap that ends it.
|
||||
if !ctl.select_mode.get() {
|
||||
arm_hold(&w, &ctl);
|
||||
}
|
||||
// Armed even when the mode is already on, which it did not used to
|
||||
// be: there was nothing left for the hold to switch on, so it was
|
||||
// skipped. But the mode being on is exactly the state a
|
||||
// multi-image drag starts from, and skipping the hold left that
|
||||
// drag with no pick-up and no cue — the one gesture that most
|
||||
// needed both. See `arm_hold`.
|
||||
arm_hold(&w, &ctl, row);
|
||||
});
|
||||
}
|
||||
|
||||
@@ -1856,16 +1910,33 @@ pub fn wire<S, R, P, C>(
|
||||
// Slint owns the gesture (see the preamble). What is left here is the
|
||||
// payload — the image ids the drop will act on — and the spring.
|
||||
|
||||
// The payload is built when the drag starts, so it is the selection as it
|
||||
// stands at that moment rather than whatever it becomes mid-flight.
|
||||
// TRACES: FR-CAT-7
|
||||
// **What arms the drag, not what it carries.**
|
||||
//
|
||||
// `DragArea` declines to start a drag while its `data` is empty, and it
|
||||
// tests that on every pointer event that reaches it — the first one
|
||||
// included, which arrives long before any drag. The binding in
|
||||
// `library.slint` calls this callback, and a binding that calls a callback
|
||||
// has nothing Slint can invalidate it on: it is evaluated once, when a
|
||||
// finger first lands on that cell, and cached. `dragging` is empty at that
|
||||
// moment and stays empty until `drag-started` fires.
|
||||
//
|
||||
// So this is answered at the wrong time, and always will be. That is
|
||||
// harmless only because **`set_user_data` is called unconditionally**: an
|
||||
// empty `Vec` is still user data, so the transfer is never `is_empty()` and
|
||||
// the `DragArea` stays armed. Skipping the call for an empty selection —
|
||||
// which looks like an obvious tidy-up — would disarm every cell a finger
|
||||
// had ever touched outside a drag, and dragging would simply stop working
|
||||
// with nothing to see.
|
||||
//
|
||||
// What the drop actually reads is `dragging`, set by `drag-started` and
|
||||
// read back by `dropped-on`. `user_data` rather than plain text so that
|
||||
// nothing outside the application can interpret it as a paste.
|
||||
{
|
||||
let ctl = ctl.clone();
|
||||
window.on_library_drag_payload(move || {
|
||||
let carried = ctl.dragging.borrow().clone();
|
||||
let mut data = slint::DataTransfer::default();
|
||||
// `user_data` rather than plain text: these are catalog ids for our
|
||||
// own drop handler, not something another application should be
|
||||
// able to interpret as a paste.
|
||||
data.set_user_data(Rc::new(carried));
|
||||
data
|
||||
});
|
||||
@@ -1899,6 +1970,15 @@ pub fn wire<S, R, P, C>(
|
||||
// meant. The pinch already cancels for exactly this reason.
|
||||
*ctl.hold_timer.borrow_mut() = None;
|
||||
|
||||
// TRACES: FR-CAT-7
|
||||
// And this press was not a tap, so it never gets to take anything
|
||||
// out of the selection — see [`Press::Deferred`]. Dropped here as
|
||||
// well as on the next press because the drag reads the selection
|
||||
// one line below, and a removal still pending would be a
|
||||
// photograph the user can see is selected and the drop would not
|
||||
// carry.
|
||||
ctl.pending_toggle.set(None);
|
||||
|
||||
// Dragging an *unselected* cell carries only that one, and makes it
|
||||
// the selection — otherwise the images that travel are not the ones
|
||||
// the user grabbed. Dragging a selected cell carries the whole
|
||||
@@ -2062,6 +2142,10 @@ pub fn wire<S, R, P, C>(
|
||||
let landed = ctl.dropped_on.borrow_mut().take();
|
||||
let to_trash = ctl.trash_requested.borrow_mut().take();
|
||||
ctl.dragging.borrow_mut().clear();
|
||||
// The grid clears this itself on the cancel that starts a drag;
|
||||
// this is for the endings that reach no cell — a drop, or a drag
|
||||
// abandoned over nothing.
|
||||
w.set_library_held_row(-1);
|
||||
*ctl.hover_id.borrow_mut() = None;
|
||||
*ctl.spring_timer.borrow_mut() = None;
|
||||
|
||||
@@ -2657,6 +2741,30 @@ fn unique_name(conn: &rusqlite::Connection, parent: Option<CollectionId>) -> Str
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
|
||||
/// A press and the release that follows it — which is what a click is.
|
||||
///
|
||||
/// These tests are about what a *user* sees, and a user only ever presses
|
||||
/// and lets go. Calling [`apply_press`] alone would drop the half of the
|
||||
/// policy that waits for the release ([`Press::Deferred`]) and quietly
|
||||
/// assert the wrong thing about ctrl.
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
fn click(
|
||||
selection: &mut BTreeSet<ImageId>,
|
||||
anchor: &mut Option<usize>,
|
||||
ids: &[ImageId],
|
||||
offset: usize,
|
||||
row: usize,
|
||||
ctrl: bool,
|
||||
shift: bool,
|
||||
span: &dyn Fn(usize, usize) -> Vec<ImageId>,
|
||||
) {
|
||||
if let Press::Deferred(id) =
|
||||
apply_press(selection, anchor, ids, offset, row, ctrl, shift, span)
|
||||
{
|
||||
selection.remove(&id);
|
||||
}
|
||||
}
|
||||
use super::*;
|
||||
|
||||
fn ids(n: u64) -> Vec<ImageId> {
|
||||
@@ -2730,8 +2838,8 @@ mod tests {
|
||||
let mut sel = BTreeSet::new();
|
||||
let mut anchor = None;
|
||||
|
||||
apply_press(&mut sel, &mut anchor, &all, 0, 0, false, false, &span);
|
||||
apply_press(&mut sel, &mut anchor, &all, 0, 2, false, false, &span);
|
||||
click(&mut sel, &mut anchor, &all, 0, 0, false, false, &span);
|
||||
click(&mut sel, &mut anchor, &all, 0, 2, false, false, &span);
|
||||
|
||||
assert_eq!(sel.iter().copied().collect::<Vec<_>>(), vec![ImageId(3)]);
|
||||
}
|
||||
@@ -2743,12 +2851,12 @@ mod tests {
|
||||
let mut sel = BTreeSet::new();
|
||||
let mut anchor = None;
|
||||
|
||||
apply_press(&mut sel, &mut anchor, &all, 0, 0, false, false, &span);
|
||||
apply_press(&mut sel, &mut anchor, &all, 0, 3, true, false, &span);
|
||||
click(&mut sel, &mut anchor, &all, 0, 0, false, false, &span);
|
||||
click(&mut sel, &mut anchor, &all, 0, 3, true, false, &span);
|
||||
assert_eq!(sel.len(), 2);
|
||||
|
||||
// Toggling: a second ctrl-press on the same cell takes it out again.
|
||||
apply_press(&mut sel, &mut anchor, &all, 0, 3, true, false, &span);
|
||||
click(&mut sel, &mut anchor, &all, 0, 3, true, false, &span);
|
||||
assert_eq!(sel.iter().copied().collect::<Vec<_>>(), vec![ImageId(1)]);
|
||||
}
|
||||
|
||||
@@ -2759,8 +2867,8 @@ mod tests {
|
||||
let mut sel = BTreeSet::new();
|
||||
let mut anchor = None;
|
||||
|
||||
apply_press(&mut sel, &mut anchor, &all, 0, 2, false, false, &span);
|
||||
apply_press(&mut sel, &mut anchor, &all, 0, 6, false, true, &span);
|
||||
click(&mut sel, &mut anchor, &all, 0, 2, false, false, &span);
|
||||
click(&mut sel, &mut anchor, &all, 0, 6, false, true, &span);
|
||||
|
||||
assert_eq!(sel.len(), 5, "rows 2..=6 inclusive");
|
||||
assert!(sel.contains(&ImageId(3)) && sel.contains(&ImageId(7)));
|
||||
@@ -2773,8 +2881,8 @@ mod tests {
|
||||
let mut sel = BTreeSet::new();
|
||||
let mut anchor = None;
|
||||
|
||||
apply_press(&mut sel, &mut anchor, &all, 0, 6, false, false, &span);
|
||||
apply_press(&mut sel, &mut anchor, &all, 0, 2, false, true, &span);
|
||||
click(&mut sel, &mut anchor, &all, 0, 6, false, false, &span);
|
||||
click(&mut sel, &mut anchor, &all, 0, 2, false, true, &span);
|
||||
assert_eq!(sel.len(), 5);
|
||||
}
|
||||
|
||||
@@ -2788,12 +2896,12 @@ mod tests {
|
||||
let mut sel = BTreeSet::new();
|
||||
let mut anchor = None;
|
||||
|
||||
apply_press(&mut sel, &mut anchor, &all, 0, 5, false, false, &span);
|
||||
apply_press(&mut sel, &mut anchor, &all, 0, 15, false, true, &span);
|
||||
click(&mut sel, &mut anchor, &all, 0, 5, false, false, &span);
|
||||
click(&mut sel, &mut anchor, &all, 0, 15, false, true, &span);
|
||||
assert_eq!(sel.len(), 11, "rows 5..=15");
|
||||
|
||||
// Corrected to a shorter range from the same anchor.
|
||||
apply_press(&mut sel, &mut anchor, &all, 0, 8, false, true, &span);
|
||||
click(&mut sel, &mut anchor, &all, 0, 8, false, true, &span);
|
||||
assert_eq!(sel.len(), 4, "rows 5..=8, and nothing from the first range");
|
||||
assert!(!sel.contains(&ImageId(16)), "row 15 is no longer selected");
|
||||
}
|
||||
@@ -2807,9 +2915,9 @@ mod tests {
|
||||
let mut sel = BTreeSet::new();
|
||||
let mut anchor = None;
|
||||
|
||||
apply_press(&mut sel, &mut anchor, &all, 0, 10, false, false, &span);
|
||||
apply_press(&mut sel, &mut anchor, &all, 0, 14, false, true, &span);
|
||||
apply_press(&mut sel, &mut anchor, &all, 0, 12, false, true, &span);
|
||||
click(&mut sel, &mut anchor, &all, 0, 10, false, false, &span);
|
||||
click(&mut sel, &mut anchor, &all, 0, 14, false, true, &span);
|
||||
click(&mut sel, &mut anchor, &all, 0, 12, false, true, &span);
|
||||
|
||||
assert_eq!(anchor, Some(10));
|
||||
assert_eq!(sel.len(), 3, "rows 10..=12");
|
||||
@@ -2830,25 +2938,14 @@ mod tests {
|
||||
let span = library(&all);
|
||||
let mut sel = BTreeSet::new();
|
||||
let mut anchor = None;
|
||||
let mut previous = None;
|
||||
|
||||
// Selecting began somewhere else, so a range gesture would have had an
|
||||
// anchor to sweep from.
|
||||
press_remembering_anchor(
|
||||
&mut sel,
|
||||
&mut anchor,
|
||||
&mut previous,
|
||||
&all,
|
||||
0,
|
||||
4,
|
||||
false,
|
||||
false,
|
||||
&span,
|
||||
);
|
||||
click(&mut sel, &mut anchor, &all, 0, 4, false, false, &span);
|
||||
assert_eq!(sel.len(), 1);
|
||||
|
||||
tap(&mut sel, &mut anchor, &mut previous, &all, 11);
|
||||
tap(&mut sel, &mut anchor, &mut previous, &all, 11);
|
||||
tap(&mut sel, &mut anchor, &all, 11);
|
||||
tap(&mut sel, &mut anchor, &all, 11);
|
||||
|
||||
assert_eq!(
|
||||
sel.iter().copied().collect::<Vec<_>>(),
|
||||
@@ -2859,24 +2956,86 @@ mod tests {
|
||||
|
||||
/// One tap in selection mode: a press reported as ctrl-held, which is what
|
||||
/// `library.slint` sends while the mode is on.
|
||||
fn tap(
|
||||
sel: &mut BTreeSet<ImageId>,
|
||||
anchor: &mut Option<usize>,
|
||||
previous: &mut Option<usize>,
|
||||
all: &[ImageId],
|
||||
row: usize,
|
||||
) {
|
||||
press_remembering_anchor(
|
||||
sel,
|
||||
anchor,
|
||||
previous,
|
||||
all,
|
||||
0,
|
||||
row,
|
||||
true,
|
||||
false,
|
||||
&library(all),
|
||||
fn tap(sel: &mut BTreeSet<ImageId>, anchor: &mut Option<usize>, all: &[ImageId], row: usize) {
|
||||
click(sel, anchor, all, 0, row, true, false, &library(all));
|
||||
}
|
||||
|
||||
/// TRACES: FR-CAT-7 | FR-UI-4
|
||||
/// The bug that made a forty-image drag file one photograph.
|
||||
///
|
||||
/// In selection mode every press arrives as a ctrl-press, so grabbing one
|
||||
/// of the selected cells to drag them all used to *deselect* the cell being
|
||||
/// grabbed. `drag-started` then saw a press on an image that was not in the
|
||||
/// selection, concluded that was the gesture, and carried it alone.
|
||||
#[test]
|
||||
fn grabbing_a_selected_cell_leaves_the_whole_selection_to_drag() {
|
||||
let all = ids(20);
|
||||
let span = library(&all);
|
||||
let mut sel = BTreeSet::new();
|
||||
let mut anchor = None;
|
||||
|
||||
// Three photographs picked in selection mode, which reports ctrl.
|
||||
for row in [2, 5, 9] {
|
||||
click(&mut sel, &mut anchor, &all, 0, row, true, false, &span);
|
||||
}
|
||||
assert_eq!(sel.len(), 3);
|
||||
|
||||
// The press that begins the drag, on one of the three.
|
||||
let outcome = apply_press(&mut sel, &mut anchor, &all, 0, 5, true, false, &span);
|
||||
|
||||
assert_eq!(
|
||||
outcome,
|
||||
Press::Deferred(ImageId(6)),
|
||||
"the removal has to be handed back, not performed"
|
||||
);
|
||||
assert_eq!(
|
||||
sel.len(),
|
||||
3,
|
||||
"the drag reads the selection next, and all three have to still be in it"
|
||||
);
|
||||
assert!(sel.contains(&ImageId(6)), "least of all the one being held");
|
||||
}
|
||||
|
||||
/// And the other half: with no drag, the release still takes it out.
|
||||
///
|
||||
/// A tap in selection mode has to toggle, or there is no way to correct a
|
||||
/// mis-tap short of leaving the mode.
|
||||
#[test]
|
||||
fn a_tap_on_a_selected_cell_still_takes_it_out() {
|
||||
let all = ids(20);
|
||||
let span = library(&all);
|
||||
let mut sel = BTreeSet::new();
|
||||
let mut anchor = None;
|
||||
|
||||
for row in [2, 5, 9] {
|
||||
click(&mut sel, &mut anchor, &all, 0, row, true, false, &span);
|
||||
}
|
||||
|
||||
click(&mut sel, &mut anchor, &all, 0, 5, true, false, &span);
|
||||
|
||||
assert_eq!(
|
||||
sel.iter().copied().collect::<Vec<_>>(),
|
||||
vec![ImageId(3), ImageId(10)],
|
||||
"the tapped photograph is out and the other two stayed"
|
||||
);
|
||||
}
|
||||
|
||||
/// The anchor moves on the press whether or not the removal does.
|
||||
///
|
||||
/// "Select to…" measures from the anchor, and a run taken after tapping a
|
||||
/// selected cell has to start where the user last touched — otherwise the
|
||||
/// gesture sweeps from wherever the anchor happened to be left.
|
||||
#[test]
|
||||
fn a_deferred_press_still_moves_the_anchor() {
|
||||
let all = ids(20);
|
||||
let span = library(&all);
|
||||
let mut sel = BTreeSet::new();
|
||||
let mut anchor = None;
|
||||
|
||||
click(&mut sel, &mut anchor, &all, 0, 3, true, false, &span);
|
||||
let _ = apply_press(&mut sel, &mut anchor, &all, 0, 3, true, false, &span);
|
||||
|
||||
assert_eq!(anchor, Some(3));
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -2888,13 +3047,13 @@ mod tests {
|
||||
let mut sel = BTreeSet::new();
|
||||
let mut anchor = None;
|
||||
|
||||
apply_press(&mut sel, &mut anchor, &all, 0, 0, false, false, &span);
|
||||
apply_press(&mut sel, &mut anchor, &all, 0, 2, false, true, &span);
|
||||
click(&mut sel, &mut anchor, &all, 0, 0, false, false, &span);
|
||||
click(&mut sel, &mut anchor, &all, 0, 2, false, true, &span);
|
||||
assert_eq!(sel.len(), 3);
|
||||
|
||||
// A new anchor by ctrl-click, then a ctrl+shift range from it.
|
||||
apply_press(&mut sel, &mut anchor, &all, 0, 10, true, false, &span);
|
||||
apply_press(&mut sel, &mut anchor, &all, 0, 12, true, true, &span);
|
||||
click(&mut sel, &mut anchor, &all, 0, 10, true, false, &span);
|
||||
click(&mut sel, &mut anchor, &all, 0, 12, true, true, &span);
|
||||
|
||||
assert_eq!(sel.len(), 6, "rows 0..=2 and 10..=12");
|
||||
assert!(sel.contains(&ImageId(1)) && sel.contains(&ImageId(13)));
|
||||
@@ -2909,13 +3068,13 @@ mod tests {
|
||||
let mut sel = BTreeSet::new();
|
||||
let mut anchor = None;
|
||||
|
||||
apply_press(&mut sel, &mut anchor, &all, 0, 0, false, false, &span);
|
||||
apply_press(&mut sel, &mut anchor, &all, 0, 1, true, false, &span);
|
||||
apply_press(&mut sel, &mut anchor, &all, 0, 2, true, false, &span);
|
||||
click(&mut sel, &mut anchor, &all, 0, 0, false, false, &span);
|
||||
click(&mut sel, &mut anchor, &all, 0, 1, true, false, &span);
|
||||
click(&mut sel, &mut anchor, &all, 0, 2, true, false, &span);
|
||||
assert_eq!(sel.len(), 3);
|
||||
|
||||
// Pressing one of the three to begin a drag.
|
||||
apply_press(&mut sel, &mut anchor, &all, 0, 1, false, false, &span);
|
||||
click(&mut sel, &mut anchor, &all, 0, 1, false, false, &span);
|
||||
assert_eq!(sel.len(), 3, "the selection survived the press");
|
||||
}
|
||||
|
||||
@@ -2932,23 +3091,22 @@ mod tests {
|
||||
let mut anchor = None;
|
||||
|
||||
// A selection built the ordinary way, and the state it leaves behind.
|
||||
apply_press(&mut sel, &mut anchor, &all, 0, 1, false, false, &span);
|
||||
apply_press(&mut sel, &mut anchor, &all, 0, 3, true, false, &span);
|
||||
click(&mut sel, &mut anchor, &all, 0, 1, false, false, &span);
|
||||
click(&mut sel, &mut anchor, &all, 0, 3, true, false, &span);
|
||||
let before = (sel.clone(), anchor);
|
||||
|
||||
// The finger that opens a pinch.
|
||||
let undo = PressUndo::capture(&sel, anchor, Some(1), Some(3));
|
||||
apply_press(&mut sel, &mut anchor, &all, 0, 5, false, false, &span);
|
||||
let undo = PressUndo::capture(&sel, anchor, Some(3));
|
||||
click(&mut sel, &mut anchor, &all, 0, 5, false, false, &span);
|
||||
assert_ne!(
|
||||
(sel.clone(), anchor),
|
||||
before,
|
||||
"the press has to change something, or this proves nothing"
|
||||
);
|
||||
|
||||
let (previous_anchor, cursor) = undo.restore(&mut sel, &mut anchor);
|
||||
let cursor = undo.restore(&mut sel, &mut anchor);
|
||||
|
||||
assert_eq!((sel, anchor), before, "selection and anchor are back");
|
||||
assert_eq!(previous_anchor, Some(1), "and so is the shift-click origin");
|
||||
assert_eq!(cursor, Some(3), "and the keyboard cursor");
|
||||
}
|
||||
|
||||
@@ -2960,7 +3118,7 @@ mod tests {
|
||||
let mut sel = BTreeSet::new();
|
||||
let mut anchor = None;
|
||||
|
||||
apply_press(&mut sel, &mut anchor, &all, 0, 99, false, false, &span);
|
||||
click(&mut sel, &mut anchor, &all, 0, 99, false, false, &span);
|
||||
assert!(sel.is_empty());
|
||||
}
|
||||
|
||||
@@ -2980,12 +3138,12 @@ mod tests {
|
||||
let mut anchor = None;
|
||||
|
||||
// Anchor on the sixth image, in a window starting at the beginning.
|
||||
apply_press(&mut sel, &mut anchor, &first, 0, 5, false, false, &span);
|
||||
click(&mut sel, &mut anchor, &first, 0, 5, false, false, &span);
|
||||
assert_eq!(anchor, Some(5), "the anchor is an ordinal, not a row");
|
||||
|
||||
// The user scrolls — the window now starts four images in — and
|
||||
// shift-clicks the image at ordinal 10.
|
||||
apply_press(&mut sel, &mut anchor, &later, 4, 6, false, true, &span);
|
||||
click(&mut sel, &mut anchor, &later, 4, 6, false, true, &span);
|
||||
|
||||
assert_eq!(sel.len(), 6, "ordinals 5..=10");
|
||||
assert!(sel.contains(&ImageId(6)), "the anchored image is still in");
|
||||
@@ -3007,7 +3165,7 @@ mod tests {
|
||||
let mut sel = BTreeSet::new();
|
||||
let mut anchor = Some(0);
|
||||
|
||||
apply_press(&mut sel, &mut anchor, &window, 10, 3, false, true, &span);
|
||||
click(&mut sel, &mut anchor, &window, 10, 3, false, true, &span);
|
||||
|
||||
assert_eq!(sel.len(), 14, "ordinals 0..=13, loaded or not");
|
||||
assert!(
|
||||
@@ -3030,7 +3188,7 @@ mod tests {
|
||||
let mut sel = BTreeSet::new();
|
||||
let mut anchor = Some(25);
|
||||
|
||||
apply_press(&mut sel, &mut anchor, &window, 0, 2, false, true, &span);
|
||||
click(&mut sel, &mut anchor, &window, 0, 2, false, true, &span);
|
||||
|
||||
assert_eq!(sel.len(), 24, "ordinals 2..=25");
|
||||
assert!(sel.contains(&ImageId(3)) && sel.contains(&ImageId(26)));
|
||||
@@ -3051,7 +3209,7 @@ mod tests {
|
||||
let mut sel = BTreeSet::new();
|
||||
let mut anchor = Some(0);
|
||||
|
||||
apply_press(
|
||||
click(
|
||||
&mut sel,
|
||||
&mut anchor,
|
||||
&window,
|
||||
@@ -3073,7 +3231,7 @@ mod tests {
|
||||
let mut sel = BTreeSet::new();
|
||||
let mut anchor = None;
|
||||
|
||||
apply_press(&mut sel, &mut anchor, &all, 0, 3, false, true, &span);
|
||||
click(&mut sel, &mut anchor, &all, 0, 3, false, true, &span);
|
||||
assert_eq!(sel.iter().copied().collect::<Vec<_>>(), vec![ImageId(4)]);
|
||||
}
|
||||
|
||||
|
||||
@@ -71,6 +71,13 @@ pub const GESTURES: &[Gesture] = &[
|
||||
pointer: "Press Done in the header",
|
||||
keys: "Escape",
|
||||
},
|
||||
Gesture {
|
||||
title: "Pick a photograph up to drag it",
|
||||
section: "Library grid",
|
||||
touch: "Press and hold it until a ring opens around it, then drag",
|
||||
pointer: "Drag it",
|
||||
keys: "",
|
||||
},
|
||||
Gesture {
|
||||
title: "Select a range",
|
||||
section: "Library grid",
|
||||
|
||||
@@ -73,6 +73,15 @@ pub use develop::DevelopSession;
|
||||
/// opened — see `library::shared_face_models_dir`.
|
||||
pub use library::shared_face_models_dir;
|
||||
|
||||
/// The scene model's three files, wherever this device keeps them.
|
||||
///
|
||||
/// Public for the same reason as the directory above: the pieces that reach for
|
||||
/// a model are not all inside this crate. Exported ahead of the scene tab that
|
||||
/// will consume it so that the packaging and unpacking added alongside it have
|
||||
/// something to be verified against — `assemble-apk.sh` writing files no lookup
|
||||
/// looks for would be a silent mistake for as long as the tab took to arrive.
|
||||
pub use library::scene_model;
|
||||
|
||||
pub mod launch;
|
||||
pub mod launch_ui;
|
||||
|
||||
|
||||
@@ -3846,6 +3846,37 @@ pub fn face_models(account: &Account) -> Option<(PathBuf, PathBuf)> {
|
||||
.or_else(|| system_face_models_dirs().into_iter().find_map(pair))
|
||||
}
|
||||
|
||||
/// The scene model, its vocabulary and its category descriptor, if all three
|
||||
/// are present.
|
||||
///
|
||||
/// All three or none, for the same reason `face_models` insists on its pair:
|
||||
/// the graph alone decodes to 150 anonymous channels, and a descriptor naming
|
||||
/// classes a different model does not have is refused by
|
||||
/// `dr_segment::scene::parse_categories` anyway. Reporting the set missing is
|
||||
/// more useful than starting and failing at the first inference.
|
||||
///
|
||||
/// Searched in the same three places, most specific first — the account's own
|
||||
/// directory, the shared one, then wherever a package installed them. Android
|
||||
/// only ever finds the second, which is where `install_bundled_models` unpacks
|
||||
/// the APK's copy before any store opens.
|
||||
///
|
||||
/// Unlike the face weights this model *is* in the repository, so a desktop
|
||||
/// build from a complete checkout has it. Absent means either a checkout
|
||||
/// without `git lfs pull` or a package that chose not to carry 24 MB, and the
|
||||
/// scene tab reports itself unavailable rather than the app refusing to run.
|
||||
pub fn scene_model(account: &Account) -> Option<(PathBuf, PathBuf, PathBuf)> {
|
||||
fn set(dir: PathBuf) -> Option<(PathBuf, PathBuf, PathBuf)> {
|
||||
let model = dir.join("yolo26s-sem-ade20k.onnx");
|
||||
let classes = dir.join("yolo26s-sem-ade20k.classes.json");
|
||||
let categories = dir.join("categories.txt");
|
||||
(model.is_file() && classes.is_file() && categories.is_file())
|
||||
.then_some((model, classes, categories))
|
||||
}
|
||||
set(face_models_dir(account))
|
||||
.or_else(|| set(shared_face_models_dir()))
|
||||
.or_else(|| system_face_models_dirs().into_iter().find_map(set))
|
||||
}
|
||||
|
||||
/// Where a *package* may have installed the models.
|
||||
///
|
||||
/// `$XDG_DATA_DIRS` rather than a hard-coded `/usr/share`, because that is the
|
||||
|
||||
@@ -4562,6 +4562,12 @@ pub fn wire<F>(
|
||||
let coll_for_click = coll_ctl.clone();
|
||||
let on_open_image = on_open_image.clone();
|
||||
window.on_library_cell_clicked(move |i| {
|
||||
let Some(w) = weak.upgrade() else { return };
|
||||
|
||||
// The press stayed put, so it was a tap and not a drag: whatever it
|
||||
// held back can be applied now. See `collections_ui::Press`.
|
||||
crate::collections_ui::commit_press(&w, &coll_for_click, &ctl.visible_ids());
|
||||
|
||||
// A ctrl- or shift-click is a selection gesture. Opening the image
|
||||
// too would throw the user out of the grid mid-selection.
|
||||
if coll_for_click.press_was_modified() {
|
||||
@@ -4572,16 +4578,12 @@ pub fn wire<F>(
|
||||
if let Some(path) = path {
|
||||
// Leave the grid for the develop view. The status bar's
|
||||
// "‹ Library" button comes back here.
|
||||
if let Some(w) = weak.upgrade() {
|
||||
w.set_show_library(false);
|
||||
// Which cell the develop view is now showing, so the photo
|
||||
// roll opens marking it rather than marking nothing.
|
||||
w.set_library_roll_current(i);
|
||||
}
|
||||
w.set_show_library(false);
|
||||
// Which cell the develop view is now showing, so the photo
|
||||
// roll opens marking it rather than marking nothing.
|
||||
w.set_library_roll_current(i);
|
||||
on_open_image(path);
|
||||
if let Some(w) = weak.upgrade() {
|
||||
report_position(&w, &ctl, i as usize);
|
||||
}
|
||||
report_position(&w, &ctl, i as usize);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
@@ -543,6 +543,13 @@ export component AppWindow inherits Window {
|
||||
/// that turns it on. The long press does the same thing without it.
|
||||
in property <bool> library-select-mode: false;
|
||||
callback library-toggle-select-mode();
|
||||
/// TRACES: FR-CAT-7 | FR-UI-4
|
||||
/// The row a press has held long enough to pick up, or `-1`.
|
||||
///
|
||||
/// Set by the same hold that turns on selection mode and cleared by the
|
||||
/// grid when the press ends — `in-out` because both ends write it. See
|
||||
/// `held-row` in `library.slint` for what it draws and what it stops.
|
||||
in-out property <int> library-held-row: -1;
|
||||
/// A press on a cell ended, so the long-press timer can be cancelled.
|
||||
callback library-cell-press-ended();
|
||||
/// TRACES: FR-UI-2 | FR-UI-4
|
||||
@@ -1579,6 +1586,7 @@ in property <bool> panel-visible: true;
|
||||
cell-press-ended() => { root.library-cell-press-ended(); }
|
||||
select-mode: root.library-select-mode;
|
||||
toggle-select-mode() => { root.library-toggle-select-mode(); }
|
||||
held-row <=> root.library-held-row;
|
||||
// The sidebar's rows, not a second model: the sheet files
|
||||
// into the same tree the sidebar draws.
|
||||
collections: root.collection-rows;
|
||||
|
||||
+139
-9
@@ -1297,6 +1297,40 @@ export component LibraryGrid inherits Rectangle {
|
||||
// keys: Escape
|
||||
in property <bool> select-mode: false;
|
||||
callback toggle-select-mode();
|
||||
|
||||
/// TRACES: FR-CAT-7 | FR-UI-4
|
||||
/// The photograph a press has held long enough to pick up, or `-1`.
|
||||
///
|
||||
/// **The visible half of a gesture that was folklore.** A finger on a cell
|
||||
/// is ambiguous — it may be starting a scroll or taking hold of a
|
||||
/// photograph — and Slint resolves that by giving the `Flickable` the first
|
||||
/// half-second: any press that travels more than a few pixels vertically
|
||||
/// inside it becomes a scroll, and the drag never begins. Only a fast
|
||||
/// sideways flick, or waiting the half-second out, ever picked a
|
||||
/// photograph up, and nothing on the screen said so. The user's account of
|
||||
/// it was that dragging "sometimes works".
|
||||
///
|
||||
/// So the wait is given a mark. The same hold that turns on selection mode
|
||||
/// sets this, a ring opens outward around the cell, and from that moment
|
||||
/// the drag is the only thing the finger can be doing — the grid below is
|
||||
/// no longer `interactive`, so there is no scroll left to lose to. The cue
|
||||
/// can only ever arrive *after* the ambiguity has passed, which is the
|
||||
/// honest direction: once the ring is open, dragging works.
|
||||
///
|
||||
/// A row of the loaded window, like every other row here. It is cleared
|
||||
/// when the press ends and when a drag finishes, and the grid cannot
|
||||
/// scroll while it is set, so it cannot outlive the window it indexes.
|
||||
// GESTURE: Pick a photograph up to drag it
|
||||
// where: Library grid
|
||||
// touch: Press and hold it until a ring opens around it, then drag
|
||||
// pointer: Drag it
|
||||
// why: A finger on a photograph might be starting a scroll, and for
|
||||
// the first half-second the grid assumes it is. Holding says
|
||||
// otherwise, and the ring is the grid saying it heard — from
|
||||
// there the drag cannot be lost to a scroll. A mouse never
|
||||
// waits: the cursor is precise enough that a sideways drag is
|
||||
// unambiguous from the first pixel.
|
||||
in-out property <int> held-row: -1;
|
||||
/// TRACES: FR-CAT-5
|
||||
// --- reordering a manual collection (FR-CAT-7) --------------------------
|
||||
//
|
||||
@@ -1379,15 +1413,29 @@ export component LibraryGrid inherits Rectangle {
|
||||
/// selected. The name arrives from the sheet below rather than being
|
||||
/// invented by Rust and corrected afterwards — see `naming`.
|
||||
callback collection-from-selection(string);
|
||||
/// The drag payload: the selected image ids, wrapped by Rust. Called when a
|
||||
/// drag starts, so it always reflects the selection as it is at that moment.
|
||||
/// **What arms the drag, not what it carries.**
|
||||
///
|
||||
/// `DragArea` refuses to begin a drag while its `data` is empty, and it
|
||||
/// asks on every pointer event — including the first one, long before
|
||||
/// anything is being dragged. A binding that calls a callback is evaluated
|
||||
/// once and cached, because there is nothing for Slint to invalidate it on,
|
||||
/// so whatever this answers the first time a finger touches a cell is what
|
||||
/// that cell's `DragArea` believes for the rest of its life.
|
||||
///
|
||||
/// It is therefore *not* the payload the drop reads, whatever it looks
|
||||
/// like: what travels is `dragging` in `collections_ui`, recorded by
|
||||
/// `drag-started` and read back by `dropped-on`. See the Rust side for why
|
||||
/// this has to answer something non-empty unconditionally.
|
||||
pure callback drag-payload() -> data-transfer;
|
||||
/// What travels under the cursor: the dragged thumbnail, or a fanned stack
|
||||
/// of them where several are being carried. Composited in Rust, because
|
||||
/// Slint accepts one bitmap here and cannot draw a pile of images into it.
|
||||
in property <image> drag-image;
|
||||
/// A drag began on this cell. Lets Rust promote an unselected cell to the
|
||||
/// selection before the payload is read.
|
||||
/// A drag began on this cell.
|
||||
///
|
||||
/// This, not `drag-payload`, is where what the drag carries is decided: it
|
||||
/// fires with the selection as it stands at the moment the drag starts, and
|
||||
/// lets Rust promote an unselected cell into the selection first.
|
||||
callback drag-started(int);
|
||||
/// The drag ended — dropped or cancelled. Clears the transient UI state.
|
||||
callback drag-finished();
|
||||
@@ -2683,11 +2731,27 @@ export component LibraryGrid inherits Rectangle {
|
||||
|
||||
// --- the grid -----------------------------------------------------
|
||||
//
|
||||
// `interactive` stays true: `DragArea` and `Flickable` arbitrate
|
||||
// properly, so dragging a cell drags the cell and dragging the
|
||||
// background still flicks the grid. (This is the part a hand-rolled
|
||||
// TouchArea gesture could not do — see the drag comments above.)
|
||||
// **Scrolls until a photograph has been picked up, and not after.**
|
||||
//
|
||||
// `DragArea` and `Flickable` do arbitrate, but not evenly: the
|
||||
// Flickable claims any press that travels more than eight pixels
|
||||
// along its own axis within half a second of landing, and it holds
|
||||
// that claim until the finger lifts. That is right for the ordinary
|
||||
// case — a finger that moves is almost always scrolling — and it is
|
||||
// why dragging the background still flicks the grid.
|
||||
//
|
||||
// It is wrong once the user has said otherwise. `held-row` is that
|
||||
// saying: a press that has stayed put long enough to be a pick-up,
|
||||
// marked on the cell so the user can see it. From there this stops
|
||||
// being interactive and the drag has nothing left to lose to.
|
||||
//
|
||||
// Today the hold outlasts the Flickable's window anyway, so this
|
||||
// mostly makes an accident into a guarantee — the arbitration stops
|
||||
// depending on two constants in different crates staying in the
|
||||
// order they happen to be in. The wheel is unaffected: `interactive`
|
||||
// does not gate it.
|
||||
if root.total > 0: grid-scroll := Flickable {
|
||||
interactive: root.held-row < 0;
|
||||
// Ctrl+wheel resizes the cells; a plain wheel is declined and
|
||||
// falls through to the Flickable's own scrolling. Two jobs on
|
||||
// one gesture, distinguished by the modifier — the convention
|
||||
@@ -2903,6 +2967,7 @@ export component LibraryGrid inherits Rectangle {
|
||||
// is what a join table means, and it is why the modifier-free
|
||||
// gesture must not be `move`.
|
||||
allow-copy: true;
|
||||
// Not the payload — the arming. See `drag-payload`.
|
||||
data: root.drag-payload();
|
||||
// What travels under the cursor is the photograph itself — and
|
||||
// where several are being dragged, a stack of them. Composited
|
||||
@@ -2978,7 +3043,19 @@ export component LibraryGrid inherits Rectangle {
|
||||
// photograph, which is exactly the wrong feedback for a
|
||||
// gesture whose whole job is to say "this one". One
|
||||
// treatment, drawn inside, in one place.
|
||||
border-width: cell-touch.has-hover ? 1px : 0px;
|
||||
//
|
||||
// **Not drawn at all once there has been a finger.** A
|
||||
// hand has no hover to give, so on a tablet this ring can
|
||||
// only ever be wrong — and it is wrong in the worst way,
|
||||
// because it is the selection ring's own colour a pixel
|
||||
// thinner. `has-hover` is not reliably cleared on touch:
|
||||
// a release normally brings an `Exit` with it, but the one
|
||||
// that ends a pinch does not, so every pinch to resize the
|
||||
// thumbnails left a ring around whichever cell a finger
|
||||
// happened to have started on. The grid then showed boxes
|
||||
// around photographs that were not selected, with no way
|
||||
// to tell them from ones that were. See `touched`.
|
||||
border-width: cell-touch.has-hover && !root.touched ? 1px : 0px;
|
||||
border-color: Theme.selected-ring;
|
||||
clip: true;
|
||||
|
||||
@@ -3218,6 +3295,8 @@ export component LibraryGrid inherits Rectangle {
|
||||
root.cell-clicked(i);
|
||||
}
|
||||
self.click-pending = false;
|
||||
// Down again, whether or not it was ever up.
|
||||
root.held-row = -1;
|
||||
root.cell-press-ended();
|
||||
}
|
||||
// `cancel` is the important ending: the Flickable
|
||||
@@ -3226,6 +3305,12 @@ export component LibraryGrid inherits Rectangle {
|
||||
// would come to rest as a long press and select it.
|
||||
if (ev.kind == PointerEventKind.cancel) {
|
||||
self.click-pending = false;
|
||||
// Including the cancel a starting drag sends:
|
||||
// by then the `DragArea` has the gesture and
|
||||
// `lifted` is the mark that matters, so there
|
||||
// is no scroll left to lose and nothing to
|
||||
// keep the ring open for.
|
||||
root.held-row = -1;
|
||||
root.cell-press-ended();
|
||||
}
|
||||
}
|
||||
@@ -3399,6 +3484,51 @@ export component LibraryGrid inherits Rectangle {
|
||||
border-radius: 1.5px;
|
||||
}
|
||||
}
|
||||
|
||||
// **The photograph is in your hand now.**
|
||||
//
|
||||
// A ring that opens outward around the cell a press has held
|
||||
// long enough to pick up (see `held-row`), so the wait the
|
||||
// gesture needs has something to end in. Without it the user
|
||||
// is holding a finger on glass with no way to know whether
|
||||
// anything has happened — which is what made dragging feel
|
||||
// like a coin toss.
|
||||
//
|
||||
// **Drawn after the cells, not on one.** Cells are one `for`,
|
||||
// and z-order inside a `for` is the loop order — a cell that
|
||||
// grew past its bounds would stand over its left and top
|
||||
// neighbours and be cut off by its right and bottom ones,
|
||||
// which reads as a rendering fault rather than as a lift. One
|
||||
// element after the loop is above every cell by construction,
|
||||
// and there is only ever one photograph in the hand.
|
||||
//
|
||||
// `visible` rather than `if`, so it has somewhere to animate
|
||||
// *from*: an `if` builds the ring at its final size and it
|
||||
// would appear rather than open.
|
||||
//
|
||||
// `Theme.active` and 3px, deliberately unlike the selection
|
||||
// ring inside the cell — two marks that meant different things
|
||||
// in one colour is the mistake the hover ring made.
|
||||
Rectangle {
|
||||
property <length> pitch: root.cell-size + Theme.gap;
|
||||
property <length> reach: root.held-row >= 0 ? 5px : 0px;
|
||||
|
||||
visible: root.held-row >= 0;
|
||||
x: Theme.gap
|
||||
+ mod(root.held-row + root.offset, root.columns) * self.pitch
|
||||
- self.reach;
|
||||
y: Theme.gap
|
||||
+ floor((root.held-row + root.offset) / root.columns) * self.pitch
|
||||
- self.reach;
|
||||
width: root.cell-size + 2 * self.reach;
|
||||
height: root.cell-size + 2 * self.reach;
|
||||
animate x, y, width, height { duration: 120ms; easing: ease-out; }
|
||||
|
||||
background: transparent;
|
||||
border-width: 3px;
|
||||
border-color: Theme.active;
|
||||
border-radius: Theme.radius;
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user