Make Best one network, and retire Medium and the mixture
Best was a mixture of two experts and a gate, 110 GMAC a megapixel; Medium a single network at 48 that was softer on real edges. fb-combo (darkroom-denoise, 20 000 steps from fb-edges2, taught by the mixture with a quarter of its crops from the edge-rich parts of the frames) is Medium's shape and holds the mixture's edges on real photographs: edge PSNR within 0.04-0.06 dB at ISO 1600/6400/25600, more sharpness kept at all three, the chart's edge 0.89 photosites wide against 0.82. It is 0.27 dB short on smooth areas at ISO 25600. It becomes Best, and the methods are Bilinear, Fast and Best. Saved edits keep their numbers: 2, which was Medium, is now Best, and 3, which was Best, is past the end and reads as the default, Best. The network ships as mosaic-hq, a new name: the result cache keys a model by name and size, and this one is byte for byte the old Medium's size. Its tablet form (A16W16) lost 0.00 dB in simulated QDQ at every ISO and at most 0.09 dB across the noise bracket.
This commit is contained in:
@@ -338,7 +338,7 @@ fn unpack_bundled_models(app: &slint::android::AndroidApp) {
|
||||
// sibling when the probe chose that rung and ignores it otherwise. The
|
||||
// segmenter's and XFeat's forms are compiled into the binary instead,
|
||||
// beside their f32 graphs.
|
||||
const BUNDLED: [(&std::ffi::CStr, &str); 23] = [
|
||||
const BUNDLED: [(&std::ffi::CStr, &str); 21] = [
|
||||
(c"models/scrfd_500m_640.onnx", "scrfd_500m_640.onnx"),
|
||||
(
|
||||
c"models/scrfd_500m_640.a16w8.onnx",
|
||||
@@ -379,15 +379,10 @@ fn unpack_bundled_models(app: &slint::android::AndroidApp) {
|
||||
c"models/mosaic-fast-1408.a16w16.onnx",
|
||||
"mosaic-fast-1408.a16w16.onnx",
|
||||
),
|
||||
(c"models/mosaic-medium-1408.onnx", "mosaic-medium-1408.onnx"),
|
||||
(c"models/mosaic-hq-1408.onnx", "mosaic-hq-1408.onnx"),
|
||||
(
|
||||
c"models/mosaic-medium-1408.a16w16.onnx",
|
||||
"mosaic-medium-1408.a16w16.onnx",
|
||||
),
|
||||
(c"models/mosaic-best-1408.onnx", "mosaic-best-1408.onnx"),
|
||||
(
|
||||
c"models/mosaic-best-1408.a16w16.onnx",
|
||||
"mosaic-best-1408.a16w16.onnx",
|
||||
c"models/mosaic-hq-1408.a16w16.onnx",
|
||||
"mosaic-hq-1408.a16w16.onnx",
|
||||
),
|
||||
];
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
//!
|
||||
//! ```sh
|
||||
//! DARKROOM_ORT_DIR=~/.local/share/darkroom/runtime \
|
||||
//! cargo run --release -p dr-denoise --features native --example denoise_raw -- IMG.CR2 out [fast|medium|best]
|
||||
//! cargo run --release -p dr-denoise --features native --example denoise_raw -- IMG.CR2 out [fast|best]
|
||||
//! ```
|
||||
//!
|
||||
//! Decode, the app's hot-pixel pass, the frame's noise from its best source,
|
||||
@@ -11,7 +11,7 @@
|
||||
//! camera RGB — for comparison with the training repo's own path
|
||||
//! (`tools/compare_rust.py` in darkroom-denoise). `DARKROOM_ORT_DIR` points
|
||||
//! at an ONNX Runtime build; the engine's cache goes to `DR_ENGINE_CACHE` or
|
||||
//! a temporary directory. The whole-frame network (`mosaic-best.onnx` beside
|
||||
//! a temporary directory. The whole-frame network (`mosaic-hq.onnx` beside
|
||||
//! the fixed file) runs where the rung takes any size; `DR_PLAN=tiles` keeps
|
||||
//! the 1408² tiles anyway, to compare the two.
|
||||
|
||||
@@ -25,15 +25,14 @@ fn main() {
|
||||
env_logger::Builder::from_env(env_logger::Env::default().default_filter_or("warn")).init();
|
||||
let mut args = std::env::args().skip(1);
|
||||
let (Some(input), Some(out)) = (args.next(), args.next()) else {
|
||||
eprintln!("usage: denoise_raw RAW OUT_PREFIX [fast|medium|best]");
|
||||
eprintln!("usage: denoise_raw RAW OUT_PREFIX [fast|best]");
|
||||
std::process::exit(2);
|
||||
};
|
||||
let shipped = match args.next().as_deref() {
|
||||
None | Some("best") => dr_denoise::BEST,
|
||||
Some("medium") => dr_denoise::MEDIUM,
|
||||
Some("fast") => dr_denoise::FAST,
|
||||
Some(other) => {
|
||||
eprintln!("no network called {other}: fast, medium or best");
|
||||
eprintln!("no network called {other}: fast or best");
|
||||
std::process::exit(2);
|
||||
}
|
||||
};
|
||||
|
||||
@@ -44,19 +44,15 @@ pub const FAST: Shipped = Shipped {
|
||||
whole: "mosaic-fast.onnx",
|
||||
halo: HALO,
|
||||
};
|
||||
/// A student of the mixture with the first release's shape: 3.2 M
|
||||
/// parameters, 48 GMAC a megapixel.
|
||||
pub const MEDIUM: Shipped = Shipped {
|
||||
file: "mosaic-medium-1408.onnx",
|
||||
whole: "mosaic-medium.onnx",
|
||||
halo: HALO,
|
||||
};
|
||||
/// The mixture: a flat expert, an edge expert and the gate that blends them.
|
||||
/// It reaches further than either, so it keeps a smaller centre of each tile.
|
||||
/// One network of the first release's shape, 3.2 M parameters and 48 GMAC a
|
||||
/// megapixel, taught by the mixture of experts that was Best until 0.24:
|
||||
/// its edges at a third of its work (denoise.md §15). A new file name, not
|
||||
/// the old Medium's or Best's: the result cache keys a model by its name
|
||||
/// and size, and this one is byte for byte the old Medium's size.
|
||||
pub const BEST: Shipped = Shipped {
|
||||
file: "mosaic-best-1408.onnx",
|
||||
whole: "mosaic-best.onnx",
|
||||
halo: 256,
|
||||
file: "mosaic-hq-1408.onnx",
|
||||
whole: "mosaic-hq.onnx",
|
||||
halo: HALO,
|
||||
};
|
||||
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
|
||||
@@ -11,11 +11,11 @@
|
||||
//! with the Bayer packing spelled `SpaceToDepth`, which QNN can hold and the
|
||||
//! 6-D reshape it replaces it cannot.
|
||||
//!
|
||||
//! Each network also ships with any height and width (`mosaic-best.onnx`
|
||||
//! beside `mosaic-best-1408.onnx`, darkroom-denoise `tools/export_whole.py`,
|
||||
//! Each network also ships with any height and width (`mosaic-hq.onnx`
|
||||
//! beside `mosaic-hq-1408.onnx`, darkroom-denoise `tools/export_whole.py`,
|
||||
//! identical to the fixed file at 1408²). Where the rung takes any size, the
|
||||
//! frame runs whole instead of in tiles whose borders are thrown away —
|
||||
//! Best computes 2.47 photosites for every one it keeps in 1408² tiles
|
||||
//! frame runs whole instead of in tiles whose borders are thrown away — a
|
||||
//! 1408² tile keeps 1024², 1.89 photosites computed for each one kept
|
||||
//! (denoise.md §14).
|
||||
|
||||
use crate::tile::{Sizes, TileNet};
|
||||
|
||||
@@ -2216,13 +2216,13 @@ mod tests {
|
||||
g.set_param(
|
||||
learned_denoise::ID,
|
||||
learned_denoise::METHOD,
|
||||
learned_denoise::Method::Medium.index(),
|
||||
learned_denoise::Method::Fast.index(),
|
||||
);
|
||||
let state = g.state();
|
||||
let mut h = EditGraph::default_chain();
|
||||
h.set_denoise_available(true);
|
||||
let _ = h.set_state(&state);
|
||||
assert_eq!(h.denoise_method(), learned_denoise::Method::Medium);
|
||||
assert_eq!(h.denoise_method(), learned_denoise::Method::Fast);
|
||||
assert!((h.denoise_grain() - 0.4).abs() < 1e-6);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -35,23 +35,27 @@ pub const GRAIN: ParamId = ParamId("grain");
|
||||
|
||||
/// TRACES: FR-DEV-3g
|
||||
/// The demosaics a photograph can be developed with, in the order the
|
||||
/// sidecar numbers them. Three networks that trade time for quality — the
|
||||
/// same training, distilled into smaller students (docs/dev/denoise.md §13)
|
||||
/// — and the classical demosaic, which is no network at all.
|
||||
/// sidecar numbers them. Two networks that trade time for quality
|
||||
/// (docs/dev/denoise.md §15) and the classical demosaic, which is no network
|
||||
/// at all.
|
||||
///
|
||||
/// Until 0.24 there were four — Bilinear, Fast, Medium, Best — and the
|
||||
/// sidecar keeps their numbers: 2, which was Medium, is now Best, and 3,
|
||||
/// which was Best, is past the end and reads as the default, which is
|
||||
/// Best. Both land on the network that replaced them, with no migration.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
|
||||
pub enum Method {
|
||||
/// The classical demosaic: the noise stays.
|
||||
Bilinear,
|
||||
/// The smallest student: a quarter of the medium network's work.
|
||||
/// The smallest student: a quarter of Best's work.
|
||||
Fast,
|
||||
/// One network the size of the first release's.
|
||||
Medium,
|
||||
/// Two experts, one for flat areas and one for edges, and a gate.
|
||||
/// One network of the first release's size, taught by the mixture of
|
||||
/// experts it replaced: the mixture's edges at a third of its work.
|
||||
Best,
|
||||
}
|
||||
|
||||
impl Method {
|
||||
pub const ALL: [Method; 4] = [Method::Bilinear, Method::Fast, Method::Medium, Method::Best];
|
||||
pub const ALL: [Method; 3] = [Method::Bilinear, Method::Fast, Method::Best];
|
||||
pub const DEFAULT: Method = Method::Best;
|
||||
|
||||
/// The sidecar's number for it.
|
||||
@@ -92,7 +96,6 @@ pub(crate) static DESCRIPTOR: LazyLock<Arc<OpDescriptor>> = LazyLock::new(|| {
|
||||
vec![
|
||||
LocalizedKey("param.learned_denoise.method.bilinear"),
|
||||
LocalizedKey("param.learned_denoise.method.fast"),
|
||||
LocalizedKey("param.learned_denoise.method.medium"),
|
||||
LocalizedKey("param.learned_denoise.method.best"),
|
||||
],
|
||||
)
|
||||
@@ -117,3 +120,20 @@ pub(crate) static DESCRIPTOR: LazyLock<Arc<OpDescriptor>> = LazyLock::new(|| {
|
||||
pub fn descriptor() -> Arc<OpDescriptor> {
|
||||
DESCRIPTOR.clone()
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
/// An edit saved before 0.24 stored Medium as 2 and Best as 3. Both
|
||||
/// now name the network that replaced them, and nothing reads as Fast
|
||||
/// or Bilinear that did not before.
|
||||
#[test]
|
||||
fn the_retired_methods_read_as_best() {
|
||||
assert_eq!(Method::from_index(0.0), Method::Bilinear);
|
||||
assert_eq!(Method::from_index(1.0), Method::Fast);
|
||||
assert_eq!(Method::from_index(2.0), Method::Best, "Medium, before 0.24");
|
||||
assert_eq!(Method::from_index(3.0), Method::Best, "Best, before 0.24");
|
||||
assert_eq!(Method::Best.index(), 2.0);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -325,8 +325,8 @@ for _dir in face scene inpaint denoise; do
|
||||
README.md) continue ;;
|
||||
# The denoisers' any-size exports run whole frames on TensorRT
|
||||
# and CUDA (denoise.md §14); the Hexagon takes fixed shapes, and
|
||||
# 42 MB of graphs it never loads stay out of the APK.
|
||||
mosaic-fast.onnx | mosaic-medium.onnx | mosaic-best.onnx) continue ;;
|
||||
# 16 MB of graphs it never loads stay out of the APK.
|
||||
mosaic-fast.onnx | mosaic-hq.onnx) continue ;;
|
||||
esac
|
||||
cp "${f}" "${OUT}/staging/assets/models/"
|
||||
_bundled="${_bundled} $(basename "${f}")"
|
||||
|
||||
File diff suppressed because one or more lines are too long
+2
-3
@@ -134,10 +134,9 @@ declined, and the InsightFace grant of D13).
|
||||
|
||||
| File | Source | Trained on | Used by |
|
||||
|---|---|---|---|
|
||||
| `denoise/mosaic-best-1408.onnx` | trained in the `darkroom-denoise` repository (2026-10-04, run `final`, 30 000 steps, from the experts of runs `m2` and `edges-100`) | 1,701 of the maintainer's own base-ISO raws and 6,000 synthetic scenes the repository draws itself, with the Canon EOS 6D's measured noise added | the learned demosaic and denoise, Best (FR-DEV-3g) |
|
||||
| `denoise/mosaic-medium-1408.onnx` | distilled from `final` in the same repository (2026-10-04, run `student-m`, 20 000 steps, from `m2`) | the same | Medium |
|
||||
| `denoise/mosaic-hq-1408.onnx` | trained in the `darkroom-denoise` repository (2026-10-07, run `fb-combo`, 20 000 steps, from `fb-edges2` ← `student-m`), taught by the mixture of experts that was Best until 0.24 (run `final`) at a half share, with 10 % drawn scenes and 25 % crops from the edge-rich parts of the training frames | 1,701 of the maintainer's own base-ISO raws and 6,000 synthetic scenes the repository draws itself, with the Canon EOS 6D's measured noise added | the learned demosaic and denoise, Best (FR-DEV-3g) |
|
||||
| `denoise/mosaic-fast-1408.onnx` | distilled from `final` (2026-10-04, run `student-s`, 30 000 steps, from scratch) | the same | Fast |
|
||||
| `denoise/mosaic-{best,medium,fast}.onnx` | the three networks above, re-exported with any height and width by `tools/export_whole.py` in the same repository (2026-10-06) from the same checkpoints; identical to the 1408 files at 1408² | the same | the same methods, a whole frame at a time on a GPU (denoise.md §14) |
|
||||
| `denoise/mosaic-{hq,fast}.onnx` | the two networks above with any height and width, by `tools/export_whole.py` in the same repository from the same checkpoints; identical to the 1408 files at 1408² | the same | the same methods, a whole frame at a time on a GPU (denoise.md §14) |
|
||||
|
||||
U-Nets of plain 3×3 convolutions, ReLU, strided and transposed
|
||||
convolutions and additive skips — no third-party architecture code or
|
||||
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
+1
-1
@@ -135,7 +135,7 @@ package() {
|
||||
# own weights, GPL — models/LICENCE.md), each at the fixed 1408 tile and
|
||||
# with any height and width for a whole frame on a GPU (denoise.md §14).
|
||||
# Same pointer check, same directory.
|
||||
for _net in fast medium best; do
|
||||
for _net in fast hq; do
|
||||
for _file in "mosaic-${_net}-1408.onnx" "mosaic-${_net}.onnx"; do
|
||||
_src="models/denoise/${_file}"
|
||||
if [[ "$(stat -c%s "${_src}")" -lt 100000 ]]; then
|
||||
|
||||
@@ -844,7 +844,7 @@ def develop_zoom():
|
||||
|
||||
# The methods in the order the scene visits them: `Best` is what the
|
||||
# photograph opens with, then each smaller network, then none.
|
||||
DENOISE_METHODS = ['Best', 'Medium', 'Fast', 'Bilinear']
|
||||
DENOISE_METHODS = ['Best', 'Fast', 'Bilinear']
|
||||
DENOISE_CLOSE_UP = 560 # pixels of canvas, square, around the lamp at 1:1
|
||||
|
||||
|
||||
|
||||
@@ -42,10 +42,8 @@ TABLE = {
|
||||
"xfeat-1024": dict(dir="keypoints", form="int8", feed="xfeat", rewrites=["unfold", "resize"]),
|
||||
"xfeat-768": dict(dir="keypoints", form="int8", feed="xfeat", rewrites=["unfold", "resize"]),
|
||||
"mosaic-fast-1408": dict(dir="denoise", form="a16w16", feed=None, rewrites=["bayer"]),
|
||||
"mosaic-medium-1408": dict(dir="denoise", form="a16w16", feed=None, rewrites=["bayer"]),
|
||||
# Exported with its packs as SpaceToDepth already; the rewrite finds
|
||||
# nothing to do.
|
||||
"mosaic-best-1408": dict(dir="denoise", form="a16w16", feed=None, rewrites=["bayer"]),
|
||||
# Best since 0.24: one network, exported as Fast is, so the same rewrite.
|
||||
"mosaic-hq-1408": dict(dir="denoise", form="a16w16", feed=None, rewrites=["bayer"]),
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
# Produce the Hexagon's form of each model (docs/dev/inference.md §1.5, §5).
|
||||
#
|
||||
# ./tools/quantise-models.sh PHOTO_DIR [MODEL ...]
|
||||
# ./tools/quantise-models.sh --ranges RANGES.json mosaic-medium-1408
|
||||
# ./tools/quantise-models.sh --ranges RANGES.json mosaic-hq-1408
|
||||
#
|
||||
# Writes `<stem>.<form>.onnx` beside each canonical file under models/: a QDQ
|
||||
# graph from QNN's own quantisation config, per-channel weights, in the form
|
||||
|
||||
@@ -35,7 +35,7 @@ pub fn init(runtime_dirs: Vec<PathBuf>) {
|
||||
(Role::EyeClassifier, crate::library::SUNGLASSES_MODEL),
|
||||
(Role::Inpainter, crate::library::INPAINT_MODEL),
|
||||
]);
|
||||
let denoisers = [dr_denoise::FAST, dr_denoise::MEDIUM, dr_denoise::BEST];
|
||||
let denoisers = [dr_denoise::FAST, dr_denoise::BEST];
|
||||
wanted.extend(denoisers.map(|n| (Role::Denoiser, n.file)));
|
||||
// Their any-size siblings, which the engine compiles only on a rung
|
||||
// that runs whole frames (TensorRT; denoise.md §14).
|
||||
|
||||
@@ -227,7 +227,6 @@ fn catalogued(key: &str) -> Option<&'static str> {
|
||||
"param.learned_denoise.method" => "Method",
|
||||
"param.learned_denoise.method.bilinear" => "Bilinear",
|
||||
"param.learned_denoise.method.fast" => "Fast",
|
||||
"param.learned_denoise.method.medium" => "Medium",
|
||||
"param.learned_denoise.method.best" => "Best",
|
||||
// How strongly: 100 % is the network's result, and less puts the
|
||||
// removed noise's brightness back as grain.
|
||||
|
||||
@@ -399,7 +399,6 @@ pub fn denoise_network(
|
||||
match method {
|
||||
Method::Bilinear => None,
|
||||
Method::Fast => Some(dr_denoise::FAST),
|
||||
Method::Medium => Some(dr_denoise::MEDIUM),
|
||||
Method::Best => Some(dr_denoise::BEST),
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user