Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
7fa3176f88 | ||
|
|
af162dd010 | ||
|
|
f5300a9f43 | ||
|
|
b19d470189 | ||
|
|
bfadd9c409 | ||
|
|
050b2a3914 | ||
|
|
195388b2e3 | ||
|
|
e166b64ee9 | ||
|
|
a773ad5c27 | ||
|
|
1e472fd251 | ||
|
|
6bf67cefc4 | ||
|
|
00e2fe6aaf | ||
|
|
402dcdc24c | ||
|
|
94b39410bc | ||
|
|
2014c80e62 | ||
|
|
6fd342680b | ||
|
|
78acc73dad | ||
|
|
954246b969 | ||
|
|
baed1c4782 | ||
|
|
fbfa891296 | ||
|
|
aee62dc7f2 | ||
|
|
84fade99ec | ||
|
|
3bfa73d1e1 | ||
|
|
4616cb0a23 | ||
|
|
cc73ea3153 | ||
|
|
f6ff5eabd9 | ||
|
|
34ac2f14d3 | ||
|
|
f71d7bacc6 | ||
|
|
14ed1dc410 | ||
|
|
38819da222 | ||
|
|
d6e9c7dc94 | ||
|
|
9c8f21b754 | ||
|
|
bd7d75522d | ||
|
|
4ef1b74f2f | ||
|
|
681486196e | ||
|
|
f5d0d57574 | ||
|
|
c96e670356 | ||
|
|
9c556364fa | ||
|
|
8d72cabff5 | ||
|
|
8a90d888d5 | ||
|
|
e86edef47c | ||
|
|
0aff5e6c8c | ||
|
|
5458314083 | ||
|
|
39a22875b1 | ||
|
|
cb7b9d71c4 | ||
|
|
beb7a5eac0 | ||
|
|
262ed2553c | ||
|
|
8d08ffd7b7 | ||
|
|
8540518022 | ||
|
|
b952f5976a | ||
|
|
a1d511fd4b | ||
|
|
050c2c9d16 | ||
|
|
bd3b993b90 | ||
|
|
59605f9fbb | ||
|
|
04949741c1 | ||
|
|
5b4ad11853 | ||
|
|
6b1aac477d | ||
|
|
2afc2a7890 | ||
|
|
6507593715 | ||
|
|
08727cff5a | ||
|
|
f4c3f425dd | ||
|
|
12f8990e09 | ||
|
|
8a897bbc01 | ||
|
|
a03e082fe2 | ||
|
|
4576499c3b | ||
|
|
2f47087223 | ||
|
|
764ad55ead |
@@ -1,6 +1,6 @@
|
|||||||
name: Benchmarks
|
name: Benchmarks
|
||||||
|
|
||||||
# The suite docs/requirements.md §8 has been promising since it was written:
|
# The suite docs/dev/requirements.md §8 has been promising since it was written:
|
||||||
# "an automated benchmark suite against a synthetic 50k catalog, run per-commit
|
# "an automated benchmark suite against a synthetic 50k catalog, run per-commit
|
||||||
# … A regression beyond stated tolerance fails the build."
|
# … A regression beyond stated tolerance fails the build."
|
||||||
#
|
#
|
||||||
@@ -29,7 +29,7 @@ name: Benchmarks
|
|||||||
# every commit to establish, every time, that this runner has no GPU. It
|
# every commit to establish, every time, that this runner has no GPU. It
|
||||||
# runs on demand (Actions → Run workflow) so that a runner that *does*
|
# runs on demand (Actions → Run workflow) so that a runner that *does*
|
||||||
# have one can be pointed at it, and the numbers it produces belong in
|
# have one can be pointed at it, and the numbers it produces belong in
|
||||||
# docs/frame-budget.md by hand, as they already are.
|
# docs/dev/frame-budget.md by hand, as they already are.
|
||||||
|
|
||||||
on:
|
on:
|
||||||
push:
|
push:
|
||||||
@@ -183,7 +183,7 @@ jobs:
|
|||||||
- name: Frame budget (FR-DSP-3)
|
- name: Frame budget (FR-DSP-3)
|
||||||
run: cargo test --release -p dr-gpu --test frame_budget -- --nocapture
|
run: cargo test --release -p dr-gpu --test frame_budget -- --nocapture
|
||||||
|
|
||||||
# The instrument behind docs/frame-budget.md. It exits non-zero with no
|
# The instrument behind docs/dev/frame-budget.md. It exits non-zero with no
|
||||||
# adapter, which is right for a tool a person runs deliberately and wrong
|
# adapter, which is right for a tool a person runs deliberately and wrong
|
||||||
# for a job that usually has none — hence continue-on-error. Its table is
|
# for a job that usually has none — hence continue-on-error. Its table is
|
||||||
# in the log for whoever asked for this run; the committed numbers are
|
# in the log for whoever asked for this run; the committed numbers are
|
||||||
|
|||||||
@@ -322,7 +322,7 @@ jobs:
|
|||||||
env:
|
env:
|
||||||
CARGO_TARGET_DIR: target-android
|
CARGO_TARGET_DIR: target-android
|
||||||
# Absent secrets mean a debug signature, which is what a fork or a
|
# Absent secrets mean a debug signature, which is what a fork or a
|
||||||
# branch build should get. Set all three (see docs/android-signing.md)
|
# branch build should get. Set all three (see docs/dev/android-signing.md)
|
||||||
# and the same job produces a release-signed APK instead.
|
# and the same job produces a release-signed APK instead.
|
||||||
ANDROID_KEYSTORE_BASE64: ${{ secrets.ANDROID_KEYSTORE_BASE64 }}
|
ANDROID_KEYSTORE_BASE64: ${{ secrets.ANDROID_KEYSTORE_BASE64 }}
|
||||||
KEYSTORE_PASS: ${{ secrets.ANDROID_KEYSTORE_PASSWORD }}
|
KEYSTORE_PASS: ${{ secrets.ANDROID_KEYSTORE_PASSWORD }}
|
||||||
@@ -370,7 +370,7 @@ jobs:
|
|||||||
|
|
||||||
# TRACES: FR-PLAT-WIN-3
|
# TRACES: FR-PLAT-WIN-3
|
||||||
# The Windows executable and its installer, cross-built from Linux
|
# The Windows executable and its installer, cross-built from Linux
|
||||||
# (docs/windows.md §7). No Windows machine anywhere in this job: what it
|
# (docs/dev/windows.md §7). No Windows machine anywhere in this job: what it
|
||||||
# can prove is that the binary links, is a Windows executable with no
|
# can prove is that the binary links, is a Windows executable with no
|
||||||
# MinGW runtime imports, starts under Wine, and that the installer installs
|
# MinGW runtime imports, starts under Wine, and that the installer installs
|
||||||
# and uninstalls under Wine. What it cannot prove — a Vulkan device, a
|
# and uninstalls under Wine. What it cannot prove — a Vulkan device, a
|
||||||
@@ -454,7 +454,12 @@ jobs:
|
|||||||
wine "$SETUP" /S 2>/dev/null
|
wine "$SETUP" /S 2>/dev/null
|
||||||
INST=$(echo "$HOME"/.wine/drive_c/users/*/AppData/Local/Programs/DarkRoom)
|
INST=$(echo "$HOME"/.wine/drive_c/users/*/AppData/Local/Programs/DarkRoom)
|
||||||
ls "$INST"
|
ls "$INST"
|
||||||
[ "$(ls "$INST/models" | wc -l)" = 7 ] || { echo "FAIL: expected 7 model files"; exit 1; }
|
# As many files as package.sh stages: everything but the READMEs in
|
||||||
|
# the directories it copies. A literal here went stale the first
|
||||||
|
# time a model was added.
|
||||||
|
WANT=$(find models/face models/scene models/inpaint -maxdepth 1 -type f ! -name README.md | wc -l)
|
||||||
|
GOT=$(ls "$INST/models" | wc -l)
|
||||||
|
[ "$GOT" = "$WANT" ] || { echo "FAIL: expected $WANT model files, installed $GOT"; exit 1; }
|
||||||
wine reg query 'HKCU\Software\Microsoft\Windows\CurrentVersion\Uninstall\DarkRoom' 2>/dev/null \
|
wine reg query 'HKCU\Software\Microsoft\Windows\CurrentVersion\Uninstall\DarkRoom' 2>/dev/null \
|
||||||
| grep -q DisplayVersion || { echo "FAIL: no uninstall registry key"; exit 1; }
|
| grep -q DisplayVersion || { echo "FAIL: no uninstall registry key"; exit 1; }
|
||||||
wine "$INST/darkroom.exe" --version 2>/dev/null | grep -q '^darkroom-desktop ' \
|
wine "$INST/darkroom.exe" --version 2>/dev/null | grep -q '^darkroom-desktop ' \
|
||||||
|
|||||||
@@ -7,7 +7,7 @@ name: Traceability
|
|||||||
# fail its own threshold. Two rules follow, and the extractor's own tests
|
# fail its own threshold. Two rules follow, and the extractor's own tests
|
||||||
# enforce both:
|
# enforce both:
|
||||||
#
|
#
|
||||||
# 1. Denominators are parsed from docs/requirements.md at run time.
|
# 1. Denominators are parsed from docs/dev/requirements.md at run time.
|
||||||
# 2. Coverage is |traced ∩ defined| / |defined|, never a raw traced count.
|
# 2. Coverage is |traced ∩ defined| / |defined|, never a raw traced count.
|
||||||
#
|
#
|
||||||
# This job is static analysis of source comments plus markdown parsing, so it
|
# This job is static analysis of source comments plus markdown parsing, so it
|
||||||
@@ -74,11 +74,11 @@ jobs:
|
|||||||
run: |
|
run: |
|
||||||
set -e
|
set -e
|
||||||
cargo run -q -p traceability -- report
|
cargo run -q -p traceability -- report
|
||||||
if ! git diff --quiet docs/traceability.md; then
|
if ! git diff --quiet docs/dev/traceability.md; then
|
||||||
echo ""
|
echo ""
|
||||||
echo "docs/traceability.md is out of date."
|
echo "docs/dev/traceability.md is out of date."
|
||||||
echo "Run: cargo run -p traceability -- report"
|
echo "Run: cargo run -p traceability -- report"
|
||||||
git diff --stat docs/traceability.md
|
git diff --stat docs/dev/traceability.md
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
@@ -127,4 +127,4 @@ jobs:
|
|||||||
|
|
||||||
- name: Summary
|
- name: Summary
|
||||||
if: always()
|
if: always()
|
||||||
run: head -30 docs/traceability.md || true
|
run: head -30 docs/dev/traceability.md || true
|
||||||
|
|||||||
@@ -26,7 +26,7 @@ fi
|
|||||||
# The artefacts are generated from the tree, so regenerating them because one
|
# The artefacts are generated from the tree, so regenerating them because one
|
||||||
# was itself edited would be circular.
|
# was itself edited would be circular.
|
||||||
case "$(tr -d '[:space:]' <<< "${staged}")" in
|
case "$(tr -d '[:space:]' <<< "${staged}")" in
|
||||||
docs/traceability.md | docs/gestures.md | ui/dr-ui/src/gesture_book.rs)
|
docs/dev/traceability.md | docs/gestures.md | ui/dr-ui/src/gesture_book.rs)
|
||||||
exit 0
|
exit 0
|
||||||
;;
|
;;
|
||||||
esac
|
esac
|
||||||
@@ -41,9 +41,9 @@ if ! cargo run -q -p traceability -- report >/dev/null 2>&1; then
|
|||||||
exit 0
|
exit 0
|
||||||
fi
|
fi
|
||||||
|
|
||||||
if ! git diff --quiet -- docs/traceability.md; then
|
if ! git diff --quiet -- docs/dev/traceability.md; then
|
||||||
git add docs/traceability.md
|
git add docs/dev/traceability.md
|
||||||
echo "pre-commit: regenerated docs/traceability.md and staged it"
|
echo "pre-commit: regenerated docs/dev/traceability.md and staged it"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# The gesture vocabulary, same discipline.
|
# The gesture vocabulary, same discipline.
|
||||||
|
|||||||
@@ -2,12 +2,12 @@
|
|||||||
|
|
||||||
Notes for anyone — person or agent — changing this code. They record what
|
Notes for anyone — person or agent — changing this code. They record what
|
||||||
went wrong once and what the fix looked like, so the same shape is not
|
went wrong once and what the fix looked like, so the same shape is not
|
||||||
written again. Requirements live in `docs/requirements.md`; this file is
|
written again. Requirements live in `docs/dev/requirements.md`; this file is
|
||||||
about habits, not features.
|
about habits, not features.
|
||||||
|
|
||||||
## Catalog reads: work is proportional to what changed, never to library size
|
## Catalog reads: work is proportional to what changed, never to library size
|
||||||
|
|
||||||
`docs/catalog.md §1` states the rule. These are the ways it was broken on
|
`docs/dev/catalog.md §1` states the rule. These are the ways it was broken on
|
||||||
the Identity screen, found when every confirm click cost half a second on a
|
the Identity screen, found when every confirm click cost half a second on a
|
||||||
24k-image library (2026-09-19), and what each fix looked like.
|
24k-image library (2026-09-19), and what each fix looked like.
|
||||||
|
|
||||||
@@ -89,6 +89,30 @@ three `405`s before each `MOVE`. The backend now remembers the collections
|
|||||||
it has confirmed (`known_dirs`) for its lifetime, which is one job. When a
|
it has confirmed (`known_dirs`) for its lifetime, which is one job. When a
|
||||||
per-file operation has a per-batch precondition, satisfy it once.
|
per-file operation has a per-batch precondition, satisfy it once.
|
||||||
|
|
||||||
|
## Providers: read the runtime's source for the version on disk, not the binding
|
||||||
|
|
||||||
|
Two things the MIGraphX rung (2026-09-20) got wrong before it was measured
|
||||||
|
right, both because `ort`'s builder was trusted to mean what its method
|
||||||
|
names say.
|
||||||
|
|
||||||
|
**A binding's option builder may fill a struct the runtime no longer
|
||||||
|
reads.** `ep::MIGraphX::with_save_model` sets fields of the legacy
|
||||||
|
`OrtMIGraphXProviderOptions`; ONNX Runtime 1.29 reads that struct for the
|
||||||
|
precision flags and ignores the rest, so every session compiled for 40 s
|
||||||
|
and the cache directory went nowhere. The option that works
|
||||||
|
(`migraphx_model_cache_dir`) exists only in the generic key/value
|
||||||
|
registration, which `session::migraphx` calls on the API table directly.
|
||||||
|
Before wiring a provider option, fetch the provider's source at the
|
||||||
|
runtime's exact version and find where the option is *read*.
|
||||||
|
|
||||||
|
**A provider's cache key may leave out what you are varying.** MIGraphX
|
||||||
|
keys a compiled program on graph, GPU and its own version — not precision.
|
||||||
|
The first fp16 measurement built in 0.3 s and matched f32 to the tenth of a
|
||||||
|
millisecond, because it had loaded the f32 program. A "from cache" build
|
||||||
|
that is suspiciously fast on the first run of a new configuration is a key
|
||||||
|
collision, not a fast provider; give each precision its own directory (the
|
||||||
|
engine does) and check the cache directory gained a file.
|
||||||
|
|
||||||
## Measuring
|
## Measuring
|
||||||
|
|
||||||
`cargo run --release -p dr-ui --example identity_bench -- CATALOG THUMBS`
|
`cargo run --release -p dr-ui --example identity_bench -- CATALOG THUMBS`
|
||||||
|
|||||||
+11
-11
@@ -90,18 +90,18 @@ break it by accident:
|
|||||||
cargo run --release -p dr-bench -- check
|
cargo run --release -p dr-bench -- check
|
||||||
```
|
```
|
||||||
|
|
||||||
That is the benchmark suite (`docs/requirements.md` §8), which builds a
|
That is the benchmark suite (`docs/dev/requirements.md` §8), which builds a
|
||||||
synthetic 50,000-image catalog and fails the build if a performance target is
|
synthetic 50,000-image catalog and fails the build if a performance target is
|
||||||
missed or a measurement has drifted past its tolerance. It runs on every push in
|
missed or a measurement has drifted past its tolerance. It runs on every push in
|
||||||
its own workflow. [`docs/benchmarks.md`](docs/benchmarks.md) says what it
|
its own workflow. [`docs/dev/benchmarks.md`](docs/dev/benchmarks.md) says what it
|
||||||
measures, what it deliberately does not, and how to read a failure. If you have
|
measures, what it deliberately does not, and how to read a failure. If you have
|
||||||
touched the catalog, the decoder, the thumbnail store or the exporter, run it
|
touched the catalog, the decoder, the thumbnail store or the exporter, run it
|
||||||
before you send.
|
before you send.
|
||||||
|
|
||||||
## Requirements and traceability
|
## Requirements and traceability
|
||||||
|
|
||||||
[`requirements.md`](docs/requirements.md) is the register of record.
|
[`requirements.md`](docs/dev/requirements.md) is the register of record.
|
||||||
[`traceability.md`](docs/traceability.md) is generated from `TRACES:` tags in
|
[`traceability.md`](docs/dev/traceability.md) is generated from `TRACES:` tags in
|
||||||
the source and must never be hand-edited:
|
the source and must never be hand-edited:
|
||||||
|
|
||||||
```rust
|
```rust
|
||||||
@@ -124,7 +124,7 @@ Note that it tracks line numbers, so a change that only moves code still moves
|
|||||||
the matrix. Never regenerate it with a stale prebuilt binary.
|
the matrix. Never regenerate it with a stale prebuilt binary.
|
||||||
|
|
||||||
**One convention that the tooling cannot enforce.** A tag proves that a tag
|
**One convention that the tooling cannot enforce.** A tag proves that a tag
|
||||||
exists, not that the code under it does the thing — `docs/code-health.md`
|
exists, not that the code under it does the thing — `docs/dev/code-health.md`
|
||||||
CH-4 has the details, and two requirements currently read as covered on the
|
CH-4 has the details, and two requirements currently read as covered on the
|
||||||
strength of plumbing a future feature would use. So: **close a requirement
|
strength of plumbing a future feature would use. So: **close a requirement
|
||||||
with a test that would fail if the behaviour were removed.** Coverage that
|
with a test that would fail if the behaviour were removed.** Coverage that
|
||||||
@@ -163,12 +163,12 @@ One commit per change. If you fixed two things, that is two commits.
|
|||||||
| Document | Read it when |
|
| Document | Read it when |
|
||||||
|---|---|
|
|---|---|
|
||||||
| [`core/dr-pipeline/ops/README.md`](core/dr-pipeline/ops/README.md) | Adding or changing a develop operation — start here regardless |
|
| [`core/dr-pipeline/ops/README.md`](core/dr-pipeline/ops/README.md) | Adding or changing a develop operation — start here regardless |
|
||||||
| [`docs/architecture.md`](docs/architecture.md) | Anything touching the render path, catalog or sync |
|
| [`docs/dev/architecture.md`](docs/dev/architecture.md) | Anything touching the render path, catalog or sync |
|
||||||
| [`docs/code-health.md`](docs/code-health.md) | Deciding what to work on; grades each seam by what it costs |
|
| [`docs/dev/code-health.md`](docs/dev/code-health.md) | Deciding what to work on; grades each seam by what it costs |
|
||||||
| [`docs/benchmarks.md`](docs/benchmarks.md) | A change that could plausibly cost time or memory |
|
| [`docs/dev/benchmarks.md`](docs/dev/benchmarks.md) | A change that could plausibly cost time or memory |
|
||||||
| [`docs/technical-debt.md`](docs/technical-debt.md) | Something looks wrong — check it was not chosen |
|
| [`docs/dev/technical-debt.md`](docs/dev/technical-debt.md) | Something looks wrong — check it was not chosen |
|
||||||
| [`docs/distribution.md`](docs/distribution.md) | Packaging a build, or adding a permission to one |
|
| [`docs/dev/distribution.md`](docs/dev/distribution.md) | Packaging a build, or adding a permission to one |
|
||||||
| [`docs/requirements.md`](docs/requirements.md) | Reference, not reading |
|
| [`docs/dev/requirements.md`](docs/dev/requirements.md) | Reference, not reading |
|
||||||
|
|
||||||
`technical-debt.md` is the one to check before "fixing" anything surprising.
|
`technical-debt.md` is the one to check before "fixing" anything surprising.
|
||||||
It records compromises that were deliberate, each with the reasoning and a
|
It records compromises that were deliberate, each with the reasoning and a
|
||||||
|
|||||||
Generated
+27
-25
@@ -1221,7 +1221,7 @@ checksum = "f27ae1dd37df86211c42e150270f82743308803d90a6f6e6651cd730d5e1732f"
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "darkroom-android"
|
name = "darkroom-android"
|
||||||
version = "0.13.3"
|
version = "0.14.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"android_logger",
|
"android_logger",
|
||||||
"dr-plat",
|
"dr-plat",
|
||||||
@@ -1234,7 +1234,7 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "darkroom-desktop"
|
name = "darkroom-desktop"
|
||||||
version = "0.13.3"
|
version = "0.14.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"anyhow",
|
"anyhow",
|
||||||
"dr-plat",
|
"dr-plat",
|
||||||
@@ -1408,7 +1408,7 @@ checksum = "d8b14ccef22fc6f5a8f4d7d768562a182c04ce9a3b3157b91390b52ddfdf1a76"
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "dr-bench"
|
name = "dr-bench"
|
||||||
version = "0.13.3"
|
version = "0.14.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"anyhow",
|
"anyhow",
|
||||||
"dr-catalog",
|
"dr-catalog",
|
||||||
@@ -1425,7 +1425,7 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "dr-catalog"
|
name = "dr-catalog"
|
||||||
version = "0.13.3"
|
version = "0.14.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"dr-face",
|
"dr-face",
|
||||||
"dr-plat",
|
"dr-plat",
|
||||||
@@ -1440,7 +1440,7 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "dr-decode"
|
name = "dr-decode"
|
||||||
version = "0.13.3"
|
version = "0.14.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"dr-types",
|
"dr-types",
|
||||||
"env_logger",
|
"env_logger",
|
||||||
@@ -1454,7 +1454,7 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "dr-export"
|
name = "dr-export"
|
||||||
version = "0.13.3"
|
version = "0.14.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"dr-decode",
|
"dr-decode",
|
||||||
"dr-gpu",
|
"dr-gpu",
|
||||||
@@ -1473,7 +1473,7 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "dr-face"
|
name = "dr-face"
|
||||||
version = "0.13.3"
|
version = "0.14.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"dr-inference-engine",
|
"dr-inference-engine",
|
||||||
"env_logger",
|
"env_logger",
|
||||||
@@ -1486,7 +1486,7 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "dr-film"
|
name = "dr-film"
|
||||||
version = "0.13.3"
|
version = "0.14.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"log",
|
"log",
|
||||||
"serde",
|
"serde",
|
||||||
@@ -1495,7 +1495,7 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "dr-gpu"
|
name = "dr-gpu"
|
||||||
version = "0.13.3"
|
version = "0.14.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"bytemuck",
|
"bytemuck",
|
||||||
"dr-decode",
|
"dr-decode",
|
||||||
@@ -1513,8 +1513,9 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "dr-inference-engine"
|
name = "dr-inference-engine"
|
||||||
version = "0.13.3"
|
version = "0.14.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
|
"env_logger",
|
||||||
"libloading",
|
"libloading",
|
||||||
"log",
|
"log",
|
||||||
"ort",
|
"ort",
|
||||||
@@ -1527,7 +1528,7 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "dr-ingest"
|
name = "dr-ingest"
|
||||||
version = "0.13.3"
|
version = "0.14.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"dr-plat",
|
"dr-plat",
|
||||||
"dr-types",
|
"dr-types",
|
||||||
@@ -1539,7 +1540,7 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "dr-lens"
|
name = "dr-lens"
|
||||||
version = "0.13.3"
|
version = "0.14.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"lensfun",
|
"lensfun",
|
||||||
"log",
|
"log",
|
||||||
@@ -1547,7 +1548,7 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "dr-pano"
|
name = "dr-pano"
|
||||||
version = "0.13.3"
|
version = "0.14.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"dr-decode",
|
"dr-decode",
|
||||||
"dr-inference-engine",
|
"dr-inference-engine",
|
||||||
@@ -1561,7 +1562,7 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "dr-pipeline"
|
name = "dr-pipeline"
|
||||||
version = "0.13.3"
|
version = "0.14.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"dr-types",
|
"dr-types",
|
||||||
"log",
|
"log",
|
||||||
@@ -1570,7 +1571,7 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "dr-plat"
|
name = "dr-plat"
|
||||||
version = "0.13.3"
|
version = "0.14.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"android-native-keyring-store",
|
"android-native-keyring-store",
|
||||||
"dr-types",
|
"dr-types",
|
||||||
@@ -1586,7 +1587,7 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "dr-preset-xmp"
|
name = "dr-preset-xmp"
|
||||||
version = "0.13.3"
|
version = "0.14.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"dr-pipeline",
|
"dr-pipeline",
|
||||||
"log",
|
"log",
|
||||||
@@ -1596,7 +1597,7 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "dr-segment"
|
name = "dr-segment"
|
||||||
version = "0.13.3"
|
version = "0.14.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"dr-inference-engine",
|
"dr-inference-engine",
|
||||||
"env_logger",
|
"env_logger",
|
||||||
@@ -1609,7 +1610,7 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "dr-sync"
|
name = "dr-sync"
|
||||||
version = "0.13.3"
|
version = "0.14.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"async-trait",
|
"async-trait",
|
||||||
"dr-plat",
|
"dr-plat",
|
||||||
@@ -1623,7 +1624,7 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "dr-sync-folder"
|
name = "dr-sync-folder"
|
||||||
version = "0.13.3"
|
version = "0.14.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"async-trait",
|
"async-trait",
|
||||||
"dr-sync",
|
"dr-sync",
|
||||||
@@ -1635,7 +1636,7 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "dr-sync-nextcloud"
|
name = "dr-sync-nextcloud"
|
||||||
version = "0.13.3"
|
version = "0.14.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"async-trait",
|
"async-trait",
|
||||||
"dr-decode",
|
"dr-decode",
|
||||||
@@ -1657,7 +1658,7 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "dr-thumbs"
|
name = "dr-thumbs"
|
||||||
version = "0.13.3"
|
version = "0.14.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"dr-types",
|
"dr-types",
|
||||||
"jpeg-encoder",
|
"jpeg-encoder",
|
||||||
@@ -1669,7 +1670,7 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "dr-types"
|
name = "dr-types"
|
||||||
version = "0.13.3"
|
version = "0.14.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"serde",
|
"serde",
|
||||||
"serde_json",
|
"serde_json",
|
||||||
@@ -1678,7 +1679,7 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "dr-ui"
|
name = "dr-ui"
|
||||||
version = "0.13.3"
|
version = "0.14.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"anyhow",
|
"anyhow",
|
||||||
"async-trait",
|
"async-trait",
|
||||||
@@ -1706,6 +1707,7 @@ dependencies = [
|
|||||||
"jni 0.22.4",
|
"jni 0.22.4",
|
||||||
"log",
|
"log",
|
||||||
"ndk-context",
|
"ndk-context",
|
||||||
|
"png",
|
||||||
"pollster",
|
"pollster",
|
||||||
"reqwest",
|
"reqwest",
|
||||||
"rusqlite",
|
"rusqlite",
|
||||||
@@ -1720,7 +1722,7 @@ dependencies = [
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "dr-xmp"
|
name = "dr-xmp"
|
||||||
version = "0.13.3"
|
version = "0.14.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"dr-types",
|
"dr-types",
|
||||||
"log",
|
"log",
|
||||||
@@ -7021,7 +7023,7 @@ checksum = "8df9b6e13f2d32c91b9bd719c00d1958837bc7dec474d94952798cc8e69eeec3"
|
|||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "traceability"
|
name = "traceability"
|
||||||
version = "0.13.3"
|
version = "0.14.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"anyhow",
|
"anyhow",
|
||||||
"serde",
|
"serde",
|
||||||
|
|||||||
+2
-2
@@ -29,7 +29,7 @@ members = [
|
|||||||
]
|
]
|
||||||
|
|
||||||
[workspace.package]
|
[workspace.package]
|
||||||
version = "0.13.3"
|
version = "0.14.0"
|
||||||
edition = "2021"
|
edition = "2021"
|
||||||
rust-version = "1.92"
|
rust-version = "1.92"
|
||||||
license = "GPL-3.0-or-later"
|
license = "GPL-3.0-or-later"
|
||||||
@@ -48,7 +48,7 @@ dr-export = { path = "core/dr-export" }
|
|||||||
dr-face = { path = "core/dr-face", default-features = false }
|
dr-face = { path = "core/dr-face", default-features = false }
|
||||||
dr-film = { path = "core/dr-film" }
|
dr-film = { path = "core/dr-film" }
|
||||||
# `tract` on by default so a test binary can open a session with nothing
|
# `tract` on by default so a test binary can open a session with nothing
|
||||||
# installed; the apps add `native` to look for a runtime file (docs/inference.md §3).
|
# installed; the apps add `native` to look for a runtime file (docs/dev/inference.md §3).
|
||||||
dr-inference-engine = { path = "core/dr-inference-engine" }
|
dr-inference-engine = { path = "core/dr-inference-engine" }
|
||||||
dr-ingest = { path = "core/dr-ingest" }
|
dr-ingest = { path = "core/dr-ingest" }
|
||||||
dr-gpu = { path = "core/dr-gpu" }
|
dr-gpu = { path = "core/dr-gpu" }
|
||||||
|
|||||||
@@ -52,7 +52,7 @@ texture directly — no readback between the GPU and the screen.
|
|||||||
|---|---|---|
|
|---|---|---|
|
||||||
| Arch Linux | [`packaging/PKGBUILD`](packaging/PKGBUILD) — `makepkg -si` | Built from every release |
|
| Arch Linux | [`packaging/PKGBUILD`](packaging/PKGBUILD) — `makepkg -si` | Built from every release |
|
||||||
| Android | The APK from each CI run, or `./docker/android/package.sh --install` | Runs on a tablet; F-Droid not yet submitted |
|
| Android | The APK from each CI run, or `./docker/android/package.sh --install` | Runs on a tablet; F-Droid not yet submitted |
|
||||||
| Windows | `DarkRoom-<version>-x86_64-setup.exe`, cross-built by CI ([windows.md](docs/windows.md)) | Verified under Wine only; unsigned |
|
| Windows | `DarkRoom-<version>-x86_64-setup.exe`, cross-built by CI ([windows.md](docs/dev/windows.md)) | Verified under Wine only; unsigned |
|
||||||
| Flatpak | [`packaging/flatpak/`](packaging/flatpak/) | Manifest in tree; choosing a library does not yet work in the sandbox |
|
| Flatpak | [`packaging/flatpak/`](packaging/flatpak/) | Manifest in tree; choosing a library does not yet work in the sandbox |
|
||||||
|
|
||||||
Or build it. Git LFS is required for the model weights, and the toolchain
|
Or build it. Git LFS is required for the model weights, and the toolchain
|
||||||
@@ -76,30 +76,30 @@ controls, its place in the chain and its tests.
|
|||||||
|
|
||||||
## Where it stands
|
## Where it stands
|
||||||
|
|
||||||
**0.13.3**, seventeen tagged releases in. 184 numbered requirements in
|
**0.14.0**, twenty-one tagged releases in. 188 numbered requirements in
|
||||||
scope, 84% of them claimed by code and [traced to it](docs/traceability.md);
|
scope, 82% of them claimed by code and [traced to it](docs/dev/traceability.md);
|
||||||
the rest are written down rather than merely absent.
|
the rest are written down rather than merely absent.
|
||||||
|
|
||||||
**Not built:** plugins (post-v1, [D12](docs/requirements.md)), compare and
|
**Not built:** plugins (post-v1, [D12](docs/dev/requirements.md)), compare and
|
||||||
survey culling, AI denoise, tiled and progressive rendering, HDR merge and
|
survey culling, AI denoise, tiled and progressive rendering, HDR merge and
|
||||||
focus stacking, most of the Android platform integration beyond running,
|
focus stacking, most of the Android platform integration beyond running,
|
||||||
and the Flatpak's library chooser. The performance targets are half
|
and the Flatpak's library chooser. The performance targets are half
|
||||||
verified: the per-commit benchmark suite §8 requires exists for everything
|
verified: the per-commit benchmark suite §8 requires exists for everything
|
||||||
that does not need a frame — the catalog, the scan, the thumbnails — and
|
that does not need a frame — the catalog, the scan, the thumbnails — and
|
||||||
not yet for the render path, so a regression there fails nothing.
|
not yet for the render path, so a regression there fails nothing.
|
||||||
[outstanding.md](docs/outstanding.md) is the list, with the reasoning for
|
[outstanding.md](docs/dev/outstanding.md) is the list, with the reasoning for
|
||||||
each.
|
each.
|
||||||
|
|
||||||
**The one deliberate compromise worth knowing about before reading
|
**The one deliberate compromise worth knowing about before reading
|
||||||
anything else:** the Android develop view reads its frame back through the
|
anything else:** the Android develop view reads its frame back through the
|
||||||
CPU, because zero-copy there needs wgpu's Vulkan swapchain and that tears a
|
CPU, because zero-copy there needs wgpu's Vulkan swapchain and that tears a
|
||||||
portrait window on a tablet whose panel is mounted landscape. It is debt,
|
portrait window on a tablet whose panel is mounted landscape. It is debt,
|
||||||
not a revision of the rule — [technical-debt.md TD-1](docs/technical-debt.md)
|
not a revision of the rule — [technical-debt.md TD-1](docs/dev/technical-debt.md)
|
||||||
has the measurements and the three things any one of which would remove it.
|
has the measurements and the three things any one of which would remove it.
|
||||||
|
|
||||||
## Documentation
|
## Documentation
|
||||||
|
|
||||||
For someone using it:
|
[docs/README.md](docs/README.md) is the index. The short version, for someone using it:
|
||||||
|
|
||||||
| | |
|
| | |
|
||||||
|---|---|
|
|---|---|
|
||||||
@@ -111,21 +111,21 @@ For someone changing it:
|
|||||||
| | |
|
| | |
|
||||||
|---|---|
|
|---|---|
|
||||||
| [CONTRIBUTING.md](CONTRIBUTING.md) | How to land a first change without reading the rest |
|
| [CONTRIBUTING.md](CONTRIBUTING.md) | How to land a first change without reading the rest |
|
||||||
| [requirements.md](docs/requirements.md) | What the software must do — the numbered register, and the decisions |
|
| [requirements.md](docs/dev/requirements.md) | What the software must do — the numbered register, and the decisions |
|
||||||
| [architecture.md](docs/architecture.md) | How it is built — crates, the GPU pipeline, the data model, sync |
|
| [architecture.md](docs/dev/architecture.md) | How it is built — crates, the GPU pipeline, the data model, sync |
|
||||||
| [technical-debt.md](docs/technical-debt.md) | Compromises taken deliberately, each with the condition that retires it |
|
| [technical-debt.md](docs/dev/technical-debt.md) | Compromises taken deliberately, each with the condition that retires it |
|
||||||
| [outstanding.md](docs/outstanding.md) | What is not built, and whether that is a decision or a gap |
|
| [outstanding.md](docs/dev/outstanding.md) | What is not built, and whether that is a decision or a gap |
|
||||||
| [code-health.md](docs/code-health.md) | What a contribution costs, per seam, measured |
|
| [code-health.md](docs/dev/code-health.md) | What a contribution costs, per seam, measured |
|
||||||
| [traceability.md](docs/traceability.md) | Generated: which requirement is claimed by which file |
|
| [traceability.md](docs/dev/traceability.md) | Generated: which requirement is claimed by which file |
|
||||||
|
|
||||||
Designs, one per subsystem:
|
Designs, one per subsystem:
|
||||||
[segmentation](docs/segmentation.md) and [mask editing](docs/mask-editing.md) ·
|
[segmentation](docs/dev/segmentation.md) and [mask editing](docs/dev/mask-editing.md) ·
|
||||||
[spot removal](docs/spot-removal.md) · [panorama](docs/panorama.md) ·
|
[spot removal](docs/dev/spot-removal.md) · [panorama](docs/dev/panorama.md) ·
|
||||||
[faces](docs/faces.md) · [inference](docs/inference.md) ·
|
[faces](docs/dev/faces.md) · [inference](docs/dev/inference.md) ·
|
||||||
[storage and sync](docs/storage.md) · [catalog](docs/catalog.md) ·
|
[storage and sync](docs/dev/storage.md) · [catalog](docs/dev/catalog.md) ·
|
||||||
[display and extension](docs/display-and-extension.md) ·
|
[display and extension](docs/dev/display-and-extension.md) ·
|
||||||
[navigation](docs/ui-navigation.md) · [distribution](docs/distribution.md) ·
|
[navigation](docs/dev/ui-navigation.md) · [distribution](docs/dev/distribution.md) ·
|
||||||
[windows](docs/windows.md) · [benchmarks](docs/benchmarks.md).
|
[windows](docs/dev/windows.md) · [benchmarks](docs/dev/benchmarks.md).
|
||||||
|
|
||||||
## Licence
|
## Licence
|
||||||
|
|
||||||
|
|||||||
@@ -240,7 +240,7 @@ fn android_main(app: slint::android::AndroidApp) {
|
|||||||
///
|
///
|
||||||
/// **Face weights are absent from the repository by design.** The InsightFace
|
/// **Face weights are absent from the repository by design.** The InsightFace
|
||||||
/// grant is research-only and incompatible with this project's licence
|
/// grant is research-only and incompatible with this project's licence
|
||||||
/// (docs/faces.md §2), so a desktop user fetches them, runs
|
/// (docs/dev/faces.md §2), so a desktop user fetches them, runs
|
||||||
/// `tools/fix-face-model-shapes.sh` over them, and drops the result in. A build
|
/// `tools/fix-face-model-shapes.sh` over them, and drops the result in. A build
|
||||||
/// that carries none is the ordinary case and face indexing simply stays off.
|
/// that carries none is the ordinary case and face indexing simply stays off.
|
||||||
///
|
///
|
||||||
@@ -321,17 +321,17 @@ fn unpack_bundled_models(app: &slint::android::AndroidApp) {
|
|||||||
// before it reports the tab available.
|
// before it reports the tab available.
|
||||||
//
|
//
|
||||||
// Three detectors, because which one runs is a setting
|
// Three detectors, because which one runs is a setting
|
||||||
// (`FaceDetector`, docs/faces.md §12.3) and a tablet has no other way to
|
// (`FaceDetector`, docs/dev/faces.md §12.3) and a tablet has no other way to
|
||||||
// obtain the one it was not shipped with. Twenty megabytes of APK for
|
// obtain the one it was not shipped with. Twenty megabytes of APK for
|
||||||
// the choice; the embedder is the same for all three.
|
// the choice; the embedder is the same for all three.
|
||||||
//
|
//
|
||||||
// Then the three eye-state models (docs/faces.md §17): landmarks, open
|
// Then the three eye-state models (docs/dev/faces.md §17): landmarks, open
|
||||||
// or closed, sunglasses. The app indexes without them; with them the
|
// or closed, sunglasses. The app indexes without them; with them the
|
||||||
// eyes-open filter has something to read, and a tablet has no other way
|
// eyes-open filter has something to read, and a tablet has no other way
|
||||||
// to get them either.
|
// to get them either.
|
||||||
//
|
//
|
||||||
// The int8 forms beside the three detectors are what the Hexagon runs
|
// The int8 forms beside the three detectors are what the Hexagon runs
|
||||||
// (docs/inference.md §5); the engine loads the sibling when the probe
|
// (docs/dev/inference.md §5); the engine loads the sibling when the probe
|
||||||
// chose that rung and ignores it otherwise.
|
// chose that rung and ignores it otherwise.
|
||||||
const BUNDLED: [(&std::ffi::CStr, &str); 14] = [
|
const BUNDLED: [(&std::ffi::CStr, &str); 14] = [
|
||||||
(c"models/scrfd_500m_640.onnx", "scrfd_500m_640.onnx"),
|
(c"models/scrfd_500m_640.onnx", "scrfd_500m_640.onnx"),
|
||||||
@@ -420,7 +420,7 @@ fn unpack_bundled_models(app: &slint::android::AndroidApp) {
|
|||||||
// on a first launch they were not on disk until this line. The runtime
|
// on a first launch they were not on disk until this line. The runtime
|
||||||
// is in the APK's native library directory beside `libdarkroom.so`,
|
// is in the APK's native library directory beside `libdarkroom.so`,
|
||||||
// which is also where Qualcomm's DSP loader has to be pointed for the
|
// which is also where Qualcomm's DSP loader has to be pointed for the
|
||||||
// Hexagon skel (docs/inference.md §3, §8).
|
// Hexagon skel (docs/dev/inference.md §3, §8).
|
||||||
dr_ui::inference::init(native_library_dir().into_iter().collect());
|
dr_ui::inference::init(native_library_dir().into_iter().collect());
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -21,7 +21,7 @@ fn main() -> anyhow::Result<()> {
|
|||||||
// build made on a machine that cannot run the application — the Linux CI
|
// build made on a machine that cannot run the application — the Linux CI
|
||||||
// producing the Windows binary, checked under Wine — has an exit that
|
// producing the Windows binary, checked under Wine — has an exit that
|
||||||
// proves the executable starts without opening a window or touching the
|
// proves the executable starts without opening a window or touching the
|
||||||
// user's directories (docs/windows.md §6).
|
// user's directories (docs/dev/windows.md §6).
|
||||||
if std::env::args().nth(1).as_deref() == Some("--version") {
|
if std::env::args().nth(1).as_deref() == Some("--version") {
|
||||||
println!("darkroom-desktop {}", env!("CARGO_PKG_VERSION"));
|
println!("darkroom-desktop {}", env!("CARGO_PKG_VERSION"));
|
||||||
return Ok(());
|
return Ok(());
|
||||||
@@ -63,7 +63,7 @@ fn main() -> anyhow::Result<()> {
|
|||||||
|
|
||||||
// Before the window: the probe runs on its own thread and the first
|
// Before the window: the probe runs on its own thread and the first
|
||||||
// frame does not wait for it, but the models a background job asks for
|
// frame does not wait for it, but the models a background job asks for
|
||||||
// should already know where the runtime is (docs/inference.md §4).
|
// should already know where the runtime is (docs/dev/inference.md §4).
|
||||||
dr_ui::inference::init(runtime_dirs());
|
dr_ui::inference::init(runtime_dirs());
|
||||||
|
|
||||||
dr_ui::run(paths)?;
|
dr_ui::run(paths)?;
|
||||||
@@ -78,7 +78,7 @@ fn main() -> anyhow::Result<()> {
|
|||||||
|
|
||||||
/// Where a desktop package may have put `libonnxruntime`, most specific
|
/// Where a desktop package may have put `libonnxruntime`, most specific
|
||||||
/// first. None of these existing is the tract build, which is a complete
|
/// first. None of these existing is the tract build, which is a complete
|
||||||
/// application and not an error (docs/inference.md §3).
|
/// application and not an error (docs/dev/inference.md §3).
|
||||||
///
|
///
|
||||||
/// `DARKROOM_ORT_DIR` is for a developer pointing at a runtime that is not
|
/// `DARKROOM_ORT_DIR` is for a developer pointing at a runtime that is not
|
||||||
/// installed — the wheel's `capi` directory, say. Then beside the executable
|
/// installed — the wheel's `capi` directory, say. Then beside the executable
|
||||||
|
|||||||
@@ -315,7 +315,7 @@ fn full_library(
|
|||||||
|
|
||||||
// The three phases, separately, because "a regroup takes n seconds" does
|
// The three phases, separately, because "a regroup takes n seconds" does
|
||||||
// not tell anyone which half to optimise — and the answer differs between
|
// not tell anyone which half to optimise — and the answer differs between
|
||||||
// a desktop and a tablet (docs/faces.md §9).
|
// a desktop and a tablet (docs/dev/faces.md §9).
|
||||||
{
|
{
|
||||||
let dim = candidates.first().map(|c| c.embedding.len()).unwrap_or(0);
|
let dim = candidates.first().map(|c| c.embedding.len()).unwrap_or(0);
|
||||||
let flat: Vec<f32> = candidates
|
let flat: Vec<f32> = candidates
|
||||||
|
|||||||
@@ -82,7 +82,7 @@
|
|||||||
//! Grouping has no natural `subject_id`: it is a property of a *run* of frames,
|
//! Grouping has no natural `subject_id`: it is a property of a *run* of frames,
|
||||||
//! so a per-image job would rebuild the world once per photograph. It is
|
//! so a per-image job would rebuild the world once per photograph. It is
|
||||||
//! therefore a debounced library-level pass, for exactly the reasons
|
//! therefore a debounced library-level pass, for exactly the reasons
|
||||||
//! docs/catalog.md §10.2 gives for face clustering, and [`regroup`] is the whole
|
//! docs/dev/catalog.md §10.2 gives for face clustering, and [`regroup`] is the whole
|
||||||
//! of it — one ordered walk, no per-pair comparison beyond adjacent frames.
|
//! of it — one ordered walk, no per-pair comparison beyond adjacent frames.
|
||||||
//!
|
//!
|
||||||
//! # Grouping is not hiding
|
//! # Grouping is not hiding
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
//! Face data as sealed shards, so a second device does not re-index the library.
|
//! Face data as sealed shards, so a second device does not re-index the library.
|
||||||
//!
|
//!
|
||||||
//! Indexing a 23,500-image library is on the order of two hours of CPU
|
//! Indexing a 23,500-image library is on the order of two hours of CPU
|
||||||
//! (docs/faces.md §12.2). It is also **byte-identical on every device**: the
|
//! (docs/dev/faces.md §12.2). It is also **byte-identical on every device**: the
|
||||||
//! same model over the same proxy produces the same embedding. Paying for it
|
//! same model over the same proxy produces the same embedding. Paying for it
|
||||||
//! once per account rather than once per device is the whole point of this
|
//! once per account rather than once per device is the whole point of this
|
||||||
//! module, and it is the same bargain the thumbnail store already makes.
|
//! module, and it is the same bargain the thumbnail store already makes.
|
||||||
@@ -178,6 +178,67 @@ impl FaceShardStore {
|
|||||||
.flatten()
|
.flatten()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The other pipelines this file is held under that share `model_id`'s
|
||||||
|
/// embedder — the generations a put of `model_id` may supersede.
|
||||||
|
fn siblings(&self, file_id: u64, model_id: &str) -> Vec<String> {
|
||||||
|
let mut stmt = match self.index.prepare(&format!(
|
||||||
|
"SELECT model_id FROM entries
|
||||||
|
WHERE file_id = ?1 AND model_id != ?2 AND {} = ?3",
|
||||||
|
crate::faces::embedder_sql("model_id")
|
||||||
|
)) {
|
||||||
|
Ok(s) => s,
|
||||||
|
Err(_) => return Vec::new(),
|
||||||
|
};
|
||||||
|
stmt.query_map(
|
||||||
|
rusqlite::params![
|
||||||
|
file_id as i64,
|
||||||
|
model_id,
|
||||||
|
crate::faces::embedder_of(model_id)
|
||||||
|
],
|
||||||
|
|r| r.get::<_, String>(0),
|
||||||
|
)
|
||||||
|
.map(|rows| rows.filter_map(|r| r.ok()).collect())
|
||||||
|
.unwrap_or_default()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whether a pass this file is already held under outranks `model_id`,
|
||||||
|
/// so a put of `model_id` would add a generation nobody would adopt.
|
||||||
|
pub fn outranked(&self, file_id: u64, model_id: &str) -> bool {
|
||||||
|
use dr_types::FaceDetector;
|
||||||
|
let Some(incoming) = FaceDetector::for_model_id(model_id) else {
|
||||||
|
return false;
|
||||||
|
};
|
||||||
|
self.siblings(file_id, model_id)
|
||||||
|
.iter()
|
||||||
|
.filter_map(|m| FaceDetector::for_model_id(m))
|
||||||
|
.any(|held| held.outranks(incoming))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Forget the index entries for generations of this file that `model_id`
|
||||||
|
/// outranks. The bytes stay where they are — a sealed shard is
|
||||||
|
/// immutable — but the store stops offering them, and a later export or
|
||||||
|
/// merge writes nothing for them again.
|
||||||
|
fn supersede(&self, file_id: u64, model_id: &str) -> Result<(), CatalogError> {
|
||||||
|
use dr_types::FaceDetector;
|
||||||
|
let Some(incoming) = FaceDetector::for_model_id(model_id) else {
|
||||||
|
return Ok(());
|
||||||
|
};
|
||||||
|
for held in self.siblings(file_id, model_id) {
|
||||||
|
let weaker = FaceDetector::for_model_id(&held).is_some_and(|h| incoming.outranks(h));
|
||||||
|
if weaker {
|
||||||
|
self.index.execute(
|
||||||
|
"DELETE FROM entries WHERE file_id = ?1 AND model_id = ?2",
|
||||||
|
rusqlite::params![file_id as i64, held],
|
||||||
|
)?;
|
||||||
|
self.index.execute(
|
||||||
|
"DELETE FROM faces_meta WHERE file_id = ?1 AND model_id = ?2",
|
||||||
|
rusqlite::params![file_id as i64, held],
|
||||||
|
)?;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
pub fn contains(&self, file_id: u64, model_id: &str) -> bool {
|
pub fn contains(&self, file_id: u64, model_id: &str) -> bool {
|
||||||
self.index
|
self.index
|
||||||
.query_row(
|
.query_row(
|
||||||
@@ -233,6 +294,15 @@ impl FaceShardStore {
|
|||||||
faces: &[SharedFace],
|
faces: &[SharedFace],
|
||||||
indexed_at: Option<i64>,
|
indexed_at: Option<i64>,
|
||||||
) -> Result<u32, CatalogError> {
|
) -> Result<u32, CatalogError> {
|
||||||
|
// One generation per image per embedder. A store carried every pass
|
||||||
|
// — 24,123 entries for 19,089 images on the reference library, a
|
||||||
|
// third of its 293 MB — and only the strongest was ever adopted.
|
||||||
|
// A weaker pass arriving after a stronger one is not written; a
|
||||||
|
// stronger one arriving retires the weaker from the index.
|
||||||
|
if self.outranked(file_id, model_id) {
|
||||||
|
return Ok(0);
|
||||||
|
}
|
||||||
|
self.supersede(file_id, model_id)?;
|
||||||
let incoming = faces
|
let incoming = faces
|
||||||
.iter()
|
.iter()
|
||||||
.map(|f| BYTES_PER_FACE + if f.crop.is_empty() { 0 } else { BYTES_PER_CROP })
|
.map(|f| BYTES_PER_FACE + if f.crop.is_empty() { 0 } else { BYTES_PER_CROP })
|
||||||
@@ -474,15 +544,28 @@ impl FaceShardStore {
|
|||||||
rusqlite::OpenFlags::SQLITE_OPEN_READ_ONLY | rusqlite::OpenFlags::SQLITE_OPEN_NO_MUTEX,
|
rusqlite::OpenFlags::SQLITE_OPEN_READ_ONLY | rusqlite::OpenFlags::SQLITE_OPEN_NO_MUTEX,
|
||||||
)?;
|
)?;
|
||||||
|
|
||||||
let mut q =
|
// The peer's marker travels with the image: it is what lets
|
||||||
src.prepare("SELECT file_id, model_id, faces_found, source_edge FROM indexed")?;
|
// `import_from_shards` record the adoption under the time the peer
|
||||||
let images: Vec<(i64, String, i64, i64)> = q
|
// indexed it, and so what keeps `export_to_shards` from reading the
|
||||||
.query_map([], |r| Ok((r.get(0)?, r.get(1)?, r.get(2)?, r.get(3)?)))?
|
// adoption as a re-index and sending the peer's faces back out under
|
||||||
|
// this device's name. A shard from before the column has none.
|
||||||
|
let mut q = src.prepare(&format!(
|
||||||
|
"SELECT file_id, model_id, faces_found, source_edge, {} FROM indexed",
|
||||||
|
match has_column(&src, "indexed", "indexed_at") {
|
||||||
|
Ok(true) => "indexed_at",
|
||||||
|
_ => "NULL",
|
||||||
|
}
|
||||||
|
))?;
|
||||||
|
let images: Vec<(i64, String, i64, i64, Option<i64>)> = q
|
||||||
|
.query_map([], |r| {
|
||||||
|
Ok((r.get(0)?, r.get(1)?, r.get(2)?, r.get(3)?, r.get(4)?))
|
||||||
|
})?
|
||||||
.collect::<Result<_, _>>()?;
|
.collect::<Result<_, _>>()?;
|
||||||
|
|
||||||
let mut adopted = 0;
|
let mut adopted = 0;
|
||||||
for (file_id, model_id, _found, edge) in images {
|
for (file_id, model_id, _found, edge, indexed_at) in images {
|
||||||
if self.contains(file_id as u64, &model_id) {
|
if self.contains(file_id as u64, &model_id) || self.outranked(file_id as u64, &model_id)
|
||||||
|
{
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
let mut fq = src.prepare(&format!(
|
let mut fq = src.prepare(&format!(
|
||||||
@@ -501,12 +584,27 @@ impl FaceShardStore {
|
|||||||
let faces: Vec<SharedFace> = fq
|
let faces: Vec<SharedFace> = fq
|
||||||
.query_map(rusqlite::params![file_id, &model_id], read_shared_face)?
|
.query_map(rusqlite::params![file_id, &model_id], read_shared_face)?
|
||||||
.collect::<Result<_, _>>()?;
|
.collect::<Result<_, _>>()?;
|
||||||
self.put_image(file_id as u64, &model_id, edge as u32, &faces)?;
|
self.put_image_at(file_id as u64, &model_id, edge as u32, &faces, indexed_at)?;
|
||||||
adopted += 1;
|
adopted += 1;
|
||||||
}
|
}
|
||||||
Ok(adopted)
|
Ok(adopted)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Record when the catalog indexed a held image, for an entry that
|
||||||
|
/// arrived without a marker — a peer's shard from before the column.
|
||||||
|
pub fn set_indexed_at(
|
||||||
|
&self,
|
||||||
|
file_id: u64,
|
||||||
|
model_id: &str,
|
||||||
|
at: i64,
|
||||||
|
) -> Result<(), CatalogError> {
|
||||||
|
self.index.execute(
|
||||||
|
"UPDATE entries SET indexed_at = ?3 WHERE file_id = ?1 AND model_id = ?2",
|
||||||
|
rusqlite::params![file_id as i64, model_id, at],
|
||||||
|
)?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
/// Read back everything held for one image.
|
/// Read back everything held for one image.
|
||||||
pub fn get_image(
|
pub fn get_image(
|
||||||
&self,
|
&self,
|
||||||
@@ -798,8 +896,21 @@ pub fn import_from_shards(
|
|||||||
})?
|
})?
|
||||||
.collect::<Result<_, _>>()?;
|
.collect::<Result<_, _>>()?;
|
||||||
|
|
||||||
|
/// Images per write transaction. Large enough that fourteen thousand
|
||||||
|
/// adoptions are a hundred and forty commits rather than fourteen
|
||||||
|
/// thousand; small enough that a read on the UI thread, queued behind
|
||||||
|
/// the lock, waits a fraction of a second and not the whole import.
|
||||||
|
const CHUNK: usize = 100;
|
||||||
|
|
||||||
let mut adopted = 0;
|
let mut adopted = 0;
|
||||||
|
let mut tx = conn.unchecked_transaction()?;
|
||||||
|
let mut in_chunk = 0;
|
||||||
for (file_id, image_id, local) in candidates {
|
for (file_id, image_id, local) in candidates {
|
||||||
|
if in_chunk == CHUNK {
|
||||||
|
tx.commit()?;
|
||||||
|
tx = conn.unchecked_transaction()?;
|
||||||
|
in_chunk = 0;
|
||||||
|
}
|
||||||
let Some(held) = store.held_model(file_id as u64, model_id) else {
|
let Some(held) = store.held_model(file_id as u64, model_id) else {
|
||||||
continue;
|
continue;
|
||||||
};
|
};
|
||||||
@@ -858,15 +969,42 @@ pub fn import_from_shards(
|
|||||||
})
|
})
|
||||||
.collect();
|
.collect();
|
||||||
|
|
||||||
crate::faces::record_detections(
|
crate::faces::record_detections_within(
|
||||||
conn,
|
&tx,
|
||||||
dr_types::ImageId(image_id as u64),
|
dr_types::ImageId(image_id as u64),
|
||||||
&held,
|
&held,
|
||||||
edge,
|
edge,
|
||||||
&local,
|
&local,
|
||||||
)?;
|
)?;
|
||||||
|
// The peer's marker, not this moment. `record_detections` stamps the
|
||||||
|
// run as now, and `export_to_shards` reads a marker newer than the
|
||||||
|
// shard's as a re-index — so every adopted image went straight back
|
||||||
|
// out as this device's own work: 14,100 adopted, 15,457 "newly
|
||||||
|
// indexed" on the next pass, and twenty-two shards of a peer's faces
|
||||||
|
// uploaded again under a second name. Where the peer's shard carried
|
||||||
|
// no marker, the store takes the catalog's, so the two agree either
|
||||||
|
// way and the export sees nothing to send.
|
||||||
|
match store.indexed_at(file_id as u64, &held) {
|
||||||
|
Some(theirs) => {
|
||||||
|
tx.execute(
|
||||||
|
"UPDATE face_index SET indexed_at = ?3
|
||||||
|
WHERE image_id = ?1 AND model_id = ?2",
|
||||||
|
rusqlite::params![image_id, held, theirs],
|
||||||
|
)?;
|
||||||
|
}
|
||||||
|
None => {
|
||||||
|
let ours: i64 = tx.query_row(
|
||||||
|
"SELECT indexed_at FROM face_index WHERE image_id = ?1 AND model_id = ?2",
|
||||||
|
rusqlite::params![image_id, held],
|
||||||
|
|r| r.get(0),
|
||||||
|
)?;
|
||||||
|
store.set_indexed_at(file_id as u64, &held, ours)?;
|
||||||
|
}
|
||||||
|
}
|
||||||
adopted += 1;
|
adopted += 1;
|
||||||
|
in_chunk += 1;
|
||||||
}
|
}
|
||||||
|
tx.commit()?;
|
||||||
Ok(adopted)
|
Ok(adopted)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1102,6 +1240,32 @@ mod tests {
|
|||||||
assert!(!s.contains(1, "lvface"));
|
assert!(!s.contains(1, "lvface"));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// One generation per image per embedder: a stronger detector's pass
|
||||||
|
/// retires a weaker one from the index, and a weaker pass arriving after
|
||||||
|
/// a stronger is not written at all.
|
||||||
|
#[test]
|
||||||
|
fn a_stronger_pass_retires_a_weaker_one_and_a_weaker_is_not_added() {
|
||||||
|
let dir = tempdir();
|
||||||
|
let mut s = FaceShardStore::open(&dir).unwrap();
|
||||||
|
s.put_image(1, "w600k_mbf", 1024, &[face(1, 1)]).unwrap();
|
||||||
|
s.put_image(1, "scrfd_10g+w600k_mbf", 1024, &[face(1, 2)])
|
||||||
|
.unwrap();
|
||||||
|
assert!(s.contains(1, "scrfd_10g+w600k_mbf"));
|
||||||
|
assert!(!s.contains(1, "w600k_mbf"), "the fast pass was not retired");
|
||||||
|
assert_eq!(s.len(), 1, "faces_meta still counts the retired pass");
|
||||||
|
|
||||||
|
s.put_image(1, "scrfd_2.5g+w600k_mbf", 1024, &[face(1, 3)])
|
||||||
|
.unwrap();
|
||||||
|
assert!(
|
||||||
|
!s.contains(1, "scrfd_2.5g+w600k_mbf"),
|
||||||
|
"a weaker pass was added"
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
s.held_model(1, "w600k_mbf").as_deref(),
|
||||||
|
Some("scrfd_10g+w600k_mbf")
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn re_storing_an_image_replaces_rather_than_doubling_it() {
|
fn re_storing_an_image_replaces_rather_than_doubling_it() {
|
||||||
let dir = tempdir();
|
let dir = tempdir();
|
||||||
@@ -1322,6 +1486,11 @@ mod catalog_round_trip {
|
|||||||
assert!((got[0].landmarks[2].0 - 0.15).abs() < 1e-5);
|
assert!((got[0].landmarks[2].0 - 0.15).abs() < 1e-5);
|
||||||
let emb = faces::embeddings(&b, "w600k_mbf").unwrap();
|
let emb = faces::embeddings(&b, "w600k_mbf").unwrap();
|
||||||
assert!(emb.iter().any(|e| e.embedding[0] == 1));
|
assert!(emb.iter().any(|e| e.embedding[0] == 1));
|
||||||
|
|
||||||
|
// And what B adopted is not B's work: its next export sends nothing.
|
||||||
|
// Adopting used to stamp the run as now, so every adopted image went
|
||||||
|
// back out under B's name as a re-index.
|
||||||
|
assert_eq!(export_to_shards(&b, &mut store_b, "w600k_mbf").unwrap(), 0);
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The desktop switched to a stronger detector part-way through the
|
/// The desktop switched to a stronger detector part-way through the
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
//! TRACES: FR-CULL-8 | FR-CULL-9 | FR-CULL-10 | FR-CULL-11 | FR-CULL-12 | NFR-SEC-5
|
//! TRACES: FR-CULL-8 | FR-CULL-9 | FR-CULL-10 | FR-CULL-11 | FR-CULL-12 | NFR-SEC-5
|
||||||
//! People and faces: what was detected, who it is, and who said so.
|
//! People and faces: what was detected, who it is, and who said so.
|
||||||
//!
|
//!
|
||||||
//! The storage half of docs/faces.md. `dr-face` finds faces and turns them into
|
//! The storage half of docs/dev/faces.md. `dr-face` finds faces and turns them into
|
||||||
//! 512 numbers; this module is where those numbers acquire an identity, and
|
//! 512 numbers; this module is where those numbers acquire an identity, and
|
||||||
//! where the user's corrections outrank the model's guesses.
|
//! where the user's corrections outrank the model's guesses.
|
||||||
//!
|
//!
|
||||||
@@ -110,7 +110,7 @@ pub struct DetectedFace {
|
|||||||
/// Raw rather than unit length, so the length ([`Self::quality`]) is in
|
/// Raw rather than unit length, so the length ([`Self::quality`]) is in
|
||||||
/// the blob and not only beside it. Readers re-normalise on load.
|
/// the blob and not only beside it. Readers re-normalise on load.
|
||||||
pub embedding: Vec<u8>,
|
pub embedding: Vec<u8>,
|
||||||
/// Source pixels across the aligned crop (docs/faces.md §7).
|
/// Source pixels across the aligned crop (docs/dev/faces.md §7).
|
||||||
pub crop_px: f32,
|
pub crop_px: f32,
|
||||||
/// Length of the raw embedding before normalisation — the model's own
|
/// Length of the raw embedding before normalisation — the model's own
|
||||||
/// reading of how recognisable the crop was, and the gate on whether
|
/// reading of how recognisable the crop was, and the gate on whether
|
||||||
@@ -211,7 +211,7 @@ pub use dr_face::Calibration;
|
|||||||
/// the same face in the same photograph, for carrying an identity across a
|
/// the same face in the same photograph, for carrying an identity across a
|
||||||
/// re-detection.
|
/// re-detection.
|
||||||
///
|
///
|
||||||
/// Set at the reference library's P≈0.95 line (docs/faces.md §9's table:
|
/// Set at the reference library's P≈0.95 line (docs/dev/faces.md §9's table:
|
||||||
/// 0.449), which is far above anything two different people in one frame
|
/// 0.449), which is far above anything two different people in one frame
|
||||||
/// reach and below what one face re-embedded from a better crop of itself
|
/// reach and below what one face re-embedded from a better crop of itself
|
||||||
/// does. The number is only ever asked about *overlapping* boxes on *one*
|
/// does. The number is only ever asked about *overlapping* boxes on *one*
|
||||||
@@ -279,7 +279,25 @@ pub fn record_detections(
|
|||||||
faces: &[DetectedFace],
|
faces: &[DetectedFace],
|
||||||
) -> Result<Vec<FaceId>, CatalogError> {
|
) -> Result<Vec<FaceId>, CatalogError> {
|
||||||
let tx = conn.unchecked_transaction()?;
|
let tx = conn.unchecked_transaction()?;
|
||||||
|
let ids = record_detections_within(&tx, image_id, model_id, source_edge, faces)?;
|
||||||
|
tx.commit()?;
|
||||||
|
Ok(ids)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// [`record_detections`] inside a transaction the caller owns.
|
||||||
|
///
|
||||||
|
/// For a caller recording many images at once — the shard import adopts
|
||||||
|
/// fourteen thousand in one pass — where a commit per image is fourteen
|
||||||
|
/// thousand fsyncs and fourteen thousand turns at the write lock that every
|
||||||
|
/// read on the UI thread queues behind. `unchecked_transaction` cannot nest,
|
||||||
|
/// so the batching has to be offered here rather than wrapped from above.
|
||||||
|
pub fn record_detections_within(
|
||||||
|
tx: &Connection,
|
||||||
|
image_id: ImageId,
|
||||||
|
model_id: &str,
|
||||||
|
source_edge: u32,
|
||||||
|
faces: &[DetectedFace],
|
||||||
|
) -> Result<Vec<FaceId>, CatalogError> {
|
||||||
// Everything the old faces knew, so it can be carried across the
|
// Everything the old faces knew, so it can be carried across the
|
||||||
// replacement. Read only when there is something to carry it onto: a
|
// replacement. Read only when there is something to carry it onto: a
|
||||||
// pass that found nothing has nothing to match, and decoding a vector
|
// pass that found nothing has nothing to match, and decoding a vector
|
||||||
@@ -287,7 +305,7 @@ pub fn record_detections(
|
|||||||
let prior = if faces.is_empty() {
|
let prior = if faces.is_empty() {
|
||||||
Vec::new()
|
Vec::new()
|
||||||
} else {
|
} else {
|
||||||
read_priors(&tx, image_id)?
|
read_priors(tx, image_id)?
|
||||||
};
|
};
|
||||||
|
|
||||||
tx.execute("DELETE FROM faces WHERE image_id = ?1", [image_id.0 as i64])?;
|
tx.execute("DELETE FROM faces WHERE image_id = ?1", [image_id.0 as i64])?;
|
||||||
@@ -389,7 +407,6 @@ pub fn record_detections(
|
|||||||
],
|
],
|
||||||
)?;
|
)?;
|
||||||
|
|
||||||
tx.commit()?;
|
|
||||||
Ok(ids)
|
Ok(ids)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -2469,7 +2486,7 @@ mod tests {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/// The reference implementation's fitted MBF curve puts the P=0.5 boundary
|
/// The reference implementation's fitted MBF curve puts the P=0.5 boundary
|
||||||
/// at cosine 0.267 (docs/faces.md §1). Our own first end-to-end run scored
|
/// at cosine 0.267 (docs/dev/faces.md §1). Our own first end-to-end run scored
|
||||||
/// 0.596 between distinct photographs of one person and 0.05 between
|
/// 0.596 between distinct photographs of one person and 0.05 between
|
||||||
/// different people, so those two must land either side.
|
/// different people, so those two must land either side.
|
||||||
#[test]
|
#[test]
|
||||||
|
|||||||
@@ -119,6 +119,8 @@ pub struct MergeReport {
|
|||||||
pub keywords_fused: usize,
|
pub keywords_fused: usize,
|
||||||
/// Keyword assignments taken from the remote.
|
/// Keyword assignments taken from the remote.
|
||||||
pub keywords_assigned: usize,
|
pub keywords_assigned: usize,
|
||||||
|
/// Images whose capture metadata was taken from the remote.
|
||||||
|
pub metadata_adopted: usize,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl MergeReport {
|
impl MergeReport {
|
||||||
@@ -133,6 +135,7 @@ impl MergeReport {
|
|||||||
|| self.keywords_deleted > 0
|
|| self.keywords_deleted > 0
|
||||||
|| self.keywords_fused > 0
|
|| self.keywords_fused > 0
|
||||||
|| self.keywords_assigned > 0
|
|| self.keywords_assigned > 0
|
||||||
|
|| self.metadata_adopted > 0
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Whether the local catalog holds anything the remote did not, and so
|
/// Whether the local catalog holds anything the remote did not, and so
|
||||||
@@ -193,10 +196,67 @@ pub fn merge_all(conn: &Connection) -> Result<MergeReport, CatalogError> {
|
|||||||
merge_collections_within(&tx, &mut report)?;
|
merge_collections_within(&tx, &mut report)?;
|
||||||
merge_keywords_within(&tx, &mut report)?;
|
merge_keywords_within(&tx, &mut report)?;
|
||||||
merge_people_within(&tx, &mut report)?;
|
merge_people_within(&tx, &mut report)?;
|
||||||
|
merge_metadata_within(&tx, &mut report)?;
|
||||||
tx.commit()?;
|
tx.commit()?;
|
||||||
Ok(report)
|
Ok(report)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Adopt capture metadata from an attached catalog, on its own.
|
||||||
|
pub fn merge_metadata(conn: &Connection) -> Result<MergeReport, CatalogError> {
|
||||||
|
let tx = conn.unchecked_transaction()?;
|
||||||
|
let mut report = MergeReport::default();
|
||||||
|
merge_metadata_within(&tx, &mut report)?;
|
||||||
|
tx.commit()?;
|
||||||
|
Ok(report)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Capture metadata a peer's sweep already read, for images this device has
|
||||||
|
/// not dated yet.
|
||||||
|
///
|
||||||
|
/// The `images` table is local state and the merge leaves it alone — except
|
||||||
|
/// for these columns, which are not: a capture time, an offset, a camera, a
|
||||||
|
/// lens and an ISO are facts about the file's bytes, identical on every
|
||||||
|
/// device, and read by fetching a header per image across the whole library
|
||||||
|
/// (`dr_ui::library::spawn_sweep`). A fresh device inherits its peers'
|
||||||
|
/// thumbnails and faces from the shards and then spent hours re-reading
|
||||||
|
/// every header for the timeline; the snapshot it had just merged held
|
||||||
|
/// every one of those dates.
|
||||||
|
///
|
||||||
|
/// Matched by `oc:fileid`, as collection membership is. Only rows still at
|
||||||
|
/// `metadata_state < 2` take anything, and only from a remote row at 2: a
|
||||||
|
/// date this device read for itself is never overwritten, and a peer that
|
||||||
|
/// has not read one has nothing to give. The sweep's own query
|
||||||
|
/// (`metadata_state < 2`) then finds nothing left to do for them.
|
||||||
|
const METADATA_BY_FILE_ID: &str = "
|
||||||
|
UPDATE main.images
|
||||||
|
SET captured_at = r.captured_at,
|
||||||
|
captured_offset = coalesce(main.images.captured_offset, r.captured_offset),
|
||||||
|
camera = coalesce(main.images.camera, r.camera),
|
||||||
|
lens = coalesce(main.images.lens, r.lens),
|
||||||
|
iso = coalesce(main.images.iso, r.iso),
|
||||||
|
metadata_state = 2
|
||||||
|
FROM (SELECT lr.image_id, ri.captured_at, ri.captured_offset,
|
||||||
|
ri.camera, ri.lens, ri.iso
|
||||||
|
FROM remote_cat.images ri
|
||||||
|
JOIN remote_cat.remote rr ON rr.image_id = ri.id
|
||||||
|
JOIN main.remote lr ON lr.file_id = rr.file_id
|
||||||
|
WHERE ri.metadata_state >= 2 AND ri.captured_at IS NOT NULL) AS r
|
||||||
|
WHERE main.images.id = r.image_id
|
||||||
|
AND main.images.metadata_state < 2";
|
||||||
|
|
||||||
|
fn merge_metadata_within(tx: &Connection, report: &mut MergeReport) -> Result<(), CatalogError> {
|
||||||
|
// A snapshot from before these columns, or from a library with no server
|
||||||
|
// behind it, has nothing to join on.
|
||||||
|
if !remote_has(tx, "remote")?
|
||||||
|
|| !remote_has_column(tx, "images", "metadata_state")?
|
||||||
|
|| !remote_has_column(tx, "images", "captured_offset")?
|
||||||
|
{
|
||||||
|
return Ok(());
|
||||||
|
}
|
||||||
|
report.metadata_adopted = tx.execute(METADATA_BY_FILE_ID, [])?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
/// Merge people and identity judgements from an attached catalog.
|
/// Merge people and identity judgements from an attached catalog.
|
||||||
///
|
///
|
||||||
/// The people half of [`merge_all`], on its own, for the same reason the other
|
/// The people half of [`merge_all`], on its own, for the same reason the other
|
||||||
@@ -729,7 +789,7 @@ fn attached_has_table(conn: &Connection, schema: &str, table: &str) -> Result<bo
|
|||||||
/// # What travels, and what is recomputed
|
/// # What travels, and what is recomputed
|
||||||
///
|
///
|
||||||
/// The rule this module already follows for the rest of the catalog: user
|
/// The rule this module already follows for the rest of the catalog: user
|
||||||
/// judgements travel, inference is rebuilt. Concretely (docs/faces.md, and the
|
/// judgements travel, inference is rebuilt. Concretely (docs/dev/faces.md, and the
|
||||||
/// asymmetry `crate::faces` opens with):
|
/// asymmetry `crate::faces` opens with):
|
||||||
///
|
///
|
||||||
/// - **People** — uuid, name, and whether the user set them aside. Merged by
|
/// - **People** — uuid, name, and whether the user set them aside. Merged by
|
||||||
@@ -1168,6 +1228,57 @@ mod tests {
|
|||||||
|
|
||||||
// ---- integration over two real catalogs ------------------------------
|
// ---- integration over two real catalogs ------------------------------
|
||||||
|
|
||||||
|
/// A fresh device takes the capture dates a peer's sweep read, matched by
|
||||||
|
/// `oc:fileid`, and never overwrites a date it read for itself.
|
||||||
|
#[test]
|
||||||
|
fn capture_metadata_arrives_for_undated_images_only() {
|
||||||
|
let c = two_catalogs();
|
||||||
|
// Three photographs on both devices: 1 undated here and dated there;
|
||||||
|
// 2 dated on both, differently; 3 undated on both.
|
||||||
|
for id in 1..=3 {
|
||||||
|
add_image_without_hash(&c, "main", id);
|
||||||
|
add_image_without_hash(&c, "remote_cat", id + 10);
|
||||||
|
add_remote_id(&c, "main", id, 100 + id);
|
||||||
|
add_remote_id(&c, "remote_cat", id + 10, 100 + id);
|
||||||
|
}
|
||||||
|
c.execute(
|
||||||
|
"UPDATE remote_cat.images
|
||||||
|
SET captured_at = 1000, captured_offset = 60, camera = 'X', metadata_state = 2
|
||||||
|
WHERE id = 11",
|
||||||
|
[],
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
c.execute(
|
||||||
|
"UPDATE remote_cat.images SET captured_at = 2000, metadata_state = 2 WHERE id = 12",
|
||||||
|
[],
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
c.execute(
|
||||||
|
"UPDATE main.images SET captured_at = 2222, metadata_state = 2 WHERE id = 2",
|
||||||
|
[],
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
|
||||||
|
let report = merge_metadata(&c).unwrap();
|
||||||
|
assert_eq!(report.metadata_adopted, 1);
|
||||||
|
|
||||||
|
let row = |id: i64| -> (Option<i64>, Option<i64>, Option<String>, i64) {
|
||||||
|
c.query_row(
|
||||||
|
"SELECT captured_at, captured_offset, camera, metadata_state
|
||||||
|
FROM main.images WHERE id = ?1",
|
||||||
|
[id],
|
||||||
|
|r| Ok((r.get(0)?, r.get(1)?, r.get(2)?, r.get(3)?)),
|
||||||
|
)
|
||||||
|
.unwrap()
|
||||||
|
};
|
||||||
|
assert_eq!(row(1), (Some(1000), Some(60), Some("X".into()), 2));
|
||||||
|
assert_eq!(row(2), (Some(2222), None, None, 2));
|
||||||
|
assert_eq!(row(3), (None, None, None, 0));
|
||||||
|
|
||||||
|
// Idempotent: a second pass finds nothing left to take.
|
||||||
|
assert_eq!(merge_metadata(&c).unwrap().metadata_adopted, 0);
|
||||||
|
}
|
||||||
|
|
||||||
fn two_catalogs() -> Connection {
|
fn two_catalogs() -> Connection {
|
||||||
attached_remote(schema::for_attached("remote_cat"))
|
attached_remote(schema::for_attached("remote_cat"))
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -16,7 +16,7 @@
|
|||||||
//! # The one thing a rebuild does not recover
|
//! # The one thing a rebuild does not recover
|
||||||
//!
|
//!
|
||||||
//! **Collections.** A manual collection is a set of images the user assembled
|
//! **Collections.** A manual collection is a set of images the user assembled
|
||||||
//! by hand and nothing in the filesystem records it (`docs/catalog.md` §8.1) —
|
//! by hand and nothing in the filesystem records it (`docs/dev/catalog.md` §8.1) —
|
||||||
//! which is the whole reason the catalog file itself syncs. So the two offers
|
//! which is the whole reason the catalog file itself syncs. So the two offers
|
||||||
//! are not interchangeable, and the interface must not present them as if they
|
//! are not interchangeable, and the interface must not present them as if they
|
||||||
//! were: a restore keeps the user's collections, a rebuild does not.
|
//! were: a restore keeps the user's collections, a rebuild does not.
|
||||||
@@ -570,7 +570,7 @@ mod tests {
|
|||||||
// The first NFR-R6 branch, asserted on the thing that distinguishes it
|
// The first NFR-R6 branch, asserted on the thing that distinguishes it
|
||||||
// from the second: a collection exists nowhere but the catalog, so it
|
// from the second: a collection exists nowhere but the catalog, so it
|
||||||
// is the evidence that the *contents* came back and not merely a
|
// is the evidence that the *contents* came back and not merely a
|
||||||
// readable file (docs/catalog.md §8.1).
|
// readable file (docs/dev/catalog.md §8.1).
|
||||||
let dir = tempdir("restore");
|
let dir = tempdir("restore");
|
||||||
let path = dir.join("catalog.sqlite");
|
let path = dir.join("catalog.sqlite");
|
||||||
fixture(&path, 500);
|
fixture(&path, 500);
|
||||||
|
|||||||
@@ -1009,7 +1009,7 @@ CREATE INDEX face_index_model ON face_index(model_id);
|
|||||||
|
|
||||||
const V8: &str = r#"
|
const V8: &str = r#"
|
||||||
-- TRACES: FR-CULL-8 | FR-CULL-9 | FR-CULL-10 | FR-CULL-11 | FR-CULL-12 | NFR-SEC-5
|
-- TRACES: FR-CULL-8 | FR-CULL-9 | FR-CULL-10 | FR-CULL-11 | FR-CULL-12 | NFR-SEC-5
|
||||||
-- People and faces (docs/faces.md, docs/catalog.md §10).
|
-- People and faces (docs/dev/faces.md, docs/dev/catalog.md §10).
|
||||||
--
|
--
|
||||||
-- Everything here is **derived data** except one column. Faces, landmarks,
|
-- Everything here is **derived data** except one column. Faces, landmarks,
|
||||||
-- embeddings, cluster assignments and suggestions are all reproducible by
|
-- embeddings, cluster assignments and suggestions are all reproducible by
|
||||||
@@ -1046,7 +1046,7 @@ CREATE TABLE faces (
|
|||||||
landmarks BLOB NOT NULL, -- 5 x (x, y) f32, normalised likewise
|
landmarks BLOB NOT NULL, -- 5 x (x, y) f32, normalised likewise
|
||||||
detector_confidence REAL NOT NULL,
|
detector_confidence REAL NOT NULL,
|
||||||
embedding BLOB NOT NULL, -- 512 x f16; unit length until V14, raw since
|
embedding BLOB NOT NULL, -- 512 x f16; unit length until V14, raw since
|
||||||
-- Source pixels across the aligned 112x112 crop (docs/faces.md §7).
|
-- Source pixels across the aligned 112x112 crop (docs/dev/faces.md §7).
|
||||||
--
|
--
|
||||||
-- Not cosmetic: it is the honest quality signal for the UI, a feature in
|
-- Not cosmetic: it is the honest quality signal for the UI, a feature in
|
||||||
-- the §8 calibration -- FR-CULL-9 names face size as an axis along which an
|
-- the §8 calibration -- FR-CULL-9 names face size as an axis along which an
|
||||||
|
|||||||
@@ -11,7 +11,7 @@ log.workspace = true
|
|||||||
|
|
||||||
# Inference. `ort` is the API; **what runs it is `dr-inference-engine`'s
|
# Inference. `ort` is the API; **what runs it is `dr-inference-engine`'s
|
||||||
# business** — tract, or an ONNX Runtime the app found on disk, on whichever
|
# business** — tract, or an ONNX Runtime the app found on disk, on whichever
|
||||||
# provider the device has (docs/inference.md). This crate never names either.
|
# provider the device has (docs/dev/inference.md). This crate never names either.
|
||||||
ort = { workspace = true, optional = true }
|
ort = { workspace = true, optional = true }
|
||||||
dr-inference-engine = { workspace = true, optional = true }
|
dr-inference-engine = { workspace = true, optional = true }
|
||||||
ndarray = { workspace = true, optional = true }
|
ndarray = { workspace = true, optional = true }
|
||||||
@@ -37,7 +37,7 @@ required-features = ["inference"]
|
|||||||
|
|
||||||
[features]
|
[features]
|
||||||
# Nothing on by default, and in particular **no `embedded-model`**: the weights
|
# Nothing on by default, and in particular **no `embedded-model`**: the weights
|
||||||
# are not a build input and never become one (docs/faces.md §2.2). A feature
|
# are not a build input and never become one (docs/dev/faces.md §2.2). A feature
|
||||||
# flag that *could* embed them is a flag someone eventually sets in a packaging
|
# flag that *could* embed them is a flag someone eventually sets in a packaging
|
||||||
# script, and the InsightFace grant does not survive that.
|
# script, and the InsightFace grant does not survive that.
|
||||||
default = []
|
default = []
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
//! Detect the faces in a JPEG and read each one's eyes (docs/faces.md §17).
|
//! Detect the faces in a JPEG and read each one's eyes (docs/dev/faces.md §17).
|
||||||
//!
|
//!
|
||||||
//! The thing worth looking at is whether the eye boxes land on eyes and
|
//! The thing worth looking at is whether the eye boxes land on eyes and
|
||||||
//! whether soft ones are refused — so with `--dump DIR` the crops the
|
//! whether soft ones are refused — so with `--dump DIR` the crops the
|
||||||
|
|||||||
@@ -7,7 +7,7 @@
|
|||||||
//! DET.onnx EMB.onnx photo.jpg [photo.jpg ...]
|
//! DET.onnx EMB.onnx photo.jpg [photo.jpg ...]
|
||||||
//!
|
//!
|
||||||
//! The models must have had their input dims frozen first; see
|
//! The models must have had their input dims frozen first; see
|
||||||
//! `tools/fix-face-model-shapes.sh` and docs/faces.md §12 M1.
|
//! `tools/fix-face-model-shapes.sh` and docs/dev/faces.md §12 M1.
|
||||||
|
|
||||||
use std::time::Instant;
|
use std::time::Instant;
|
||||||
|
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
//! M1 (docs/faces.md §12) — will tract load these graphs at all?
|
//! M1 (docs/dev/faces.md §12) — will tract load these graphs at all?
|
||||||
//!
|
//!
|
||||||
//! The one measurement everything else in the face subsystem is conditional
|
//! The one measurement everything else in the face subsystem is conditional
|
||||||
//! on. `det_500m.onnx` has a dynamic H/W input, which is exactly what tract
|
//! on. `det_500m.onnx` has a dynamic H/W input, which is exactly what tract
|
||||||
|
|||||||
@@ -9,7 +9,7 @@
|
|||||||
//!
|
//!
|
||||||
//! # What it is for
|
//! # What it is for
|
||||||
//!
|
//!
|
||||||
//! docs/faces.md §9 has the desktop numbers and the question they leave open:
|
//! docs/dev/faces.md §9 has the desktop numbers and the question they leave open:
|
||||||
//! a GPU GEMM is worth roughly 1.5× of a regroup on a twenty-core desktop,
|
//! a GPU GEMM is worth roughly 1.5× of a regroup on a twenty-core desktop,
|
||||||
//! because the scan is under a third of the pass there. On a tablet the CPU is
|
//! because the scan is under a third of the pass there. On a tablet the CPU is
|
||||||
//! several times slower and the GPU is not, so the same optimisation is worth
|
//! several times slower and the GPU is not, so the same optimisation is worth
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
//! Five-point face alignment (docs/faces.md §5).
|
//! Five-point face alignment (docs/dev/faces.md §5).
|
||||||
//!
|
//!
|
||||||
//! ArcFace embeddings are trained on faces warped to a canonical 112×112
|
//! ArcFace embeddings are trained on faces warped to a canonical 112×112
|
||||||
//! arrangement. Feeding the model a plain bounding-box crop *works* — it
|
//! arrangement. Feeding the model a plain bounding-box crop *works* — it
|
||||||
@@ -208,7 +208,7 @@ impl Similarity {
|
|||||||
///
|
///
|
||||||
/// # Why least squares and not RANSAC
|
/// # Why least squares and not RANSAC
|
||||||
///
|
///
|
||||||
/// The reference C++ implementation (docs/faces.md §1.1) fits this with
|
/// The reference C++ implementation (docs/dev/faces.md §1.1) fits this with
|
||||||
/// OpenCV's `estimateAffinePartial2D` under RANSAC. RANSAC over five points is
|
/// OpenCV's `estimateAffinePartial2D` under RANSAC. RANSAC over five points is
|
||||||
/// a strange fit: the minimal sample for a similarity is two, so it can discard
|
/// a strange fit: the minimal sample for a similarity is two, so it can discard
|
||||||
/// landmarks it judges outliers and solve from a subset — and on a profile face
|
/// landmarks it judges outliers and solve from a subset — and on a profile face
|
||||||
@@ -427,7 +427,7 @@ fn sample_window(
|
|||||||
// ── eyes ──────────────────────────────────────────────────────────────────
|
// ── eyes ──────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
/// Width of an eye crop as the classifier reads it, in pixels. Fixed by the
|
/// Width of an eye crop as the classifier reads it, in pixels. Fixed by the
|
||||||
/// OCEC input (`docs/faces.md` §17): 40 wide, 24 high.
|
/// OCEC input (`docs/dev/faces.md` §17): 40 wide, 24 high.
|
||||||
pub const EYE_PATCH_WIDTH: usize = 40;
|
pub const EYE_PATCH_WIDTH: usize = 40;
|
||||||
/// Height of an eye crop as the classifier reads it, in pixels.
|
/// Height of an eye crop as the classifier reads it, in pixels.
|
||||||
pub const EYE_PATCH_HEIGHT: usize = 24;
|
pub const EYE_PATCH_HEIGHT: usize = 24;
|
||||||
@@ -438,7 +438,7 @@ pub const EYE_PATCH_HEIGHT: usize = 24;
|
|||||||
/// The classifier was trained on a whole-body detector's *eye* boxes — tight
|
/// The classifier was trained on a whole-body detector's *eye* boxes — tight
|
||||||
/// round the palpebral fissure — and measured on 25 open-eyed faces from the
|
/// round the palpebral fissure — and measured on 25 open-eyed faces from the
|
||||||
/// reference library, a tight box is what it wants: 22 of 25 read open at
|
/// reference library, a tight box is what it wants: 22 of 25 read open at
|
||||||
/// 0 and 0.1, 18 at 0.4, 14 at 0.6 (docs/faces.md §17.2). A tenth, so a
|
/// 0 and 0.1, 18 at 0.4, 14 at 0.6 (docs/dev/faces.md §17.2). A tenth, so a
|
||||||
/// contour landing a pixel short of the lashes still holds them.
|
/// contour landing a pixel short of the lashes still holds them.
|
||||||
pub const EYE_BOX_MARGIN: f32 = 0.1;
|
pub const EYE_BOX_MARGIN: f32 = 0.1;
|
||||||
|
|
||||||
@@ -571,7 +571,7 @@ pub const SUNGLASSES_EDGE: usize = 48;
|
|||||||
/// clear glasses, at 0.68. Erring towards "sunglasses" is the safe direction
|
/// clear glasses, at 0.68. Erring towards "sunglasses" is the safe direction
|
||||||
/// for what this feeds: a face called sunglasses is left alone by the
|
/// for what this feeds: a face called sunglasses is left alone by the
|
||||||
/// eyes-open filter, where a pair of sunglasses missed hands the eye
|
/// eyes-open filter, where a pair of sunglasses missed hands the eye
|
||||||
/// classifier a lens to guess at (docs/faces.md §17).
|
/// classifier a lens to guess at (docs/dev/faces.md §17).
|
||||||
pub const SUNGLASSES_WINDOWS: [(f32, f32, f32, f32); 2] =
|
pub const SUNGLASSES_WINDOWS: [(f32, f32, f32, f32); 2] =
|
||||||
[(0.0, 0.0, 112.0, 112.0), (-5.0, -14.0, 122.0, 122.0)];
|
[(0.0, 0.0, 112.0, 112.0), (-5.0, -14.0, 122.0, 122.0)];
|
||||||
|
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
//! Cosine to probability (docs/faces.md §8, FR-CULL-9).
|
//! Cosine to probability (docs/dev/faces.md §8, FR-CULL-9).
|
||||||
//!
|
//!
|
||||||
//! FR-CULL-9 is a hard requirement rather than an implementation detail: no
|
//! FR-CULL-9 is a hard requirement rather than an implementation detail: no
|
||||||
//! code path may threshold a bare cosine, every threshold in the subsystem is
|
//! code path may threshold a bare cosine, every threshold in the subsystem is
|
||||||
@@ -28,7 +28,7 @@
|
|||||||
//! calibration to the belief it was supposed to test — and that is the whole
|
//! calibration to the belief it was supposed to test — and that is the whole
|
||||||
//! of the alternative.
|
//! of the alternative.
|
||||||
//!
|
//!
|
||||||
//! docs/faces.md §8.1 names one more that would cost no labelling at all: two
|
//! docs/dev/faces.md §8.1 names one more that would cost no labelling at all: two
|
||||||
//! faces in adjacent frames of one burst are near-certainly the same person,
|
//! faces in adjacent frames of one burst are near-certainly the same person,
|
||||||
//! and FR-CULL-5's grouping is sitting there. Nothing draws on it. This crate
|
//! and FR-CULL-5's grouping is sitting there. Nothing draws on it. This crate
|
||||||
//! cannot see a catalog, let alone the bursts in one — it is handed cosines by
|
//! cannot see a catalog, let alone the bursts in one — it is handed cosines by
|
||||||
@@ -83,7 +83,7 @@ pub struct Calibration {
|
|||||||
}
|
}
|
||||||
|
|
||||||
impl Default for Calibration {
|
impl Default for Calibration {
|
||||||
/// The reference implementation's fitted MBF curve (docs/faces.md §1):
|
/// The reference implementation's fitted MBF curve (docs/dev/faces.md §1):
|
||||||
/// steepness 16.2, P=0.5 at cosine 0.267.
|
/// steepness 16.2, P=0.5 at cosine 0.267.
|
||||||
///
|
///
|
||||||
/// **`valid` is false**, and that is the point. It is a documented
|
/// **`valid` is false**, and that is the point. It is a documented
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
//! TRACES: FR-CULL-8a
|
//! TRACES: FR-CULL-8a
|
||||||
//! The two small classifiers behind a face's eye state (docs/faces.md §17).
|
//! The two small classifiers behind a face's eye state (docs/dev/faces.md §17).
|
||||||
//!
|
//!
|
||||||
//! **OCEC** — *open closed eyes classification*, Hyodo 2025 — reads one
|
//! **OCEC** — *open closed eyes classification*, Hyodo 2025 — reads one
|
||||||
//! 40×24 eye and answers P(open). **SGC** — *sunglasses classification*,
|
//! 40×24 eye and answers P(open). **SGC** — *sunglasses classification*,
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
//! Grouping faces into people (docs/faces.md §9, FR-CULL-10).
|
//! Grouping faces into people (docs/dev/faces.md §9, FR-CULL-10).
|
||||||
//!
|
//!
|
||||||
//! Model-free: this is arithmetic over embeddings, and it is where the
|
//! Model-free: this is arithmetic over embeddings, and it is where the
|
||||||
//! subsystem's accuracy actually lives, so it is testable with no weights on
|
//! subsystem's accuracy actually lives, so it is testable with no weights on
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
//! SCRFD face detection (docs/faces.md §4).
|
//! SCRFD face detection (docs/dev/faces.md §4).
|
||||||
//!
|
//!
|
||||||
//! One forward pass produces a box, a confidence and **five landmarks** per
|
//! One forward pass produces a box, a confidence and **five landmarks** per
|
||||||
//! face — the landmarks being the reason for this detector rather than a
|
//! face — the landmarks being the reason for this detector rather than a
|
||||||
@@ -138,7 +138,7 @@ impl Detection {
|
|||||||
pub struct Detector {
|
pub struct Detector {
|
||||||
session: Model,
|
session: Model,
|
||||||
/// f32 or int8 — the int8 form finds a different set of faces and is a
|
/// f32 or int8 — the int8 form finds a different set of faces and is a
|
||||||
/// different detector in `model_id` (docs/inference.md §7).
|
/// different detector in `model_id` (docs/dev/inference.md §7).
|
||||||
form: Form,
|
form: Form,
|
||||||
/// Feature-map count: 3 for strides {8,16,32}, 4 for {8,16,32,64}.
|
/// Feature-map count: 3 for strides {8,16,32}, 4 for {8,16,32,64}.
|
||||||
///
|
///
|
||||||
@@ -346,7 +346,7 @@ fn iou(a: &(f32, f32, f32, f32), b: &(f32, f32, f32, f32)) -> f32 {
|
|||||||
/// How the image is fitted into the graph's fixed square input.
|
/// How the image is fitted into the graph's fixed square input.
|
||||||
///
|
///
|
||||||
/// The forward and inverse mappings live in one struct on purpose:
|
/// The forward and inverse mappings live in one struct on purpose:
|
||||||
/// docs/faces.md §4.1 notes that what matters is not *where* the padding goes
|
/// docs/dev/faces.md §4.1 notes that what matters is not *where* the padding goes
|
||||||
/// but that the two agree. A mismatch offsets every box and landmark by the
|
/// but that the two agree. A mismatch offsets every box and landmark by the
|
||||||
/// padding, producing detections that look plausible and embeddings that
|
/// padding, producing detections that look plausible and embeddings that
|
||||||
/// quietly cluster badly three stages later.
|
/// quietly cluster badly three stages later.
|
||||||
@@ -372,7 +372,7 @@ impl Letterbox {
|
|||||||
///
|
///
|
||||||
/// `(x·255 − 127.5) / 128` — note `/128`, not `/127.5`. The reference
|
/// `(x·255 − 127.5) / 128` — note `/128`, not `/127.5`. The reference
|
||||||
/// implementation this is ported from uses `/128` for both models, and
|
/// implementation this is ported from uses `/128` for both models, and
|
||||||
/// every measured number in docs/faces.md §1 came from it.
|
/// every measured number in docs/dev/faces.md §1 came from it.
|
||||||
///
|
///
|
||||||
/// Padding is grey, matching the reference's `114`: the value the network
|
/// Padding is grey, matching the reference's `114`: the value the network
|
||||||
/// reads least as an edge, where black would draw a hard border across the
|
/// reads least as an edge, where black would draw a hard border across the
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
//! ArcFace / MobileFaceNet inference (docs/faces.md §6).
|
//! ArcFace / MobileFaceNet inference (docs/dev/faces.md §6).
|
||||||
//!
|
//!
|
||||||
//! Takes an aligned crop and returns 512 L2-normalised floats. The alignment is
|
//! Takes an aligned crop and returns 512 L2-normalised floats. The alignment is
|
||||||
//! not optional and cannot be skipped by accident: [`Embedder::embed`] takes an
|
//! not optional and cannot be skipped by accident: [`Embedder::embed`] takes an
|
||||||
@@ -66,7 +66,7 @@ impl Embedder {
|
|||||||
|
|
||||||
pub fn from_bytes(bytes: &[u8], model: ModelId) -> Result<Self, FaceError> {
|
pub fn from_bytes(bytes: &[u8], model: ModelId) -> Result<Self, FaceError> {
|
||||||
// Always the f32 form: an embedding must compare across devices
|
// Always the f32 form: an embedding must compare across devices
|
||||||
// (docs/inference.md §7), and the engine pins this role to it.
|
// (docs/dev/inference.md §7), and the engine pins this role to it.
|
||||||
let loaded = dr_inference_engine::open(Role::Embedder, Form::F32, bytes)?;
|
let loaded = dr_inference_engine::open(Role::Embedder, Form::F32, bytes)?;
|
||||||
let acquired = loaded.acquire()?;
|
let acquired = loaded.acquire()?;
|
||||||
let session = acquired.lock();
|
let session = acquired.lock();
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
//! What an embedder produces, and how it is stored (docs/faces.md §6).
|
//! What an embedder produces, and how it is stored (docs/dev/faces.md §6).
|
||||||
//!
|
//!
|
||||||
//! Deliberately **model-free**: the vector, its identity, its comparison and
|
//! Deliberately **model-free**: the vector, its identity, its comparison and
|
||||||
//! its storage encoding are arithmetic, and `calibrate` and `cluster` are built
|
//! its storage encoding are arithmetic, and `calibrate` and `cluster` are built
|
||||||
@@ -273,7 +273,7 @@ mod tests {
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The claim docs/faces.md §6 makes about the storage format: the f16
|
/// The claim docs/dev/faces.md §6 makes about the storage format: the f16
|
||||||
/// round-trip costs ~1e-3 of cosine, three orders below the separation
|
/// round-trip costs ~1e-3 of cosine, three orders below the separation
|
||||||
/// between a match and a non-match.
|
/// between a match and a non-match.
|
||||||
#[test]
|
#[test]
|
||||||
|
|||||||
@@ -67,14 +67,14 @@ pub const SUNGLASSES_THRESHOLD: f32 = 0.5;
|
|||||||
/// The classifier was trained on eyes down to about a dozen pixels wide
|
/// The classifier was trained on eyes down to about a dozen pixels wide
|
||||||
/// (its reference footage averaged 15–21); below that the 40-pixel patch is
|
/// (its reference footage averaged 15–21); below that the 40-pixel patch is
|
||||||
/// an interpolation of nothing, and the answer is noise that reads as
|
/// an interpolation of nothing, and the answer is noise that reads as
|
||||||
/// "closed". docs/faces.md §17.3 has the measurement behind the number.
|
/// "closed". docs/dev/faces.md §17.3 has the measurement behind the number.
|
||||||
pub const MIN_EYE_PX: f32 = 12.0;
|
pub const MIN_EYE_PX: f32 = 12.0;
|
||||||
|
|
||||||
/// Least [`Eye::sharpness`] for the eye to be read.
|
/// Least [`Eye::sharpness`] for the eye to be read.
|
||||||
///
|
///
|
||||||
/// The same measure as the face's `min_sharpness`, over the eye patch, and
|
/// The same measure as the face's `min_sharpness`, over the eye patch, and
|
||||||
/// chosen the same way: the value under which the open-eyed faces of the
|
/// chosen the same way: the value under which the open-eyed faces of the
|
||||||
/// reference sample were being called closed. docs/faces.md §17.3.
|
/// reference sample were being called closed. docs/dev/faces.md §17.3.
|
||||||
pub const MIN_EYE_SHARPNESS: f32 = 0.02;
|
pub const MIN_EYE_SHARPNESS: f32 = 0.02;
|
||||||
|
|
||||||
/// An eye narrower than this fraction of its partner is the far eye of a
|
/// An eye narrower than this fraction of its partner is the far eye of a
|
||||||
@@ -82,7 +82,7 @@ pub const MIN_EYE_SHARPNESS: f32 = 0.02;
|
|||||||
///
|
///
|
||||||
/// A landmark model's contour for a hidden eye collapses towards the nose.
|
/// A landmark model's contour for a hidden eye collapses towards the nose.
|
||||||
/// Measured on twenty native renders of the reference library
|
/// Measured on twenty native renders of the reference library
|
||||||
/// (docs/faces.md §17.4): profiles put the far eye at 0.02–0.43 of the near
|
/// (docs/dev/faces.md §17.4): profiles put the far eye at 0.02–0.43 of the near
|
||||||
/// one, two three-quarter faces whose far eye read closed sat at 0.54, and
|
/// one, two three-quarter faces whose far eye read closed sat at 0.54, and
|
||||||
/// every face looking at the camera — winks included, since a shut eye's
|
/// every face looking at the camera — winks included, since a shut eye's
|
||||||
/// box keeps its width — sat at 0.78 or more. 0.6 splits the gap.
|
/// box keeps its width — sat at 0.78 or more. 0.6 splits the gap.
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
//! TRACES: FR-CULL-8a
|
//! TRACES: FR-CULL-8a
|
||||||
//! Dense facial landmarks — InsightFace's `2d106det` (docs/faces.md §17.2).
|
//! Dense facial landmarks — InsightFace's `2d106det` (docs/dev/faces.md §17.2).
|
||||||
//!
|
//!
|
||||||
//! SCRFD's five points place a face; they do not place an eye. Its eye
|
//! SCRFD's five points place a face; they do not place an eye. Its eye
|
||||||
//! point is loose enough that a window centred on it left the eye in a
|
//! point is loose enough that a window centred on it left the eye in a
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
//! Faces and identity (S14, docs/faces.md).
|
//! Faces and identity (S14, docs/dev/faces.md).
|
||||||
//!
|
//!
|
||||||
//! Two models, run over the native render, producing per face a box, five
|
//! Two models, run over the native render, producing per face a box, five
|
||||||
//! landmarks, a confidence and a 512-d embedding (FR-CULL-8) — and then the
|
//! landmarks, a confidence and a 512-d embedding (FR-CULL-8) — and then the
|
||||||
@@ -21,9 +21,9 @@
|
|||||||
//! for a packaging script to switch on. The application obtains a model at
|
//! for a packaging script to switch on. The application obtains a model at
|
||||||
//! runtime; this crate takes bytes and never fetches anything.
|
//! runtime; this crate takes bytes and never fetches anything.
|
||||||
//!
|
//!
|
||||||
//! docs/faces.md §2 is the full reading, including what would have to change
|
//! docs/dev/faces.md §2 is the full reading, including what would have to change
|
||||||
//! for that to stop being true. The eye-state models are the exception: MIT,
|
//! for that to stop being true. The eye-state models are the exception: MIT,
|
||||||
//! weights and all, and shipped in `models/face/` (docs/faces.md §17).
|
//! weights and all, and shipped in `models/face/` (docs/dev/faces.md §17).
|
||||||
//!
|
//!
|
||||||
//! # Why the runtime is split behind a feature
|
//! # Why the runtime is split behind a feature
|
||||||
//!
|
//!
|
||||||
@@ -50,11 +50,12 @@ pub mod eyes;
|
|||||||
pub mod landmarks;
|
pub mod landmarks;
|
||||||
pub mod naming;
|
pub mod naming;
|
||||||
pub mod neighbours;
|
pub mod neighbours;
|
||||||
|
pub mod references;
|
||||||
|
|
||||||
/// Smallest long edge a face crop may be sampled from.
|
/// Smallest long edge a face crop may be sampled from.
|
||||||
///
|
///
|
||||||
/// **A floor on the crop source, not on the detector input.** The distinction
|
/// **A floor on the crop source, not on the detector input.** The distinction
|
||||||
/// is the whole of FR-CULL-8 and `docs/faces.md` §7: detection letterboxes
|
/// is the whole of FR-CULL-8 and `docs/dev/faces.md` §7: detection letterboxes
|
||||||
/// every buffer into 640×640, so its input resolution decides nothing, while
|
/// every buffer into 640×640, so its input resolution decides nothing, while
|
||||||
/// [`warp`] samples the 112×112 the embedder sees and so converts source
|
/// [`warp`] samples the 112×112 the embedder sees and so converts source
|
||||||
/// resolution directly into embedding quality. FR-CULL-8 requires that crop to
|
/// resolution directly into embedding quality. FR-CULL-8 requires that crop to
|
||||||
|
|||||||
@@ -115,7 +115,7 @@ pub struct Faces<'a> {
|
|||||||
/// Source pixels across the aligned crop, for the calibration's size term.
|
/// Source pixels across the aligned crop, for the calibration's size term.
|
||||||
pub crop_px: &'a [f32],
|
pub crop_px: &'a [f32],
|
||||||
/// Which photograph each face came from. Two faces in one frame are not
|
/// Which photograph each face came from. Two faces in one frame are not
|
||||||
/// the same person, so those pairs are never returned (docs/faces.md §9).
|
/// the same person, so those pairs are never returned (docs/dev/faces.md §9).
|
||||||
pub images: &'a [u64],
|
pub images: &'a [u64],
|
||||||
/// Which faces may be compared *against* — the gallery
|
/// Which faces may be compared *against* — the gallery
|
||||||
/// ([`crate::embedding::MIN_GALLERY_QUALITY`]).
|
/// ([`crate::embedding::MIN_GALLERY_QUALITY`]).
|
||||||
|
|||||||
@@ -0,0 +1,232 @@
|
|||||||
|
//! TRACES: FR-CULL-10 | NFR-P9
|
||||||
|
//! Which of a person's faces stand for them in a grouping pass.
|
||||||
|
//!
|
||||||
|
//! # Why not all of them
|
||||||
|
//!
|
||||||
|
//! Every face the user has ruled on enters [`crate::cluster`] as an anchor,
|
||||||
|
//! and the pass compares every face against every other
|
||||||
|
//! ([`crate::neighbours`] is exhaustive by design). So a person with 750
|
||||||
|
//! confirmed faces costs 750 comparisons against each of the library's other
|
||||||
|
//! faces, and the cost of naming a library well grows with how well it is
|
||||||
|
//! named: a fully confirmed library of 25,000 faces spends almost the whole
|
||||||
|
//! scan re-comparing faces whose identity is already settled against each
|
||||||
|
//! other.
|
||||||
|
//!
|
||||||
|
//! Most of those comparisons say nothing new. A person's confirmed faces are
|
||||||
|
//! heavily redundant — thirty frames from one afternoon are one point of
|
||||||
|
//! view, not thirty — and a new face that matches one of them matches the
|
||||||
|
//! others too. What a new face needs to be measured against is the person's
|
||||||
|
//! *range*: the angles, ages and lights they have been photographed in, each
|
||||||
|
//! represented once.
|
||||||
|
//!
|
||||||
|
//! # The choice: the most diverse of the good ones
|
||||||
|
//!
|
||||||
|
//! Two rules, in order.
|
||||||
|
//!
|
||||||
|
//! **Good enough to vouch.** Only faces whose raw embedding was at least
|
||||||
|
//! [`MIN_REFERENCE_QUALITY`] long are eligible — a stricter floor than the
|
||||||
|
//! gallery's ([`crate::embedding::MIN_GALLERY_QUALITY`]), because a reference
|
||||||
|
//! is asked to speak *for* a person rather than merely be admitted to the
|
||||||
|
//! comparison. A face whose length was never recorded is admitted, as it is
|
||||||
|
//! everywhere else: a rule that cannot be checked admits rather than excludes.
|
||||||
|
//!
|
||||||
|
//! **As far apart as possible.** From the eligible pool, up to
|
||||||
|
//! [`MAX_REFERENCES`] faces are chosen to maximise the volume they span —
|
||||||
|
//! the determinant of their Gram matrix — greedily: start from the longest
|
||||||
|
//! vector, and at each step add the face with the largest component
|
||||||
|
//! orthogonal to everything chosen so far. That is Gram–Schmidt with a
|
||||||
|
//! pivot, and the product of the squared residuals it picks *is* the
|
||||||
|
//! determinant, so the greedy step is the exact greedy on the objective.
|
||||||
|
//! The effect is that a near-duplicate of a chosen face has almost no
|
||||||
|
//! residual and is passed over, while the one profile shot among two
|
||||||
|
//! hundred frontal frames is taken early.
|
||||||
|
//!
|
||||||
|
//! What is not chosen still belongs to the person. Those faces keep their
|
||||||
|
//! confirmations and are not touched by the pass; they are simply not
|
||||||
|
//! compared, which is the whole saving.
|
||||||
|
|
||||||
|
/// The most faces that stand for one person.
|
||||||
|
///
|
||||||
|
/// A hundred is far more points of view than a person has. What it bounds
|
||||||
|
/// is the cost: with every person at the cap, a scan against the named part
|
||||||
|
/// of a library is `people × 100` comparisons per face rather than
|
||||||
|
/// `confirmations`, and the two part company as soon as a library is used.
|
||||||
|
pub const MAX_REFERENCES: usize = 100;
|
||||||
|
|
||||||
|
/// The shortest raw embedding that may stand for a person.
|
||||||
|
///
|
||||||
|
/// One above the gallery floor: a reference vouches for someone, and the
|
||||||
|
/// margin keeps the faces that only just cleared the gallery — the ones
|
||||||
|
/// nearest the middle of the sphere — out of the set that speaks for a
|
||||||
|
/// person.
|
||||||
|
pub const MIN_REFERENCE_QUALITY: f32 = 15.0;
|
||||||
|
|
||||||
|
/// Whether a face of this quality may stand for a person.
|
||||||
|
///
|
||||||
|
/// `None` is "never measured" and is admitted, as in
|
||||||
|
/// [`crate::embedding::in_gallery`].
|
||||||
|
pub fn eligible(quality: Option<f32>) -> bool {
|
||||||
|
quality.is_none_or(|q| q >= MIN_REFERENCE_QUALITY)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Choose which of one person's faces stand for them.
|
||||||
|
///
|
||||||
|
/// `embeddings` and `quality` are one entry per face, the embeddings unit
|
||||||
|
/// length and all of one dimension. Returns the indices chosen, in the order
|
||||||
|
/// chosen — the first is the longest eligible vector, and each after it is
|
||||||
|
/// the one furthest from the span of those before. Every eligible face is
|
||||||
|
/// returned when there are `max` or fewer of them, so a person under the
|
||||||
|
/// cap loses nothing.
|
||||||
|
///
|
||||||
|
/// Deterministic: equal residuals break on the longer vector, then the lower
|
||||||
|
/// index, so two devices holding the same faces choose the same references
|
||||||
|
/// and group the same way (`cluster::clustering_is_deterministic`).
|
||||||
|
pub fn select(embeddings: &[&[f32]], quality: &[Option<f32>], max: usize) -> Vec<usize> {
|
||||||
|
debug_assert_eq!(embeddings.len(), quality.len());
|
||||||
|
let mut pool: Vec<usize> = (0..embeddings.len())
|
||||||
|
.filter(|&i| eligible(quality[i]))
|
||||||
|
.collect();
|
||||||
|
if pool.len() <= max {
|
||||||
|
return pool;
|
||||||
|
}
|
||||||
|
// Longest first, so the seed is the pool's front and a tie on residual
|
||||||
|
// resolves to the earlier position. A missing reading ranks below any
|
||||||
|
// measured one for this purpose only: it is admitted, but a face that
|
||||||
|
// was measured and found long is the better seed.
|
||||||
|
pool.sort_by(|&a, &b| {
|
||||||
|
let qa = quality[a].unwrap_or(0.0);
|
||||||
|
let qb = quality[b].unwrap_or(0.0);
|
||||||
|
qb.total_cmp(&qa).then(a.cmp(&b))
|
||||||
|
});
|
||||||
|
|
||||||
|
// Residuals: what remains of each pool vector outside the span of the
|
||||||
|
// chosen ones. Copied, since they are rewritten in place.
|
||||||
|
let mut residual: Vec<Vec<f32>> = pool.iter().map(|&i| embeddings[i].to_vec()).collect();
|
||||||
|
let mut taken = vec![false; pool.len()];
|
||||||
|
let mut chosen = Vec::with_capacity(max);
|
||||||
|
|
||||||
|
while chosen.len() < max {
|
||||||
|
// The face with the most left outside the span. The seed is the
|
||||||
|
// pool's front by construction: every unit vector has the same
|
||||||
|
// residual before anything is chosen, up to rounding, and rounding
|
||||||
|
// is not a reason to prefer one. After that `> best` and not `>=`,
|
||||||
|
// so a genuine tie keeps the earlier (longer) candidate.
|
||||||
|
let mut pick = None;
|
||||||
|
let mut best = 0.0_f32;
|
||||||
|
if chosen.is_empty() {
|
||||||
|
pick = Some(0);
|
||||||
|
best = residual[0].iter().map(|x| x * x).sum();
|
||||||
|
} else {
|
||||||
|
for (k, r) in residual.iter().enumerate() {
|
||||||
|
if taken[k] {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
let n2: f32 = r.iter().map(|x| x * x).sum();
|
||||||
|
if n2 > best {
|
||||||
|
best = n2;
|
||||||
|
pick = Some(k);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Nothing left outside the span: every remaining face is a
|
||||||
|
// combination of the chosen ones and adds no volume.
|
||||||
|
let Some(k) = pick.filter(|_| best > 1e-6) else {
|
||||||
|
break;
|
||||||
|
};
|
||||||
|
taken[k] = true;
|
||||||
|
chosen.push(pool[k]);
|
||||||
|
|
||||||
|
// Project the chosen direction out of every remaining residual.
|
||||||
|
let inv = best.sqrt().recip();
|
||||||
|
let q: Vec<f32> = residual[k].iter().map(|x| x * inv).collect();
|
||||||
|
for (j, r) in residual.iter_mut().enumerate() {
|
||||||
|
if taken[j] {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
let d: f32 = r.iter().zip(&q).map(|(a, b)| a * b).sum();
|
||||||
|
for (x, y) in r.iter_mut().zip(&q) {
|
||||||
|
*x -= d * y;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
chosen
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
fn unit(v: &[f32]) -> Vec<f32> {
|
||||||
|
let n = v.iter().map(|x| x * x).sum::<f32>().sqrt();
|
||||||
|
v.iter().map(|x| x / n).collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_person_under_the_cap_keeps_every_eligible_face() {
|
||||||
|
let e = [unit(&[1.0, 0.0]), unit(&[0.0, 1.0]), unit(&[1.0, 1.0])];
|
||||||
|
let refs: Vec<&[f32]> = e.iter().map(Vec::as_slice).collect();
|
||||||
|
let q = [Some(20.0), None, Some(16.0)];
|
||||||
|
assert_eq!(select(&refs, &q, 100), vec![0, 1, 2]);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_short_vector_never_stands_for_a_person() {
|
||||||
|
let e = [unit(&[1.0, 0.0]), unit(&[0.0, 1.0])];
|
||||||
|
let refs: Vec<&[f32]> = e.iter().map(Vec::as_slice).collect();
|
||||||
|
let q = [Some(20.0), Some(MIN_REFERENCE_QUALITY - 0.01)];
|
||||||
|
assert_eq!(select(&refs, &q, 100), vec![0]);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Two hundred frames from one afternoon and one profile shot: the
|
||||||
|
/// profile is the second choice, not the two-hundred-and-first.
|
||||||
|
#[test]
|
||||||
|
fn the_odd_one_out_is_chosen_before_any_duplicate() {
|
||||||
|
let mut e: Vec<Vec<f32>> = Vec::new();
|
||||||
|
let mut q = Vec::new();
|
||||||
|
for i in 0..200 {
|
||||||
|
// Near-duplicates of one direction, with a little noise.
|
||||||
|
let t = (i as f32) * 1e-3;
|
||||||
|
e.push(unit(&[1.0, t, t * 0.5]));
|
||||||
|
q.push(Some(20.0 + (i % 7) as f32));
|
||||||
|
}
|
||||||
|
e.push(unit(&[0.0, 0.0, 1.0]));
|
||||||
|
q.push(Some(16.0));
|
||||||
|
let refs: Vec<&[f32]> = e.iter().map(Vec::as_slice).collect();
|
||||||
|
let chosen = select(&refs, &q, 3);
|
||||||
|
assert_eq!(chosen.len(), 3);
|
||||||
|
assert_eq!(
|
||||||
|
chosen[1], 200,
|
||||||
|
"the profile shot was not second: {chosen:?}"
|
||||||
|
);
|
||||||
|
// Seeded on the longest vector.
|
||||||
|
assert_eq!(q[chosen[0]], Some(26.0));
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Faces inside the span of the chosen ones add no volume and are not
|
||||||
|
/// taken to fill the cap.
|
||||||
|
#[test]
|
||||||
|
fn the_cap_is_not_filled_from_inside_the_span() {
|
||||||
|
let e = [
|
||||||
|
unit(&[1.0, 0.0]),
|
||||||
|
unit(&[0.0, 1.0]),
|
||||||
|
unit(&[1.0, 1.0]),
|
||||||
|
unit(&[2.0, -1.0]),
|
||||||
|
];
|
||||||
|
let refs: Vec<&[f32]> = e.iter().map(Vec::as_slice).collect();
|
||||||
|
let q = [Some(20.0); 4];
|
||||||
|
assert_eq!(select(&refs, &q, 3).len(), 2);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn the_choice_is_deterministic() {
|
||||||
|
let e: Vec<Vec<f32>> = (0..50)
|
||||||
|
.map(|i| {
|
||||||
|
let a = (i as f32) * 0.37;
|
||||||
|
unit(&[a.cos(), a.sin(), (a * 3.0).sin(), 0.2])
|
||||||
|
})
|
||||||
|
.collect();
|
||||||
|
let refs: Vec<&[f32]> = e.iter().map(Vec::as_slice).collect();
|
||||||
|
let q = vec![Some(18.0); 50];
|
||||||
|
assert_eq!(select(&refs, &q, 5), select(&refs, &q, 5));
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
//! What a frame actually costs — the measurement FR-DSP-2 is waiting on.
|
//! What a frame actually costs — the measurement FR-DSP-2 is waiting on.
|
||||||
//!
|
//!
|
||||||
//! `docs/display-and-extension.md` §2 argues that tiled computation predates
|
//! `docs/dev/display-and-extension.md` §2 argues that tiled computation predates
|
||||||
//! the fused-shader design and may not need to exist: the composer folds every
|
//! the fused-shader design and may not need to exist: the composer folds every
|
||||||
//! active operation into **one dispatch over a viewport-sized target**, so the
|
//! active operation into **one dispatch over a viewport-sized target**, so the
|
||||||
//! problem tiles were invented to solve may already be solved. That argument
|
//! problem tiles were invented to solve may already be solved. That argument
|
||||||
@@ -28,7 +28,7 @@
|
|||||||
//! the per-frame CPU half is dominated by shader-source assembly, which is
|
//! the per-frame CPU half is dominated by shader-source assembly, which is
|
||||||
//! string formatting and is several times slower unoptimised.
|
//! string formatting and is several times slower unoptimised.
|
||||||
//!
|
//!
|
||||||
//! The committed numbers live in `docs/frame-budget.md`. Rerun this and diff
|
//! The committed numbers live in `docs/dev/frame-budget.md`. Rerun this and diff
|
||||||
//! that file; a regression should be a diff rather than somebody's memory.
|
//! that file; a regression should be a diff rather than somebody's memory.
|
||||||
//!
|
//!
|
||||||
//! # Why the 99th percentile and not the mean
|
//! # Why the 99th percentile and not the mean
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
//! Segment an image and write the granularity ladder as false-coloured PPMs.
|
//! Segment an image and write the granularity ladder as false-coloured PPMs.
|
||||||
//!
|
//!
|
||||||
//! The whole point of S15 step 2 (docs/segmentation.md §11): look at the
|
//! The whole point of S15 step 2 (docs/dev/segmentation.md §11): look at the
|
||||||
//! ladder and decide whether clicking through it would land on the things a
|
//! ladder and decide whether clicking through it would land on the things a
|
||||||
//! person means. No amount of design settles that — the pictures do.
|
//! person means. No amount of design settles that — the pictures do.
|
||||||
//!
|
//!
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
//! Watershed segmentation — arm A's GPU half (S15, docs/segmentation.md).
|
//! Watershed segmentation — arm A's GPU half (S15, docs/dev/segmentation.md).
|
||||||
//!
|
//!
|
||||||
//! Runs the five passes in `shaders/watershed.wgsl` over a demosaiced image
|
//! Runs the five passes in `shaders/watershed.wgsl` over a demosaiced image
|
||||||
//! and leaves a basin label per pixel on the GPU. The hierarchy built from
|
//! and leaves a basin label per pixel on the GPU. The hierarchy built from
|
||||||
@@ -20,7 +20,7 @@
|
|||||||
//! one AC-8 forbids is per frame in the render loop, and sharing a switch
|
//! one AC-8 forbids is per frame in the render loop, and sharing a switch
|
||||||
//! would force a build wanting local masking to unlock the other.
|
//! would force a build wanting local masking to unlock the other.
|
||||||
//!
|
//!
|
||||||
//! It is still a real cost and still unfinished. F3 in docs/segmentation.md
|
//! It is still a real cost and still unfinished. F3 in docs/dev/segmentation.md
|
||||||
//! §12 stands: the adjacency accumulation belongs GPU-side with atomics, and
|
//! §12 stands: the adjacency accumulation belongs GPU-side with atomics, and
|
||||||
//! until it moves there every segmentation pays a full-resolution transfer.
|
//! until it moves there every segmentation pays a full-resolution transfer.
|
||||||
//! Read the feature name as a description of a known gap rather than as
|
//! Read the feature name as a description of a known gap rather than as
|
||||||
@@ -36,7 +36,7 @@ pub struct SegmentOptions {
|
|||||||
/// Longest proxy edge. The segmentation runs here, not at sensor
|
/// Longest proxy edge. The segmentation runs here, not at sensor
|
||||||
/// resolution: a 24 MP watershed costs 12× the memory to place boundaries
|
/// resolution: a 24 MP watershed costs 12× the memory to place boundaries
|
||||||
/// a person cannot see, and the boundary refinement that matters at 1:1
|
/// a person cannot see, and the boundary refinement that matters at 1:1
|
||||||
/// is a separate stage (docs/segmentation.md §4).
|
/// is a separate stage (docs/dev/segmentation.md §4).
|
||||||
pub max_edge: u32,
|
pub max_edge: u32,
|
||||||
/// Pre-smoothing radius in proxy pixels. The caller's to raise with ISO —
|
/// Pre-smoothing radius in proxy pixels. The caller's to raise with ISO —
|
||||||
/// this is the single knob that decides whether a noisy file segments
|
/// this is the single knob that decides whether a noisy file segments
|
||||||
@@ -69,7 +69,7 @@ impl Default for SegmentOptions {
|
|||||||
w_chroma: 0.5,
|
w_chroma: 0.5,
|
||||||
// **Zero: the pass is off.** It is implemented, dispatched
|
// **Zero: the pass is off.** It is implemented, dispatched
|
||||||
// correctly and measurably changes nothing — see the ignored test
|
// correctly and measurably changes nothing — see the ignored test
|
||||||
// below and §12 of docs/segmentation.md. Until that is understood,
|
// below and §12 of docs/dev/segmentation.md. Until that is understood,
|
||||||
// running it would buy 64 dispatches per segmentation and no
|
// running it would buy 64 dispatches per segmentation and no
|
||||||
// improvement, so the default declines to pay.
|
// improvement, so the default declines to pay.
|
||||||
plateau_iterations: 0,
|
plateau_iterations: 0,
|
||||||
@@ -486,7 +486,7 @@ impl Segmentation {
|
|||||||
/// a region graph of a few thousand nodes that every later interaction
|
/// a region graph of a few thousand nodes that every later interaction
|
||||||
/// reads from the CPU anyway.
|
/// reads from the CPU anyway.
|
||||||
///
|
///
|
||||||
/// What it is *not* is finished. F3 in docs/segmentation.md §12 stands:
|
/// What it is *not* is finished. F3 in docs/dev/segmentation.md §12 stands:
|
||||||
/// the adjacency accumulation belongs on the GPU with atomics, and until
|
/// the adjacency accumulation belongs on the GPU with atomics, and until
|
||||||
/// it moves there a segmentation costs one full-resolution transfer of the
|
/// it moves there a segmentation costs one full-resolution transfer of the
|
||||||
/// label and gradient buffers. That is a real cost on a phone and the
|
/// label and gradient buffers. That is a real cost on a phone and the
|
||||||
@@ -724,7 +724,7 @@ mod tests {
|
|||||||
px
|
px
|
||||||
}
|
}
|
||||||
#[test]
|
#[test]
|
||||||
#[ignore = "the plateau pass is a measured no-op; see docs/segmentation.md §12"]
|
#[ignore = "the plateau pass is a measured no-op; see docs/dev/segmentation.md §12"]
|
||||||
fn lower_completion_drains_a_plateau_instead_of_shattering_it() {
|
fn lower_completion_drains_a_plateau_instead_of_shattering_it() {
|
||||||
// F1, asserted rather than eyeballed, and asserted at the level where
|
// F1, asserted rather than eyeballed, and asserted at the level where
|
||||||
// it matters.
|
// it matters.
|
||||||
@@ -741,7 +741,7 @@ mod tests {
|
|||||||
// with no exit anywhere — cannot be drained by a distance that has
|
// with no exit anywhere — cannot be drained by a distance that has
|
||||||
// nowhere to descend to, and collapsing it fully would need connected
|
// nowhere to descend to, and collapsing it fully would need connected
|
||||||
// component labelling rather than a local rule. It is not worth it:
|
// component labelling rather than a local rule. It is not worth it:
|
||||||
// see docs/segmentation.md §12.
|
// see docs/dev/segmentation.md §12.
|
||||||
let Some(ctx) = ctx() else { return };
|
let Some(ctx) = ctx() else { return };
|
||||||
let (w, h) = (96u32, 96u32);
|
let (w, h) = (96u32, 96u32);
|
||||||
let src = DemosaicedImage::from_rgba8(&ctx, &ramp(w, h), w, h).expect("source");
|
let src = DemosaicedImage::from_rgba8(&ctx, &ramp(w, h), w, h).expect("source");
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
// Watershed segmentation — the passes behind arm A of S15 (docs/segmentation.md).
|
// Watershed segmentation — the passes behind arm A of S15 (docs/dev/segmentation.md).
|
||||||
//
|
//
|
||||||
// Seven entry points forming one chain:
|
// Seven entry points forming one chain:
|
||||||
//
|
//
|
||||||
@@ -197,7 +197,7 @@ fn gradient(@builtin(global_invocation_id) gid: vec3<u32>) {
|
|||||||
// lowest-indexed neighbour, which is up and to the left. Each pixel therefore
|
// lowest-indexed neighbour, which is up and to the left. Each pixel therefore
|
||||||
// walks diagonally until it falls off the plateau, and one flat region becomes
|
// walks diagonally until it falls off the plateau, and one flat region becomes
|
||||||
// a fan of diagonal chains rather than one basin — visible as hatching across
|
// a fan of diagonal chains rather than one basin — visible as hatching across
|
||||||
// what should be a single area (docs/segmentation.md §12, F1).
|
// what should be a single area (docs/dev/segmentation.md §12, F1).
|
||||||
//
|
//
|
||||||
// The fix is the standard lower-completion: give each plateau pixel its
|
// The fix is the standard lower-completion: give each plateau pixel its
|
||||||
// geodesic distance to the nearest pixel that *does* have a lower neighbour,
|
// geodesic distance to the nearest pixel that *does* have a lower neighbour,
|
||||||
|
|||||||
@@ -2,11 +2,11 @@
|
|||||||
//!
|
//!
|
||||||
//! FR-DSP-3 says a slider updates the visible region within one frame budget at
|
//! FR-DSP-3 says a slider updates the visible region within one frame budget at
|
||||||
//! proxy resolution. Until this file existed nothing checked it, which made it
|
//! proxy resolution. Until this file existed nothing checked it, which made it
|
||||||
//! a wish — `docs/display-and-extension.md` §3 is blunt about that, and §7 is
|
//! a wish — `docs/dev/display-and-extension.md` §3 is blunt about that, and §7 is
|
||||||
//! blunt about what tagging an unchecked requirement does to the coverage
|
//! blunt about what tagging an unchecked requirement does to the coverage
|
||||||
//! figure.
|
//! figure.
|
||||||
//!
|
//!
|
||||||
//! The measurements this guards are in [`docs/frame-budget.md`], produced by
|
//! The measurements this guards are in [`docs/dev/frame-budget.md`], produced by
|
||||||
//! `examples/frame_budget.rs`. This file is the part of them that has to keep
|
//! `examples/frame_budget.rs`. This file is the part of them that has to keep
|
||||||
//! being true: it renders the **whole point-operation chain** through the real
|
//! being true: it renders the **whole point-operation chain** through the real
|
||||||
//! `render_detailed` for a hundred frames, moving a slider between each, and
|
//! `render_detailed` for a hundred frames, moving a slider between each, and
|
||||||
@@ -17,7 +17,7 @@
|
|||||||
//! **The neighbourhood stage is deliberately not in the asserted chain.** It is
|
//! **The neighbourhood stage is deliberately not in the asserted chain.** It is
|
||||||
//! over the budget today — clarity alone is 34 ms at 4K, because its kernel is
|
//! over the budget today — clarity alone is 34 ms at 4K, because its kernel is
|
||||||
//! a fraction of the frame and reaches a 52-pixel radius there — and
|
//! a fraction of the frame and reaches a 52-pixel radius there — and
|
||||||
//! `docs/frame-budget.md` records that, names the fix (a base computed at
|
//! `docs/dev/frame-budget.md` records that, names the fix (a base computed at
|
||||||
//! reduced resolution) and does not pretend otherwise. Asserting a budget the
|
//! reduced resolution) and does not pretend otherwise. Asserting a budget the
|
||||||
//! code does not meet would produce a red suite that everyone learns to ignore;
|
//! code does not meet would produce a red suite that everyone learns to ignore;
|
||||||
//! asserting it on a chain that quietly excluded the expensive stage *without
|
//! asserting it on a chain that quietly excluded the expensive stage *without
|
||||||
@@ -81,7 +81,7 @@ const SOURCE: (u32, u32) = (6000, 4000);
|
|||||||
/// The viewport the budget is asserted at: a 16:10 desktop display.
|
/// The viewport the budget is asserted at: a 16:10 desktop display.
|
||||||
///
|
///
|
||||||
/// Not 4K, and the reason is worth stating. At 4K the fused chain still passes
|
/// Not 4K, and the reason is worth stating. At 4K the fused chain still passes
|
||||||
/// with room to spare (4.5 ms of GPU; see `docs/frame-budget.md`), but a test
|
/// with room to spare (4.5 ms of GPU; see `docs/dev/frame-budget.md`), but a test
|
||||||
/// that renders 8.3 M pixels a hundred times twice over is four seconds of
|
/// that renders 8.3 M pixels a hundred times twice over is four seconds of
|
||||||
/// suite time to re-establish a conclusion 4.1 M pixels already establishes.
|
/// suite time to re-establish a conclusion 4.1 M pixels already establishes.
|
||||||
const VIEWPORT: (u32, u32) = (2560, 1600);
|
const VIEWPORT: (u32, u32) = (2560, 1600);
|
||||||
@@ -166,7 +166,7 @@ impl Run {
|
|||||||
judged <= BUDGET_MS,
|
judged <= BUDGET_MS,
|
||||||
"{case} at {}x{}: p99 of {FRAMES} frames was {judged:.2} ms, over the \
|
"{case} at {}x{}: p99 of {FRAMES} frames was {judged:.2} ms, over the \
|
||||||
{BUDGET_MS:.0} ms budget (cpu {:.2} ms, gpu {:.2} ms, total {:.2} ms). \
|
{BUDGET_MS:.0} ms budget (cpu {:.2} ms, gpu {:.2} ms, total {:.2} ms). \
|
||||||
FR-DSP-3 is what this violates; docs/frame-budget.md holds the \
|
FR-DSP-3 is what this violates; docs/dev/frame-budget.md holds the \
|
||||||
numbers it used to be.",
|
numbers it used to be.",
|
||||||
viewport.0,
|
viewport.0,
|
||||||
viewport.1,
|
viewport.1,
|
||||||
|
|||||||
@@ -326,7 +326,7 @@ fn a_proxy_and_an_export_agree_about_the_effect() {
|
|||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn crossing_the_reduction_threshold_does_not_change_the_picture() {
|
fn crossing_the_reduction_threshold_does_not_change_the_picture() {
|
||||||
// TRACES: FR-DSP-3 — `docs/technical-debt.md` TD-4, held in pixels.
|
// TRACES: FR-DSP-3 — `docs/dev/technical-debt.md` TD-4, held in pixels.
|
||||||
//
|
//
|
||||||
// Clarity's base is computed on a reduced grid, and how reduced depends on
|
// Clarity's base is computed on a reduced grid, and how reduced depends on
|
||||||
// the viewport: `LocalContrast::reduction` steps 4 -> 2 -> 1 as sigma
|
// the viewport: `LocalContrast::reduction` steps 4 -> 2 -> 1 as sigma
|
||||||
|
|||||||
@@ -10,7 +10,7 @@
|
|||||||
//! code path — the zoom is the full-resolution path — which is why the
|
//! code path — the zoom is the full-resolution path — which is why the
|
||||||
//! requirement has been satisfied for some time without anyone tagging it.
|
//! requirement has been satisfied for some time without anyone tagging it.
|
||||||
//!
|
//!
|
||||||
//! `docs/display-and-extension.md` §7 is the reason this file exists rather
|
//! `docs/dev/display-and-extension.md` §7 is the reason this file exists rather
|
||||||
//! than a tag on `framing.rs`: a requirement counts as covered when a `TRACES`
|
//! than a tag on `framing.rs`: a requirement counts as covered when a `TRACES`
|
||||||
//! comment names it, and nothing checks that the code under the tag does the
|
//! comment names it, and nothing checks that the code under the tag does the
|
||||||
//! thing. `FR-DEV-8` is tagged against plumbing a future operation would use.
|
//! thing. `FR-DEV-8` is tagged against plumbing a future operation would use.
|
||||||
|
|||||||
@@ -6,7 +6,7 @@ rust-version.workspace = true
|
|||||||
license.workspace = true
|
license.workspace = true
|
||||||
|
|
||||||
# The one crate that names a runtime, a provider, a vendor library or a
|
# The one crate that names a runtime, a provider, a vendor library or a
|
||||||
# device (docs/inference.md §8). `dr-face` and `dr-segment` ask it for a
|
# device (docs/dev/inference.md §8). `dr-face` and `dr-segment` ask it for a
|
||||||
# session by role and never see which of these answered.
|
# session by role and never see which of these answered.
|
||||||
|
|
||||||
[dependencies]
|
[dependencies]
|
||||||
@@ -28,7 +28,10 @@ ort-sys = { version = "2.0.0-rc.13", default-features = false, features = ["disa
|
|||||||
# The NVIDIA rungs exist on the desktop only. These features add `ort`'s
|
# The NVIDIA rungs exist on the desktop only. These features add `ort`'s
|
||||||
# option builders and nothing else — no linking under `alternative-backend` —
|
# option builders and nothing else — no linking under `alternative-backend` —
|
||||||
# but an Android binary has no business carrying even the option names, and
|
# but an Android binary has no business carrying even the option names, and
|
||||||
# the packaging must never be tempted to (§2, §3.1).
|
# the packaging must never be tempted to (§2, §3.1). The AMD rung needs no
|
||||||
|
# feature: MIGraphX is registered through the runtime's generic key/value
|
||||||
|
# entry point (`session::migraphx`), because `ort`'s own builder cannot
|
||||||
|
# name the compiled-program cache.
|
||||||
[target.'cfg(not(target_os = "android"))'.dependencies]
|
[target.'cfg(not(target_os = "android"))'.dependencies]
|
||||||
ort = { workspace = true, features = ["cuda", "tensorrt"] }
|
ort = { workspace = true, features = ["cuda", "tensorrt"] }
|
||||||
|
|
||||||
@@ -42,3 +45,8 @@ default = ["tract"]
|
|||||||
tract = ["dep:ort-tract"]
|
tract = ["dep:ort-tract"]
|
||||||
# Look for `libonnxruntime` on disk and hand its table to `ort`.
|
# Look for `libonnxruntime` on disk and hand its table to `ort`.
|
||||||
native = ["dep:libloading", "dep:ort-sys"]
|
native = ["dep:libloading", "dep:ort-sys"]
|
||||||
|
|
||||||
|
[dev-dependencies]
|
||||||
|
# The `ep_probe` example prints the provider's own diagnostics, which is most
|
||||||
|
# of what a failed rung tells you.
|
||||||
|
env_logger.workspace = true
|
||||||
|
|||||||
@@ -0,0 +1,207 @@
|
|||||||
|
//! Time each execution provider a runtime offers, on the models this
|
||||||
|
//! repository ships — the measurement docs/inference.md §1 requires before a
|
||||||
|
//! rung is added to §2's ladder.
|
||||||
|
//!
|
||||||
|
//! DARKROOM_ORT_DIR=/usr/lib \
|
||||||
|
//! cargo run --release -p dr-inference-engine --features native,tract \
|
||||||
|
//! --example ep_probe -- models/face/scrfd_500m_640.onnx ...
|
||||||
|
//!
|
||||||
|
//! Prints one row per (model, provider): the median of timed runs after
|
||||||
|
//! warm-ups, and the build time, which for a compiling provider is the
|
||||||
|
//! number that decides whether it needs an engine cache. MIGraphX is built
|
||||||
|
//! twice per precision — cold, then again from the cache it just wrote —
|
||||||
|
//! so both numbers are on the page.
|
||||||
|
//!
|
||||||
|
//! The ROCm provider is not in the list: ONNX Runtime removed it in 1.23,
|
||||||
|
//! and 1.29's `onnxruntime-rocm` ships `libonnxruntime_providers_migraphx.so`
|
||||||
|
//! and nothing else for AMD.
|
||||||
|
|
||||||
|
use std::path::{Path, PathBuf};
|
||||||
|
use std::time::Instant;
|
||||||
|
|
||||||
|
#[derive(Clone, Copy, PartialEq)]
|
||||||
|
enum Ep {
|
||||||
|
Cpu,
|
||||||
|
MiGraphX,
|
||||||
|
MiGraphXFp16,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Ep {
|
||||||
|
fn label(self) -> &'static str {
|
||||||
|
match self {
|
||||||
|
Ep::Cpu => "CPU",
|
||||||
|
Ep::MiGraphX => "MIGraphX f32",
|
||||||
|
Ep::MiGraphXFp16 => "MIGraphX fp16",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn build(ep: Ep, bytes: &[u8], threads: usize, cache: &Path) -> ort::Result<ort::session::Session> {
|
||||||
|
let mut b = ort::session::Session::builder()?.with_intra_threads(threads)?;
|
||||||
|
match ep {
|
||||||
|
Ep::Cpu => {}
|
||||||
|
Ep::MiGraphX => migraphx(&mut b, false, &cache.join("f32"))?,
|
||||||
|
Ep::MiGraphXFp16 => migraphx(&mut b, true, &cache.join("fp16"))?,
|
||||||
|
}
|
||||||
|
b.commit_from_memory(bytes)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Register MIGraphX through the generic key/value API. `ort`'s own
|
||||||
|
/// builder fills the legacy `OrtMIGraphXProviderOptions`, which 1.29 reads
|
||||||
|
/// for its precision flags and nothing else: the model cache directory —
|
||||||
|
/// the difference between a 40 s load and a 0.3 s one — only travels this
|
||||||
|
/// way. The cache key is the graph, the GPU and the MIGraphX version, not
|
||||||
|
/// the precision, so each precision gets its own directory.
|
||||||
|
fn migraphx(
|
||||||
|
b: &mut ort::session::builder::SessionBuilder,
|
||||||
|
fp16: bool,
|
||||||
|
cache: &Path,
|
||||||
|
) -> ort::Result<()> {
|
||||||
|
use ort::AsPointer;
|
||||||
|
use std::ffi::CString;
|
||||||
|
std::fs::create_dir_all(cache).map_err(|e| ort::Error::new(e.to_string()))?;
|
||||||
|
let keys = [c"migraphx_fp16_enable", c"migraphx_model_cache_dir"];
|
||||||
|
let values = [
|
||||||
|
CString::new(if fp16 { "1" } else { "0" }).unwrap(),
|
||||||
|
CString::new(cache.to_string_lossy().as_bytes()).unwrap(),
|
||||||
|
];
|
||||||
|
let key_ptrs: Vec<_> = keys.iter().map(|k| k.as_ptr()).collect();
|
||||||
|
let value_ptrs: Vec<_> = values.iter().map(|v| v.as_ptr()).collect();
|
||||||
|
// SAFETY: the documented C call, over arrays that outlive it; the
|
||||||
|
// runtime copies the strings into its own options map.
|
||||||
|
unsafe {
|
||||||
|
let status = (ort::api().SessionOptionsAppendExecutionProvider)(
|
||||||
|
b.ptr_mut(),
|
||||||
|
c"MIGraphX".as_ptr(),
|
||||||
|
key_ptrs.as_ptr(),
|
||||||
|
value_ptrs.as_ptr(),
|
||||||
|
keys.len(),
|
||||||
|
);
|
||||||
|
ort::Error::result_from_status(status)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Median of `runs` timed runs over zeros, in milliseconds, after warm-ups.
|
||||||
|
fn time(session: &mut ort::session::Session, warmups: usize, runs: usize) -> Result<f64, String> {
|
||||||
|
let shape: Vec<usize> = session.inputs()[0]
|
||||||
|
.dtype()
|
||||||
|
.tensor_shape()
|
||||||
|
.ok_or("input is not a tensor")?
|
||||||
|
.iter()
|
||||||
|
.map(|&d| if d > 0 { d as usize } else { 1 })
|
||||||
|
.collect();
|
||||||
|
let zeros = vec![0f32; shape.iter().product()];
|
||||||
|
let once = |s: &mut ort::session::Session| -> Result<f64, String> {
|
||||||
|
let input = ort::value::Tensor::from_array((shape.clone(), zeros.clone()))
|
||||||
|
.map_err(|e| e.to_string())?;
|
||||||
|
let t = Instant::now();
|
||||||
|
let out = s.run(ort::inputs![input]).map_err(|e| e.to_string())?;
|
||||||
|
let _ = out[0]
|
||||||
|
.try_extract_tensor::<f32>()
|
||||||
|
.map_err(|e| e.to_string())?;
|
||||||
|
Ok(t.elapsed().as_secs_f64() * 1e3)
|
||||||
|
};
|
||||||
|
for _ in 0..warmups {
|
||||||
|
once(session)?;
|
||||||
|
}
|
||||||
|
let mut times = Vec::with_capacity(runs);
|
||||||
|
for _ in 0..runs {
|
||||||
|
times.push(once(session)?);
|
||||||
|
}
|
||||||
|
times.sort_by(|a, b| a.partial_cmp(b).unwrap());
|
||||||
|
Ok(times[times.len() / 2])
|
||||||
|
}
|
||||||
|
|
||||||
|
fn first_line(s: &str) -> String {
|
||||||
|
s.lines().next().unwrap_or("").chars().take(120).collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn main() {
|
||||||
|
env_logger::Builder::from_env(env_logger::Env::default().default_filter_or("info")).init();
|
||||||
|
|
||||||
|
let models: Vec<PathBuf> = std::env::args_os().skip(1).map(PathBuf::from).collect();
|
||||||
|
if models.is_empty() {
|
||||||
|
eprintln!("usage: ep_probe MODEL.onnx [MODEL.onnx ...]");
|
||||||
|
std::process::exit(2);
|
||||||
|
}
|
||||||
|
|
||||||
|
dr_inference_engine::ensure_runtime();
|
||||||
|
let runtime = dr_inference_engine::status().runtime;
|
||||||
|
println!("runtime: {}", runtime.label());
|
||||||
|
if !runtime.is_native() {
|
||||||
|
println!("(tract: no provider to compare; set DARKROOM_ORT_DIR)");
|
||||||
|
}
|
||||||
|
|
||||||
|
let threads = std::thread::available_parallelism()
|
||||||
|
.map(|n| n.get().saturating_sub(2).max(1))
|
||||||
|
.unwrap_or(1);
|
||||||
|
println!("intra-op threads: {threads}");
|
||||||
|
let cache = std::env::temp_dir().join("darkroom-ep-probe");
|
||||||
|
let _ = std::fs::remove_dir_all(&cache);
|
||||||
|
println!("compiled-program cache: {}\n", cache.display());
|
||||||
|
|
||||||
|
println!(
|
||||||
|
"{:<28} {:<15} {:>10} {:>10}",
|
||||||
|
"model", "provider", "build s", "median ms"
|
||||||
|
);
|
||||||
|
for model in &models {
|
||||||
|
let bytes = match std::fs::read(model) {
|
||||||
|
Ok(b) => b,
|
||||||
|
Err(e) => {
|
||||||
|
println!("{:<28} read failed: {e}", name(model));
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
// A compiling provider is built twice: the second build reads the
|
||||||
|
// program the first wrote, and its time is what a launch after the
|
||||||
|
// first costs.
|
||||||
|
let plan = [
|
||||||
|
(Ep::Cpu, false),
|
||||||
|
(Ep::MiGraphX, false),
|
||||||
|
(Ep::MiGraphX, true),
|
||||||
|
(Ep::MiGraphXFp16, false),
|
||||||
|
(Ep::MiGraphXFp16, true),
|
||||||
|
];
|
||||||
|
for (ep, cached) in plan {
|
||||||
|
let started = Instant::now();
|
||||||
|
match build(ep, &bytes, threads, &cache) {
|
||||||
|
Ok(mut session) => {
|
||||||
|
let built = started.elapsed().as_secs_f64();
|
||||||
|
match time(&mut session, 3, 15) {
|
||||||
|
Ok(ms) => println!(
|
||||||
|
"{:<28} {:<15} {:>10.1} {:>10.1}{}",
|
||||||
|
name(model),
|
||||||
|
ep.label(),
|
||||||
|
built,
|
||||||
|
ms,
|
||||||
|
if cached { " (from cache)" } else { "" }
|
||||||
|
),
|
||||||
|
Err(e) => println!(
|
||||||
|
"{:<28} {:<15} {:>10.1} {:>10} {}",
|
||||||
|
name(model),
|
||||||
|
ep.label(),
|
||||||
|
built,
|
||||||
|
"ran ✗",
|
||||||
|
first_line(&e)
|
||||||
|
),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Err(e) => println!(
|
||||||
|
"{:<28} {:<15} {:>21} {}",
|
||||||
|
name(model),
|
||||||
|
ep.label(),
|
||||||
|
"build ✗",
|
||||||
|
first_line(&e.to_string())
|
||||||
|
),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
println!();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn name(p: &Path) -> String {
|
||||||
|
p.file_name()
|
||||||
|
.unwrap_or(p.as_os_str())
|
||||||
|
.to_string_lossy()
|
||||||
|
.into_owned()
|
||||||
|
}
|
||||||
@@ -0,0 +1,100 @@
|
|||||||
|
//! Walk the ladder as the app does — probe, engines, then a session — and
|
||||||
|
//! say what each step chose. The M5 check of docs/inference.md §6 without
|
||||||
|
//! the app around it.
|
||||||
|
//!
|
||||||
|
//! DARKROOM_ORT_DIR=/usr/lib \
|
||||||
|
//! cargo run --release -p dr-inference-engine --features native,tract \
|
||||||
|
//! --example ladder -- CACHE_DIR models/face/scrfd_500m_640.onnx [MODEL.onnx ...]
|
||||||
|
//!
|
||||||
|
//! Every model named is a `Detector` for the config's purposes, which is
|
||||||
|
//! enough to see the rung taken, the engines compiled and a session land
|
||||||
|
//! on it. Delete `CACHE_DIR` to see the first run again; keep it to see the
|
||||||
|
//! second.
|
||||||
|
|
||||||
|
use std::path::PathBuf;
|
||||||
|
use std::time::{Duration, Instant};
|
||||||
|
|
||||||
|
fn main() {
|
||||||
|
env_logger::Builder::from_env(env_logger::Env::default().default_filter_or("info")).init();
|
||||||
|
let mut args = std::env::args_os().skip(1).map(PathBuf::from);
|
||||||
|
let (Some(cache_dir), models) = (args.next(), args.collect::<Vec<_>>()) else {
|
||||||
|
eprintln!("usage: ladder CACHE_DIR MODEL.onnx [MODEL.onnx ...]");
|
||||||
|
std::process::exit(2);
|
||||||
|
};
|
||||||
|
if models.is_empty() {
|
||||||
|
eprintln!("usage: ladder CACHE_DIR MODEL.onnx [MODEL.onnx ...]");
|
||||||
|
std::process::exit(2);
|
||||||
|
}
|
||||||
|
|
||||||
|
let runtime_dirs: Vec<PathBuf> = std::env::var_os("DARKROOM_ORT_DIR")
|
||||||
|
.map(PathBuf::from)
|
||||||
|
.into_iter()
|
||||||
|
.collect();
|
||||||
|
let started = Instant::now();
|
||||||
|
dr_inference_engine::init(dr_inference_engine::Config {
|
||||||
|
runtime_dirs,
|
||||||
|
cache_dir: cache_dir.clone(),
|
||||||
|
models: models
|
||||||
|
.iter()
|
||||||
|
.map(|p| (dr_inference_engine::Role::Detector, p.clone()))
|
||||||
|
.collect(),
|
||||||
|
embedded: Vec::new(),
|
||||||
|
ceiling: None,
|
||||||
|
threads: 0,
|
||||||
|
decay: Duration::ZERO,
|
||||||
|
});
|
||||||
|
|
||||||
|
let mut last = String::new();
|
||||||
|
loop {
|
||||||
|
let s = dr_inference_engine::status();
|
||||||
|
let line = format!(
|
||||||
|
"{} · {} · engines {}/{}{}",
|
||||||
|
s.line(),
|
||||||
|
if s.probing {
|
||||||
|
"probing"
|
||||||
|
} else {
|
||||||
|
s.reason.as_str()
|
||||||
|
},
|
||||||
|
s.engines.0,
|
||||||
|
s.engines.1,
|
||||||
|
if s.failed.is_empty() {
|
||||||
|
String::new()
|
||||||
|
} else {
|
||||||
|
format!(
|
||||||
|
" · tried {}",
|
||||||
|
s.failed
|
||||||
|
.iter()
|
||||||
|
.map(|(r, why)| format!("{}: {why}", r.label()))
|
||||||
|
.collect::<Vec<_>>()
|
||||||
|
.join(" · ")
|
||||||
|
)
|
||||||
|
}
|
||||||
|
);
|
||||||
|
if line != last {
|
||||||
|
println!("{:>6.1} s {line}", started.elapsed().as_secs_f64());
|
||||||
|
last = line;
|
||||||
|
}
|
||||||
|
if !s.probing && s.engines.0 >= s.engines.1 {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
std::thread::sleep(Duration::from_millis(500));
|
||||||
|
}
|
||||||
|
|
||||||
|
for path in &models {
|
||||||
|
let bytes = std::fs::read(path).expect("read model");
|
||||||
|
let t = Instant::now();
|
||||||
|
let model = dr_inference_engine::open(
|
||||||
|
dr_inference_engine::Role::Detector,
|
||||||
|
dr_inference_engine::Form::F32,
|
||||||
|
&bytes,
|
||||||
|
)
|
||||||
|
.expect("open model");
|
||||||
|
let acquired = model.acquire().expect("acquire session");
|
||||||
|
println!(
|
||||||
|
"{} on {} in {:.2} s",
|
||||||
|
path.file_name().unwrap().to_string_lossy(),
|
||||||
|
acquired.rung().label(),
|
||||||
|
t.elapsed().as_secs_f64()
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
//! The API table `ort` runs on, chosen once (docs/inference.md §3).
|
//! The API table `ort` runs on, chosen once (docs/dev/inference.md §3).
|
||||||
//!
|
//!
|
||||||
//! `ort` with `alternative-backend` links no runtime and asks, on first use,
|
//! `ort` with `alternative-backend` links no runtime and asks, on first use,
|
||||||
//! for an `OrtApi` — a struct of function pointers. Two things can fill it:
|
//! for an `OrtApi` — a struct of function pointers. Two things can fill it:
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
//! Compiled engines: what a rung builds once per device, and the thread that
|
//! Compiled engines: what a rung builds once per device, and the thread that
|
||||||
//! builds them before anyone asks (docs/inference.md §5, §6).
|
//! builds them before anyone asks (docs/dev/inference.md §5, §6).
|
||||||
//!
|
//!
|
||||||
//! TensorRT keeps its own engine cache keyed by graph hash; QNN writes a
|
//! TensorRT keeps its own engine cache keyed by graph hash; QNN writes a
|
||||||
//! context model. Both are opaque to this crate, which tracks only *that* a
|
//! context model. Both are opaque to this crate, which tracks only *that* a
|
||||||
|
|||||||
@@ -1,10 +1,10 @@
|
|||||||
//! Which runtime, which provider and which model form — decided once per
|
//! Which runtime, which provider and which model form — decided once per
|
||||||
//! device, and the only crate that knows the answer (docs/inference.md).
|
//! device, and the only crate that knows the answer (docs/dev/inference.md).
|
||||||
//!
|
//!
|
||||||
//! Consumers ask for a session by [`Role`] and get `ort`'s `Session` back;
|
//! Consumers ask for a session by [`Role`] and get `ort`'s `Session` back;
|
||||||
//! what built it — tract on one core, ONNX Runtime's CPU pool, a TensorRT
|
//! what built it — tract on one core, ONNX Runtime's CPU pool, a TensorRT
|
||||||
//! engine, the Hexagon — is this crate's business and shows up in
|
//! engine, a MIGraphX program, the Hexagon — is this crate's business and
|
||||||
//! [`status`] for the settings row and nowhere else.
|
//! shows up in [`status`] for the settings row and nowhere else.
|
||||||
//!
|
//!
|
||||||
//! The shape follows §3 of the spec: `ort` links nothing (`alternative-backend`),
|
//! The shape follows §3 of the spec: `ort` links nothing (`alternative-backend`),
|
||||||
//! and the first call hands it an API table from either a `libonnxruntime`
|
//! and the first call hands it an API table from either a `libonnxruntime`
|
||||||
@@ -35,13 +35,13 @@ pub enum Role {
|
|||||||
Embedder,
|
Embedder,
|
||||||
Segmenter,
|
Segmenter,
|
||||||
Scene,
|
Scene,
|
||||||
/// The dense landmark model behind the eye reading (docs/faces.md §7c).
|
/// The dense landmark model behind the eye reading (docs/dev/faces.md §7c).
|
||||||
Landmarks,
|
Landmarks,
|
||||||
/// The eye-state and sunglasses classifiers, a few hundred kilobytes.
|
/// The eye-state and sunglasses classifiers, a few hundred kilobytes.
|
||||||
EyeClassifier,
|
EyeClassifier,
|
||||||
/// XFeat, the panorama keypoint detector (docs/panorama.md).
|
/// XFeat, the panorama keypoint detector (docs/dev/panorama.md).
|
||||||
Keypoints,
|
Keypoints,
|
||||||
/// MI-GAN, the panorama border filler (docs/panorama.md §12). Plain
|
/// MI-GAN, the panorama border filler (docs/dev/panorama.md §12). Plain
|
||||||
/// convolutions, so any rung serves it; fp16 on TensorRT and int8 on
|
/// convolutions, so any rung serves it; fp16 on TensorRT and int8 on
|
||||||
/// the Hexagon are the point of it.
|
/// the Hexagon are the point of it.
|
||||||
Inpainter,
|
Inpainter,
|
||||||
@@ -60,7 +60,9 @@ pub enum Form {
|
|||||||
|
|
||||||
/// A rung of the ladder (§2). Ordered: a user override names the highest rung
|
/// A rung of the ladder (§2). Ordered: a user override names the highest rung
|
||||||
/// the probe may take, and a compiling rung falls back to the one below it
|
/// the probe may take, and a compiling rung falls back to the one below it
|
||||||
/// until its engine exists.
|
/// until its engine exists. The order is within a vendor's ladder — a
|
||||||
|
/// machine has NVIDIA rungs or an AMD rung, never both — so a ceiling is
|
||||||
|
/// read as "no higher than this on whichever ladder the device has".
|
||||||
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, PartialOrd, Ord, Serialize, Deserialize)]
|
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, PartialOrd, Ord, Serialize, Deserialize)]
|
||||||
pub enum Rung {
|
pub enum Rung {
|
||||||
/// ONNX Runtime's CPU provider, or tract when no runtime file was found.
|
/// ONNX Runtime's CPU provider, or tract when no runtime file was found.
|
||||||
@@ -69,6 +71,11 @@ pub enum Rung {
|
|||||||
Cuda,
|
Cuda,
|
||||||
/// NVIDIA, through a TensorRT engine compiled on this device. Desktop only.
|
/// NVIDIA, through a TensorRT engine compiled on this device. Desktop only.
|
||||||
TensorRt,
|
TensorRt,
|
||||||
|
/// AMD, through a MIGraphX program compiled on this device. Desktop
|
||||||
|
/// only. ONNX Runtime's ROCm provider, the CUDA provider's twin, was
|
||||||
|
/// removed in ONNX Runtime 1.23, so there is no non-compiling AMD rung
|
||||||
|
/// to fall back to: this one falls back to the CPU.
|
||||||
|
MiGraphX,
|
||||||
/// Qualcomm's Hexagon NPU through QNN, int8 models only. Android only.
|
/// Qualcomm's Hexagon NPU through QNN, int8 models only. Android only.
|
||||||
Hexagon,
|
Hexagon,
|
||||||
}
|
}
|
||||||
@@ -79,6 +86,7 @@ impl Rung {
|
|||||||
Rung::Cpu => "CPU",
|
Rung::Cpu => "CPU",
|
||||||
Rung::Cuda => "CUDA",
|
Rung::Cuda => "CUDA",
|
||||||
Rung::TensorRt => "TensorRT",
|
Rung::TensorRt => "TensorRT",
|
||||||
|
Rung::MiGraphX => "MIGraphX",
|
||||||
Rung::Hexagon => "Hexagon NPU",
|
Rung::Hexagon => "Hexagon NPU",
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -88,13 +96,13 @@ impl Rung {
|
|||||||
fn fallback(self) -> Rung {
|
fn fallback(self) -> Rung {
|
||||||
match self {
|
match self {
|
||||||
Rung::TensorRt => Rung::Cuda,
|
Rung::TensorRt => Rung::Cuda,
|
||||||
Rung::Hexagon | Rung::Cuda | Rung::Cpu => Rung::Cpu,
|
Rung::MiGraphX | Rung::Hexagon | Rung::Cuda | Rung::Cpu => Rung::Cpu,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Whether a session on this rung needs an engine built first.
|
/// Whether a session on this rung needs an engine built first.
|
||||||
fn compiles(self) -> bool {
|
fn compiles(self) -> bool {
|
||||||
matches!(self, Rung::TensorRt | Rung::Hexagon)
|
matches!(self, Rung::TensorRt | Rung::MiGraphX | Rung::Hexagon)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The model form this rung wants for a role.
|
/// The model form this rung wants for a role.
|
||||||
@@ -164,7 +172,7 @@ impl Status {
|
|||||||
pub fn line(&self) -> String {
|
pub fn line(&self) -> String {
|
||||||
let form = match self.rung {
|
let form = match self.rung {
|
||||||
Rung::Hexagon => " · int8",
|
Rung::Hexagon => " · int8",
|
||||||
Rung::TensorRt => " · fp16",
|
Rung::TensorRt | Rung::MiGraphX => " · fp16",
|
||||||
_ => "",
|
_ => "",
|
||||||
};
|
};
|
||||||
format!("{}{} · {}", self.rung.label(), form, self.runtime.label())
|
format!("{}{} · {}", self.rung.label(), form, self.runtime.label())
|
||||||
@@ -269,8 +277,8 @@ fn acquire(role: Role, form: Form, bytes: &Arc<[u8]>, hash: u64) -> Result<Acqui
|
|||||||
return Ok(Acquired { entry });
|
return Ok(Acquired { entry });
|
||||||
}
|
}
|
||||||
|
|
||||||
// Built outside the registry lock: a TensorRT engine load is long enough
|
// Built outside the registry lock: a TensorRT or MIGraphX engine load
|
||||||
// that another role's acquire should not wait on it.
|
// is long enough that another role's acquire should not wait on it.
|
||||||
let session = session::build(rung, role, bytes, &cfg)?;
|
let session = session::build(rung, role, bytes, &cfg)?;
|
||||||
log::debug!("inference: {role:?} loaded on {}", rung.label());
|
log::debug!("inference: {role:?} loaded on {}", rung.label());
|
||||||
let entry = Arc::new(Loaded {
|
let entry = Arc::new(Loaded {
|
||||||
@@ -401,7 +409,16 @@ pub fn status() -> Status {
|
|||||||
runtime: api::runtime(),
|
runtime: api::runtime(),
|
||||||
rung,
|
rung,
|
||||||
reason: s.cache.reason.clone(),
|
reason: s.cache.reason.clone(),
|
||||||
failed: s.cache.failed.clone(),
|
// Only what explains the selection: on an AMD machine the NVIDIA
|
||||||
|
// rungs "not enabled in this build" say nothing about why MIGraphX
|
||||||
|
// was taken. With the floor selected, everything tried is above it.
|
||||||
|
failed: s
|
||||||
|
.cache
|
||||||
|
.failed
|
||||||
|
.iter()
|
||||||
|
.filter(|(r, _)| *r > rung)
|
||||||
|
.cloned()
|
||||||
|
.collect(),
|
||||||
probing: s.probing,
|
probing: s.probing,
|
||||||
engines: if rung.compiles() {
|
engines: if rung.compiles() {
|
||||||
(s.cache.compiled.len(), s.wanted)
|
(s.cache.compiled.len(), s.wanted)
|
||||||
@@ -499,7 +516,7 @@ mod tests {
|
|||||||
|
|
||||||
/// The smallest shipped graph, if this checkout has the weights; a test
|
/// The smallest shipped graph, if this checkout has the weights; a test
|
||||||
/// suite that needs a research-licensed download is one that does not
|
/// suite that needs a research-licensed download is one that does not
|
||||||
/// run in CI (docs/faces.md §3), so absence is a skip.
|
/// run in CI (docs/dev/faces.md §3), so absence is a skip.
|
||||||
fn probe_bytes() -> Option<Vec<u8>> {
|
fn probe_bytes() -> Option<Vec<u8>> {
|
||||||
let path = concat!(
|
let path = concat!(
|
||||||
env!("CARGO_MANIFEST_DIR"),
|
env!("CARGO_MANIFEST_DIR"),
|
||||||
@@ -614,8 +631,33 @@ mod tests {
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn the_status_reports_only_the_rungs_above_the_selection() {
|
||||||
|
let _serial = serial();
|
||||||
|
let failed = vec![
|
||||||
|
(Rung::TensorRt, "not enabled".to_string()),
|
||||||
|
(Rung::Cuda, "not enabled".to_string()),
|
||||||
|
];
|
||||||
|
let before = state().lock().unwrap().cache.clone();
|
||||||
|
state().lock().unwrap().cache = Cache {
|
||||||
|
rung: Some(Rung::MiGraphX),
|
||||||
|
failed: failed.clone(),
|
||||||
|
..Cache::default()
|
||||||
|
};
|
||||||
|
// An AMD desktop: the NVIDIA rungs below MIGraphX are not the story.
|
||||||
|
assert!(status().failed.is_empty());
|
||||||
|
// An NVIDIA desktop on the CUDA provider: TensorRT's failure is.
|
||||||
|
state().lock().unwrap().cache.rung = Some(Rung::Cuda);
|
||||||
|
assert_eq!(status().failed, vec![failed[0].clone()]);
|
||||||
|
// The floor: everything tried explains it.
|
||||||
|
state().lock().unwrap().cache.rung = Some(Rung::Cpu);
|
||||||
|
assert_eq!(status().failed.len(), 2);
|
||||||
|
state().lock().unwrap().cache = before;
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn the_status_line_reads_as_the_floor_before_init() {
|
fn the_status_line_reads_as_the_floor_before_init() {
|
||||||
|
let _serial = serial();
|
||||||
let s = status();
|
let s = status();
|
||||||
assert_eq!(s.rung, Rung::Cpu);
|
assert_eq!(s.rung, Rung::Cpu);
|
||||||
assert!(s.line().starts_with("CPU"), "{}", s.line());
|
assert!(s.line().starts_with("CPU"), "{}", s.line());
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
//! Walk the ladder, once, by building real sessions (docs/inference.md §4).
|
//! Walk the ladder, once, by building real sessions (docs/dev/inference.md §4).
|
||||||
//!
|
//!
|
||||||
//! A rung is taken when a session builds on it, runs, and is faster than
|
//! A rung is taken when a session builds on it, runs, and is faster than
|
||||||
//! the floor. Both halves matter: a provider can register and then fail at
|
//! the floor. Both halves matter: a provider can register and then fail at
|
||||||
@@ -16,8 +16,11 @@ use crate::{api::Runtime, state, Cache, Config, Form, Role, Rung};
|
|||||||
fn ladder(ceiling: Option<Rung>) -> Vec<Rung> {
|
fn ladder(ceiling: Option<Rung>) -> Vec<Rung> {
|
||||||
#[cfg(target_os = "android")]
|
#[cfg(target_os = "android")]
|
||||||
let all = [Rung::Hexagon];
|
let all = [Rung::Hexagon];
|
||||||
|
// A desktop has one vendor's GPU; the other vendor's providers are
|
||||||
|
// "not enabled in this build" or a library that fails to load, and
|
||||||
|
// either answer arrives in milliseconds.
|
||||||
#[cfg(not(target_os = "android"))]
|
#[cfg(not(target_os = "android"))]
|
||||||
let all = [Rung::TensorRt, Rung::Cuda];
|
let all = [Rung::TensorRt, Rung::Cuda, Rung::MiGraphX];
|
||||||
all.into_iter()
|
all.into_iter()
|
||||||
.filter(|r| ceiling.is_none_or(|c| *r <= c))
|
.filter(|r| ceiling.is_none_or(|c| *r <= c))
|
||||||
.collect()
|
.collect()
|
||||||
@@ -206,15 +209,22 @@ fn first_line(s: &str) -> String {
|
|||||||
line[start..].chars().take(200).collect()
|
line[start..].chars().take(200).collect()
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Everything a change of which should re-probe: the runtime and where it
|
/// Everything a change of which should re-probe: the runtime, where it
|
||||||
/// came from, this crate, the platform, the driver or SoC, and the models.
|
/// came from and which providers sit beside it, this crate, the platform,
|
||||||
|
/// the driver or SoC, and the models.
|
||||||
fn fingerprint(runtime: &Runtime, cfg: &Config) -> String {
|
fn fingerprint(runtime: &Runtime, cfg: &Config) -> String {
|
||||||
let mut parts = vec![
|
let mut parts = vec![
|
||||||
format!("engine {}", env!("CARGO_PKG_VERSION")),
|
format!("engine {}", env!("CARGO_PKG_VERSION")),
|
||||||
format!("{} {}", std::env::consts::OS, std::env::consts::ARCH),
|
format!("{} {}", std::env::consts::OS, std::env::consts::ARCH),
|
||||||
match runtime {
|
match runtime {
|
||||||
Runtime::Tract => "tract".to_string(),
|
Runtime::Tract => "tract".to_string(),
|
||||||
Runtime::OnnxRuntime { path, version } => format!("ort {version} {}", path.display()),
|
Runtime::OnnxRuntime { path, version } => {
|
||||||
|
format!(
|
||||||
|
"ort {version} {} [{}]",
|
||||||
|
path.display(),
|
||||||
|
providers_beside(path)
|
||||||
|
)
|
||||||
|
}
|
||||||
},
|
},
|
||||||
device_identity(),
|
device_identity(),
|
||||||
];
|
];
|
||||||
@@ -237,13 +247,41 @@ fn fingerprint(runtime: &Runtime, cfg: &Config) -> String {
|
|||||||
parts.join("\n")
|
parts.join("\n")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The `libonnxruntime_providers_*.so` files in the runtime's directory.
|
||||||
|
/// A distribution's CPU-only and ROCm builds are the same version at the
|
||||||
|
/// same path; the provider libraries beside them are what differs.
|
||||||
|
fn providers_beside(runtime: &Path) -> String {
|
||||||
|
let Some(dir) = runtime.parent() else {
|
||||||
|
return String::new();
|
||||||
|
};
|
||||||
|
let mut names: Vec<String> = std::fs::read_dir(dir)
|
||||||
|
.into_iter()
|
||||||
|
.flatten()
|
||||||
|
.filter_map(|e| e.ok())
|
||||||
|
.filter_map(|e| e.file_name().into_string().ok())
|
||||||
|
.filter(|n| {
|
||||||
|
n.starts_with("libonnxruntime_providers_") || n.starts_with("onnxruntime_providers_")
|
||||||
|
})
|
||||||
|
.collect();
|
||||||
|
names.sort();
|
||||||
|
names.join(" ")
|
||||||
|
}
|
||||||
|
|
||||||
#[cfg(target_os = "linux")]
|
#[cfg(target_os = "linux")]
|
||||||
fn device_identity() -> String {
|
fn device_identity() -> String {
|
||||||
// The NVIDIA driver's version line; absent means no NVIDIA driver.
|
// The NVIDIA driver's version line, or the ROCm release the AMD stack
|
||||||
std::fs::read_to_string("/proc/driver/nvidia/version")
|
// came from (`rocm-core` writes it; the kernel driver has no version
|
||||||
|
// of its own). Absent means neither.
|
||||||
|
if let Some(line) = std::fs::read_to_string("/proc/driver/nvidia/version")
|
||||||
.ok()
|
.ok()
|
||||||
.and_then(|s| s.lines().next().map(str::to_string))
|
.and_then(|s| s.lines().next().map(str::to_string))
|
||||||
.unwrap_or_else(|| "no nvidia driver".into())
|
{
|
||||||
|
return line;
|
||||||
|
}
|
||||||
|
if let Ok(rocm) = std::fs::read_to_string("/opt/rocm/.info/version") {
|
||||||
|
return format!("rocm {}", rocm.trim());
|
||||||
|
}
|
||||||
|
"no nvidia driver, no rocm".into()
|
||||||
}
|
}
|
||||||
|
|
||||||
#[cfg(target_os = "android")]
|
#[cfg(target_os = "android")]
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
//! One session builder per rung (docs/inference.md §2, §7, §9).
|
//! One session builder per rung (docs/dev/inference.md §2, §7, §9).
|
||||||
|
|
||||||
use ort::session::Session;
|
use ort::session::Session;
|
||||||
|
|
||||||
@@ -83,10 +83,65 @@ fn providers(
|
|||||||
ep::CUDA::default().build(),
|
ep::CUDA::default().build(),
|
||||||
])?)
|
])?)
|
||||||
}
|
}
|
||||||
|
Rung::MiGraphX => {
|
||||||
|
// fp16 on the same terms as TensorRT (§7). MIGraphX compiles a
|
||||||
|
// program per graph — 20–60 s here — and keeps it in the cache
|
||||||
|
// directory, keyed on the graph, the GPU and its own version
|
||||||
|
// but not the precision: hence one directory per precision.
|
||||||
|
// The CPU takes any node it declines.
|
||||||
|
let fp16 = role != Role::Embedder;
|
||||||
|
let cache = cfg
|
||||||
|
.cache_dir
|
||||||
|
.join("migraphx")
|
||||||
|
.join(if fp16 { "fp16" } else { "f32" });
|
||||||
|
let _ = std::fs::create_dir_all(&cache);
|
||||||
|
let mut b = b;
|
||||||
|
migraphx(&mut b, fp16, &cache)?;
|
||||||
|
Ok(b)
|
||||||
|
}
|
||||||
Rung::Hexagon => unreachable!("the Hexagon rung is not on a desktop ladder"),
|
Rung::Hexagon => unreachable!("the Hexagon rung is not on a desktop ladder"),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Register MIGraphX through ONNX Runtime's generic key/value entry point.
|
||||||
|
///
|
||||||
|
/// `ort`'s own builder (`ep::MIGraphX`) fills the legacy
|
||||||
|
/// `OrtMIGraphXProviderOptions`, and 1.29 reads that struct for its
|
||||||
|
/// precision flags and nothing else — the compiled-program cache directory
|
||||||
|
/// is only a key in the generic map (`migraphx_model_cache_dir`), and
|
||||||
|
/// without it every session is a full compile. Registration through the
|
||||||
|
/// generic entry point needs no `ort` feature: it is one call on the API
|
||||||
|
/// table, which is why the crate's `ort` dependency names no AMD feature.
|
||||||
|
#[cfg(not(target_os = "android"))]
|
||||||
|
fn migraphx(
|
||||||
|
b: &mut ort::session::builder::SessionBuilder,
|
||||||
|
fp16: bool,
|
||||||
|
cache: &std::path::Path,
|
||||||
|
) -> ort::Result<()> {
|
||||||
|
use ort::AsPointer;
|
||||||
|
use std::ffi::CString;
|
||||||
|
let keys = [c"migraphx_fp16_enable", c"migraphx_model_cache_dir"];
|
||||||
|
let values = [
|
||||||
|
CString::new(if fp16 { "1" } else { "0" }).unwrap(),
|
||||||
|
CString::new(cache.to_string_lossy().as_bytes())
|
||||||
|
.map_err(|e| ort::Error::new(e.to_string()))?,
|
||||||
|
];
|
||||||
|
let key_ptrs: Vec<_> = keys.iter().map(|k| k.as_ptr()).collect();
|
||||||
|
let value_ptrs: Vec<_> = values.iter().map(|v| v.as_ptr()).collect();
|
||||||
|
// SAFETY: the documented C call over arrays that outlive it; the
|
||||||
|
// runtime copies the strings into its own options map before returning.
|
||||||
|
unsafe {
|
||||||
|
let status = (ort::api().SessionOptionsAppendExecutionProvider)(
|
||||||
|
b.ptr_mut(),
|
||||||
|
c"MIGraphX".as_ptr(),
|
||||||
|
key_ptrs.as_ptr(),
|
||||||
|
value_ptrs.as_ptr(),
|
||||||
|
keys.len(),
|
||||||
|
);
|
||||||
|
ort::Error::result_from_status(status)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
#[cfg(target_os = "android")]
|
#[cfg(target_os = "android")]
|
||||||
fn providers(
|
fn providers(
|
||||||
b: ort::session::builder::SessionBuilder,
|
b: ort::session::builder::SessionBuilder,
|
||||||
@@ -120,6 +175,8 @@ fn providers(
|
|||||||
.build()
|
.build()
|
||||||
.error_on_failure()])?)
|
.error_on_failure()])?)
|
||||||
}
|
}
|
||||||
Rung::Cuda | Rung::TensorRt => unreachable!("no NVIDIA rung on Android"),
|
Rung::Cuda | Rung::TensorRt | Rung::MiGraphX => {
|
||||||
|
unreachable!("no desktop GPU rung on Android")
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -13,7 +13,7 @@ log.workspace = true
|
|||||||
|
|
||||||
# Inference for the learned keypoint detector, on the same footing as
|
# Inference for the learned keypoint detector, on the same footing as
|
||||||
# `dr-segment`: `ort` is the API, `dr-inference-engine` decides what runs
|
# `dr-segment`: `ort` is the API, `dr-inference-engine` decides what runs
|
||||||
# it (docs/inference.md), and both are optional so that the geometry —
|
# it (docs/dev/inference.md), and both are optional so that the geometry —
|
||||||
# matching, the rotation solve, the projections — is a dependency-free crate
|
# matching, the rotation solve, the projections — is a dependency-free crate
|
||||||
# that tests without a model.
|
# that tests without a model.
|
||||||
ort = { workspace = true, optional = true }
|
ort = { workspace = true, optional = true }
|
||||||
|
|||||||
+62
-10
@@ -95,7 +95,7 @@ pub struct Params {
|
|||||||
/// and, beyond the band being filled, still unknown. That is what the
|
/// and, beyond the band being filled, still unknown. That is what the
|
||||||
/// shipped model was trained on (a fine-tune of MI-GAN on voids cut
|
/// shipped model was trained on (a fine-tune of MI-GAN on voids cut
|
||||||
/// from photographs the way a cylindrical merge cuts them, see
|
/// from photographs the way a cylindrical merge cuts them, see
|
||||||
/// `docs/panorama.md` §14); a ring would give it a fold to continue.
|
/// `docs/dev/panorama.md` §14); a ring would give it a fold to continue.
|
||||||
///
|
///
|
||||||
/// Non-zero is the stock model's crutch: a plain reflection of a deep
|
/// Non-zero is the stock model's crutch: a plain reflection of a deep
|
||||||
/// hole pulls in whatever is that far from the edge — a ridge, a peak —
|
/// hole pulls in whatever is that far from the edge — a ridge, a peak —
|
||||||
@@ -303,7 +303,7 @@ fn fill_once(
|
|||||||
}
|
}
|
||||||
|
|
||||||
// The padded canvas with mirrored context, and the hole within it.
|
// The padded canvas with mirrored context, and the hole within it.
|
||||||
let ctx = MirroredContext::build(rgb, width, height, known, mirror_depth);
|
let ctx = MirroredContext::build(rgb, width, height, known, mirror_depth, t);
|
||||||
let (pw, ph) = (ctx.width, ctx.height);
|
let (pw, ph) = (ctx.width, ctx.height);
|
||||||
|
|
||||||
// Tiles that touch the hole, on a grid that reaches both far edges.
|
// Tiles that touch the hole, on a grid that reaches both far edges.
|
||||||
@@ -499,16 +499,36 @@ struct MirroredContext {
|
|||||||
}
|
}
|
||||||
|
|
||||||
impl MirroredContext {
|
impl MirroredContext {
|
||||||
fn build(rgb: &[f32], width: usize, height: usize, known: &[bool], depth: usize) -> Self {
|
fn build(
|
||||||
|
rgb: &[f32],
|
||||||
|
width: usize,
|
||||||
|
height: usize,
|
||||||
|
known: &[bool],
|
||||||
|
depth: usize,
|
||||||
|
tile: usize,
|
||||||
|
) -> Self {
|
||||||
if depth == 0 {
|
if depth == 0 {
|
||||||
// Open: the picture as it is, the hole as it is. What the hole
|
// Open: the picture as it is, the hole as it is. What the hole
|
||||||
// holds does not matter — the model masks it out.
|
// holds does not matter — the model masks it out. A picture
|
||||||
|
// smaller than a tile (the merge page's preview) sits at the
|
||||||
|
// origin of a tile-sized canvas whose rest is hole: still the
|
||||||
|
// void as it is, and the only way a tile fits at all.
|
||||||
|
let (pw, ph) = (width.max(tile), height.max(tile));
|
||||||
|
let mut canvas = vec![0.0f32; pw * ph * 3];
|
||||||
|
let mut hole = vec![true; pw * ph];
|
||||||
|
for y in 0..height {
|
||||||
|
canvas[y * pw * 3..(y * pw + width) * 3]
|
||||||
|
.copy_from_slice(&rgb[y * width * 3..(y + 1) * width * 3]);
|
||||||
|
for x in 0..width {
|
||||||
|
hole[y * pw + x] = !known[y * width + x];
|
||||||
|
}
|
||||||
|
}
|
||||||
return MirroredContext {
|
return MirroredContext {
|
||||||
width,
|
width: pw,
|
||||||
height,
|
height: ph,
|
||||||
ring: 0,
|
ring: 0,
|
||||||
rgb: rgb.to_vec(),
|
rgb: canvas,
|
||||||
hole: known.iter().map(|&k| !k).collect(),
|
hole,
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
let fold = |d: usize| fold(d, depth);
|
let fold = |d: usize| fold(d, depth);
|
||||||
@@ -715,6 +735,38 @@ mod tests {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_picture_smaller_than_the_tile_is_still_filled_when_the_void_is_open() {
|
||||||
|
// The merge page's preview is 1600 wide and a few hundred tall —
|
||||||
|
// shorter than a 512 tile. With no ring the canvas is padded to a
|
||||||
|
// tile, the padding hole, and the border is still filled.
|
||||||
|
let (mut rgb, known) = picture(300, 40, 8);
|
||||||
|
let mut model = Flat {
|
||||||
|
tile: 64,
|
||||||
|
seen: Vec::new(),
|
||||||
|
};
|
||||||
|
let tiles = fill_border(
|
||||||
|
&mut rgb,
|
||||||
|
300,
|
||||||
|
40,
|
||||||
|
&known,
|
||||||
|
&mut model,
|
||||||
|
test_params(0),
|
||||||
|
&mut |_, _| {},
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
assert!(tiles > 0, "no tile fitted a picture shorter than the tile");
|
||||||
|
for i in 0..300 * 40 {
|
||||||
|
if !known[i] {
|
||||||
|
assert!((rgb[i * 3] - 0.5).abs() < 1e-4, "pixel {i}");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// And the model saw the padding as hole, never as black content.
|
||||||
|
for (_, k) in &model.seen {
|
||||||
|
assert_eq!(k.len(), 64 * 64);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn the_fine_passes_run_in_bands_after_the_coarse_one() {
|
fn the_fine_passes_run_in_bands_after_the_coarse_one() {
|
||||||
// A 150-tall hole above and below a picture: the coarse pass sees
|
// A 150-tall hole above and below a picture: the coarse pass sees
|
||||||
@@ -824,7 +876,7 @@ mod tests {
|
|||||||
#[test]
|
#[test]
|
||||||
fn the_context_mirrors_the_top_rows_upward() {
|
fn the_context_mirrors_the_top_rows_upward() {
|
||||||
let (rgb, known) = picture(40, 30, 5);
|
let (rgb, known) = picture(40, 30, 5);
|
||||||
let ctx = MirroredContext::build(&rgb, 40, 30, &known, 48);
|
let ctx = MirroredContext::build(&rgb, 40, 30, &known, 48, 64);
|
||||||
let x = RING + 10;
|
let x = RING + 10;
|
||||||
let first = RING + 5;
|
let first = RING + 5;
|
||||||
for k in 1..=4 {
|
for k in 1..=4 {
|
||||||
@@ -846,7 +898,7 @@ mod tests {
|
|||||||
for x in 0..40 {
|
for x in 0..40 {
|
||||||
rgb[(ridge * 40 + x) * 3..(ridge * 40 + x) * 3 + 3].copy_from_slice(&[0.9, 0.1, 0.1]);
|
rgb[(ridge * 40 + x) * 3..(ridge * 40 + x) * 3 + 3].copy_from_slice(&[0.9, 0.1, 0.1]);
|
||||||
}
|
}
|
||||||
let ctx = MirroredContext::build(&rgb, 40, 400, &known, 48);
|
let ctx = MirroredContext::build(&rgb, 40, 400, &known, 48, 64);
|
||||||
let x = RING + 10;
|
let x = RING + 10;
|
||||||
for y in 0..RING + 5 {
|
for y in 0..RING + 5 {
|
||||||
let p = (y * ctx.width + x) * 3;
|
let p = (y * ctx.width + x) * 3;
|
||||||
|
|||||||
@@ -9,7 +9,7 @@
|
|||||||
//! on every rung, and what they cost is the whole story of whether a fill
|
//! on every rung, and what they cost is the whole story of whether a fill
|
||||||
//! is interactive: 7.4 s a tile under tract, 0.4 s under ONNX Runtime's
|
//! is interactive: 7.4 s a tile under tract, 0.4 s under ONNX Runtime's
|
||||||
//! CPU pool, 23 ms in fp16 and 13 ms in int8 on a laptop's TensorRT
|
//! CPU pool, 23 ms in fp16 and 13 ms in int8 on a laptop's TensorRT
|
||||||
//! (2026-09-19, docs/panorama.md §12).
|
//! (2026-09-19, docs/dev/panorama.md §12).
|
||||||
//!
|
//!
|
||||||
//! The model's contract, from the reference `export_inference_model.py`:
|
//! The model's contract, from the reference `export_inference_model.py`:
|
||||||
//! input `1×4×512×512` float — channel 0 is `mask − 0.5` with 1 where the
|
//! input `1×4×512×512` float — channel 0 is `mask − 0.5` with 1 where the
|
||||||
|
|||||||
@@ -4,7 +4,7 @@
|
|||||||
//! Apache-2.0 weights (`models/LICENCE.md`), exported at a fixed shape by
|
//! Apache-2.0 weights (`models/LICENCE.md`), exported at a fixed shape by
|
||||||
//! `tools/export-xfeat.sh` and loaded through the same `dr-inference-engine`
|
//! `tools/export-xfeat.sh` and loaded through the same `dr-inference-engine`
|
||||||
//! `dr-segment` and `dr-face` use, so this adds no runtime and no C to the
|
//! `dr-segment` and `dr-face` use, so this adds no runtime and no C to the
|
||||||
//! tree; what runs it is the device's business (docs/inference.md). ~300 ms
|
//! tree; what runs it is the device's business (docs/dev/inference.md). ~300 ms
|
||||||
//! per frame on tract on the reference desktop, ~400 ms on the tablet
|
//! per frame on tract on the reference desktop, ~400 ms on the tablet
|
||||||
//! (S15.2, S15.4).
|
//! (S15.2, S15.4).
|
||||||
|
|
||||||
@@ -37,7 +37,7 @@ pub struct XFeat {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/// The bytes of both exports compiled into the binary, for whoever compiles
|
/// The bytes of both exports compiled into the binary, for whoever compiles
|
||||||
/// engines ahead of the first request (docs/inference.md §6).
|
/// engines ahead of the first request (docs/dev/inference.md §6).
|
||||||
#[cfg(feature = "embedded-model")]
|
#[cfg(feature = "embedded-model")]
|
||||||
pub fn embedded_model_bytes() -> [&'static [u8]; 2] {
|
pub fn embedded_model_bytes() -> [&'static [u8]; 2] {
|
||||||
[EMBEDDED_LANDSCAPE, EMBEDDED_PORTRAIT]
|
[EMBEDDED_LANDSCAPE, EMBEDDED_PORTRAIT]
|
||||||
|
|||||||
@@ -315,7 +315,7 @@ pub struct DetailPass {
|
|||||||
/// 52 render pixels at 4K — holds no spatial frequency a quarter-scale
|
/// 52 render pixels at 4K — holds no spatial frequency a quarter-scale
|
||||||
/// grid cannot represent. Computing it at the render size therefore buys
|
/// grid cannot represent. Computing it at the render size therefore buys
|
||||||
/// nothing and costs everything: 105 taps over 8.3 M pixels, twice, which
|
/// nothing and costs everything: 105 taps over 8.3 M pixels, twice, which
|
||||||
/// measured at 34 ms and is where `docs/technical-debt.md` TD-4 came from.
|
/// measured at 34 ms and is where `docs/dev/technical-debt.md` TD-4 came from.
|
||||||
/// At a quarter it is a sixteenth of the pixels at a quarter of the
|
/// At a quarter it is a sixteenth of the pixels at a quarter of the
|
||||||
/// radius, and the result is not an approximation of the full-resolution
|
/// radius, and the result is not an approximation of the full-resolution
|
||||||
/// base — it is the same band-limited function, sampled where it is still
|
/// base — it is the same band-limited function, sampled where it is still
|
||||||
@@ -568,7 +568,7 @@ pub fn compose_detail(
|
|||||||
/// photograph the photographer thinks they are sharpening.
|
/// photograph the photographer thinks they are sharpening.
|
||||||
///
|
///
|
||||||
/// It also means ARCH §5.2's stage list, which draws spot removal after
|
/// It also means ARCH §5.2's stage list, which draws spot removal after
|
||||||
/// texture and clarity, is not what this does — see `docs/spot-removal.md`
|
/// texture and clarity, is not what this does — see `docs/dev/spot-removal.md`
|
||||||
/// §5.1, which is where the disagreement is written down.
|
/// §5.1, which is where the disagreement is written down.
|
||||||
pub fn compose_detail_with(
|
pub fn compose_detail_with(
|
||||||
ops: &[Box<dyn Operation>],
|
ops: &[Box<dyn Operation>],
|
||||||
|
|||||||
@@ -113,7 +113,7 @@ pub struct EditGraph {
|
|||||||
/// a sidecar comes to name one stock while the shader draws another.
|
/// a sidecar comes to name one stock while the shader draws another.
|
||||||
film: Option<Film>,
|
film: Option<Film>,
|
||||||
/// TRACES: FR-DEV-8
|
/// TRACES: FR-DEV-8
|
||||||
/// The repairs (`docs/spot-removal.md`).
|
/// The repairs (`docs/dev/spot-removal.md`).
|
||||||
///
|
///
|
||||||
/// Apart from `ops` for the third time and the same reason: a spot is not
|
/// Apart from `ops` for the third time and the same reason: a spot is not
|
||||||
/// a scalar, and a list of them is not a slider. It sits beside the masks
|
/// a scalar, and a list of them is not a slider. It sits beside the masks
|
||||||
@@ -122,7 +122,7 @@ pub struct EditGraph {
|
|||||||
/// photograph comes from.
|
/// photograph comes from.
|
||||||
spots: SpotSet,
|
spots: SpotSet,
|
||||||
/// The lens corrections that rewrite coordinates: distortion and lateral
|
/// The lens corrections that rewrite coordinates: distortion and lateral
|
||||||
/// chromatic aberration (`docs/architecture.md` §5.2).
|
/// chromatic aberration (`docs/dev/architecture.md` §5.2).
|
||||||
///
|
///
|
||||||
/// Apart from `ops` for the fourth time, and this one is not about shape
|
/// Apart from `ops` for the fourth time, and this one is not about shape
|
||||||
/// but about direction. Every [`Operation`] is a function from colour to
|
/// but about direction. Every [`Operation`] is a function from colour to
|
||||||
@@ -857,6 +857,39 @@ impl EditGraph {
|
|||||||
crate::operation::compose_camera_linear(&self.warps, self.framing.baseline(), view)
|
crate::operation::compose_camera_linear(&self.warps, self.framing.baseline(), view)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// TRACES: FR-DEV-3
|
||||||
|
/// The camera-space tap over one patch of what is on the canvas.
|
||||||
|
///
|
||||||
|
/// `patch` is in fractions of the visible region — the coordinates a
|
||||||
|
/// click on the canvas arrives in — and is laid over this edit's own
|
||||||
|
/// framing: crop, view, rotation and all, so a fraction of the canvas is
|
||||||
|
/// a fraction of the probe. Nothing else of the edit: no operation, no
|
||||||
|
/// mask, no repair. Rendering only the patch is what lets a small target
|
||||||
|
/// cover every sensor pixel under it rather than sampling one in fifty;
|
||||||
|
/// see [`crate::operation::compose_camera_probe`] for why the white
|
||||||
|
/// balance picker reads from here and not from the display.
|
||||||
|
///
|
||||||
|
/// The patch is centred where asked and held to the view's own minimum
|
||||||
|
/// extent: at a deep zoom a patch a fraction of the view would be
|
||||||
|
/// smaller than a view may be, and letting `set_view` widen it from one
|
||||||
|
/// corner would move the sample off the point that was clicked.
|
||||||
|
pub fn compose_camera_probe(&self, patch: crate::framing::CropRect) -> ComposedShader {
|
||||||
|
use crate::framing::CropRect;
|
||||||
|
let mut framing = self.framing;
|
||||||
|
let view = framing.view();
|
||||||
|
let width = (patch.width * view.width).max(CropRect::MIN_EXTENT);
|
||||||
|
let height = (patch.height * view.height).max(CropRect::MIN_EXTENT);
|
||||||
|
let cx = view.x + (patch.x + patch.width * 0.5) * view.width;
|
||||||
|
let cy = view.y + (patch.y + patch.height * 0.5) * view.height;
|
||||||
|
framing.set_view(CropRect {
|
||||||
|
x: (cx - width * 0.5).clamp(0.0, 1.0 - width),
|
||||||
|
y: (cy - height * 0.5).clamp(0.0, 1.0 - height),
|
||||||
|
width,
|
||||||
|
height,
|
||||||
|
});
|
||||||
|
crate::operation::compose_camera_probe(&self.warps, &framing)
|
||||||
|
}
|
||||||
|
|
||||||
/// TRACES: FR-DEV-19c
|
/// TRACES: FR-DEV-19c
|
||||||
/// [`Self::compose_for`], with one layer's mask drawn over the picture.
|
/// [`Self::compose_for`], with one layer's mask drawn over the picture.
|
||||||
///
|
///
|
||||||
|
|||||||
@@ -65,7 +65,8 @@ pub use history::{Edit, Entry as HistoryEntry, History, Step};
|
|||||||
pub use lens::{compose_warps, ComposedWarp, LensProfile, Tca, Warp};
|
pub use lens::{compose_warps, ComposedWarp, LensProfile, Tca, Warp};
|
||||||
pub use operation::{
|
pub use operation::{
|
||||||
compose, compose_with_framing, Affects, ComposedShader, Helper, Invalidation, Operation,
|
compose, compose_with_framing, Affects, ComposedShader, Helper, Invalidation, Operation,
|
||||||
OutputMode, Uniform, BASE_CURVE_POINTS, BASE_CURVE_UNIFORM_OFFSET, RESERVED_UNIFORM_FIELDS,
|
OutputMode, Uniform, BASE_CURVE_POINTS, BASE_CURVE_UNIFORM_OFFSET, CLIP_ONSET,
|
||||||
|
RESERVED_UNIFORM_FIELDS,
|
||||||
};
|
};
|
||||||
pub use preset::{LibraryParseError, NameError, Preset, PresetLibrary, Scope};
|
pub use preset::{LibraryParseError, NameError, Preset, PresetLibrary, Scope};
|
||||||
pub use sidecar::{Sidecar, Version};
|
pub use sidecar::{Sidecar, Version};
|
||||||
|
|||||||
@@ -27,7 +27,7 @@
|
|||||||
//! [`MaskSource::Regions`] stores integers naming regions in the segmentation
|
//! [`MaskSource::Regions`] stores integers naming regions in the segmentation
|
||||||
//! hierarchy (`dr-segment`). That choice is what makes a mask diffable, cheap
|
//! hierarchy (`dr-segment`). That choice is what makes a mask diffable, cheap
|
||||||
//! in a sidecar, and mergeable per-field under FR-NC-9 — three properties a
|
//! in a sidecar, and mergeable per-field under FR-NC-9 — three properties a
|
||||||
//! stored raster has none of (docs/segmentation.md §1). Two devices that
|
//! stored raster has none of (docs/dev/segmentation.md §1). Two devices that
|
||||||
//! select the same subject produce the same small sorted list, and a sync
|
//! select the same subject produce the same small sorted list, and a sync
|
||||||
//! conflict between them is resolvable rather than a binary blob fight.
|
//! conflict between them is resolvable rather than a binary blob fight.
|
||||||
//!
|
//!
|
||||||
@@ -564,7 +564,7 @@ pub enum MaskSource {
|
|||||||
/// This is what the watershed and the semantic model exist to produce.
|
/// This is what the watershed and the semantic model exist to produce.
|
||||||
/// Selecting a subject means "the regions the model's instance covers",
|
/// Selecting a subject means "the regions the model's instance covers",
|
||||||
/// and the resulting edge is the watershed's, which is to say the image's
|
/// and the resulting edge is the watershed's, which is to say the image's
|
||||||
/// own (docs/segmentation.md §5).
|
/// own (docs/dev/segmentation.md §5).
|
||||||
Regions {
|
Regions {
|
||||||
/// Which segmentation these ids index into.
|
/// Which segmentation these ids index into.
|
||||||
///
|
///
|
||||||
@@ -590,7 +590,7 @@ pub enum MaskSource {
|
|||||||
/// **The primary way a local adjustment is made.** The watershed hierarchy
|
/// **The primary way a local adjustment is made.** The watershed hierarchy
|
||||||
/// this crate was first built around does not survive a photograph: its
|
/// this crate was first built around does not survive a photograph: its
|
||||||
/// saddles are near zero almost everywhere, so a global cut collapses the
|
/// saddles are near zero almost everywhere, so a global cut collapses the
|
||||||
/// frame into one region plus noise (docs/segmentation.md §15). A model
|
/// frame into one region plus noise (docs/dev/segmentation.md §15). A model
|
||||||
/// instance is a whole object, found as one thing, and needs no ladder.
|
/// instance is a whole object, found as one thing, and needs no ladder.
|
||||||
///
|
///
|
||||||
/// The trade is that the boundary is the model's — a quarter-resolution
|
/// The trade is that the boundary is the model's — a quarter-resolution
|
||||||
@@ -625,7 +625,7 @@ pub enum MaskSource {
|
|||||||
/// reason both exist. A subject is *one* instance — this dog, not that one
|
/// reason both exist. A subject is *one* instance — this dog, not that one
|
||||||
/// — found by a COCO-trained instance model. A category is *all* the sky,
|
/// — found by a COCO-trained instance model. A category is *all* the sky,
|
||||||
/// or all the foliage, from an ADE20K-trained semantic model that has no
|
/// or all the foliage, from an ADE20K-trained semantic model that has no
|
||||||
/// notion of instances at all (docs/segmentation.md §16).
|
/// notion of instances at all (docs/dev/segmentation.md §16).
|
||||||
///
|
///
|
||||||
/// So this is what a global grade attaches to: lift the sky, desaturate
|
/// So this is what a global grade attaches to: lift the sky, desaturate
|
||||||
/// the vegetation, warm the architecture. Asking it for "that person
|
/// the vegetation, warm the architecture. Asking it for "that person
|
||||||
@@ -2231,7 +2231,7 @@ fn reveal_block(slot: usize, layer: &MaskLayer, style: RevealStyle, colour: [f32
|
|||||||
/// **not** a hash of the label field: that would be a readback on a path that
|
/// **not** a hash of the label field: that would be a readback on a path that
|
||||||
/// must not have one (ARCH §6.1), and would also make the signature depend on
|
/// must not have one (ARCH §6.1), and would also make the signature depend on
|
||||||
/// float arithmetic whose cross-vendor determinism is exactly the open
|
/// float arithmetic whose cross-vendor determinism is exactly the open
|
||||||
/// question (docs/segmentation.md §6, M5).
|
/// question (docs/dev/segmentation.md §6, M5).
|
||||||
pub fn segmentation_signature(width: u32, height: u32, regions: u32, tuning: u64) -> u64 {
|
pub fn segmentation_signature(width: u32, height: u32, regions: u32, tuning: u64) -> u64 {
|
||||||
// FNV-1a over the four fields. Small, dependency-free, and adequate: this
|
// FNV-1a over the four fields. Small, dependency-free, and adequate: this
|
||||||
// guards against accidental mismatch, not against a forged sidecar.
|
// guards against accidental mismatch, not against a forged sidecar.
|
||||||
|
|||||||
@@ -69,20 +69,26 @@ const ROUNDS: u32 = 3;
|
|||||||
|
|
||||||
/// Below this a channel carries no ratio worth balancing.
|
/// Below this a channel carries no ratio worth balancing.
|
||||||
///
|
///
|
||||||
/// A sample in the deep shadows, or one taken on a blown highlight where a
|
/// A sample in the deep shadows has no white balance in it: the logarithms
|
||||||
/// channel has already clipped to nothing, has no white balance in it: the
|
/// below would run away and the picker would slam a slider to its stop.
|
||||||
/// logarithms below would run away and the picker would slam a slider to its
|
/// Refusing is the honest answer, and the caller reports that the point was
|
||||||
/// stop. Refusing is the honest answer, and the caller reports that the point
|
/// not usable rather than moving the photograph. The other end — a blown
|
||||||
/// was not usable rather than moving the photograph.
|
/// highlight, where every channel has stopped counting — is refused by the
|
||||||
|
/// caller before the sample is taken, because only the caller can see the
|
||||||
|
/// sensor value; see [`crate::operation::CLIP_ONSET`].
|
||||||
const FLOOR: f32 = 1e-4;
|
const FLOOR: f32 = 1e-4;
|
||||||
|
|
||||||
/// TRACES: FR-DEV-3
|
/// TRACES: FR-DEV-3
|
||||||
/// Move the graph so that `sample` renders neutral.
|
/// Move the graph so that `sample` renders neutral.
|
||||||
///
|
///
|
||||||
/// `sample` is linear RGB, as the operation's own gains multiply it — that is,
|
/// `sample` is the linear triple the operation's own gains multiply — camera
|
||||||
/// measured with the sampling operation at its defaults. Returns whether the
|
/// RGB with the camera's as-shot balance on, *before* the body's base curve
|
||||||
/// graph was moved: `false` where the chain offers no white point widget, or
|
/// and matrix, and with the sampling operation at its defaults. Not the
|
||||||
/// where the colour has no balance in it to correct.
|
/// pixel on the screen: the matrix mixes the channels on the way there, so
|
||||||
|
/// a colour read after it does not answer to these gains, and a solve over
|
||||||
|
/// one lands somewhere no sample asked for. Returns whether the graph was
|
||||||
|
/// moved: `false` where the chain offers no white point widget, or where the
|
||||||
|
/// colour has no balance in it to correct.
|
||||||
///
|
///
|
||||||
/// **Absolute, not relative.** The values written depend on the colour and not
|
/// **Absolute, not relative.** The values written depend on the colour and not
|
||||||
/// on where the sliders happened to be, so sampling the same wall twice lands
|
/// on where the sliders happened to be, so sampling the same wall twice lands
|
||||||
|
|||||||
@@ -54,7 +54,7 @@ pub enum Affects {
|
|||||||
/// A pixel's *neighbourhood* — sharpening, noise reduction, clarity,
|
/// A pixel's *neighbourhood* — sharpening, noise reduction, clarity,
|
||||||
/// texture, dehaze, spot removal.
|
/// texture, dehaze, spot removal.
|
||||||
///
|
///
|
||||||
/// The seam `docs/requirements.md` §3.3 designed and nothing cut until
|
/// The seam `docs/dev/requirements.md` §3.3 designed and nothing cut until
|
||||||
/// [`crate::detail`] existed. It is a separate variant rather than a flavour
|
/// [`crate::detail`] existed. It is a separate variant rather than a flavour
|
||||||
/// of `Colour` because it is a separate *dispatch*: a fragment in the fused
|
/// of `Colour` because it is a separate *dispatch*: a fragment in the fused
|
||||||
/// pass is handed a colour and has no way back to a coordinate, so a
|
/// pass is handed a colour and has no way back to a coordinate, so a
|
||||||
@@ -431,9 +431,11 @@ pub enum OutputMode {
|
|||||||
/// no operations, and the caller fills the reserved uniforms neutral —
|
/// no operations, and the caller fills the reserved uniforms neutral —
|
||||||
/// unit white balance, identity matrix, base curve off — so what is
|
/// unit white balance, identity matrix, base curve off — so what is
|
||||||
/// stored is the sensor's own numbers, demosaiced and undistorted. Only
|
/// stored is the sensor's own numbers, demosaiced and undistorted. Only
|
||||||
/// [`compose_camera_linear`] produces it, and only
|
/// [`compose_camera_probe`] produces it — for a merge through
|
||||||
/// `AdjustPass::render_camera_linear` accepts it, so the neutral
|
/// [`compose_camera_linear`], and for the white balance picker under the
|
||||||
/// uniforms cannot be forgotten by a caller that composed it by mistake.
|
/// edit's own framing — and only `AdjustPass::render_camera_linear`
|
||||||
|
/// accepts it, so the neutral uniforms cannot be forgotten by a caller
|
||||||
|
/// that composed it by mistake.
|
||||||
///
|
///
|
||||||
/// Thirty-two bits rather than sixteen because the composite is written
|
/// Thirty-two bits rather than sixteen because the composite is written
|
||||||
/// back as a RAW at the sensor's scale (FR-MRG-3): a 14-bit sensor has
|
/// back as a RAW at the sensor's scale (FR-MRG-3): a 14-bit sensor has
|
||||||
@@ -490,6 +492,19 @@ pub const BASE_CURVE_UNIFORM_OFFSET: usize = 16;
|
|||||||
/// selection in [`compose_full`].
|
/// selection in [`compose_full`].
|
||||||
pub const BASE_CURVE_POINTS: usize = 5;
|
pub const BASE_CURVE_POINTS: usize = 5;
|
||||||
|
|
||||||
|
/// Where the highlight desaturation begins: the fraction of the white level
|
||||||
|
/// above which a photosite is treated as clipped.
|
||||||
|
///
|
||||||
|
/// A photosite this close to saturation has stopped counting, so its ratio
|
||||||
|
/// to its neighbours is not a colour. The generated prologue fades a pixel
|
||||||
|
/// above this toward a neutral of the same brightness before any operation
|
||||||
|
/// runs, and the white balance probe refuses to sample one: a blown sky is
|
||||||
|
/// sensor white, which the as-shot multipliers make magenta, and a solve
|
||||||
|
/// over that slams tint to its stop. One number, so the two cannot drift
|
||||||
|
/// apart — a probe that accepted what the shader had already desaturated
|
||||||
|
/// would be balancing against a pixel the photographer cannot see.
|
||||||
|
pub const CLIP_ONSET: f32 = 0.985;
|
||||||
|
|
||||||
/// Where an operation's own uniforms begin in the generated block.
|
/// Where an operation's own uniforms begin in the generated block.
|
||||||
///
|
///
|
||||||
/// The base fields, then framing's. Exported because `dr-gpu` writes the
|
/// The base fields, then framing's. Exported because `dr-gpu` writes the
|
||||||
@@ -589,7 +604,9 @@ pub fn compose_full_revealing(
|
|||||||
warps: &[Box<dyn crate::lens::Warp>],
|
warps: &[Box<dyn crate::lens::Warp>],
|
||||||
reveal: Option<&crate::mask::Reveal>,
|
reveal: Option<&crate::mask::Reveal>,
|
||||||
) -> ComposedShader {
|
) -> ComposedShader {
|
||||||
compose_inner(ops, framing, output, masks, spots, warps, reveal, None)
|
compose_inner(
|
||||||
|
ops, framing, output, masks, spots, warps, reveal, None, false,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// TRACES: FR-MRG-2
|
/// TRACES: FR-MRG-2
|
||||||
@@ -614,15 +631,50 @@ pub fn compose_camera_linear(
|
|||||||
// upright too, and `view` is a fraction of the upright frame.
|
// upright too, and `view` is a fraction of the upright frame.
|
||||||
framing.set_baseline(baseline);
|
framing.set_baseline(baseline);
|
||||||
framing.set_view(view);
|
framing.set_view(view);
|
||||||
|
compose_camera_tap(warps, &framing, false)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// TRACES: FR-DEV-3
|
||||||
|
/// The same tap under a framing the caller chose: the white balance probe.
|
||||||
|
///
|
||||||
|
/// A neutral picked off the canvas has to be measured in the space the
|
||||||
|
/// white balance gains multiply, and that is camera RGB — the operation
|
||||||
|
/// runs before the body's matrix, and a probe read after the matrix would
|
||||||
|
/// be solving the wrong equation on any body whose matrix mixes the
|
||||||
|
/// channels, which is every body. It also has to be measured at the pixel
|
||||||
|
/// the canvas is showing, which is why this takes the edit's own framing
|
||||||
|
/// where a merge passes the file's orientation and a tile.
|
||||||
|
///
|
||||||
|
/// **Interpolated whatever the framing says.** The point of rendering a
|
||||||
|
/// patch is to average what is under it, and the nearest sampling an
|
||||||
|
/// unrotated frame otherwise gets is a comb: at two source pixels per
|
||||||
|
/// probe pixel it lands on the same column of any pattern every time, and
|
||||||
|
/// the average of a thousand samples is then the average of nothing.
|
||||||
|
pub fn compose_camera_probe(
|
||||||
|
warps: &[Box<dyn crate::lens::Warp>],
|
||||||
|
framing: &Framing,
|
||||||
|
) -> ComposedShader {
|
||||||
|
compose_camera_tap(warps, framing, true)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The camera-space tap proper: no operations, `rgba32float`, and the
|
||||||
|
/// profile uniforms left for the GPU side to fill neutral. `smooth` forces
|
||||||
|
/// the interpolating sampler; see the two callers for who wants it and why.
|
||||||
|
fn compose_camera_tap(
|
||||||
|
warps: &[Box<dyn crate::lens::Warp>],
|
||||||
|
framing: &Framing,
|
||||||
|
smooth: bool,
|
||||||
|
) -> ComposedShader {
|
||||||
compose_inner(
|
compose_inner(
|
||||||
&[],
|
&[],
|
||||||
&framing,
|
framing,
|
||||||
ColourSpace::Srgb,
|
ColourSpace::Srgb,
|
||||||
&MaskStack::new(),
|
&MaskStack::new(),
|
||||||
&crate::spot::SpotSet::new(),
|
&crate::spot::SpotSet::new(),
|
||||||
warps,
|
warps,
|
||||||
None,
|
None,
|
||||||
Some(OutputMode::CameraLinear),
|
Some(OutputMode::CameraLinear),
|
||||||
|
smooth,
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -636,6 +688,7 @@ fn compose_inner(
|
|||||||
warps: &[Box<dyn crate::lens::Warp>],
|
warps: &[Box<dyn crate::lens::Warp>],
|
||||||
reveal: Option<&crate::mask::Reveal>,
|
reveal: Option<&crate::mask::Reveal>,
|
||||||
forced: Option<OutputMode>,
|
forced: Option<OutputMode>,
|
||||||
|
smooth: bool,
|
||||||
) -> ComposedShader {
|
) -> ComposedShader {
|
||||||
// The lens corrections, composed into one coordinate transform. Beside
|
// The lens corrections, composed into one coordinate transform. Beside
|
||||||
// `framing` because they are the other half of the same stage: framing
|
// `framing` because they are the other half of the same stage: framing
|
||||||
@@ -668,7 +721,7 @@ fn compose_inner(
|
|||||||
// twice and be bound to a texture of the wrong format.
|
// twice and be bound to a texture of the wrong format.
|
||||||
//
|
//
|
||||||
// `forced` is the one exception, and it is not a caller flag in the
|
// `forced` is the one exception, and it is not a caller flag in the
|
||||||
// sense above: `compose_camera_linear` is the only function that passes
|
// sense above: `compose_camera_probe` is the only function that passes
|
||||||
// it, with an empty operation list, and the mode it forces has its own
|
// it, with an empty operation list, and the mode it forces has its own
|
||||||
// storage format and its own render entry on the GPU side.
|
// storage format and its own render entry on the GPU side.
|
||||||
let output_mode = forced.unwrap_or(
|
let output_mode = forced.unwrap_or(
|
||||||
@@ -842,7 +895,10 @@ fn compose_inner(
|
|||||||
// framing alone — which is what this did before the warps existed — would
|
// framing alone — which is what this did before the warps existed — would
|
||||||
// have nearest-neighboured a distortion correction on an unstraightened
|
// have nearest-neighboured a distortion correction on an unstraightened
|
||||||
// frame, and the aliasing would have looked like a bad profile.
|
// frame, and the aliasing would have looked like a bad profile.
|
||||||
let interpolate = framing.needs_interpolation() || warp.is_active();
|
// `smooth` is the third reason, and the only one a caller states: the
|
||||||
|
// white balance probe averages a patch and cannot do that through a
|
||||||
|
// nearest-neighbour comb (see `compose_camera_probe`).
|
||||||
|
let interpolate = smooth || framing.needs_interpolation() || warp.is_active();
|
||||||
|
|
||||||
// Declared ahead of the warp block, which assigns to them. They enter
|
// Declared ahead of the warp block, which assigns to them. They enter
|
||||||
// equal to `p` so that a chain mixing a splitting warp with a
|
// equal to `p` so that a chain mixing a splitting warp with a
|
||||||
@@ -997,6 +1053,9 @@ fn compose_inner(
|
|||||||
.to_string()
|
.to_string()
|
||||||
};
|
};
|
||||||
|
|
||||||
|
// Formatted with Rust's `Display` so the shader reads the same threshold
|
||||||
|
// the probe checks against; see `CLIP_ONSET`.
|
||||||
|
let clip_onset = CLIP_ONSET;
|
||||||
let source = format!(
|
let source = format!(
|
||||||
"// GENERATED — do not edit.
|
"// GENERATED — do not edit.
|
||||||
//
|
//
|
||||||
@@ -1067,7 +1126,7 @@ fn main(@builtin(global_invocation_id) gid: vec3<u32>) {{
|
|||||||
// A photosite at its white level carries no colour information — every
|
// A photosite at its white level carries no colour information — every
|
||||||
// channel simply stopped counting — so the balance below must not be
|
// channel simply stopped counting — so the balance below must not be
|
||||||
// allowed to tint it.
|
// allowed to tint it.
|
||||||
let clipped = smoothstep(0.985, 1.0, max(c.r, max(c.g, c.b)));
|
let clipped = smoothstep({clip_onset}, 1.0, max(c.r, max(c.g, c.b)));
|
||||||
|
|
||||||
c = c * u.as_shot_wb.rgb;
|
c = c * u.as_shot_wb.rgb;
|
||||||
|
|
||||||
|
|||||||
@@ -99,7 +99,7 @@
|
|||||||
//! A minimum over a patch is separable, as a Gaussian is: minimum along x,
|
//! A minimum over a patch is separable, as a Gaussian is: minimum along x,
|
||||||
//! then along y. That alone is not enough. The patch is 1% of the shorter edge
|
//! then along y. That alone is not enough. The patch is 1% of the shorter edge
|
||||||
//! — 61 taps across at 4K — and two passes of 61 taps is the arithmetic that
|
//! — 61 taps across at 4K — and two passes of 61 taps is the arithmetic that
|
||||||
//! measured 34 ms for clarity and became `docs/technical-debt.md` TD-4.
|
//! measured 34 ms for clarity and became `docs/dev/technical-debt.md` TD-4.
|
||||||
//!
|
//!
|
||||||
//! A minimum has a property a Gaussian does not: **erosions compose by adding
|
//! A minimum has a property a Gaussian does not: **erosions compose by adding
|
||||||
//! their structuring elements**. The minimum over a contiguous run of `d`
|
//! their structuring elements**. The minimum over a contiguous run of `d`
|
||||||
|
|||||||
@@ -150,7 +150,7 @@
|
|||||||
//! the artefact this control must not have.
|
//! the artefact this control must not have.
|
||||||
//!
|
//!
|
||||||
//! Run at the render size, that measured **34 ms at 4K** — seven times the
|
//! Run at the render size, that measured **34 ms at 4K** — seven times the
|
||||||
//! entire fused point chain, for one slider — which is `docs/technical-debt.md`
|
//! entire fused point chain, for one slider — which is `docs/dev/technical-debt.md`
|
||||||
//! TD-4 and is what [`Recipe::base_scale`] now answers. The base is computed on
|
//! TD-4 and is what [`Recipe::base_scale`] now answers. The base is computed on
|
||||||
//! a grid a quarter the size on each axis: a sixteenth of the pixels at a
|
//! a grid a quarter the size on each axis: a sixteenth of the pixels at a
|
||||||
//! quarter of the radius.
|
//! quarter of the radius.
|
||||||
@@ -271,7 +271,7 @@ impl Band for Coarse {
|
|||||||
threshold: 0.35,
|
threshold: 0.35,
|
||||||
gain: 1.0,
|
gain: 1.0,
|
||||||
midtone_taper: true,
|
midtone_taper: true,
|
||||||
// A quarter, which is what `docs/technical-debt.md` TD-4 bought back.
|
// A quarter, which is what `docs/dev/technical-debt.md` TD-4 bought back.
|
||||||
//
|
//
|
||||||
// σ is 1.2% of the shorter edge — 26 px at 4K — so the base holds no
|
// σ is 1.2% of the shorter edge — 26 px at 4K — so the base holds no
|
||||||
// spatial frequency anywhere near the quarter-scale Nyquist of one
|
// spatial frequency anywhere near the quarter-scale Nyquist of one
|
||||||
|
|||||||
@@ -217,7 +217,7 @@ pub struct Version {
|
|||||||
/// graph's film cleared and the caller re-bakes — see `EditGraph::set_film`.
|
/// graph's film cleared and the caller re-bakes — see `EditGraph::set_film`.
|
||||||
pub film: Option<FilmRef>,
|
pub film: Option<FilmRef>,
|
||||||
/// TRACES: FR-DEV-8 | FR-NC-9
|
/// TRACES: FR-DEV-8 | FR-NC-9
|
||||||
/// The repairs (`docs/spot-removal.md`).
|
/// The repairs (`docs/dev/spot-removal.md`).
|
||||||
///
|
///
|
||||||
/// A line per spot, keyed `spot.<id>`, rather than a block per spot as a
|
/// A line per spot, keyed `spot.<id>`, rather than a block per spot as a
|
||||||
/// mask gets: a spot is eight numbers, and sixty-four blocks would bury the
|
/// mask gets: a spot is eight numbers, and sixty-four blocks would bury the
|
||||||
|
|||||||
@@ -6,7 +6,7 @@
|
|||||||
//! are blended. No pixels are stored, here or anywhere: the shader draws the
|
//! are blended. No pixels are stored, here or anywhere: the shader draws the
|
||||||
//! repair from these numbers every time the photograph is rendered, which is
|
//! repair from these numbers every time the photograph is rendered, which is
|
||||||
//! what makes it non-destructive, cheap to sync, and undoable
|
//! what makes it non-destructive, cheap to sync, and undoable
|
||||||
//! (`docs/spot-removal.md`).
|
//! (`docs/dev/spot-removal.md`).
|
||||||
//!
|
//!
|
||||||
//! # Why this is not an operation
|
//! # Why this is not an operation
|
||||||
//!
|
//!
|
||||||
@@ -61,7 +61,7 @@ pub const MAX_SPOTS: usize = 64;
|
|||||||
/// bounds something that is otherwise unbounded: a detail pass declares how far
|
/// bounds something that is otherwise unbounded: a detail pass declares how far
|
||||||
/// it reads from the pixel it writes, and for a spot that is the offset plus
|
/// it reads from the pixel it writes, and for a spot that is the offset plus
|
||||||
/// the radius. An unbounded offset is an unbounded halo, which is a pass the
|
/// the radius. An unbounded offset is an unbounded halo, which is a pass the
|
||||||
/// tile scheduler cannot plan (ARCH §5.3, `docs/spot-removal.md` §5.3).
|
/// tile scheduler cannot plan (ARCH §5.3, `docs/dev/spot-removal.md` §5.3).
|
||||||
pub const MAX_SOURCE_DISTANCE: f32 = 0.5;
|
pub const MAX_SOURCE_DISTANCE: f32 = 0.5;
|
||||||
|
|
||||||
/// The radius a new spot starts at, in frame units.
|
/// The radius a new spot starts at, in frame units.
|
||||||
@@ -210,7 +210,7 @@ impl Spot {
|
|||||||
/// **FR-DEV-8 asks for automatic source placement, and this is the cheap
|
/// **FR-DEV-8 asks for automatic source placement, and this is the cheap
|
||||||
/// half of it.** The good half searches the photograph for a patch whose
|
/// half of it.** The good half searches the photograph for a patch whose
|
||||||
/// surroundings match — a compute dispatch scoring candidate offsets, and
|
/// surroundings match — a compute dispatch scoring candidate offsets, and
|
||||||
/// one small readback when the spot is created (`docs/spot-removal.md`
|
/// one small readback when the spot is created (`docs/dev/spot-removal.md`
|
||||||
/// §8). This is what stands in for it, and it is worth having on its own
|
/// §8). This is what stands in for it, and it is worth having on its own
|
||||||
/// terms rather than as a placeholder: dust sits on skies, skies are
|
/// terms rather than as a placeholder: dust sits on skies, skies are
|
||||||
/// smooth, and a patch two and a half radii away is nearly always the same
|
/// smooth, and a patch two and a half radii away is nearly always the same
|
||||||
|
|||||||
@@ -13,7 +13,7 @@ log.workspace = true
|
|||||||
|
|
||||||
# Inference. `ort` is the API; **what runs it is `dr-inference-engine`'s
|
# Inference. `ort` is the API; **what runs it is `dr-inference-engine`'s
|
||||||
# business** — tract, or an ONNX Runtime the app found on disk, on whichever
|
# business** — tract, or an ONNX Runtime the app found on disk, on whichever
|
||||||
# provider the device has (docs/inference.md). This crate never names either.
|
# provider the device has (docs/dev/inference.md). This crate never names either.
|
||||||
ort = { workspace = true, optional = true }
|
ort = { workspace = true, optional = true }
|
||||||
dr-inference-engine = { workspace = true, optional = true }
|
dr-inference-engine = { workspace = true, optional = true }
|
||||||
ndarray = { workspace = true, optional = true }
|
ndarray = { workspace = true, optional = true }
|
||||||
|
|||||||
@@ -34,7 +34,7 @@
|
|||||||
//! And doing it here buys two things a shader could not. It is **exactly
|
//! And doing it here buys two things a shader could not. It is **exactly
|
||||||
//! deterministic**, which matters because masks reach the sidecar as indices
|
//! deterministic**, which matters because masks reach the sidecar as indices
|
||||||
//! and a field that varied by vendor would mean a mask meaning one thing on
|
//! and a field that varied by vendor would mean a mask meaning one thing on
|
||||||
//! the desktop and another on the phone (docs/segmentation.md §6, M5). And it
|
//! the desktop and another on the phone (docs/dev/segmentation.md §6, M5). And it
|
||||||
//! is testable against hand-computed distances with no adapter present.
|
//! is testable against hand-computed distances with no adapter present.
|
||||||
//!
|
//!
|
||||||
//! # The transform
|
//! # The transform
|
||||||
|
|||||||
@@ -40,7 +40,7 @@ pub struct Edge {
|
|||||||
|
|
||||||
/// A partition of the image into labelled regions, plus how they adjoin.
|
/// A partition of the image into labelled regions, plus how they adjoin.
|
||||||
///
|
///
|
||||||
/// The shared interface from docs/segmentation.md §2: arm A produces this
|
/// The shared interface from docs/dev/segmentation.md §2: arm A produces this
|
||||||
/// from a watershed, arm B would produce it from a class map, and the
|
/// from a watershed, arm B would produce it from a class map, and the
|
||||||
/// consumers above cannot tell which.
|
/// consumers above cannot tell which.
|
||||||
#[derive(Debug, Clone, PartialEq)]
|
#[derive(Debug, Clone, PartialEq)]
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
//! TRACES: FR-DEV-3i
|
//! TRACES: FR-DEV-3i
|
||||||
//! Region segmentation for local masking (S15, docs/segmentation.md).
|
//! Region segmentation for local masking (S15, docs/dev/segmentation.md).
|
||||||
//!
|
//!
|
||||||
//! Local adjustments need to know where the image's regions are before they
|
//! Local adjustments need to know where the image's regions are before they
|
||||||
//! can snap a mask to one. This crate is that map, and it is deliberately
|
//! can snap a mask to one. This crate is that map, and it is deliberately
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
//! Arm C — semantic instances as a prior over the watershed merge order.
|
//! Arm C — semantic instances as a prior over the watershed merge order.
|
||||||
//!
|
//!
|
||||||
//! docs/segmentation.md §5. The spec calls this the expected winner and it is
|
//! docs/dev/segmentation.md §5. The spec calls this the expected winner and it is
|
||||||
//! what ships, for a reason that survives the model turning out to be narrower
|
//! what ships, for a reason that survives the model turning out to be narrower
|
||||||
//! than §4 assumed: the two arms fail in *opposite* directions, so each one
|
//! than §4 assumed: the two arms fail in *opposite* directions, so each one
|
||||||
//! covers the other's failure.
|
//! covers the other's failure.
|
||||||
@@ -233,7 +233,7 @@ pub fn apply_semantic_prior(
|
|||||||
/// This is the interaction the whole spike exists to enable, and the reason it
|
/// This is the interaction the whole spike exists to enable, and the reason it
|
||||||
/// returns *region ids* rather than a raster: a mask that is a set of integers
|
/// returns *region ids* rather than a raster: a mask that is a set of integers
|
||||||
/// is diffable, mergeable at node level under FR-NC-9, and cheap in a sidecar
|
/// is diffable, mergeable at node level under FR-NC-9, and cheap in a sidecar
|
||||||
/// (docs/segmentation.md §1). A raster is none of those.
|
/// (docs/dev/segmentation.md §1). A raster is none of those.
|
||||||
///
|
///
|
||||||
/// The returned ids are sorted, so the same click always produces the same
|
/// The returned ids are sorted, so the same click always produces the same
|
||||||
/// mask — which is what lets it be a cache key.
|
/// mask — which is what lets it be a cache key.
|
||||||
|
|||||||
@@ -60,7 +60,7 @@
|
|||||||
//! So the last step is a **marker-based watershed**. The mask is eroded to
|
//! So the last step is a **marker-based watershed**. The mask is eroded to
|
||||||
//! give two markers — confidently inside, confidently outside — and the flood
|
//! give two markers — confidently inside, confidently outside — and the flood
|
||||||
//! runs in the ribbon left between them, meeting along the most expensive line
|
//! runs in the ribbon left between them, meeting along the most expensive line
|
||||||
//! it can find. The cost is a sum of terms, as docs/segmentation.md §2 says it
|
//! it can find. The cost is a sum of terms, as docs/dev/segmentation.md §2 says it
|
||||||
//! should be: the photograph's own edges, and the colour model's disagreement.
|
//! should be: the photograph's own edges, and the colour model's disagreement.
|
||||||
//!
|
//!
|
||||||
//! Markers are what make this the right shape rather than the watershed §15
|
//! Markers are what make this the right shape rather than the watershed §15
|
||||||
@@ -244,7 +244,7 @@ pub struct RefineOptions {
|
|||||||
/// How much the photograph's own edges count against the colour model in
|
/// How much the photograph's own edges count against the colour model in
|
||||||
/// the flood's cost, `0.0..=1.0`.
|
/// the flood's cost, `0.0..=1.0`.
|
||||||
///
|
///
|
||||||
/// docs/segmentation.md §2 specifies the cost as *a sum of terms* — image
|
/// docs/dev/segmentation.md §2 specifies the cost as *a sum of terms* — image
|
||||||
/// gradient always available, semantic evidence added when a model is
|
/// gradient always available, semantic evidence added when a model is
|
||||||
/// present — and this is the mix. At one the boundary lands purely on the
|
/// present — and this is the mix. At one the boundary lands purely on the
|
||||||
/// strongest edge in the band; at zero purely where the colour verdict
|
/// strongest edge in the band; at zero purely where the colour verdict
|
||||||
@@ -378,7 +378,7 @@ const MAX_SAMPLES: usize = 20_000;
|
|||||||
/// floating-point comparison is a stopping rule that can differ between
|
/// floating-point comparison is a stopping rule that can differ between
|
||||||
/// machines, and a mask that differs between machines reaches the sidecar as
|
/// machines, and a mask that differs between machines reaches the sidecar as
|
||||||
/// indices meaning one thing on the desktop and another on the phone
|
/// indices meaning one thing on the desktop and another on the phone
|
||||||
/// (docs/segmentation.md §6).
|
/// (docs/dev/segmentation.md §6).
|
||||||
const ITERATIONS: usize = 12;
|
const ITERATIONS: usize = 12;
|
||||||
|
|
||||||
/// Half-width of the verdict scale, in nats.
|
/// Half-width of the verdict scale, in nats.
|
||||||
@@ -676,7 +676,7 @@ impl Refinement {
|
|||||||
/// fronts meet along the most expensive line in the ribbon — which is the
|
/// fronts meet along the most expensive line in the ribbon — which is the
|
||||||
/// watershed, and which is where the boundary belongs.
|
/// watershed, and which is where the boundary belongs.
|
||||||
///
|
///
|
||||||
/// The cost is a sum of terms, as docs/segmentation.md §2 says it should
|
/// The cost is a sum of terms, as docs/dev/segmentation.md §2 says it should
|
||||||
/// be: the photograph's own edges, and the colour model's disagreement.
|
/// be: the photograph's own edges, and the colour model's disagreement.
|
||||||
/// Neither alone is right. An edge with no colour meaning is a texture,
|
/// Neither alone is right. An edge with no colour meaning is a texture,
|
||||||
/// and a colour change with no edge is a gradient.
|
/// and a colour change with no edge is a gradient.
|
||||||
@@ -889,7 +889,7 @@ fn neighbours(p: usize, w: usize, h: usize) -> impl Iterator<Item = usize> {
|
|||||||
/// Edge strength over the opponent features, as one byte per pixel.
|
/// Edge strength over the opponent features, as one byte per pixel.
|
||||||
///
|
///
|
||||||
/// Sobel over the same three numbers the colour model is fitted on, rather
|
/// Sobel over the same three numbers the colour model is fitted on, rather
|
||||||
/// than over plain luma — docs/segmentation.md §3 is explicit that a
|
/// than over plain luma — docs/dev/segmentation.md §3 is explicit that a
|
||||||
/// channel-weighted RGB gradient reads a saturated red edge as weaker than it
|
/// channel-weighted RGB gradient reads a saturated red edge as weaker than it
|
||||||
/// looks, and a flag against sky is exactly that edge.
|
/// looks, and a flag against sky is exactly that edge.
|
||||||
///
|
///
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
//! Semantic segmentation — arm B (S15, docs/segmentation.md §4).
|
//! Semantic segmentation — arm B (S15, docs/dev/segmentation.md §4).
|
||||||
//!
|
//!
|
||||||
//! Runs a YOLO instance-segmentation graph over a proxy-resolution image and
|
//! Runs a YOLO instance-segmentation graph over a proxy-resolution image and
|
||||||
//! returns the instances it found: a class, a score, a box, and a soft mask
|
//! returns the instances it found: a class, a score, a box, and a soft mask
|
||||||
@@ -209,7 +209,7 @@ const EMBEDDED_MODEL: &[u8] = include_bytes!("../../../models/segment/yolo26n-se
|
|||||||
const EMBEDDED_CLASSES: &str = include_str!("../../../models/segment/yolo26n-seg.classes.json");
|
const EMBEDDED_CLASSES: &str = include_str!("../../../models/segment/yolo26n-seg.classes.json");
|
||||||
|
|
||||||
/// The bytes of the model that ships with this crate, for whoever compiles
|
/// The bytes of the model that ships with this crate, for whoever compiles
|
||||||
/// engines ahead of the first request (docs/inference.md §6).
|
/// engines ahead of the first request (docs/dev/inference.md §6).
|
||||||
#[cfg(feature = "embedded-model")]
|
#[cfg(feature = "embedded-model")]
|
||||||
pub fn embedded_model_bytes() -> &'static [u8] {
|
pub fn embedded_model_bytes() -> &'static [u8] {
|
||||||
EMBEDDED_MODEL
|
EMBEDDED_MODEL
|
||||||
@@ -237,7 +237,7 @@ impl SemanticModel {
|
|||||||
|
|
||||||
pub fn from_bytes(bytes: &[u8], classes: Vec<Arc<str>>) -> Result<Self, SegmentError> {
|
pub fn from_bytes(bytes: &[u8], classes: Vec<Arc<str>>) -> Result<Self, SegmentError> {
|
||||||
// The f32 graph on whatever the device's backend is. An int8 form
|
// The f32 graph on whatever the device's backend is. An int8 form
|
||||||
// for the Hexagon waits on docs/inference.md §10 M7 — the mask
|
// for the Hexagon waits on docs/dev/inference.md §10 M7 — the mask
|
||||||
// boundary has to be measured before it moves.
|
// boundary has to be measured before it moves.
|
||||||
let session = dr_inference_engine::open(
|
let session = dr_inference_engine::open(
|
||||||
dr_inference_engine::Role::Segmenter,
|
dr_inference_engine::Role::Segmenter,
|
||||||
|
|||||||
@@ -177,7 +177,7 @@ pub struct FaceSettings {
|
|||||||
/// Which SCRFD graph the indexing pass detects with.
|
/// Which SCRFD graph the indexing pass detects with.
|
||||||
///
|
///
|
||||||
/// Three exports of one architecture, differing only in how much computation
|
/// Three exports of one architecture, differing only in how much computation
|
||||||
/// they spend, and docs/faces.md §12.3 is the measurement that made this a
|
/// they spend, and docs/dev/faces.md §12.3 is the measurement that made this a
|
||||||
/// choice rather than a constant: over the same photographs the cheapest one
|
/// choice rather than a constant: over the same photographs the cheapest one
|
||||||
/// misses the small faces in a group and reports a dog a dozen times, the
|
/// misses the small faces in a group and reports a dog a dozen times, the
|
||||||
/// middle one finds 14% more faces for 12% more time, and the largest a
|
/// middle one finds 14% more faces for 12% more time, and the largest a
|
||||||
@@ -261,7 +261,7 @@ impl FaceDetector {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The id when the detector runs in its int8 form (docs/inference.md §7).
|
/// The id when the detector runs in its int8 form (docs/dev/inference.md §7).
|
||||||
///
|
///
|
||||||
/// A different detector: it finds a different set of faces, so it is a
|
/// A different detector: it finds a different set of faces, so it is a
|
||||||
/// different population of detections. The embedder half is unchanged,
|
/// different population of detections. The embedder half is unchanged,
|
||||||
|
|||||||
@@ -48,7 +48,7 @@ pointers there; the build detects that and stops rather than shipping them.
|
|||||||
|
|
||||||
This exists because Android offers no other route to a model: the directory the app reads from is
|
This exists because Android offers no other route to a model: the directory the app reads from is
|
||||||
inside app-private storage, `run-as` needs a debuggable build, and the app has no picker and no
|
inside app-private storage, `run-as` needs a debuggable build, and the app has no picker and no
|
||||||
fetch. docs/faces.md §2.2a is the decision and its limits — these files come back out before anything
|
fetch. docs/dev/faces.md §2.2a is the decision and its limits — these files come back out before anything
|
||||||
is published.
|
is published.
|
||||||
|
|
||||||
## Java in the APK
|
## Java in the APK
|
||||||
|
|||||||
@@ -258,7 +258,7 @@ fi
|
|||||||
cp "${SO}" "${OUT}/staging/lib/${ABI}/libdarkroom.so"
|
cp "${SO}" "${OUT}/staging/lib/${ABI}/libdarkroom.so"
|
||||||
cp "${DEX}" "${OUT}/staging/classes.dex"
|
cp "${DEX}" "${OUT}/staging/classes.dex"
|
||||||
|
|
||||||
# The inference runtime (docs/inference.md §3): ONNX Runtime and Qualcomm's
|
# The inference runtime (docs/dev/inference.md §3): ONNX Runtime and Qualcomm's
|
||||||
# Hexagon backend, beside libdarkroom.so so the app finds them in its own
|
# Hexagon backend, beside libdarkroom.so so the app finds them in its own
|
||||||
# native library directory. The build links none of it — the app dlopens
|
# native library directory. The build links none of it — the app dlopens
|
||||||
# `libonnxruntime.so` at launch and runs on tract if it is not there — so an
|
# `libonnxruntime.so` at launch and runs on tract if it is not there — so an
|
||||||
@@ -278,7 +278,7 @@ else
|
|||||||
fi
|
fi
|
||||||
|
|
||||||
# The models. Android has no other route to one — app-private storage is not
|
# The models. Android has no other route to one — app-private storage is not
|
||||||
# user-reachable and the in-app fetch is unbuilt (docs/faces.md §2.2a) — so
|
# user-reachable and the in-app fetch is unbuilt (docs/dev/faces.md §2.2a) — so
|
||||||
# they go in the APK and `android_main` unpacks them on first launch. The
|
# they go in the APK and `android_main` unpacks them on first launch. The
|
||||||
# sources are `models/face/` and `models/scene/`, shared with the Arch package
|
# sources are `models/face/` and `models/scene/`, shared with the Arch package
|
||||||
# rather than living under this one platform's directory.
|
# rather than living under this one platform's directory.
|
||||||
|
|||||||
@@ -73,7 +73,7 @@ SO="${CACHE}/target/jniLibs/${ABI}/libdarkroom.so"
|
|||||||
# of KEYSTORE_PASS (see its header), and the keystore has to be reachable from
|
# of KEYSTORE_PASS (see its header), and the keystore has to be reachable from
|
||||||
# inside the container, so a host path in KEYSTORE is copied under the mounted
|
# inside the container, so a host path in KEYSTORE is copied under the mounted
|
||||||
# target directory for the duration of the build and removed after. The
|
# target directory for the duration of the build and removed after. The
|
||||||
# passwords travel as environment, never as arguments -- docs/android-signing.md
|
# passwords travel as environment, never as arguments -- docs/dev/android-signing.md
|
||||||
# has the incantation.
|
# has the incantation.
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
echo "==> packaging APK"
|
echo "==> packaging APK"
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# DarkRoom — reproducible Windows cross-build environment
|
# DarkRoom — reproducible Windows cross-build environment
|
||||||
#
|
#
|
||||||
# Everything docs/windows.md §2 names: Rust with the GNU Windows target, the
|
# Everything docs/dev/windows.md §2 names: Rust with the GNU Windows target, the
|
||||||
# MinGW-w64 cross compiler it links with, NSIS to build the installer, and Wine
|
# MinGW-w64 cross compiler it links with, NSIS to build the installer, and Wine
|
||||||
# to smoke-test the result. Both CI and local builds use this image, so "works
|
# to smoke-test the result. Both CI and local builds use this image, so "works
|
||||||
# on my machine" and "works in CI" are the same machine — the same argument
|
# on my machine" and "works in CI" are the same machine — the same argument
|
||||||
@@ -77,7 +77,7 @@ RUN curl -fsSL https://sh.rustup.rs | sh -s -- \
|
|||||||
# libwinpthread the Rust target's own MinGW pieces were built against, and
|
# libwinpthread the Rust target's own MinGW pieces were built against, and
|
||||||
# picking the other produces link errors that read as if std were missing.
|
# picking the other produces link errors that read as if std were missing.
|
||||||
#
|
#
|
||||||
# The runtime is linked statically (docs/windows.md §2) so the installer
|
# The runtime is linked statically (docs/dev/windows.md §2) so the installer
|
||||||
# carries one file. `-static-libgcc` is all it takes: rustc's windows-gnu
|
# carries one file. `-static-libgcc` is all it takes: rustc's windows-gnu
|
||||||
# target links its own copy of winpthread in self-contained mode, so nothing
|
# target links its own copy of winpthread in self-contained mode, so nothing
|
||||||
# imports libwinpthread-1.dll — the smoke test's objdump step is what checks
|
# imports libwinpthread-1.dll — the smoke test's objdump step is what checks
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
# Windows cross-build environment
|
# Windows cross-build environment
|
||||||
|
|
||||||
Reproducible container for building the Windows executable and its installer from Linux. The
|
Reproducible container for building the Windows executable and its installer from Linux. The
|
||||||
specification is [docs/windows.md](../../docs/windows.md); this directory is what it turned into,
|
specification is [docs/dev/windows.md](../../docs/dev/windows.md); this directory is what it turned into,
|
||||||
and every departure from the spec's first draft is recorded in the Dockerfile's comments.
|
and every departure from the spec's first draft is recorded in the Dockerfile's comments.
|
||||||
|
|
||||||
## Use
|
## Use
|
||||||
@@ -16,7 +16,7 @@ and every departure from the spec's first draft is recorded in the Dockerfile's
|
|||||||
# Build the installer from that binary
|
# Build the installer from that binary
|
||||||
./docker/windows/build.sh docker/windows/package.sh
|
./docker/windows/build.sh docker/windows/package.sh
|
||||||
|
|
||||||
# Smoke-test under Wine (docs/windows.md §6)
|
# Smoke-test under Wine (docs/dev/windows.md §6)
|
||||||
./docker/windows/build.sh wine target-windows/x86_64-pc-windows-gnu/release/darkroom-desktop.exe --version
|
./docker/windows/build.sh wine target-windows/x86_64-pc-windows-gnu/release/darkroom-desktop.exe --version
|
||||||
./docker/windows/build.sh wine target-windows/installer/DarkRoom-0.12.0-x86_64-setup.exe /S
|
./docker/windows/build.sh wine target-windows/installer/DarkRoom-0.12.0-x86_64-setup.exe /S
|
||||||
|
|
||||||
|
|||||||
@@ -7,7 +7,7 @@
|
|||||||
# to have run in the same target directory. Produces
|
# to have run in the same target directory. Produces
|
||||||
# DarkRoom-<version>-x86_64-setup.exe in $OUT (default: target-windows/installer).
|
# DarkRoom-<version>-x86_64-setup.exe in $OUT (default: target-windows/installer).
|
||||||
#
|
#
|
||||||
# Runs inside the container, where makensis is; docs/windows.md §5 is the
|
# Runs inside the container, where makensis is; docs/dev/windows.md §5 is the
|
||||||
# specification this implements.
|
# specification this implements.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
@@ -48,7 +48,13 @@ sed 's/$/\r/' "${REPO}/LICENSE" > "${STAGE}/LICENSE"
|
|||||||
# tract on the user's machine with a message about a broken graph rather than
|
# tract on the user's machine with a message about a broken graph rather than
|
||||||
# a checkout that needed `git lfs pull`. Only the weights are checked; the
|
# a checkout that needed `git lfs pull`. Only the weights are checked; the
|
||||||
# scene model's vocabulary and category descriptor are legitimately small.
|
# scene model's vocabulary and category descriptor are legitimately small.
|
||||||
for dir in face scene; do
|
#
|
||||||
|
# The directories are the ones the APK stages (assemble-apk.sh) and the Arch
|
||||||
|
# package installs: the face pair and its eye-state models, the scene model
|
||||||
|
# with its two descriptors, and the panorama border filler. The installer
|
||||||
|
# smoke test counts the same directories, so a model added here is expected
|
||||||
|
# there without a number to update.
|
||||||
|
for dir in face scene inpaint; do
|
||||||
for f in "${REPO}/models/${dir}"/*; do
|
for f in "${REPO}/models/${dir}"/*; do
|
||||||
case "$(basename "${f}")" in
|
case "$(basename "${f}")" in
|
||||||
README.md) continue ;;
|
README.md) continue ;;
|
||||||
|
|||||||
@@ -0,0 +1,86 @@
|
|||||||
|
# Documentation
|
||||||
|
|
||||||
|
Two audiences, two folders. Most people want the first table and never the
|
||||||
|
second.
|
||||||
|
|
||||||
|
## Using DarkRoom
|
||||||
|
|
||||||
|
| | |
|
||||||
|
|---|---|
|
||||||
|
| [The manual](manual/README.md) | Every feature, pictured from the application itself — opening a library, rating and filing, developing, local masks, repair, film, panoramas, export |
|
||||||
|
| [How it is driven](gestures.md) | Every gesture and shortcut, by screen. Generated from the code, so it cannot describe one the application does not have |
|
||||||
|
|
||||||
|
The [top-level README](../README.md) says what DarkRoom is, how to get it on
|
||||||
|
each platform, and what is still missing.
|
||||||
|
|
||||||
|
## Changing DarkRoom
|
||||||
|
|
||||||
|
Everything under [`dev/`](dev/) is for someone working on the code. Start with
|
||||||
|
[CONTRIBUTING.md](../CONTRIBUTING.md), which says how to land a first change
|
||||||
|
without reading the rest.
|
||||||
|
|
||||||
|
**The register and the record.** What must be built, how it is built, and how
|
||||||
|
far along it is.
|
||||||
|
|
||||||
|
| | |
|
||||||
|
|---|---|
|
||||||
|
| [requirements.md](dev/requirements.md) | What the software must do — the numbered register, the decisions (D-numbers) and the spikes (S-numbers) |
|
||||||
|
| [architecture.md](dev/architecture.md) | How it is built — crates, the GPU pipeline, the data model, sync |
|
||||||
|
| [traceability.md](dev/traceability.md) | Generated: which requirement is claimed by which file. Never edited by hand |
|
||||||
|
| [outstanding.md](dev/outstanding.md) | What is specified and not built, and whether that is a decision or a gap |
|
||||||
|
| [technical-debt.md](dev/technical-debt.md) | Compromises taken deliberately, each with the condition that retires it |
|
||||||
|
| [code-health.md](dev/code-health.md) | What a contribution costs, per seam, measured |
|
||||||
|
|
||||||
|
**Designs, one per subsystem.** Each is the specification the code was built
|
||||||
|
to, kept current as the code moved.
|
||||||
|
|
||||||
|
| | |
|
||||||
|
|---|---|
|
||||||
|
| [catalog.md](dev/catalog.md) | The index, the library view, incremental scan, the job queue |
|
||||||
|
| [storage.md](dev/storage.md) | Storage backends: the seam a folder, a sync client and a Nextcloud account share |
|
||||||
|
| [faces.md](dev/faces.md) | Face detection, identity, clustering and the eye-state models |
|
||||||
|
| [segmentation.md](dev/segmentation.md) | How the application finds the regions a local mask snaps to |
|
||||||
|
| [mask-editing.md](dev/mask-editing.md) | Painting, erasing and combining masks |
|
||||||
|
| [spot-removal.md](dev/spot-removal.md) | Clone and heal as parameters in the edit graph |
|
||||||
|
| [panorama.md](dev/panorama.md) | Alignment, projection, the chunked composite and the border fill |
|
||||||
|
| [inference.md](dev/inference.md) | The neural runtime and model chosen per device, with the measurements |
|
||||||
|
| [display-and-extension.md](dev/display-and-extension.md) | The display contract, and why the fused pipeline is already most of a plugin format |
|
||||||
|
| [view-composition.md](dev/view-composition.md) | A controller for the display layer |
|
||||||
|
| [ui-navigation.md](dev/ui-navigation.md) | Finding things in the interface once there are many |
|
||||||
|
|
||||||
|
**Measurements.** Numbers committed so a regression is a diff rather than a
|
||||||
|
recollection.
|
||||||
|
|
||||||
|
| | |
|
||||||
|
|---|---|
|
||||||
|
| [benchmarks.md](dev/benchmarks.md) | The per-commit suite: what it covers, what it does not, how to read a failure |
|
||||||
|
| [bench-baseline.json](dev/bench-baseline.json) | The committed numbers the suite checks against |
|
||||||
|
| [frame-budget.md](dev/frame-budget.md) | What a frame costs on each device, and the decision those figures settled |
|
||||||
|
|
||||||
|
**Platforms and distribution.**
|
||||||
|
|
||||||
|
| | |
|
||||||
|
|---|---|
|
||||||
|
| [distribution.md](dev/distribution.md) | Which channels v1 targets and what each one constrains |
|
||||||
|
| [windows.md](dev/windows.md) | The Windows installer, cross-built from the Linux CI |
|
||||||
|
| [android-signing.md](dev/android-signing.md) | Which key signs the APK, and keeping it |
|
||||||
|
|
||||||
|
**Archive.** Kept as the record of what was asked for, not as plans.
|
||||||
|
|
||||||
|
| | |
|
||||||
|
|---|---|
|
||||||
|
| [milestone-v0.1.md](dev/archive/milestone-v0.1.md) | The first milestone, delivered 2026-08-30 and superseded |
|
||||||
|
| [ui-refinement.md](dev/archive/ui-refinement.md) | How the interface should look; succeeded by [ui-navigation.md](dev/ui-navigation.md) |
|
||||||
|
|
||||||
|
## Conventions
|
||||||
|
|
||||||
|
Two files here are generated and must not be edited by hand:
|
||||||
|
`gestures.md` and `dev/traceability.md`. Both come from
|
||||||
|
`cargo run -p traceability` and the pre-commit hook keeps them in step with
|
||||||
|
the tree. The manual's pictures are recorded by
|
||||||
|
[`tools/manual`](../tools/manual/README.md) and live in LFS.
|
||||||
|
|
||||||
|
A design document links to the requirements it satisfies and to the code
|
||||||
|
that satisfies them. When the code moves, the link moves with it; a document
|
||||||
|
that has stopped being true goes to `dev/archive/` with a note saying what
|
||||||
|
replaced it, rather than being deleted.
|
||||||
@@ -4,8 +4,8 @@
|
|||||||
|
|
||||||
> Kept as the record of what the first milestone asked for, not as a plan.
|
> Kept as the record of what the first milestone asked for, not as a plan.
|
||||||
> Everything below shipped, and the application went well past it — see
|
> Everything below shipped, and the application went well past it — see
|
||||||
> [outstanding.md](outstanding.md) for what is still missing at 0.9.0.
|
> [outstanding.md](../outstanding.md) for what is still missing at 0.9.0.
|
||||||
**Companion to:** [requirements.md](requirements.md) · [architecture.md](architecture.md)
|
**Companion to:** [requirements.md](../requirements.md) · [architecture.md](../architecture.md)
|
||||||
|
|
||||||
The first buildable milestone: connect to a Nextcloud folder, index it locally, and display RAW
|
The first buildable milestone: connect to a Nextcloud folder, index it locally, and display RAW
|
||||||
previews on both Linux and Android.
|
previews on both Linux and Android.
|
||||||
@@ -39,7 +39,7 @@ building on sand.
|
|||||||
|
|
||||||
## 2. The four assumptions under test
|
## 2. The four assumptions under test
|
||||||
|
|
||||||
Each maps to a spike in [requirements.md §9](requirements.md).
|
Each maps to a spike in [requirements.md §9](../requirements.md).
|
||||||
|
|
||||||
| # | Assumption | If wrong | Spike |
|
| # | Assumption | If wrong | Spike |
|
||||||
|---|---|---|---|
|
|---|---|---|---|
|
||||||
@@ -55,7 +55,7 @@ all, so it should be proven in the first week, before catalog or sync work begin
|
|||||||
|
|
||||||
## 3. Functional scope
|
## 3. Functional scope
|
||||||
|
|
||||||
Requirement IDs reference [requirements.md](requirements.md); a v0.1 suffix marks a reduced subset
|
Requirement IDs reference [requirements.md](../requirements.md); a v0.1 suffix marks a reduced subset
|
||||||
of the full requirement.
|
of the full requirement.
|
||||||
|
|
||||||
### 3.1 Account and connection
|
### 3.1 Account and connection
|
||||||
@@ -2,9 +2,9 @@
|
|||||||
|
|
||||||
**Status:** Built, not yet recorded · 2026-08-30
|
**Status:** Built, not yet recorded · 2026-08-30
|
||||||
**Companion to:** [requirements.md](requirements.md) §4.1 (performance targets) · §8 (verification)
|
**Companion to:** [requirements.md](requirements.md) §4.1 (performance targets) · §8 (verification)
|
||||||
**Instrument:** [`tools/bench`](../tools/bench) — `cargo run --release -p dr-bench -- check`
|
**Instrument:** [`tools/bench`](../../tools/bench) — `cargo run --release -p dr-bench -- check`
|
||||||
**Committed numbers:** [`bench-baseline.json`](bench-baseline.json)
|
**Committed numbers:** [`bench-baseline.json`](bench-baseline.json)
|
||||||
**GPU half:** [`core/dr-gpu/tests/frame_budget.rs`](../core/dr-gpu/tests/frame_budget.rs) ·
|
**GPU half:** [`core/dr-gpu/tests/frame_budget.rs`](../../core/dr-gpu/tests/frame_budget.rs) ·
|
||||||
[frame-budget.md](frame-budget.md)
|
[frame-budget.md](frame-budget.md)
|
||||||
|
|
||||||
§8 has said since it was written that performance is verified by *"an automated
|
§8 has said since it was written that performance is verified by *"an automated
|
||||||
@@ -60,7 +60,7 @@ Two of those rows carry a qualifier, and the qualifiers are the point.
|
|||||||
library view cannot paint without: `Catalog::open` (which connects, migrates and
|
library view cannot paint without: `Catalog::open` (which connects, migrates and
|
||||||
**backfills**, and the backfill is three passes over the images table on every
|
**backfills**, and the backfill is three passes over the images table on every
|
||||||
open), `count`, the first 400-row `window`, and the monthly `timeline`. Tagged
|
open), `count`, the first 400-row `window`, and the monthly `timeline`. Tagged
|
||||||
`TRACES: NFR-P1` in [`tools/bench/src/catalog_open.rs`](../tools/bench/src/catalog_open.rs),
|
`TRACES: NFR-P1` in [`tools/bench/src/catalog_open.rs`](../../tools/bench/src/catalog_open.rs),
|
||||||
because a build that breaks it fails this gate.
|
because a build that breaks it fails this gate.
|
||||||
|
|
||||||
**NFR-P3 — ≥ 100 images per second on the embedded preview path.** The
|
**NFR-P3 — ≥ 100 images per second on the embedded preview path.** The
|
||||||
@@ -69,7 +69,7 @@ per-image work is exactly what `spawn_thumbnail_sweep` does — `decode_jpeg`,
|
|||||||
`ThumbStore::put` — arranged in the same shape: chunks of 96, lanes owning
|
`ThumbStore::put` — arranged in the same shape: chunks of 96, lanes owning
|
||||||
disjoint slices, and the single thread that owns the store writing the finished
|
disjoint slices, and the single thread that owns the store writing the finished
|
||||||
chunk. Tagged `TRACES: NFR-P3` in
|
chunk. Tagged `TRACES: NFR-P3` in
|
||||||
[`tools/bench/src/thumbnails.rs`](../tools/bench/src/thumbnails.rs).
|
[`tools/bench/src/thumbnails.rs`](../../tools/bench/src/thumbnails.rs).
|
||||||
|
|
||||||
### Requirements this can only half-answer, and is not tagged for
|
### Requirements this can only half-answer, and is not tagged for
|
||||||
|
|
||||||
@@ -114,7 +114,7 @@ instead of ~2 TB, and neither half is flattered by that.
|
|||||||
| Rows | 50,000 images, 50,000 default versions, 400 folders, one root |
|
| Rows | 50,000 images, 50,000 default versions, 400 folders, one root |
|
||||||
| Capture times | Twelve years from a fixed epoch, so the timeline has ~144 monthly buckets |
|
| Capture times | Twelve years from a fixed epoch, so the timeline has ~144 monthly buckets |
|
||||||
| Sources | 12 synthesised JPEGs at 1620 × 1080 — the size `dr-decode` records a CR2 carrying in IFD2 |
|
| Sources | 12 synthesised JPEGs at 1620 × 1080 — the size `dr-decode` records a CR2 carrying in IFD2 |
|
||||||
| Seed | 20260829, in [`tools/bench/src/main.rs`](../tools/bench/src/main.rs) |
|
| Seed | 20260829, in [`tools/bench/src/main.rs`](../../tools/bench/src/main.rs) |
|
||||||
| Location | `$DR_BENCH_DIR`, else the system temporary directory |
|
| Location | `$DR_BENCH_DIR`, else the system temporary directory |
|
||||||
|
|
||||||
It is reproducible from the seed, and a `stamp.json` beside it records what it
|
It is reproducible from the seed, and a `stamp.json` beside it records what it
|
||||||
@@ -530,6 +530,15 @@ Three invariants, each tested:
|
|||||||
| Re-storing an existing id updates in place, never migrates | Migrating would rewrite a sealed shard |
|
| Re-storing an existing id updates in place, never migrates | Migrating would rewrite a sealed shard |
|
||||||
| Merging another client's shard is insert-only and idempotent | Both copies derive from the same bytes by the same code, so neither is better; preferring ours avoids dirtying a shard others have synced |
|
| Merging another client's shard is insert-only and idempotent | Both copies derive from the same bytes by the same code, so neither is better; preferring ours avoids dirtying a shard others have synced |
|
||||||
|
|
||||||
|
**When the exchange runs.** Corrected 2026-09-20. It fired only after the metadata sweep — hours
|
||||||
|
on a large library — so a fresh device re-derived every thumbnail it looked at, re-detected faces
|
||||||
|
and re-read every header before adopting the shards and snapshot that held all of it. It now also
|
||||||
|
fires the moment the scan completes, which is the first moment the rows the merges key on exist,
|
||||||
|
and the sweep starts behind it. In steady state that pass is one listing. The catalog merge also
|
||||||
|
takes **capture metadata** (`captured_at`, offset, camera, lens, ISO) for images still at
|
||||||
|
`metadata_state < 2`, matched by `oc:fileid` — a date is a fact about the file's bytes, not local
|
||||||
|
state, and the snapshot already carried it; the sweep's per-chunk query then finds nothing left.
|
||||||
|
|
||||||
**The transfer**, in `dr-ui`'s `derived_sync`, exchanges shards with `.darkroom-derived/` under the
|
**The transfer**, in `dr-ui`'s `derived_sync`, exchanges shards with `.darkroom-derived/` under the
|
||||||
library root. `ThumbStore::shards()` reports which are sealed, so an up-to-date client's whole pass
|
library root. `ThumbStore::shards()` reports which are sealed, so an up-to-date client's whole pass
|
||||||
is one listing plus whichever shard is still open.
|
is one listing plus whichever shard is still open.
|
||||||
@@ -17,8 +17,8 @@ permission to a package rather than after.
|
|||||||
|
|
||||||
| Platform | Channel | State | What it constrains |
|
| Platform | Channel | State | What it constrains |
|
||||||
|---|---|---|---|
|
|---|---|---|---|
|
||||||
| Linux | Arch source package — [`packaging/PKGBUILD`](../packaging/PKGBUILD) | Built, in tree | Nothing. Full filesystem access, system Vulkan, system secret daemon |
|
| Linux | Arch source package — [`packaging/PKGBUILD`](../../packaging/PKGBUILD) | Built, in tree | Nothing. Full filesystem access, system Vulkan, system secret daemon |
|
||||||
| Linux | Flatpak — [`packaging/flatpak/`](../packaging/flatpak/) | Manifest in tree, **library selection does not work** (§4) | Portals only. No `--filesystem=`, no host mount table, no typed paths |
|
| Linux | Flatpak — [`packaging/flatpak/`](../../packaging/flatpak/) | Manifest in tree, **library selection does not work** (§4) | Portals only. No `--filesystem=`, no host mount table, no typed paths |
|
||||||
| Linux | AppImage | v1 channel, **recipe not yet written** (§5) | Oldest supported glibc, and no sandbox at all |
|
| Linux | AppImage | v1 channel, **recipe not yet written** (§5) | Oldest supported glibc, and no sandbox at all |
|
||||||
| Android | F-Droid | v1 channel, not yet submitted | GPLv3-clean build, reproducible, no proprietary blobs |
|
| Android | F-Droid | v1 channel, not yet submitted | GPLv3-clean build, reproducible, no proprietary blobs |
|
||||||
| Android | Play Store | **Not v1** (§6) | Would make ARCH §6.9 binding as policy rather than as engineering |
|
| Android | Play Store | **Not v1** (§6) | Would make ARCH §6.9 binding as policy rather than as engineering |
|
||||||
@@ -39,7 +39,7 @@ somewhere:
|
|||||||
that misses one of them costs the icon in the shell or the association in the
|
that misses one of them costs the icon in the shell or the association in the
|
||||||
software centre, and neither failure announces itself.
|
software centre, and neither failure announces itself.
|
||||||
- **The metainfo, not just the desktop entry.**
|
- **The metainfo, not just the desktop entry.**
|
||||||
[`packaging/paris.tourolle.darkroom.metainfo.xml`](../packaging/paris.tourolle.darkroom.metainfo.xml)
|
[`packaging/paris.tourolle.darkroom.metainfo.xml`](../../packaging/paris.tourolle.darkroom.metainfo.xml)
|
||||||
is the single description of the application, installed by every channel that
|
is the single description of the application, installed by every channel that
|
||||||
has somewhere to put it. Its `metadata_license` is CC0-1.0 and its
|
has somewhere to put it. Its `metadata_license` is CC0-1.0 and its
|
||||||
`project_license` is GPL-3.0-or-later; those differ on purpose — see the
|
`project_license` is GPL-3.0-or-later; those differ on purpose — see the
|
||||||
@@ -173,7 +173,7 @@ Two changes, in this order:
|
|||||||
chooser instead of a volume list.
|
chooser instead of a volume list.
|
||||||
|
|
||||||
**Done when:** a Flatpak built from
|
**Done when:** a Flatpak built from
|
||||||
[`packaging/flatpak/paris.tourolle.darkroom.yml`](../packaging/flatpak/paris.tourolle.darkroom.yml),
|
[`packaging/flatpak/paris.tourolle.darkroom.yml`](../../packaging/flatpak/paris.tourolle.darkroom.yml),
|
||||||
with its `finish-args` unchanged and no `flatpak override` applied, can select a
|
with its `finish-args` unchanged and no `flatpak override` applied, can select a
|
||||||
library root, scan it, and write a sidecar back into it.
|
library root, scan it, and write a sidecar back into it.
|
||||||
|
|
||||||
@@ -821,6 +821,16 @@ here, and it is the phone and tablet story that should decide whether it gets bu
|
|||||||
already has, the raw vector written over the old one and its id, box and identity untouched
|
already has, the raw vector written over the old one and its id, box and identity untouched
|
||||||
(`faces::record_updates`). No detector runs and no suggestion is lost — the cost is the
|
(`faces::record_updates`). No detector runs and no suggestion is lost — the cost is the
|
||||||
original fetched once more, since the length exists only at the moment of embedding.
|
original fetched once more, since the length exists only at the moment of embedding.
|
||||||
|
- **A person is stood for by their references.** Every face the user has ruled on is an anchor,
|
||||||
|
and the scan is exhaustive, so a person with 750 confirmations would cost 750 comparisons
|
||||||
|
against every other face — and the cost of a library would grow with how well it was named.
|
||||||
|
Instead each person enters through at most `MAX_REFERENCES` (100) of their anchored faces,
|
||||||
|
chosen by `dr_face::references`: those whose raw embedding is at least `MIN_REFERENCE_QUALITY`
|
||||||
|
(15) long, and among them the set spanning the greatest volume — greedy max-determinant, the
|
||||||
|
longest vector first and then, at each step, the face with the largest component orthogonal to
|
||||||
|
the chosen so far. Thirty frames from one afternoon contribute one reference; the single profile
|
||||||
|
shot is taken early. The faces not chosen keep their confirmations and are not touched by the
|
||||||
|
pass; they are simply not compared.
|
||||||
|
|
||||||
**The algorithm.** Constrained average-link agglomeration over the probability graph, merging while
|
**The algorithm.** Constrained average-link agglomeration over the probability graph, merging while
|
||||||
the average pairwise probability exceeds **0.9** and no cannot-link is violated. Average-link rather
|
the average pairwise probability exceeds **0.9** and no cannot-link is violated. Average-link rather
|
||||||
@@ -3,8 +3,8 @@
|
|||||||
**Status:** Measured · 2026-08-27
|
**Status:** Measured · 2026-08-27
|
||||||
**Companion to:** [display-and-extension.md](display-and-extension.md) §2–3 ·
|
**Companion to:** [display-and-extension.md](display-and-extension.md) §2–3 ·
|
||||||
[requirements.md](requirements.md) §3.4 FR-DSP-2, FR-DSP-3, FR-DSP-4
|
[requirements.md](requirements.md) §3.4 FR-DSP-2, FR-DSP-3, FR-DSP-4
|
||||||
**Instrument:** [`core/dr-gpu/examples/frame_budget.rs`](../core/dr-gpu/examples/frame_budget.rs)
|
**Instrument:** [`core/dr-gpu/examples/frame_budget.rs`](../../core/dr-gpu/examples/frame_budget.rs)
|
||||||
**Guard:** [`core/dr-gpu/tests/frame_budget.rs`](../core/dr-gpu/tests/frame_budget.rs)
|
**Guard:** [`core/dr-gpu/tests/frame_budget.rs`](../../core/dr-gpu/tests/frame_budget.rs)
|
||||||
|
|
||||||
[display-and-extension.md](display-and-extension.md) §2 fixed a decision rule in
|
[display-and-extension.md](display-and-extension.md) §2 fixed a decision rule in
|
||||||
advance and made three measurements the thing that settles it. This file is
|
advance and made three measurements the thing that settles it. This file is
|
||||||
@@ -75,7 +75,49 @@ TensorRT's job.
|
|||||||
worst: nearly five minutes), because it is compiling an engine for this exact GPU. The engine caches to disk and the second load is milliseconds. That
|
worst: nearly five minutes), because it is compiling an engine for this exact GPU. The engine caches to disk and the second load is milliseconds. That
|
||||||
number is what §6 is designed around.
|
number is what §6 is designed around.
|
||||||
|
|
||||||
### 1.3 What the numbers say
|
### 1.3 The desktop — Radeon RX 7900 XT, Threadripper 2920X, 24 threads · 2026-09-20
|
||||||
|
|
||||||
|
Arch's `onnxruntime-rocm` 1.29.0 against ROCm 7.2.4 and MIGraphX 7.2.3 (gfx1100), through the
|
||||||
|
same `ep_probe` harness (`core/dr-inference-engine/examples/ep_probe.rs`). Zero input, three
|
||||||
|
warm-ups, the median of 15 runs, on a machine doing nothing else. The build columns are the wall
|
||||||
|
clock of `Session` construction: cold is a MIGraphX compile of the graph for this GPU, cached is the
|
||||||
|
same session loading the program the cold build wrote.
|
||||||
|
|
||||||
|
| Model | ORT CPU f32 | MIGraphX f32 | **MIGraphX fp16** | Compile f32 / fp16 (s) | Cached load (s) |
|
||||||
|
|---|---|---|---|---|---|
|
||||||
|
| scrfd_500m (Fast) | 10.4 | 2.8 | **2.4** | 40 / 58 | 0.3 |
|
||||||
|
| scrfd_2.5g (Balanced) | 20.7 | 3.3 | **2.8** | 37 / 40 | 0.3 |
|
||||||
|
| scrfd_10g (Thorough) | 57.9 | 4.5 | **3.4** | 40 / 48 | 0.4 |
|
||||||
|
| arcface_mbf (per face) | 12.8 | 1.8 | 1.6 | 15 / 21 | 0.4 |
|
||||||
|
| yolo26n-seg | 49.3 | 8.4 | **7.5** | 110 / 136 | 0.9 |
|
||||||
|
| yolo26s-sem-ade20k | 55.8 | 4.8 | **3.8** | 50 / 60 | 0.5 |
|
||||||
|
| 2d106det (landmarks) | 9.9 | 1.2 | 1.0 | 16 / 21 | 0.2 |
|
||||||
|
| ocec_s (eye state) | 2.5 | 0.5 | 0.4 | 17 / 17 | 0.1 |
|
||||||
|
| xfeat-1024 | 26.8 | 10.5 | 9.9 | 37 / 53 | 0.3 |
|
||||||
|
| migan-512 (per tile) | 514 | 12.7 | **8.3** | 102 / 132 | 0.8 |
|
||||||
|
|
||||||
|
Three things the table settles.
|
||||||
|
|
||||||
|
- **The ROCm execution provider does not exist any more.** It was ONNX Runtime's CUDA-provider twin
|
||||||
|
for AMD, removed in the 1.23 release (AMD's builds dropped it from ROCm 7.1); 1.29's ROCm build ships
|
||||||
|
`libonnxruntime_providers_migraphx.so` and nothing else for AMD, and asking for `ROCm` answers
|
||||||
|
"not enabled in this build". So there is no non-compiling AMD rung to sit under MIGraphX the way
|
||||||
|
the CUDA provider sits under TensorRT: the AMD ladder is MIGraphX, then the CPU.
|
||||||
|
- **MIGraphX is a compiling provider, and its cache has to be asked for by name.** 15–135 s per
|
||||||
|
graph cold, under a second from its cache — TensorRT's shape exactly, and §6's design covers it.
|
||||||
|
Two things the provider does that the code has to know: `ort`'s builder fills the legacy options
|
||||||
|
struct, which 1.29 reads for the precision flags only, so the cache directory
|
||||||
|
(`migraphx_model_cache_dir`) reaches it only through the generic key/value registration; and
|
||||||
|
the cache key is the graph, the GPU and the MIGraphX version *without the precision*, so an fp16
|
||||||
|
session pointed at the f32 program's directory silently loads the f32 program (the first fp16
|
||||||
|
row measured here was that, before the directories were split).
|
||||||
|
- **fp16 is worth 10–35% over f32 on this card, not the 3× it is worth on TensorRT**, because
|
||||||
|
MIGraphX f32 is already 3–8× the CPU provider and the small graphs are launch-bound. The
|
||||||
|
detectors at 2.4–3.4 ms sit beside TensorRT fp16's 1.8–3.3 ms on the RTX 3050; the embedder is
|
||||||
|
1.6–1.8 ms on either precision and stays f32 (§7). The whole face pipeline for one image
|
||||||
|
(detector + landmarks + eyes + embedder) is under 6 ms.
|
||||||
|
|
||||||
|
### 1.4 What the numbers say
|
||||||
|
|
||||||
- **`tract` is single-threaded.** The tablet's one X4 core and one Raptor Lake core give the same
|
- **`tract` is single-threaded.** The tablet's one X4 core and one Raptor Lake core give the same
|
||||||
tract numbers. Replacing it with ONNX Runtime's CPU provider, *no accelerator involved*, is 3–6×
|
tract numbers. Replacing it with ONNX Runtime's CPU provider, *no accelerator involved*, is 3–6×
|
||||||
@@ -92,6 +134,8 @@ number is what §6 is designed around.
|
|||||||
- **On NVIDIA, TensorRT fp16 ≈ 3× the CUDA provider**, and the CUDA provider ≈ 2× the
|
- **On NVIDIA, TensorRT fp16 ≈ 3× the CUDA provider**, and the CUDA provider ≈ 2× the
|
||||||
multi-threaded CPU; at fp16 the detectors are 1.8–3.3 ms with no quantisation at all. Both leave the twenty cores free for decoding during a batch index, which the table does not
|
multi-threaded CPU; at fp16 the detectors are 1.8–3.3 ms with no quantisation at all. Both leave the twenty cores free for decoding during a batch index, which the table does not
|
||||||
show and which matters more than the ratio.
|
show and which matters more than the ratio.
|
||||||
|
- **On AMD, MIGraphX fp16 is 4–17× the CPU provider** on the detectors and 60× on the
|
||||||
|
inpainter, with the same first-run compile cost as TensorRT and no rung between it and the CPU.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -105,7 +149,8 @@ winning:
|
|||||||
| Android, Qualcomm with a Hexagon the shipped QNN skel covers (V68–V81) | QNN HTP, int8 model | ORT CPU, f32 model | — | tract |
|
| Android, Qualcomm with a Hexagon the shipped QNN skel covers (V68–V81) | QNN HTP, int8 model | ORT CPU, f32 model | — | tract |
|
||||||
| Android, any other SoC | ORT CPU, f32 | — | — | tract |
|
| Android, any other SoC | ORT CPU, f32 | — | — | tract |
|
||||||
| Linux / Windows, NVIDIA GPU | TensorRT, f32 model, fp16 engine | CUDA provider, f32 | ORT CPU, f32 | tract |
|
| Linux / Windows, NVIDIA GPU | TensorRT, f32 model, fp16 engine | CUDA provider, f32 | ORT CPU, f32 | tract |
|
||||||
| Linux / Windows, no NVIDIA | ORT CPU, f32 | — | — | tract |
|
| Linux, AMD GPU with ROCm | MIGraphX, f32 model, fp16 program | ORT CPU, f32 | — | tract |
|
||||||
|
| Linux / Windows, no GPU stack | ORT CPU, f32 | — | — | tract |
|
||||||
| macOS ⁵ | ORT CPU, f32 | — | — | tract |
|
| macOS ⁵ | ORT CPU, f32 | — | — | tract |
|
||||||
|
|
||||||
⁵ CoreML is the obvious rung and is unmeasured; it is listed so its absence is a gap and not an
|
⁵ CoreML is the obvious rung and is unmeasured; it is listed so its absence is a gap and not an
|
||||||
@@ -113,8 +158,13 @@ oversight.
|
|||||||
|
|
||||||
Deliberately **not** on any ladder, with the measurement that excluded each: NNAPI (no driver),
|
Deliberately **not** on any ladder, with the measurement that excluded each: NNAPI (no driver),
|
||||||
XNNPACK (slower than CPU, aborts on SCRFD), WebGPU (slower than CPU), the Adreno through QNN (works,
|
XNNPACK (slower than CPU, aborts on SCRFD), WebGPU (slower than CPU), the Adreno through QNN (works,
|
||||||
but never where the Hexagon does not also), CUDA int8 (slower than CUDA f32). A rung is added to this
|
but never where the Hexagon does not also), CUDA int8 (slower than CUDA f32), the ROCm provider
|
||||||
table by a measurement on this page, not by a provider existing.
|
(gone: §1.3). A rung is added to this table by a measurement on this page, not by a provider
|
||||||
|
existing.
|
||||||
|
|
||||||
|
The AMD ladder has no middle rung. TensorRT falls back to the CUDA provider while its engines
|
||||||
|
compile; MIGraphX has no such twin, so its fallback is the CPU provider, and the minute or two of
|
||||||
|
compiling on first run (§6) is spent at the floor's speed rather than at half the GPU's.
|
||||||
|
|
||||||
Two things the ladder is *not*: it is not a per-model choice — one backend serves every model on a
|
Two things the ladder is *not*: it is not a per-model choice — one backend serves every model on a
|
||||||
device, because §7's identity rule needs the detector and embedder on the same runtime for the
|
device, because §7's identity rule needs the detector and embedder on the same runtime for the
|
||||||
@@ -174,9 +224,10 @@ is cheaper than discovering it at packaging time.
|
|||||||
| ONNX Runtime | MIT | Yes |
|
| ONNX Runtime | MIT | Yes |
|
||||||
| Qualcomm QNN runtime (`com.qualcomm.qti:qnn-runtime` on Maven) | Qualcomm AI Engine Direct SDK licence — proprietary, redistribution permitted for applications using it | Yes for the APK, with the licence text shipped; not for a source distribution. **To be read in full, not summarised from memory, before the APK gains it.** |
|
| Qualcomm QNN runtime (`com.qualcomm.qti:qnn-runtime` on Maven) | Qualcomm AI Engine Direct SDK licence — proprietary, redistribution permitted for applications using it | Yes for the APK, with the licence text shipped; not for a source distribution. **To be read in full, not summarised from memory, before the APK gains it.** |
|
||||||
| CUDA runtime, cuDNN, TensorRT | NVIDIA EULAs — redistributable with an application, with the licence text, not modifiable | Yes for a package that bundles them. 600 MB. The alternative is to load them from the user's system install if present and skip the rung otherwise — which is what §4's probe does anyway. |
|
| CUDA runtime, cuDNN, TensorRT | NVIDIA EULAs — redistributable with an application, with the licence text, not modifiable | Yes for a package that bundles them. 600 MB. The alternative is to load them from the user's system install if present and skip the rung otherwise — which is what §4's probe does anyway. |
|
||||||
|
| ROCm (HIP, MIOpen, rocBLAS), MIGraphX | MIT | Yes, but the HIP SDK MIGraphX needs is ~15 GB installed. Same answer as NVIDIA: the user's system install, or the rung is skipped. |
|
||||||
|
|
||||||
The position this takes: the **NVIDIA libraries are not bundled**. The desktop package probes for a
|
The position this takes: the **GPU vendors' libraries are not bundled**. The desktop package probes for a
|
||||||
system CUDA/TensorRT install and uses it if it is version-compatible; a desktop without one runs on
|
system CUDA/TensorRT or ROCm/MIGraphX install and uses it if it is version-compatible; a desktop without one runs on
|
||||||
ORT CPU, which is still 8–10× today. Bundling 600 MB for a rung that is 2× again is not a trade
|
ORT CPU, which is still 8–10× today. Bundling 600 MB for a rung that is 2× again is not a trade
|
||||||
worth making unmeasured, and it can be revisited by a measurement on a batch index. The **QNN
|
worth making unmeasured, and it can be revisited by a measurement on a batch index. The **QNN
|
||||||
runtime is bundled** in the APK, because the Hexagon is the difference between a tablet that
|
runtime is bundled** in the APK, because the Hexagon is the difference between a tablet that
|
||||||
@@ -237,6 +288,7 @@ of which form they load:
|
|||||||
| f32 ONNX, shape-fixed, **opset ≥ 13** | `tools/fix-face-model-shapes.sh`, `tools/export-seg-model.sh` | Release time, once | Every rung except Hexagon |
|
| f32 ONNX, shape-fixed, **opset ≥ 13** | `tools/fix-face-model-shapes.sh`, `tools/export-seg-model.sh` | Release time, once | Every rung except Hexagon |
|
||||||
| int8 QDQ ONNX, per-channel, uint8 activations | `tools/quantise-models.sh` (new) | Release time, once, **calibrated on real photographs** | Hexagon |
|
| int8 QDQ ONNX, per-channel, uint8 activations | `tools/quantise-models.sh` (new) | Release time, once, **calibrated on real photographs** | Hexagon |
|
||||||
| TensorRT engine (`.engine`, per GPU architecture and TensorRT version) | The app, from the f32 file | First run on that device, in the background | TensorRT rung |
|
| TensorRT engine (`.engine`, per GPU architecture and TensorRT version) | The app, from the f32 file | First run on that device, in the background | TensorRT rung |
|
||||||
|
| MIGraphX program (`.mxr`, per GPU architecture, MIGraphX version and precision) | The app, from the f32 file | First run on that device, in the background | MIGraphX rung |
|
||||||
| QNN context binary | The app, from the int8 file | First run on that device, in the background | Hexagon rung |
|
| QNN context binary | The app, from the int8 file | First run on that device, in the background | Hexagon rung |
|
||||||
|
|
||||||
Two rules.
|
Two rules.
|
||||||
@@ -256,9 +308,12 @@ prefers. That is a change to the canonical file and so a change to the shipped m
|
|||||||
happens in the same model release as the int8 files.
|
happens in the same model release as the int8 files.
|
||||||
|
|
||||||
**Compilation is a device-time step, and it is cached.** A TensorRT engine is specific to the GPU
|
**Compilation is a device-time step, and it is cached.** A TensorRT engine is specific to the GPU
|
||||||
it was built on and the TensorRT that built it; a QNN context binary is specific to the Hexagon
|
it was built on and the TensorRT that built it; a MIGraphX program to the GPU and the MIGraphX
|
||||||
generation. Neither can ship. Both are built by the app the first time that rung is selected, in
|
that built it; a QNN context binary to the Hexagon generation. None can ship. All are built by
|
||||||
the background (§6), and written beside the probe cache keyed by the same inputs. They are
|
the app the first time that rung is selected, in the background (§6), and written beside the
|
||||||
|
probe cache keyed by the same inputs. MIGraphX's own key leaves out the precision, so the app
|
||||||
|
gives its f32 and fp16 programs separate directories — otherwise the embedder's f32 build would
|
||||||
|
be served the detector's fp16 program, or the reverse. They are
|
||||||
**derived, disposable, and regenerable**: deleting the cache directory costs the next launch a
|
**derived, disposable, and regenerable**: deleting the cache directory costs the next launch a
|
||||||
rebuild and nothing else, and the directory is excluded from anything that syncs (it is a peer of
|
rebuild and nothing else, and the directory is excluded from anything that syncs (it is a peer of
|
||||||
`thumbs`, not of the catalog).
|
`thumbs`, not of the catalog).
|
||||||
@@ -267,17 +322,18 @@ rebuild and nothing else, and the directory is excluded from anything that syncs
|
|||||||
|
|
||||||
## 6. First run — building engines without the user waiting for them
|
## 6. First run — building engines without the user waiting for them
|
||||||
|
|
||||||
The sequence on a device where a compiling rung (TensorRT, Hexagon) is selected:
|
The sequence on a device where a compiling rung (TensorRT, MIGraphX, Hexagon) is selected:
|
||||||
|
|
||||||
1. **Launch.** The runtime loads; the probe (§4) starts in the background; the app serves every
|
1. **Launch.** The runtime loads; the probe (§4) starts in the background; the app serves every
|
||||||
model request from the floor. Face indexing, segmentation and scene grading all work, at
|
model request from the floor. Face indexing, segmentation and scene grading all work, at
|
||||||
today's speed or better (ORT CPU).
|
today's speed or better (ORT CPU).
|
||||||
2. **Probe reports** — say, TensorRT. The compiling rung is now *selected* but has **no engines**.
|
2. **Probe reports** — say, TensorRT. The compiling rung is now *selected* but has **no engines**.
|
||||||
Model requests continue on the fallback rung below it (CUDA provider for TensorRT; ORT CPU for
|
Model requests continue on the fallback rung below it (CUDA provider for TensorRT; ORT CPU for
|
||||||
Hexagon), which needs no compilation and is already faster than the floor.
|
MIGraphX and Hexagon), which needs no compilation and is already faster than the floor.
|
||||||
3. **Engines build**, one model at a time, on a single low-priority background thread, smallest
|
3. **Engines build**, one model at a time, on a single low-priority background thread, smallest
|
||||||
model first so the detector — the one that runs per image — is ready soonest. On the reference
|
model first so the detector — the one that runs per image — is ready soonest. On the reference
|
||||||
desktop that is ~1 minute for the first detector and ~10 minutes for all six at fp16; on the tablet the QNN
|
desktop that is ~1 minute for the first detector and ~10 minutes for all six at fp16; on the AMD
|
||||||
|
desktop 40 s for the first detector and ~8 minutes for the set; on the tablet the QNN
|
||||||
context binaries take 0.8–1.7 s each and the whole set is ready before the user has opened a
|
context binaries take 0.8–1.7 s each and the whole set is ready before the user has opened a
|
||||||
library. Each engine is written to a temporary name and renamed into place, so a request never
|
library. Each engine is written to a temporary name and renamed into place, so a request never
|
||||||
sees a half-written file.
|
sees a half-written file.
|
||||||
@@ -313,7 +369,7 @@ enough to be *offered* at all. f32 on tract, ORT CPU, CUDA and TensorRT-f32 are
|
|||||||
same graph, the same arithmetic, differences at the last bit.
|
same graph, the same arithmetic, differences at the last bit.
|
||||||
|
|
||||||
**The embedder** is where comparability across devices is the whole point, and it is the one
|
**The embedder** is where comparability across devices is the whole point, and it is the one
|
||||||
model that no accelerator helps (§1.3). So: **the embedder runs in f32 on every rung.** On TensorRT
|
model that no accelerator helps (§1.4). So: **the embedder runs in f32 on every rung.** On TensorRT
|
||||||
that means the embedder's engine is built without fp16 while the detector's is built with it; on
|
that means the embedder's engine is built without fp16 while the detector's is built with it; on
|
||||||
the Hexagon it means the embedder is not on the NPU at all — it runs on the ORT CPU rung at 9 ms,
|
the Hexagon it means the embedder is not on the NPU at all — it runs on the ORT CPU rung at 9 ms,
|
||||||
and the ladder's "one backend per device" is, precisely, one backend *per model role*, with the
|
and the ladder's "one backend per device" is, precisely, one backend *per model role*, with the
|
||||||
@@ -344,7 +400,7 @@ core/dr-inference-engine
|
|||||||
src/probe.rs §4 — the ladder per platform, the session-build probe, the cache file
|
src/probe.rs §4 — the ladder per platform, the session-build probe, the cache file
|
||||||
src/engines.rs §6 — background compilation, the cache directory, progress
|
src/engines.rs §6 — background compilation, the cache directory, progress
|
||||||
src/session.rs open(role, bytes) -> ort::Session, applying the rung and the role's precision rule
|
src/session.rs open(role, bytes) -> ort::Session, applying the rung and the role's precision rule
|
||||||
src/api.rs the one unsafe block: dlopen libonnxruntime, fetch OrtApi, ort::set_api — or ort_tract::api()
|
src/api.rs the unsafe block that matters: dlopen libonnxruntime, fetch OrtApi, ort::set_api — or ort_tract::api()
|
||||||
```
|
```
|
||||||
|
|
||||||
- `dr-face` and `dr-segment` **delete** their private `install_backend` and their direct
|
- `dr-face` and `dr-segment` **delete** their private `install_backend` and their direct
|
||||||
@@ -354,7 +410,11 @@ core/dr-inference-engine
|
|||||||
`qnn`: those features add option builders, not linking, under `alternative-backend`. **Verified
|
`qnn`: those features add option builders, not linking, under `alternative-backend`. **Verified
|
||||||
for the QNN, CUDA and TensorRT builders on 2026-09-19** — they go through the API table's generic
|
for the QNN, CUDA and TensorRT builders on 2026-09-19** — they go through the API table's generic
|
||||||
`SessionOptionsAppendExecutionProvider*`. The NNAPI builder resolves a symbol directly and would
|
`SessionOptionsAppendExecutionProvider*`. The NNAPI builder resolves a symbol directly and would
|
||||||
not; it is not needed and is not enabled.
|
not; it is not needed and is not enabled. MIGraphX uses no `ort` feature at all: `ort`'s builder
|
||||||
|
fills the legacy options struct, which ONNX Runtime 1.29 reads for the precision flags and
|
||||||
|
nothing else, and the compiled-program cache directory only travels through the generic
|
||||||
|
key/value entry point (`migraphx_model_cache_dir`). `session::migraphx` makes that one call
|
||||||
|
on the API table itself.
|
||||||
- `dr-ui` owns the settings row, the about-screen line and the progress row; it holds one
|
- `dr-ui` owns the settings row, the about-screen line and the progress row; it holds one
|
||||||
`Sessions` per process, created at launch, and passes it down. `dr_ui::library` gains
|
`Sessions` per process, created at launch, and passes it down. `dr_ui::library` gains
|
||||||
`inference_cache_dir()` beside `shared_face_models_dir()`, on the same account-independent
|
`inference_cache_dir()` beside `shared_face_models_dir()`, on the same account-independent
|
||||||
@@ -18,11 +18,11 @@ The core is further along than the interface, and it is worth being exact
|
|||||||
about which half is missing, because it changes the size of the work.
|
about which half is missing, because it changes the size of the work.
|
||||||
|
|
||||||
**Painting exists everywhere except where a finger is.**
|
**Painting exists everywhere except where a finger is.**
|
||||||
[`MaskSource::Brush`](../core/dr-pipeline/src/mask.rs), [`Stroke`], the
|
[`MaskSource::Brush`](../../core/dr-pipeline/src/mask.rs), [`Stroke`], the
|
||||||
simplification and the point budgets, the sidecar's `stroke = …` line and its
|
simplification and the point budgets, the sidecar's `stroke = …` line and its
|
||||||
parser, the GPU's per-stroke bounding-box draw with add and erase blend
|
parser, the GPU's per-stroke bounding-box draw with add and erase blend
|
||||||
states — all of it is written, tested, and reachable from no control in the
|
states — all of it is written, tested, and reachable from no control in the
|
||||||
application. [`toolrail.slint:138`](../ui/dr-ui/ui/toolrail.slint#L138) says so
|
application. [`toolrail.slint:138`](../../ui/dr-ui/ui/toolrail.slint#L138) says so
|
||||||
in as many words: *"what is missing is the canvas interaction"*.
|
in as many words: *"what is missing is the canvas interaction"*.
|
||||||
|
|
||||||
**A mask has exactly one source.** A layer is one `MaskSource` and a shaping
|
**A mask has exactly one source.** A layer is one `MaskSource` and a shaping
|
||||||
@@ -37,7 +37,7 @@ shoulder and leaks four pixels into the hair, no global number fixes both, and
|
|||||||
that is the ordinary case rather than a corner one.
|
that is the ordinary case rather than a corner one.
|
||||||
|
|
||||||
**And you cannot see the mask.** *(Built — see §6.)* The overlay on the canvas
|
**And you cannot see the mask.** *(Built — see §6.)* The overlay on the canvas
|
||||||
was [`overlay_rgba`](../ui/dr-ui/src/segmentation.rs) and nothing else — a
|
was [`overlay_rgba`](../../ui/dr-ui/src/segmentation.rs) and nothing else — a
|
||||||
CPU-built false-colour picture of *what the model detected*, at proxy
|
CPU-built false-colour picture of *what the model detected*, at proxy
|
||||||
resolution. It is not the layer's alpha: it knows nothing of the layer's
|
resolution. It is not the layer's alpha: it knows nothing of the layer's
|
||||||
feather, its falloff, its morphology, its invert, or its opacity. Nobody can
|
feather, its falloff, its morphology, its invert, or its opacity. Nobody can
|
||||||
@@ -234,7 +234,7 @@ writes with one part is byte-identical to what it writes today**:
|
|||||||
3. `stroke = …` lines keep their position and their meaning. The mode token
|
3. `stroke = …` lines keep their position and their meaning. The mode token
|
||||||
gains `push`, and `cling` is a fourth number written only when non-zero.
|
gains `push`, and `cling` is a fourth number written only when non-zero.
|
||||||
An older build meeting either drops *that stroke and only that stroke*,
|
An older build meeting either drops *that stroke and only that stroke*,
|
||||||
which is the rule [`parse_stroke`](../core/dr-pipeline/src/sidecar.rs)
|
which is the rule [`parse_stroke`](../../core/dr-pipeline/src/sidecar.rs)
|
||||||
already documents and already implements.
|
already documents and already implements.
|
||||||
|
|
||||||
**Merge (FR-NC-9).** A part is a block with an id, so two devices that added
|
**Merge (FR-NC-9).** A part is a block with an id, so two devices that added
|
||||||
@@ -272,7 +272,7 @@ The set operations are already expressible in fixed-function blending over
|
|||||||
| Intersect | `Zero`, `Src`, `Add` | `dst · src` | M2 |
|
| Intersect | `Zero`, `Src`, `Add` | `dst · src` | M2 |
|
||||||
|
|
||||||
The middle row is `brush_erase`, already constructed in
|
The middle row is `brush_erase`, already constructed in
|
||||||
[`MaskPass::new`](../core/dr-gpu/src/mask.rs). The other two are the same
|
[`MaskPass::new`](../../core/dr-gpu/src/mask.rs). The other two are the same
|
||||||
three vertices with a different `BlendState`, and nothing is read back.
|
three vertices with a different `BlendState`, and nothing is read back.
|
||||||
|
|
||||||
**One correction to the first draft of this section, found in the building.**
|
**One correction to the first draft of this section, found in the building.**
|
||||||
@@ -289,7 +289,7 @@ the first time any layer has more than one part) and blended from there. A
|
|||||||
layer of one part still takes the old path exactly — straight into its slice,
|
layer of one part still takes the old path exactly — straight into its slice,
|
||||||
no scratch, no combine pass — which is what keeps every existing mask
|
no scratch, no combine pass — which is what keeps every existing mask
|
||||||
rendering as it did. `an_erase_stroke_holes_its_own_part_and_not_the_mask` in
|
rendering as it did. `an_erase_stroke_holes_its_own_part_and_not_the_mask` in
|
||||||
[`local_adjustments.rs`](../core/dr-gpu/tests/local_adjustments.rs) is the test
|
[`local_adjustments.rs`](../../core/dr-gpu/tests/local_adjustments.rs) is the test
|
||||||
that holds this in place.
|
that holds this in place.
|
||||||
|
|
||||||
`Max` blending on `r8unorm` is core WGPU and universally supported on the
|
`Max` blending on `r8unorm` is core WGPU and universally supported on the
|
||||||
@@ -342,7 +342,7 @@ Three honest costs:
|
|||||||
|
|
||||||
### 5.4 Painting has to be incremental
|
### 5.4 Painting has to be incremental
|
||||||
|
|
||||||
Today [`MaskPass::render`](../core/dr-gpu/src/mask.rs) clears each slice and
|
Today [`MaskPass::render`](../../core/dr-gpu/src/mask.rs) clears each slice and
|
||||||
redraws every stroke of the layer. That is right when a shape changes and
|
redraws every stroke of the layer. That is right when a shape changes and
|
||||||
wrong while a finger is down: at 120 reports a second, a layer holding 4096
|
wrong while a finger is down: at 120 reports a second, a layer holding 4096
|
||||||
points redraws all of them per dab, and the cost of a stroke grows as it is
|
points redraws all of them per dab, and the cost of a stroke grows as it is
|
||||||
@@ -359,14 +359,14 @@ changing falls back to the full rebuild it does now.
|
|||||||
|
|
||||||
This is required, not an optimisation to schedule later: it is what decides
|
This is required, not an optimisation to schedule later: it is what decides
|
||||||
whether painting is usable on the phone, and it is the specific failure
|
whether painting is usable on the phone, and it is the specific failure
|
||||||
[`mask.rs`'s module docs](../core/dr-pipeline/src/mask.rs) say this whole
|
[`mask.rs`'s module docs](../../core/dr-pipeline/src/mask.rs) say this whole
|
||||||
design exists to avoid.
|
design exists to avoid.
|
||||||
|
|
||||||
### 5.5 Distance fields become per part
|
### 5.5 Distance fields become per part
|
||||||
|
|
||||||
[`SubjectMasks`](../core/dr-gpu/src/mask.rs) uploads one signed distance field
|
[`SubjectMasks`](../../core/dr-gpu/src/mask.rs) uploads one signed distance field
|
||||||
**per active layer, in stack order**, and
|
**per active layer, in stack order**, and
|
||||||
[`DevelopSession`](../ui/dr-ui/src/develop.rs) builds them on the same
|
[`DevelopSession`](../../ui/dr-ui/src/develop.rs) builds them on the same
|
||||||
indexing. With parts, a field belongs to the part that shaped it: the upload
|
indexing. With parts, a field belongs to the part that shaped it: the upload
|
||||||
becomes one field per *model-backed part*, flattened in `(layer, part)` order,
|
becomes one field per *model-backed part*, flattened in `(layer, part)` order,
|
||||||
and the rasteriser indexes it by a running counter rather than by `slot`.
|
and the rasteriser indexes it by a running counter rather than by `slot`.
|
||||||
@@ -540,7 +540,7 @@ layer's.
|
|||||||
or painted. One code path, three joins.
|
or painted. One code path, three joins.
|
||||||
|
|
||||||
**Folding two layers into one** uses the multi-selection
|
**Folding two layers into one** uses the multi-selection
|
||||||
[`masks_ui.rs`](../ui/dr-ui/src/masks_ui.rs) already supports: with two layers
|
[`masks_ui.rs`](../../ui/dr-ui/src/masks_ui.rs) already supports: with two layers
|
||||||
selected, "Combine" appends the second's parts to the first and removes it.
|
selected, "Combine" appends the second's parts to the first and removes it.
|
||||||
Offered only when the second layer's adjustments are neutral, and otherwise
|
Offered only when the second layer's adjustments are neutral, and otherwise
|
||||||
offered with a warning that names what will be lost — quietly discarding an
|
offered with a warning that names what will be lost — quietly discarding an
|
||||||
@@ -560,7 +560,7 @@ who sets a small eraser expects it to still be small the next time they erase.
|
|||||||
### 7.4 Gestures
|
### 7.4 Gestures
|
||||||
|
|
||||||
Each of these needs a `GESTURE:` block beside its implementation — that is the
|
Each of these needs a `GESTURE:` block beside its implementation — that is the
|
||||||
only place [gestures.md](gestures.md) can be written from.
|
only place [gestures.md](../gestures.md) can be written from.
|
||||||
|
|
||||||
| Gesture | Touch | Pointer | Keyboard |
|
| Gesture | Touch | Pointer | Keyboard |
|
||||||
|---------|-------|---------|----------|
|
|---------|-------|---------|----------|
|
||||||
@@ -611,7 +611,7 @@ removing or re-joining a part.
|
|||||||
The mask stack is already snapshotted per step and shared by `Arc` when a step
|
The mask stack is already snapshotted per step and shared by `Arc` when a step
|
||||||
does not touch it, so the cost of an undoable stroke is a clone of one layer's
|
does not touch it, so the cost of an undoable stroke is a clone of one layer's
|
||||||
parts, not of the picture. New keys in
|
parts, not of the picture. New keys in
|
||||||
[`labels.rs`](../ui/dr-ui/src/labels.rs):
|
[`labels.rs`](../../ui/dr-ui/src/labels.rs):
|
||||||
|
|
||||||
```
|
```
|
||||||
history.mask_painted "Paint Mask"
|
history.mask_painted "Paint Mask"
|
||||||
@@ -196,7 +196,7 @@ inside `#[cfg(test)]` — `LocalStorage::open` refuses it, and the test that pro
|
|||||||
`dr_plat::imports_supported`, which *returns false on Android* and whose own documentation says it
|
`dr_plat::imports_supported`, which *returns false on Android* and whose own documentation says it
|
||||||
"stops being false when a SAF implementation lands". The second tag documented the absence of the
|
"stops being false when a SAF implementation lands". The second tag documented the absence of the
|
||||||
thing it was counted as evidence for. Both have been removed; this is the "plumbing a future feature
|
thing it was counted as evidence for. Both have been removed; this is the "plumbing a future feature
|
||||||
would use" case [CONTRIBUTING.md](../CONTRIBUTING.md) and [code-health.md CH-4](code-health.md) both
|
would use" case [CONTRIBUTING.md](../../CONTRIBUTING.md) and [code-health.md CH-4](code-health.md) both
|
||||||
warn about. Android reaches a library through a Nextcloud account or a folder, over paths, like the
|
warn about. Android reaches a library through a Nextcloud account or a folder, over paths, like the
|
||||||
desktop.
|
desktop.
|
||||||
|
|
||||||
@@ -355,10 +355,10 @@ synthetic 50k catalog, run per commit, where **"a regression beyond a stated tol
|
|||||||
failure, not a notification."** For most of this project's life it did not exist — no `benches/`, no
|
failure, not a notification."** For most of this project's life it did not exist — no `benches/`, no
|
||||||
criterion, no synthetic catalog, and three CI workflows that between them measured nothing.
|
criterion, no synthetic catalog, and three CI workflows that between them measured nothing.
|
||||||
|
|
||||||
**It exists now, for everything that does not need a frame.** [`tools/bench`](../tools/bench) builds
|
**It exists now, for everything that does not need a frame.** [`tools/bench`](../../tools/bench) builds
|
||||||
a deterministic 50,000-row catalog over a pool of a dozen real files, measures against it, and fails
|
a deterministic 50,000-row catalog over a pool of a dozen real files, measures against it, and fails
|
||||||
the build on a violated budget or a drift past tolerance;
|
the build on a violated budget or a drift past tolerance;
|
||||||
[`.gitea/workflows/benchmark.yml`](../.gitea/workflows/benchmark.yml) runs it on every push, and
|
[`.gitea/workflows/benchmark.yml`](../../.gitea/workflows/benchmark.yml) runs it on every push, and
|
||||||
[benchmarks.md](benchmarks.md) is the account of what it does and does not cover. **NFR-P1** and
|
[benchmarks.md](benchmarks.md) is the account of what it does and does not cover. **NFR-P1** and
|
||||||
**NFR-P3** are now genuinely gated, and R2's "catalog opens in under 2s" clause with them.
|
**NFR-P3** are now genuinely gated, and R2's "catalog opens in under 2s" clause with them.
|
||||||
|
|
||||||
@@ -404,7 +404,7 @@ tag anything against.
|
|||||||
NFR-OPS-1 was covered by tags that were real rather than fixtures, which is the worse case of the
|
NFR-OPS-1 was covered by tags that were real rather than fixtures, which is the worse case of the
|
||||||
two: one on `compute_coverage` and one on the gesture extractor, both on the traceability tool. A
|
two: one on `compute_coverage` and one on the gesture extractor, both on the traceability tool. A
|
||||||
coverage calculation and a documentation generator are not diagnostics under any reading, so both
|
coverage calculation and a documentation generator are not diagnostics under any reading, so both
|
||||||
tags were removed. It is the case [CONTRIBUTING.md](../CONTRIBUTING.md) warns about in its own words:
|
tags were removed. It is the case [CONTRIBUTING.md](../../CONTRIBUTING.md) warns about in its own words:
|
||||||
a tag proves a tag exists. The requirement has since been built where it says: the rotating,
|
a tag proves a tag exists. The requirement has since been built where it says: the rotating,
|
||||||
size-capped log and its redaction in `platform/dr-plat/src/diagnostics.rs` (2026-08-30), and the
|
size-capped log and its redaction in `platform/dr-plat/src/diagnostics.rs` (2026-08-30), and the
|
||||||
bundle in `diagnostics/bundle.rs` (2026-09-12) — the log, the crash records, the version, the schema
|
bundle in `diagnostics/bundle.rs` (2026-09-12) — the log, the crash records, the version, the schema
|
||||||
@@ -179,7 +179,7 @@ exports the network alone at 768×1024 — thirteen operator types, all
|
|||||||
standard: `Conv`, `InstanceNormalization`, `AveragePool`, `Resize`, `Slice`,
|
standard: `Conv`, `InstanceNormalization`, `AveragePool`, `Resize`, `Slice`,
|
||||||
`Transpose`, `Reshape`, `Concat`, `Add`, `Relu`, `Sigmoid`, `ReduceMean`,
|
`Transpose`, `Reshape`, `Concat`, `Add`, `Relu`, `Sigmoid`, `ReduceMean`,
|
||||||
`Unsqueeze` — and
|
`Unsqueeze` — and
|
||||||
[`examples/onnx_probe.rs`](../core/dr-segment/examples/onnx_probe.rs) loads
|
[`examples/onnx_probe.rs`](../../core/dr-segment/examples/onnx_probe.rs) loads
|
||||||
the 2.8 MB file through the app's own `ort`-over-tract backend with nothing
|
the 2.8 MB file through the app's own `ort`-over-tract backend with nothing
|
||||||
unsupported, in 28 ms, and runs it in **~300 ms on the reference desktop's
|
unsupported, in 28 ms, and runs it in **~300 ms on the reference desktop's
|
||||||
CPU**. The weights ship as `models/keypoints/xfeat-1024.onnx`, recorded in
|
CPU**. The weights ship as `models/keypoints/xfeat-1024.onnx`, recorded in
|
||||||
@@ -255,7 +255,7 @@ Two containers were candidates and S15.1 decided, on 2026-09-19:
|
|||||||
the existing encoder with a different sample type, and nothing about it is
|
the existing encoder with a different sample type, and nothing about it is
|
||||||
uncertain.
|
uncertain.
|
||||||
|
|
||||||
**Linear DNG.** [`examples/linear_dng.rs`](../core/dr-decode/examples/linear_dng.rs)
|
**Linear DNG.** [`examples/linear_dng.rs`](../../core/dr-decode/examples/linear_dng.rs)
|
||||||
hand-rolls a 64 × 48 `LinearRaw` DNG — one IFD, uncompressed 16-bit RGB,
|
hand-rolls a 64 × 48 `LinearRaw` DNG — one IFD, uncompressed 16-bit RGB,
|
||||||
`DNGVersion`, `ColorMatrix1`, `AsShotNeutral`, `CalibrationIlluminant1` — and
|
`DNGVersion`, `ColorMatrix1`, `AsShotNeutral`, `CalibrationIlluminant1` — and
|
||||||
rawler 0.7 reads it back: `cpp 3`, the samples interleaved as written, the
|
rawler 0.7 reads it back: `cpp 3`, the samples interleaved as written, the
|
||||||
@@ -538,9 +538,24 @@ each loss weighting did, and the two runs abandoned (blur under L1 in
|
|||||||
the hole; a brick pattern under a strong adversarial term against a
|
the hole; a brick pattern under a strong adversarial term against a
|
||||||
discriminator that had not learned) — is `runs/` in `darkroom-infill`.
|
discriminator that had not learned) — is `runs/` in `darkroom-infill`.
|
||||||
|
|
||||||
**What is still wrong.** The ground fill is softer than its context —
|
**Second model, the same day.** The morning's fill was soft in the deep
|
||||||
texture, not structure, is what a night on a laptop GPU could not finish.
|
ground bands. The afternoon's run trained the generator against MI-GAN's
|
||||||
The levers, in order: a discriminator that learns (a pretrained one —
|
own pretrained discriminator (non-saturating loss, lazy R1, feature
|
||||||
MI-GAN's own from the unfused checkpoint — instead of a PatchGAN from
|
matching), with fresh noise inputs while training and flip/translation
|
||||||
scratch), feature matching, and more steps at 512. FR-MRG-4's
|
augmentation of the discriminator's input — both needed, or the generator
|
||||||
*experimental* stays.
|
settles into a periodic texture the discriminator cannot see. The shipped
|
||||||
|
weights (step 4 750 of `runs/border-v6`) are level with the stock model on
|
||||||
|
LPIPS (edge 0.125 / corner 0.187 against 0.121 / 0.183) while keeping the
|
||||||
|
PSNR gain (edge 18.0 / corner 15.7 against 16.9 / 14.6). On the fixture
|
||||||
|
the ground bands now carry texture at the right tone; at 1:1 a faint
|
||||||
|
regular hatch is visible in the deepest part.
|
||||||
|
|
||||||
|
**What is still wrong.** The hatch, and any deep textured void the
|
||||||
|
generator must invent. The better answer for those is not generative:
|
||||||
|
seed the void with the picture's own texture in hexagonal cells, let the
|
||||||
|
discriminator rank the candidates, and let the generator heal only the
|
||||||
|
gaps between cells — built and measured in `darkroom-infill`
|
||||||
|
(`infill/hexfill.py`), the most convincing scree corner produced so far,
|
||||||
|
and the next thing to port into `dr_pano::fill` (it needs the
|
||||||
|
discriminator as a second model, ~80 MB fp16). FR-MRG-4's *experimental*
|
||||||
|
stays.
|
||||||
@@ -1814,6 +1814,28 @@ tool for a hand-held set, and a photograph with real parallax is not a panorama.
|
|||||||
in one operation until HDR merge exists on its own. No live re-stitch: a different projection or
|
in one operation until HDR merge exists on its own. No live re-stitch: a different projection or
|
||||||
crop after the fact is a new file, not an edit. No video.
|
crop after the fact is a new file, not an edit. No video.
|
||||||
|
|
||||||
|
### 3.12 Inference runtime
|
||||||
|
|
||||||
|
The face, segmentation and border-fill models run under an inference engine the application
|
||||||
|
selects per device. [inference.md](inference.md) is the design: its §3 sets the dependency policy
|
||||||
|
the runtime may reopen, its §10 the milestones (M1–M7) the clauses below cite as acceptance. The
|
||||||
|
clauses are the register entries inference.md §12 promised; they are stated here so the
|
||||||
|
traceability gate can count them.
|
||||||
|
|
||||||
|
**FR-INF-1 — Runtime selection.** On launch the application shall determine, per device and
|
||||||
|
without blocking the first frame, the fastest inference backend that can build and run a session
|
||||||
|
for the shipped models, by attempting it; shall record and reuse that determination until the
|
||||||
|
runtime, driver, hardware or models change; and shall display the backend in use in Settings and
|
||||||
|
on the about screen. *Acceptance:* inference.md §10 M1 and M5.
|
||||||
|
|
||||||
|
**FR-INF-2 — Derived engines.** Backends that require device-specific compilation shall compile in
|
||||||
|
the background after selection, shall serve requests from the next lower backend until each engine
|
||||||
|
is ready, and shall not change the backend of a job in progress. *Acceptance:* M5.
|
||||||
|
|
||||||
|
**FR-INF-3 — Model forms.** Quantised model forms are produced at release time from real
|
||||||
|
calibration data and are shipped only when they meet inference.md §10's accuracy gates against the
|
||||||
|
canonical form; the application never quantises on the device. *Acceptance:* M2, M7.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 4. Non-functional requirements
|
## 4. Non-functional requirements
|
||||||
@@ -2111,6 +2133,13 @@ it is far cheaper to discover now than after the UI is built.
|
|||||||
clipping indicators (FR-DSP-7), and the HSL mixer carry a shape or text affordance. This matters
|
clipping indicators (FR-DSP-7), and the HSL mixer carry a shape or text affordance. This matters
|
||||||
more in a colour-grading application than in most software.
|
more in a colour-grading application than in most software.
|
||||||
|
|
||||||
|
### 4.10 Inference
|
||||||
|
|
||||||
|
**NFR-INF-1 — Embedding comparability.** Face embeddings shall be computed at a precision whose
|
||||||
|
deviation from the f32 reference is within [inference.md](inference.md) §7's gate, on every
|
||||||
|
backend, so that embeddings from any device are comparable — FR-CULL-9's calibration depends on
|
||||||
|
it. *Acceptance:* inference.md §10 M3.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 5. Data model and architecture
|
## 5. Data model and architecture
|
||||||
@@ -2140,7 +2169,7 @@ Rationale, evidence, and the eliminated alternatives are recorded in
|
|||||||
|---|---|---|
|
|---|---|---|
|
||||||
| D1 | Language and UI framework | Rust + Slint, rendering through wgpu |
|
| D1 | Language and UI framework | Rust + Slint, rendering through wgpu |
|
||||||
| D2 | RAW decoder | rawler; LibRaw fallback behind a trait |
|
| D2 | RAW decoder | rawler; LibRaw fallback behind a trait |
|
||||||
| D3 | First milestone | Delivered — [milestone-v0.1.md](milestone-v0.1.md), closed 2026-08-30 |
|
| D3 | First milestone | Delivered — [milestone-v0.1.md](archive/milestone-v0.1.md), closed 2026-08-30 |
|
||||||
| D4 | Nextcloud sync mechanism | ETag pruning, chunked upload v2, Login Flow v2 |
|
| D4 | Nextcloud sync mechanism | ETag pruning, chunked upload v2, Login Flow v2 |
|
||||||
| D5 | Colour management | lcms2 + GPU-side matrix/LUT transforms |
|
| D5 | Colour management | lcms2 + GPU-side matrix/LUT transforms |
|
||||||
| D6 | Shader authoring | Hand-written WGSL |
|
| D6 | Shader authoring | Hand-written WGSL |
|
||||||
@@ -21,11 +21,11 @@ one in its class and the workflow still breaks at the first frame with a mark on
|
|||||||
the sky.
|
the sky.
|
||||||
|
|
||||||
It is also, unusually, a feature whose cost has already been paid twice over.
|
It is also, unusually, a feature whose cost has already been paid twice over.
|
||||||
The neighbourhood stage exists ([`crate::detail`](../core/dr-pipeline/src/detail.rs)),
|
The neighbourhood stage exists ([`crate::detail`](../../core/dr-pipeline/src/detail.rs)),
|
||||||
the convention for storing geometry in normalised source coordinates exists
|
the convention for storing geometry in normalised source coordinates exists
|
||||||
([`mask.rs`](../core/dr-pipeline/src/mask.rs)), the canvas-drag pattern exists
|
([`mask.rs`](../../core/dr-pipeline/src/mask.rs)), the canvas-drag pattern exists
|
||||||
([`gradient.rs`](../ui/dr-ui/src/gradient.rs)), and the merge-by-id rule exists
|
([`gradient.rs`](../../ui/dr-ui/src/gradient.rs)), and the merge-by-id rule exists
|
||||||
([`sidecar.rs`](../core/dr-pipeline/src/sidecar.rs)). What is genuinely new is
|
([`sidecar.rs`](../../core/dr-pipeline/src/sidecar.rs)). What is genuinely new is
|
||||||
small and is named in §3.
|
small and is named in §3.
|
||||||
|
|
||||||
## 2. Non-goals
|
## 2. Non-goals
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user