Record the manual's scenes by name, and each against what it depends on

scenes.py aimed every press at window pixels, and the develop column had
already moved under it: Compose now sits above Adjust, so the old
exposure coordinate lands on a straighten slider. Every scene now names
what it presses by its accessible label through the automation hook,
places points on the photograph relative to the canvas, and opens its
photographs by file name. Each starts from a known place and undoes what
it did, so one can be recorded alone; the few that continue another's
state name it, and running one runs that first into a scratch folder.

Each scene also declares the pictures it makes and the sources they
depend on. `record.sh --check` fails when the manual shows a picture no
scene makes, or a scene makes one it does not show; it reads two files.
`record.sh --changed` re-records the scenes whose sources, or own code,
changed since the commit that last touched their pictures. record.sh
builds with the automation feature, restores the library from
DR_LIBRARY_SNAPSHOT, starts from a fresh profile and pins inference to
the CPU; the launch screen is recorded from an empty profile of its own.

Re-recorded with the ported scenes, and looked at frame by frame. What
differs from the pictures they replace:
- develop, presets, settings, local, compose, film, wb, light: the
  current develop column (Compose with Vertical and Horizontal above
  Adjust, the Label button), otherwise the same moments.
- library pictures: the filter bar's colour-label chips; no collection
  left over from an earlier run in the sidebar; library-selection is the
  twelve alpine frames rather than eight of them and four New York ones.
- library-rating rates two frames nobody had rated, so the stars are set
  and not cleared.
- develop-zoom goes on past 1:1 with the wheel and ends on the file's
  pixels as hard-edged blocks.
- repair covers a real mark on the road, with a size that fits it; film
  is shown on the Chinatown frame instead of the road.
- panorama tries Perspective, Spherical and Cylindrical before filling.
- launch, launch-folder and panorama-aligned came out byte-identical.

tools/manual/README.md

@@ -1,20 +1,51 @@
 # tools/manual — the pictures in docs/manual
 
-`record.sh <library>` drives the desktop build on a private X server and
+`record.sh` drives the desktop build on a private X server and records the
-records every scene in `scenes.py` into `docs/manual/media/`. `drive.py` is
+scenes in `scenes.py` into `docs/manual/media/`:
-the puppeteer underneath — launch, click, drag, type, screenshot, record —
-and is usable on its own to look at a panel after changing it:
 
 ```bash
-DR_HOME=/var/tmp/x tools/manual/drive.py launch /some/folder
+tools/manual/record.sh /var/tmp/dr-demo/library              # every scene
-tools/manual/drive.py click 1543 22        # Settings
+tools/manual/record.sh /var/tmp/dr-demo/library develop_zoom # one, and what it needs
-tools/manual/drive.py shot /tmp/settings.png
+tools/manual/record.sh --changed /var/tmp/dr-demo/library    # those whose sources moved
+tools/manual/record.sh --check                               # the manual and the scenes agree
+tools/manual/record.sh --list
+```
+
+Set `DR_LIBRARY_SNAPSHOT` to a pristine copy of the library: a recording
+rates, files and merges, and the snapshot is copied over the library first.
+Each run starts from a fresh profile under `DR_HOME`.
+
+## Controls by name
+
+The binary is built with `--features automation`, which lets the app answer
+"where is the control labelled *Exposure*?" over the Unix socket named by
+`DR_AUTOMATION` (`ui/dr-ui/src/automation.rs`). No other build has the
+feature, and a build that has it listens only when the variable is set. The
+names are the accessible labels a screen reader reads, so a scene says
+`click_on('Select@Button')` or `slide('Exposure', 24)` rather than a pixel,
+and survives a panel that moves. The pointer itself is still xdotool.
+
+```bash
+DR_HOME=/var/tmp/x tools/manual/drive.py launch /some/folder   # a feature build in DR_BIN
+tools/manual/drive.py labels                 # every name on screen
+tools/manual/drive.py click-on 'Settings@Button'
+tools/manual/drive.py ids canvas             # things with no name: id:canvas-image
 tools/manual/drive.py stop
 ```
 
-The demo library is the author's: seventy frames with no people in them,
+Grid cells are named by file (`_MG_8393@ListItem`), rating stars `1 star`
-the `fixtures/pano` set among them. The scenes' cell coordinates are for
+to `5 stars`. What a scene cannot name — a point on the photograph, the crop
-that grid, at 1600×1100; the panel coordinates hold for any library.
+rectangle's corner — it places relative to something it can
+(`drive.photo(fx, fy)`, `id:move-area`).
+
+## Scenes and the manual
+
+Each scene in `scenes.py` declares the pictures it makes, as the manual names
+them, and the globs of the sources that could change them. `--check` fails
+when the manual shows a picture no scene makes, or a scene makes one the
+manual does not show; it reads two files and runs in CI. `--changed`
+re-records a scene when one of its sources, or its own code, has changed
+since the commit that last touched its pictures.
 
 Two things the scripts know that are not obvious: a Slint `TouchArea` wants
 a held press, not xdotool's `click`; and the app is driven on Xvfb rather

tools/manual/drive.py

@@ -31,8 +31,10 @@ Suffixes narrow it — `@ROLE` takes only elements of that role (`Button`,
 `#-1` the last (a sheet is drawn after what it covers). A leading `id:`
 names an element by its id in the markup or its component type instead
 (`id:canvas-image`, `id:move-area`, `id:Timeline`), for the few things a
-scene aims at that are not controls. Only what Slint draws is answered:
+scene aims at that are not controls. Only what Slint draws is answered,
-nothing hidden, nothing scrolled out of its list.
+and only where its centre is in the window: nothing hidden, nothing
+scrolled out of its list. An element half under the edge of a scrolled
+list still counts, so a scene that needs all of one scrolls it into view.
 
 The input itself is still xdotool — a real pointer, pressed and held — so a
 scene exercises exactly what a hand would. The hook only says where to aim.
@@ -229,6 +231,17 @@ def wait_ready(timeout=120):
         time.sleep(0.5)
 
 
+_window = None
+
+
+def window_size():
+    global _window
+    if _window is None:
+        w = ask('window')
+        _window = (w['w'], w['h'])
+    return _window
+
+
 def parse_name(name):
     """`LABEL`, `id:ID`, each with optional `@ROLE` and `#N`."""
     m = re.match(r'^(.*?)(?:@(\w+))?(?:#(-?\d+))?$', name, re.S)
@@ -244,7 +257,9 @@ def matches(name, within=None):
         hits = ask(f'locate-id {what[3:]}')
     else:
         hits = ask(f'locate {what}')
-    hits = [e for e in hits if e['w'] > 0 and e['h'] > 0 and e['opacity'] > 0]
+    W, H = window_size()
+    hits = [e for e in hits if e['w'] > 0 and e['h'] > 0 and e['opacity'] > 0
+            and 0 <= e['x'] + e['w'] / 2 < W and 0 <= e['y'] + e['h'] / 2 < H]
     if role:
         hits = [e for e in hits if (e['role'] or '').lower() == role.lower()]
     if within:
@@ -351,7 +366,11 @@ def rec_start(out):
 
 
 def rec_stop():
-    pid = int(open(f'{HOME}/rec.pid').read())
+    """Stop the recording `rec_start` began. The pid file goes with it, so
+    a second stop — or one after a scene failed — signals nothing."""
+    path = f'{HOME}/rec.pid'
+    pid = int(open(path).read())
+    os.remove(path)
     os.kill(pid, 2)
     for _ in range(50):
         try:

tools/manual/record.sh

@@ -1,57 +1,145 @@
 #!/usr/bin/env bash
 # Re-record the manual: docs/manual/README.md's pictures, from the app itself.
 #
-#   tools/manual/record.sh <library-folder> [scene...]
+#   tools/manual/record.sh LIBRARY [SCENE...]      every scene, or those named
+#   tools/manual/record.sh --changed LIBRARY       the scenes whose sources
+#                                                  changed since their pictures
+#   tools/manual/record.sh --check                 the manual and the scenes
+#                                                  agree (no app, no display)
+#   tools/manual/record.sh --list                  the scenes and their pictures
 #
-# Needs Xvfb, xdotool, ImageMagick's `import` and ffmpeg. Builds the desktop
+# Needs Xvfb, xdotool, ImageMagick's `import`, ffmpeg and optipng. Builds the
-# binary if it is not there, starts a private X server, opens the folder as
+# desktop binary with the `automation` feature — the hook scenes.py finds
-# a library in a throwaway profile, runs the scenes (all of them by default)
+# controls by name through — unless DR_BIN names one. Starts a private X
-# and writes PNGs and GIFs into docs/manual/media/.
+# server, opens LIBRARY in a fresh throwaway profile under DR_HOME, runs the
+# scenes and writes PNGs and GIFs into docs/manual/media/.
 #
-# The library folder is the author's demo set — seventy face-free frames,
+# LIBRARY is the author's demo set — seventy face-free frames, the
-# the twelve-frame panorama from fixtures/pano among them — and scenes.py's
+# twelve-frame panorama from fixtures/pano among them — and the scenes open
-# cell coordinates assume its grid. Another folder needs those looked at.
+# its photographs by file name. Recording changes it (ratings, collections,
+# the merged DNG), so DR_LIBRARY_SNAPSHOT, if set, is copied over it first.
+#
+# Environment: DR_HOME (/var/tmp/dr-manual), DR_DISPLAY (:7), DR_BIN,
+# DR_LIBRARY_SNAPSHOT, DR_EXPORT_DIR (LIBRARY/../export), DR_INFERENCE
+# (`cpu`, the default, pins inference to the CPU so a recording does not
+# fight a training run for the GPU; `probe` lets the app choose), CARGO
+# (cargo; a wrapper taking cargo's arguments works too).
 set -euo pipefail
 
 here="$(cd "$(dirname "$0")" && pwd)"
 repo="$(cd "$here/../.." && pwd)"
-library="${1:?library folder}"
+scenes="$here/scenes.py"
-shift || true
+
+mode=record
+case "${1:-}" in
+    --check) exec python3 "$scenes" check ;;
+    --list) exec python3 "$scenes" list ;;
+    --changed) mode=changed; shift ;;
+    ""|-h|--help) sed -n '2,/^set -e/p' "$0" | sed '$d; s/^# \{0,1\}//'; exit 0 ;;
+esac
+
+library="$(cd "${1:?library folder}" && pwd)"
+shift
+
+if [ "$mode" = changed ]; then
+    python3 "$scenes" changed | tee /dev/stderr > /dev/null
+    mapfile -t chosen < <(python3 "$scenes" changed | cut -f1)
+    if [ ${#chosen[@]} -eq 0 ]; then
+        echo "nothing to re-record: every picture is newer than what it depends on"
+        exit 0
+    fi
+    set -- "${chosen[@]}"
+fi
 
 export DR_HOME="${DR_HOME:-/var/tmp/dr-manual}"
 export DR_DISPLAY="${DR_DISPLAY:-:7}"
-export DR_BIN="${DR_BIN:-$repo/target/release/darkroom-desktop}"
 export DR_LIBRARY="$library"
+export DR_AUTOMATION="$DR_HOME/automation.sock"
 media="$repo/docs/manual/media"
-mkdir -p "$media" "$DR_HOME/xdg/config/darkroom" "$DR_HOME/xdg/data/darkroom"
 
-[ -x "$DR_BIN" ] || (cd "$repo" && cargo build --release -p darkroom-desktop)
+if [ -z "${DR_BIN:-}" ]; then
+    (cd "$repo" && ${CARGO:-cargo} build --release -p darkroom-desktop --features automation)
+    DR_BIN="$repo/target/release/darkroom-desktop"
+fi
+export DR_BIN
+
+if [ -n "${DR_LIBRARY_SNAPSHOT:-}" ]; then
+    rsync -a --delete "$DR_LIBRARY_SNAPSHOT/" "$library/"
+fi
+
+# A fresh profile every time, so a scene sees the library and not whatever
+# the last run left in the catalog. The inference cache is kept — it is
+# minutes of compiling — and so are the links to the models. The marker is
+# what makes the wipe safe: nothing without it is deleted.
+profile="$DR_HOME/xdg"
+if [ -d "$profile" ] && [ ! -e "$DR_HOME/.manual-profile" ]; then
+    echo "$profile exists and was not made by record.sh; set DR_HOME elsewhere" >&2
+    exit 1
+fi
+mkdir -p "$DR_HOME" && touch "$DR_HOME/.manual-profile"
+rm -rf "$profile/config" "$profile/state"
+if [ -d "$profile/data/darkroom" ]; then
+    find "$profile/data/darkroom" -mindepth 1 -maxdepth 1 \
+        ! -name inference ! -name models ! -name runtime -exec rm -rf {} +
+fi
+mkdir -p "$profile/config/darkroom" "$profile/data/darkroom"
 
 # The models and the runtime are shared with the real profile: a segmenter
 # that is not there makes "Find subjects" a picture of nothing.
 for d in models runtime; do
     src="${XDG_DATA_HOME:-$HOME/.local/share}/darkroom/$d"
-    [ -e "$src" ] && ln -sfn "$src" "$DR_HOME/xdg/data/darkroom/$d"
+    [ -e "$src" ] && ln -sfn "$src" "$profile/data/darkroom/$d"
 done
 
+# Keep the inference choice the app last made, but on the CPU unless asked.
+backend="$profile/data/darkroom/inference/backend.json"
+if [ "${DR_INFERENCE:-cpu}" = cpu ] && [ -e "$backend" ]; then
+    python3 - "$backend" <<'PY'
+import json, sys
+p = sys.argv[1]
+d = json.load(open(p))
+d['rung'], d['reason'] = 'Cpu', 'forced for the demo'
+json.dump(d, open(p, 'w'), indent=2)
+PY
+fi
+
 # A folder library, chosen: no launch screen on the way in.
-cat > "$DR_HOME/xdg/config/darkroom/sessions.json" <<JSON
+cat > "$profile/config/darkroom/sessions.json" <<JSON
 { "version": 0, "sessions": [ { "backend": "folder", "server": "$library",
   "login": "", "user_id": "", "root": "", "root_chosen": true,
   "formats": [], "last_scan": null } ] }
 JSON
+export_dir="${DR_EXPORT_DIR:-$(dirname "$library")/export}"
+cat > "$profile/config/darkroom/settings.json" <<JSON
+{ "export": { "destination": "$export_dir" } }
+JSON
 
+started_x=""
 if ! DISPLAY="$DR_DISPLAY" xdpyinfo >/dev/null 2>&1; then
     Xvfb "$DR_DISPLAY" -screen 0 1920x1200x24 +extension GLX +render -noreset \
         > "$DR_HOME/xvfb.log" 2>&1 &
     echo $! > "$DR_HOME/xvfb.pid"
+    started_x=1
     sleep 2
 fi
+cleanup() {
+    python3 "$here/drive.py" stop || true
+    if [ -n "$started_x" ]; then kill "$(cat "$DR_HOME/xvfb.pid")" || true; fi
+}
+trap cleanup EXIT
 
+stamp="$DR_HOME/.recording-started"
+touch "$stamp"
 python3 "$here/drive.py" launch
-sleep 12
+DR_TOOLS="$here" python3 -c '
-python3 "$here/scenes.py" "$media" "${@:-all}"
+import os, sys, time
-python3 "$here/drive.py" stop
+sys.path.insert(0, os.environ["DR_TOOLS"])
+import drive as dr
+dr.wait_ready()
+dr.wait_for("id:grid-scroll", 120)
+time.sleep(10)   # the first thumbnails, and the scan settling
+'
+python3 "$scenes" record "$media" "${@:-all}"
 
 # GIFs for the page; the MP4s are working files and are not kept.
 for mp4 in "$media"/*.mp4; do
@@ -59,4 +147,4 @@ for mp4 in "$media"/*.mp4; do
     "$here/gif.sh" "$mp4" "${mp4%.mp4}.gif" 960 10
     rm "$mp4"
 done
-optipng -quiet -o2 "$media"/*.png || true
+find "$media" -name '*.png' -newer "$stamp" -exec optipng -quiet -o2 {} + || true

ui/dr-ui/src/automation.rs

@@ -24,8 +24,9 @@
 //!
 //! - `ping` — `"ok"`.
 //! - `locate LABEL` — every element whose accessible label is exactly
-//!   `LABEL`: `[{"label","role","x","y","w","h","opacity","enabled",
+//!   `LABEL`: `[{"label","id","type","role","x","y","w","h","opacity",
-//!   "checked","value"}, …]`, in physical window pixels, tree order.
+//!   "enabled","checked","value","min","max"}, …]`, in physical window
+//!   pixels, tree order.
 //! - `labels [SUBSTRING]` — every labelled element, or those whose label
 //!   contains `SUBSTRING`; the same objects. For finding what a scene can
 //!   name.
@@ -180,5 +181,7 @@ fn describe(e: &ElementHandle, scale: f32) -> Value {
         "enabled": e.accessible_enabled(),
         "checked": e.accessible_checked(),
         "value": e.accessible_value().map(|s| s.to_string()),
+        "min": e.accessible_value_minimum(),
+        "max": e.accessible_value_maximum(),
     })
 }

ui/dr-ui/ui/merge.slint

@@ -176,7 +176,7 @@ export component MergePage inherits Rectangle {
                     if root.preview.width > 0: Rectangle {
                         height: min(root.height * 0.55, parent.width * root.preview.height / max(1, root.preview.width));
                         background: Theme.rule;
-                        Image {
+                        preview-image := Image {
                             source: root.preview;
                             width: 100%;
                             height: 100%;