perf: make the benchmark able to answer the question, then ask it

PERF_PLAN phase 0, plus B1/B2 which turned out to cost seconds rather
than the minutes budgeted for them. No library code is touched.

The harness could not support the conclusions drawn from it. items_for()
shrank the sample as work per item grew, so exactly the rows under
investigation -- chain-16 and chain-32 -- ran 50 to 200 items and swung
4-8x between passes. Sample size now derives from a time budget with a
floor, using work_us * stages / units as the per-item cost. The old
ladder's error was treating depth as a throughput cost: past the core
count it is, below it depth costs only latency.

Rows now report median of N repetitions after a discarded warm-up, with
IQR and range, so an unreliable row says so instead of being averaged
into a table. The CSV header records nproc, governor and AC state, which
immediately caught this laptop running on battery under powersave.

A3 needed no experiment in the end: ru_nivcsw and ru_nvcsw are captured
around every timed region and reported per item, so involuntary switches
against depth is a column rather than a run.

bench_dispatch answers B1 and B2 without instrumenting the scheduler.
Sleeping is inferred from ru_nvcsw, since a thread blocking on a
condition variable books a voluntary context switch. B1: a ThreadPool(1)
dispatch is 291 ns null, 466 ns with a payload, against the ~290 ns the
plan estimated -- so the abandon criterion is not met and workstream B
stays alive.

B2's answer is not the one the question expected. It is not whether
workers sleep but which pool: on a private ThreadPool(1) the worker never
sleeps, because it resubmits into its own queue and finds the work
already there; on any pool of two or more it sleeps exactly once per
task, because submit() round-robins to a different worker, which is
asleep. That is the whole 466 ns to 1.7 us difference, and it inverts
half the plan. B5 (bounded spin) buys nothing in the default
configuration, and A5 must not make a shared pool the default until the
wake cost is fixed, or every graph that already fits its cores gets 3-4x
worse per dispatch.

G1 lands as tests/soak_wedge.cpp, superseding benchmarks/repro_wedge.cpp,
which was never wired into any build. Always compiled so it cannot rot;
its CTest cases register only under -DKPN_ENABLE_SOAK_TESTS=ON, so the
default test count is unchanged. A wedge is a hang, and a hang under
CTest is an unattributable timeout, so it carries a watchdog that aborts
naming the iteration and phase.

Phase 0's gate is not yet cleared: the acceptance run belongs on the
reference machine, not here. A 3-pass check lands every row within 0.7%
against the 4-8x swings described above, which is encouraging and is not
the same thing.

Provisional, recorded so it can be checked: chain-16 came out 6% behind
TBB rather than 28.5%. If that survives a proper run, the deep-chain
deficit is substantially an artefact of the N=200 rows.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

scripts/bench_repro_check.py

@@ -0,0 +1,122 @@
+#!/usr/bin/env python3
+"""Check the PERF_PLAN Phase-0 acceptance criterion.
+
+Runs bench_pipeline several times and reports, per row, how far the passes
+spread around their median. The plan's gate is: the same configuration run 7x
+lands within +/-5% on every row. Until that holds, no measured difference
+between KPN and TBB is worth acting on.
+
+Exits non-zero if any row exceeds the tolerance, so it can gate a session of
+performance work rather than merely inform one.
+
+Usage:
+    scripts/bench_repro_check.py ./build_bench/benchmarks/bench_pipeline \\
+        --passes 7 --tolerance 5 -- --work=10 --topos=chain,wide --reps=5
+"""
+
+import argparse
+import statistics
+import subprocess
+import sys
+
+KEY_COLS = ("topology", "size", "work_us", "threads")
+
+# bench_pipeline reports throughput, bench_dispatch reports per-dispatch cost.
+# Either is a valid thing to demand reproducibility of; deviation from the
+# median is symmetric, so it does not matter which direction is "better".
+METRIC_COLS = ("items_per_sec", "ns_per_dispatch")
+
+
+def parse_csv(text, metric=None):
+    """Return ({(topology, size, work_us, threads): value}, metric_name)."""
+    rows = {}
+    header = None
+    for line in text.splitlines():
+        line = line.strip()
+        if not line or line.startswith("#"):
+            continue
+        fields = line.split(",")
+        if header is None:
+            if fields[0] != "topology":
+                continue
+            header = fields
+            if metric is None:
+                for cand in METRIC_COLS:
+                    if cand in header:
+                        metric = cand
+                        break
+                else:
+                    sys.exit(f"no metric column found in header: {header}")
+            elif metric not in header:
+                sys.exit(f"metric {metric!r} not in header: {header}")
+            continue
+        rec = dict(zip(header, fields))
+        try:
+            key = tuple(rec[c] for c in KEY_COLS)
+            rows[key] = float(rec[metric])
+        except (KeyError, ValueError):
+            continue
+    return rows, metric
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("binary", help="path to bench_pipeline")
+    ap.add_argument("--passes", type=int, default=7)
+    ap.add_argument("--tolerance", type=float, default=5.0,
+                    help="max allowed deviation from the median, percent")
+    ap.add_argument("--metric", default=None, choices=METRIC_COLS,
+                    help="column to check (default: whichever the CSV carries)")
+
+    # Everything after a standalone `--` goes to bench_pipeline verbatim.
+    # argparse.REMAINDER would swallow this script's own flags instead.
+    argv = sys.argv[1:]
+    extra = []
+    if "--" in argv:
+        cut = argv.index("--")
+        argv, extra = argv[:cut], argv[cut + 1:]
+    args = ap.parse_args(argv)
+
+    passes = []
+    metric = args.metric
+    for i in range(args.passes):
+        print(f"pass {i + 1}/{args.passes} ...", file=sys.stderr, flush=True)
+        proc = subprocess.run([args.binary] + extra,
+                              capture_output=True, text=True)
+        if proc.returncode != 0:
+            print(proc.stderr, file=sys.stderr)
+            sys.exit(f"{args.binary} failed with {proc.returncode}")
+        rows, metric = parse_csv(proc.stdout, metric)
+        passes.append(rows)
+
+    keys = set(passes[0])
+    for p in passes[1:]:
+        keys &= set(p)
+    if not keys:
+        sys.exit("no rows common to every pass")
+
+    print(f"\n{'row':<34} {'median ' + metric:>20} {'worst dev':>10}  verdict")
+    print("-" * 72)
+
+    failures = 0
+    for key in sorted(keys):
+        values = [p[key] for p in passes]
+        med = statistics.median(values)
+        worst = max(abs(v - med) / med * 100 for v in values) if med else 0.0
+        ok = worst <= args.tolerance
+        failures += not ok
+        label = "{}-{} w={} s={}".format(*key)
+        print(f"{label:<34} {med:>20.1f} {worst:>9.1f}%  {'ok' if ok else 'NOISY'}")
+
+    print("-" * 72)
+    if failures:
+        print(f"{failures}/{len(keys)} rows exceed +/-{args.tolerance:g}% — "
+              f"the Phase-0 gate is not met.")
+        return 1
+    print(f"all {len(keys)} rows within +/-{args.tolerance:g}% "
+          f"over {args.passes} passes — Phase-0 gate met.")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())