perf(bench): make a multi-hour acceptance run observable and restartable

The Phase-0 gate is 7 passes over the full row set, which is long enough
that capture_output=True was the wrong default: no rows existed anywhere
until a pass ended, so a slow run and a wedged one looked identical, and
killing either threw away everything measured.

Rows now stream to --out-dir/pass-NN.csv with a flush per line, so an
interrupted run keeps what it had. --resume reuses complete pass files
and reruns incomplete ones -- checked against the row count rather than
merely existing, because a partial file silently averaged in as a whole
pass would corrupt the verdict rather than fail it.

Progress is a tqdm bar over rows, not passes; a pass counter would sit
at 1/7 for twenty minutes and report nothing useful. The row total comes
from probing the binary with --reps=0 (0.8s) rather than reimplementing
the sweep in Python, which would drift from the C++ defaults. There is a
plain-stderr fallback when tqdm is absent -- refusing to start a
benchmark over a missing progress dependency is the wrong trade.

bench_runs/ is gitignored: the new default output path would otherwise
land in git status.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-08 22:25:00 +02:00
co-authored by Claude Opus 5
parent a3f61fcb3c
commit 73828bcffe
2 changed files with 172 additions and 33 deletions
+3
View File
@@ -25,6 +25,9 @@ venv/
.DS_Store
Thumbs.db
# Benchmark output (scripts/bench_repro_check.py --out-dir)
bench_runs/
# Claude Code local settings
.claude/settings.local.json
include/kpn/ort_cache/