build: requirement traceability extractor, gate, and CI workflow

Ports JellyTau's traceability tooling, rewritten in stdlib Python because
this repo is C++/Python and adding a bun/node toolchain to check source
comments would be a worse trade than writing the scanner.

scripts/traceability/extract_traces.py scans .cpp/.hpp/.py under src, tests,
scripts, experiments and eval for the house tag format

    /// TRACES: AR-012, AR-013 | SR-002

and reports EXCEPTION tags separately. An exception is a recorded decision to
depart from an invariant, so folding it into coverage would invert its
meaning; it is listed with its reason, and a missing reason is flagged.

Two rules carried over from JellyTau's gate repair:

  * Denominators are parsed out of docs/requirements.md at run time. A
    requirement is defined only by a row in a table whose header is
    `| ID | Requirement | ... |`, so references in the Traces to column, in
    prose, and in the verification-plan table do not inflate the count.
    Adding a register row lowers coverage until it is traced - the property
    that dies the moment a denominator is frozen.
  * Coverage above 100% is a hard failure. It cannot happen through the
    intersection, which is the point: if it ever does, the arithmetic is
    broken and the run must not be reported as a pass.

One rule specific to this repo: CI is an Intel N100 with no discrete GPU. The
extractor reads each requirement's verification tier from requirements.md and
reports T4/GPU-only requirements as tagged but unexecuted, never as covered.
Counting a test that can never run is the same failure mode as the 158% bug.

MIN_COVERAGE starts at 0 because almost nothing is tagged yet - tags are added
as the pipeline is built. That is not a gate that cannot fail: orphan tags, a
>100% ratio, a register that parses to nothing, and an empty source scan are
all hard failures from day one. The threshold lives in traceability-gate.sh
alone, never duplicated into the workflow YAML.

53 tests over fixture strings, so their meaning does not drift as requirements
are added.
This commit is contained in:
2026-07-30 18:16:58 +02:00
parent 43d2c976c3
commit 054c4c8b8f
5 changed files with 2257 additions and 0 deletions
+64
View File
@@ -0,0 +1,64 @@
#!/bin/sh
#
# Requirement traceability gate. Run locally exactly as CI runs it:
#
# scripts/traceability/traceability-gate.sh
#
# Writes traces-report.json and docs/traceability.md, prints the coverage
# report, and exits non-zero when the gate fails.
#
# Environment:
# MIN_COVERAGE minimum overall coverage percent (default 0 - see below)
# ALLOW_ORPHANS set to 1 to report orphan tags without failing
# TRACES_JSON JSON report path (default traces-report.json)
# TRACES_MD markdown matrix path (default docs/traceability.md)
# SYSTEM_SPEC optional path to the umbrella SPEC.md, which defines the
# PR/SR IDs; when given, PR/SR orphans are reported too. That
# file lives in the parent project, not in this repo, so CI
# normally leaves it unset.
#
# Threshold policy lives here and nowhere else. It is deliberately NOT
# duplicated into the workflow YAML: a threshold written in two places is a
# threshold that will disagree with itself.
#
# MIN_COVERAGE defaults to 0 because almost nothing is tagged yet - tags are
# added as the pipeline is built, so a low number today is accurate rather than
# alarming. A zero threshold does NOT mean the gate cannot fail: orphan tags,
# a >100% ratio, a register that parses to nothing, and an empty source scan
# are all hard failures from day one. Raise MIN_COVERAGE as tags land; treat
# every raise as a ratchet, never a reset.
#
# POSIX sh, no bashisms, no jq - the extractor does its own arithmetic and
# printing so CI needs nothing beyond python3.
set -eu
SCRIPT_DIR=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
REPO_ROOT=$(CDPATH= cd -- "$SCRIPT_DIR/../.." && pwd)
MIN_COVERAGE="${MIN_COVERAGE:-0}"
TRACES_JSON="${TRACES_JSON:-$REPO_ROOT/traces-report.json}"
TRACES_MD="${TRACES_MD:-$REPO_ROOT/docs/traceability.md}"
PYTHON="${PYTHON:-python3}"
command -v "$PYTHON" >/dev/null 2>&1 || {
echo "FAILED: $PYTHON not found. The traceability gate needs Python 3.9+" >&2
exit 2
}
set -- \
--root "$REPO_ROOT" \
--format coverage \
--json-out "$TRACES_JSON" \
--markdown-out "$TRACES_MD" \
--min-coverage "$MIN_COVERAGE"
if [ "${ALLOW_ORPHANS:-0}" = "1" ]; then
set -- "$@" --allow-orphans
fi
if [ -n "${SYSTEM_SPEC:-}" ]; then
set -- "$@" --system-spec "$SYSTEM_SPEC"
fi
exec "$PYTHON" "$SCRIPT_DIR/extract_traces.py" "$@"