From fd078c399fa31c4c2c1d9bf39888ed403cf3fe19 Mon Sep 17 00:00:00 2001 From: Duncan Tourolle Date: Wed, 5 Aug 2026 15:17:17 +0200 Subject: [PATCH] ci(tests): actually run the tier the verification strategy is built on docs/requirements.md describes four verification tiers and argues that T1 (functor unit) and T2 (replay) are "the only tiers that can exist in CI at all". The traceability gate then reports a CI-scope coverage fraction over exactly those tiers. Nothing ran them: the only workflow was the gate itself, which reads source comments, and SAE_BUILD_TESTS defaults to OFF. "Covered" meant a TRACES tag existed in a file. That is the same failure the gate's own config warns about one level up -- counting a test that cannot run -- and the gate cannot see it, because a tag is all a static reader has. Runs in the pinned DP-007 CPU builder image, which the image script already expected this workflow to exist (it names unit-tests.yml and asserts its tag). Nothing here calls a model: T1 constructs node functors directly and T2 replays a precomputed dump, so the GPU-free N100 runner is sufficient by construction rather than by concession. Two deliberate hard failures. A missing replay fixture fails the job instead of skipping, because pull_artifacts.sh warns-and-continues and a T2 test whose input never arrived must not look like a pass. And the image reporting a tag other than the pinned one fails rather than building against an unknown toolchain. It found a real bug on its first run: see the preceding commit. ctest runs each case in its own process, which turned a 1-in-4 heap corruption from noise in the aggregate binary into a reproducible failure. TRACES: DP-007 | PR-004 --- .gitea/workflows/unit-tests.yml | 140 ++++++++++++++++++++++++++++++++ 1 file changed, 140 insertions(+) create mode 100644 .gitea/workflows/unit-tests.yml diff --git a/.gitea/workflows/unit-tests.yml b/.gitea/workflows/unit-tests.yml new file mode 100644 index 0000000..665dfac --- /dev/null +++ b/.gitea/workflows/unit-tests.yml @@ -0,0 +1,140 @@ +name: Unit tests + +# TRACES: DP-007 | PR-004 +# +# The tier the verification strategy is built on, finally executing. +# +# docs/requirements.md describes a four-tier plan in which T1 (functor unit) +# and T2 (replay) are "the only tiers that can exist in CI at all", and the +# traceability gate reports a CI-scope coverage fraction over exactly those +# tiers. Until this workflow existed, nothing ran them: "covered" meant a +# TRACES tag was present in a file, not that any test had been executed. That +# is the same failure mode as counting a test that cannot run, one level up, +# and the gate cannot detect it because a tag is all it can see. +# +# The runner is an Intel N100 with no discrete GPU. Nothing here calls a model: +# T1 constructs node functors directly, and T2 replays a precomputed HDF5 dump. +# T3 (ORT CPU smoke) and T4 (GPU) are deliberately absent -- the embedder is +# ~930 ms/frame on this hardware, so a 77 s clip at 5 fps would be six minutes +# of inference alone. + +on: + push: + branches: + - main + - master + - develop + pull_request: + branches: + - main + - master + - develop + +jobs: + unit-tests: + runs-on: linux/amd64 + name: Build and run the GPU-free suite + + # Pinned by tag, never `latest`, so rebuilding the image cannot silently + # change what a previous green build meant. Bumping the dependency set means + # bumping the tag in scripts/ci/build_builder_image.sh AND here, in one + # commit -- see that script's header. + container: + image: gitea.tourolle.paris/dtourolle/sae-builder-cpu:v1 + + steps: + - name: Checkout repository + uses: actions/checkout@v4 + with: + fetch-depth: 0 + # KPN is a submodule and the pipeline does not build without it. + # + # NOTE: this checks out the commit this repo PINS, which is the whole + # point and is also the first thing this job will disagree with a + # developer about. A local KPN working copy that is ahead of + # origin/master builds and passes here while CI builds something else + # entirely; the AR-004 evidence in docs/requirements.md was gathered + # that way. If this job fails on tests that pass locally, check + # `git -C external/KPN log origin/master..HEAD` before suspecting the + # tests. + # LFS is deliberately NOT fetched: SAE_MODELS_DIR is baked into the + # binary as a path string and nothing in T1/T2 opens a model file, so + # pulling ~hundreds of MB of ONNX would cost the job everything and + # buy it nothing. + submodules: recursive + lfs: false + + - name: Assert the builder image is the pinned one + run: | + set -e + echo "builder=$SAE_BUILDER version=$SAE_BUILDER_VERSION" + echo "ort=$SAE_ORT_VERSION opencv=$SAE_OPENCV_VERSION" + # The image reports its own tag. A mismatch means the `container:` + # line above and the image that actually landed disagree, which is + # exactly the drift the pinning exists to prevent -- so it fails the + # job rather than building against an unknown toolchain. + [ "$SAE_BUILDER_VERSION" = "v1" ] || { + echo "image reports version '$SAE_BUILDER_VERSION', workflow pins v1" >&2 + exit 1 + } + + - name: Fetch replay fixtures + env: + GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }} + # bash, not sh: the script declares #!/bin/bash and uses `set -o + # pipefail` and arrays, which dash does not have. + run: bash scripts/artifacts/pull_artifacts.sh replay-fixtures latest + + # pull_artifacts.sh warns and continues when a package version is missing, + # which is right for a developer pulling one artifact of several and wrong + # here. A T2 test whose fixture never arrived must not look like a pass: + # the dumps are the entire input to the replay tier, and VR-002's claim is + # that replay drives the real nodes over real data. + - name: Verify the fixtures actually arrived + run: | + set -e + missing=0 + for f in tests/fixtures/dumps/superhero.h5; do + if [ -s "$f" ]; then + echo " ok: $f ($(wc -c < "$f") bytes)" + else + echo " MISSING: $f" >&2 + missing=1 + fi + done + if [ "$missing" -ne 0 ]; then + echo "" >&2 + echo "Replay fixtures are absent, so the T2 tier cannot run." >&2 + echo "They are not in git (tests/fixtures/dumps/.gitignore) -- they" >&2 + echo "live in the Gitea generic package registry and are pulled by" >&2 + echo "the step above, which needs GITEA_TOKEN to resolve 'latest'." >&2 + exit 1 + fi + + - name: Configure + run: | + set -e + # SAE_GEMM_BACKEND defaults to ROCM and the auto-detect prefers a GPU + # backend where it finds one; CPU is stated explicitly so this job + # cannot start depending on what happens to be installed on the runner. + # The CPU kernel is OpenBLAS in this image (tests/CMakeLists.txt fails + # the configure if it is not), so the suite exercises the kernel the + # CPU release actually ships. + cmake -S . -B build \ + -DCMAKE_BUILD_TYPE=Release \ + -DSAE_BUILD_TESTS=ON \ + -DSAE_GEMM_BACKEND=CPU + + - name: Build the test suite + run: cmake --build build --target sae_tests --parallel + + - name: Run the tests + run: ctest --test-dir build --output-on-failure + + - name: Save test output + if: always() + uses: actions/upload-artifact@v3 + with: + name: unit-test-results + path: build/Testing/ + retention-days: 30