Files
dtourolle fd078c399f ci(tests): actually run the tier the verification strategy is built on
docs/requirements.md describes four verification tiers and argues that
T1 (functor unit) and T2 (replay) are "the only tiers that can exist in
CI at all". The traceability gate then reports a CI-scope coverage
fraction over exactly those tiers. Nothing ran them: the only workflow
was the gate itself, which reads source comments, and SAE_BUILD_TESTS
defaults to OFF. "Covered" meant a TRACES tag existed in a file.

That is the same failure the gate's own config warns about one level up
-- counting a test that cannot run -- and the gate cannot see it,
because a tag is all a static reader has.

Runs in the pinned DP-007 CPU builder image, which the image script
already expected this workflow to exist (it names unit-tests.yml and
asserts its tag). Nothing here calls a model: T1 constructs node
functors directly and T2 replays a precomputed dump, so the GPU-free
N100 runner is sufficient by construction rather than by concession.

Two deliberate hard failures. A missing replay fixture fails the job
instead of skipping, because pull_artifacts.sh warns-and-continues and a
T2 test whose input never arrived must not look like a pass. And the
image reporting a tag other than the pinned one fails rather than
building against an unknown toolchain.

It found a real bug on its first run: see the preceding commit. ctest
runs each case in its own process, which turned a 1-in-4 heap corruption
from noise in the aggregate binary into a reproducible failure.

TRACES: DP-007 | PR-004
2026-08-05 15:17:17 +02:00

141 lines
5.7 KiB
YAML

name: Unit tests
# TRACES: DP-007 | PR-004
#
# The tier the verification strategy is built on, finally executing.
#
# docs/requirements.md describes a four-tier plan in which T1 (functor unit)
# and T2 (replay) are "the only tiers that can exist in CI at all", and the
# traceability gate reports a CI-scope coverage fraction over exactly those
# tiers. Until this workflow existed, nothing ran them: "covered" meant a
# TRACES tag was present in a file, not that any test had been executed. That
# is the same failure mode as counting a test that cannot run, one level up,
# and the gate cannot detect it because a tag is all it can see.
#
# The runner is an Intel N100 with no discrete GPU. Nothing here calls a model:
# T1 constructs node functors directly, and T2 replays a precomputed HDF5 dump.
# T3 (ORT CPU smoke) and T4 (GPU) are deliberately absent -- the embedder is
# ~930 ms/frame on this hardware, so a 77 s clip at 5 fps would be six minutes
# of inference alone.
on:
push:
branches:
- main
- master
- develop
pull_request:
branches:
- main
- master
- develop
jobs:
unit-tests:
runs-on: linux/amd64
name: Build and run the GPU-free suite
# Pinned by tag, never `latest`, so rebuilding the image cannot silently
# change what a previous green build meant. Bumping the dependency set means
# bumping the tag in scripts/ci/build_builder_image.sh AND here, in one
# commit -- see that script's header.
container:
image: gitea.tourolle.paris/dtourolle/sae-builder-cpu:v1
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
# KPN is a submodule and the pipeline does not build without it.
#
# NOTE: this checks out the commit this repo PINS, which is the whole
# point and is also the first thing this job will disagree with a
# developer about. A local KPN working copy that is ahead of
# origin/master builds and passes here while CI builds something else
# entirely; the AR-004 evidence in docs/requirements.md was gathered
# that way. If this job fails on tests that pass locally, check
# `git -C external/KPN log origin/master..HEAD` before suspecting the
# tests.
# LFS is deliberately NOT fetched: SAE_MODELS_DIR is baked into the
# binary as a path string and nothing in T1/T2 opens a model file, so
# pulling ~hundreds of MB of ONNX would cost the job everything and
# buy it nothing.
submodules: recursive
lfs: false
- name: Assert the builder image is the pinned one
run: |
set -e
echo "builder=$SAE_BUILDER version=$SAE_BUILDER_VERSION"
echo "ort=$SAE_ORT_VERSION opencv=$SAE_OPENCV_VERSION"
# The image reports its own tag. A mismatch means the `container:`
# line above and the image that actually landed disagree, which is
# exactly the drift the pinning exists to prevent -- so it fails the
# job rather than building against an unknown toolchain.
[ "$SAE_BUILDER_VERSION" = "v1" ] || {
echo "image reports version '$SAE_BUILDER_VERSION', workflow pins v1" >&2
exit 1
}
- name: Fetch replay fixtures
env:
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
# bash, not sh: the script declares #!/bin/bash and uses `set -o
# pipefail` and arrays, which dash does not have.
run: bash scripts/artifacts/pull_artifacts.sh replay-fixtures latest
# pull_artifacts.sh warns and continues when a package version is missing,
# which is right for a developer pulling one artifact of several and wrong
# here. A T2 test whose fixture never arrived must not look like a pass:
# the dumps are the entire input to the replay tier, and VR-002's claim is
# that replay drives the real nodes over real data.
- name: Verify the fixtures actually arrived
run: |
set -e
missing=0
for f in tests/fixtures/dumps/superhero.h5; do
if [ -s "$f" ]; then
echo " ok: $f ($(wc -c < "$f") bytes)"
else
echo " MISSING: $f" >&2
missing=1
fi
done
if [ "$missing" -ne 0 ]; then
echo "" >&2
echo "Replay fixtures are absent, so the T2 tier cannot run." >&2
echo "They are not in git (tests/fixtures/dumps/.gitignore) -- they" >&2
echo "live in the Gitea generic package registry and are pulled by" >&2
echo "the step above, which needs GITEA_TOKEN to resolve 'latest'." >&2
exit 1
fi
- name: Configure
run: |
set -e
# SAE_GEMM_BACKEND defaults to ROCM and the auto-detect prefers a GPU
# backend where it finds one; CPU is stated explicitly so this job
# cannot start depending on what happens to be installed on the runner.
# The CPU kernel is OpenBLAS in this image (tests/CMakeLists.txt fails
# the configure if it is not), so the suite exercises the kernel the
# CPU release actually ships.
cmake -S . -B build \
-DCMAKE_BUILD_TYPE=Release \
-DSAE_BUILD_TESTS=ON \
-DSAE_GEMM_BACKEND=CPU
- name: Build the test suite
run: cmake --build build --target sae_tests --parallel
- name: Run the tests
run: ctest --test-dir build --output-on-failure
- name: Save test output
if: always()
uses: actions/upload-artifact@v3
with:
name: unit-test-results
path: build/Testing/
retention-days: 30