Found by CI itself, on run 1437 -- the first run that ever got far enough
to try. Everything ahead of it worked: checkout brought both submodules,
v2 pulled and passed its own version assert, and the fixtures resolved at
the pinned version. Then:
scripts/artifacts/pull_artifacts.sh: line 54: unzip: command not found
The replay fixtures ship as a zip in the generic package registry and
pull_artifacts.sh unpacks them with unzip. Without it the step exits 127
and the T2 tier has no input -- reported as a missing fixture by the
verify step below, which is a missing tool wearing the wrong label.
Worth naming why the local rehearsal missed this and CI did not: the
rehearsal had the dumps staged in the working tree by hand, so it built
and ran the tier without ever executing the script that fetches them. The
image was checked against the build, not against the job.
Tag v2 -> v3, in all three places, in one commit.
TRACES: DP-007 | PR-004
v1 has no libfftw3. SAE_SCENE_XGB is on by default and pulls the learned
scene-boundary detector into result_sink, whose find_library(FFTW3_LIB
fftw3 REQUIRED) at CMakeLists.txt:377 runs at CONFIGURE time -- so cmake
fails before any target is chosen, and building only `sae_tests` does not
get you past it. The image predates the detector; nothing rebuilt it when
the detector landed, which is precisely the gap DP-007's own register row
records.
-DSAE_SCENE_XGB=OFF would also have made the configure pass. It is the
wrong fix and worth saying why: CI is the only place these tests run, so
a flag that silences a subsystem there silences it everywhere, and the
job would have gone green over a detector nobody was compiling.
The image's self-check gains a line, because an image that ships a
library CMake cannot find the way CMake looks for it is the failure this
block exists to catch. FFTW is checked as a bare .so rather than through
pkg-config, since find_library is what CMakeLists.txt actually calls --
checking fftw3.pc would pass on an image whose libfftw3.so was absent.
Tag bumped v1 -> v2 here, in build_builder_image.sh and in the workflow's
run-time assert, in one commit, as this image's contract requires. v2 is
built and published. Rehearsed in the image before pinning it: configure,
build and 151 tests now run.
TRACES: DP-007 | PR-004
DP-007's row says the builder image is "pinned by tag in the Gitea
container registry". The tag was pinned; the image was never pushed.
Every run since the workflow landed died at
"Error response from daemon: manifest unknown" before a single step ran,
so the tier this workflow exists to execute has still never executed.
sae-builder-cpu:v1 is now in the registry, built from ff3b8eb by
scripts/ci/build_builder_image.sh and reporting SAE_BUILDER_VERSION=v1,
which is what the job's own assert-the-image step demands. The
v1-ff3b8eb audit tag went with it.
Behind that, the same absence one layer down: no replay-fixtures package
existed either, so "Verify the fixtures actually arrived" would have
failed next. tests/fixtures/dumps is now published at version ff3b8eb.
Which makes the `latest` here worth removing rather than keeping. Two
reasons, either sufficient. It is the same argument DP-007 already
makes about the image tag -- a dump is an input to the tests, so a
re-upload under a moving `latest` retroactively changes what an earlier
green build proved. And `latest` was the only thing in this job that
wanted a credential: package downloads are anonymous while the repo is
public, and only resolving `latest` needs a token for the list endpoint.
No GITEA_TOKEN secret is configured on this repo, so that step could
never have resolved `latest` even once the image existed. Pinning
deletes the dependency instead of documenting it.
The failure message below it said the step "needs GITEA_TOKEN to resolve
'latest'"; it now says to check the pinned version still exists, which
is the thing that can actually go wrong -- pull_artifacts.sh warns and
continues on a missing version rather than failing, which is why that
verify step is there at all.
TRACES: DP-007 | PR-004
The traceability job has been failing since it was written, and never on
anything it checks. It declared no `container:`, so it ran in the
runner's default image, which has no node; act_runner executes JS
actions with the node it finds inside the job container, and both
actions/checkout and upload-artifact are JS. The job died at
"Cannot find: node in PATH" before the repository was checked out, and
every later step then failed on an empty working tree -- the summary
step's "head: cannot open 'docs/traceability.md'" is that, not a missing
report.
node:20-bookworm, and the choice is not arbitrary: node is the part with
no workaround, while Debian 12 already carries python3.11 (tomllib, so
the stdlib-only extractor reads traceability.toml) and git (for the
pull_request diff step). Nothing in it names this repo, so the file
stays copyable into the other two components unedited, which its header
claims and this commit keeps true.
The kpnpp-builder and jellytau-builder images bake node in for exactly
this reason, and Dockerfile.builder-cpu says so in a comment. That
knowledge just had not reached the one job with no image of its own.
Second failure behind the first: the jray-project submodule was pinned
by SSH URL. The runner has no key, so `submodules: recursive` could not
have fetched the extractor even with node present. https, like the KPN
submodule beside it.
Gate and static check both pass locally on this tree -- 42/72 traced,
73.7% in CI scope, 0 orphans, 0 bare-cosine violations -- so what CI
reports next is a fact about CI, not about the tree.
AR-024's register row gives its verification tier as "Static check -- no
bare cosine outside a tagged EXCEPTION". No such check existed, so the
invariant was enforced by reading, and reading had missed a live
violation.
scripts/ci/check_raw_cosine.py is that check, wired into the
traceability workflow as a blocking step. It is honest about its reach:
it catches direct cosine_similarity() uses not routed through a
calibration, and it cannot follow a cosine through a variable across
statements. That limit is documented in the script rather than left for
someone to discover after trusting a pass.
What it caught, and what this commit removes with it:
The identity matcher's no-calibration fallback thresholded raw cosine
distance (match_threshold) plus a ratio test (match_ratio,
match_ratio_ceil). Worse than the invariant breach: it fed
max(0, cosine) into TrackRegistry::observe, whose contract reads
"posterior is a calibrated probability, never a raw cosine (AR-024) ...
so the accumulation cannot be fed an uncalibrated number by a careless
caller". It could, and did. And it disagreed with the rest of the
pipeline about what "the fit failed" means -- same_person_probability
answers that with the untuned default sigmoid and a loud warning, so
association stayed in probability space while matching alone left it.
One run, two policies, no announcement.
Now one rule: cal_.probability() always, with a warning when the fit is
not real. A worse answer than a fitted calibration, a better one than a
number whose units nothing else shares.
TrackGallery::set_calibration is mandatory for the same reason. Its
default was max(0, cosine), which made expand_band_lo = 0.90 mean
"cosine > 0.9" in a test and "P(same person) > 0.9" in production.
FaceTrackerFunc already threw without one; the expansion store now
matches.
One exception is recorded, in the calibration's own dedup. It is not a
close call: at 1 - 1e-7 it asks whether two vectors are the same vector,
and it runs on the fit's input, so a calibrated comparison there would
have to be calibrated by the fit it is feeding.
Also drops seven dead keys from the optimizer's CFG_KEYS. Config keys
are read with a contains() check, so each one had been silently inert
since the field behind it was deleted -- a sweep varying one of them
measured nothing and reported an ordinary-looking F1.
TRACES: AR-024, AR-023 | SR-002
docs/requirements.md describes four verification tiers and argues that
T1 (functor unit) and T2 (replay) are "the only tiers that can exist in
CI at all". The traceability gate then reports a CI-scope coverage
fraction over exactly those tiers. Nothing ran them: the only workflow
was the gate itself, which reads source comments, and SAE_BUILD_TESTS
defaults to OFF. "Covered" meant a TRACES tag existed in a file.
That is the same failure the gate's own config warns about one level up
-- counting a test that cannot run -- and the gate cannot see it,
because a tag is all a static reader has.
Runs in the pinned DP-007 CPU builder image, which the image script
already expected this workflow to exist (it names unit-tests.yml and
asserts its tag). Nothing here calls a model: T1 constructs node
functors directly and T2 replays a precomputed dump, so the GPU-free
N100 runner is sufficient by construction rather than by concession.
Two deliberate hard failures. A missing replay fixture fails the job
instead of skipping, because pull_artifacts.sh warns-and-continues and a
T2 test whose input never arrived must not look like a pass. And the
image reporting a tag other than the pinned one fails rather than
building against an unknown toolchain.
It found a real bug on its first run: see the preceding commit. ctest
runs each case in its own process, which turned a 1-in-4 heap corruption
from noise in the aggregate binary into a reproducible failure.
TRACES: DP-007 | PR-004
jray-project is added at scripts/vendor/jray-project and the extractor is used
from there. Only two files are repo-local: traceability.toml, which carries
everything repo-specific, and the CI workflow that invokes the vendored gate.
The extractor is deliberately NOT copied in. One implementation, parameterised
by config — a second copy would drift from the first, and the tool already
proves it works unchanged against all three registers.
Enables system_spec so PR/SR orphan checking runs: previously uncheckable,
because the system spec lived outside every component's checkout.
Gate is green at 0.0% of 63 requirements, which is correct — nothing is tagged
yet. Eleven are flagged unverifiable on this CI host and excluded from the
numerator rather than counted.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
dtourolle
