6 Commits
Author SHA1 Message Date
dtourolle 9bcc765408 fix(ci): the builder image has no unzip, so the replay tier had no input
Traceability Validation / Check requirement traces (pull_request) Successful in 19s
Unit tests / Build and run the GPU-free suite (pull_request) Successful in 3m25s
Found by CI itself, on run 1437 -- the first run that ever got far enough
to try. Everything ahead of it worked: checkout brought both submodules,
v2 pulled and passed its own version assert, and the fixtures resolved at
the pinned version. Then:

    scripts/artifacts/pull_artifacts.sh: line 54: unzip: command not found

The replay fixtures ship as a zip in the generic package registry and
pull_artifacts.sh unpacks them with unzip. Without it the step exits 127
and the T2 tier has no input -- reported as a missing fixture by the
verify step below, which is a missing tool wearing the wrong label.

Worth naming why the local rehearsal missed this and CI did not: the
rehearsal had the dumps staged in the working tree by hand, so it built
and ran the tier without ever executing the script that fetches them. The
image was checked against the build, not against the job.

Tag v2 -> v3, in all three places, in one commit.

TRACES: DP-007 | PR-004
2026-08-31 08:04:43 +02:00
dtourolle 3966e19a83 test: two tests still describe the registry as it was before 1477c53
Both failed the first time CI was able to run this suite, and neither is
a new break. They assert semantics that two deliberate changes replaced,
and nothing noticed because nothing had ever executed them.

1477c53 (2026-08-09) made a dormant track associable only if it had been
identified: an unowned dormant track has no actor to re-attach to, so it
only enlarges the matcher's comparison set and invites a new face landing
on an anonymous stub. Its message records the trade -- "a small recall
cost (~1-3pp on some films, e.g. Lord of War -2.5) for a cleaner, bounded
pool. Kept deliberately." Both tests build tracks that are never
identified, so since that commit they are dropped from the pool one line
before the extinction horizon they were written to exercise. The tests
date from 2026-07-31 and 2026-08-08 -- they are older than the gate.
Fixed by owning the track while it is still on screen, which is what a
real run does and what makes the horizon reachable at all.

The second test needed more than that, and the rename says why. It was
built on association and reaping answering on DIFFERENT clocks -- the
tracker's for "may this associate?", the watermark for "is this
finished?" -- which is exactly the arrangement track_registry.hpp:184-203
now rules out: one clock, one threshold, "nothing is offered past its
reap horizon, nothing is reaped while still offerable". So there is no
longer an interval in which a track is retired from association but still
alive, and a test named for that interval cannot pass. Rewritten to
assert what the unified clock actually promises: a tracker racing 50 s
past the horizon retires nothing while the matcher lags, the votes in
flight still land, and the watermark passing the horizon retires, reaps
and emits in one step.

Kept: the AR-008 tag, the late-vote check and dropped_votes() == 0, which
were the point of the test and still hold.

Full suite now 151/151 in sae-builder-cpu.

TRACES: AR-008, AR-013 | SR-002
2026-08-31 08:04:17 +02:00
dtourolle 5ce6394416 chore: update KPN — diagnostics.hpp includes the <vector> it uses
Traceability Validation / Check requirement traces (pull_request) Successful in 10s
Unit tests / Build and run the GPU-free suite (pull_request) Failing after 40s
af9da7d on top of 5375ab4, and the reason it exists is this repository.

kpn/diagnostics.hpp declares NetworkSnapshot with two std::vector members
and has never included <vector>. It compiled everywhere we build because
something earlier in each translation unit dragged the definition in
first. src/benchmark.hpp includes <kpn/diagnostics.hpp> before its own
<vector>, and under Debian 12 / GCC 12 -- the CI image, and nothing we
develop on -- nothing supplies it, so the two members fail to declare and
every later use of .nodes or .channels fails after them. The errors all
name benchmark.hpp, so it reads as our bug until you look at the first
line of compiler output rather than the last.

This bump also crosses the four perf/phase0-harness commits between
5375ab4 and the pin, which is a larger move than a header fix and worth
being explicit about. Checked rather than assumed: the two tests failing
in CI fail identically with the pin at 771b9f8 plus the include fix
alone, so they are not this bump's doing. The perf work changes the
scheduler and pool; the failures are tracker extinction-window logic and
are reachable with no pool at all.

Also found while checking KPN's public headers standalone under GCC 12,
and NOT fixed there: network.hpp names IChannelProbe (channel.hpp:557)
without including channel.hpp -- the same latent bug, waiting for the
same kind of consumer. web_debug.hpp needs httplib.h, which is an
optional external dependency and not a defect.
2026-08-30 22:42:26 +02:00
dtourolle 80c23d39d1 fix(ci): sae-builder-cpu could not configure this repository at all
v1 has no libfftw3. SAE_SCENE_XGB is on by default and pulls the learned
scene-boundary detector into result_sink, whose find_library(FFTW3_LIB
fftw3 REQUIRED) at CMakeLists.txt:377 runs at CONFIGURE time -- so cmake
fails before any target is chosen, and building only `sae_tests` does not
get you past it. The image predates the detector; nothing rebuilt it when
the detector landed, which is precisely the gap DP-007's own register row
records.

-DSAE_SCENE_XGB=OFF would also have made the configure pass. It is the
wrong fix and worth saying why: CI is the only place these tests run, so
a flag that silences a subsystem there silences it everywhere, and the
job would have gone green over a detector nobody was compiling.

The image's self-check gains a line, because an image that ships a
library CMake cannot find the way CMake looks for it is the failure this
block exists to catch. FFTW is checked as a bare .so rather than through
pkg-config, since find_library is what CMakeLists.txt actually calls --
checking fftw3.pc would pass on an image whose libfftw3.so was absent.

Tag bumped v1 -> v2 here, in build_builder_image.sh and in the workflow's
run-time assert, in one commit, as this image's contract requires. v2 is
built and published. Rehearsed in the image before pinning it: configure,
build and 151 tests now run.

TRACES: DP-007 | PR-004
2026-08-30 22:42:15 +02:00
dtourolle c5a8502bbb ci(tests): publish what the job was told to pull, and pin it
Traceability Validation / Check requirement traces (pull_request) Successful in 40s
Unit tests / Build and run the GPU-free suite (pull_request) Failing after 36s
DP-007's row says the builder image is "pinned by tag in the Gitea
container registry". The tag was pinned; the image was never pushed.
Every run since the workflow landed died at
"Error response from daemon: manifest unknown" before a single step ran,
so the tier this workflow exists to execute has still never executed.

sae-builder-cpu:v1 is now in the registry, built from ff3b8eb by
scripts/ci/build_builder_image.sh and reporting SAE_BUILDER_VERSION=v1,
which is what the job's own assert-the-image step demands. The
v1-ff3b8eb audit tag went with it.

Behind that, the same absence one layer down: no replay-fixtures package
existed either, so "Verify the fixtures actually arrived" would have
failed next. tests/fixtures/dumps is now published at version ff3b8eb.

Which makes the `latest` here worth removing rather than keeping. Two
reasons, either sufficient. It is the same argument DP-007 already
makes about the image tag -- a dump is an input to the tests, so a
re-upload under a moving `latest` retroactively changes what an earlier
green build proved. And `latest` was the only thing in this job that
wanted a credential: package downloads are anonymous while the repo is
public, and only resolving `latest` needs a token for the list endpoint.
No GITEA_TOKEN secret is configured on this repo, so that step could
never have resolved `latest` even once the image existed. Pinning
deletes the dependency instead of documenting it.

The failure message below it said the step "needs GITEA_TOKEN to resolve
'latest'"; it now says to check the pinned version still exists, which
is the thing that can actually go wrong -- pull_artifacts.sh warns and
continues on a missing version rather than failing, which is why that
verify step is there at all.

TRACES: DP-007 | PR-004
2026-08-30 22:09:02 +02:00
dtourolle fb4e4d4abc ci(traceability): the job container is where act_runner looks for node
The traceability job has been failing since it was written, and never on
anything it checks. It declared no `container:`, so it ran in the
runner's default image, which has no node; act_runner executes JS
actions with the node it finds inside the job container, and both
actions/checkout and upload-artifact are JS. The job died at
"Cannot find: node in PATH" before the repository was checked out, and
every later step then failed on an empty working tree -- the summary
step's "head: cannot open 'docs/traceability.md'" is that, not a missing
report.

node:20-bookworm, and the choice is not arbitrary: node is the part with
no workaround, while Debian 12 already carries python3.11 (tomllib, so
the stdlib-only extractor reads traceability.toml) and git (for the
pull_request diff step). Nothing in it names this repo, so the file
stays copyable into the other two components unedited, which its header
claims and this commit keeps true.

The kpnpp-builder and jellytau-builder images bake node in for exactly
this reason, and Dockerfile.builder-cpu says so in a comment. That
knowledge just had not reached the one job with no image of its own.

Second failure behind the first: the jray-project submodule was pinned
by SSH URL. The runner has no key, so `submodules: recursive` could not
have fetched the extractor even with node present. https, like the KPN
submodule beside it.

Gate and static check both pass locally on this tree -- 42/72 traced,
73.7% in CI scope, 0 orphans, 0 bare-cosine violations -- so what CI
reports next is a fact about CI, not about the tree.
2026-08-30 22:08:51 +02:00
8 changed files with 108 additions and 28 deletions
+16 -1
View File
@@ -7,7 +7,8 @@ name: Traceability Validation
# prefixes count, which file suffixes are source, which directories to scan,
# the threshold - lives in traceability.toml at the repo root, and the same
# extractor is shared by all three JRay components. Copying this file into
# another component needs no edits.
# another component needs no edits - including the container image below, which
# is a stock public one and names nothing about this repo.
#
# NOTE: the runner here is an Intel N100 with no discrete GPU. This job is only
# ever static analysis of source comments plus markdown parsing, so it is cheap;
@@ -31,6 +32,20 @@ jobs:
runs-on: linux/amd64
name: Check requirement traces
# Gitea's act_runner executes JS actions - actions/checkout and
# upload-artifact are both JS - with the `node` binary found INSIDE the job
# container, not one it supplies. The runner's default image has none, so
# without this block the job dies at "Cannot find: node in PATH" before the
# repository is even checked out, and every later step fails on a missing
# working tree rather than on anything it was meant to check.
#
# node:20-bookworm rather than a Python image because node is the part that
# cannot be worked around: bookworm's python3 is 3.11, which has tomllib and
# is therefore already everything the stdlib-only extractor needs. Debian 12
# also carries the git the pull_request diff step below shells out to.
container:
image: node:20-bookworm
steps:
- name: Checkout repository
uses: actions/checkout@v4
+19 -7
View File
@@ -40,7 +40,7 @@ jobs:
# bumping the tag in scripts/ci/build_builder_image.sh AND here, in one
# commit -- see that script's header.
container:
image: gitea.tourolle.paris/dtourolle/sae-builder-cpu:v1
image: gitea.tourolle.paris/dtourolle/sae-builder-cpu:v3
steps:
- name: Checkout repository
@@ -73,17 +73,27 @@ jobs:
# line above and the image that actually landed disagree, which is
# exactly the drift the pinning exists to prevent -- so it fails the
# job rather than building against an unknown toolchain.
[ "$SAE_BUILDER_VERSION" = "v1" ] || {
echo "image reports version '$SAE_BUILDER_VERSION', workflow pins v1" >&2
[ "$SAE_BUILDER_VERSION" = "v3" ] || {
echo "image reports version '$SAE_BUILDER_VERSION', workflow pins v3" >&2
exit 1
}
# Pinned to a version, never `latest`, for the same reason the builder
# image above is: a dump is an input to the tests, so a moving `latest`
# would let a re-upload retroactively change what an earlier green build
# proved. It also removes a credential from this job entirely -- package
# DOWNLOADS are anonymous while the repo is public, and only resolving
# `latest` needs a token (the list-packages endpoint requires auth on this
# instance). `latest` was the sole reason this step wanted GITEA_TOKEN,
# and no such secret is configured, so it could never have resolved.
#
# Bumping the fixtures means uploading a new version with
# scripts/artifacts/push_artifacts.sh replay-fixtures and editing the SHA
# here, in the same commit -- as with the image tag.
- name: Fetch replay fixtures
env:
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
# bash, not sh: the script declares #!/bin/bash and uses `set -o
# pipefail` and arrays, which dash does not have.
run: bash scripts/artifacts/pull_artifacts.sh replay-fixtures latest
run: bash scripts/artifacts/pull_artifacts.sh replay-fixtures ff3b8eb
# pull_artifacts.sh warns and continues when a package version is missing,
# which is right for a developer pulling one artifact of several and wrong
@@ -107,7 +117,9 @@ jobs:
echo "Replay fixtures are absent, so the T2 tier cannot run." >&2
echo "They are not in git (tests/fixtures/dumps/.gitignore) -- they" >&2
echo "live in the Gitea generic package registry and are pulled by" >&2
echo "the step above, which needs GITEA_TOKEN to resolve 'latest'." >&2
echo "the step above, at the version pinned there. Check that the" >&2
echo "version still exists in the registry: pull_artifacts.sh warns" >&2
echo "and continues on a missing one rather than failing." >&2
exit 1
fi
+1 -1
View File
@@ -4,4 +4,4 @@
branch = master
[submodule "jray-project"]
path = scripts/vendor/jray-project
url = git@gitea.tourolle.paris:dtourolle/jray-project.git
url = https://gitea.tourolle.paris/dtourolle/jray-project.git
+23 -3
View File
@@ -92,6 +92,11 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
git \
ca-certificates \
curl \
# unzip is not optional here: the T2 replay fixtures ship as a zip in the
# generic package registry and scripts/artifacts/pull_artifacts.sh unpacks
# them with it. Without unzip that step exits 127 and the replay tier has no
# input -- which is a missing tool reported as a missing fixture.
unzip \
# Gitea's act_runner executes JS actions (actions/checkout, upload-artifact)
# with the `node` found *inside* the container. Without this the job cannot
# even check the repository out. Same reason as the kpnpp-builder image.
@@ -116,6 +121,17 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
# Both the main build (CMakeLists.txt:162) and the test target
# (tests/CMakeLists.txt:42) discover it through pkg-config `openblas`.
libopenblas-dev \
# FFTW3. The learned scene-boundary detector is compiled into result_sink
# whenever SAE_SCENE_XGB is on -- which is the default -- and it pulls in
# xgboost + FFTW + ffmpeg together (CMakeLists.txt:377). The find_library is
# REQUIRED and runs at CONFIGURE time, so a missing libfftw3 fails the whole
# cmake step: the `sae_tests` target never gets as far as being irrelevant
# to it. This image predates the detector, which is why it was absent.
#
# Turning SAE_SCENE_XGB off in CI would also make the configure pass, and it
# is the wrong fix: CI is the only place these tests run, so the flag that
# silences a subsystem there silences it everywhere.
libfftw3-dev \
# Python: the build itself needs the interpreter and headers
# (find_package(Python COMPONENTS Interpreter Development.Module) at
# CMakeLists.txt:254, for the nanobind modules). numpy/h5py/scipy are for
@@ -140,14 +156,18 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
zlib1g-dev \
&& rm -rf /var/lib/apt/lists/*
# Fail the image build, not the CI run, if OpenBLAS or swresample are not
# discoverable the way CMakeLists.txt discovers them. An image that ships
# Fail the image build, not the CI run, if OpenBLAS, swresample or FFTW are
# not discoverable the way CMakeLists.txt discovers them. An image that ships
# libopenblas but no openblas.pc would compile the scalar fallback in silence.
# FFTW is checked as a bare .so rather than by pkg-config because that is how
# CMake looks for it -- find_library(FFTW3_LIB fftw3), not pkg_check_modules.
RUN set -eux; \
pkg-config --exists openblas; \
echo "openblas $(pkg-config --modversion openblas)"; \
pkg-config --exists libswresample; \
echo "swresample $(pkg-config --modversion libswresample)"
echo "swresample $(pkg-config --modversion libswresample)"; \
ls /usr/lib/*/libfftw3.so > /dev/null; \
echo "fftw3 $(pkg-config --modversion fftw3)"
# ─── ONNX Runtime, CPU provider only ─────────────────────────────────────────
#
+1 -1
+7 -3
View File
@@ -5,9 +5,9 @@
# TRACES: DP-007 | PR-004
#
# Usage:
# scripts/ci/build_builder_image.sh # build only, tag v1
# scripts/ci/build_builder_image.sh # build only, tag v3
# scripts/ci/build_builder_image.sh --push # build and push
# scripts/ci/build_builder_image.sh --tag v2 --push # bump the pinned tag
# scripts/ci/build_builder_image.sh --tag v4 --push # bump the pinned tag
# scripts/ci/build_builder_image.sh --no-cache # force a clean rebuild
#
# The tag is the contract with CI. .gitea/workflows/unit-tests.yml names an
@@ -34,7 +34,11 @@ DOCKERFILE="Dockerfile.builder-cpu"
# .gitea/workflows/unit-tests.yml; the workflow asserts at run time that the
# image it landed in reports this same version, so a drift shows up as a failed
# job rather than as a build against the wrong toolchain.
TAG="v1"
#
# v2 added libfftw3-dev: the learned scene-boundary detector's find_library is
# REQUIRED at configure time, so v1 cannot configure this repository at all.
# v3 adds unzip, which pull_artifacts.sh needs to unpack the replay fixtures.
TAG="v3"
PUSH=0
EXTRA_ARGS=()
+7
View File
@@ -192,6 +192,13 @@ TEST_CASE("a track within the extinction window is still a candidate",
Embedding person = at_sim(0, 1, 0.99f);
int id_pre = r.track_of(frame(0.0, 10, 10, person));
// Identify it while it is still on screen. Since 1477c53 a DORMANT track is
// only associable if it was owned -- an anonymous one has no actor to
// re-attach to, so it is dropped from the pool a line before the horizon is
// ever consulted. Without this the test cannot reach what it is about.
r.reg->observe(id_pre, /*actor=*/0, /*posterior=*/0.99f, person);
r.track_of(frame(1.0, 300, 300, axis(7), /*is_cut=*/true));
// Back inside the window: the same person continues the same track, so the
+34 -12
View File
@@ -415,15 +415,21 @@ TEST_CASE("a track is not reaped until the evidence clock passes it",
}
/// TRACES: UT-001 | AR-008, AR-013 | SR-002
TEST_CASE("a track retired from association is still open to evidence",
TEST_CASE("association and reaping retire a track on the same clock",
"[registry][AR-008]") {
// The two clocks answer different questions and must not share an answer.
// Association asks "may this detection link to that track?" on the tracker's
// clock; reaping asks "is that track finished?" and cannot answer until the
// votes are in. Deferring both to the evidence clock was the second half of
// this bug: retired tracks lingered in the candidate pool for as long as the
// matcher lagged, so a new face re-associated onto a long-dead track and two
// people merged into one window.
// Renamed and rewritten for the unified clock. This test used to assert the
// opposite arrangement -- association answered on the tracker's clock while
// reaping answered on the evidence watermark -- and it was written when that
// was true. It is not true now: candidates() and reap_locked() take the SAME
// clock and the SAME threshold, so "retired from association" and "reaped"
// are one event, and the interval the old name described does not exist.
//
// Both halves of the AR-013 bug are visible here. Offering on a LOOSER
// horizon than the reap left retired tracks in the candidate pool for as
// long as the matcher lagged, so a new face re-associated onto a long-dead
// track and two people merged into one window. Reaping on a looser horizon
// than the offer killed tracks whose votes were still in flight. One clock
// makes both unrepresentable.
TrackRegistry reg(cfg(/*extinction=*/5.0), disc());
Sink sink; sink.attach(reg);
reg.expect_evidence();
@@ -432,21 +438,37 @@ TEST_CASE("a track retired from association is still open to evidence",
{
auto s = reg.begin_frame(0.0);
id = s.create(0.0, axis(1));
}
// Owned before it goes dormant, for the reason above: an unidentified
// dormant track is not a candidate at any horizon (1477c53).
reg.observe(id, 7, 0.99f, axis(1));
{
auto s = reg.begin_frame(0.0);
s.mark_lost(id, 0.0);
}
{
auto s = reg.begin_frame(3.0); // inside the window
auto s = reg.begin_frame(3.0); // watermark inside the window
CHECK(s.candidates().size() == 1); // still associable
}
// The tracker races far past the horizon while the matcher lags. This
// retires nothing, and that is the point of the watermark: the clock that
// decides has not moved, so the track is still offered.
{
auto s = reg.begin_frame(50.0); // far outside it
CHECK(s.candidates().empty()); // retired from association...
auto s = reg.begin_frame(50.0);
CHECK(s.candidates().size() == 1);
}
// ...but not gone, and still able to receive the votes in flight for it.
// Which is what keeps it able to receive the votes still in flight for it.
reg.observe(id, 7, 0.99f, axis(1));
CHECK(reg.dropped_votes() == 0);
// Only the watermark passing last_seen + extinction retires it -- and the
// same call reaps and emits it, so there is no interval in between.
reg.advance_evidence(50.0);
{
auto s = reg.begin_frame(50.0);
CHECK(s.candidates().empty());
}
REQUIRE(sink.claims.size() == 1);
CHECK(sink.claims[0].actor_idx == 7);
}