Author SHA1 Message Date
dtourolle 5ce6394416 chore: update KPN — diagnostics.hpp includes the <vector> it uses
Traceability Validation / Check requirement traces (pull_request) Successful in 10s
Unit tests / Build and run the GPU-free suite (pull_request) Failing after 40s
af9da7d on top of 5375ab4, and the reason it exists is this repository.

kpn/diagnostics.hpp declares NetworkSnapshot with two std::vector members
and has never included <vector>. It compiled everywhere we build because
something earlier in each translation unit dragged the definition in
first. src/benchmark.hpp includes <kpn/diagnostics.hpp> before its own
<vector>, and under Debian 12 / GCC 12 -- the CI image, and nothing we
develop on -- nothing supplies it, so the two members fail to declare and
every later use of .nodes or .channels fails after them. The errors all
name benchmark.hpp, so it reads as our bug until you look at the first
line of compiler output rather than the last.

This bump also crosses the four perf/phase0-harness commits between
5375ab4 and the pin, which is a larger move than a header fix and worth
being explicit about. Checked rather than assumed: the two tests failing
in CI fail identically with the pin at 771b9f8 plus the include fix
alone, so they are not this bump's doing. The perf work changes the
scheduler and pool; the failures are tracker extinction-window logic and
are reachable with no pool at all.

Also found while checking KPN's public headers standalone under GCC 12,
and NOT fixed there: network.hpp names IChannelProbe (channel.hpp:557)
without including channel.hpp -- the same latent bug, waiting for the
same kind of consumer. web_debug.hpp needs httplib.h, which is an
optional external dependency and not a defect.
2026-08-30 22:42:26 +02:00
dtourolle 80c23d39d1 fix(ci): sae-builder-cpu could not configure this repository at all
v1 has no libfftw3. SAE_SCENE_XGB is on by default and pulls the learned
scene-boundary detector into result_sink, whose find_library(FFTW3_LIB
fftw3 REQUIRED) at CMakeLists.txt:377 runs at CONFIGURE time -- so cmake
fails before any target is chosen, and building only `sae_tests` does not
get you past it. The image predates the detector; nothing rebuilt it when
the detector landed, which is precisely the gap DP-007's own register row
records.

-DSAE_SCENE_XGB=OFF would also have made the configure pass. It is the
wrong fix and worth saying why: CI is the only place these tests run, so
a flag that silences a subsystem there silences it everywhere, and the
job would have gone green over a detector nobody was compiling.

The image's self-check gains a line, because an image that ships a
library CMake cannot find the way CMake looks for it is the failure this
block exists to catch. FFTW is checked as a bare .so rather than through
pkg-config, since find_library is what CMakeLists.txt actually calls --
checking fftw3.pc would pass on an image whose libfftw3.so was absent.

Tag bumped v1 -> v2 here, in build_builder_image.sh and in the workflow's
run-time assert, in one commit, as this image's contract requires. v2 is
built and published. Rehearsed in the image before pinning it: configure,
build and 151 tests now run.

TRACES: DP-007 | PR-004
2026-08-30 22:42:15 +02:00
dtourolle c5a8502bbb ci(tests): publish what the job was told to pull, and pin it
Traceability Validation / Check requirement traces (pull_request) Successful in 40s
Unit tests / Build and run the GPU-free suite (pull_request) Failing after 36s
DP-007's row says the builder image is "pinned by tag in the Gitea
container registry". The tag was pinned; the image was never pushed.
Every run since the workflow landed died at
"Error response from daemon: manifest unknown" before a single step ran,
so the tier this workflow exists to execute has still never executed.

sae-builder-cpu:v1 is now in the registry, built from ff3b8eb by
scripts/ci/build_builder_image.sh and reporting SAE_BUILDER_VERSION=v1,
which is what the job's own assert-the-image step demands. The
v1-ff3b8eb audit tag went with it.

Behind that, the same absence one layer down: no replay-fixtures package
existed either, so "Verify the fixtures actually arrived" would have
failed next. tests/fixtures/dumps is now published at version ff3b8eb.

Which makes the `latest` here worth removing rather than keeping. Two
reasons, either sufficient. It is the same argument DP-007 already
makes about the image tag -- a dump is an input to the tests, so a
re-upload under a moving `latest` retroactively changes what an earlier
green build proved. And `latest` was the only thing in this job that
wanted a credential: package downloads are anonymous while the repo is
public, and only resolving `latest` needs a token for the list endpoint.
No GITEA_TOKEN secret is configured on this repo, so that step could
never have resolved `latest` even once the image existed. Pinning
deletes the dependency instead of documenting it.

The failure message below it said the step "needs GITEA_TOKEN to resolve
'latest'"; it now says to check the pinned version still exists, which
is the thing that can actually go wrong -- pull_artifacts.sh warns and
continues on a missing version rather than failing, which is why that
verify step is there at all.

TRACES: DP-007 | PR-004
2026-08-30 22:09:02 +02:00
dtourolle fb4e4d4abc ci(traceability): the job container is where act_runner looks for node
The traceability job has been failing since it was written, and never on
anything it checks. It declared no `container:`, so it ran in the
runner's default image, which has no node; act_runner executes JS
actions with the node it finds inside the job container, and both
actions/checkout and upload-artifact are JS. The job died at
"Cannot find: node in PATH" before the repository was checked out, and
every later step then failed on an empty working tree -- the summary
step's "head: cannot open 'docs/traceability.md'" is that, not a missing
report.

node:20-bookworm, and the choice is not arbitrary: node is the part with
no workaround, while Debian 12 already carries python3.11 (tomllib, so
the stdlib-only extractor reads traceability.toml) and git (for the
pull_request diff step). Nothing in it names this repo, so the file
stays copyable into the other two components unedited, which its header
claims and this commit keeps true.

The kpnpp-builder and jellytau-builder images bake node in for exactly
this reason, and Dockerfile.builder-cpu says so in a comment. That
knowledge just had not reached the one job with no image of its own.

Second failure behind the first: the jray-project submodule was pinned
by SSH URL. The runner has no key, so `submodules: recursive` could not
have fetched the extractor even with node present. https, like the KPN
submodule beside it.

Gate and static check both pass locally on this tree -- 42/72 traced,
73.7% in CI scope, 0 orphans, 0 bare-cosine violations -- so what CI
reports next is a fact about CI, not about the tree.
2026-08-30 22:08:51 +02:00
dtourolle ff3b8ebf1d Merge feature/opencv5: correct scene-boundary detector F1 numbers
Traceability Validation / Check requirement traces (push) Failing after 9s
Unit tests / Build and run the GPU-free suite (push) Failing after 2s
Fold in the documentation accuracy fix: the boundary-detection F1 numbers now
reflect the measured values at the shipped ±20s tolerance (44.1% leave-one-out /
29.8% grayscale / 72.9% train-all), replacing the stale pre-retrain '~34%'
figure, and the evolution figure is split so the ±2s development curve is not
mistaken for the shipped result.
2026-08-11 20:56:06 +02:00
dtourolle 30b5ad7da7 Merge feature/opencv5: learned scene-boundary flood-fill pipeline
Traceability Validation / Check requirement traces (push) Failing after 4s
Unit tests / Build and run the GPU-free suite (push) Failing after 1s
The opencv5 rework of the detect/track/match/scene pipeline. Headline result:
flood-fill actor presence on a learned XGBoost scene-boundary detector lifts
per-second Amazon X-Ray presence F1 from 62.6% (track-extent) to 74.9% under
leave-one-out across the nine-film benchmark, improving every film and fixing
the low-contrast grades (Scarface, Downton) that naive flood-fill broke.
2026-08-11 19:47:23 +02:00
6 changed files with 61 additions and 16 deletions
+16 -1
View File
@@ -7,7 +7,8 @@ name: Traceability Validation
# prefixes count, which file suffixes are source, which directories to scan,
# the threshold - lives in traceability.toml at the repo root, and the same
# extractor is shared by all three JRay components. Copying this file into
# another component needs no edits.
# another component needs no edits - including the container image below, which
# is a stock public one and names nothing about this repo.
#
# NOTE: the runner here is an Intel N100 with no discrete GPU. This job is only
# ever static analysis of source comments plus markdown parsing, so it is cheap;
@@ -31,6 +32,20 @@ jobs:
runs-on: linux/amd64
name: Check requirement traces
# Gitea's act_runner executes JS actions - actions/checkout and
# upload-artifact are both JS - with the `node` binary found INSIDE the job
# container, not one it supplies. The runner's default image has none, so
# without this block the job dies at "Cannot find: node in PATH" before the
# repository is even checked out, and every later step fails on a missing
# working tree rather than on anything it was meant to check.
#
# node:20-bookworm rather than a Python image because node is the part that
# cannot be worked around: bookworm's python3 is 3.11, which has tomllib and
# is therefore already everything the stdlib-only extractor needs. Debian 12
# also carries the git the pull_request diff step below shells out to.
container:
image: node:20-bookworm
steps:
- name: Checkout repository
uses: actions/checkout@v4
+19 -7
View File
@@ -40,7 +40,7 @@ jobs:
# bumping the tag in scripts/ci/build_builder_image.sh AND here, in one
# commit -- see that script's header.
container:
image: gitea.tourolle.paris/dtourolle/sae-builder-cpu:v1
image: gitea.tourolle.paris/dtourolle/sae-builder-cpu:v2
steps:
- name: Checkout repository
@@ -73,17 +73,27 @@ jobs:
# line above and the image that actually landed disagree, which is
# exactly the drift the pinning exists to prevent -- so it fails the
# job rather than building against an unknown toolchain.
[ "$SAE_BUILDER_VERSION" = "v1" ] || {
echo "image reports version '$SAE_BUILDER_VERSION', workflow pins v1" >&2
[ "$SAE_BUILDER_VERSION" = "v2" ] || {
echo "image reports version '$SAE_BUILDER_VERSION', workflow pins v2" >&2
exit 1
}
# Pinned to a version, never `latest`, for the same reason the builder
# image above is: a dump is an input to the tests, so a moving `latest`
# would let a re-upload retroactively change what an earlier green build
# proved. It also removes a credential from this job entirely -- package
# DOWNLOADS are anonymous while the repo is public, and only resolving
# `latest` needs a token (the list-packages endpoint requires auth on this
# instance). `latest` was the sole reason this step wanted GITEA_TOKEN,
# and no such secret is configured, so it could never have resolved.
#
# Bumping the fixtures means uploading a new version with
# scripts/artifacts/push_artifacts.sh replay-fixtures and editing the SHA
# here, in the same commit -- as with the image tag.
- name: Fetch replay fixtures
env:
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
# bash, not sh: the script declares #!/bin/bash and uses `set -o
# pipefail` and arrays, which dash does not have.
run: bash scripts/artifacts/pull_artifacts.sh replay-fixtures latest
run: bash scripts/artifacts/pull_artifacts.sh replay-fixtures ff3b8eb
# pull_artifacts.sh warns and continues when a package version is missing,
# which is right for a developer pulling one artifact of several and wrong
@@ -107,7 +117,9 @@ jobs:
echo "Replay fixtures are absent, so the T2 tier cannot run." >&2
echo "They are not in git (tests/fixtures/dumps/.gitignore) -- they" >&2
echo "live in the Gitea generic package registry and are pulled by" >&2
echo "the step above, which needs GITEA_TOKEN to resolve 'latest'." >&2
echo "the step above, at the version pinned there. Check that the" >&2
echo "version still exists in the registry: pull_artifacts.sh warns" >&2
echo "and continues on a missing one rather than failing." >&2
exit 1
fi
+1 -1
View File
@@ -4,4 +4,4 @@
branch = master
[submodule "jray-project"]
path = scripts/vendor/jray-project
url = git@gitea.tourolle.paris:dtourolle/jray-project.git
url = https://gitea.tourolle.paris/dtourolle/jray-project.git
+18 -3
View File
@@ -116,6 +116,17 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
# Both the main build (CMakeLists.txt:162) and the test target
# (tests/CMakeLists.txt:42) discover it through pkg-config `openblas`.
libopenblas-dev \
# FFTW3. The learned scene-boundary detector is compiled into result_sink
# whenever SAE_SCENE_XGB is on -- which is the default -- and it pulls in
# xgboost + FFTW + ffmpeg together (CMakeLists.txt:377). The find_library is
# REQUIRED and runs at CONFIGURE time, so a missing libfftw3 fails the whole
# cmake step: the `sae_tests` target never gets as far as being irrelevant
# to it. This image predates the detector, which is why it was absent.
#
# Turning SAE_SCENE_XGB off in CI would also make the configure pass, and it
# is the wrong fix: CI is the only place these tests run, so the flag that
# silences a subsystem there silences it everywhere.
libfftw3-dev \
# Python: the build itself needs the interpreter and headers
# (find_package(Python COMPONENTS Interpreter Development.Module) at
# CMakeLists.txt:254, for the nanobind modules). numpy/h5py/scipy are for
@@ -140,14 +151,18 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
zlib1g-dev \
&& rm -rf /var/lib/apt/lists/*
# Fail the image build, not the CI run, if OpenBLAS or swresample are not
# discoverable the way CMakeLists.txt discovers them. An image that ships
# Fail the image build, not the CI run, if OpenBLAS, swresample or FFTW are
# not discoverable the way CMakeLists.txt discovers them. An image that ships
# libopenblas but no openblas.pc would compile the scalar fallback in silence.
# FFTW is checked as a bare .so rather than by pkg-config because that is how
# CMake looks for it -- find_library(FFTW3_LIB fftw3), not pkg_check_modules.
RUN set -eux; \
pkg-config --exists openblas; \
echo "openblas $(pkg-config --modversion openblas)"; \
pkg-config --exists libswresample; \
echo "swresample $(pkg-config --modversion libswresample)"
echo "swresample $(pkg-config --modversion libswresample)"; \
ls /usr/lib/*/libfftw3.so > /dev/null; \
echo "fftw3 $(pkg-config --modversion fftw3)"
# ─── ONNX Runtime, CPU provider only ─────────────────────────────────────────
#
+1 -1
+6 -3
View File
@@ -5,9 +5,9 @@
# TRACES: DP-007 | PR-004
#
# Usage:
# scripts/ci/build_builder_image.sh # build only, tag v1
# scripts/ci/build_builder_image.sh # build only, tag v2
# scripts/ci/build_builder_image.sh --push # build and push
# scripts/ci/build_builder_image.sh --tag v2 --push # bump the pinned tag
# scripts/ci/build_builder_image.sh --tag v3 --push # bump the pinned tag
# scripts/ci/build_builder_image.sh --no-cache # force a clean rebuild
#
# The tag is the contract with CI. .gitea/workflows/unit-tests.yml names an
@@ -34,7 +34,10 @@ DOCKERFILE="Dockerfile.builder-cpu"
# .gitea/workflows/unit-tests.yml; the workflow asserts at run time that the
# image it landed in reports this same version, so a drift shows up as a failed
# job rather than as a build against the wrong toolchain.
TAG="v1"
#
# v2 adds libfftw3-dev: the learned scene-boundary detector's find_library is
# REQUIRED at configure time, so v1 cannot configure this repository at all.
TAG="v2"
PUSH=0
EXTRA_ARGS=()