dtourolle
9bcc765408
Found by CI itself, on run 1437 -- the first run that ever got far enough
to try. Everything ahead of it worked: checkout brought both submodules,
v2 pulled and passed its own version assert, and the fixtures resolved at
the pinned version. Then:
scripts/artifacts/pull_artifacts.sh: line 54: unzip: command not found
The replay fixtures ship as a zip in the generic package registry and
pull_artifacts.sh unpacks them with unzip. Without it the step exits 127
and the T2 tier has no input -- reported as a missing fixture by the
verify step below, which is a missing tool wearing the wrong label.
Worth naming why the local rehearsal missed this and CI did not: the
rehearsal had the dumps staged in the working tree by hand, so it built
and ran the tier without ever executing the script that fetches them. The
image was checked against the build, not against the job.
Tag v2 -> v3, in all three places, in one commit.
TRACES: DP-007 | PR-004
123 lines
4.7 KiB
Bash
123 lines
4.7 KiB
Bash
#!/bin/bash
|
|
# build_builder_image.sh — build and publish the DP-007 CI builder image to the
|
|
# Gitea container registry.
|
|
#
|
|
# TRACES: DP-007 | PR-004
|
|
#
|
|
# Usage:
|
|
# scripts/ci/build_builder_image.sh # build only, tag v3
|
|
# scripts/ci/build_builder_image.sh --push # build and push
|
|
# scripts/ci/build_builder_image.sh --tag v4 --push # bump the pinned tag
|
|
# scripts/ci/build_builder_image.sh --no-cache # force a clean rebuild
|
|
#
|
|
# The tag is the contract with CI. .gitea/workflows/unit-tests.yml names an
|
|
# explicit tag in its `container:` block and never `latest`, so that rebuilding
|
|
# the image cannot silently change what a previous green build meant. Bumping
|
|
# the dependency set means bumping the tag AND editing the workflow — the two
|
|
# edits landing in the same commit is the point, not an inconvenience.
|
|
#
|
|
# Registry auth: this script does not log in. Do it once, out of band:
|
|
# docker login gitea.tourolle.paris
|
|
# The CI host is already authenticated this way (its cached credentials in
|
|
# ~/.docker/config.json are what the kpnpp-builder push relies on), so a
|
|
# workflow that calls this script needs no secret plumbing.
|
|
set -euo pipefail
|
|
|
|
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
|
|
|
REGISTRY="gitea.tourolle.paris"
|
|
OWNER="dtourolle"
|
|
IMAGE="sae-builder-cpu"
|
|
DOCKERFILE="Dockerfile.builder-cpu"
|
|
|
|
# The tag CI pins to today. Keep this in step with the `container.image` line in
|
|
# .gitea/workflows/unit-tests.yml; the workflow asserts at run time that the
|
|
# image it landed in reports this same version, so a drift shows up as a failed
|
|
# job rather than as a build against the wrong toolchain.
|
|
#
|
|
# v2 added libfftw3-dev: the learned scene-boundary detector's find_library is
|
|
# REQUIRED at configure time, so v1 cannot configure this repository at all.
|
|
# v3 adds unzip, which pull_artifacts.sh needs to unpack the replay fixtures.
|
|
TAG="v3"
|
|
|
|
PUSH=0
|
|
EXTRA_ARGS=()
|
|
|
|
while [ $# -gt 0 ]; do
|
|
case "$1" in
|
|
--push) PUSH=1 ;;
|
|
--tag) TAG="${2:?--tag needs a value}"; shift ;;
|
|
--no-cache) EXTRA_ARGS+=(--no-cache) ;;
|
|
-h|--help) sed -n '2,30p' "${BASH_SOURCE[0]}"; exit 0 ;;
|
|
*) echo "error: unknown argument '$1'" >&2; exit 2 ;;
|
|
esac
|
|
shift
|
|
done
|
|
|
|
if [ "$TAG" = "latest" ]; then
|
|
echo "error: refusing to build the tag 'latest'." >&2
|
|
echo "DP-007 requires CI to pin an immutable tag. A moving 'latest' means a" >&2
|
|
echo "rebuild retroactively changes what every earlier green build proved." >&2
|
|
exit 2
|
|
fi
|
|
|
|
REF="${REGISTRY}/${OWNER}/${IMAGE}:${TAG}"
|
|
# A second tag carrying the commit that produced the image. The workflow pins
|
|
# the human-readable tag; this one is the audit trail — given any image you can
|
|
# recover the Dockerfile that built it.
|
|
SHA="$(git -C "$REPO_ROOT" rev-parse --short HEAD)"
|
|
REF_SHA="${REGISTRY}/${OWNER}/${IMAGE}:${TAG}-${SHA}"
|
|
|
|
# The Dockerfile COPYs nothing from the repository on purpose (see its closing
|
|
# comment), so the build context is an empty directory rather than the repo
|
|
# root. Sending ~1 GB of models, fixtures and experiment data to the daemon for
|
|
# a build that reads none of it is pure latency.
|
|
CONTEXT="$(mktemp -d)"
|
|
trap 'rm -rf "$CONTEXT"' EXIT
|
|
|
|
echo "=== building ${REF}"
|
|
echo " dockerfile: ${REPO_ROOT}/${DOCKERFILE}"
|
|
echo " context: (empty — the image embeds no repository content)"
|
|
echo
|
|
echo " Expect this to take a while: OpenCV 5 is compiled from source because"
|
|
echo " no Debian release ships it. That cost is paid once per image, which is"
|
|
echo " the entire reason DP-007 asks for a prebuilt image instead of"
|
|
echo " installing dependencies inside each CI run."
|
|
echo
|
|
|
|
docker build \
|
|
"${EXTRA_ARGS[@]}" \
|
|
--build-arg "IMAGE_TAG=${TAG}" \
|
|
-f "${REPO_ROOT}/${DOCKERFILE}" \
|
|
-t "${REF}" \
|
|
-t "${REF_SHA}" \
|
|
"${CONTEXT}"
|
|
|
|
echo
|
|
echo "=== built"
|
|
docker image inspect "${REF}" --format ' {{.RepoTags}} {{.Size}} bytes'
|
|
docker run --rm "${REF}" sh -c 'echo " SAE_BUILDER=$SAE_BUILDER version=$SAE_BUILDER_VERSION ort=$SAE_ORT_VERSION opencv=$SAE_OPENCV_VERSION"'
|
|
|
|
if [ "$PUSH" -eq 0 ]; then
|
|
echo
|
|
echo "Not pushed. Re-run with --push, or push by hand:"
|
|
echo " docker push ${REF}"
|
|
echo " docker push ${REF_SHA}"
|
|
exit 0
|
|
fi
|
|
|
|
echo
|
|
echo "=== pushing"
|
|
# No `latest` tag is pushed, by design. Publishing one invites a workflow to use
|
|
# it, and DP-007 exists to prevent exactly that.
|
|
docker push "${REF}"
|
|
docker push "${REF_SHA}"
|
|
|
|
echo
|
|
echo "=== published ${REF}"
|
|
echo "If this was a dependency-set change, bump the tag in"
|
|
echo " .gitea/workflows/unit-tests.yml (container.image)"
|
|
echo " scripts/ci/build_builder_image.sh (TAG, above)"
|
|
echo "in the same commit, so no run can build against an image the repository"
|
|
echo "does not describe."
|