From b044a8c06708034fc6f890397e7401768cda9b85 Mon Sep 17 00:00:00 2001 From: Duncan Tourolle Date: Sun, 16 Aug 2026 14:59:57 +0200 Subject: [PATCH] Build the Android CI image in CI, not on a laptop MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Android job ran in gitea.tourolle.paris/dtourolle/darkroom-android:latest, a tag that had never been pushed. The image existed only as a local darkroom-android:latest on one machine, so every Android job died at docker pull with "manifest unknown" before reaching a step. The registry API confirms it: that manifest is a 404 while the other builder images answer 200. android-image.yml now builds and pushes it, following KPN's docker.yaml — host runner rather than a container, so it has the Docker daemon and the host's cached registry credentials, and a plain-git checkout because that host has no Node for actions/checkout. Where it diverges from KPN: that workflow gates on dorny/paths-filter running inside the builder image, which here would need the very image that is missing. The tag is the git tree hash of docker/android instead, which changes when and only when a file there changes. An unrelated push reuses the image, a Dockerfile edit cannot keep serving a stale latest, and a missing tag rebuilds itself without a manual step. The presence probe is curl against the registry API, not `docker manifest inspect`. The latter exits 1 on this registry even for tags that are plainly there — jellytau-builder:latest answers HTTP 200 while docker reports "manifest unknown" for it — and trusting it would have rebuilt 7 GB on every push. The HEAD request also yields Docker-Content-Digest, so latest is repointed only when the digests actually disagree, without pulling any layers. A probe that cannot authenticate falls through to building. Rebuilding when it was unnecessary costs minutes; skipping a build that was needed is the failure this commit exists to remove. Co-Authored-By: Claude Opus 5 --- .gitea/workflows/android-image.yml | 170 ++++++++++++++++++++++++++++ .gitea/workflows/build-and-test.yml | 11 ++ docker/android/README.md | 15 +++ 3 files changed, 196 insertions(+) create mode 100644 .gitea/workflows/android-image.yml diff --git a/.gitea/workflows/android-image.yml b/.gitea/workflows/android-image.yml new file mode 100644 index 0000000..998c5d2 --- /dev/null +++ b/.gitea/workflows/android-image.yml @@ -0,0 +1,170 @@ +name: '🐳 Android image' + +# Builds and pushes gitea.tourolle.paris/dtourolle/darkroom-android, the job +# container for the Android leg of build-and-test.yml. +# +# It exists because that image previously lived only on a developer's laptop: +# the workflow referenced a tag that had never been pushed, and every Android +# job died at `docker pull` with "manifest unknown" before running a step. The +# image is now reproducible from the repo rather than from one machine. +# +# Called by build-and-test.yml on every push, and runnable by hand via +# workflow_dispatch. It is cheap when nothing changed — see the guard below. +on: + workflow_call: + inputs: + force: + description: 'Rebuild even if the registry already has this image ("true"/"false")' + type: string + default: 'false' + workflow_dispatch: + inputs: + force: + description: 'Rebuild even if the registry already has this image ("true"/"false")' + type: string + default: 'false' + +# Gitea's act_runner mangles boolean workflow inputs passed through an +# expression — they arrive as false regardless of what was sent. Every input +# here is a string compared with == 'true', as in KPN's docker.yaml. + +env: + IMAGE: gitea.tourolle.paris/dtourolle/darkroom-android + +jobs: + build: + runs-on: linux/amd64 + name: Build and push + # Deliberately NOT in a container: this job needs the host Docker daemon to + # build an image, and the host's cached ~/.docker/config.json to push it. + # That is also why there is no `docker login` step — the runner host was + # authenticated to the registry during setup. + + steps: + # The host has no Node, so the JS-based actions/checkout cannot run here. + # A minimal shallow fetch with plain git gets the same tree. + - name: Checkout + run: | + set -e + git init -q . + git remote add origin "${{ github.server_url }}/${{ github.repository }}.git" + git -c http.extraheader="AUTHORIZATION: basic $(printf '%s' '${{ github.actor }}:${{ github.token }}' | base64 -w0)" \ + fetch --depth 1 origin "${{ github.sha }}" + git checkout -q FETCH_HEAD + + # The image is tagged by the content of docker/android, not by the commit + # that happened to touch it. `git rev-parse HEAD:` is the tree object + # id — it changes when and only when a file in that directory changes, so + # an unrelated push reuses the existing image and a Dockerfile edit can + # never silently keep serving a stale `latest`. + # + # Using the commit sha instead would rebuild 7 GB on every push; using a + # paths-filter action would need a container that has Node, and the only + # one this repo would reach for is the very image being built. + - name: Resolve image tag + id: tag + run: | + set -e + TREE=$(git rev-parse HEAD:docker/android) + echo "tree=$TREE" >> "$GITHUB_OUTPUT" + echo "docker/android tree: $TREE" + + # Skip the build when the registry already holds this exact content. This + # is what keeps the job a few seconds long on a normal push, and what + # makes it self-healing: if the tag is missing for any reason, including + # the image having never been pushed at all, it gets built here. + # + # The probe is curl against the registry API, NOT `docker manifest + # inspect`. The latter exits 1 on this registry even for tags that are + # demonstrably present — jellytau-builder:latest answers HTTP 200 to the + # API while `docker manifest inspect` reports "manifest unknown" for it. + # Trusting that would have rebuilt 7 GB on every single push. + # + # A HEAD request also gives the digest for free, which is how the repoint + # decision below is made without pulling any layers. + - name: Query registry + id: check + env: + # The runner's own credentials, so this does not depend on how the + # host's ~/.docker/config.json happens to be set up. + REG_USER: ${{ github.actor }} + REG_PASS: ${{ github.token }} + TREE: ${{ steps.tag.outputs.tree }} + run: | + set -eu + ACCEPT='application/vnd.oci.image.index.v1+json,application/vnd.docker.distribution.manifest.v2+json,application/vnd.oci.image.manifest.v1+json,application/vnd.docker.distribution.manifest.list.v2+json' + API="https://gitea.tourolle.paris/v2/dtourolle/darkroom-android/manifests" + + # Prints " " for a tag. + probe() { + curl -sI -u "$REG_USER:$REG_PASS" -H "Accept: $ACCEPT" "$API/$1" \ + | tr -d '\r' \ + | awk 'BEGIN{s="000";d=""} /^HTTP/{s=$2} tolower($1)=="docker-content-digest:"{d=$2} END{print s, d}' + } + + read -r TREE_STATUS TREE_DIGEST < HTTP $TREE_STATUS ${TREE_DIGEST:-(no digest)}" + echo "tag latest -> HTTP $LATEST_STATUS ${LATEST_DIGEST:-(no digest)}" + + # Build unless the registry definitively confirms this content is + # already there. An auth failure or an unreachable registry lands + # here too, and rebuilding needlessly is the safe direction to fail — + # skipping a build that was needed is what breaks the Android job. + if [ "${{ inputs.force }}" = "true" ]; then + echo "forced rebuild requested" + echo "build=true" >> "$GITHUB_OUTPUT" + echo "repoint=false" >> "$GITHUB_OUTPUT" + elif [ "$TREE_STATUS" != "200" ]; then + echo "registry does not have this content — building" + echo "build=true" >> "$GITHUB_OUTPUT" + echo "repoint=false" >> "$GITHUB_OUTPUT" + elif [ -n "$TREE_DIGEST" ] && [ "$TREE_DIGEST" = "$LATEST_DIGEST" ]; then + echo "registry is already correct — nothing to do" + echo "build=false" >> "$GITHUB_OUTPUT" + echo "repoint=false" >> "$GITHUB_OUTPUT" + else + echo "content is present but latest points elsewhere — repointing" + echo "build=false" >> "$GITHUB_OUTPUT" + echo "repoint=true" >> "$GITHUB_OUTPUT" + fi + + # Context is docker/android, matching the README's build command. The + # Dockerfile COPYs nothing from the repo, so it needs no wider context — + # and a narrow context keeps the daemon from tarring up the whole tree, + # target/ included. + - name: Build + if: ${{ steps.check.outputs.build == 'true' }} + run: | + set -e + docker build \ + -t "$IMAGE:${{ steps.tag.outputs.tree }}" \ + -t "$IMAGE:latest" \ + docker/android + + # Both tags are pushed: the tree tag is what the guard above looks for on + # the next run, and `latest` is what build-and-test.yml pulls. + - name: Push + if: ${{ steps.check.outputs.build == 'true' }} + run: | + set -e + docker push "$IMAGE:${{ steps.tag.outputs.tree }}" + docker push "$IMAGE:latest" + + # A cache hit on the tree tag says nothing about where `latest` points — a + # reverted Dockerfile or a build from another branch can leave it on + # different content. This runs only when the digests above actually + # disagree, so the common case costs nothing; the layers are already in + # the registry, so the push that follows uploads a manifest, not 7 GB. + - name: Repoint latest + if: ${{ steps.check.outputs.repoint == 'true' }} + run: | + set -e + docker pull "$IMAGE:${{ steps.tag.outputs.tree }}" + docker tag "$IMAGE:${{ steps.tag.outputs.tree }}" "$IMAGE:latest" + docker push "$IMAGE:latest" diff --git a/.gitea/workflows/build-and-test.yml b/.gitea/workflows/build-and-test.yml index 4eaa4b1..cf6cb29 100644 --- a/.gitea/workflows/build-and-test.yml +++ b/.gitea/workflows/build-and-test.yml @@ -11,6 +11,13 @@ on: branches: [main, master, develop] jobs: + # The Android job runs inside an image that this repo builds. Ensure it is in + # the registry before anything tries to pull it — see android-image.yml for + # why this is a job rather than a documented manual step. It is a no-op of a + # few seconds unless docker/android actually changed. + android-image: + uses: ./.gitea/workflows/android-image.yml + desktop: runs-on: linux/amd64 name: Desktop (Linux) @@ -68,6 +75,10 @@ jobs: android: runs-on: linux/amd64 name: Android (aarch64) + # Waits for the image build. Without this the pull races the push and the + # job dies with "manifest unknown" before its first step, which is the + # failure mode this ordering exists to remove. + needs: android-image container: image: gitea.tourolle.paris/dtourolle/darkroom-android:latest diff --git a/docker/android/README.md b/docker/android/README.md index b667803..d7a2bdf 100644 --- a/docker/android/README.md +++ b/docker/android/README.md @@ -25,6 +25,21 @@ image, so a break appears in one place rather than two. Prefers `podman`, falls back to `docker`. First run builds the image, which takes several minutes; after that it is cached. +## In CI + +`.gitea/workflows/android-image.yml` builds this image and pushes it to +`gitea.tourolle.paris/dtourolle/darkroom-android`, which the Android job then runs inside. It is +called on every push and finishes in seconds unless something here changed — the image is tagged +with the git tree hash of `docker/android/`, so a rebuild happens when and only when a file in this +directory does. + +Nothing needs pushing by hand. Editing the Dockerfile is the trigger; `latest` follows +automatically. To force a rebuild without a content change, run the workflow from the Gitea UI with +`force` set to `true`. + +The job runs on the host runner rather than in a container, because it needs the Docker daemon and +the runner host's cached registry credentials. + ## Pinned versions | Component | Version | Why this one |