Files
jellytau/docs/build/ci-operations.md
T
dtourolle 9c75e74ea3
🏗️ Build and Test JellyTau / Run Tests (push) Successful in 15m52s
🏗️ Build and Test JellyTau / Supply Chain (push) Failing after 29s
Publish Documentation / Build & publish docs to gitea-pages (push) Successful in 5m35s
Traceability Validation / Check Requirement Traces (push) Successful in 11s
Build & Release / Run Tests (push) Successful in 14m53s
🏗️ Build and Test JellyTau / Android Compile Check (push) Successful in 4m22s
Build & Release / Build Linux (push) Successful in 20m53s
Build & Release / Build Windows (push) Successful in 15m41s
Build & Release / Build Android (push) Successful in 30m46s
Build & Release / Create Release (push) Successful in 38s
fix(ci): give the builder image what linuxdeploy needs for the AppImage
The v0.10.0 release build failed in Build Linux after 16 minutes:

  failed to bundle project: xdg-open binary not found
  /usr/bin/xdg-open: No such file or directory

linuxdeploy embeds xdg-open into the AppImage and aborts the whole bundle
when it is absent. deb and rpm had already bundled fine; only AppImage
was affected.

This is the one failure tonight that building locally could not have
caught, and the reason is worth writing down: a developer machine is a
desktop and always has xdg-utils, so the AppImage builds there and fails
on a minimal server image. The asymmetry is the bug. Every other release
defect this evening was found by building locally first; this one needed
the runner.

xdg-utils, desktop-file-utils and zsync are added together rather than
one at a time. Each round trip costs an image rebuild plus a failed
release build, and those three are what linuxdeploy commonly reaches for
(xdg-open, desktop-file-validate, and zsync for delta updates).

Workflows move to jellytau-builder:2026.08.1, built and pushed with all
three verified present inside it before this commit.

ci-operations.md gains two things learned here: that an apt addition
invalidates the layer above the cargo-install steps, so it is a ~20 minute
rebuild rather than the ~2 minutes the trailing layer normally gives; and
that Tauri's AppImage bundler downloads linuxdeploy, AppRun and two plugin
scripts from GitHub during the build, so an AppImage build depends on
GitHub being reachable from the runner.
2026-08-22 02:52:32 +02:00

9.3 KiB

CI operations

How the pipeline is kept working: the builder image, the secrets it needs, the gates that must stay required, and the things that only a human with access to the Gitea instance can do.

CI is Gitea Actions (.gitea/workflows/) on gitea.tourolle.paris, not GitHub.

The workflows

Workflow Trigger What it protects
build-and-test.yml push/PR to master Frontend + Rust gates, Android compile check, supply chain
traceability-check.yml push/PR Requirement coverage ratchet, dangling IDs
build-release.yml tag v* Builds, signs, publishes, and writes the update manifest
publish-docs.yml push to master Docs site on the gitea-pages branch

🔴 CI installs no system tools

Every build, test and packaging tool lives in the Docker image the job runs in. Never add apt-get, rustup, sdkmanager, or a curl | tar -xz of a binary to a workflow step.

Fetching the project's own declared dependencies is not a toolchain install and is fine: bun install, cargo pulling crates from the lockfile, cargo deny fetching the RustSec advisory database. The distinction is tool versus data.

This rule has been broken twice, both times invisibly until something else failed. publish-docs.yml downloaded mdBook from GitHub releases into /usr/local/bin at job time — a hard dependency on GitHub's CDN being up whenever docs were published. Both mdBook and the supply-chain tools are in the image now.

The builder image

Dockerfile.buildergitea.tourolle.paris/dtourolle/jellytau-builder. It carries: the pinned Rust toolchain plus rustfmt/clippy and the Android, Windows-MSVC targets; bun and Node; the Android SDK/NDK and a local Gradle distribution; Linux desktop and packaging deps (WebKitGTK, libmpv, rpm, NSIS, cargo-xwin); and the tooling — cargo-deny, cargo-cyclonedx, mdbook.

Arch packages build in a separate Dockerfile.arch, because makepkg is Arch-specific.

Tags are pinned, and why

Workflows name an immutable dated tag (:2026.08), never :latest. While every job said :latest, rebuilding the image silently changed what every build compiled against — including a rebuild of an old release tag, which is the opposite of reproducible.

:latest is still pushed alongside, for local docker compose runs and manual pulls.

Date tags rather than per-commit SHA tags on purpose: the runner shares a 74 GB disk with two other projects, and SHA-tagged images accumulated there until it filled. Keep a couple of dated tags live and prune the rest.

Changing the image

The order matters — CI breaks if the workflow lands before the image exists.

A caveat learned the hard way: the trailing layer is only fast for cargo install tools. Adding an apt package invalidates the packaging layer, which sits above the cargo-xwin/cargo-deny installs, so those recompile too — a ~20 minute rebuild rather than ~2.

# 1. Edit Dockerfile.builder. Put new tools in the TRAILING layer: it exists so
#    a tool change is a ~2 min rebuild instead of ~15.
# 2. Build and push, tagged with the new month:
./scripts/build-builder-image.sh 2026.09
# 3. Repoint every workflow at the new tag, in the same commit as whatever
#    needed the new tool:
sed -i 's|jellytau-builder:2026.08|jellytau-builder:2026.09|g' .gitea/workflows/*.yml
# 4. Verify the tools are actually in it:
docker run --rm gitea.tourolle.paris/dtourolle/jellytau-builder:2026.09 \
  -c "cargo deny --version; mdbook --version"

🔴 The Rust version is pinned in two places that must agree: RUST_VERSION in Dockerfile.builder and channel in src-tauri/rust-toolchain.toml. If they drift, rustup downloads the pinned toolchain inside the job — a toolchain install in CI. Bump both, rebuild, push, then merge.

Tauri plugin versions are pinned in pairs

Every Tauri plugin exists twice: a Rust crate in src-tauri/Cargo.toml and an npm package in package.json. The Tauri CLI refuses to build when the two are on different minor versions — not a warning, a hard stop before compilation.

Both sides are therefore pinned exactly ("2.8.0", not "^2.8.0"). A caret range is what let them drift apart in the first place: bun add took the latest npm package while cargo held an older crate, and nothing noticed until a release build refused to start.

Nothing in build-and-test.yml runs tauri build — that happens only on a tag — so this class of breakage used to be invisible until release day. The Check Tauri plugin versions match step runs tauri info, which performs the same comparison without building.

To upgrade a plugin, move both sides together and re-run that step. Expect the Rust side to be the constraint: a newer plugin crate may pull a large transitive upgrade (bumping tauri-plugin-log to 2.9.0 also moved wry, wasm-bindgen, web-sys and webkit2gtk), which touches the webview and therefore video playback. That is a change to make deliberately, with a full build and a playback check — not one to slip into a release.

AppImage needs more than the Rust toolchain

linuxdeploy (which Tauri downloads at build time to assemble the AppImage) shells out to distro tools that a minimal server image does not have. It aborts the whole bundle on the first one missing:

failed to bundle project: xdg-open binary not found

The image therefore carries xdg-utils, desktop-file-utils and zsync. This is a class of failure that cannot be caught by building locally: a developer machine is a desktop and has all three, so the AppImage builds there and fails in CI. It cost one release build to find.

Tauri's AppImage bundler also downloads linuxdeploy, AppRun and two plugin scripts from GitHub during the build. That is Tauri's behaviour, not ours, but it means an AppImage build depends on GitHub being reachable from the runner.

Secrets

Managed with the tea CLI (tea actions secrets list) or the repo settings UI.

Secret Used by Notes
ANDROID_KEYSTORE_BASE64 release Base64 of the release keystore
ANDROID_KEYSTORE_PASSWORD release
ANDROID_KEY_ALIAS release
ANDROID_KEY_PASSWORD release
TAURI_SIGNING_PRIVATE_KEY release minisign key for the desktop updater
TAURI_SIGNING_PRIVATE_KEY_PASSWORD release
GITEA_TOKEN release, docs PAT; falls back to the auto-provided token

The updater keypair's public half is committed in src-tauri/tauri.conf.json — that one is meant to be public; it is what clients verify against. The private half exists in the Gitea secret and in the maintainer's local .env (which is gitignored) and at ~/.tauri/jellytau.key.

Losing the private key means losing the ability to ship updates to installed desktop clients, because they will only accept payloads signed by the key matching the public key they were built with. Recovering means generating a new pair, shipping a build carrying the new public key, and telling everyone on an older build to reinstall by hand. Back it up.

Required status checks

Gitea → repo Settings → Branches → protect master, requiring:

  • Run Tests
  • Android Compile Check
  • Supply Chain
  • the traceability job

Without branch protection, every gate in this document is advisory: a push straight to master lands whether or not CI is red. That is the state the repo was in for its whole history before this was set up.

The runner

One self-hosted runner, one ~74 GB disk shared with two other projects. It fills, and when it does the symptoms are misleading: cargo dying mid-link, docker refusing to pull, actions/cache quietly not saving — anything except an obvious out-of-space error. Check the disk first.

There is deliberately no scheduled job watching this. On a single-slot runner a daily job occupies the slot and pulls the builder image to run df, and df inside a container does not reliably describe the host's disk anyway — it would cost real build capacity to report a number that might be wrong. Check it by hand on the runner:

df -h /
docker system df -v

When it does fill:

docker image prune -a
docker volume prune -a     # the -a matters: without it, NAMED volumes are kept,
                           # which is exactly how this filled up unnoticed

Never cache src-tauri/target — it is ~16 GB, and caching it under several keys is what filled the disk at ~1.15 GB/day. The workflows cache only the cargo registry index and .crate tarballs; cargo re-extracts registry/src for free.

Release verification

The steps that catch a broken release before users do are in release-checklist.md — in particular the update path: latest.json must be live on the updater branch, both platform entries must carry a non-empty signature, and the previous release should be installed and asked to update to the new one.

Bus factor

The Gitea instance holds the canonical remote, the signing secrets, the container registry and the CI runner. It is not backed up as part of this repository, and nothing in this repository can restore it. That is the largest single risk to the project — larger than any gate in this document — and the backup lives outside it.