ci: run tests in a prebuilt container image
Python CI / test (3.10) (push) Successful in 1m21s
Python CI / test (3.11) (push) Successful in 1m7s
Python CI / test (3.12) (push) Successful in 1m9s
Python CI / test (3.13) (push) Successful in 1m23s

Matches the convention used by pyPhotoAlbum and the other projects here:
runs-on: linux/amd64 with a container image from the Gitea registry,
instead of setup-python plus an ad-hoc `pip install pytest pytest-cov
flake8 coverage-badge interrogate` on a self-hosted runner.

pyWebLayout is a library, so the image carries all four interpreters
pyproject.toml claims to support - 3.10, 3.11, 3.12, 3.13 - each in its
own venv at /opt/py<version> with every dependency pre-installed. The
matrix picks one per job. A CI run now downloads nothing, and coverage
widens from 3.10/3.12/3.13 to the full declared range.

Ubuntu marks its system Python externally-managed, so per-interpreter
venvs are used rather than --break-system-packages; that also keeps the
four dependency sets isolated.

Two defects in the existing workflow are fixed while rewriting it:

- pytest runs under continue-on-error so the badge steps still execute,
  but nothing afterwards checked its outcome - the job reported green on
  a red suite. An explicit gate now fails the job.
- Every matrix leg ran the badge steps and force-pushed the badges
  branch, so three jobs raced to publish. Badges and artifacts are now
  produced by the 3.13 leg only.

setuptools is pinned below 81 in the image: that release dropped
pkg_resources, which coverage-badge imports at startup, and without the
pin the badge step dies with ModuleNotFoundError. Found by running the
workflow's own commands in the image rather than assuming they work.

Also raises the test Flask server's readiness budget from 5s to 30s.
Making that check raise instead of silently falling through (737cf07)
turned runner load into a hard failure; it showed up as 17 spurious
errors in one containerised run and did not reproduce in three repeats.
The loop still exits as soon as the server answers.

Verified locally: image builds, and tests/ passes 916 on each of 3.10,
3.11, 3.12 and 3.13 inside it. The publishing leg was run end to end -
clean-install dependency check, pytest with coverage, both badges,
coverage summary at 81.5%.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-08 14:10:05 +02:00
co-authored by Claude Opus 5
parent 3761e00398
commit 745fc8687e
5 changed files with 252 additions and 112 deletions
+43
View File
@@ -0,0 +1,43 @@
# Dockerfile.ci copies nothing from the context, but keeping it small makes
# `docker build` fast and avoids shipping local state into the build.
# Virtual environments
venv/
.venv/
# Python cache and build output
__pycache__/
*.py[cod]
*$py.class
*.so
build/
dist/
*.egg-info/
*.egg
# Git
.git/
# Test/coverage output
.coverage
coverage.json
coverage.xml
htmlcov/
cov_info/
.pytest_cache/
.tox/
.mypy_cache/
# IDE
.idea/
.vscode/
*.swp
*.swo
.claude/
# Generated docs/images
docs/images/
# OS files
.DS_Store
Thumbs.db
+82 -96
View File
@@ -11,25 +11,47 @@ on:
jobs:
test:
runs-on: self-hosted
runs-on: linux/amd64
container:
# Built from Dockerfile.ci at the repo root. Carries Python 3.10-3.13,
# each in its own venv at /opt/py<version> with every dependency
# pre-installed, so a run downloads nothing.
image: gitea.tourolle.paris/dtourolle/pyweblayout-ci:latest
strategy:
matrix:
python-version: ['3.10', '3.12', '3.13']
python-version: ['3.10', '3.11', '3.12', '3.13']
fail-fast: false
env:
# pyWebLayout is a library: it is tested on every interpreter
# pyproject.toml's requires-python claims to support.
PYBIN: /opt/py${{ matrix.python-version }}/bin
# Badges and artifacts are published once, not once per matrix leg -
# four jobs racing to force-push the same branch is not a publish
# strategy. This leg is the one that publishes.
PUBLISH: ${{ matrix.python-version == '3.13' }}
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- name: Install project
run: |
# --no-deps: dependencies are baked into the image. If a new one is
# added to pyproject.toml, add it to Dockerfile.ci and rebuild;
# the check below is what catches forgetting to.
$PYBIN/pip install -e . --no-deps
$PYBIN/python -c "import pyWebLayout; print('pyWebLayout', pyWebLayout.__file__)"
- name: Verify declared dependencies are sufficient
if: env.PUBLISH == 'true'
run: |
# A clean venv with ONLY the declared runtime deps. If an import here
# fails, install_requires is incomplete and a real `pip install
# pyWebLayout` would fail the same way for a user.
python -m venv /tmp/clean-install
# A clean venv with ONLY the declared runtime deps, installed from
# the index rather than from the image. If an import here fails,
# pyproject.toml is incomplete and a real `pip install pyWebLayout`
# fails the same way for a user. This is the one step that is
# allowed to reach the network.
$PYBIN/python -m venv /tmp/clean-install
/tmp/clean-install/bin/pip install --upgrade pip
/tmp/clean-install/bin/pip install .
/tmp/clean-install/bin/python -c "
@@ -40,152 +62,116 @@ jobs:
print('clean install imports OK')
"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
# Install package in development mode, with the declared dev extra.
# Test dependencies belong in setup.cfg, not in an ad-hoc pip line.
pip install -e '.[dev]'
- name: Download initial failed badges
run: |
echo "Downloading initial failed badges..."
# Create cov_info directory first
mkdir -p cov_info
# Download failed badges as defaults
curl -o cov_info/coverage.svg "https://img.shields.io/badge/coverage-failed-red.svg"
curl -o cov_info/coverage-docs.svg "https://img.shields.io/badge/docs-failed-red.svg"
echo "Initial failed badges created:"
ls -la cov_info/coverage*.svg
- name: Run tests with pytest
id: pytest
continue-on-error: true
run: |
# Run tests with coverage
python -m pytest tests/ -v --cov=pyWebLayout --cov-report=term-missing --cov-report=json --cov-report=html --cov-report=xml
$PYBIN/python -m pytest tests/ -v \
--cov=pyWebLayout \
--cov-report=term-missing \
--cov-report=json \
--cov-report=html \
--cov-report=xml
- name: Check documentation coverage
id: docs
continue-on-error: true
run: |
# Generate documentation coverage report
interrogate -v --ignore-init-method --ignore-init-module --ignore-magic --ignore-private --ignore-property-decorators --ignore-semiprivate --fail-under=80 pyWebLayout/
$PYBIN/interrogate -v \
--ignore-init-method --ignore-init-module --ignore-magic \
--ignore-private --ignore-property-decorators --ignore-semiprivate \
--fail-under=80 pyWebLayout/
- name: Lint with flake8
run: |
# Stop the build if there are Python syntax errors or undefined names
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
$PYBIN/flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# Exit-zero treats all errors as warnings
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
$PYBIN/flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Create coverage info directory
if: always()
- name: Fail the job if tests failed
if: steps.pytest.outcome != 'success'
run: |
# pytest runs with continue-on-error so the badge steps below still
# execute; without this the job would report green on a red suite.
echo "::error::pytest failed on Python ${{ matrix.python-version }}"
exit 1
# ------------------------------------------------------------------
# Badges and artifacts - publishing leg only
# ------------------------------------------------------------------
- name: Prepare badge directory
if: always() && env.PUBLISH == 'true'
run: |
mkdir -p cov_info
echo "Created cov_info directory for coverage data"
# Default to failed badges; the steps below overwrite them on success
curl -o cov_info/coverage.svg "https://img.shields.io/badge/coverage-failed-red.svg"
curl -o cov_info/coverage-docs.svg "https://img.shields.io/badge/docs-failed-red.svg"
- name: Update test coverage badge on success
if: steps.pytest.outcome == 'success' && always()
if: always() && env.PUBLISH == 'true' && steps.pytest.outcome == 'success'
run: |
echo "Tests passed! Generating successful coverage badge..."
if [ -f coverage.json ]; then
coverage-badge -o cov_info/coverage.svg -f
echo "✅ Test coverage badge updated with actual results"
$PYBIN/coverage-badge -o cov_info/coverage.svg -f
echo "✅ Test coverage badge updated"
else
echo "⚠️ No coverage.json found, keeping failed badge"
fi
- name: Update docs coverage badge on success
if: steps.docs.outcome == 'success' && always()
if: always() && env.PUBLISH == 'true' && steps.docs.outcome == 'success'
run: |
echo "Docs check passed! Generating successful docs badge..."
# Remove existing badge first to avoid overwrite error
rm -f cov_info/coverage-docs.svg
interrogate --generate-badge cov_info/coverage-docs.svg pyWebLayout/
echo "✅ Docs coverage badge updated with actual results"
$PYBIN/interrogate --generate-badge cov_info/coverage-docs.svg pyWebLayout/
echo "✅ Docs coverage badge updated"
- name: Generate coverage reports
if: steps.pytest.outcome == 'success'
if: env.PUBLISH == 'true' && steps.pytest.outcome == 'success'
run: |
# Generate coverage summary for README
python -c "
import json
import os
# Read coverage data
$PYBIN/python -c "
import json, os
if os.path.exists('coverage.json'):
with open('coverage.json', 'r') as f:
coverage_data = json.load(f)
total_coverage = round(coverage_data['totals']['percent_covered'], 1)
# Create coverage summary file in cov_info directory
with open('coverage.json') as f:
data = json.load(f)
total = round(data['totals']['percent_covered'], 1)
with open('cov_info/coverage-summary.txt', 'w') as f:
f.write(f'{total_coverage}%')
print(f'Test Coverage: {total_coverage}%')
covered_lines = coverage_data['totals']['covered_lines']
total_lines = coverage_data['totals']['num_statements']
print(f'Lines Covered: {covered_lines}/{total_lines}')
f.write(f'{total}%')
print(f\"Test Coverage: {total}%\")
print(f\"Lines Covered: {data['totals']['covered_lines']}/{data['totals']['num_statements']}\")
else:
print('No coverage data found')
"
# Copy other coverage files to cov_info
if [ -f coverage.json ]; then cp coverage.json cov_info/; fi
if [ -f coverage.xml ]; then cp coverage.xml cov_info/; fi
if [ -d htmlcov ]; then cp -r htmlcov cov_info/; fi
- name: Final badge status
if: always()
if: always() && env.PUBLISH == 'true'
run: |
echo "=== FINAL BADGE STATUS ==="
echo "Test outcome: ${{ steps.pytest.outcome }}"
echo "Docs outcome: ${{ steps.docs.outcome }}"
if [ -f cov_info/coverage.svg ]; then
echo "✅ Test coverage badge: $(ls -lh cov_info/coverage.svg)"
else
echo "❌ Test coverage badge: MISSING"
fi
if [ -f cov_info/coverage-docs.svg ]; then
echo "✅ Docs coverage badge: $(ls -lh cov_info/coverage-docs.svg)"
else
echo "❌ Docs coverage badge: MISSING"
fi
echo "Coverage info directory contents:"
ls -la cov_info/ 2>/dev/null || echo "No cov_info directory found"
ls -la cov_info/ 2>/dev/null || echo "No cov_info directory"
- name: Upload coverage artifacts
if: always() && env.PUBLISH == 'true'
uses: actions/upload-artifact@v3
with:
name: coverage-reports
path: |
cov_info/
path: cov_info/
- name: Commit badges to badges branch
if: github.ref == 'refs/heads/master'
if: env.PUBLISH == 'true' && github.ref == 'refs/heads/master'
run: |
git config --local user.email "action@gitea.local"
git config --local user.name "Gitea Action"
# Set the remote URL to use the token
git remote set-url origin https://${{ secrets.PUSH_TOKEN }}@gitea.tourolle.paris/dtourolle/pyWebLayout.git
# Create a new orphan branch for badges (this discards any existing badges branch)
# Orphan branch holding only the badges, force-pushed each time
git checkout --orphan badges
# Remove all files except cov_info
find . -maxdepth 1 -not -name '.git' -not -name 'cov_info' -exec rm -rf {} + 2>/dev/null || true
# Add only the coverage info directory
git add -f cov_info/
# Always commit (force overwrite)
echo "Force updating badges branch with new coverage data..."
git commit -m "Update coverage badges [skip ci]"
git push -f origin badges
+80
View File
@@ -0,0 +1,80 @@
# CI test image for pyWebLayout
# Build: docker build -f Dockerfile.ci -t gitea.tourolle.paris/dtourolle/pyweblayout-ci:latest .
# Push: docker push gitea.tourolle.paris/dtourolle/pyweblayout-ci:latest
#
# pyWebLayout is a library, so CI tests every interpreter pyproject.toml claims
# to support rather than just one. All four are in this image and the workflow
# matrix picks one per job; dependencies are pre-installed into each, so a CI
# run downloads nothing.
FROM ubuntu:24.04
ENV DEBIAN_FRONTEND=noninteractive
# deadsnakes carries the Python versions Ubuntu 24.04 does not ship
RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates \
gnupg \
software-properties-common \
&& add-apt-repository -y ppa:deadsnakes/ppa \
&& apt-get update && apt-get install -y --no-install-recommends \
# Interpreters. 3.12 is Ubuntu 24.04's own; the rest come from deadsnakes.
python3.10 python3.10-venv \
python3.11 python3.11-venv \
python3.12 python3.12-venv \
python3.13 python3.13-venv \
# Pillow needs these at runtime for font rasterisation and image IO
libfreetype6 \
libjpeg-turbo8 \
libopenjp2-7 \
libtiff6 \
zlib1g \
# Used by the workflow itself
curl \
git \
nodejs \
&& rm -rf /var/lib/apt/lists/*
# One venv per interpreter at a predictable path, /opt/py<version>. Ubuntu marks
# its system Python externally-managed, so installing into venvs sidesteps that
# without --break-system-packages, and keeps the four dependency sets isolated.
#
# The package list is written once so versions cannot drift between
# interpreters. It mirrors pyproject.toml's runtime deps plus the test and dev
# extras; keep the two in step.
#
# Deliberately NOT installed: pyWebLayout itself. The workflow installs the
# checkout with --no-deps, so a job always tests the code under review.
#
# setuptools is pinned below 81 because that release dropped pkg_resources,
# which coverage-badge still imports at startup. Without the pin the badge step
# dies with ModuleNotFoundError. Revisit when coverage-badge stops using it.
RUN for v in 3.10 3.11 3.12 3.13; do \
python$v -m venv /opt/py$v && \
/opt/py$v/bin/pip install --no-cache-dir --upgrade pip wheel && \
/opt/py$v/bin/pip install --no-cache-dir --upgrade "setuptools<81" && \
/opt/py$v/bin/pip install --no-cache-dir \
Pillow \
numpy \
pyphen \
beautifulsoup4 \
lxml \
pytest \
pytest-cov \
flask \
werkzeug \
ebooklib \
requests \
flake8 \
coverage-badge \
interrogate \
; \
done
# Fail the build rather than ship an image whose dependencies do not import
RUN for v in 3.10 3.11 3.12 3.13; do \
echo "--- python$v ---" && \
/opt/py$v/bin/python -c \
"import sys, PIL, numpy, pyphen, bs4, pytest, flask, ebooklib, requests; \
print(sys.version.split()[0], 'deps OK')" \
; \
done
+27
View File
@@ -231,6 +231,33 @@ current = manager.get_font_family()
- **[pyWebLayout/layout/README_EREADER_API.md](pyWebLayout/layout/README_EREADER_API.md)** - EbookReader API reference
- **API Reference** - See docstrings in source code
## Continuous integration
CI runs in a prebuilt container image rather than installing dependencies per
job. The image carries Python 3.10, 3.11, 3.12 and 3.13, each in its own venv at
`/opt/py<version>` with every dependency installed, so a run downloads nothing
and the test matrix covers the whole range `pyproject.toml` claims to support.
Rebuild and push the image whenever `Dockerfile.ci` changes — most often
because a dependency was added to `pyproject.toml`:
```bash
docker build -f Dockerfile.ci -t gitea.tourolle.paris/dtourolle/pyweblayout-ci:latest .
docker push gitea.tourolle.paris/dtourolle/pyweblayout-ci:latest
```
To reproduce a CI job locally:
```bash
docker run --rm -v "$PWD:/src:ro" gitea.tourolle.paris/dtourolle/pyweblayout-ci:latest bash -c '
mkdir -p /work && cp -a /src/. /work/ && cd /work && rm -rf venv .git
/opt/py3.13/bin/pip install -e . --no-deps -q
/opt/py3.13/bin/python -m pytest tests/ -q'
```
The workflow is [.gitea/workflows/ci.yml](.gitea/workflows/ci.yml). Badges and
coverage artifacts are published from the 3.13 leg only.
## License
MIT License
+6 -2
View File
@@ -372,8 +372,12 @@ class TestImagePIL(unittest.TestCase):
cls.flask_thread = threading.Thread(target=cls.flask_server.serve_forever, daemon=True)
cls.flask_thread.start()
# Wait for server to be ready with health check
max_wait = 5 # Maximum 5 seconds
# Wait for server to be ready with health check.
# Generous, because this now raises rather than falling through
# silently: on a loaded CI runner the accept loop can take several
# seconds to get scheduled, and a spurious failure here is worse than
# a slow one. The loop exits as soon as the server answers.
max_wait = 30
wait_interval = 0.1 # Check every 100ms
elapsed = 0