Compare commits

...
3 Commits
Author SHA1 Message Date
dtourolleandClaude Opus 5 cfc4230713 fix(ereader): restore cover position when navigating back to the cover
Python CI / test (3.10) (push) Successful in 1m2s
Python CI / test (3.11) (push) Successful in 1m5s
Python CI / test (3.12) (push) Successful in 1m5s
Python CI / test (3.13) (push) Successful in 1m43s
previous_page() set the on-cover flag but left current_position at the
first content block, so "showing the cover" had two different internal
representations depending on how you got there: block 0 on a fresh load,
block 1 after going forward and back.

current_position is what gets persisted, so closing the book while on the
cover reopened it past the cover, silently losing it.

Reset current_position to block 0 when returning to the cover, matching
where a fresh load sits.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 22:26:38 +02:00
dtourolleandClaude Opus 5 0bb34a4a32 perf(layout): cut page layout time by ~40%
Python CI / test (3.10) (push) Successful in 1m4s
Python CI / test (3.11) (push) Successful in 1m8s
Python CI / test (3.12) (push) Successful in 1m5s
Python CI / test (3.13) (push) Successful in 1m24s
Layout was dominated by work that was either repeated per word or thrown
away. Rendered output is unchanged: every page hashed byte-for-byte
identical across 3 page sizes, 2 font scales, 2 font families, and 4
alignments x 4 column widths chosen to force heavy hyphenation.

  layout, 600x800, 40 pages     ~200ms -> ~124ms
  layout, 1404x1872, 11 pages   ~168ms -> ~111ms

Measured, not guessed. The reflex fix - swapping list comprehensions for
generators - measures slower here (602ns vs 425ns for the width sum), so
those are left alone.

- Line asked its font for the advance width of a space on every
  construction. FreeTypeFont.getlength(" ") costs ~18us, two orders of
  magnitude more than getmetrics(), and it landed once per line. Memoise
  per font object.

- Line.add_word was quadratic in the words on a line. Each candidate word
  re-summed every width and rebuilt the whole per-gap spacing list, when
  fitting only ever reads the first gap. Gap spacings are now a plan
  materialised on demand (only render() reads the list), and widths come
  from a prefix sum. The prefix list, rather than one accumulator, is what
  keeps this exact: widths are floats, (total + w) - w need not give back
  total, and one ulp flips an overflow decision on a line that ends flush.

- RenderingPosition.copy/__eq__/__hash__ all went through
  dataclasses.asdict, a deep recursive walk, over 8 immutable scalars.

- paragraph_layouter built a Text per line purely to discard it.

- AbstractStyle.__hash__ rebuilt a 15-tuple containing 5 enums on every
  dict lookup; memoised on the frozen instance (1037ns -> 160ns).

- The pyphen dictionary wrapper was rebuilt for every word that overflowed
  its line, and word extraction stripped before splitting and tested each
  split result for emptiness, neither of which str.split() needs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 14:27:20 +02:00
dtourolleandClaude Opus 5 745fc8687e ci: run tests in a prebuilt container image
Python CI / test (3.10) (push) Successful in 1m21s
Python CI / test (3.11) (push) Successful in 1m7s
Python CI / test (3.12) (push) Successful in 1m9s
Python CI / test (3.13) (push) Successful in 1m23s
Matches the convention used by pyPhotoAlbum and the other projects here:
runs-on: linux/amd64 with a container image from the Gitea registry,
instead of setup-python plus an ad-hoc `pip install pytest pytest-cov
flake8 coverage-badge interrogate` on a self-hosted runner.

pyWebLayout is a library, so the image carries all four interpreters
pyproject.toml claims to support - 3.10, 3.11, 3.12, 3.13 - each in its
own venv at /opt/py<version> with every dependency pre-installed. The
matrix picks one per job. A CI run now downloads nothing, and coverage
widens from 3.10/3.12/3.13 to the full declared range.

Ubuntu marks its system Python externally-managed, so per-interpreter
venvs are used rather than --break-system-packages; that also keeps the
four dependency sets isolated.

Two defects in the existing workflow are fixed while rewriting it:

- pytest runs under continue-on-error so the badge steps still execute,
  but nothing afterwards checked its outcome - the job reported green on
  a red suite. An explicit gate now fails the job.
- Every matrix leg ran the badge steps and force-pushed the badges
  branch, so three jobs raced to publish. Badges and artifacts are now
  produced by the 3.13 leg only.

setuptools is pinned below 81 in the image: that release dropped
pkg_resources, which coverage-badge imports at startup, and without the
pin the badge step dies with ModuleNotFoundError. Found by running the
workflow's own commands in the image rather than assuming they work.

Also raises the test Flask server's readiness budget from 5s to 30s.
Making that check raise instead of silently falling through (737cf07)
turned runner load into a hard failure; it showed up as 17 spurious
errors in one containerised run and did not reproduce in three repeats.
The loop still exits as soon as the server answers.

Verified locally: image builds, and tests/ passes 916 on each of 3.10,
3.11, 3.12 and 3.13 inside it. The publishing leg was run end to end -
clean-install dependency check, pytest with coverage, both badges,
coverage summary at 81.5%.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 14:10:05 +02:00
12 changed files with 480 additions and 184 deletions
+43
View File
@@ -0,0 +1,43 @@
# Dockerfile.ci copies nothing from the context, but keeping it small makes
# `docker build` fast and avoids shipping local state into the build.
# Virtual environments
venv/
.venv/
# Python cache and build output
__pycache__/
*.py[cod]
*$py.class
*.so
build/
dist/
*.egg-info/
*.egg
# Git
.git/
# Test/coverage output
.coverage
coverage.json
coverage.xml
htmlcov/
cov_info/
.pytest_cache/
.tox/
.mypy_cache/
# IDE
.idea/
.vscode/
*.swp
*.swo
.claude/
# Generated docs/images
docs/images/
# OS files
.DS_Store
Thumbs.db
+82 -96
View File
@@ -11,25 +11,47 @@ on:
jobs: jobs:
test: test:
runs-on: self-hosted runs-on: linux/amd64
container:
# Built from Dockerfile.ci at the repo root. Carries Python 3.10-3.13,
# each in its own venv at /opt/py<version> with every dependency
# pre-installed, so a run downloads nothing.
image: gitea.tourolle.paris/dtourolle/pyweblayout-ci:latest
strategy: strategy:
matrix: matrix:
python-version: ['3.10', '3.12', '3.13'] python-version: ['3.10', '3.11', '3.12', '3.13']
fail-fast: false
env:
# pyWebLayout is a library: it is tested on every interpreter
# pyproject.toml's requires-python claims to support.
PYBIN: /opt/py${{ matrix.python-version }}/bin
# Badges and artifacts are published once, not once per matrix leg -
# four jobs racing to force-push the same branch is not a publish
# strategy. This leg is the one that publishes.
PUBLISH: ${{ matrix.python-version == '3.13' }}
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@v4 uses: actions/checkout@v4
- name: Set up Python - name: Install project
uses: actions/setup-python@v4 run: |
with: # --no-deps: dependencies are baked into the image. If a new one is
python-version: ${{ matrix.python-version }} # added to pyproject.toml, add it to Dockerfile.ci and rebuild;
# the check below is what catches forgetting to.
$PYBIN/pip install -e . --no-deps
$PYBIN/python -c "import pyWebLayout; print('pyWebLayout', pyWebLayout.__file__)"
- name: Verify declared dependencies are sufficient - name: Verify declared dependencies are sufficient
if: env.PUBLISH == 'true'
run: | run: |
# A clean venv with ONLY the declared runtime deps. If an import here # A clean venv with ONLY the declared runtime deps, installed from
# fails, install_requires is incomplete and a real `pip install # the index rather than from the image. If an import here fails,
# pyWebLayout` would fail the same way for a user. # pyproject.toml is incomplete and a real `pip install pyWebLayout`
python -m venv /tmp/clean-install # fails the same way for a user. This is the one step that is
# allowed to reach the network.
$PYBIN/python -m venv /tmp/clean-install
/tmp/clean-install/bin/pip install --upgrade pip /tmp/clean-install/bin/pip install --upgrade pip
/tmp/clean-install/bin/pip install . /tmp/clean-install/bin/pip install .
/tmp/clean-install/bin/python -c " /tmp/clean-install/bin/python -c "
@@ -40,152 +62,116 @@ jobs:
print('clean install imports OK') print('clean install imports OK')
" "
- name: Install dependencies
run: |
python -m pip install --upgrade pip
# Install package in development mode, with the declared dev extra.
# Test dependencies belong in setup.cfg, not in an ad-hoc pip line.
pip install -e '.[dev]'
- name: Download initial failed badges
run: |
echo "Downloading initial failed badges..."
# Create cov_info directory first
mkdir -p cov_info
# Download failed badges as defaults
curl -o cov_info/coverage.svg "https://img.shields.io/badge/coverage-failed-red.svg"
curl -o cov_info/coverage-docs.svg "https://img.shields.io/badge/docs-failed-red.svg"
echo "Initial failed badges created:"
ls -la cov_info/coverage*.svg
- name: Run tests with pytest - name: Run tests with pytest
id: pytest id: pytest
continue-on-error: true continue-on-error: true
run: | run: |
# Run tests with coverage $PYBIN/python -m pytest tests/ -v \
python -m pytest tests/ -v --cov=pyWebLayout --cov-report=term-missing --cov-report=json --cov-report=html --cov-report=xml --cov=pyWebLayout \
--cov-report=term-missing \
--cov-report=json \
--cov-report=html \
--cov-report=xml
- name: Check documentation coverage - name: Check documentation coverage
id: docs id: docs
continue-on-error: true continue-on-error: true
run: | run: |
# Generate documentation coverage report $PYBIN/interrogate -v \
interrogate -v --ignore-init-method --ignore-init-module --ignore-magic --ignore-private --ignore-property-decorators --ignore-semiprivate --fail-under=80 pyWebLayout/ --ignore-init-method --ignore-init-module --ignore-magic \
--ignore-private --ignore-property-decorators --ignore-semiprivate \
--fail-under=80 pyWebLayout/
- name: Lint with flake8 - name: Lint with flake8
run: | run: |
# Stop the build if there are Python syntax errors or undefined names # Stop the build if there are Python syntax errors or undefined names
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics $PYBIN/flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# Exit-zero treats all errors as warnings # Exit-zero treats all errors as warnings
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics $PYBIN/flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Create coverage info directory - name: Fail the job if tests failed
if: always() if: steps.pytest.outcome != 'success'
run: |
# pytest runs with continue-on-error so the badge steps below still
# execute; without this the job would report green on a red suite.
echo "::error::pytest failed on Python ${{ matrix.python-version }}"
exit 1
# ------------------------------------------------------------------
# Badges and artifacts - publishing leg only
# ------------------------------------------------------------------
- name: Prepare badge directory
if: always() && env.PUBLISH == 'true'
run: | run: |
mkdir -p cov_info mkdir -p cov_info
echo "Created cov_info directory for coverage data" # Default to failed badges; the steps below overwrite them on success
curl -o cov_info/coverage.svg "https://img.shields.io/badge/coverage-failed-red.svg"
curl -o cov_info/coverage-docs.svg "https://img.shields.io/badge/docs-failed-red.svg"
- name: Update test coverage badge on success - name: Update test coverage badge on success
if: steps.pytest.outcome == 'success' && always() if: always() && env.PUBLISH == 'true' && steps.pytest.outcome == 'success'
run: | run: |
echo "Tests passed! Generating successful coverage badge..."
if [ -f coverage.json ]; then if [ -f coverage.json ]; then
coverage-badge -o cov_info/coverage.svg -f $PYBIN/coverage-badge -o cov_info/coverage.svg -f
echo "✅ Test coverage badge updated with actual results" echo "✅ Test coverage badge updated"
else else
echo "⚠️ No coverage.json found, keeping failed badge" echo "⚠️ No coverage.json found, keeping failed badge"
fi fi
- name: Update docs coverage badge on success - name: Update docs coverage badge on success
if: steps.docs.outcome == 'success' && always() if: always() && env.PUBLISH == 'true' && steps.docs.outcome == 'success'
run: | run: |
echo "Docs check passed! Generating successful docs badge..."
# Remove existing badge first to avoid overwrite error
rm -f cov_info/coverage-docs.svg rm -f cov_info/coverage-docs.svg
interrogate --generate-badge cov_info/coverage-docs.svg pyWebLayout/ $PYBIN/interrogate --generate-badge cov_info/coverage-docs.svg pyWebLayout/
echo "✅ Docs coverage badge updated with actual results" echo "✅ Docs coverage badge updated"
- name: Generate coverage reports - name: Generate coverage reports
if: steps.pytest.outcome == 'success' if: env.PUBLISH == 'true' && steps.pytest.outcome == 'success'
run: | run: |
# Generate coverage summary for README $PYBIN/python -c "
python -c " import json, os
import json
import os
# Read coverage data
if os.path.exists('coverage.json'): if os.path.exists('coverage.json'):
with open('coverage.json', 'r') as f: with open('coverage.json') as f:
coverage_data = json.load(f) data = json.load(f)
total_coverage = round(coverage_data['totals']['percent_covered'], 1) total = round(data['totals']['percent_covered'], 1)
# Create coverage summary file in cov_info directory
with open('cov_info/coverage-summary.txt', 'w') as f: with open('cov_info/coverage-summary.txt', 'w') as f:
f.write(f'{total_coverage}%') f.write(f'{total}%')
print(f'Test Coverage: {total_coverage}%') print(f\"Test Coverage: {total}%\")
covered_lines = coverage_data['totals']['covered_lines'] print(f\"Lines Covered: {data['totals']['covered_lines']}/{data['totals']['num_statements']}\")
total_lines = coverage_data['totals']['num_statements']
print(f'Lines Covered: {covered_lines}/{total_lines}')
else: else:
print('No coverage data found') print('No coverage data found')
" "
# Copy other coverage files to cov_info
if [ -f coverage.json ]; then cp coverage.json cov_info/; fi if [ -f coverage.json ]; then cp coverage.json cov_info/; fi
if [ -f coverage.xml ]; then cp coverage.xml cov_info/; fi if [ -f coverage.xml ]; then cp coverage.xml cov_info/; fi
if [ -d htmlcov ]; then cp -r htmlcov cov_info/; fi if [ -d htmlcov ]; then cp -r htmlcov cov_info/; fi
- name: Final badge status - name: Final badge status
if: always() if: always() && env.PUBLISH == 'true'
run: | run: |
echo "=== FINAL BADGE STATUS ===" echo "=== FINAL BADGE STATUS ==="
echo "Test outcome: ${{ steps.pytest.outcome }}" echo "Test outcome: ${{ steps.pytest.outcome }}"
echo "Docs outcome: ${{ steps.docs.outcome }}" echo "Docs outcome: ${{ steps.docs.outcome }}"
ls -la cov_info/ 2>/dev/null || echo "No cov_info directory"
if [ -f cov_info/coverage.svg ]; then
echo "✅ Test coverage badge: $(ls -lh cov_info/coverage.svg)"
else
echo "❌ Test coverage badge: MISSING"
fi
if [ -f cov_info/coverage-docs.svg ]; then
echo "✅ Docs coverage badge: $(ls -lh cov_info/coverage-docs.svg)"
else
echo "❌ Docs coverage badge: MISSING"
fi
echo "Coverage info directory contents:"
ls -la cov_info/ 2>/dev/null || echo "No cov_info directory found"
- name: Upload coverage artifacts - name: Upload coverage artifacts
if: always() && env.PUBLISH == 'true'
uses: actions/upload-artifact@v3 uses: actions/upload-artifact@v3
with: with:
name: coverage-reports name: coverage-reports
path: | path: cov_info/
cov_info/
- name: Commit badges to badges branch - name: Commit badges to badges branch
if: github.ref == 'refs/heads/master' if: env.PUBLISH == 'true' && github.ref == 'refs/heads/master'
run: | run: |
git config --local user.email "action@gitea.local" git config --local user.email "action@gitea.local"
git config --local user.name "Gitea Action" git config --local user.name "Gitea Action"
# Set the remote URL to use the token
git remote set-url origin https://${{ secrets.PUSH_TOKEN }}@gitea.tourolle.paris/dtourolle/pyWebLayout.git git remote set-url origin https://${{ secrets.PUSH_TOKEN }}@gitea.tourolle.paris/dtourolle/pyWebLayout.git
# Create a new orphan branch for badges (this discards any existing badges branch) # Orphan branch holding only the badges, force-pushed each time
git checkout --orphan badges git checkout --orphan badges
# Remove all files except cov_info
find . -maxdepth 1 -not -name '.git' -not -name 'cov_info' -exec rm -rf {} + 2>/dev/null || true find . -maxdepth 1 -not -name '.git' -not -name 'cov_info' -exec rm -rf {} + 2>/dev/null || true
# Add only the coverage info directory
git add -f cov_info/ git add -f cov_info/
# Always commit (force overwrite)
echo "Force updating badges branch with new coverage data..."
git commit -m "Update coverage badges [skip ci]" git commit -m "Update coverage badges [skip ci]"
git push -f origin badges git push -f origin badges
+80
View File
@@ -0,0 +1,80 @@
# CI test image for pyWebLayout
# Build: docker build -f Dockerfile.ci -t gitea.tourolle.paris/dtourolle/pyweblayout-ci:latest .
# Push: docker push gitea.tourolle.paris/dtourolle/pyweblayout-ci:latest
#
# pyWebLayout is a library, so CI tests every interpreter pyproject.toml claims
# to support rather than just one. All four are in this image and the workflow
# matrix picks one per job; dependencies are pre-installed into each, so a CI
# run downloads nothing.
FROM ubuntu:24.04
ENV DEBIAN_FRONTEND=noninteractive
# deadsnakes carries the Python versions Ubuntu 24.04 does not ship
RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates \
gnupg \
software-properties-common \
&& add-apt-repository -y ppa:deadsnakes/ppa \
&& apt-get update && apt-get install -y --no-install-recommends \
# Interpreters. 3.12 is Ubuntu 24.04's own; the rest come from deadsnakes.
python3.10 python3.10-venv \
python3.11 python3.11-venv \
python3.12 python3.12-venv \
python3.13 python3.13-venv \
# Pillow needs these at runtime for font rasterisation and image IO
libfreetype6 \
libjpeg-turbo8 \
libopenjp2-7 \
libtiff6 \
zlib1g \
# Used by the workflow itself
curl \
git \
nodejs \
&& rm -rf /var/lib/apt/lists/*
# One venv per interpreter at a predictable path, /opt/py<version>. Ubuntu marks
# its system Python externally-managed, so installing into venvs sidesteps that
# without --break-system-packages, and keeps the four dependency sets isolated.
#
# The package list is written once so versions cannot drift between
# interpreters. It mirrors pyproject.toml's runtime deps plus the test and dev
# extras; keep the two in step.
#
# Deliberately NOT installed: pyWebLayout itself. The workflow installs the
# checkout with --no-deps, so a job always tests the code under review.
#
# setuptools is pinned below 81 because that release dropped pkg_resources,
# which coverage-badge still imports at startup. Without the pin the badge step
# dies with ModuleNotFoundError. Revisit when coverage-badge stops using it.
RUN for v in 3.10 3.11 3.12 3.13; do \
python$v -m venv /opt/py$v && \
/opt/py$v/bin/pip install --no-cache-dir --upgrade pip wheel && \
/opt/py$v/bin/pip install --no-cache-dir --upgrade "setuptools<81" && \
/opt/py$v/bin/pip install --no-cache-dir \
Pillow \
numpy \
pyphen \
beautifulsoup4 \
lxml \
pytest \
pytest-cov \
flask \
werkzeug \
ebooklib \
requests \
flake8 \
coverage-badge \
interrogate \
; \
done
# Fail the build rather than ship an image whose dependencies do not import
RUN for v in 3.10 3.11 3.12 3.13; do \
echo "--- python$v ---" && \
/opt/py$v/bin/python -c \
"import sys, PIL, numpy, pyphen, bs4, pytest, flask, ebooklib, requests; \
print(sys.version.split()[0], 'deps OK')" \
; \
done
+27
View File
@@ -231,6 +231,33 @@ current = manager.get_font_family()
- **[pyWebLayout/layout/README_EREADER_API.md](pyWebLayout/layout/README_EREADER_API.md)** - EbookReader API reference - **[pyWebLayout/layout/README_EREADER_API.md](pyWebLayout/layout/README_EREADER_API.md)** - EbookReader API reference
- **API Reference** - See docstrings in source code - **API Reference** - See docstrings in source code
## Continuous integration
CI runs in a prebuilt container image rather than installing dependencies per
job. The image carries Python 3.10, 3.11, 3.12 and 3.13, each in its own venv at
`/opt/py<version>` with every dependency installed, so a run downloads nothing
and the test matrix covers the whole range `pyproject.toml` claims to support.
Rebuild and push the image whenever `Dockerfile.ci` changes — most often
because a dependency was added to `pyproject.toml`:
```bash
docker build -f Dockerfile.ci -t gitea.tourolle.paris/dtourolle/pyweblayout-ci:latest .
docker push gitea.tourolle.paris/dtourolle/pyweblayout-ci:latest
```
To reproduce a CI job locally:
```bash
docker run --rm -v "$PWD:/src:ro" gitea.tourolle.paris/dtourolle/pyweblayout-ci:latest bash -c '
mkdir -p /work && cp -a /src/. /work/ && cd /work && rm -rf venv .git
/opt/py3.13/bin/pip install -e . --no-deps -q
/opt/py3.13/bin/python -m pytest tests/ -q'
```
The workflow is [.gitea/workflows/ci.yml](.gitea/workflows/ci.yml). Badges and
coverage artifacts are published from the 3.13 leg only.
## License ## License
MIT License MIT License
+14 -2
View File
@@ -3,12 +3,25 @@ from pyWebLayout.core import Hierarchical
from pyWebLayout.style import Font from pyWebLayout.style import Font
from pyWebLayout.style.abstract_style import AbstractStyle from pyWebLayout.style.abstract_style import AbstractStyle
from typing import Tuple, Union, List, Optional, Dict, Any, Callable from typing import Tuple, Union, List, Optional, Dict, Any, Callable
from functools import lru_cache
import pyphen import pyphen
# Import LinkType for type hints (imported at module level to avoid F821 linting error) # Import LinkType for type hints (imported at module level to avoid F821 linting error)
from pyWebLayout.abstract.functional import LinkType from pyWebLayout.abstract.functional import LinkType
@lru_cache(maxsize=16)
def _hyphen_dict(language: Optional[str]) -> pyphen.Pyphen:
"""
The pyphen dictionary for a language, reused across words.
Pyphen caches the parsed dictionary file itself, but rebuilding the wrapper
per word still costs about 40% of a hyphenation call, and hyphenation is
attempted for every word that overflows its line.
"""
return pyphen.Pyphen(lang=language)
class Word: class Word:
""" """
An abstract representation of a word in a document. Words can be split across An abstract representation of a word in a document. Words can be split across
@@ -186,8 +199,7 @@ class Word:
bool: True if the word was hyphenated, False otherwise. bool: True if the word was hyphenated, False otherwise.
""" """
dic = pyphen.Pyphen(lang=self._style.language) return list(_hyphen_dict(self._style.language).iterate(self._text))
return list(dic.iterate(self._text))
... ...
+169 -53
View File
@@ -58,6 +58,15 @@ _width_cache: UsageCache = UsageCache(DEFAULT_WIDTH_CACHE_ENTRIES)
_glyph_cache: SizedUsageCache = SizedUsageCache(DEFAULT_GLYPH_CACHE_BYTES, _glyph_entry_bytes) _glyph_cache: SizedUsageCache = SizedUsageCache(DEFAULT_GLYPH_CACHE_BYTES, _glyph_entry_bytes)
_glyph_subpixel_steps: int = DEFAULT_GLYPH_SUBPIXEL_STEPS _glyph_subpixel_steps: int = DEFAULT_GLYPH_SUBPIXEL_STEPS
# Every Line asks its font for the advance width of a space. That single
# FreeTypeFont.getlength(" ") call costs ~18us -- two orders of magnitude more
# than getmetrics() -- because PIL shapes the string from scratch each time, and
# it lands once per line created, which dominates the cost of laying a line out.
# There are only ever a handful of distinct fonts in play, so memoise per font
# object. Values are wrapped in a 1-tuple because None is itself a legitimate
# result (fonts that cannot report a length) and must not read as a cache miss.
_space_advance_cache: Dict[Any, Tuple[Optional[int]]] = {}
# Set to False the first time the fast rasterisation path is found to be # Set to False the first time the fast rasterisation path is found to be
# unavailable (e.g. a PIL build without the private ImageDraw internals it uses), # unavailable (e.g. a PIL build without the private ImageDraw internals it uses),
# after which every Text falls back to ImageDraw.text(). # after which every Text falls back to ImageDraw.text().
@@ -98,6 +107,35 @@ def clear_text_caches():
"""Drop all cached widths and glyph bitmaps.""" """Drop all cached widths and glyph bitmaps."""
_width_cache.clear() _width_cache.clear()
_glyph_cache.clear() _glyph_cache.clear()
_space_advance_cache.clear()
def _space_advance(font) -> Optional[int]:
"""
The font's own advance width for a space, in whole pixels.
None when the font cannot report one, which is the signal for callers to fall
back to their configured spacing range.
"""
try:
cached = _space_advance_cache.get(font)
except TypeError:
# Unhashable font object; measure without caching.
cached = None
else:
if cached is not None:
return cached[0]
try:
value = int(round(font.getlength(" ")))
except (AttributeError, TypeError, ValueError):
value = None
try:
_space_advance_cache[font] = (value,)
except TypeError:
pass
return value
def text_cache_stats() -> Dict[str, Any]: def text_cache_stats() -> Dict[str, Any]:
@@ -215,7 +253,8 @@ class AlignmentHandler(ABC):
def calculate_spacing_and_position(self, text_objects: List['Text'], def calculate_spacing_and_position(self, text_objects: List['Text'],
available_width: int, min_spacing: int, available_width: int, min_spacing: int,
max_spacing: int, max_spacing: int,
natural_spacing: Optional[int] = None natural_spacing: Optional[int] = None,
total_width: Optional[float] = None
) -> Tuple[int, int, bool]: ) -> Tuple[int, int, bool]:
""" """
Calculate the spacing between words and starting position for the line. Calculate the spacing between words and starting position for the line.
@@ -228,6 +267,11 @@ class AlignmentHandler(ABC):
natural_spacing: The font's own space width. Ragged alignments use it natural_spacing: The font's own space width. Ragged alignments use it
as a constant gap; justification ignores it. Defaults to as a constant gap; justification ignores it. Defaults to
min_spacing when not supplied. min_spacing when not supplied.
total_width: The summed width of `text_objects`, when the caller
already knows it. Purely an optimisation: a line asks its handler
to re-measure once per candidate word, and summing the whole line
each time makes filling a line quadratic in its word count. Omit
it and the sum is taken here as before.
Returns: Returns:
Tuple of (spacing_between_words, starting_x_position, overflow) Tuple of (spacing_between_words, starting_x_position, overflow)
@@ -242,7 +286,8 @@ class LeftAlignmentHandler(AlignmentHandler):
available_width: int, available_width: int,
min_spacing: int, min_spacing: int,
max_spacing: int, max_spacing: int,
natural_spacing: Optional[int] = None natural_spacing: Optional[int] = None,
total_width: Optional[float] = None
) -> Tuple[int, int, bool]: ) -> Tuple[int, int, bool]:
""" """
Calculate spacing and position for left-aligned text objects. Calculate spacing and position for left-aligned text objects.
@@ -269,7 +314,8 @@ class LeftAlignmentHandler(AlignmentHandler):
spacing = min_spacing if natural_spacing is None else natural_spacing spacing = min_spacing if natural_spacing is None else natural_spacing
spacing = max(min_spacing, min(max_spacing, int(spacing))) spacing = max(min_spacing, min(max_spacing, int(spacing)))
text_length = sum([text.width for text in text_objects]) text_length = (sum([text.width for text in text_objects])
if total_width is None else total_width)
num_gaps = len(text_objects) - 1 num_gaps = len(text_objects) - 1
# The spacing is constant whether or not the content fits: tightening a # The spacing is constant whether or not the content fits: tightening a
@@ -290,7 +336,8 @@ class CenterRightAlignmentHandler(AlignmentHandler):
def calculate_spacing_and_position(self, text_objects: List['Text'], def calculate_spacing_and_position(self, text_objects: List['Text'],
available_width: int, min_spacing: int, available_width: int, min_spacing: int,
max_spacing: int, max_spacing: int,
natural_spacing: Optional[int] = None natural_spacing: Optional[int] = None,
total_width: Optional[float] = None
) -> Tuple[int, int, bool]: ) -> Tuple[int, int, bool]:
""" """
Centre/right alignment: constant word space, line shifted as a block. Centre/right alignment: constant word space, line shifted as a block.
@@ -300,7 +347,8 @@ class CenterRightAlignmentHandler(AlignmentHandler):
the same spacing that will actually be used, so the line lands where it the same spacing that will actually be used, so the line lands where it
was measured to land. was measured to land.
""" """
word_length = sum([word.width for word in text_objects]) word_length = (sum([word.width for word in text_objects])
if total_width is None else total_width)
# Handle single word case # Handle single word case
if len(text_objects) <= 1: if len(text_objects) <= 1:
@@ -329,13 +377,42 @@ class JustifyAlignmentHandler(AlignmentHandler):
"""Handler for justified text with full justification.""" """Handler for justified text with full justification."""
def __init__(self): def __init__(self):
# Store variable spacing for each gap to distribute remainder pixels # The per-gap spacings are described by a plan rather than stored outright,
self._gap_spacings: List[int] = [] # and materialised on demand by the _gap_spacings property below. Fitting a
# line calls this handler once per candidate word and only ever looks at the
# first gap; building the whole list on each of those probes made adding n
# words to a line O(n^2). Only render() reads the full list.
self._gap_uniform: Optional[int] = None
self._gap_residual: int = 0
self._gap_count: int = 0
self._gap_cache: Optional[List[int]] = []
@property
def _gap_spacings(self) -> List[int]:
"""The spacing to apply at each gap, left to right."""
if self._gap_cache is None:
if self._gap_uniform is not None:
self._gap_cache = [self._gap_uniform] * self._gap_count
else:
self._gap_cache = self._distribute(self._gap_residual, self._gap_count)
return self._gap_cache
@staticmethod
def _distribute(total: int, num_gaps: int) -> List[int]:
"""Split `total` pixels across `num_gaps` gaps by cumulative rounding."""
gaps = []
placed = 0
for i in range(1, num_gaps + 1):
cumulative = int(round(total * i / num_gaps))
gaps.append(cumulative - placed)
placed = cumulative
return gaps
def calculate_spacing_and_position(self, text_objects: List['Text'], def calculate_spacing_and_position(self, text_objects: List['Text'],
available_width: int, min_spacing: int, available_width: int, min_spacing: int,
max_spacing: int, max_spacing: int,
natural_spacing: Optional[int] = None natural_spacing: Optional[int] = None,
total_width: Optional[float] = None
) -> Tuple[int, int, bool]: ) -> Tuple[int, int, bool]:
""" """
Justified alignment distributes space to fill the entire line width. Justified alignment distributes space to fill the entire line width.
@@ -347,14 +424,17 @@ class JustifyAlignmentHandler(AlignmentHandler):
is min_spacing to ensure readability. is min_spacing to ensure readability.
""" """
word_length = sum([word.width for word in text_objects]) word_length = (sum([word.width for word in text_objects])
if total_width is None else total_width)
residual_space = available_width - word_length residual_space = available_width - word_length
num_gaps = max(1, len(text_objects) - 1) num_gaps = max(1, len(text_objects) - 1)
# Check if we have enough space for minimum spacing # Check if we have enough space for minimum spacing
if residual_space // num_gaps < min_spacing: if residual_space // num_gaps < min_spacing:
# Not enough space - this is overflow # Not enough space - this is overflow
self._gap_spacings = [min_spacing] * num_gaps self._gap_uniform = min_spacing
self._gap_count = num_gaps
self._gap_cache = None
return min_spacing, 0, True return min_spacing, 0, True
# Distribute the residual by cumulative rounding rather than by taking a # Distribute the residual by cumulative rounding rather than by taking a
@@ -365,14 +445,14 @@ class JustifyAlignmentHandler(AlignmentHandler):
# ragged right edge on otherwise justified text. Rounding the running # ragged right edge on otherwise justified text. Rounding the running
# total makes the gaps sum to the residual exactly. # total makes the gaps sum to the residual exactly.
total = int(round(residual_space)) total = int(round(residual_space))
self._gap_spacings = [] self._gap_uniform = None
placed = 0 self._gap_residual = total
for i in range(1, num_gaps + 1): self._gap_count = num_gaps
cumulative = int(round(total * i / num_gaps)) self._gap_cache = None
self._gap_spacings.append(cumulative - placed)
placed = cumulative
return self._gap_spacings[0], 0, False # The first gap is the whole of the plan that fitting needs, and it falls
# out of the same cumulative rounding as _distribute would give it.
return int(round(total / num_gaps)), 0, False
class Text(Renderable, Queriable): class Text(Renderable, Queriable):
@@ -689,6 +769,9 @@ class Line(Box):
""" """
super().__init__(origin, size, callback, sheet, mode, halign, valign) super().__init__(origin, size, callback, sheet, mode, halign, valign)
self._text_objects: List['Text'] = [] # Store Text objects directly self._text_objects: List['Text'] = [] # Store Text objects directly
# Prefix sums of the widths in _text_objects, kept in step by _push_text /
# _pop_text. Element 0 is the empty sum. See _push_text for the rationale.
self._width_prefix: List[float] = [0.0]
self._spacing = spacing # (min_spacing, max_spacing) self._spacing = spacing # (min_spacing, max_spacing)
self._font = font if font else Font() # Use default font if none provided self._font = font if font else Font() # Use default font if none provided
self._current_width = 0 # Track the current width used self._current_width = 0 # Track the current width used
@@ -704,10 +787,7 @@ class Line(Box):
# The font's own space advance. Ragged alignments use this as their # The font's own space advance. Ragged alignments use this as their
# constant word gap rather than stretching to fill the measure. # constant word gap rather than stretching to fill the measure.
try: self._natural_spacing = _space_advance(self._font.font)
self._natural_spacing = int(round(self._font.font.getlength(" ")))
except (AttributeError, TypeError, ValueError):
self._natural_spacing = None
# Hyphenation configuration parameters # Hyphenation configuration parameters
self._min_word_length_for_brute_force = min_word_length_for_brute_force self._min_word_length_for_brute_force = min_word_length_for_brute_force
@@ -771,6 +851,45 @@ class Line(Box):
"""Set the next line in sequence""" """Set the next line in sequence"""
self._next = line self._next = line
@property
def _content_width(self) -> float:
"""Summed width of the line's current contents."""
return self._width_prefix[-1]
def _push_text(self, text: 'Text'):
"""
Append a Text to the line, keeping the running width sum in step.
Fitting a word is a trial: the candidate is pushed, measured, and popped
again if it did not fit, so the line's contents churn far more often than
they grow. Tracking the sum here rather than re-adding every width on each
measurement is what keeps filling a line linear in its word count.
The sum is kept as a prefix list rather than as one accumulator that is
added to and subtracted from. Widths are floats, so `(total + w) - w` need
not give back `total` exactly, and a drift of one ulp is enough to flip an
overflow decision on a line that ends flush. Truncating a prefix list
restores the earlier total bit for bit, and each entry is built by the same
left-to-right addition sum() would perform.
"""
self._text_objects.append(text)
self._width_prefix.append(self._width_prefix[-1] + text.width)
def _pop_text(self) -> 'Text':
"""Remove the last Text from the line, keeping the width sum in step."""
text = self._text_objects.pop()
self._width_prefix.pop()
return text
def _measure(self, handler: Optional[AlignmentHandler] = None
) -> Tuple[int, int, bool]:
"""Ask an alignment handler to place the line's current contents."""
if handler is None:
handler = self._alignment_handler
return handler.calculate_spacing_and_position(
self._text_objects, self._size[0], self._spacing[0], self._spacing[1],
self._natural_spacing, self._content_width)
def add_word(self, def add_word(self,
word: 'Word', word: 'Word',
part: Optional[Text] = None) -> Tuple[bool, part: Optional[Text] = None) -> Tuple[bool,
@@ -789,7 +908,7 @@ class Line(Box):
""" """
# First, add any pretext from previous hyphenation # First, add any pretext from previous hyphenation
if part is not None: if part is not None:
self._text_objects.append(part) self._push_text(part)
self._words.append(word) self._words.append(word)
part.add_line(self) part.add_line(self)
@@ -818,10 +937,8 @@ class Line(Box):
line=self) line=self)
else: else:
text = Text.from_word(word, self._draw) text = Text.from_word(word, self._draw)
self._text_objects.append(text) self._push_text(text)
spacing, position, overflow = self._alignment_handler.calculate_spacing_and_position( spacing, position, overflow = self._measure()
self._text_objects, self._size[0], self._spacing[0], self._spacing[1],
self._natural_spacing)
if not overflow: if not overflow:
# Word fits! Add it completely # Word fits! Add it completely
@@ -833,7 +950,7 @@ class Line(Box):
return True, None return True, None
# Word doesn't fit, remove it and try hyphenation # Word doesn't fit, remove it and try hyphenation
_ = self._text_objects.pop() self._pop_text()
# Step 1: Try pyphen hyphenation # Step 1: Try pyphen hyphenation
pyphen_splits = word.possible_hyphenation() pyphen_splits = word.possible_hyphenation()
@@ -866,11 +983,9 @@ class Line(Box):
source=word) source=word)
# Check if first part fits # Check if first part fits
self._text_objects.append(first_text) self._push_text(first_text)
spacing, position, overflow = self._alignment_handler.calculate_spacing_and_position( spacing, position, overflow = self._measure()
self._text_objects, self._size[0], self._spacing[0], self._spacing[1], self._pop_text()
self._natural_spacing)
_ = self._text_objects.pop()
if not overflow: if not overflow:
# This split fits! Add it to valid options # This split fits! Add it to valid options
@@ -883,7 +998,7 @@ class Line(Box):
first_text, second_text, spacing, position = best_split first_text, second_text, spacing, position = best_split
# Apply the split # Apply the split
self._text_objects.append(first_text) self._push_text(first_text)
first_text.line = self first_text.line = self
word.add_concete((first_text, second_text)) word.add_concete((first_text, second_text))
self._spacing_render = spacing self._spacing_render = spacing
@@ -894,7 +1009,7 @@ class Line(Box):
# Step 3: Try brute force hyphenation (only for long words) # Step 3: Try brute force hyphenation (only for long words)
if len(word.text) >= self._min_word_length_for_brute_force: if len(word.text) >= self._min_word_length_for_brute_force:
# Calculate available space for the word # Calculate available space for the word
word_length = sum([text.width for text in self._text_objects]) word_length = self._content_width
spacing_length = self._spacing[0] * max(0, len(self._text_objects) - 1) spacing_length = self._spacing[0] * max(0, len(self._text_objects) - 1)
remaining = self._size[0] - word_length - spacing_length remaining = self._size[0] - word_length - spacing_length
@@ -938,10 +1053,8 @@ class Line(Box):
source=word) source=word)
# Verify the first part actually fits # Verify the first part actually fits
self._text_objects.append(first_text) self._push_text(first_text)
spacing, position, overflow = self._alignment_handler.calculate_spacing_and_position( spacing, position, overflow = self._measure()
self._text_objects, self._size[0], self._spacing[0], self._spacing[1],
self._natural_spacing)
if not overflow: if not overflow:
# Brute force split works! # Brute force split works!
@@ -954,7 +1067,7 @@ class Line(Box):
return True, second_text return True, second_text
else: else:
# Doesn't fit, remove it # Doesn't fit, remove it
_ = self._text_objects.pop() self._pop_text()
# Step 4: Word cannot be hyphenated or split, move to next line # Step 4: Word cannot be hyphenated or split, move to next line
return False, None return False, None
@@ -972,9 +1085,7 @@ class Line(Box):
# justified paragraph. # justified paragraph.
handler = self.render_alignment_handler handler = self.render_alignment_handler
if len(self._text_objects) > 0: if len(self._text_objects) > 0:
spacing, position, overflow = handler.calculate_spacing_and_position( spacing, position, overflow = self._measure(handler)
self._text_objects, self._size[0], self._spacing[0], self._spacing[1],
self._natural_spacing)
self._spacing_render = spacing self._spacing_render = spacing
self._position_render = position self._position_render = position
@@ -982,28 +1093,33 @@ class Line(Box):
# Start x_cursor at line origin plus any alignment offset # Start x_cursor at line origin plus any alignment offset
x_cursor = self._origin[0] + self._position_render x_cursor = self._origin[0] + self._position_render
for i, text in enumerate(self._text_objects):
# Everything the loop needs that does not vary per word is resolved once.
# Only justified lines carry per-gap spacings; every other alignment uses
# the single spacing figured above.
texts = self._text_objects
last = len(texts) - 1
draw = self._draw
default_spacing = self._spacing_render
gaps = handler._gap_spacings if isinstance(handler, JustifyAlignmentHandler) else ()
gap_count = len(gaps)
for i, text in enumerate(texts):
# Update text draw context to current draw context # Update text draw context to current draw context
text._draw = self._draw text._draw = draw
text.set_origin(np.array([x_cursor, y_cursor])) text.set_origin(np.array([x_cursor, y_cursor]))
# Determine next text object for continuous decoration # Determine next text object for continuous decoration
next_text = self._text_objects[i + 1] if i + \ next_text = texts[i + 1] if i < last else None
1 < len(self._text_objects) else None
# Get the spacing for this specific gap (variable for justified text) # Get the spacing for this specific gap (variable for justified text)
if isinstance(handler, JustifyAlignmentHandler) and \ current_spacing = gaps[i] if i < gap_count else default_spacing
hasattr(handler, '_gap_spacings') and \
i < len(handler._gap_spacings):
current_spacing = handler._gap_spacings[i]
else:
current_spacing = self._spacing_render
# Render with next text information for continuous underline/strikethrough # Render with next text information for continuous underline/strikethrough
text.render(next_text, current_spacing) text.render(next_text, current_spacing)
# Add text width, then spacing only if there are more words # Add text width, then spacing only if there are more words
x_cursor += text.width x_cursor += text.width
if i < len(self._text_objects) - 1: if i < last:
x_cursor += current_spacing x_cursor += current_spacing
def query_point(self, point: Tuple[int, int]) -> Optional['QueryResult']: def query_point(self, point: Tuple[int, int]) -> Optional['QueryResult']:
+7 -7
View File
@@ -403,13 +403,13 @@ def extract_words_from_nodes(nodes: List, context: StyleContext) -> List[Word]:
continue continue
if isinstance(child, NavigableString): if isinstance(child, NavigableString):
# Plain text - split into words # Plain text - split into words. Argument-less str.split() already
text = str(child).strip() # discards surrounding whitespace and never yields an empty string, so
if text: # it needs neither a preceding strip() nor a per-word emptiness test.
word_texts = text.split() font = context.font
for word_text in word_texts: background = context.background
if word_text: words.extend([Word(word_text, font, background)
words.append(Word(word_text, context.font, context.background)) for word_text in str(child).split()])
elif isinstance(child, Tag): elif isinstance(child, Tag):
# Special handling for <a> tags (hyperlinks) # Special handling for <a> tags (hyperlinks)
if child.name.lower() == "a": if child.name.lower() == "a":
+3 -6
View File
@@ -163,12 +163,9 @@ def paragraph_layouter(paragraph: Paragraph,
y_cursor = page._current_y_offset y_cursor = page._current_y_offset
x_cursor = page.content_origin[0] x_cursor = page.content_origin[0]
# Create a temporary Text object to calculate word width # `word` is accepted for call-site readability only: the line that is about
if word: # to be created measures it when it is added, so measuring it here as well
temp_text = Text.from_word(word, page.measurement_draw) # only paid for a Text object that was immediately discarded.
temp_text.width
else:
pass
return Line( return Line(
spacing=word_spacing_constraints, spacing=word_spacing_constraints,
+16 -3
View File
@@ -43,6 +43,19 @@ class RenderingPosition:
remaining_pretext: Optional[str] = None # Hyphenated word continuation remaining_pretext: Optional[str] = None # Hyphenated word continuation
page_y_offset: int = 0 # Vertical position on page page_y_offset: int = 0 # Vertical position on page
def _key(self) -> Tuple[Any, ...]:
"""
The fields in declaration order.
Copying, comparing and hashing a position all used to go through
dataclasses.asdict, which walks the field list and deep-copies each value.
Every field here is an immutable scalar, so that traversal bought nothing
and these three run constantly during page navigation and buffer lookups.
"""
return (self.chapter_index, self.block_index, self.word_index,
self.table_row, self.table_col, self.list_item_index,
self.remaining_pretext, self.page_y_offset)
def to_dict(self) -> Dict[str, Any]: def to_dict(self) -> Dict[str, Any]:
"""Serialize position for saving to file/database""" """Serialize position for saving to file/database"""
return asdict(self) return asdict(self)
@@ -54,17 +67,17 @@ class RenderingPosition:
def copy(self) -> 'RenderingPosition': def copy(self) -> 'RenderingPosition':
"""Create a copy of this position""" """Create a copy of this position"""
return RenderingPosition(**asdict(self)) return RenderingPosition(*self._key())
def __eq__(self, other) -> bool: def __eq__(self, other) -> bool:
"""Check if two positions are equal""" """Check if two positions are equal"""
if not isinstance(other, RenderingPosition): if not isinstance(other, RenderingPosition):
return False return False
return asdict(self) == asdict(other) return self._key() == other._key()
def __hash__(self) -> int: def __hash__(self) -> int:
"""Make position hashable for use as dict key""" """Make position hashable for use as dict key"""
return hash(tuple(asdict(self).values())) return hash(self._key())
class ChapterInfo: class ChapterInfo:
+6
View File
@@ -453,6 +453,12 @@ class EreaderLayoutManager:
# Special case: if at the beginning of content and there's a cover, go back to it # Special case: if at the beginning of content and there's a cover, go back to it
if self._has_cover and self._is_at_beginning() and not self._on_cover_page: if self._has_cover and self._is_at_beginning() and not self._on_cover_page:
self._on_cover_page = True self._on_cover_page = True
# Restore the canonical cover position. Being on the cover must have a
# single representation: a fresh load sits at block 0 with the cover
# showing, so returning to the cover has to land there too. Leaving the
# position at the first content block saves a position that reopens past
# the cover, silently losing it.
self.current_position = RenderingPosition()
self._notify_position_changed() self._notify_position_changed()
return self.get_current_page() return self.get_current_page()
+13 -1
View File
@@ -112,7 +112,17 @@ class AbstractStyle:
Since this is a frozen dataclass, it should be hashable by default, Since this is a frozen dataclass, it should be hashable by default,
but we provide a custom implementation to ensure all fields are but we provide a custom implementation to ensure all fields are
properly considered and to handle the Union types correctly. properly considered and to handle the Union types correctly.
The result is memoised on first use. Styles are used as dictionary keys
throughout parsing and style resolution, and five of the fields are enum
members whose own __hash__ is a Python-level call, so rebuilding the
15-tuple on every lookup was a measurable share of document parsing. The
class is frozen, so the value cannot go stale.
""" """
cached = self.__dict__.get('_hash_cache')
if cached is not None:
return cached
# Convert all values to hashable forms # Convert all values to hashable forms
hashable_values = ( hashable_values = (
self.font_family, self.font_family,
@@ -132,7 +142,9 @@ class AbstractStyle:
self.parent_style_id self.parent_style_id
) )
return hash(hashable_values) result = hash(hashable_values)
object.__setattr__(self, '_hash_cache', result)
return result
def merge_with(self, other: 'AbstractStyle') -> 'AbstractStyle': def merge_with(self, other: 'AbstractStyle') -> 'AbstractStyle':
""" """
+6 -2
View File
@@ -372,8 +372,12 @@ class TestImagePIL(unittest.TestCase):
cls.flask_thread = threading.Thread(target=cls.flask_server.serve_forever, daemon=True) cls.flask_thread = threading.Thread(target=cls.flask_server.serve_forever, daemon=True)
cls.flask_thread.start() cls.flask_thread.start()
# Wait for server to be ready with health check # Wait for server to be ready with health check.
max_wait = 5 # Maximum 5 seconds # Generous, because this now raises rather than falling through
# silently: on a loaded CI runner the accept loop can take several
# seconds to get scheduled, and a spurious failure here is worse than
# a slow one. The loop exits as soon as the server answers.
max_wait = 30
wait_interval = 0.1 # Check every 100ms wait_interval = 0.1 # Check every 100ms
elapsed = 0 elapsed = 0