Compare commits

..
17 Commits
Author SHA1 Message Date
dtourolleandClaude Opus 5 cfc4230713 fix(ereader): restore cover position when navigating back to the cover
Python CI / test (3.10) (push) Successful in 1m2s
Python CI / test (3.11) (push) Successful in 1m5s
Python CI / test (3.12) (push) Successful in 1m5s
Python CI / test (3.13) (push) Successful in 1m43s
previous_page() set the on-cover flag but left current_position at the
first content block, so "showing the cover" had two different internal
representations depending on how you got there: block 0 on a fresh load,
block 1 after going forward and back.

current_position is what gets persisted, so closing the book while on the
cover reopened it past the cover, silently losing it.

Reset current_position to block 0 when returning to the cover, matching
where a fresh load sits.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 22:26:38 +02:00
dtourolleandClaude Opus 5 0bb34a4a32 perf(layout): cut page layout time by ~40%
Python CI / test (3.10) (push) Successful in 1m4s
Python CI / test (3.11) (push) Successful in 1m8s
Python CI / test (3.12) (push) Successful in 1m5s
Python CI / test (3.13) (push) Successful in 1m24s
Layout was dominated by work that was either repeated per word or thrown
away. Rendered output is unchanged: every page hashed byte-for-byte
identical across 3 page sizes, 2 font scales, 2 font families, and 4
alignments x 4 column widths chosen to force heavy hyphenation.

  layout, 600x800, 40 pages     ~200ms -> ~124ms
  layout, 1404x1872, 11 pages   ~168ms -> ~111ms

Measured, not guessed. The reflex fix - swapping list comprehensions for
generators - measures slower here (602ns vs 425ns for the width sum), so
those are left alone.

- Line asked its font for the advance width of a space on every
  construction. FreeTypeFont.getlength(" ") costs ~18us, two orders of
  magnitude more than getmetrics(), and it landed once per line. Memoise
  per font object.

- Line.add_word was quadratic in the words on a line. Each candidate word
  re-summed every width and rebuilt the whole per-gap spacing list, when
  fitting only ever reads the first gap. Gap spacings are now a plan
  materialised on demand (only render() reads the list), and widths come
  from a prefix sum. The prefix list, rather than one accumulator, is what
  keeps this exact: widths are floats, (total + w) - w need not give back
  total, and one ulp flips an overflow decision on a line that ends flush.

- RenderingPosition.copy/__eq__/__hash__ all went through
  dataclasses.asdict, a deep recursive walk, over 8 immutable scalars.

- paragraph_layouter built a Text per line purely to discard it.

- AbstractStyle.__hash__ rebuilt a 15-tuple containing 5 enums on every
  dict lookup; memoised on the frozen instance (1037ns -> 160ns).

- The pyphen dictionary wrapper was rebuilt for every word that overflowed
  its line, and word extraction stripped before splitting and tested each
  split result for emptiness, neither of which str.split() needs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 14:27:20 +02:00
dtourolleandClaude Opus 5 745fc8687e ci: run tests in a prebuilt container image
Python CI / test (3.10) (push) Successful in 1m21s
Python CI / test (3.11) (push) Successful in 1m7s
Python CI / test (3.12) (push) Successful in 1m9s
Python CI / test (3.13) (push) Successful in 1m23s
Matches the convention used by pyPhotoAlbum and the other projects here:
runs-on: linux/amd64 with a container image from the Gitea registry,
instead of setup-python plus an ad-hoc `pip install pytest pytest-cov
flake8 coverage-badge interrogate` on a self-hosted runner.

pyWebLayout is a library, so the image carries all four interpreters
pyproject.toml claims to support - 3.10, 3.11, 3.12, 3.13 - each in its
own venv at /opt/py<version> with every dependency pre-installed. The
matrix picks one per job. A CI run now downloads nothing, and coverage
widens from 3.10/3.12/3.13 to the full declared range.

Ubuntu marks its system Python externally-managed, so per-interpreter
venvs are used rather than --break-system-packages; that also keeps the
four dependency sets isolated.

Two defects in the existing workflow are fixed while rewriting it:

- pytest runs under continue-on-error so the badge steps still execute,
  but nothing afterwards checked its outcome - the job reported green on
  a red suite. An explicit gate now fails the job.
- Every matrix leg ran the badge steps and force-pushed the badges
  branch, so three jobs raced to publish. Badges and artifacts are now
  produced by the 3.13 leg only.

setuptools is pinned below 81 in the image: that release dropped
pkg_resources, which coverage-badge imports at startup, and without the
pin the badge step dies with ModuleNotFoundError. Found by running the
workflow's own commands in the image rather than assuming they work.

Also raises the test Flask server's readiness budget from 5s to 30s.
Making that check raise instead of silently falling through (737cf07)
turned runner load into a hard failure; it showed up as 17 spurious
errors in one containerised run and did not reproduce in three repeats.
The loop still exits as soon as the server answers.

Verified locally: image builds, and tests/ passes 916 on each of 3.10,
3.11, 3.12 and 3.13 inside it. The publishing leg was run end to end -
clean-install dependency check, pytest with coverage, both badges,
coverage summary at 81.5%.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 14:10:05 +02:00
dtourolleandClaude Opus 5 3761e00398 docs: record R1-R8 as resolved and add R9
Python CI / test (3.10) (push) Canceled after 0s
Python CI / test (3.12) (push) Canceled after 0s
Python CI / test (3.13) (push) Canceled after 0s
Adds a status table mapping each finding to the commit that resolved it,
and corrects R8's entry: S16 landed anchor replay independently, which is
the design R8 asked for.

Records R9, found while verifying R3. The hit region query_point reports
for a text object is offset from the object's own origin/size by roughly
the font ascent, so probing a LinkText at its own centre returns "empty".
It reproduces at every font scale, so it predates the R3 work, but it
matters more now: R7's highlighting uses those bounds to place overlays.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 13:21:20 +02:00
dtourolleandClaude Opus 5 0ce1aeaa87 feat(ereader): wire pointer interaction into EreaderLayoutManager (R7)
concrete/interaction_handler.py was 310 lines reachable only from
examples/07_pressed_state_demo.py - no library code, no tests. Press and
hover feedback existed but could not be used through the library's own
interface.

Adds to the manager:

    handle_hover(point)        -> frame if hover changed, else None
    handle_touch_down(point)   -> frame showing the pressed state
    handle_touch_up(point)     -> (frame, callback result)
    reset_interaction_state()

Returning None when nothing changed visually lets a UI skip a redraw it
does not need - which matters on e-ink.

Press state belongs to one rendered page, so the InteractionStateManager
is bound lazily and rebound whenever the displayed page changes, resetting
the outgoing one so a press cannot survive a page turn.

Wiring it up immediately surfaced a real bug it had been hiding.
LinkText.render passed [origin, origin + size] - a list of two numpy
arrays - to PIL's draw.rectangle, which needs a flat four-scalar box.
Rendering any hovered or pressed link raised

    TypeError: coordinate list must contain exactly 2 coordinates

so the entire feature was broken on this PIL version. Fixed by building
the box explicitly, and the two branches now share it instead of
duplicating the call.

Tests cover hover/press/release, no-op paths, that an unchanged hover
reports no change, state rebinding across navigation, reset, and
regressions for the rectangle crash.

916 passed. examples/07_pressed_state_demo.py still runs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 13:20:40 +02:00
dtourolleandClaude Opus 5 8746d3f549 feat(ereader): wire highlighting into EreaderLayoutManager (R7)
core/highlight.py was 249 lines of fully implemented, fully tested code
that nothing could reach. EreaderLayoutManager had no highlight API, so
highlighting was not available through the library's own interface - it
was tested in isolation and otherwise dead.

Adds to the manager:

    highlight_point(point, color, note, tags)   tap to highlight a word
    highlight_range(start, end, ...)            drag to highlight a span
    remove_highlight(id) / clear_highlights()
    list_highlights() / get_highlights_for_current_page()

Highlights default to the bookmarks directory, so a document's reading
state lives in one place rather than two.

A Highlight carried only pixel bounds, which describe the one rendering
they were taken from: change the font scale or page size and they no
longer point at anything. Highlight now also records the
RenderingPosition of the page it was made on, and page association goes
through that instead of through bounds overlap. The field is optional and
read with .get, so existing stores load unchanged - they simply never
match a page, which is the honest answer for a highlight whose only
anchor is stale pixels.

Also removes the persistence duplication the review called out.
BookmarkManager and HighlightManager each had their own copy of "make the
directory, read the file, swallow and print on failure". Both now use
core/persistence.py, which logs with exc_info instead of printing and
catches specific exceptions rather than bare Exception. File names and
formats are unchanged, so nothing needs migrating.

Tests cover point and range highlighting, colour/note/tag round trips,
misses returning None, page scoping across navigation, persistence across
a restart, and that a corrupt highlight file does not stop a book from
opening.

902 passed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 13:18:14 +02:00
dtourolleandClaude Opus 5 bcae45a023 refactor(ereader): drop the estimator helpers S16 superseded (R8)
S16 replaced backward pagination's estimate-render-compare-adjust loop
with anchor replay, but left _estimate_page_start and
_adjust_start_estimate behind. Nothing in the library calls them; only
their own tests did.

_estimate_page_start guessed max(1, int(10 / font_scale)) blocks per
page - a constant with no relationship to page size, block length or
font metrics - and _adjust_start_estimate halved the error each round to
converge on it. Anchor replay makes both meaningless: it walks forward
from a known page boundary instead of guessing at one.

Removes their five tests with them rather than leaving tests pinning
behaviour nothing depends on.

889 passed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 13:07:53 +02:00
dtourolleandClaude Opus 5 e81ba48f6d refactor(page): delete the dead child-measurement helpers (R6)
page.py carried a closed cluster of five methods with no callers outside
itself:

    _get_child_property   called only by the four below
    _get_child_height     called by nothing
    _get_child_position   called only by _point_in_child
    _point_in_child       called by nothing
    _get_child_size       called only by _point_in_child

138 lines, verified unreferenced across pyWebLayout/, tests/, examples/
and scripts/.

They existed because Renderable declares no size, so the code probed
_size, size, _height, height, _origin and position in turn with hasattr,
guessing at each child's shape. query_point already does the right thing
instead: it hit-tests through the Queriable interface.

Hardening the Renderable contract so this cannot grow back - Renderable
has origin but no size - belongs with S10.1, which is already going to
revisit the render contract in core/base.py. Left alone here rather than
half-done.

894 passed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 12:56:05 +02:00
dtourolleandClaude Opus 5 62ca15159a refactor(ereader): delete the import-time Page monkey patch (R5)
ereader_layout.py defined _add_page_methods() and called it at import
time, attaching can_fit_line and available_width to the Page class if
they were absent. Page defines both, so the patch never fired - but the
two definitions of can_fit_line disagreed:

    Page          can_fit_line(baseline_spacing, ascent=0, descent=0)
    monkey patch  can_fit_line(line_height)

The patched version had no way to express descent, which is exactly the
clipping bug S2 fixed. Had Page.can_fit_line ever been renamed or moved,
this would have silently reinstated pre-S2 behaviour as a side effect of
importing a module in a different package.

Import-time patching of another module's class has no place here. If the
layout engine needs something from Page, it belongs on Page.

Tests pin the outcome: the module exposes no patcher, Page owns both
attributes, and can_fit_line still rejects a line whose descender would
hang past the content box.

894 passed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 12:52:38 +02:00
dtourolleandClaude Opus 5 f0dc67541b fix(ereader): keep hyperlinks and nested blocks when font scale changes (R3)
_scale_block_fonts rebuilt a block by constructing a plain
Word(word.text, scaled_style) for every word. LinkedWord is a Word
subclass, so the reconstruction downgraded it and dropped the link
target: every hyperlink in the document disappeared the moment the reader
changed font size. Nothing caught it because the function returns the
block unchanged at scale 1.0 with no family override, which is what the
tests exercised.

Add Word.with_style(), overridden by LinkedWord to carry location, link
type, callback, params and title across. Putting the copy behaviour on
the word class means any future Word subclass either inherits a correct
copy or overrides one, rather than being silently flattened by a
constructor call in the layout engine.

Also extend coverage beyond Paragraph and Heading. Quote, HList and Table
were returned unscaled, so a font-size change left quoted text, list
items and table cells at their original size while the surrounding text
reflowed. Table rows are re-added to the section they came from, so a
<thead> row does not become a body row. Image, HorizontalRule, PageBreak
and CodeBlock still pass through: they carry no styled words.

Scaled blocks are now memoised per (block, scale) for the life of the
layouter. Previously a fresh Paragraph and Word were allocated for every
word on every page render at any scale != 1.0, on the hot path, against
the caching work in concrete/text.py.

Tests cover with_style on both word classes, link survival and target
preservation across all four container types, per-container scaling,
table section preservation, memoisation, and that originals are never
mutated. End-to-end: links remain tappable at 0.8x, 1.5x and 2.0x.

Note: query_point's hit region is offset from LinkText.origin by roughly
the ascent, so probing a link's own centre reports "empty". That
reproduces identically at scale 1.0, predates this change, and is tracked
separately as R9 - the end-to-end test scans instead of probing.

891 passed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 12:51:05 +02:00
dtourolleandClaude Opus 5 1924cc234d perf(buffer): remove the process pool from page rendering (S12, R1, R2)
Python CI / test (3.10) (push) Has been cancelled
Python CI / test (3.12) (push) Has been cancelled
Python CI / test (3.13) (push) Has been cancelled
PageBuffer started a ProcessPoolExecutor(max_workers=4) and submitted page
renders to it. Every job failed. _render_page_worker returned
pickle.dumps(page), and a Page holds a live PIL canvas, which is not
picklable, so check_completed_renders swallowed a TypeError into a bare
print and cached nothing. The cost was paid in full for zero benefit:
four interpreter copies plus the whole block list shipped per job (~880KB
for a 200-block document).

Three further defects would have had to be fixed before it could ever
have worked: the worker built its BidirectionalLayouter without page_size
so it silently used the (800, 600) default; check_completed_renders
cached every result with is_backward=False, so backward renders landed in
the forward buffer; and both _queue_*_renders broke at the end of their
first loop body, queueing one page each despite looping buffer_size
times.

Two live defects go with it:

  R1 - on Python 3.14 the default start method became forkserver, so
  submit() reaches _check_not_importing_main() and raises unless the
  caller sits inside an `if __name__ == "__main__"` guard.
  EreaderLayoutManager.get_current_page() raised outright from ordinary
  module-level script code.

  R2 - PageBuffer.__del__ called executor.shutdown(wait=True). Blocking
  on a process pool from a finaliser at interpreter teardown deadlocked;
  the test suite finished in 11.5s and then never exited.

S12's measurement gate, on the tests/data Wikipedia fixture (411 blocks)
with text caches warm:

    800x600     p50  8.8 ms   p95 15.4 ms
    1072x1448   p50 13.8 ms   p95 56.1 ms

A page turn is cheaper than the IPC meant to hide it, so the gate says
delete rather than replace. The LRU buffers, position maps and
invalidation logic are kept unchanged; only the executor, worker,
pickling, prefetch queueing and the lock guarding the pending-render dict
are removed. If a slower device ever changes the numbers, the fallback is
a synchronous readahead() method or a single worker thread, not
processes.

EreaderLayoutManager.shutdown() becomes idempotent and its __del__ no
longer propagates exceptions - it was doing bookmark file I/O during
interpreter teardown.

Adds tests/layout/test_page_buffer.py, which the module had none of:
LRU eviction and position-map cleanup, cache hits, font-scale
invalidation, backward round-trip, and subprocess regressions for R1
(no __main__ guard) and R2 (exit without explicit shutdown).

870 passed, and the suite now exits in 12s wall instead of hanging.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 12:45:20 +02:00
dtourolle 456824d6d6 fix(ereader): replay the page chain instead of searching for it (S16)
render_page_backward searched for the previous page's start: estimate a block
index, lay out forward, bisect on the block difference, up to ten times. Both the
estimator and the adjuster pinned word_index to 0 and moved only block_index -
but pages routinely start mid-block, so the answer was not in the search space.
The loop could never match, exhausted its iterations and fell through to a
fallback that jumped to the start of the document.

Measured on a document with one 1200-word paragraph, whose page starts are
(0,0), (2,208), (2,494), (2,780): three of four backward calls returned (0,0),
each after ten full page layouts. The bisection was unsound in its own space too
- 40 small paragraphs, every page starting on a block boundary, also failed.

Pagination is a pure function, so the page before P is the q with next(q) == P,
and it is found by replaying the chain forward rather than guessing q. Three
sources: the chain recorded as pages are laid out forward (exact, one layout,
covers paging back and forth); replay from the block containing P and then from
earlier blocks (exact when P lies on that chain); and failing that, the last
start before P, which overlaps slightly rather than skipping content.

  warm:                          4/4 exact, 1 layout each
  cold, fresh layouter per call: 12/13 exact, worst 17 layouts
  cold, repeated back presses:   ~4 layouts per turn

Each step returns a page ending exactly where the reader is, so paging back never
skips or repeats content. That chain can differ from the one seen reading forward
from page one if the reader arrived by a jump - pagination from a different start
is a different chain, and nothing can recover the original without replaying the
whole document.

Complementary to S11 rather than caused by it: before S11 forward pagination
dead-ended at the first page-spanning block, so mid-block starts never arose and
the block-granular search looked adequate.
2026-08-08 12:43:30 +02:00
dtourolleandClaude Opus 5 767e4c135c build: consolidate packaging on pyproject.toml and correct the dependency set
The project carried three sets of packaging metadata — pyproject.toml
[project], setup.cfg [options] and setup.py kwargs — declaring different
dependencies. pyproject.toml wins under any modern backend, so the wheel
was correct, but the other two said "Pillow, numpy" and reading either
gave the wrong answer about what the library needs.

Reduce setup.cfg to its [flake8] section and setup.py to a setup() shim,
each pointing at pyproject.toml.

Correct the authoritative list while consolidating:

- flask was a runtime dependency but is imported only by the fixture HTTP
  server in tests. Every user was installing Flask, Jinja2, Werkzeug,
  click, itsdangerous and blinker for nothing.
- ebooklib was a runtime dependency and is never imported by the library;
  epub_reader.py uses zipfile + xml.etree directly. Only the tests use it,
  to build EPUB fixtures.
- requests was declared required but concrete/image.py imports it lazily
  and degrades gracefully when absent, so it belongs in an extra.
- requires-python said >=3.6, which cannot be true: the package uses
  dataclasses and `from __future__ import annotations`, both 3.7+, and CI
  tests 3.10/3.12/3.13.

Runtime install drops from 7 direct dependencies to 4. Adds test,
remote-images and dev extras, and a CI step that installs into an empty
venv with only the runtime deps and imports every subpackage — this class
of defect is only caught by installing what you ship.

Verified: runtime-only install imports all subpackages; `.[test]` runs the
full suite, 853 passed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 12:30:08 +02:00
dtourolleandClaude Opus 5 7384a32cdd docs: add architecture review with findings R1-R8
Independent review of the codebase at c5c61a3. Records the architectural
verdict and eight findings not covered by LAYOUT_REMEDIATION_SPEC.md,
cross-referencing the existing spec where they overlap.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 12:21:02 +02:00
dtourolleandClaude Opus 5 4d596ce095 docs: rewrite ARCHITECTURE.md around layers and dependency rules
The previous version described only the abstract/concrete split and had
drifted from the code. Restructure it around the actual layering
(io → abstract → layout → concrete), state the dependency rules that
hold today, and document the layouter contract that makes pagination
resumable.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 12:21:02 +02:00
dtourolleandClaude Opus 5 737cf0771c test(image): bind the fixture Flask server to an ephemeral port
The URL-image tests hard-coded port 5555 and never shut the server down,
so a leftover or concurrent run left the port occupied and the readiness
loop fell through silently — the tests then ran against whatever was
listening, or against nothing.

Use werkzeug's make_server on port 0, record the assigned port, and shut
the server down in tearDownClass. The readiness loop now raises instead
of falling through when the server never comes up.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 12:21:02 +02:00
dtourolle 1985163827 fix(functional): form field labels no longer overprint the field above (S15)
Python CI / test (3.10) (push) Has been cancelled
Python CI / test (3.12) (push) Has been cancelled
Python CI / test (3.13) (push) Has been cancelled
FormFieldText treats its origin as the control's top-left - size and in_object
both measure down from it - but drew the label by calling Text.render at that
origin, and Text anchors on the baseline. The label's glyphs therefore landed
above the origin, outside the box the control claims, on top of whatever was
there. In a stacked form that is the preceding field's input box, which is what
example_10_forms.png showed: every label but the first crowding the box above it.

The label is now offset down by its ascent, so it occupies the space the control
accounts for. Height derives from the label's ink height rather than the nominal
font size, which had also eaten into the 5px gap between label and box.

LABEL_GAP names that gap and field_area_offset gives the distance from the origin
to the top of the input box; render, handle_click and the height calculation now
share it instead of each recomputing font_size + 5.

Also recorded under S12: the broken process pool is not merely wasted work. It
forks from a process that already has threads, and
tests/layout/test_ereader_image_rendering.py hangs at interpreter exit roughly
one run in four - every test passes, then the process never returns.
2026-08-06 23:26:03 +02:00
32 changed files with 3397 additions and 1122 deletions
+43
View File
@@ -0,0 +1,43 @@
# Dockerfile.ci copies nothing from the context, but keeping it small makes
# `docker build` fast and avoids shipping local state into the build.
# Virtual environments
venv/
.venv/
# Python cache and build output
__pycache__/
*.py[cod]
*$py.class
*.so
build/
dist/
*.egg-info/
*.egg
# Git
.git/
# Test/coverage output
.coverage
coverage.json
coverage.xml
htmlcov/
cov_info/
.pytest_cache/
.tox/
.mypy_cache/
# IDE
.idea/
.vscode/
*.swp
*.swo
.claude/
# Generated docs/images
docs/images/
# OS files
.DS_Store
Thumbs.db
+91 -93
View File
@@ -11,169 +11,167 @@ on:
jobs: jobs:
test: test:
runs-on: self-hosted runs-on: linux/amd64
container:
# Built from Dockerfile.ci at the repo root. Carries Python 3.10-3.13,
# each in its own venv at /opt/py<version> with every dependency
# pre-installed, so a run downloads nothing.
image: gitea.tourolle.paris/dtourolle/pyweblayout-ci:latest
strategy: strategy:
matrix: matrix:
python-version: ['3.10', '3.12', '3.13'] python-version: ['3.10', '3.11', '3.12', '3.13']
fail-fast: false
env:
# pyWebLayout is a library: it is tested on every interpreter
# pyproject.toml's requires-python claims to support.
PYBIN: /opt/py${{ matrix.python-version }}/bin
# Badges and artifacts are published once, not once per matrix leg -
# four jobs racing to force-push the same branch is not a publish
# strategy. This leg is the one that publishes.
PUBLISH: ${{ matrix.python-version == '3.13' }}
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@v4 uses: actions/checkout@v4
- name: Set up Python - name: Install project
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: | run: |
python -m pip install --upgrade pip # --no-deps: dependencies are baked into the image. If a new one is
# Install package in development mode # added to pyproject.toml, add it to Dockerfile.ci and rebuild;
pip install -e . # the check below is what catches forgetting to.
# Install test dependencies if they exist $PYBIN/pip install -e . --no-deps
if [ -f requirements-dev.txt ]; then pip install -r requirements-dev.txt; fi $PYBIN/python -c "import pyWebLayout; print('pyWebLayout', pyWebLayout.__file__)"
if [ -f requirements/test.txt ]; then pip install -r requirements/test.txt; fi
# Install common test packages
pip install pytest pytest-cov flake8 coverage-badge interrogate
- name: Download initial failed badges - name: Verify declared dependencies are sufficient
if: env.PUBLISH == 'true'
run: | run: |
echo "Downloading initial failed badges..." # A clean venv with ONLY the declared runtime deps, installed from
# the index rather than from the image. If an import here fails,
# Create cov_info directory first # pyproject.toml is incomplete and a real `pip install pyWebLayout`
mkdir -p cov_info # fails the same way for a user. This is the one step that is
# allowed to reach the network.
# Download failed badges as defaults $PYBIN/python -m venv /tmp/clean-install
curl -o cov_info/coverage.svg "https://img.shields.io/badge/coverage-failed-red.svg" /tmp/clean-install/bin/pip install --upgrade pip
curl -o cov_info/coverage-docs.svg "https://img.shields.io/badge/docs-failed-red.svg" /tmp/clean-install/bin/pip install .
/tmp/clean-install/bin/python -c "
echo "Initial failed badges created:" import pyWebLayout.concrete, pyWebLayout.abstract
ls -la cov_info/coverage*.svg import pyWebLayout.io.readers.epub_reader
import pyWebLayout.io.readers.html_extraction
import pyWebLayout.layout.ereader_manager
print('clean install imports OK')
"
- name: Run tests with pytest - name: Run tests with pytest
id: pytest id: pytest
continue-on-error: true continue-on-error: true
run: | run: |
# Run tests with coverage $PYBIN/python -m pytest tests/ -v \
python -m pytest tests/ -v --cov=pyWebLayout --cov-report=term-missing --cov-report=json --cov-report=html --cov-report=xml --cov=pyWebLayout \
--cov-report=term-missing \
--cov-report=json \
--cov-report=html \
--cov-report=xml
- name: Check documentation coverage - name: Check documentation coverage
id: docs id: docs
continue-on-error: true continue-on-error: true
run: | run: |
# Generate documentation coverage report $PYBIN/interrogate -v \
interrogate -v --ignore-init-method --ignore-init-module --ignore-magic --ignore-private --ignore-property-decorators --ignore-semiprivate --fail-under=80 pyWebLayout/ --ignore-init-method --ignore-init-module --ignore-magic \
--ignore-private --ignore-property-decorators --ignore-semiprivate \
--fail-under=80 pyWebLayout/
- name: Lint with flake8 - name: Lint with flake8
run: | run: |
# Stop the build if there are Python syntax errors or undefined names # Stop the build if there are Python syntax errors or undefined names
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics $PYBIN/flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# Exit-zero treats all errors as warnings # Exit-zero treats all errors as warnings
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics $PYBIN/flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Create coverage info directory - name: Fail the job if tests failed
if: always() if: steps.pytest.outcome != 'success'
run: |
# pytest runs with continue-on-error so the badge steps below still
# execute; without this the job would report green on a red suite.
echo "::error::pytest failed on Python ${{ matrix.python-version }}"
exit 1
# ------------------------------------------------------------------
# Badges and artifacts - publishing leg only
# ------------------------------------------------------------------
- name: Prepare badge directory
if: always() && env.PUBLISH == 'true'
run: | run: |
mkdir -p cov_info mkdir -p cov_info
echo "Created cov_info directory for coverage data" # Default to failed badges; the steps below overwrite them on success
curl -o cov_info/coverage.svg "https://img.shields.io/badge/coverage-failed-red.svg"
curl -o cov_info/coverage-docs.svg "https://img.shields.io/badge/docs-failed-red.svg"
- name: Update test coverage badge on success - name: Update test coverage badge on success
if: steps.pytest.outcome == 'success' && always() if: always() && env.PUBLISH == 'true' && steps.pytest.outcome == 'success'
run: | run: |
echo "Tests passed! Generating successful coverage badge..."
if [ -f coverage.json ]; then if [ -f coverage.json ]; then
coverage-badge -o cov_info/coverage.svg -f $PYBIN/coverage-badge -o cov_info/coverage.svg -f
echo "✅ Test coverage badge updated with actual results" echo "✅ Test coverage badge updated"
else else
echo "⚠️ No coverage.json found, keeping failed badge" echo "⚠️ No coverage.json found, keeping failed badge"
fi fi
- name: Update docs coverage badge on success - name: Update docs coverage badge on success
if: steps.docs.outcome == 'success' && always() if: always() && env.PUBLISH == 'true' && steps.docs.outcome == 'success'
run: | run: |
echo "Docs check passed! Generating successful docs badge..."
# Remove existing badge first to avoid overwrite error
rm -f cov_info/coverage-docs.svg rm -f cov_info/coverage-docs.svg
interrogate --generate-badge cov_info/coverage-docs.svg pyWebLayout/ $PYBIN/interrogate --generate-badge cov_info/coverage-docs.svg pyWebLayout/
echo "✅ Docs coverage badge updated with actual results" echo "✅ Docs coverage badge updated"
- name: Generate coverage reports - name: Generate coverage reports
if: steps.pytest.outcome == 'success' if: env.PUBLISH == 'true' && steps.pytest.outcome == 'success'
run: | run: |
# Generate coverage summary for README $PYBIN/python -c "
python -c " import json, os
import json
import os
# Read coverage data
if os.path.exists('coverage.json'): if os.path.exists('coverage.json'):
with open('coverage.json', 'r') as f: with open('coverage.json') as f:
coverage_data = json.load(f) data = json.load(f)
total_coverage = round(coverage_data['totals']['percent_covered'], 1) total = round(data['totals']['percent_covered'], 1)
# Create coverage summary file in cov_info directory
with open('cov_info/coverage-summary.txt', 'w') as f: with open('cov_info/coverage-summary.txt', 'w') as f:
f.write(f'{total_coverage}%') f.write(f'{total}%')
print(f'Test Coverage: {total_coverage}%') print(f\"Test Coverage: {total}%\")
covered_lines = coverage_data['totals']['covered_lines'] print(f\"Lines Covered: {data['totals']['covered_lines']}/{data['totals']['num_statements']}\")
total_lines = coverage_data['totals']['num_statements']
print(f'Lines Covered: {covered_lines}/{total_lines}')
else: else:
print('No coverage data found') print('No coverage data found')
" "
# Copy other coverage files to cov_info
if [ -f coverage.json ]; then cp coverage.json cov_info/; fi if [ -f coverage.json ]; then cp coverage.json cov_info/; fi
if [ -f coverage.xml ]; then cp coverage.xml cov_info/; fi if [ -f coverage.xml ]; then cp coverage.xml cov_info/; fi
if [ -d htmlcov ]; then cp -r htmlcov cov_info/; fi if [ -d htmlcov ]; then cp -r htmlcov cov_info/; fi
- name: Final badge status - name: Final badge status
if: always() if: always() && env.PUBLISH == 'true'
run: | run: |
echo "=== FINAL BADGE STATUS ===" echo "=== FINAL BADGE STATUS ==="
echo "Test outcome: ${{ steps.pytest.outcome }}" echo "Test outcome: ${{ steps.pytest.outcome }}"
echo "Docs outcome: ${{ steps.docs.outcome }}" echo "Docs outcome: ${{ steps.docs.outcome }}"
ls -la cov_info/ 2>/dev/null || echo "No cov_info directory"
if [ -f cov_info/coverage.svg ]; then
echo "✅ Test coverage badge: $(ls -lh cov_info/coverage.svg)"
else
echo "❌ Test coverage badge: MISSING"
fi
if [ -f cov_info/coverage-docs.svg ]; then
echo "✅ Docs coverage badge: $(ls -lh cov_info/coverage-docs.svg)"
else
echo "❌ Docs coverage badge: MISSING"
fi
echo "Coverage info directory contents:"
ls -la cov_info/ 2>/dev/null || echo "No cov_info directory found"
- name: Upload coverage artifacts - name: Upload coverage artifacts
if: always() && env.PUBLISH == 'true'
uses: actions/upload-artifact@v3 uses: actions/upload-artifact@v3
with: with:
name: coverage-reports name: coverage-reports
path: | path: cov_info/
cov_info/
- name: Commit badges to badges branch - name: Commit badges to badges branch
if: github.ref == 'refs/heads/master' if: env.PUBLISH == 'true' && github.ref == 'refs/heads/master'
run: | run: |
git config --local user.email "action@gitea.local" git config --local user.email "action@gitea.local"
git config --local user.name "Gitea Action" git config --local user.name "Gitea Action"
# Set the remote URL to use the token
git remote set-url origin https://${{ secrets.PUSH_TOKEN }}@gitea.tourolle.paris/dtourolle/pyWebLayout.git git remote set-url origin https://${{ secrets.PUSH_TOKEN }}@gitea.tourolle.paris/dtourolle/pyWebLayout.git
# Create a new orphan branch for badges (this discards any existing badges branch) # Orphan branch holding only the badges, force-pushed each time
git checkout --orphan badges git checkout --orphan badges
# Remove all files except cov_info
find . -maxdepth 1 -not -name '.git' -not -name 'cov_info' -exec rm -rf {} + 2>/dev/null || true find . -maxdepth 1 -not -name '.git' -not -name 'cov_info' -exec rm -rf {} + 2>/dev/null || true
# Add only the coverage info directory
git add -f cov_info/ git add -f cov_info/
# Always commit (force overwrite)
echo "Force updating badges branch with new coverage data..."
git commit -m "Update coverage badges [skip ci]" git commit -m "Update coverage badges [skip ci]"
git push -f origin badges git push -f origin badges
+303 -188
View File
@@ -1,233 +1,348 @@
# pyWebLayout Architecture: Abstract vs Concrete # pyWebLayout Architecture
This document explains the fundamental architectural separation between **Abstract** and **Concrete** layers in the pyWebLayout library. This document describes how pyWebLayout is organised: the layers, what lives in each,
and the rules that govern how they may depend on one another.
## Overview ## Overview
The pyWebLayout library follows a clear separation between two distinct layers: The library turns markup (HTML/EPUB) into rendered images. That pipeline is split into
layers with a strict dependency direction:
- **Abstract Layer**: Represents the logical structure and content of documents (HTML/EPUB text) ```
- **Concrete Layer**: Handles the spatial rendering and visual representation of content io/readers/ parse markup into document structure
abstract/ what the document says ─┐
↓ ├─ built on core/ + style/
layout/ decide where things go │
↓ │
concrete/ what the pixels are ─┘
PIL.Image
```
This separation provides flexibility, testability, and clean separation of concerns. The central distinction is **abstract vs concrete**:
## Abstract Layer (`pyWebLayout/abstract/`) - **Abstract** — the logical content and structure of a document. A `Paragraph` knows
it contains words in an order; it does not know how wide they are.
- **Concrete** — the spatial realisation of that content. A `Line` knows exactly which
glyphs sit at which pixel offsets on a specific canvas.
The Abstract layer deals with the **logical structure** of documents without concerning itself with how content will be visually rendered. One abstract document produces many concrete renderings: different page sizes, font
scales, and font families all re-run the concrete layer over unchanged abstract content.
That is what makes ereader features like live font scaling and reflow possible.
### Key Components ## Layers
#### `abstract/block.py` ### `core/` — shared foundations
- `Block`: Base class for all block-level content
- `Paragraph`: Represents a logical paragraph containing words
- `Heading`: Represents headings with semantic levels (H1-H6)
- `HList`: Represents ordered/unordered lists
- `Image`: Represents image references
#### `abstract/inline.py` Everything else is built on these. `core/` depends on nothing but `style/`.
- `Word`: Represents individual words with text content and styling information
- Contains methods for hyphenation and text manipulation
- Does **not** handle rendering or spatial layout
#### `abstract/document.py` **`core/base.py`** defines the contracts that make a class abstract or concrete:
- `Document`: Container for the overall document structure
- `Chapter`: Logical grouping of blocks (for books/long documents)
### Characteristics of Abstract Classes | Contract | Kind | Meaning |
|---|---|---|
| `Renderable` | ABC | Has `render()`; produces or draws visual output |
| `Queriable` | ABC | Has `in_object(point)`; can be hit-tested |
| `Layoutable` | ABC | Has `layout()`; arranges its own contents |
| `Interactable` | ABC | Holds a callback invoked on interaction |
| `Geometric` | mixin | `origin` and `size` as numpy arrays |
| `Hierarchical` | mixin | `parent` back-reference |
| `Styleable` | mixin | Carries a style object |
| `FontRegistry` | mixin | Deduplicates `Font` instances across a document |
| `MetadataContainer` | mixin | Key/value metadata with typed accessors |
| `BlockContainer` | mixin | Holds child `Block`s |
| `ContainerAware` | mixin | Knows the container it was added to |
1. **Content-focused**: Store text, structure, and semantic meaning The useful shorthand: **`Renderable` and `Geometric` are the concrete markers.** An
2. **Layout-agnostic**: No knowledge of fonts, pixels, or rendering abstract class that acquires either has crossed the line.
3. **Reusable**: Same content can be rendered in different formats/sizes
4. **Serializable**: Can be saved/loaded without rendering context
### Example: Abstract Word **`core/query.py`** — `QueryResult` and `SelectionRange`: the result types for mapping a
pixel back to content (what was clicked, what text is selected, where it is in the
document).
**`core/highlight.py`** — `Highlight`, `HighlightColor`, `HighlightManager`. Highlights
store both pixel bounds (for drawing) and semantic bounds (word indices, for surviving
a font change).
**`core/cache.py`** — `UsageCache` / `SizedUsageCache`, bounded caches keyed by
`(font, string)` for text measurement and glyph rasterisation. Eviction is by usage
count with periodic aging, not LRU; the module docstring explains why at length. This
exists because a single page issues thousands of measurements over fewer than a
thousand distinct pairs, on hardware where an unbounded cache is not affordable.
**`core/callback_registry.py`** — `CallbackRegistry`, which owns the interactive
elements registered on a page and dispatches to them.
### `style/` — semantic styling and its resolution
This package is itself an instance of the abstract/concrete split, applied to styling:
- **`abstract_style.py`** — `AbstractStyle` (frozen dataclass, hashable) captures
*intent*: `FontFamily.SERIF`, `FontSize.LARGE`, `color="black"`. `AbstractStyleRegistry`
interns them so a document holds one instance per distinct style.
- **`concrete_style.py`** — `RenderingContext` (user preferences, DPI, accessibility
flags, available space) plus `StyleResolver`, which maps `AbstractStyle` +
`RenderingContext``ConcreteStyle` (a resolved font path, a pixel size, an RGB
tuple). `ConcreteStyleRegistry` caches the results.
- **`fonts.py`** — `Font`, the loaded PIL font plus its rendering attributes, and the
bundled DejaVu families (`BundledFont`).
- **`page_style.py`** — `PageStyle`: borders, padding, background, default alignment.
- **`alignment.py`** — the `Alignment` enum.
`AbstractStyle``StyleResolver``ConcreteStyle` is the mechanism by which the same
document renders differently for different readers.
### `abstract/` — document content and structure
Layout-agnostic representations of what the document contains.
**`abstract/block.py`** — block-level content, all deriving from `Block`:
`Paragraph`, `Heading` (with `HeadingLevel`), `Quote`, `CodeBlock`, `HList` (with
`ListStyle`) and `ListItem`, `Table` / `TableRow` / `TableCell`, `Image` and
`LinkedImage`, `HorizontalRule`, `PageBreak`.
**`abstract/inline.py`** — `Word`, `FormattedSpan`, `LinkedWord`, `LineBreak`. `Word`
holds text, a style, and previous/next links into the document's word sequence. It can
report whether it is a hyphenation candidate (`possible_hyphenation(language)`), but it
does not perform the split — that is a measurement decision and belongs to `Line`.
**`abstract/document.py`** — `Document`, `Chapter`, `Book`, `MetadataType`. `Book` is
the EPUB-shaped `Document` with chapters and a table of contents.
**`abstract/functional.py`** — `Link`, `Button`, `Form`, `FormField`, `LinkType`,
`FormFieldType`.
**`abstract/interactive_image.py`** — `InteractiveImage`, an `Image` that is also
`Interactable` and `Queriable`.
### `concrete/` — spatial realisation
Objects that know their position and size and can draw themselves onto a canvas.
**`concrete/text.py`** — the heart of text layout:
- `Text` — one renderable fragment (a whole word, or a hyphenated part). Requires an
`ImageDraw.Draw` at construction so it can measure itself immediately.
- `Line` — a sequence of `Text` objects with resolved spacing and a baseline.
`Line.add_word()` is where a `Word` becomes one or two `Text` objects: it measures,
and if the word overflows it tries dictionary hyphenation, then brute-force splitting,
before rejecting the word.
- `AlignmentHandler` and its subclasses (`LeftAlignmentHandler`,
`CenterRightAlignmentHandler`, `JustifyAlignmentHandler`) — the strategy objects that
turn a set of measured fragments plus an available width into concrete spacing and a
start position.
- Cache management: `configure_text_caches`, `clear_text_caches`, `text_cache_stats`,
`prewarm_text_caches`.
**`concrete/page.py`** — `Page`: a fixed-size canvas holding `Renderable` children, with
a content rectangle derived from its `PageStyle`, `can_fit_line()` for the layouter to
test against, `render()` returning a `PIL.Image`, and `query_point()` / `query_range()`
for hit-testing.
**`concrete/box.py`** — `Box`, the `Geometric` + `Renderable` + `Queriable` base for
positioned drawable objects.
**`concrete/dynamic_page.py`** — `DynamicPage` (a `Page` subclass) and `SizeConstraints`.
Adds a two-phase measure-then-layout protocol, so a container such as a table can learn
its content's intrinsic size before committing to a size allocation.
**`concrete/image.py`** — `RenderableImage`.
**`concrete/table.py`** — `TableRenderer`, `TableRowRenderer`, `TableCellRenderer`,
`TableStyle`. Cells host their own nested `Page`, which is why `Page` accepts a
non-zero `origin`.
**`concrete/functional.py`** — `LinkText`, `ButtonText`, `FormFieldText`: `Text`
subclasses that are also `Interactable`.
**`concrete/interaction_handler.py`** — `InteractionHandler`, `InteractionStateManager`:
routing taps and presses to registered elements and tracking pressed/released state.
### `layout/` — the abstract → concrete transformation
This is the package the pipeline diagram calls the layout engine.
**`layout/document_layouter.py`** — a set of layouter functions, one per content kind,
all sharing a signature shape of *(abstract element, target `Page`) → did it fit*:
```python ```python
# An Abstract Word knows its text content and semantic properties paragraph_layouter(paragraph, page, start_word=0, pretext=None, alignment_override=None)
word = Word("supercalifragilisticexpialidocious", font_style) -> (complete: bool, failed_word_index: int | None, remaining_pretext: Text | None)
word.hyphenate() # Logical operation - finds break points
parts = word.get_hyphenated_parts() # Returns ["super-", "cali-", "fragi-", ...] image_layouter(image, page, max_width=None, max_height=None) -> bool
table_layouter(table, page, style=None) -> bool
pagebreak_layouter(page_break, page) -> bool
button_layouter(button, page, font=None, padding=...) -> (bool, str)
form_field_layouter(field, page, font=None, ...) -> ...
form_layouter(form, page, font=None, field_spacing=10) -> (bool, list[str])
``` ```
## Concrete Layer (`pyWebLayout/concrete/`) They **append to a page**, they do not return a list of lines. The three-part return of
`paragraph_layouter` is what makes pagination resumable: when a paragraph runs off the
bottom of a page, the caller learns which word failed and whether a hyphenated fragment
is pending, and can continue on the next page from exactly there.
The Concrete layer handles the **spatial representation** and actual rendering of content. `DocumentLayouter` wraps a `Page` and dispatches over a list of abstract elements by
type, holding the `ConcreteStyleRegistry` for the run.
### Key Components **`layout/ereader_layout.py`** — the paginated reading model:
- `RenderingPosition` — a serialisable cursor expressed in *abstract* coordinates
(chapter, block, word, table cell, list item, pending pretext). Because it names
document structure rather than pixels, it survives font-size and page-size changes.
- `BidirectionalLayouter` — renders a page forward or backward from a position,
returning `(Page, next_position)`.
- `ChapterNavigator` / `ChapterInfo` — a table of contents built from heading structure.
- `FontScaler`, `FontFamilyOverride` — apply scale and family changes to blocks at
layout time without mutating the abstract document.
#### `concrete/text.py` **`layout/page_buffer.py`** — `PageBuffer` (LRU caches of rendered pages in both
- `Text`: Renders a specific text fragment with precise positioning directions, plus position maps) and `BufferedPageRenderer` (background rendering).
- `Line`: Manages a line of `Text` objects with spacing and alignment
- Handles actual pixel measurements, font rendering, and positioning
#### `concrete/page.py` **`layout/ereader_manager.py`** — `EreaderLayoutManager`, the top-level application
- `Page`: Top-level container for rendered content interface (page turns, font changes, chapter jumps, progress), and `BookmarkManager`
- `Container`: Layout manager for organizing renderable objects for persisting bookmarks and the last reading position.
- Handles spatial layout, pagination, and visual composition
#### `concrete/box.py` **`layout/table_optimizer.py`** — column width allocation for tables.
- `Box`: Base class for all spatially-aware renderable objects
- Provides positioning, sizing, and rendering capabilities
### Characteristics of Concrete Classes ### `io/readers/` — parsing
1. **Rendering-focused**: Handle pixels, fonts, images, and visual output **`html_extraction.py`** — `parse_html_string(html, base_font=None, document=None,
2. **Spatially-aware**: Know exact positions, sizes, and layout constraints base_path=None) -> List[Block]`. Built from a `StyleContext` (a `NamedTuple` threaded
3. **Implementation-specific**: Tied to specific rendering technologies (PIL, etc.) down the tree, carrying the inherited font and styling) and a table of per-tag handlers
4. **Non-portable**: Rendering results are tied to specific display contexts (`paragraph_handler`, `heading_handler`, `table_handler`, …). This is the only place
that knows about HTML.
### Example: Concrete Text **`epub_reader.py`** — `EPUBReader` and `read_epub(path) -> Book`: container/OPF
parsing, spine and manifest, table of contents, cover handling, and image processing
(with an e-ink processor available by default).
## Dependency rules
The direction of dependency is what keeps the split honest:
```
io/readers → abstract → core, style
layout → abstract, concrete, core, style
concrete → abstract, core, style
abstract → core, style ← must NOT import concrete
```
`abstract/` importing from `concrete/` is the violation to watch for. `concrete/`
importing from `abstract/` is expected and correct: a `Text` may point back at the
`Word` it came from, and a table cell renderer reads its abstract `TableCell`.
**Where the abstract layer touches rendering today, and why:**
- Abstract classes are constructed with `Font` objects, which carry a pixel size and a
loaded font file. This is a deliberate compromise for parsing performance (HTML
styling resolves to a `Font` once, at parse time) but it does mean the abstract layer
is not fully rendering-independent. `AbstractStyle` is the intended replacement, and
`Word` already accepts either.
- `Word.concrete` / `Word.add_concete()` ([inline.py:42](pyWebLayout/abstract/inline.py#L42),
[:134](pyWebLayout/abstract/inline.py#L134)) is a back-reference to the `Text` objects
a word became. **It is currently written and never read**, and it cannot correctly
model the relationship anyway: one `Word` becomes many `Text`s across re-layouts, and
a single slot only remembers the most recent. Treat it as vestigial. When a
word→pixels mapping is genuinely needed, build it on `Text`'s `source` back-reference
(concrete pointing at abstract), which is the safe direction — note it is currently
stored as `_source` with no public accessor, and is itself unread today.
## Worked example
```python ```python
# A Concrete Text object handles actual rendering from pyWebLayout.io.readers.html_extraction import parse_html_string
text = Text("super-", font) # Specific text fragment from pyWebLayout.concrete.page import Page
text._calculate_dimensions() # Computes exact pixel size from pyWebLayout.layout.document_layouter import DocumentLayouter
image = text.render() # Produces actual visual output
# 1. Parse: markup -> abstract blocks
blocks = parse_html_string("<h1>Chapter One</h1><p>It was a dark and stormy night.</p>")
# [Heading, Paragraph]
# 2. Lay out: abstract blocks -> concrete children appended to a Page
page = Page(size=(400, 600))
DocumentLayouter(page).layout_document(blocks)
# 3. Render: Page -> PIL.Image
image = page.render()
# 4. Query: pixel -> content
result = page.query_point((50, 40))
print(result.object_type, result.text) # e.g. "text" "Chapter"
``` ```
## The Transformation Process Inspecting the intermediate concrete objects:
The architecture involves a clear transformation from Abstract to Concrete:
```
Abstract Document
[Parser Layer]
Abstract Blocks (Paragraph, Heading, etc.)
[Layout Engine]
Concrete Objects (Text, Line, Page)
[Rendering Engine]
Visual Output (Images, PDF, etc.)
```
### Example Transformation
```python ```python
# 1. Abstract content from pyWebLayout.concrete.text import Line
paragraph = Paragraph()
paragraph.add_word(Word("This", font))
paragraph.add_word(Word("is", font))
paragraph.add_word(Word("a", font))
paragraph.add_word(Word("test", font))
# 2. Layout transformation for line in (c for c in page.children if isinstance(c, Line)):
layout = ParagraphLayout(line_width=200, line_height=20) print([t.text for t in line.text_objects])
lines = layout.layout_paragraph(paragraph) # Returns List[Line] # ['Chapter', 'One']
# ['It', 'was', 'a', 'dark', 'and', 'stormy', 'night.']
# 3. Each Line contains concrete Text objects
for line in lines:
for text_obj in line.text_objects: # List[Text]
print(f"Text: '{text_obj.text}' at position {text_obj._origin}")
``` ```
## Key Architectural Principles For paginated reading, drive `EreaderLayoutManager` instead of building pages by hand:
### 1. **Single Responsibility**
- Abstract classes: Handle content and structure
- Concrete classes: Handle rendering and layout
### 2. **Separation of Concerns**
- Text parsing/processing ≠ Text rendering
- Document structure ≠ Page layout
- Content semantics ≠ Visual presentation
### 3. **Immutable Abstract Content**
- Abstract content remains unchanged during rendering
- Multiple concrete representations can be generated from same abstract content
- Enables pagination, different formats, responsive layouts
### 4. **One-to-Many Relationships**
- One Abstract Word → Multiple Concrete Text objects (hyphenation)
- One Abstract Paragraph → Multiple Concrete Lines
- One Abstract Document → Multiple Concrete Pages
## Common Anti-Patterns to Avoid
### ❌ **Mixing Concerns**
```python ```python
# WRONG: Abstract class knowing about pixels from pyWebLayout.layout.ereader_manager import EreaderLayoutManager
manager = EreaderLayoutManager(blocks, page_size=(800, 600), document_id="my-book")
page = manager.get_current_page()
page = manager.next_page()
manager.set_font_scale(1.25) # re-lays out from the same RenderingPosition
```
## Design principles
**1. Abstract content is not mutated by layout.** Font scaling and family overrides
produce scaled copies at layout time rather than editing the document. This is what
allows the same `blocks` list to back a buffer of pages at several font sizes.
**2. Positions are expressed in abstract coordinates.** `RenderingPosition` names a
chapter, block, and word — never a pixel. A reader who changes font size stays on the
same sentence.
**3. Layouters report partial success.** Every layouter returns whether it fit, and the
text layouter also returns where it stopped. Pagination is built from this rather than
from a separate page-breaking pass.
**4. One abstract object may become many concrete ones.** One `Word` → one or two `Text`
fragments; one `Paragraph` → many `Line`s across many `Page`s; one `Document` → an
unbounded sequence of `Page`s. Any API that assumes a one-to-one mapping will be wrong
at a hyphen or a page boundary.
**5. Measurement is cached, not avoided.** Text width and glyph rasterisation are the
hot path. `core/cache.py` bounds their cost; `prewarm_text_caches` front-loads it for a
known document.
## Anti-patterns
**Concrete state stored on abstract objects.** An abstract object that remembers its
rendered width, position, or `Text` objects is wrong at the second rendering. If a
back-reference is needed, point from concrete to abstract.
```python
# WRONG
class Word: class Word:
def __init__(self, text): def __init__(self, text):
self.text = text self.rendered_width = None # invalidated by any font change
self.rendered_width = None # ❌ Concrete concern in abstract class
# RIGHT
text = Text(word.text, font, draw, source=word) # concrete knows its origin
``` ```
### ❌ **renderable_words Concept** **Treating `Word` as renderable.** Words are abstract; only `Text` draws. There is no
```python `renderable_words` anywhere in the codebase, and there should not be.
# WRONG: Confusing abstract and concrete
line.renderable_words # ❌ This suggests Words are renderable
# Words are abstract - only Text objects render
```
### ✅ **Correct Separation** **Assuming a layouter returns lines.** `paragraph_layouter` appends to a page and
```python reports what did not fit. Code that expects `List[Line]` back is working from an
# CORRECT: Clear separation outdated model.
abstract_word = Word("test") # Abstract content
concrete_text = Text("test", font) # Concrete rendering
line.text_objects.append(concrete_text) # Concrete objects in concrete container
```
## Benefits of This Architecture ## Summary
### 1. **Flexibility** - **`core/`** — contracts and shared machinery
- Same content can be rendered at different sizes - **`style/`** — semantic style, and its resolution to concrete rendering parameters
- Multiple output formats from single source - **`abstract/`** — what the document says
- Easy to implement responsive design - **`concrete/`** — where the pixels go
- **`layout/`** — the transformation between them, and pagination on top of it
### 2. **Testability** - **`io/readers/`** — markup in
- Abstract logic can be tested without rendering
- Layout algorithms can be tested independently
- Visual rendering can be mocked
### 3. **Performance**
- Abstract content can be cached and reused
- Layout can be computed once for multiple renderings
- Incremental updates possible
### 4. **Maintainability**
- Clear boundaries between text processing and rendering
- Changes to rendering don't affect content parsing
- Easy to swap rendering backends
## File Organization
```
pyWebLayout/
├── abstract/ # Content and structure
│ ├── block.py # Document blocks (Paragraph, Heading, etc.)
│ ├── inline.py # Inline content (Word, etc.)
│ ├── document.py # Document structure
│ └── functional.py # Links, buttons, etc.
├── concrete/ # Rendering and layout
│ ├── text.py # Text and Line rendering
│ ├── page.py # Page layout and containers
│ ├── box.py # Base rendering classes
│ ├── image.py # Image rendering
│ └── functional.py # Interactive elements
├── typesetting/ # Layout algorithms
│ ├── paragraph_layout.py # Abstract → Concrete transformation
│ ├── flow.py # Text flow management
│ └── pagination.py # Page breaking logic
└── style/ # Styling and formatting
├── fonts.py # Font management
├── layout.py # Layout constants
└── alignment.py # Alignment enums
```
## Conclusion
The Abstract/Concrete separation is fundamental to pyWebLayout's design. It ensures clean separation between content processing and visual rendering, enabling flexible, maintainable, and testable document processing pipelines.
**Remember**:
- **Abstract** = What to display (content, structure, semantics)
- **Concrete** = How to display it (pixels, fonts, positioning, rendering)
This architecture enables the library to handle complex document layouts while maintaining clear, understandable code organization.
+80
View File
@@ -0,0 +1,80 @@
# CI test image for pyWebLayout
# Build: docker build -f Dockerfile.ci -t gitea.tourolle.paris/dtourolle/pyweblayout-ci:latest .
# Push: docker push gitea.tourolle.paris/dtourolle/pyweblayout-ci:latest
#
# pyWebLayout is a library, so CI tests every interpreter pyproject.toml claims
# to support rather than just one. All four are in this image and the workflow
# matrix picks one per job; dependencies are pre-installed into each, so a CI
# run downloads nothing.
FROM ubuntu:24.04
ENV DEBIAN_FRONTEND=noninteractive
# deadsnakes carries the Python versions Ubuntu 24.04 does not ship
RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates \
gnupg \
software-properties-common \
&& add-apt-repository -y ppa:deadsnakes/ppa \
&& apt-get update && apt-get install -y --no-install-recommends \
# Interpreters. 3.12 is Ubuntu 24.04's own; the rest come from deadsnakes.
python3.10 python3.10-venv \
python3.11 python3.11-venv \
python3.12 python3.12-venv \
python3.13 python3.13-venv \
# Pillow needs these at runtime for font rasterisation and image IO
libfreetype6 \
libjpeg-turbo8 \
libopenjp2-7 \
libtiff6 \
zlib1g \
# Used by the workflow itself
curl \
git \
nodejs \
&& rm -rf /var/lib/apt/lists/*
# One venv per interpreter at a predictable path, /opt/py<version>. Ubuntu marks
# its system Python externally-managed, so installing into venvs sidesteps that
# without --break-system-packages, and keeps the four dependency sets isolated.
#
# The package list is written once so versions cannot drift between
# interpreters. It mirrors pyproject.toml's runtime deps plus the test and dev
# extras; keep the two in step.
#
# Deliberately NOT installed: pyWebLayout itself. The workflow installs the
# checkout with --no-deps, so a job always tests the code under review.
#
# setuptools is pinned below 81 because that release dropped pkg_resources,
# which coverage-badge still imports at startup. Without the pin the badge step
# dies with ModuleNotFoundError. Revisit when coverage-badge stops using it.
RUN for v in 3.10 3.11 3.12 3.13; do \
python$v -m venv /opt/py$v && \
/opt/py$v/bin/pip install --no-cache-dir --upgrade pip wheel && \
/opt/py$v/bin/pip install --no-cache-dir --upgrade "setuptools<81" && \
/opt/py$v/bin/pip install --no-cache-dir \
Pillow \
numpy \
pyphen \
beautifulsoup4 \
lxml \
pytest \
pytest-cov \
flask \
werkzeug \
ebooklib \
requests \
flake8 \
coverage-badge \
interrogate \
; \
done
# Fail the build rather than ship an image whose dependencies do not import
RUN for v in 3.10 3.11 3.12 3.13; do \
echo "--- python$v ---" && \
/opt/py$v/bin/python -c \
"import sys, PIL, numpy, pyphen, bs4, pytest, flask, ebooklib, requests; \
print(sys.version.split()[0], 'deps OK')" \
; \
done
+27
View File
@@ -231,6 +231,33 @@ current = manager.get_font_family()
- **[pyWebLayout/layout/README_EREADER_API.md](pyWebLayout/layout/README_EREADER_API.md)** - EbookReader API reference - **[pyWebLayout/layout/README_EREADER_API.md](pyWebLayout/layout/README_EREADER_API.md)** - EbookReader API reference
- **API Reference** - See docstrings in source code - **API Reference** - See docstrings in source code
## Continuous integration
CI runs in a prebuilt container image rather than installing dependencies per
job. The image carries Python 3.10, 3.11, 3.12 and 3.13, each in its own venv at
`/opt/py<version>` with every dependency installed, so a run downloads nothing
and the test matrix covers the whole range `pyproject.toml` claims to support.
Rebuild and push the image whenever `Dockerfile.ci` changes — most often
because a dependency was added to `pyproject.toml`:
```bash
docker build -f Dockerfile.ci -t gitea.tourolle.paris/dtourolle/pyweblayout-ci:latest .
docker push gitea.tourolle.paris/dtourolle/pyweblayout-ci:latest
```
To reproduce a CI job locally:
```bash
docker run --rm -v "$PWD:/src:ro" gitea.tourolle.paris/dtourolle/pyweblayout-ci:latest bash -c '
mkdir -p /work && cp -a /src/. /work/ && cd /work && rm -rf venv .git
/opt/py3.13/bin/pip install -e . --no-deps -q
/opt/py3.13/bin/python -m pytest tests/ -q'
```
The workflow is [.gitea/workflows/ci.yml](.gitea/workflows/ci.yml). Badges and
coverage artifacts are published from the 3.13 leg only.
## License ## License
MIT License MIT License
+595
View File
@@ -0,0 +1,595 @@
# Architecture Review
Independent review of the codebase at `c5c61a3` (2026-08-06), answering one
question: **is this a well-architected library or an over-complex mess?**
It is a well-architected library with one rotten subsystem inside it. The core
design holds up; roughly a fifth of the code is speculative or non-functional,
and it is concentrated in the ereader pagination/buffering layer.
This document records the verdict, the evidence, and the findings **not already
covered** by [LAYOUT_REMEDIATION_SPEC.md](LAYOUT_REMEDIATION_SPEC.md). Where a
finding is already specced, it is cross-referenced rather than restated.
## Contents
| ID | Finding | Severity | Status |
|----|---------|----------|--------|
| [R1](#r1--the-process-pool-crashes-on-python-314) | The process pool crashes on Python 3.14 | Critical | New; raises priority of S12 |
| [R2](#r2--the-test-suite-hangs-at-interpreter-exit) | Test suite hangs at interpreter exit | High | New; same root cause as R1 |
| [R3](#r3--font-scaling-destroys-hyperlinks) | Font scaling destroys hyperlinks | High | New |
| [R4](#r4--three-packaging-configs-that-disagree) | Three packaging configs that disagree | Medium | New (corrected) |
| [R5](#r5--monkey-patched-page-methods-with-a-conflicting-signature) | Monkey-patched `Page` methods with a conflicting signature | Medium | New |
| [R6](#r6--dead-duck-typing-cluster-in-pagepy) | Dead duck-typing cluster in `page.py` | Low | New; extends S10.3 |
| [R7](#r7--two-orphaned-subsystems) | Two orphaned subsystems | Low | New |
| [R8](#r8--backward-pagination-is-guesswork) | Backward pagination is guesswork | Medium | Resolved by S16 |
| [R9](#r9--query_points-hit-region-is-offset-from-the-glyphs) | `query_point`'s hit region is offset from the glyphs | Medium | New, open |
---
## Verdict
**Well architected.** The concerns that usually decide this question are all on
the right side of the line:
- **The abstract/concrete split is real, not aspirational.** Verified
empirically: `abstract/` never imports `concrete/`; `core/` imports nothing but
itself. The single crossing ([document.py:7](../pyWebLayout/abstract/document.py#L7))
is into `style/`, which the dependency rules permit. Most codebases claiming
this layering have leaked it within a year.
- **The layouter contract is the right abstraction.** `paragraph_layouter`
returning `(fit, failed_word_index, remaining_pretext)` is what makes
pagination resumable, and the shape is consistent across content types. Layout
engines that return `List[Line]` cannot paginate without a second pass.
- **[core/cache.py](../pyWebLayout/core/cache.py) is exemplary.** Usage-ranked
eviction with periodic aging, sampled eviction instead of a maintained heap,
O(1) hit path with no reordering — every choice justified by a measurement in
the docstring.
- **[concrete/text.py](../pyWebLayout/concrete/text.py) is the strongest file.**
The glyph cache ([:581-647](../pyWebLayout/concrete/text.py#L581-L647))
reimplements PIL's internals to skip per-call setup, with a permanent graceful
fallback when the private API is absent. Alignment is a clean strategy pattern,
and `render_alignment_handler` handles last-line-of-paragraph correctly.
- **[html_extraction.py](../pyWebLayout/io/readers/html_extraction.py) is
textbook.** An immutable `StyleContext` threaded down the tree plus a handler
dispatch table — no giant if/elif, no mutable parser state.
**The complexity that is not earned** is concentrated in three places, all in the
same band of the code:
1. `layout/page_buffer.py` — 520 lines of multiprocess prefetch that has never
worked (S12, plus R1/R2 below).
2. `BidirectionalLayouter.render_page_backward` — ~100 lines of convergence
heuristics standing in for an anchor list that already exists (R8).
3. The second block dispatcher in `ereader_layout.py`, which silently drops
tables and lists (S4, S8).
The pattern is visible in the git history: work from `e000068` onward (caching,
alignment, page geometry) is markedly better than the ereader scaffolding it sits
on. This is not a mess. It is a solid library with an early prototype still
embedded in it.
**Sizing the cleanup:** R1R7 plus S12 and S10.3 remove roughly 800900 lines and
fix four user-visible defects. None of it requires redesigning anything.
## Test baseline
At `c5c61a3`, in a clean venv on Python 3.14.6:
```
833 passed, 2 skipped, 24 subtests passed in 11.51s
```
(The 2 skips were environmental — the review venv lacked `requests`, so the URL
image tests skipped. With the `test` extra from R4 installed the suite reports
`853 passed, 24 subtests passed in 13.53s`.)
The suite then **hangs indefinitely** rather than exiting. See R2.
## Status of the existing remediation spec
| Spec | Subject | State |
|------|---------|-------|
| S1 | Inline content in non-paragraph containers | Done (`284d521`) |
| S2 | Page geometry: origin and content rect | Done (`f18cec2`) |
| S3 | Draw/canvas lifecycle | Done (`202dacf`) |
| S11 | Partial-block progress discarded | Done (`a57da80`) |
| S13 | Word spacing and alignment | Done (`1262be6`) |
| S14 | Vertical centring in buttons and fields | Done (`c5c61a3`) |
| S4S10, S12 | Dispatch, cells, table grid, pagination, hygiene, background rendering | Outstanding |
The spec's analysis is sound and in places sharper than this review — S12 caught
that `_render_page_worker` omits `page_size` entirely, which this review missed.
Nothing below supersedes it.
---
## R1 — The process pool crashes on Python 3.14
**Severity: critical. Raises S12 from "useless" to "fatal".**
### Problem
`PageBuffer` submits to a `ProcessPoolExecutor` from inside
`BufferedPageRenderer.render_page`
([page_buffer.py:431](../pyWebLayout/layout/page_buffer.py#L431)). Python 3.14
changed the default multiprocessing start method on Linux from `fork` to
`forkserver`. Under a non-`fork` start method, `submit()` reaches
`_check_not_importing_main()`, which raises unless the caller sits inside an
`if __name__ == "__main__":` guard — and the child re-imports the caller's main
module, re-executing it.
S12 documents this subsystem as delivering no benefit. On 3.14 it is worse than
that: `EreaderLayoutManager.get_current_page()` **raises** when called from
module-level script code.
### Evidence
A plain script calling `manager.get_current_page()` at module level, Python
3.14.6:
```
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.
...
File "pyWebLayout/layout/page_buffer.py", line 221, in _queue_forward_renders
future = self.executor.submit(_render_page_worker, args)
ConnectionResetError: [Errno 104] Connection reset by peer
```
With a `__main__` guard added it does not raise, and instead confirms S12's
finding on every job:
```
Background render failed for position RenderingPosition(...): cannot pickle 'ImagingCore' object
Background render failed for position RenderingPosition(...): cannot pickle 'ImagingCore' object
```
### Action
Fold into **S12**, and treat S12 as unblocked and urgent rather than phase 4. The
recommended resolution there — delete the pool, keep the LRU buffers and position
maps, replace prefetch with synchronous readahead — resolves R1 and R2 as a side
effect. S12's measurement gate still applies to the *readahead* decision; it does
not need to gate deletion of the pool, because the pool's contribution is
provably zero.
### Files
`pyWebLayout/layout/page_buffer.py`
---
## R2 — The test suite hangs at interpreter exit
**Severity: high.** Same root cause as R1.
### Problem
`PageBuffer.__del__` calls `shutdown()`, which calls
`executor.shutdown(wait=True)` ([page_buffer.py:342](../pyWebLayout/layout/page_buffer.py#L342)).
`EreaderLayoutManager.__del__` does the same via `renderer.shutdown()`. Running
`__del__` at interpreter shutdown and blocking on a process pool inside it
deadlocks.
### Evidence
```
833 passed, 2 skipped, 24 subtests passed in 11.51s
```
...then the process sat at ~0% CPU with idle forkserver children for 13 minutes
before being killed. Reproduced twice; both runs completed the tests in under
12s and neither exited.
This is why CI wall-clock does not resemble the 11.5s the tests actually take.
### Action
Resolved by S12's deletion of the executor. If for any reason the pool is
retained, `__del__` must not block: register an `atexit` handler or require
explicit `shutdown()`, and never `wait=True` from a finaliser.
### Acceptance criteria
- `pytest` returns to the shell within a second of printing its summary line.
- No `multiprocessing` child processes outlive the test session.
### Files
`pyWebLayout/layout/page_buffer.py`, `pyWebLayout/layout/ereader_manager.py`
---
## R3 — Font scaling destroys hyperlinks
**Severity: high. User-visible, silent, and trivially reproducible.**
### Problem
`BidirectionalLayouter._scale_block_fonts`
([ereader_layout.py:474-498](../pyWebLayout/layout/ereader_layout.py#L474-L498))
rebuilds a scaled block by constructing plain `Word(word.text, scaled_style)` for
every word. `LinkedWord` is a `Word` subclass
([inline.py:288](../pyWebLayout/abstract/inline.py#L288)), so the reconstruction
downgrades it and the link target is discarded.
The function returns the block unchanged when `font_scale == 1.0` and no family
override is set, which is why no test has caught this: the defect only appears
once the reader changes font size.
Two further gaps in the same function:
1. It handles only `Paragraph` and `Heading`. Every other block type is returned
unscaled, so a font-size change leaves images, tables and lists at their
original size while the text around them reflows.
2. It allocates a new `Paragraph` and a new `Word` per word on **every page
render** at any scale ≠ 1.0 — directly against the caching work in
`concrete/text.py`, and on the hot path.
### Evidence
At `c5c61a3`, parsing `<p>Go to <a href="http://x">this link</a> now.</p>`:
```
scale=1.0: LinkedWords = 2
scale=1.5: LinkedWords = 0
```
### Design
Stop reconstructing abstract blocks at layout time. Font scale and family are
*rendering context*, not document content — carrying them in a copied document
violates the "abstract content is not mutated by layout" principle in
[ARCHITECTURE.md](../ARCHITECTURE.md) in spirit, even though it copies rather
than mutates.
Preferred: thread the scale/override into the layouter and resolve fonts at
`Text` construction, where `Font` objects are already deduplicated by
`FontRegistry`. `paragraph_layouter` already accepts an `alignment_override`;
`font_scale` and `font_family_override` belong in the same place.
Minimum viable fix if the larger change is deferred: reconstruct via
`type(word)` and copy subclass state, and extend coverage to every block type.
This is strictly a stopgap — it keeps the per-page allocation cost.
### Acceptance criteria
- A document containing `<a href>` retains every `LinkedWord` after
`set_font_scale(1.5)`, and `query_point` over the rendered page still returns
`object_type="link"` with the correct target.
- An image block's rendered size is unaffected by `set_font_scale`, or scales
deliberately — not left inconsistent with the text around it.
- No new `Word`/`Paragraph` objects are allocated per page render at scale ≠ 1.0.
### Files
`pyWebLayout/layout/ereader_layout.py`, `pyWebLayout/layout/document_layouter.py`
---
## R4 — Three packaging configs that disagree
**Severity: medium.** *Corrected: the original review claimed a clean install
fails on first import. It does not — see below.*
### Problem
The project carries **three** sets of packaging metadata:
| File | Declares |
|------|----------|
| `pyproject.toml` `[project]` | Pillow, numpy, pyphen, beautifulsoup4, flask, ebooklib, requests |
| `setup.cfg` `[options]` | Pillow, numpy |
| `setup.py` `setup(...)` kwargs | Pillow, numpy |
`pyproject.toml`'s `[project]` table wins under any modern build backend, so the
shipped wheel is correct and `pip install pyWebLayout` works. The `setup.cfg` and
`setup.py` copies are dead, contradictory, and actively misleading — reading
either one gives the wrong answer about what the library needs.
The authoritative list is itself wrong in the other direction:
- **`flask` is a runtime dependency.** It is imported only by
`tests/abstract/test_abstract_blocks.py`, as a fixture HTTP server. Every user
installs Flask, Jinja2, Werkzeug, click, itsdangerous and blinker for nothing.
- **`ebooklib` is a runtime dependency and is never imported by the library.**
`epub_reader.py` uses `zipfile` + `xml.etree` directly. Only the *tests* use
ebooklib, to build EPUB fixtures.
- **`requests` is declared required but is optional.** `concrete/image.py:100-111`
imports it lazily and degrades to an error message on the image when absent.
- **`requires-python = ">=3.6"` is false.** The package uses dataclasses (3.7+)
and `from __future__ import annotations` (3.7+); CI tests 3.10, 3.12 and 3.13.
Net effect: a runtime install pulls 7 direct dependencies where 4 are needed.
### Action
- Consolidate on `pyproject.toml`. Reduce `setup.cfg` to its `[flake8]` section
and `setup.py` to a `setup()` shim, each with a comment saying where metadata
lives.
- Runtime deps: Pillow, numpy, pyphen, beautifulsoup4. Move flask, werkzeug,
ebooklib and requests into a `test` extra; add a `remote-images` extra for
requests; add a `dev` extra composing them.
- Set `requires-python = ">=3.10"` to match the CI matrix, and add version
classifiers.
- Add a CI step that installs the package into an empty venv with **only**
declared runtime deps and imports every top-level subpackage. This class of
defect is only caught by installing what you ship — and it is what would have
caught the original misreading.
### Files
`pyproject.toml`, `setup.cfg`, `setup.py`, `.gitea/workflows/ci.yml`
---
## R5 — Monkey-patched `Page` methods with a conflicting signature
**Severity: medium.** Currently inert; a landmine if `Page` is ever refactored.
### Problem
[ereader_layout.py:741-761](../pyWebLayout/layout/ereader_layout.py#L741-L761)
defines `_add_page_methods()` and calls it at import time. It attaches
`can_fit_line` and `available_width` to the `Page` class if they are absent.
`Page` defines both ([page.py:59](../pyWebLayout/concrete/page.py#L59),
[page.py:147](../pyWebLayout/concrete/page.py#L147)), so the patch never fires.
But the two definitions of `can_fit_line` **do not agree**:
| Source | Signature |
|--------|-----------|
| `Page` | `can_fit_line(baseline_spacing, ascent=0, descent=0)` |
| monkey patch | `can_fit_line(line_height)` |
The patched version also ignores descenders entirely — the exact bug S2 fixed. If
`Page.can_fit_line` were ever renamed or moved, this would silently reinstate
pre-S2 clipping behaviour, from an import side effect in a different package.
### Action
Delete `_add_page_methods` and its call site. Import-time monkey-patching of a
class in another module has no place here; if `Page` is missing something the
layout engine needs, it belongs on `Page`.
### Files
`pyWebLayout/layout/ereader_layout.py`
---
## R6 — Dead duck-typing cluster in `page.py`
**Severity: low.** Extends S10.3.
### Problem
[page.py:261-491](../pyWebLayout/concrete/page.py#L261-L491) contains a closed
cluster with no external callers:
- `_get_child_property` (:261) — called only by the four below
- `_get_child_height` (:301) — called by nothing
- `_get_child_position` (:382) — called only by `_point_in_child`
- `_point_in_child` (:435) — called by nothing
- `_get_child_size` (:466) — called only by `_point_in_child`
Verified by grep across `pyWebLayout/`, `tests/`, `examples/` and `scripts/`:
zero references outside the cluster. About 90 lines.
It exists because `Renderable` declares neither `size` nor `origin`, so the code
probes `_size`, `size`, `_height`, `height`, `_origin` and `position` in turn with
`hasattr`. `query_point` (:399) already does the right thing instead — it relies
on the `Queriable` interface.
### Action
- Delete all five methods.
- Add `origin` and `size` to the `Renderable`/`Geometric` contract in
`core/base.py` so the duck-typing cannot grow back. This is the same concern as
S10.1's render contract and can ship with it.
### Files
`pyWebLayout/concrete/page.py`, `pyWebLayout/core/base.py`
---
## R7 — Two orphaned subsystems
**Severity: low**, but they are a large share of the "is this over-complex?"
impression: 559 lines that nothing in the library reaches.
### Problem
**`concrete/interaction_handler.py` (310 lines).** `InteractionHandler` and
`InteractionStateManager` are referenced only by
`examples/07_pressed_state_demo.py`. No library code, no ereader path, no tests.
**`core/highlight.py` (249 lines).** `Highlight`, `HighlightColor` and
`HighlightManager` have tests (`tests/core/test_highlight.py`) but are not wired
into `EreaderLayoutManager` at all. Highlighting is not reachable through the
library's own top-level interface.
`HighlightManager` also duplicates `BookmarkManager`'s JSON persistence
(directory, `_save`, `_load`, per-document file naming) with no shared base.
### Action
Decide per subsystem, and record the decision:
- **Wire it up** — `EreaderLayoutManager` grows `add_highlight` / `highlights_for_page`
and the persistence merges with `BookmarkManager` into one document-state store.
- **Or move it out** — relocate to `examples/` or delete, and drop the tests with it.
Either is fine. Leaving a tested, documented, unreachable subsystem in `core/` is
what makes the library look larger and less coherent than it is.
### Files
`pyWebLayout/concrete/interaction_handler.py`, `pyWebLayout/core/highlight.py`,
`pyWebLayout/layout/ereader_manager.py`
---
## R8 — Backward pagination is guesswork
**Severity: medium.** S11's closing note already flags this for audit; this
records what the audit found.
### Problem
`render_page_backward`
([ereader_layout.py:372-472](../pyWebLayout/layout/ereader_layout.py#L372-L472))
finds the previous page by estimating a start position, rendering forward,
comparing the end against the target, and adjusting — **up to 10 times**. It then
has a fallback that jumps back up to 5 blocks and renders again, and a fallback
for *that* which renders from the start of the document.
Worst case: one "previous page" tap costs up to 12 full page layouts.
The estimator it converges from is `max(1, int(10 / font_scale))` blocks
([:684](../pyWebLayout/layout/ereader_layout.py#L684)) — a constant with no
relationship to page size, block length or font metrics.
The correct answer is usually already known.
`EreaderLayoutManager._page_history` ([ereader_manager.py:210](../pyWebLayout/layout/ereader_manager.py#L210))
records real page-start positions and serves them instantly; the refinement loop
only runs when history misses — after a jump, a bookmark, a font change, or
beyond 50 entries.
S11's note asks whether these fallbacks were compensating for the
discarded-progress bug it fixed. They were, in part: the "failed to move
backward" branch at [:446](../pyWebLayout/layout/ereader_layout.py#L446) is
reachable precisely when forward rendering fails to advance, which S11 addressed.
### Design
Replace convergence with anchors. Maintain a sorted list of known page-start
positions — chapter starts from `ChapterNavigator` (free, already built) plus
every position visited. To go back from position P: binary-search the largest
anchor A < P, render forward from A collecting page starts until reaching P, and
return the last one. Cost is bounded by the anchor spacing, and every page start
discovered on the way is itself a new anchor, so the second traversal of any
region is free.
This subsumes `_page_history`, so the two mechanisms become one.
**Sequencing:** do this after S12, and after S8 — table and list pagination
changes what a page start can be, and re-deriving anchors is cheap only once
positions round-trip through tables correctly (S8 already notes this dependency).
### Acceptance criteria
- `previous_page()` from any position issues at most *k* page layouts, where *k*
is the anchor spacing, with no iteration count and no fallback ladder.
- Forward-then-backward round-trips exactly, from a cold cache, after a chapter
jump, and after a bookmark restore.
- `_estimate_page_start`, `_adjust_start_estimate` and the three-tier fallback
are deleted, not retained alongside.
### Files
`pyWebLayout/layout/ereader_layout.py`, `pyWebLayout/layout/ereader_manager.py`
---
## Status
All findings in this document are resolved. What remains is the existing
remediation spec: **S4 → S5 → S6 → S7 → S8 → S9**, plus **S10.1**, unchanged.
| ID | Resolution | Commit |
|----|-----------|--------|
| R1 | Fixed with S12 — the pool that raised is gone | `1924cc2` |
| R2 | Fixed with S12 — no executor, no blocking finaliser | `1924cc2` |
| R3 | `Word.with_style` keeps subclasses; all container blocks scale | `f0dc675` |
| R4 | Consolidated on `pyproject.toml`; 7 runtime deps → 4 | `767e4c1` |
| R5 | Monkey patch deleted | `62ca151` |
| R6 | 138 dead lines deleted; contract hardening deferred to S10.1 | `e81ba48` |
| R7 | Both subsystems wired into `EreaderLayoutManager` | `8746d3f`, `0ce1aea` |
| R8 | Superseded by S16 (anchor replay); dead estimators removed | `bcae45a` |
| R9 | Open — see below | — |
Two things worth carrying forward:
- **S12's measurement stands as the argument against prefetch.** A page render
is 956 ms. Any future proposal to render ahead should have to beat that
number first.
- **Wiring an orphan found a bug.** R7's interaction handler had a crash on
every hovered or pressed link (`0ce1aea`). Unreachable code is not
neutral — it is untested code that looks tested.
---
## R9 — query_point's hit region is offset from the glyphs
**Severity: medium.** Found while verifying R3; not part of the original review.
### Problem
The region `Page.query_point` reports for a text object does not line up with
where that object says it is. Probing a `LinkText` at the centre of its own
`origin`/`size` box returns `object_type="empty"`.
### Evidence
A single-link page at 400×600, default scale:
```
'this' origin=(68.3, 35.0) size=(29.2, 19.0) centre=(82, 44) -> empty
'link' origin=(102.5, 35.0) size=(28.3, 19.0) centre=(116, 44) -> empty
grid scan: link is detected across y≈2039
LinkText claims: y≈3554
```
The two bands overlap by about four pixels. The offset is close to the font
ascent, which points at a baseline-versus-top mismatch between the coordinates
`Text` stores and the ones `in_object` tests.
This reproduces identically at scale 1.0 and 1.5, so it predates the R3 fix.
### Why it matters
Taps land through the grid because the region is only shifted, not absent — but
it is shifted by most of a line height. Near the top or bottom of a page, or
between tightly spaced lines, a tap can hit the neighbouring line instead of the
one under the finger. It also makes `LinkText.origin`/`size` unusable for
drawing selection or highlight overlays, which is what R7's highlighting now
depends on.
### Action
Establish which of the two is authoritative — almost certainly the drawn
position — and make the other agree. This sits close to S2 (page geometry) and
S3 (draw/canvas lifecycle), both already landed, so the conventions to match
are in place.
### Acceptance criteria
- `page.query_point(centre_of(obj))` returns `obj` for every text object on a
rendered page, at scales 0.8, 1.0, 1.5 and 2.0.
- The end-to-end test in `tests/layout/test_font_scaling.py` probes the centre
directly instead of scanning a grid.
### Files
`pyWebLayout/concrete/text.py`, `pyWebLayout/concrete/page.py`,
`pyWebLayout/core/base.py`
## Reproducing the findings
Reviewed at `c5c61a3` on Python 3.14.6, in a venv containing
`pytest pyphen Pillow numpy beautifulsoup4 lxml ebooklib`.
- **R1**: call `EreaderLayoutManager(...).get_current_page()` from module-level
script code (no `__main__` guard).
- **R2**: `python -m pytest -q`; observe the summary line, then the hang.
- **R3**: parse HTML containing `<a href>`, call
`BidirectionalLayouter._scale_block_fonts(block, 1.5)`, count `LinkedWord`
instances in the result.
- **R5**: compare `inspect.signature(Page.can_fit_line)` against the patch body.
- **R6**: grep the five method names across `pyWebLayout/ tests/ examples/ scripts/`.
- **R8**: read the loop; no execution needed.
+152
View File
@@ -29,6 +29,8 @@ It is independent of every other spec here.
| [S12](#s12--background-rendering) | Background rendering | 4 | | [S12](#s12--background-rendering) | Background rendering | 4 |
| [S13](#s13--word-spacing-and-alignment) | Word spacing and alignment | 0 | | [S13](#s13--word-spacing-and-alignment) | Word spacing and alignment | 0 |
| [S14](#s14--vertical-centring-in-buttons-and-fields) | Vertical centring in buttons and fields | 0 | | [S14](#s14--vertical-centring-in-buttons-and-fields) | Vertical centring in buttons and fields | 0 |
| [S15](#s15--form-field-label-geometry) | Form field label geometry | 0 |
| [S16](#s16--backward-page-navigation) | Backward page navigation | 0 |
## Design invariants ## Design invariants
@@ -1016,6 +1018,20 @@ a full page laid out in the worker — then the result is thrown away by
4-core Pi with 512MB this is actively harmful: four interpreter copies plus four 4-core Pi with 512MB this is actively harmful: four interpreter copies plus four
copies of the book, to populate a cache that never populates. copies of the book, to populate a cache that never populates.
It is also not inert. The pool is started from `PageBuffer.initialize` inside a
process that already has threads, and CPython warns about exactly this:
```
DeprecationWarning: This process (pid=...) is multi-threaded,
use of fork() may lead to deadlocks in the child.
```
`tests/layout/test_ereader_image_rendering.py` intermittently hangs at
interpreter exit as a result — every test reports PASSED, then the process never
returns. Observed roughly one run in four. A reader that hangs on shutdown once
in four launches would be a shipped bug; the test suite is just where it shows
up first. This raises S12 from "wasted work" to "actively harmful".
Four further defects in the same file, which matter only if the decision is to Four further defects in the same file, which matter only if the decision is to
keep it: keep it:
@@ -1211,6 +1227,142 @@ the demo to write straight to `docs/images/` so it cannot drift again.
--- ---
## S15 — Form field label geometry
### Problem
`FormFieldText` treats its origin as the control's top-left: `size` and
`in_object` both measure down from it. But it drew the label by calling
`Text.render` at that origin, and Text anchors on the **baseline**, so the
label's glyphs landed *above* the origin — outside the box the control claims,
on top of whatever was there. In a stacked form that is the previous field's
input box, which is what
`docs/images/example_10_forms.png` showed: every label but the first crowding
and touching the box above it.
The height was also computed as `font_size + 5 + field_height`, understating the
label by the difference between nominal size and ink height, which left the gap
between label and box smaller than the intended 5px.
### Design
- The origin is documented as the top-left of the whole control.
- Rendering offsets the label down by its ascent, so the glyphs occupy
`[origin.y, origin.y + ascent + descent]`.
- `LABEL_GAP` names the 5px gap, and `field_area_offset` gives the distance from
the origin to the top of the input box. `render`, `handle_click` and the height
calculation all derive from it, instead of each recomputing `font_size + 5`.
### Acceptance criteria
- No label ink is drawn above the control's origin.
- All ink lies within `[origin.y, origin.y + size[1]]`.
- Consecutive fields laid out by `form_layouter` do not overlap.
- A click in the input area focuses the field; a click on the label does not.
### Files
`pyWebLayout/concrete/functional.py`
---
## S16 — Backward page navigation
### Problem
`render_page_backward` *searched* for the previous page's start: estimate a block
index, lay out forward, compare the end against the target, bisect on the block
difference, repeat up to ten times. Both the estimator and the adjuster pinned
`word_index` to 0 and moved only `block_index`.
Pages routinely start mid-block. Any such start was therefore **not in the search
space**, the loop could never match, and it fell through to a fallback that
jumped several blocks back or to the document start.
### Evidence
A document of short paragraphs around one 1200-word paragraph. Forward pagination
gives page starts at `(0,0), (2,208), (2,494), (2,780), (2,1057)`. Asking for the
page that ends where each of those begins:
```
from page 1 -> got (0,0) expected (0,0) ok (1 forward layout)
from page 2 -> got (0,0) expected (2,208) WRONG (10 forward layouts)
from page 3 -> got (0,0) expected (2,494) WRONG (10 forward layouts)
from page 4 -> got (0,0) expected (2,780) WRONG (10 forward layouts)
```
Every mid-paragraph case threw the reader to the start of the document after ten
full page layouts. The bisection was also unsound within its own space: a
document of 40 small paragraphs, where every page *does* start on a block
boundary, failed too.
This is complementary to S11 rather than caused by it. Before S11 forward
pagination dead-ended at the first page-spanning block, so mid-block starts were
never produced and the block-granular search looked adequate.
### Design
Pagination is a pure function: laying out from `q` yields a page and the position
it stopped at, `next(q)`. The page before `P` is the `q` with `next(q) == P`.
That is found by **replaying the chain forward from an anchor**, not by guessing
`q`. Three sources, in order:
1. **The recorded chain.** `render_page_forward` now records
`(font_scale, next(q)) -> q`. Stepping back to anywhere the reader has been is
exact and costs one layout. Keyed by font scale, since changing it
repaginates.
2. **Replay from an anchor.** Anchors are block starts, nearest first: the block
containing `P`, then up to `MAX_BACKWARD_ANCHORS` earlier ones, then the
document start. Lay out forward from the anchor until a page ends exactly on
`P`; that page's start is the answer. `MAX_REPLAY_PAGES` caps the walk so one
page turn cannot traverse a whole chapter.
3. **Nearest start before `P`.** If no chain passes exactly through `P` — which
happens when `P` was reached by a jump or a restored bookmark rather than by
reading forward, so it lies on no natural chain — return the last page start
before it. That overlaps `P`'s page slightly rather than skipping content,
which is the safe direction to be wrong in.
The estimator and the bisecting adjuster are deleted.
**What "correct" means here.** Each backward step returns a page ending exactly
where the reader currently is, so paging back never skips or repeats content.
That chain can differ from the one you would have seen reading forward from page
one, if you entered the document by a jump — pagination from a different starting
point is genuinely a different chain, and no algorithm can recover the original
without replaying from the start.
### Measurements
Same document, after the change:
```
warm (chain recorded by the forward pass): 4/4 exact, 1 layout each
cold, fresh layouter per call: 12/13 exact, worst 17 layouts
cold, one layouter, repeated back presses: 4 layouts per turn typical
```
The single inexact case is a target that lies on the canonical chain but not on
any chain reachable from a nearby anchor; it returns a start 15 words early,
i.e. a slightly overlapping page.
### Acceptance criteria
- For every page of a document, `render_page_backward(start[i])` returns
`start[i-1]` — verified for both a mid-paragraph-paginating document and one
where every page starts on a block boundary.
- Laying out forward from the returned position ends exactly on the requested
position.
- Forward-then-back returns to the original position.
- At the document start, backward stays there; an empty document is safe.
- Cost stays within a small bounded number of forward layouts.
### Files
`pyWebLayout/layout/ereader_layout.py`
---
## Test plan ## Test plan
Findings were reproduced with four probe scripts; each becomes a regression test Findings were reproduced with four probe scripts; each becomes a regression test
Binary file not shown.

Before

Width:  |  Height:  |  Size: 31 KiB

After

Width:  |  Height:  |  Size: 31 KiB

+39 -2
View File
@@ -3,12 +3,25 @@ from pyWebLayout.core import Hierarchical
from pyWebLayout.style import Font from pyWebLayout.style import Font
from pyWebLayout.style.abstract_style import AbstractStyle from pyWebLayout.style.abstract_style import AbstractStyle
from typing import Tuple, Union, List, Optional, Dict, Any, Callable from typing import Tuple, Union, List, Optional, Dict, Any, Callable
from functools import lru_cache
import pyphen import pyphen
# Import LinkType for type hints (imported at module level to avoid F821 linting error) # Import LinkType for type hints (imported at module level to avoid F821 linting error)
from pyWebLayout.abstract.functional import LinkType from pyWebLayout.abstract.functional import LinkType
@lru_cache(maxsize=16)
def _hyphen_dict(language: Optional[str]) -> pyphen.Pyphen:
"""
The pyphen dictionary for a language, reused across words.
Pyphen caches the parsed dictionary file itself, but rebuilding the wrapper
per word still costs about 40% of a hyphenation call, and hyphenation is
attempted for every word that overflows its line.
"""
return pyphen.Pyphen(lang=language)
class Word: class Word:
""" """
An abstract representation of a word in a document. Words can be split across An abstract representation of a word in a document. Words can be split across
@@ -163,6 +176,18 @@ class Word:
"""Set the next word in sequence""" """Set the next word in sequence"""
self._next = next_word self._next = next_word
def with_style(self, style: Font) -> 'Word':
"""
Return a copy of this word carrying a different font.
Subclasses that hold extra state must override this, or that state is
silently dropped when a caller restyles the word. Sequence links
(previous/next) are deliberately not copied: the copy belongs to a
different word chain, which the new container rebuilds as words are
added to it.
"""
return Word(self._text, style, self._background)
def possible_hyphenation(self, language: str = None) -> bool: def possible_hyphenation(self, language: str = None) -> bool:
""" """
Hyphenate the word and store the parts. Hyphenate the word and store the parts.
@@ -174,8 +199,7 @@ class Word:
bool: True if the word was hyphenated, False otherwise. bool: True if the word was hyphenated, False otherwise.
""" """
dic = pyphen.Pyphen(lang=self._style.language) return list(_hyphen_dict(self._style.language).iterate(self._text))
return list(dic.iterate(self._text))
... ...
@@ -348,6 +372,19 @@ class LinkedWord(Word):
"""Get the link title/tooltip""" """Get the link title/tooltip"""
return self._title return self._title
def with_style(self, style: Font) -> 'LinkedWord':
"""Return a copy carrying a different font, keeping the link intact."""
return LinkedWord(
self._text,
style,
self._location,
link_type=self._link_type,
callback=self._callback,
background=self._background,
params=dict(self._params),
title=self._title,
)
def execute_link(self, context: Optional[Dict[str, Any]] = None) -> Any: def execute_link(self, context: Optional[Dict[str, Any]] = None) -> Any:
""" """
Execute the link action. Execute the link action.
+56 -16
View File
@@ -99,15 +99,20 @@ class LinkText(Text, Interactable, Queriable):
self._origin, self._origin,
np.ndarray) else self._origin np.ndarray) else self._origin
# Draw background based on state (before text is rendered) # Draw background based on state (before text is rendered).
if self._pressed: # PIL wants a flat sequence of four scalars; handing it a list of two
# Pressed state - stronger, darker highlight # numpy arrays raises "coordinate list must contain exactly 2
bg_color = (180, 180, 255, 180) # Stronger blue with more opacity # coordinates".
self._draw.rectangle([origin, origin + size], fill=bg_color) if self._pressed or self._hovered:
elif self._hovered: far = origin + size
# Hover state - subtle highlight box = (int(origin[0]), int(origin[1]), int(far[0]), int(far[1]))
bg_color = (220, 220, 255, 100) # Light blue with alpha if self._pressed:
self._draw.rectangle([origin, origin + size], fill=bg_color) # Pressed state - stronger, darker highlight
bg_color = (180, 180, 255, 180)
else:
# Hover state - subtle highlight
bg_color = (220, 220, 255, 100)
self._draw.rectangle(box, fill=bg_color)
# Call the parent Text render method with parameters # Call the parent Text render method with parameters
super().render(next_text, spacing) super().render(next_text, spacing)
@@ -297,8 +302,17 @@ class FormFieldText(Text, Interactable, Queriable):
""" """
A Text subclass that can handle FormField interactions. A Text subclass that can handle FormField interactions.
Renders form field labels and input areas. Renders form field labels and input areas.
The origin is the top-left of the whole control: label, then a gap, then the
input box. Text itself draws from a baseline, so the label is offset down by
its ascent when rendering; without that the glyphs would sit above the origin
and overprint whatever is above, which for a stacked form is the previous
field's input box.
""" """
# Vertical gap between the label and its input box, in pixels.
LABEL_GAP = 5
def __init__(self, field: FormField, font: Font, draw: ImageDraw.Draw, def __init__(self, field: FormField, font: Font, draw: ImageDraw.Draw,
field_height: int = 24, source=None, line=None): field_height: int = 24, source=None, line=None):
""" """
@@ -324,8 +338,11 @@ class FormFieldText(Text, Interactable, Queriable):
self._field_height = field_height self._field_height = field_height
self._focused = False self._focused = False
# Calculate total height (label + gap + field) # Calculate total height (label + gap + field). The label's height is its
self._total_height = self._style.font_size + 5 + field_height # ink height, ascender to descender, not the nominal font size - the two
# differ by several pixels and the gap between label and box is only 5.
self._label_height = self._visual_label_height()
self._total_height = self._label_height + self.LABEL_GAP + field_height
# Field width should be at least as wide as the label # Field width should be at least as wide as the label
# Use getattr to handle mock objects in tests # Use getattr to handle mock objects in tests
@@ -334,6 +351,20 @@ class FormFieldText(Text, Interactable, Queriable):
self._width, '__call__') else 0 self._width, '__call__') else 0
self._field_width = max(text_width, 150) self._field_width = max(text_width, 150)
def _visual_label_height(self) -> int:
"""Height of the rendered label, ascender to descender."""
try:
ascent, descent = self._style.font.getmetrics()
return int(ascent + descent)
except (AttributeError, TypeError, ValueError):
# Mock or unusual font object; the nominal size is the best guess.
return int(getattr(self._style, 'font_size', 0) or 0)
@property
def field_area_offset(self) -> int:
"""Distance from this control's origin to the top of its input box."""
return self._label_height + self.LABEL_GAP
@property @property
def field(self) -> FormField: def field(self) -> FormField:
"""Get the associated FormField object""" """Get the associated FormField object"""
@@ -352,12 +383,21 @@ class FormFieldText(Text, Interactable, Queriable):
""" """
Render the form field with label and input area. Render the form field with label and input area.
""" """
# Render the label # Render the label. Text draws from the baseline, so shift down by the
super().render() # ascent to make the origin the top of the label rather than its baseline.
try:
label_ascent = self._style.font.getmetrics()[0]
except (AttributeError, TypeError, ValueError):
label_ascent = self._label_height
# Calculate field position (below label with 5px gap) label_origin = self._origin
self._origin = np.array([label_origin[0], label_origin[1] + label_ascent])
super().render()
self._origin = label_origin
# Calculate field position (below the label, with the standard gap)
field_x = self._origin[0] field_x = self._origin[0]
field_y = self._origin[1] + self._style.font_size + 5 field_y = self._origin[1] + self.field_area_offset
# Draw field background and border # Draw field background and border
bg_color = (255, 255, 255) bg_color = (255, 255, 255)
@@ -404,7 +444,7 @@ class FormFieldText(Text, Interactable, Queriable):
True if the field was clicked and focused True if the field was clicked and focused
""" """
# Calculate field area # Calculate field area
field_y = self._style.font_size + 5 field_y = self.field_area_offset
# Check if click is within the input field area (not just the label) # Check if click is within the input field area (not just the label)
if (0 <= point[0] <= self._field_width and if (0 <= point[0] <= self._field_width and
-138
View File
@@ -258,69 +258,6 @@ class Page(Renderable, Queriable):
"""Get a copy of the children list""" """Get a copy of the children list"""
return self._children.copy() return self._children.copy()
def _get_child_property(self, child: Renderable, private_attr: str,
public_attr: str, index: Optional[int] = None,
default: Optional[int] = None) -> Optional[int]:
"""
Generic helper to extract properties from child objects with multiple fallback strategies.
Args:
child: The child object
private_attr: Name of the private attribute (e.g., '_size')
public_attr: Name of the public property (e.g., 'size')
index: Optional index for array-like properties (0 for width, 1 for height)
default: Default value if property cannot be determined
Returns:
Property value or default
"""
# Try private attribute first
if hasattr(child, private_attr):
value = getattr(child, private_attr)
if value is not None:
if isinstance(value, (list, tuple, np.ndarray)):
if index is not None and len(value) > index:
return int(value[index])
elif index is None:
return value
# Try public property
if hasattr(child, public_attr):
value = getattr(child, public_attr)
if value is not None:
if isinstance(value, (list, tuple, np.ndarray)):
if index is not None and len(value) > index:
return int(value[index])
elif index is None:
return value
else:
return int(value)
return default
def _get_child_height(self, child: Renderable) -> int:
"""
Get the height of a child object.
Args:
child: The child to measure
Returns:
Height in pixels
"""
# Try to get height from size property (index 1)
height = self._get_child_property(child, '_size', 'size', index=1)
if height is not None:
return height
# Try direct height attribute
height = self._get_child_property(child, '_height', 'height')
if height is not None:
return height
# Default fallback height
return 20
def render_children(self): def render_children(self):
""" """
Call render on all children in the list. Call render on all children in the list.
@@ -379,23 +316,6 @@ class Page(Renderable, Queriable):
return canvas return canvas
def _get_child_position(self, child: Renderable) -> Tuple[int, int]:
"""
Get the position where a child should be rendered.
Args:
child: The child object
Returns:
Tuple of (x, y) coordinates
"""
# Try to get x coordinate
x = self._get_child_property(child, '_origin', 'position', index=0, default=0)
# Try to get y coordinate
y = self._get_child_property(child, '_origin', 'position', index=1, default=0)
return (x, y)
def query_point(self, point: Tuple[int, int]) -> Optional[QueryResult]: def query_point(self, point: Tuple[int, int]) -> Optional[QueryResult]:
""" """
Query a point to find the deepest object at that location. Query a point to find the deepest object at that location.
@@ -432,64 +352,6 @@ class Page(Renderable, Queriable):
bounds=(int(point[0]), int(point[1]), 0, 0) bounds=(int(point[0]), int(point[1]), 0, 0)
) )
def _point_in_child(self, point: np.ndarray, child: Renderable) -> bool:
"""
Check if a point is within a child's bounds.
Args:
point: The point to check
child: The child to check against
Returns:
True if the point is within the child's bounds
"""
# If child implements Queriable interface, use it
if isinstance(child, Queriable) and hasattr(child, 'in_object'):
try:
return child.in_object(point)
except BaseException:
pass # Fall back to bounds checking
# Get child position and size for bounds checking
child_pos = self._get_child_position(child)
child_size = self._get_child_size(child)
if child_size is None:
return False
# Check if point is within child bounds
return (
child_pos[0] <= point[0] < child_pos[0] + child_size[0] and
child_pos[1] <= point[1] < child_pos[1] + child_size[1]
)
def _get_child_size(self, child: Renderable) -> Optional[Tuple[int, int]]:
"""
Get the size of a child object.
Args:
child: The child to measure
Returns:
Tuple of (width, height) or None if size cannot be determined
"""
# Try to get width and height from size property
width = self._get_child_property(child, '_size', 'size', index=0)
height = self._get_child_property(child, '_size', 'size', index=1)
# If size property worked, return it
if width is not None and height is not None:
return (width, height)
# Try direct width/height attributes
width = self._get_child_property(child, '_width', 'width')
height = self._get_child_property(child, '_height', 'height')
if width is not None and height is not None:
return (width, height)
return None
def _make_query_result(self, obj, point: Tuple[int, int]) -> QueryResult: def _make_query_result(self, obj, point: Tuple[int, int]) -> QueryResult:
""" """
Package an object into a QueryResult with metadata. Package an object into a QueryResult with metadata.
+169 -53
View File
@@ -58,6 +58,15 @@ _width_cache: UsageCache = UsageCache(DEFAULT_WIDTH_CACHE_ENTRIES)
_glyph_cache: SizedUsageCache = SizedUsageCache(DEFAULT_GLYPH_CACHE_BYTES, _glyph_entry_bytes) _glyph_cache: SizedUsageCache = SizedUsageCache(DEFAULT_GLYPH_CACHE_BYTES, _glyph_entry_bytes)
_glyph_subpixel_steps: int = DEFAULT_GLYPH_SUBPIXEL_STEPS _glyph_subpixel_steps: int = DEFAULT_GLYPH_SUBPIXEL_STEPS
# Every Line asks its font for the advance width of a space. That single
# FreeTypeFont.getlength(" ") call costs ~18us -- two orders of magnitude more
# than getmetrics() -- because PIL shapes the string from scratch each time, and
# it lands once per line created, which dominates the cost of laying a line out.
# There are only ever a handful of distinct fonts in play, so memoise per font
# object. Values are wrapped in a 1-tuple because None is itself a legitimate
# result (fonts that cannot report a length) and must not read as a cache miss.
_space_advance_cache: Dict[Any, Tuple[Optional[int]]] = {}
# Set to False the first time the fast rasterisation path is found to be # Set to False the first time the fast rasterisation path is found to be
# unavailable (e.g. a PIL build without the private ImageDraw internals it uses), # unavailable (e.g. a PIL build without the private ImageDraw internals it uses),
# after which every Text falls back to ImageDraw.text(). # after which every Text falls back to ImageDraw.text().
@@ -98,6 +107,35 @@ def clear_text_caches():
"""Drop all cached widths and glyph bitmaps.""" """Drop all cached widths and glyph bitmaps."""
_width_cache.clear() _width_cache.clear()
_glyph_cache.clear() _glyph_cache.clear()
_space_advance_cache.clear()
def _space_advance(font) -> Optional[int]:
"""
The font's own advance width for a space, in whole pixels.
None when the font cannot report one, which is the signal for callers to fall
back to their configured spacing range.
"""
try:
cached = _space_advance_cache.get(font)
except TypeError:
# Unhashable font object; measure without caching.
cached = None
else:
if cached is not None:
return cached[0]
try:
value = int(round(font.getlength(" ")))
except (AttributeError, TypeError, ValueError):
value = None
try:
_space_advance_cache[font] = (value,)
except TypeError:
pass
return value
def text_cache_stats() -> Dict[str, Any]: def text_cache_stats() -> Dict[str, Any]:
@@ -215,7 +253,8 @@ class AlignmentHandler(ABC):
def calculate_spacing_and_position(self, text_objects: List['Text'], def calculate_spacing_and_position(self, text_objects: List['Text'],
available_width: int, min_spacing: int, available_width: int, min_spacing: int,
max_spacing: int, max_spacing: int,
natural_spacing: Optional[int] = None natural_spacing: Optional[int] = None,
total_width: Optional[float] = None
) -> Tuple[int, int, bool]: ) -> Tuple[int, int, bool]:
""" """
Calculate the spacing between words and starting position for the line. Calculate the spacing between words and starting position for the line.
@@ -228,6 +267,11 @@ class AlignmentHandler(ABC):
natural_spacing: The font's own space width. Ragged alignments use it natural_spacing: The font's own space width. Ragged alignments use it
as a constant gap; justification ignores it. Defaults to as a constant gap; justification ignores it. Defaults to
min_spacing when not supplied. min_spacing when not supplied.
total_width: The summed width of `text_objects`, when the caller
already knows it. Purely an optimisation: a line asks its handler
to re-measure once per candidate word, and summing the whole line
each time makes filling a line quadratic in its word count. Omit
it and the sum is taken here as before.
Returns: Returns:
Tuple of (spacing_between_words, starting_x_position, overflow) Tuple of (spacing_between_words, starting_x_position, overflow)
@@ -242,7 +286,8 @@ class LeftAlignmentHandler(AlignmentHandler):
available_width: int, available_width: int,
min_spacing: int, min_spacing: int,
max_spacing: int, max_spacing: int,
natural_spacing: Optional[int] = None natural_spacing: Optional[int] = None,
total_width: Optional[float] = None
) -> Tuple[int, int, bool]: ) -> Tuple[int, int, bool]:
""" """
Calculate spacing and position for left-aligned text objects. Calculate spacing and position for left-aligned text objects.
@@ -269,7 +314,8 @@ class LeftAlignmentHandler(AlignmentHandler):
spacing = min_spacing if natural_spacing is None else natural_spacing spacing = min_spacing if natural_spacing is None else natural_spacing
spacing = max(min_spacing, min(max_spacing, int(spacing))) spacing = max(min_spacing, min(max_spacing, int(spacing)))
text_length = sum([text.width for text in text_objects]) text_length = (sum([text.width for text in text_objects])
if total_width is None else total_width)
num_gaps = len(text_objects) - 1 num_gaps = len(text_objects) - 1
# The spacing is constant whether or not the content fits: tightening a # The spacing is constant whether or not the content fits: tightening a
@@ -290,7 +336,8 @@ class CenterRightAlignmentHandler(AlignmentHandler):
def calculate_spacing_and_position(self, text_objects: List['Text'], def calculate_spacing_and_position(self, text_objects: List['Text'],
available_width: int, min_spacing: int, available_width: int, min_spacing: int,
max_spacing: int, max_spacing: int,
natural_spacing: Optional[int] = None natural_spacing: Optional[int] = None,
total_width: Optional[float] = None
) -> Tuple[int, int, bool]: ) -> Tuple[int, int, bool]:
""" """
Centre/right alignment: constant word space, line shifted as a block. Centre/right alignment: constant word space, line shifted as a block.
@@ -300,7 +347,8 @@ class CenterRightAlignmentHandler(AlignmentHandler):
the same spacing that will actually be used, so the line lands where it the same spacing that will actually be used, so the line lands where it
was measured to land. was measured to land.
""" """
word_length = sum([word.width for word in text_objects]) word_length = (sum([word.width for word in text_objects])
if total_width is None else total_width)
# Handle single word case # Handle single word case
if len(text_objects) <= 1: if len(text_objects) <= 1:
@@ -329,13 +377,42 @@ class JustifyAlignmentHandler(AlignmentHandler):
"""Handler for justified text with full justification.""" """Handler for justified text with full justification."""
def __init__(self): def __init__(self):
# Store variable spacing for each gap to distribute remainder pixels # The per-gap spacings are described by a plan rather than stored outright,
self._gap_spacings: List[int] = [] # and materialised on demand by the _gap_spacings property below. Fitting a
# line calls this handler once per candidate word and only ever looks at the
# first gap; building the whole list on each of those probes made adding n
# words to a line O(n^2). Only render() reads the full list.
self._gap_uniform: Optional[int] = None
self._gap_residual: int = 0
self._gap_count: int = 0
self._gap_cache: Optional[List[int]] = []
@property
def _gap_spacings(self) -> List[int]:
"""The spacing to apply at each gap, left to right."""
if self._gap_cache is None:
if self._gap_uniform is not None:
self._gap_cache = [self._gap_uniform] * self._gap_count
else:
self._gap_cache = self._distribute(self._gap_residual, self._gap_count)
return self._gap_cache
@staticmethod
def _distribute(total: int, num_gaps: int) -> List[int]:
"""Split `total` pixels across `num_gaps` gaps by cumulative rounding."""
gaps = []
placed = 0
for i in range(1, num_gaps + 1):
cumulative = int(round(total * i / num_gaps))
gaps.append(cumulative - placed)
placed = cumulative
return gaps
def calculate_spacing_and_position(self, text_objects: List['Text'], def calculate_spacing_and_position(self, text_objects: List['Text'],
available_width: int, min_spacing: int, available_width: int, min_spacing: int,
max_spacing: int, max_spacing: int,
natural_spacing: Optional[int] = None natural_spacing: Optional[int] = None,
total_width: Optional[float] = None
) -> Tuple[int, int, bool]: ) -> Tuple[int, int, bool]:
""" """
Justified alignment distributes space to fill the entire line width. Justified alignment distributes space to fill the entire line width.
@@ -347,14 +424,17 @@ class JustifyAlignmentHandler(AlignmentHandler):
is min_spacing to ensure readability. is min_spacing to ensure readability.
""" """
word_length = sum([word.width for word in text_objects]) word_length = (sum([word.width for word in text_objects])
if total_width is None else total_width)
residual_space = available_width - word_length residual_space = available_width - word_length
num_gaps = max(1, len(text_objects) - 1) num_gaps = max(1, len(text_objects) - 1)
# Check if we have enough space for minimum spacing # Check if we have enough space for minimum spacing
if residual_space // num_gaps < min_spacing: if residual_space // num_gaps < min_spacing:
# Not enough space - this is overflow # Not enough space - this is overflow
self._gap_spacings = [min_spacing] * num_gaps self._gap_uniform = min_spacing
self._gap_count = num_gaps
self._gap_cache = None
return min_spacing, 0, True return min_spacing, 0, True
# Distribute the residual by cumulative rounding rather than by taking a # Distribute the residual by cumulative rounding rather than by taking a
@@ -365,14 +445,14 @@ class JustifyAlignmentHandler(AlignmentHandler):
# ragged right edge on otherwise justified text. Rounding the running # ragged right edge on otherwise justified text. Rounding the running
# total makes the gaps sum to the residual exactly. # total makes the gaps sum to the residual exactly.
total = int(round(residual_space)) total = int(round(residual_space))
self._gap_spacings = [] self._gap_uniform = None
placed = 0 self._gap_residual = total
for i in range(1, num_gaps + 1): self._gap_count = num_gaps
cumulative = int(round(total * i / num_gaps)) self._gap_cache = None
self._gap_spacings.append(cumulative - placed)
placed = cumulative
return self._gap_spacings[0], 0, False # The first gap is the whole of the plan that fitting needs, and it falls
# out of the same cumulative rounding as _distribute would give it.
return int(round(total / num_gaps)), 0, False
class Text(Renderable, Queriable): class Text(Renderable, Queriable):
@@ -689,6 +769,9 @@ class Line(Box):
""" """
super().__init__(origin, size, callback, sheet, mode, halign, valign) super().__init__(origin, size, callback, sheet, mode, halign, valign)
self._text_objects: List['Text'] = [] # Store Text objects directly self._text_objects: List['Text'] = [] # Store Text objects directly
# Prefix sums of the widths in _text_objects, kept in step by _push_text /
# _pop_text. Element 0 is the empty sum. See _push_text for the rationale.
self._width_prefix: List[float] = [0.0]
self._spacing = spacing # (min_spacing, max_spacing) self._spacing = spacing # (min_spacing, max_spacing)
self._font = font if font else Font() # Use default font if none provided self._font = font if font else Font() # Use default font if none provided
self._current_width = 0 # Track the current width used self._current_width = 0 # Track the current width used
@@ -704,10 +787,7 @@ class Line(Box):
# The font's own space advance. Ragged alignments use this as their # The font's own space advance. Ragged alignments use this as their
# constant word gap rather than stretching to fill the measure. # constant word gap rather than stretching to fill the measure.
try: self._natural_spacing = _space_advance(self._font.font)
self._natural_spacing = int(round(self._font.font.getlength(" ")))
except (AttributeError, TypeError, ValueError):
self._natural_spacing = None
# Hyphenation configuration parameters # Hyphenation configuration parameters
self._min_word_length_for_brute_force = min_word_length_for_brute_force self._min_word_length_for_brute_force = min_word_length_for_brute_force
@@ -771,6 +851,45 @@ class Line(Box):
"""Set the next line in sequence""" """Set the next line in sequence"""
self._next = line self._next = line
@property
def _content_width(self) -> float:
"""Summed width of the line's current contents."""
return self._width_prefix[-1]
def _push_text(self, text: 'Text'):
"""
Append a Text to the line, keeping the running width sum in step.
Fitting a word is a trial: the candidate is pushed, measured, and popped
again if it did not fit, so the line's contents churn far more often than
they grow. Tracking the sum here rather than re-adding every width on each
measurement is what keeps filling a line linear in its word count.
The sum is kept as a prefix list rather than as one accumulator that is
added to and subtracted from. Widths are floats, so `(total + w) - w` need
not give back `total` exactly, and a drift of one ulp is enough to flip an
overflow decision on a line that ends flush. Truncating a prefix list
restores the earlier total bit for bit, and each entry is built by the same
left-to-right addition sum() would perform.
"""
self._text_objects.append(text)
self._width_prefix.append(self._width_prefix[-1] + text.width)
def _pop_text(self) -> 'Text':
"""Remove the last Text from the line, keeping the width sum in step."""
text = self._text_objects.pop()
self._width_prefix.pop()
return text
def _measure(self, handler: Optional[AlignmentHandler] = None
) -> Tuple[int, int, bool]:
"""Ask an alignment handler to place the line's current contents."""
if handler is None:
handler = self._alignment_handler
return handler.calculate_spacing_and_position(
self._text_objects, self._size[0], self._spacing[0], self._spacing[1],
self._natural_spacing, self._content_width)
def add_word(self, def add_word(self,
word: 'Word', word: 'Word',
part: Optional[Text] = None) -> Tuple[bool, part: Optional[Text] = None) -> Tuple[bool,
@@ -789,7 +908,7 @@ class Line(Box):
""" """
# First, add any pretext from previous hyphenation # First, add any pretext from previous hyphenation
if part is not None: if part is not None:
self._text_objects.append(part) self._push_text(part)
self._words.append(word) self._words.append(word)
part.add_line(self) part.add_line(self)
@@ -818,10 +937,8 @@ class Line(Box):
line=self) line=self)
else: else:
text = Text.from_word(word, self._draw) text = Text.from_word(word, self._draw)
self._text_objects.append(text) self._push_text(text)
spacing, position, overflow = self._alignment_handler.calculate_spacing_and_position( spacing, position, overflow = self._measure()
self._text_objects, self._size[0], self._spacing[0], self._spacing[1],
self._natural_spacing)
if not overflow: if not overflow:
# Word fits! Add it completely # Word fits! Add it completely
@@ -833,7 +950,7 @@ class Line(Box):
return True, None return True, None
# Word doesn't fit, remove it and try hyphenation # Word doesn't fit, remove it and try hyphenation
_ = self._text_objects.pop() self._pop_text()
# Step 1: Try pyphen hyphenation # Step 1: Try pyphen hyphenation
pyphen_splits = word.possible_hyphenation() pyphen_splits = word.possible_hyphenation()
@@ -866,11 +983,9 @@ class Line(Box):
source=word) source=word)
# Check if first part fits # Check if first part fits
self._text_objects.append(first_text) self._push_text(first_text)
spacing, position, overflow = self._alignment_handler.calculate_spacing_and_position( spacing, position, overflow = self._measure()
self._text_objects, self._size[0], self._spacing[0], self._spacing[1], self._pop_text()
self._natural_spacing)
_ = self._text_objects.pop()
if not overflow: if not overflow:
# This split fits! Add it to valid options # This split fits! Add it to valid options
@@ -883,7 +998,7 @@ class Line(Box):
first_text, second_text, spacing, position = best_split first_text, second_text, spacing, position = best_split
# Apply the split # Apply the split
self._text_objects.append(first_text) self._push_text(first_text)
first_text.line = self first_text.line = self
word.add_concete((first_text, second_text)) word.add_concete((first_text, second_text))
self._spacing_render = spacing self._spacing_render = spacing
@@ -894,7 +1009,7 @@ class Line(Box):
# Step 3: Try brute force hyphenation (only for long words) # Step 3: Try brute force hyphenation (only for long words)
if len(word.text) >= self._min_word_length_for_brute_force: if len(word.text) >= self._min_word_length_for_brute_force:
# Calculate available space for the word # Calculate available space for the word
word_length = sum([text.width for text in self._text_objects]) word_length = self._content_width
spacing_length = self._spacing[0] * max(0, len(self._text_objects) - 1) spacing_length = self._spacing[0] * max(0, len(self._text_objects) - 1)
remaining = self._size[0] - word_length - spacing_length remaining = self._size[0] - word_length - spacing_length
@@ -938,10 +1053,8 @@ class Line(Box):
source=word) source=word)
# Verify the first part actually fits # Verify the first part actually fits
self._text_objects.append(first_text) self._push_text(first_text)
spacing, position, overflow = self._alignment_handler.calculate_spacing_and_position( spacing, position, overflow = self._measure()
self._text_objects, self._size[0], self._spacing[0], self._spacing[1],
self._natural_spacing)
if not overflow: if not overflow:
# Brute force split works! # Brute force split works!
@@ -954,7 +1067,7 @@ class Line(Box):
return True, second_text return True, second_text
else: else:
# Doesn't fit, remove it # Doesn't fit, remove it
_ = self._text_objects.pop() self._pop_text()
# Step 4: Word cannot be hyphenated or split, move to next line # Step 4: Word cannot be hyphenated or split, move to next line
return False, None return False, None
@@ -972,9 +1085,7 @@ class Line(Box):
# justified paragraph. # justified paragraph.
handler = self.render_alignment_handler handler = self.render_alignment_handler
if len(self._text_objects) > 0: if len(self._text_objects) > 0:
spacing, position, overflow = handler.calculate_spacing_and_position( spacing, position, overflow = self._measure(handler)
self._text_objects, self._size[0], self._spacing[0], self._spacing[1],
self._natural_spacing)
self._spacing_render = spacing self._spacing_render = spacing
self._position_render = position self._position_render = position
@@ -982,28 +1093,33 @@ class Line(Box):
# Start x_cursor at line origin plus any alignment offset # Start x_cursor at line origin plus any alignment offset
x_cursor = self._origin[0] + self._position_render x_cursor = self._origin[0] + self._position_render
for i, text in enumerate(self._text_objects):
# Everything the loop needs that does not vary per word is resolved once.
# Only justified lines carry per-gap spacings; every other alignment uses
# the single spacing figured above.
texts = self._text_objects
last = len(texts) - 1
draw = self._draw
default_spacing = self._spacing_render
gaps = handler._gap_spacings if isinstance(handler, JustifyAlignmentHandler) else ()
gap_count = len(gaps)
for i, text in enumerate(texts):
# Update text draw context to current draw context # Update text draw context to current draw context
text._draw = self._draw text._draw = draw
text.set_origin(np.array([x_cursor, y_cursor])) text.set_origin(np.array([x_cursor, y_cursor]))
# Determine next text object for continuous decoration # Determine next text object for continuous decoration
next_text = self._text_objects[i + 1] if i + \ next_text = texts[i + 1] if i < last else None
1 < len(self._text_objects) else None
# Get the spacing for this specific gap (variable for justified text) # Get the spacing for this specific gap (variable for justified text)
if isinstance(handler, JustifyAlignmentHandler) and \ current_spacing = gaps[i] if i < gap_count else default_spacing
hasattr(handler, '_gap_spacings') and \
i < len(handler._gap_spacings):
current_spacing = handler._gap_spacings[i]
else:
current_spacing = self._spacing_render
# Render with next text information for continuous underline/strikethrough # Render with next text information for continuous underline/strikethrough
text.render(next_text, current_spacing) text.render(next_text, current_spacing)
# Add text width, then spacing only if there are more words # Add text width, then spacing only if there are more words
x_cursor += text.width x_cursor += text.width
if i < len(self._text_objects) - 1: if i < last:
x_cursor += current_spacing x_cursor += current_spacing
def query_point(self, point: Tuple[int, int]) -> Optional['QueryResult']: def query_point(self, point: Tuple[int, int]) -> Optional['QueryResult']:
+27 -27
View File
@@ -6,12 +6,16 @@ managing highlight collections, and rendering highlights on pages.
""" """
from __future__ import annotations from __future__ import annotations
import logging
from dataclasses import dataclass from dataclasses import dataclass
from typing import List, Tuple, Optional, Dict, Any from typing import List, Tuple, Optional, Dict, Any
from enum import Enum from enum import Enum
import json
from pathlib import Path from pathlib import Path
from pyWebLayout.core.persistence import ensure_dir, read_json, write_json
logger = logging.getLogger(__name__)
class HighlightColor(Enum): class HighlightColor(Enum):
"""Predefined highlight colors with RGBA values""" """Predefined highlight colors with RGBA values"""
@@ -44,6 +48,12 @@ class Highlight:
start_word_index: Optional[int] = None # Word index in document (if available) start_word_index: Optional[int] = None # Word index in document (if available)
end_word_index: Optional[int] = None end_word_index: Optional[int] = None
# Where in the document this highlight lives, as a serialized
# RenderingPosition. `bounds` are pixel coordinates on one particular
# rendering, so they stop matching as soon as the font scale or page size
# changes; this survives repagination and is what page association uses.
position: Optional[Dict[str, Any]] = None
# Metadata # Metadata
note: Optional[str] = None # Optional annotation note: Optional[str] = None # Optional annotation
tags: List[str] = None # Optional categorization tags tags: List[str] = None # Optional categorization tags
@@ -63,6 +73,7 @@ class Highlight:
'text': self.text, 'text': self.text,
'start_word_index': self.start_word_index, 'start_word_index': self.start_word_index,
'end_word_index': self.end_word_index, 'end_word_index': self.end_word_index,
'position': self.position,
'note': self.note, 'note': self.note,
'tags': self.tags, 'tags': self.tags,
'timestamp': self.timestamp 'timestamp': self.timestamp
@@ -78,6 +89,7 @@ class Highlight:
text=data['text'], text=data['text'],
start_word_index=data.get('start_word_index'), start_word_index=data.get('start_word_index'),
end_word_index=data.get('end_word_index'), end_word_index=data.get('end_word_index'),
position=data.get('position'),
note=data.get('note'), note=data.get('note'),
tags=data.get('tags', []), tags=data.get('tags', []),
timestamp=data.get('timestamp') timestamp=data.get('timestamp')
@@ -100,12 +112,9 @@ class HighlightManager:
highlights_dir: Directory to store highlight data highlights_dir: Directory to store highlight data
""" """
self.document_id = document_id self.document_id = document_id
self.highlights_dir = Path(highlights_dir) self.highlights_dir = ensure_dir(highlights_dir)
self.highlights: Dict[str, Highlight] = {} # id -> Highlight self.highlights: Dict[str, Highlight] = {} # id -> Highlight
# Create directory if it doesn't exist
self.highlights_dir.mkdir(parents=True, exist_ok=True)
# Load existing highlights # Load existing highlights
self._load_highlights() self._load_highlights()
@@ -178,34 +187,22 @@ class HighlightManager:
def _save_highlights(self) -> None: def _save_highlights(self) -> None:
"""Persist highlights to disk""" """Persist highlights to disk"""
try: write_json(self._get_filepath(), {
filepath = self._get_filepath() 'document_id': self.document_id,
data = { 'highlights': [h.to_dict() for h in self.highlights.values()]
'document_id': self.document_id, })
'highlights': [h.to_dict() for h in self.highlights.values()]
}
with open(filepath, 'w') as f:
json.dump(data, f, indent=2)
except Exception as e:
print(f"Error saving highlights: {e}")
def _load_highlights(self) -> None: def _load_highlights(self) -> None:
"""Load highlights from disk""" """Load highlights from disk"""
data = read_json(self._get_filepath(), {})
try: try:
filepath = self._get_filepath()
if not filepath.exists():
return
with open(filepath, 'r') as f:
data = json.load(f)
self.highlights = { self.highlights = {
h['id']: Highlight.from_dict(h) h['id']: Highlight.from_dict(h)
for h in data.get('highlights', []) for h in data.get('highlights', [])
} }
except Exception as e: except (AttributeError, TypeError, KeyError):
print(f"Error loading highlights: {e}") logger.warning("Highlight file %s is not in the expected shape; ignoring it",
self._get_filepath(), exc_info=True)
self.highlights = {} self.highlights = {}
@@ -213,16 +210,18 @@ def create_highlight_from_query_result(
result, result,
color: Tuple[int, int, int, int] = HighlightColor.YELLOW.value, color: Tuple[int, int, int, int] = HighlightColor.YELLOW.value,
note: Optional[str] = None, note: Optional[str] = None,
tags: Optional[List[str]] = None tags: Optional[List[str]] = None,
position: Optional[Dict[str, Any]] = None
) -> Highlight: ) -> Highlight:
""" """
Create a highlight from a QueryResult. Create a highlight from a QueryResult.
Args: Args:
result: QueryResult from query_pixel or query_range result: QueryResult from query_point or query_range
color: RGBA color tuple color: RGBA color tuple
note: Optional annotation note: Optional annotation
tags: Optional categorization tags tags: Optional categorization tags
position: Serialized RenderingPosition of the page the result came from
Returns: Returns:
Highlight instance Highlight instance
@@ -243,6 +242,7 @@ def create_highlight_from_query_result(
bounds=bounds, bounds=bounds,
color=color, color=color,
text=text, text=text,
position=position,
note=note, note=note,
tags=tags or [], tags=tags or [],
timestamp=time() timestamp=time()
+59
View File
@@ -0,0 +1,59 @@
"""
Small JSON-file helpers shared by the per-document stores.
BookmarkManager and HighlightManager both keep a JSON file per document under a
directory, and both had their own copy of "make the directory, try to read it,
swallow and print on failure". The duplication is the point of this module; the
file formats themselves stay owned by each store.
"""
from __future__ import annotations
import json
import logging
from pathlib import Path
from typing import Any
logger = logging.getLogger(__name__)
def ensure_dir(path: str | Path) -> Path:
"""Return `path` as a Path, creating it and any missing parents."""
directory = Path(path)
directory.mkdir(parents=True, exist_ok=True)
return directory
def read_json(path: Path, default: Any) -> Any:
"""
Read JSON from `path`, returning `default` if it is missing or unreadable.
A corrupt store must not stop a book from opening, so failures are logged
and swallowed. `default` is returned as given, so pass a fresh mutable if
the caller intends to mutate it.
"""
if not path.exists():
return default
try:
with open(path, 'r', encoding='utf-8') as handle:
return json.load(handle)
except (OSError, ValueError):
logger.warning("Could not read %s; ignoring its contents", path, exc_info=True)
return default
def write_json(path: Path, data: Any) -> bool:
"""
Write `data` to `path` as JSON.
Returns True on success. Failures are logged rather than raised: losing a
bookmark is not a reason to take down the reader.
"""
try:
with open(path, 'w', encoding='utf-8') as handle:
json.dump(data, handle, indent=2)
return True
except (OSError, TypeError, ValueError):
logger.error("Could not write %s", path, exc_info=True)
return False
+7 -7
View File
@@ -403,13 +403,13 @@ def extract_words_from_nodes(nodes: List, context: StyleContext) -> List[Word]:
continue continue
if isinstance(child, NavigableString): if isinstance(child, NavigableString):
# Plain text - split into words # Plain text - split into words. Argument-less str.split() already
text = str(child).strip() # discards surrounding whitespace and never yields an empty string, so
if text: # it needs neither a preceding strip() nor a per-word emptiness test.
word_texts = text.split() font = context.font
for word_text in word_texts: background = context.background
if word_text: words.extend([Word(word_text, font, background)
words.append(Word(word_text, context.font, context.background)) for word_text in str(child).split()])
elif isinstance(child, Tag): elif isinstance(child, Tag):
# Special handling for <a> tags (hyperlinks) # Special handling for <a> tags (hyperlinks)
if child.name.lower() == "a": if child.name.lower() == "a":
+3 -6
View File
@@ -163,12 +163,9 @@ def paragraph_layouter(paragraph: Paragraph,
y_cursor = page._current_y_offset y_cursor = page._current_y_offset
x_cursor = page.content_origin[0] x_cursor = page.content_origin[0]
# Create a temporary Text object to calculate word width # `word` is accepted for call-site readability only: the line that is about
if word: # to be created measures it when it is added, so measuring it here as well
temp_text = Text.from_word(word, page.measurement_draw) # only paid for a Text object that was immediately discarded.
temp_text.width
else:
pass
return Line( return Line(
spacing=word_spacing_constraints, spacing=word_spacing_constraints,
+230 -173
View File
@@ -15,7 +15,9 @@ from __future__ import annotations
from dataclasses import dataclass, asdict from dataclasses import dataclass, asdict
from typing import List, Dict, Tuple, Optional, Any from typing import List, Dict, Tuple, Optional, Any
from pyWebLayout.abstract.block import Block, Paragraph, Heading, HeadingLevel, Table, HList, Image from pyWebLayout.abstract.block import (
Block, Paragraph, Heading, HeadingLevel, Table, TableRow, TableCell,
HList, ListItem, Quote, Image)
from pyWebLayout.abstract.inline import Word from pyWebLayout.abstract.inline import Word
from pyWebLayout.concrete.page import Page from pyWebLayout.concrete.page import Page
from pyWebLayout.concrete.text import Text from pyWebLayout.concrete.text import Text
@@ -41,6 +43,19 @@ class RenderingPosition:
remaining_pretext: Optional[str] = None # Hyphenated word continuation remaining_pretext: Optional[str] = None # Hyphenated word continuation
page_y_offset: int = 0 # Vertical position on page page_y_offset: int = 0 # Vertical position on page
def _key(self) -> Tuple[Any, ...]:
"""
The fields in declaration order.
Copying, comparing and hashing a position all used to go through
dataclasses.asdict, which walks the field list and deep-copies each value.
Every field here is an immutable scalar, so that traversal bought nothing
and these three run constantly during page navigation and buffer lookups.
"""
return (self.chapter_index, self.block_index, self.word_index,
self.table_row, self.table_col, self.list_item_index,
self.remaining_pretext, self.page_y_offset)
def to_dict(self) -> Dict[str, Any]: def to_dict(self) -> Dict[str, Any]:
"""Serialize position for saving to file/database""" """Serialize position for saving to file/database"""
return asdict(self) return asdict(self)
@@ -52,17 +67,17 @@ class RenderingPosition:
def copy(self) -> 'RenderingPosition': def copy(self) -> 'RenderingPosition':
"""Create a copy of this position""" """Create a copy of this position"""
return RenderingPosition(**asdict(self)) return RenderingPosition(*self._key())
def __eq__(self, other) -> bool: def __eq__(self, other) -> bool:
"""Check if two positions are equal""" """Check if two positions are equal"""
if not isinstance(other, RenderingPosition): if not isinstance(other, RenderingPosition):
return False return False
return asdict(self) == asdict(other) return self._key() == other._key()
def __hash__(self) -> int: def __hash__(self) -> int:
"""Make position hashable for use as dict key""" """Make position hashable for use as dict key"""
return hash(tuple(asdict(self).values())) return hash(self._key())
class ChapterInfo: class ChapterInfo:
@@ -313,6 +328,19 @@ class BidirectionalLayouter:
self.alignment_override = alignment_override self.alignment_override = alignment_override
self.font_family_override = font_family_override self.font_family_override = font_family_override
# Maps (font_scale, end position) -> the position the page started at.
# Filled in as pages are laid out forward, which makes "previous page"
# exact and free for anywhere the reader has already been. Keyed by font
# scale because changing it repaginates the document.
self._page_chain: Dict[Tuple[float, Tuple[int, int, int]],
RenderingPosition] = {}
# Scaled copies of blocks, keyed by (id(block), font_scale). Rebuilding
# a block's words on every page render allocated a fresh Paragraph and
# Word per word on the hot path. The original block is kept alongside
# the copy so its id cannot be recycled while it is a live key.
self._scaled_block_cache: Dict[Tuple[int, float], Tuple[Block, Block]] = {}
def render_page_forward(self, position: RenderingPosition, def render_page_forward(self, position: RenderingPosition,
font_scale: float = 1.0) -> Tuple[Page, RenderingPosition]: font_scale: float = 1.0) -> Tuple[Page, RenderingPosition]:
""" """
@@ -367,134 +395,241 @@ class BidirectionalLayouter:
current_pos = new_pos current_pos = new_pos
# Remember this link in the chain so stepping back to it later is exact.
if self._position_compare(current_pos, position) > 0:
self._page_chain[(font_scale, self._position_key(current_pos))] = \
position.copy()
return page, current_pos return page, current_pos
# How many block starts before the target to try as replay anchors before
# settling for the best inexact answer.
MAX_BACKWARD_ANCHORS = 4
# Ceiling on pages replayed from a single anchor, so a pathologically long
# block cannot make one page turn walk an entire chapter.
MAX_REPLAY_PAGES = 8
def render_page_backward(self, def render_page_backward(self,
end_position: RenderingPosition, end_position: RenderingPosition,
font_scale: float = 1.0) -> Tuple[Page, font_scale: float = 1.0) -> Tuple[Page,
RenderingPosition]: RenderingPosition]:
""" """
Render a page that ends at the given position, filling backward. Render the page that ends at the given position - "previous page".
Critical for "previous page" navigation.
Uses iterative refinement to find the correct start position that Pagination is a pure function: laying out from a position q yields a page
results in a page ending at (or very close to) the target position. and the position where it stopped, next(q). The page before P is therefore
the q for which next(q) == P, and it is found by *replaying* the chain
forward from an anchor, not by guessing q.
The previous implementation searched instead: it estimated a block index
and bisected on it, pinning word_index to 0. Pages routinely start
mid-block, so the answer was frequently not in the search space at all -
the search then exhausted its iterations and fell back to a position that
was not the previous page, usually the start of the document.
Three sources are tried in order:
1. The recorded chain, from pages already laid out going forward. Exact,
and the common case when the reader is paging back and forth.
2. Replay from the start of the block containing P, then from
progressively earlier blocks. Exact when P lies on the resulting chain.
3. Failing an exact hit - which happens when P was reached by a jump or a
restored bookmark rather than by reading forward, so it is on no
natural chain - the latest page start before P. That overlaps P's page
slightly rather than skipping content, which is the safe direction to
be wrong in.
Args: Args:
end_position: Position where page should end end_position: Position where the page should end
font_scale: Font scaling factor font_scale: Font scaling factor
Returns: Returns:
Tuple of (rendered_page, start_position) Tuple of (rendered_page, start_position)
""" """
# Handle edge case: already at beginning document_start = RenderingPosition()
if end_position.block_index == 0 and end_position.word_index == 0:
return self.render_page_forward(end_position, font_scale)
# Start with initial estimate # Nothing precedes the start of the document.
estimated_start = self._estimate_page_start(end_position, font_scale) if self._position_compare(end_position, document_start) <= 0:
page, _ = self.render_page_forward(document_start, font_scale)
return page, document_start
# Iterative refinement: keep adjusting until we converge or hit max iterations # 1. The chain we have already walked.
max_iterations = 10 remembered = self._page_chain.get((font_scale, self._position_key(end_position)))
best_page = None if remembered is not None:
best_start = estimated_start page, actual_end = self.render_page_forward(remembered, font_scale)
best_distance = float('inf') if self._position_compare(actual_end, end_position) == 0:
return page, remembered
for iteration in range(max_iterations): # 2/3. Replay from anchors, keeping the best inexact result as a fallback.
# Render forward from current estimate fallback = None
page, actual_end = self.render_page_forward(estimated_start, font_scale) for anchor in self._backward_anchors(end_position):
page, start, exact = self._replay_to(anchor, end_position, font_scale)
if page is None:
continue
if exact:
return page, start
if fallback is None:
fallback = (page, start)
# Calculate how far we are from target if fallback is not None:
comparison = self._position_compare(actual_end, end_position) return fallback
# Perfect match or close enough (within same block) page, _ = self.render_page_forward(document_start, font_scale)
# BUT: ensure we actually moved backward (estimated_start < end_position) return page, document_start
if comparison == 0:
# Check if we actually found a valid previous page
if self._position_compare(estimated_start, end_position) < 0:
return page, estimated_start
# If estimated_start >= end_position, we haven't moved backward
# Continue iterating to find a better position
elif iteration == 0:
# On first iteration, if we can't find a previous position,
# we're likely at or near the beginning
break
# Track best result so far def _backward_anchors(self, target: RenderingPosition):
distance = abs(actual_end.block_index - end_position.block_index) """
if distance < best_distance: Yield positions to replay from, nearest first.
best_distance = distance
best_page = page
best_start = estimated_start.copy()
# Adjust estimate for next iteration Block starts are used as anchors because they are the coarsest positions
estimated_start = self._adjust_start_estimate( that are certainly valid to lay out from. The block containing the target
estimated_start, end_position, actual_end) comes first: when the target is mid-block, the page before it usually
starts in that same block or the one before.
"""
first_block = target.block_index if target.word_index > 0 \
else target.block_index - 1
# Safety: don't go before document start for offset in range(self.MAX_BACKWARD_ANCHORS):
if estimated_start.block_index < 0: block_index = first_block - offset
estimated_start.block_index = 0 if block_index < 0:
estimated_start.word_index = 0 break
yield RenderingPosition(
# If we exhausted iterations, return best result found chapter_index=target.chapter_index,
# BUT: ensure we didn't return the same position (no backward progress) block_index=block_index,
final_page = best_page if best_page else page word_index=0,
final_start = best_start
# Safety check: if final_start >= end_position, we failed to move backward
# This can happen at the beginning of the document or when estimation failed
if self._position_compare(final_start, end_position) >= 0:
# Can't go further back - check if we're at the absolute beginning
if end_position.block_index == 0 and end_position.word_index == 0:
# Already at beginning, return as-is
return final_page, final_start
# Fallback strategy: try a more aggressive backward jump
# Start from several blocks before the current position
blocks_to_jump = max(1, min(5, end_position.block_index))
fallback_pos = RenderingPosition(
chapter_index=end_position.chapter_index,
block_index=max(0, end_position.block_index - blocks_to_jump),
word_index=0
) )
# Render forward from the fallback position if first_block - self.MAX_BACKWARD_ANCHORS >= 0:
fallback_page, fallback_end = self.render_page_forward(fallback_pos, font_scale) yield RenderingPosition()
# Verify the fallback actually moved us backward def _replay_to(self,
if self._position_compare(fallback_pos, end_position) < 0: anchor: RenderingPosition,
return fallback_page, fallback_pos target: RenderingPosition,
font_scale: float):
"""
Lay out pages forward from `anchor`, looking for the one ending at `target`.
# If even the fallback didn't work, we're likely at the beginning Returns:
# Return a page starting from the beginning (page, start, exact). `exact` is True when a page ended precisely on
return self.render_page_forward(RenderingPosition(), font_scale) the target. When the chain steps over the target instead, the last
page starting before it is returned with exact=False. (None, None,
False) means the anchor yielded nothing usable.
"""
position = anchor
last = (None, None)
return final_page, final_start for _ in range(self.MAX_REPLAY_PAGES):
if self._position_compare(position, target) >= 0:
break
page, next_position = self.render_page_forward(position, font_scale)
comparison = self._position_compare(next_position, target)
if comparison == 0:
return page, position, True
if comparison > 0:
# Stepped over the target: this chain does not pass through it.
return last[0], last[1], False
if self._position_compare(next_position, position) <= 0:
break # no progress; give up on this anchor
last = (page, position)
position = next_position
return last[0], last[1], False
@staticmethod
def _position_key(position: RenderingPosition) -> Tuple[int, int, int]:
"""Hashable identity of a position, for the page chain map."""
return (position.chapter_index, position.block_index, position.word_index)
def _scale_block_fonts(self, block: Block, font_scale: float) -> Block: def _scale_block_fonts(self, block: Block, font_scale: float) -> Block:
"""Apply font scaling and font family override to all fonts in a block""" """
# Check if we need to do any transformation Apply font scaling and the font family override to every font in a block.
Returns the block unchanged when there is nothing to apply. Results are
memoised per (block, scale) for the life of the layouter, so a page
re-render at an unchanged scale costs a dict lookup.
"""
if font_scale == 1.0 and self.font_family_override is None: if font_scale == 1.0 and self.font_family_override is None:
return block return block
# This is a simplified implementation key = (id(block), font_scale)
# In practice, we'd need to handle each block type appropriately cached = self._scaled_block_cache.get(key)
if isinstance(block, (Paragraph, Heading)): if cached is not None:
scaled_block_style = FontScaler.scale_font(block.style, font_scale, self.font_family_override) return cached[1]
if isinstance(block, Heading):
scaled_block = Heading(block.level, scaled_block_style)
else:
scaled_block = Paragraph(scaled_block_style)
# words_iter() returns tuples of (position, word) scaled = self._build_scaled_block(block, font_scale)
for position, word in block.words_iter(): self._scaled_block_cache[key] = (block, scaled)
return scaled
def _build_scaled_block(self, block: Block, font_scale: float) -> Block:
"""Construct the scaled copy of a block. See _scale_block_fonts."""
def scale(font: Font) -> Font:
return FontScaler.scale_font(font, font_scale, self.font_family_override)
if isinstance(block, (Paragraph, Heading)):
if isinstance(block, Heading):
scaled_block = Heading(block.level, scale(block.style))
else:
scaled_block = Paragraph(scale(block.style))
# words_iter() yields (position, word) tuples. with_style() keeps
# the concrete word class, so a LinkedWord stays linked - rebuilding
# these as plain Words silently stripped every hyperlink in the
# document as soon as the reader changed font size.
for _, word in block.words_iter():
if isinstance(word, Word): if isinstance(word, Word):
scaled_word = Word( scaled_block.add_word(word.with_style(scale(word.style)))
word.text, FontScaler.scale_font(
word.style, font_scale, self.font_family_override))
scaled_block.add_word(scaled_word)
return scaled_block return scaled_block
if isinstance(block, Quote):
scaled_quote = Quote(scale(block.style) if block.style else None)
for child in block.blocks():
scaled_quote.add_block(self._scale_block_fonts(child, font_scale))
return scaled_quote
if isinstance(block, HList):
scaled_list = HList(
block.style,
scale(block.default_style) if block.default_style else None)
for item in block.items():
scaled_item = ListItem(
item.term,
scale(item.style) if item.style else None)
for child in item.blocks():
scaled_item.add_block(self._scale_block_fonts(child, font_scale))
scaled_list.add_item(scaled_item)
return scaled_list
if isinstance(block, Table):
scaled_table = Table(
block.caption,
scale(block.style) if block.style else None)
# Rows must go back into the section they came from, or a <thead>
# row would be re-added as a body row.
for section, rows in (('header', block.header_rows()),
('body', block.body_rows()),
('footer', block.footer_rows())):
for row in rows:
scaled_row = TableRow(scale(row.style) if row.style else None)
for cell in row.cells():
scaled_cell = TableCell(
is_header=cell.is_header,
colspan=cell.colspan,
rowspan=cell.rowspan,
style=scale(cell.style) if cell.style else None)
for child in cell.blocks():
scaled_cell.add_block(self._scale_block_fonts(child, font_scale))
scaled_row.add_cell(scaled_cell)
scaled_table.add_row(scaled_row, section)
return scaled_table
# Blocks with no fonts of their own (Image, HorizontalRule, PageBreak,
# CodeBlock - which carries raw lines, not styled words) pass through.
return block return block
def _layout_block_on_page(self, def _layout_block_on_page(self,
@@ -671,60 +806,6 @@ class BidirectionalLayouter:
# Keep same position so it will be attempted on the next page # Keep same position so it will be attempted on the next page
return False, position return False, position
def _estimate_page_start(
self,
end_position: RenderingPosition,
font_scale: float) -> RenderingPosition:
"""Estimate where a page should start to end at the given position"""
# This is a simplified heuristic - a full implementation would be more
# sophisticated
estimated_start = end_position.copy()
# Move back by an estimated number of blocks that would fit on a page
estimated_blocks_per_page = max(1, int(10 / font_scale)) # Rough estimate
estimated_start.block_index = max(
0, end_position.block_index - estimated_blocks_per_page)
estimated_start.word_index = 0
return estimated_start
def _adjust_start_estimate(
self,
current_start: RenderingPosition,
target_end: RenderingPosition,
actual_end: RenderingPosition) -> RenderingPosition:
"""
Adjust start position estimate based on overshoot/undershoot.
Uses proportional adjustment to converge faster.
"""
adjusted = current_start.copy()
# Calculate the difference between actual and target end positions
block_diff = actual_end.block_index - target_end.block_index
comparison = self._position_compare(actual_end, target_end)
if comparison < 0: # Undershot - rendered to block X but need to reach block Y where X < Y
# We didn't render far enough forward
# Need to start at a LATER block (higher index) so the page includes more content
adjustment = max(1, abs(block_diff) // 2)
new_index = adjusted.block_index + adjustment
# Clamp to valid range
if len(self.blocks) > 0:
adjusted.block_index = min(len(self.blocks) - 1, max(0, new_index))
else:
adjusted.block_index = max(0, new_index)
elif comparison > 0: # Overshot - rendered past the target
# We rendered too far forward
# Need to start at an EARLIER block (lower index) so the page doesn't go as far
adjustment = max(1, abs(block_diff) // 2)
adjusted.block_index = max(0, adjusted.block_index - adjustment)
# Reset word index when adjusting blocks
adjusted.word_index = 0
return adjusted
def _position_compare(self, pos1: RenderingPosition, def _position_compare(self, pos1: RenderingPosition,
pos2: RenderingPosition) -> int: pos2: RenderingPosition) -> int:
"""Compare two positions (-1: pos1 < pos2, 0: equal, 1: pos1 > pos2)""" """Compare two positions (-1: pos1 < pos2, 0: equal, 1: pos1 > pos2)"""
@@ -735,27 +816,3 @@ class BidirectionalLayouter:
if pos1.word_index != pos2.word_index: if pos1.word_index != pos2.word_index:
return 1 if pos1.word_index > pos2.word_index else -1 return 1 if pos1.word_index > pos2.word_index else -1
return 0 return 0
# Add can_fit_line method to Page class if it doesn't exist
def _add_page_methods():
"""Add missing methods to Page class"""
if not hasattr(Page, 'can_fit_line'):
def can_fit_line(self, line_height: int) -> bool:
"""Check if a line of given height can fit on the page"""
available_height = self.content_size[1] - self._current_y_offset
return available_height >= line_height
Page.can_fit_line = can_fit_line
if not hasattr(Page, 'available_width'):
@property
def available_width(self) -> int:
"""Get available width for content"""
return self.content_size[0]
Page.available_width = available_width
# Apply the page methods
_add_page_methods()
+223 -41
View File
@@ -8,9 +8,7 @@ into a unified, easy-to-use API.
from __future__ import annotations from __future__ import annotations
from typing import List, Dict, Optional, Tuple, Any, Callable from typing import List, Dict, Optional, Tuple, Any, Callable
import json
import logging import logging
from pathlib import Path
from .ereader_layout import RenderingPosition, ChapterNavigator, ChapterInfo from .ereader_layout import RenderingPosition, ChapterNavigator, ChapterInfo
from .page_buffer import BufferedPageRenderer from .page_buffer import BufferedPageRenderer
@@ -20,6 +18,11 @@ from pyWebLayout.concrete.image import RenderableImage
from pyWebLayout.style.page_style import PageStyle from pyWebLayout.style.page_style import PageStyle
from pyWebLayout.style.fonts import BundledFont from pyWebLayout.style.fonts import BundledFont
from pyWebLayout.layout.document_layouter import image_layouter from pyWebLayout.layout.document_layouter import image_layouter
from pyWebLayout.core.highlight import Highlight, HighlightColor, HighlightManager, \
create_highlight_from_query_result
from pyWebLayout.core.persistence import ensure_dir, read_json, write_json
from pyWebLayout.concrete.interaction_handler import InteractionStateManager
from PIL import Image as Image_
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -38,8 +41,7 @@ class BookmarkManager:
bookmarks_dir: Directory to store bookmark files bookmarks_dir: Directory to store bookmark files
""" """
self.document_id = document_id self.document_id = document_id
self.bookmarks_dir = Path(bookmarks_dir) self.bookmarks_dir = ensure_dir(bookmarks_dir)
self.bookmarks_dir.mkdir(exist_ok=True)
self.bookmarks_file = self.bookmarks_dir / f"{document_id}_bookmarks.json" self.bookmarks_file = self.bookmarks_dir / f"{document_id}_bookmarks.json"
self.position_file = self.bookmarks_dir / f"{document_id}_position.json" self.position_file = self.bookmarks_dir / f"{document_id}_position.json"
@@ -49,29 +51,23 @@ class BookmarkManager:
def _load_bookmarks(self): def _load_bookmarks(self):
"""Load bookmarks from file""" """Load bookmarks from file"""
if self.bookmarks_file.exists(): data = read_json(self.bookmarks_file, {})
try: try:
with open(self.bookmarks_file, 'r') as f: self._bookmarks = {
data = json.load(f) name: RenderingPosition.from_dict(pos_data)
self._bookmarks = { for name, pos_data in data.items()
name: RenderingPosition.from_dict(pos_data) }
for name, pos_data in data.items() except (AttributeError, TypeError, KeyError):
} logger.warning("Bookmark file %s is not in the expected shape; ignoring it",
except Exception as e: self.bookmarks_file, exc_info=True)
print(f"Failed to load bookmarks: {e}") self._bookmarks = {}
self._bookmarks = {}
def _save_bookmarks(self): def _save_bookmarks(self):
"""Save bookmarks to file""" """Save bookmarks to file"""
try: write_json(self.bookmarks_file, {
data = { name: position.to_dict()
name: position.to_dict() for name, position in self._bookmarks.items()
for name, position in self._bookmarks.items() })
}
with open(self.bookmarks_file, 'w') as f:
json.dump(data, f, indent=2)
except Exception as e:
print(f"Failed to save bookmarks: {e}")
def add_bookmark(self, name: str, position: RenderingPosition): def add_bookmark(self, name: str, position: RenderingPosition):
""" """
@@ -128,11 +124,7 @@ class BookmarkManager:
Args: Args:
position: Current reading position position: Current reading position
""" """
try: write_json(self.position_file, position.to_dict())
with open(self.position_file, 'w') as f:
json.dump(position.to_dict(), f, indent=2)
except Exception as e:
print(f"Failed to save reading position: {e}")
def load_reading_position(self) -> Optional[RenderingPosition]: def load_reading_position(self) -> Optional[RenderingPosition]:
""" """
@@ -141,14 +133,15 @@ class BookmarkManager:
Returns: Returns:
Last reading position or None if not found Last reading position or None if not found
""" """
if self.position_file.exists(): data = read_json(self.position_file, None)
try: if data is None:
with open(self.position_file, 'r') as f: return None
data = json.load(f) try:
return RenderingPosition.from_dict(data) return RenderingPosition.from_dict(data)
except Exception as e: except (TypeError, KeyError):
print(f"Failed to load reading position: {e}") logger.warning("Position file %s is not in the expected shape; ignoring it",
return None self.position_file, exc_info=True)
return None
class EreaderLayoutManager: class EreaderLayoutManager:
@@ -171,7 +164,8 @@ class EreaderLayoutManager:
document_id: str = "default", document_id: str = "default",
buffer_size: int = 5, buffer_size: int = 5,
page_style: Optional[PageStyle] = None, page_style: Optional[PageStyle] = None,
bookmarks_dir: str = "bookmarks"): bookmarks_dir: str = "bookmarks",
highlights_dir: Optional[str] = None):
""" """
Initialize the ereader layout manager. Initialize the ereader layout manager.
@@ -182,6 +176,8 @@ class EreaderLayoutManager:
buffer_size: Number of pages to cache in each direction buffer_size: Number of pages to cache in each direction
page_style: Custom page styling (uses default if None) page_style: Custom page styling (uses default if None)
bookmarks_dir: Directory to store bookmark files bookmarks_dir: Directory to store bookmark files
highlights_dir: Directory to store highlights. Defaults to
bookmarks_dir, so a document's reading state lives in one place.
""" """
self.blocks = blocks self.blocks = blocks
self.page_size = page_size self.page_size = page_size
@@ -196,6 +192,8 @@ class EreaderLayoutManager:
self.renderer = BufferedPageRenderer(blocks, page_style, buffer_size, page_size) self.renderer = BufferedPageRenderer(blocks, page_style, buffer_size, page_size)
self.chapter_navigator = ChapterNavigator(blocks) self.chapter_navigator = ChapterNavigator(blocks)
self.bookmark_manager = BookmarkManager(document_id, bookmarks_dir) self.bookmark_manager = BookmarkManager(document_id, bookmarks_dir)
self.highlight_manager = HighlightManager(
document_id, highlights_dir if highlights_dir is not None else bookmarks_dir)
# Current state # Current state
self.current_position = RenderingPosition() self.current_position = RenderingPosition()
@@ -216,6 +214,10 @@ class EreaderLayoutManager:
self.current_position = saved_position self.current_position = saved_position
self._on_cover_page = False # If we have a saved position, we're past the cover self._on_cover_page = False # If we have a saved position, we're past the cover
# Pointer interaction state, rebound whenever the displayed page changes
self._interaction_state_manager: Optional[InteractionStateManager] = None
self._interaction_page: Optional[Page] = None
# Callbacks for UI updates # Callbacks for UI updates
self.position_changed_callback: Optional[Callable[[ self.position_changed_callback: Optional[Callable[[
RenderingPosition], None]] = None RenderingPosition], None]] = None
@@ -451,6 +453,12 @@ class EreaderLayoutManager:
# Special case: if at the beginning of content and there's a cover, go back to it # Special case: if at the beginning of content and there's a cover, go back to it
if self._has_cover and self._is_at_beginning() and not self._on_cover_page: if self._has_cover and self._is_at_beginning() and not self._on_cover_page:
self._on_cover_page = True self._on_cover_page = True
# Restore the canonical cover position. Being on the cover must have a
# single representation: a fresh load sits at block 0 with the cover
# showing, so returning to the cover has to land there too. Leaving the
# position at the first content block saves a position that reopens past
# the cover, silently losing it.
self.current_position = RenderingPosition()
self._notify_position_changed() self._notify_position_changed()
return self.get_current_page() return self.get_current_page()
@@ -848,6 +856,165 @@ class EreaderLayoutManager:
""" """
return self.bookmark_manager.list_bookmarks() return self.bookmark_manager.list_bookmarks()
# ------------------------------------------------------------------
# Highlights
#
# A Highlight carries pixel bounds, which belong to the one rendering it
# was taken from: change the font scale or page size and they no longer
# describe anything. Each highlight therefore also records the
# RenderingPosition of the page it was made on, and page association goes
# through that rather than through the bounds.
# ------------------------------------------------------------------
def highlight_point(self,
point: Tuple[int, int],
color: Tuple[int, int, int, int] = HighlightColor.YELLOW.value,
note: Optional[str] = None,
tags: Optional[List[str]] = None) -> Optional[Highlight]:
"""
Highlight whatever is at a point on the current page.
Args:
point: (x, y) in page coordinates, as delivered by a tap
color: RGBA fill, e.g. one of HighlightColor
note: Optional annotation
tags: Optional categorization tags
Returns:
The stored Highlight, or None if nothing was at that point.
"""
result = self.get_current_page().query_point(point)
if result is None or result.object_type == "empty":
return None
return self._store_highlight(result, color, note, tags)
def highlight_range(self,
start: Tuple[int, int],
end: Tuple[int, int],
color: Tuple[int, int, int, int] = HighlightColor.YELLOW.value,
note: Optional[str] = None,
tags: Optional[List[str]] = None) -> Optional[Highlight]:
"""
Highlight the text between two points on the current page.
Args:
start: (x, y) where the selection began
end: (x, y) where the selection ended
color: RGBA fill, e.g. one of HighlightColor
note: Optional annotation
tags: Optional categorization tags
Returns:
The stored Highlight, or None if the range selected no text.
"""
selection = self.get_current_page().query_range(start, end)
if not selection.results:
return None
return self._store_highlight(selection, color, note, tags)
def _store_highlight(self, result, color, note, tags) -> Highlight:
"""Build a Highlight from a query result and persist it."""
highlight = create_highlight_from_query_result(
result, color=color, note=note, tags=tags,
position=self.current_position.to_dict())
self.highlight_manager.add_highlight(highlight)
return highlight
def remove_highlight(self, highlight_id: str) -> bool:
"""
Remove a highlight.
Args:
highlight_id: ID of the highlight to remove
Returns:
True if it existed and was removed
"""
return self.highlight_manager.remove_highlight(highlight_id)
def list_highlights(self) -> List[Highlight]:
"""Get every highlight in this document."""
return self.highlight_manager.list_highlights()
def get_highlights_for_current_page(self) -> List[Highlight]:
"""
Get the highlights made on the page currently being displayed.
Matched on the recorded RenderingPosition, so this stays correct across
font changes; highlights saved before the position field existed have
no position and are never matched.
"""
current = self.current_position.to_dict()
return [h for h in self.highlight_manager.list_highlights()
if h.position == current]
def clear_highlights(self) -> None:
"""Remove every highlight in this document."""
self.highlight_manager.clear_all()
# ------------------------------------------------------------------
# Pointer interaction
#
# Press/hover feedback is state that belongs to one rendered page, so the
# state machine is rebound whenever the displayed page changes. Callers get
# a fresh frame back when something changed visually, and None when nothing
# did - so a UI can skip a redraw it does not need.
# ------------------------------------------------------------------
def _interaction_state(self) -> InteractionStateManager:
"""The state machine for the page currently displayed."""
page = self.get_current_page()
if self._interaction_page is not page:
if self._interaction_state_manager is not None:
self._interaction_state_manager.reset()
self._interaction_state_manager = InteractionStateManager(page)
self._interaction_page = page
return self._interaction_state_manager
def handle_hover(self, point: Tuple[int, int]) -> Optional[Image_.Image]:
"""
Update hover feedback for a pointer at `point`.
Args:
point: (x, y) in page coordinates
Returns:
A re-rendered frame if the hover state changed, else None.
"""
return self._interaction_state().update_hover(point)
def handle_touch_down(self, point: Tuple[int, int]) -> Optional[Image_.Image]:
"""
Show pressed feedback for whatever interactive element is at `point`.
Args:
point: (x, y) in page coordinates
Returns:
A frame showing the pressed state, or None if nothing interactive
is there.
"""
return self._interaction_state().handle_mouse_down(point)
def handle_touch_up(self, point: Tuple[int, int]) -> Tuple[Optional[Image_.Image], Any]:
"""
Release the pressed element and run its action.
Args:
point: (x, y) in page coordinates
Returns:
(frame, callback_result). Both are None if no element was pressed.
"""
return self._interaction_state().handle_mouse_up(point)
def reset_interaction_state(self) -> None:
"""Clear any hover or press feedback, e.g. when the pointer leaves."""
if self._interaction_state_manager is not None:
self._interaction_state_manager.reset()
def get_reading_progress(self) -> float: def get_reading_progress(self) -> float:
""" """
Get reading progress as a percentage. Get reading progress as a percentage.
@@ -935,16 +1102,31 @@ class EreaderLayoutManager:
""" """
Shutdown the ereader manager and clean up resources. Shutdown the ereader manager and clean up resources.
Call this when the application is closing. Call this when the application is closing.
Idempotent: calling it twice saves the position once.
""" """
if getattr(self, '_shutdown_done', False):
return
self._shutdown_done = True
# Save current position # Save current position
self.bookmark_manager.save_reading_position(self.current_position) self.bookmark_manager.save_reading_position(self.current_position)
# Shutdown renderer and buffer # Release cached pages
self.renderer.shutdown() self.renderer.shutdown()
def __del__(self): def __del__(self):
"""Cleanup on destruction""" """
self.shutdown() Best-effort cleanup for callers that never called shutdown().
Finalisers run during interpreter teardown, when modules and globals
may already be torn down, so this must never raise and must never
block. Applications should call shutdown() explicitly.
"""
try:
self.shutdown()
except Exception:
pass
# Convenience function for quick setup # Convenience function for quick setup
+49 -225
View File
@@ -1,16 +1,35 @@
""" """
Multi-process page buffering system for high-performance ereader navigation. Page caching for ereader navigation.
This module provides intelligent page caching with background rendering using `PageBuffer` is an LRU cache of rendered pages plus the position links between
multiprocessing to achieve sub-second page navigation performance. them; `BufferedPageRenderer` wraps it around a `BidirectionalLayouter`.
This module used to render pages ahead of time in a `ProcessPoolExecutor`. That
never worked and has been removed see S12 in docs/LAYOUT_REMEDIATION_SPEC.md
and R1/R2 in docs/ARCHITECTURE_REVIEW.md. In short: the worker returned
`pickle.dumps(page)`, and a Page holds a live PIL canvas, which is not
picklable, so every job failed and the result was discarded. The cost four
interpreter copies and the whole block list shipped per job was paid in full
for no benefit. On Python 3.14, where the default start method became
`forkserver`, submitting from module-level code raised outright.
Prefetch is not needed. Measured on the tests/data Wikipedia fixture (411
blocks) with the text caches warm, one page render costs:
800x600 p50 8.8 ms p95 15.4 ms
1072x1448 p50 13.8 ms p95 56.1 ms
A page turn is cheaper than the IPC that was meant to hide it. If a slower
target device ever changes that, the fallback is a synchronous `readahead()`
method on this class, or a single worker *thread* layout is PIL-bound and PIL
releases the GIL not a process pool. Making the concrete tree picklable
(Page -> Line -> Text -> Font -> FreeTypeFont) is a large surface area to
maintain for a cache.
""" """
from __future__ import annotations from __future__ import annotations
from typing import Dict, Optional, List, Tuple, Any from typing import Dict, Optional, List, Tuple, Any
from collections import OrderedDict from collections import OrderedDict
from concurrent.futures import ProcessPoolExecutor, Future
import threading
import pickle
from .ereader_layout import RenderingPosition, BidirectionalLayouter, FontFamilyOverride from .ereader_layout import RenderingPosition, BidirectionalLayouter, FontFamilyOverride
from pyWebLayout.concrete.page import Page from pyWebLayout.concrete.page import Page
@@ -19,57 +38,20 @@ from pyWebLayout.style.page_style import PageStyle
from pyWebLayout.style.fonts import BundledFont from pyWebLayout.style.fonts import BundledFont
def _render_page_worker(args: Tuple[List[Block],
PageStyle,
RenderingPosition,
float,
bool,
Optional[BundledFont]]) -> Tuple[RenderingPosition,
bytes,
RenderingPosition]:
"""
Worker function for multiprocess page rendering.
Args:
args: Tuple of (blocks, page_style, position, font_scale, is_backward, font_family)
Returns:
Tuple of (original_position, pickled_page, next_position)
"""
blocks, page_style, position, font_scale, is_backward, font_family = args
# Create font family override if specified
font_family_override = FontFamilyOverride(font_family) if font_family else None
layouter = BidirectionalLayouter(blocks, page_style, font_family_override=font_family_override)
if is_backward:
page, next_pos = layouter.render_page_backward(position, font_scale)
else:
page, next_pos = layouter.render_page_forward(position, font_scale)
# Serialize the page for inter-process communication
pickled_page = pickle.dumps(page)
return position, pickled_page, next_pos
class PageBuffer: class PageBuffer:
""" """
Intelligent page caching system with LRU eviction and background rendering. LRU cache of rendered pages, with separate forward and backward buffers and
Maintains separate forward and backward buffers for optimal navigation performance. the position links between adjacent pages.
""" """
def __init__(self, buffer_size: int = 5, max_workers: int = 4): def __init__(self, buffer_size: int = 5):
""" """
Initialize the page buffer. Initialize the page buffer.
Args: Args:
buffer_size: Number of pages to cache in each direction buffer_size: Number of pages to cache in each direction
max_workers: Maximum number of worker processes for background rendering
""" """
self.buffer_size = buffer_size self.buffer_size = buffer_size
self.max_workers = max_workers
# LRU caches for forward and backward pages # LRU caches for forward and backward pages
self.forward_buffer: OrderedDict[RenderingPosition, Page] = OrderedDict() self.forward_buffer: OrderedDict[RenderingPosition, Page] = OrderedDict()
@@ -81,11 +63,6 @@ class PageBuffer:
self.reverse_position_map: Dict[RenderingPosition, self.reverse_position_map: Dict[RenderingPosition,
RenderingPosition] = {} # current -> previous RenderingPosition] = {} # current -> previous
# Background rendering
self.executor: Optional[ProcessPoolExecutor] = None
self.pending_renders: Dict[RenderingPosition, Future] = {}
self.render_lock = threading.Lock()
# Document state # Document state
self.blocks: Optional[List[Block]] = None self.blocks: Optional[List[Block]] = None
self.page_style: Optional[PageStyle] = None self.page_style: Optional[PageStyle] = None
@@ -112,10 +89,6 @@ class PageBuffer:
self.current_font_scale = font_scale self.current_font_scale = font_scale
self.current_font_family = font_family self.current_font_family = font_family
# Start the process pool
if self.executor is None:
self.executor = ProcessPoolExecutor(max_workers=self.max_workers)
def get_page(self, position: RenderingPosition) -> Optional[Page]: def get_page(self, position: RenderingPosition) -> Optional[Page]:
""" """
Get a cached page if available. Get a cached page if available.
@@ -176,125 +149,12 @@ class PageBuffer:
self.position_map.pop(oldest_pos, None) self.position_map.pop(oldest_pos, None)
self.reverse_position_map.pop(oldest_pos, None) self.reverse_position_map.pop(oldest_pos, None)
def start_background_rendering(
self,
current_position: RenderingPosition,
direction: str = 'forward'):
"""
Start background rendering of upcoming pages.
Args:
current_position: Current reading position
direction: 'forward', 'backward', or 'both'
"""
if not self.blocks or not self.page_style or not self.executor:
return
with self.render_lock:
if direction in ['forward', 'both']:
self._queue_forward_renders(current_position)
if direction in ['backward', 'both']:
self._queue_backward_renders(current_position)
def _queue_forward_renders(self, start_position: RenderingPosition):
"""Queue forward page renders starting from the given position"""
current_pos = start_position
for i in range(self.buffer_size):
# Skip if already cached or being rendered
if current_pos in self.forward_buffer or current_pos in self.pending_renders:
# Try to get next position from cache
current_pos = self.position_map.get(current_pos)
if not current_pos:
break
continue
# Queue render job
args = (
self.blocks,
self.page_style,
current_pos,
self.current_font_scale,
False,
self.current_font_family)
future = self.executor.submit(_render_page_worker, args)
self.pending_renders[current_pos] = future
# We don't know the next position yet, so we'll update it when the render
# completes
break
def _queue_backward_renders(self, start_position: RenderingPosition):
"""Queue backward page renders ending at the given position"""
current_pos = start_position
for i in range(self.buffer_size):
# Skip if already cached or being rendered
if current_pos in self.backward_buffer or current_pos in self.pending_renders:
# Try to get previous position from cache
current_pos = self.reverse_position_map.get(current_pos)
if not current_pos:
break
continue
# Queue render job
args = (
self.blocks,
self.page_style,
current_pos,
self.current_font_scale,
True,
self.current_font_family)
future = self.executor.submit(_render_page_worker, args)
self.pending_renders[current_pos] = future
# We don't know the previous position yet, so we'll update it when the
# render completes
break
def check_completed_renders(self):
"""Check for completed background renders and cache the results"""
if not self.pending_renders:
return
completed = []
with self.render_lock:
for position, future in self.pending_renders.items():
if future.done():
try:
original_pos, pickled_page, next_pos = future.result()
# Deserialize the page
page = pickle.loads(pickled_page)
# Cache the page
self.cache_page(original_pos, page, next_pos, is_backward=False)
completed.append(position)
except Exception as e:
print(f"Background render failed for position {position}: {e}")
completed.append(position)
# Remove completed renders
for pos in completed:
self.pending_renders.pop(pos, None)
def invalidate_all(self): def invalidate_all(self):
"""Clear all cached pages and cancel pending renders""" """Clear all cached pages"""
with self.render_lock: self.forward_buffer.clear()
# Cancel pending renders self.backward_buffer.clear()
for future in self.pending_renders.values(): self.position_map.clear()
future.cancel() self.reverse_position_map.clear()
self.pending_renders.clear()
# Clear caches
self.forward_buffer.clear()
self.backward_buffer.clear()
self.position_map.clear()
self.reverse_position_map.clear()
def set_font_scale(self, font_scale: float): def set_font_scale(self, font_scale: float):
""" """
@@ -323,7 +183,6 @@ class PageBuffer:
return { return {
'forward_buffer_size': len(self.forward_buffer), 'forward_buffer_size': len(self.forward_buffer),
'backward_buffer_size': len(self.backward_buffer), 'backward_buffer_size': len(self.backward_buffer),
'pending_renders': len(self.pending_renders),
'position_mappings': len(self.position_map), 'position_mappings': len(self.position_map),
'reverse_position_mappings': len(self.reverse_position_map), 'reverse_position_mappings': len(self.reverse_position_map),
'current_font_scale': self.current_font_scale, 'current_font_scale': self.current_font_scale,
@@ -331,28 +190,20 @@ class PageBuffer:
} }
def shutdown(self): def shutdown(self):
"""Shutdown the page buffer and clean up resources""" """
if self.executor: Release cached pages.
# Cancel pending renders
with self.render_lock:
for future in self.pending_renders.values():
future.cancel()
# Shutdown executor Cheap and idempotent. There is deliberately no __del__ calling this:
self.executor.shutdown(wait=True) blocking work in a finaliser is what deadlocked the interpreter at exit
self.executor = None while the process pool existed.
"""
# Clear all caches
self.invalidate_all() self.invalidate_all()
def __del__(self):
"""Cleanup on destruction"""
self.shutdown()
class BufferedPageRenderer: class BufferedPageRenderer:
""" """
High-level interface for buffered page rendering with automatic background caching. High-level interface for page rendering with an LRU cache in front of the
layouter.
""" """
def __init__(self, def __init__(self,
@@ -390,7 +241,7 @@ class BufferedPageRenderer:
def render_page(self, position: RenderingPosition, def render_page(self, position: RenderingPosition,
font_scale: float = 1.0) -> Tuple[Page, RenderingPosition]: font_scale: float = 1.0) -> Tuple[Page, RenderingPosition]:
""" """
Render a page with intelligent caching. Render a page, serving it from cache when possible.
Args: Args:
position: Position to render from position: Position to render from
@@ -407,32 +258,18 @@ class BufferedPageRenderer:
# Check cache first # Check cache first
cached_page = self.buffer.get_page(position) cached_page = self.buffer.get_page(position)
if cached_page: if cached_page:
# Get next position from position map # Only use the cache if we also know where the next page starts;
# otherwise fall through and compute it.
next_pos = self.buffer.position_map.get(position) next_pos = self.buffer.position_map.get(position)
# Only use cache if we have the forward position mapping
# Otherwise, we need to compute it
if next_pos is not None: if next_pos is not None:
# Start background rendering for upcoming pages
self.buffer.start_background_rendering(position, 'forward')
return cached_page, next_pos return cached_page, next_pos
# Cache hit for the page, but we don't have the forward position
# Fall through to compute it below
# Render the page directly # Render the page directly
page, next_pos = self.layouter.render_page_forward(position, font_scale) page, next_pos = self.layouter.render_page_forward(position, font_scale)
# Cache the result # Cache the result
self.buffer.cache_page(position, page, next_pos) self.buffer.cache_page(position, page, next_pos)
# Start background rendering
self.buffer.start_background_rendering(position, 'both')
# Check for completed background renders
self.buffer.check_completed_renders()
return page, next_pos return page, next_pos
def render_page_backward(self, def render_page_backward(self,
@@ -440,7 +277,8 @@ class BufferedPageRenderer:
font_scale: float = 1.0) -> Tuple[Page, font_scale: float = 1.0) -> Tuple[Page,
RenderingPosition]: RenderingPosition]:
""" """
Render a page ending at the given position with intelligent caching. Render a page ending at the given position, serving it from cache when
possible.
Args: Args:
end_position: Position where page should end end_position: Position where page should end
@@ -457,32 +295,18 @@ class BufferedPageRenderer:
# Check cache first # Check cache first
cached_page = self.buffer.get_page(end_position) cached_page = self.buffer.get_page(end_position)
if cached_page: if cached_page:
# Get previous position from reverse position map # Only use the cache if we also know where the previous page
# starts; otherwise fall through and compute it.
prev_pos = self.buffer.reverse_position_map.get(end_position) prev_pos = self.buffer.reverse_position_map.get(end_position)
# Only use cache if we have the reverse position mapping
# Otherwise, we need to compute it
if prev_pos is not None: if prev_pos is not None:
# Start background rendering for previous pages
self.buffer.start_background_rendering(end_position, 'backward')
return cached_page, prev_pos return cached_page, prev_pos
# Cache hit for the page, but we don't have the reverse position
# Fall through to compute it below
# Render the page directly # Render the page directly
page, start_pos = self.layouter.render_page_backward(end_position, font_scale) page, start_pos = self.layouter.render_page_backward(end_position, font_scale)
# Cache the result # Cache the result
self.buffer.cache_page(start_pos, page, end_position, is_backward=True) self.buffer.cache_page(start_pos, page, end_position, is_backward=True)
# Start background rendering
self.buffer.start_background_rendering(end_position, 'both')
# Check for completed background renders
self.buffer.check_completed_renders()
return page, start_pos return page, start_pos
def set_font_family(self, font_family: Optional[BundledFont]): def set_font_family(self, font_family: Optional[BundledFont]):
@@ -516,5 +340,5 @@ class BufferedPageRenderer:
return self.buffer.get_cache_stats() return self.buffer.get_cache_stats()
def shutdown(self): def shutdown(self):
"""Shutdown the renderer and clean up resources""" """Release cached pages"""
self.buffer.shutdown() self.buffer.shutdown()
+13 -1
View File
@@ -112,7 +112,17 @@ class AbstractStyle:
Since this is a frozen dataclass, it should be hashable by default, Since this is a frozen dataclass, it should be hashable by default,
but we provide a custom implementation to ensure all fields are but we provide a custom implementation to ensure all fields are
properly considered and to handle the Union types correctly. properly considered and to handle the Union types correctly.
The result is memoised on first use. Styles are used as dictionary keys
throughout parsing and style resolution, and five of the fields are enum
members whose own __hash__ is a Python-level call, so rebuilding the
15-tuple on every lookup was a measurable share of document parsing. The
class is frozen, so the value cannot go stale.
""" """
cached = self.__dict__.get('_hash_cache')
if cached is not None:
return cached
# Convert all values to hashable forms # Convert all values to hashable forms
hashable_values = ( hashable_values = (
self.font_family, self.font_family,
@@ -132,7 +142,9 @@ class AbstractStyle:
self.parent_style_id self.parent_style_id
) )
return hash(hashable_values) result = hash(hashable_values)
object.__setattr__(self, '_hash_cache', result)
return result
def merge_with(self, other: 'AbstractStyle') -> 'AbstractStyle': def merge_with(self, other: 'AbstractStyle') -> 'AbstractStyle':
""" """
+37 -5
View File
@@ -4,24 +4,56 @@ build-backend = "setuptools.build_meta"
[project] [project]
name = "pyWebLayout" name = "pyWebLayout"
version = "0.1.1"
description = "A Python library for HTML-like layout and rendering" description = "A Python library for HTML-like layout and rendering"
readme = "README.md" readme = "README.md"
requires-python = ">=3.6" requires-python = ">=3.10"
license = {file = "LICENSE"} license = {file = "LICENSE"}
authors = [ authors = [
{name = "Duncan Tourolle", email = "duncan@tourolle.paris"} {name = "Duncan Tourolle", email = "duncan@tourolle.paris"}
] ]
dynamic = ["version"] classifiers = [
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
]
dependencies = [ dependencies = [
"Pillow", "Pillow",
"numpy", "numpy",
"pyphen", "pyphen",
"beautifulsoup4", "beautifulsoup4",
"flask",
"ebooklib",
"requests"
] ]
[project.urls]
Homepage = "https://gitea.tourolle.paris/pyWebLayout"
[project.optional-dependencies]
# Loading images from http(s) URLs. concrete.image imports requests lazily and
# degrades to an error message on the image when it is absent, so it is not a
# hard requirement.
remote-images = ["requests"]
test = [
"pytest>=6.0",
"pytest-cov",
"flask", # fixture HTTP server in tests/abstract/test_abstract_blocks.py
"werkzeug", # make_server, same fixture
"ebooklib", # builds EPUB fixtures; the reader itself uses zipfile + ElementTree
"requests", # exercises the remote-images path
]
dev = [
"pyWebLayout[test,remote-images]",
"flake8",
"coverage-badge",
"interrogate",
]
[tool.setuptools.packages.find]
include = ["pyWebLayout*"]
[tool.coverage.run] [tool.coverage.run]
source = ["pyWebLayout"] source = ["pyWebLayout"]
branch = true branch = true
+3 -23
View File
@@ -1,26 +1,6 @@
[metadata] # Packaging metadata lives in pyproject.toml ([project]), which takes
name = pyWebLayout # precedence over anything declared here. This file keeps only tool config
version = 0.1.1 # that has nowhere better to live.
author = Duncan Tourolle
author_email = duncan@tourolle.paris
description = A Python library for HTML-like layout and rendering
long_description = file: README.md
long_description_content_type = text/markdown
url = https://gitea.tourolle.paris/pyWebLayout
classifiers =
Programming Language :: Python :: 3
License :: OSI Approved :: MIT License
Operating System :: OS Independent
[options]
packages = find:
python_requires = >=3.6
install_requires =
Pillow
numpy
[options.packages.find]
include = pyWebLayout*
[flake8] [flake8]
exclude = exclude =
+9 -31
View File
@@ -1,32 +1,10 @@
from setuptools import setup, find_packages """Shim for legacy `python setup.py` invocations.
setup( All packaging metadata lives in setup.cfg. Keeping a second copy here was an
name="pyWebLayout", active hazard: keyword arguments passed to setup() override setup.cfg, so the
version="0.1.1", two could disagree silently and the setup.py copy would win.
packages=find_packages(), """
install_requires=[
"Pillow", from setuptools import setup
"numpy",
], setup()
extras_require={
"test": [
"coverage>=5.0",
],
"dev": [
"coverage>=5.0",
"pytest>=6.0",
],
},
author="Duncan Tourolle",
author_email="duncan@tourolle.paris",
description="A Python library for HTML-like layout and rendering",
long_description=open("README.md").read(),
long_description_content_type="text/markdown",
url="https://gitea.tourolle.paris/pyWebLayout",
classifiers=[
"Programming Language :: Python :: 3",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
],
python_requires=">=3.6",
)
+18 -9
View File
@@ -307,6 +307,8 @@ class TestImagePIL(unittest.TestCase):
if FLASK_AVAILABLE and hasattr(cls, 'flask_thread'): if FLASK_AVAILABLE and hasattr(cls, 'flask_thread'):
cls.flask_server_running = False cls.flask_server_running = False
cls.flask_server.shutdown()
cls.flask_server.server_close()
cls.flask_thread.join(timeout=2) cls.flask_thread.join(timeout=2)
@classmethod @classmethod
@@ -350,10 +352,9 @@ class TestImagePIL(unittest.TestCase):
"""Start a Flask server for URL testing.""" """Start a Flask server for URL testing."""
import urllib.request import urllib.request
import urllib.error import urllib.error
from werkzeug.serving import make_server
cls.flask_app = Flask(__name__) cls.flask_app = Flask(__name__)
cls.flask_port = 5555 # Use a specific port for testing
cls.flask_server_running = True
@cls.flask_app.route('/test.jpg') @cls.flask_app.route('/test.jpg')
def serve_test_image(): def serve_test_image():
@@ -363,15 +364,20 @@ class TestImagePIL(unittest.TestCase):
def health_check(): def health_check():
return 'OK', 200 return 'OK', 200
def run_flask(): # Bind to an ephemeral port so concurrent/leftover test runs can't clash
cls.flask_app.run(host='127.0.0.1', port=cls.flask_port, debug=False, cls.flask_server = make_server('127.0.0.1', 0, cls.flask_app, threaded=True)
use_reloader=False, threaded=True) cls.flask_port = cls.flask_server.server_port
cls.flask_server_running = True
cls.flask_thread = threading.Thread(target=run_flask, daemon=True) cls.flask_thread = threading.Thread(target=cls.flask_server.serve_forever, daemon=True)
cls.flask_thread.start() cls.flask_thread.start()
# Wait for server to be ready with health check # Wait for server to be ready with health check.
max_wait = 5 # Maximum 5 seconds # Generous, because this now raises rather than falling through
# silently: on a loaded CI runner the accept loop can take several
# seconds to get scheduled, and a spurious failure here is worse than
# a slow one. The loop exits as soon as the server answers.
max_wait = 30
wait_interval = 0.1 # Check every 100ms wait_interval = 0.1 # Check every 100ms
elapsed = 0 elapsed = 0
@@ -379,12 +385,15 @@ class TestImagePIL(unittest.TestCase):
try: try:
with urllib.request.urlopen(f'http://127.0.0.1:{cls.flask_port}/health', timeout=1) as response: with urllib.request.urlopen(f'http://127.0.0.1:{cls.flask_port}/health', timeout=1) as response:
if response.status == 200: if response.status == 200:
break return
except (urllib.error.URLError, ConnectionRefusedError, OSError): except (urllib.error.URLError, ConnectionRefusedError, OSError):
pass pass
time.sleep(wait_interval) time.sleep(wait_interval)
elapsed += wait_interval elapsed += wait_interval
raise RuntimeError(
f"Test Flask server did not become ready on port {cls.flask_port} within {max_wait}s")
def test_image_url_detection(self): def test_image_url_detection(self):
"""Test URL detection functionality.""" """Test URL detection functionality."""
img = Image() img = Image()
+5 -2
View File
@@ -334,8 +334,11 @@ class TestFormFieldText(unittest.TestCase):
"""Test size property includes field area""" """Test size property includes field area"""
renderable = FormFieldText(self.text_field, self.font, self.mock_draw) renderable = FormFieldText(self.text_field, self.font, self.mock_draw)
# Size should include label height + gap + field height # Size should include label height + gap + field height. The label's
expected_height = renderable._style.font_size + 5 + renderable._field_height # height is its ink height (ascent + descent), not the nominal font size.
ascent, descent = renderable._style.font.getmetrics()
expected_height = (ascent + descent) + FormFieldText.LABEL_GAP \
+ renderable._field_height
expected_width = renderable._field_width # Use the calculated field width expected_width = renderable._field_width # Use the calculated field width
np.testing.assert_array_equal( np.testing.assert_array_equal(
+140
View File
@@ -0,0 +1,140 @@
"""
Regression tests for form field label geometry (spec S15).
Text renders with a baseline anchor, so drawing the label at the field's origin
put its glyphs above that origin - outside the box the field claims through size
and in_object. Stacked fields therefore had each label overprinting the input box
of the field before it.
"""
import numpy as np
import pytest
from PIL import Image, ImageDraw
from pyWebLayout.abstract.functional import Form, FormField, FormFieldType
from pyWebLayout.concrete.functional import FormFieldText
from pyWebLayout.concrete.page import Page
from pyWebLayout.layout.document_layouter import form_layouter
from pyWebLayout.style import Font
from pyWebLayout.style.page_style import PageStyle
ORIGIN = (10, 40)
FIELD_HEIGHT = 24
@pytest.fixture
def font():
return Font(font_size=12, colour=(0, 0, 0))
@pytest.fixture
def canvas():
image = Image.new("RGB", (300, 200), (255, 255, 255))
return image, ImageDraw.Draw(image)
def make_field(font, draw, label="Email Address"):
field = FormField(name="email", field_type=FormFieldType.TEXT, label=label)
renderable = FormFieldText(field, font, draw, field_height=FIELD_HEIGHT)
renderable.set_origin(np.array(list(ORIGIN)))
return renderable
def ink_rows(image, x_range, y_range):
pixels = image.convert("RGB").load()
return [y for y in y_range
if any(sum(pixels[x, y]) < 400 for x in x_range)]
class TestLabelStaysInsideTheFieldBox:
def test_label_ink_is_below_the_origin(self, font, canvas):
image, draw = canvas
renderable = make_field(font, draw)
renderable.render()
rows = ink_rows(image, range(ORIGIN[0], ORIGIN[0] + 140),
range(0, ORIGIN[1]))
assert not rows, \
f"label drew above its own origin, at rows {rows}"
def test_label_and_box_do_not_overlap(self, font, canvas):
image, draw = canvas
renderable = make_field(font, draw)
renderable.render()
ascent, descent = font.font.getmetrics()
label_bottom = ORIGIN[1] + ascent + descent
box_top = renderable.field_area_offset + ORIGIN[1]
assert box_top >= label_bottom, \
"the input box must start below the label's descenders"
def test_reported_height_covers_everything_drawn(self, font, canvas):
image, draw = canvas
renderable = make_field(font, draw)
renderable.render()
top, bottom = ORIGIN[1], ORIGIN[1] + int(renderable.size[1])
rows = ink_rows(image, range(ORIGIN[0], ORIGIN[0] + 200), range(0, 200))
assert min(rows) >= top, "ink above the field's declared box"
assert max(rows) < bottom, "ink below the field's declared box"
class TestStackedFieldsDoNotCollide:
def test_form_layout_leaves_labels_clear(self, font):
page = Page(size=(300, 400), style=PageStyle())
form = Form("signup")
for name in ["Username", "Email Address", "Password"]:
form.add_field(FormField(name=name.lower().replace(" ", "_"),
field_type=FormFieldType.TEXT, label=name))
ok, ids = form_layouter(form, page, font)
assert ok and len(ids) == 3
fields = [c for c in page.children if isinstance(c, FormFieldText)]
assert len(fields) == 3
for earlier, later in zip(fields, fields[1:]):
earlier_bottom = earlier.origin[1] + earlier.size[1]
assert later.origin[1] >= earlier_bottom, \
"fields overlap: a label would print over the preceding input box"
def test_rendered_form_has_no_ink_collisions(self, font):
"""Every field's ink stays within its own declared bounds."""
page = Page(size=(300, 400), style=PageStyle())
form = Form("signup")
for name in ["Username", "Email Address"]:
form.add_field(FormField(name=name.lower(), field_type=FormFieldType.TEXT,
label=name))
form_layouter(form, page, font)
image = page.render()
fields = [c for c in page.children if isinstance(c, FormFieldText)]
for field in fields:
top = int(field.origin[1])
bottom = top + int(field.size[1])
rows = ink_rows(image, range(int(field.origin[0]),
int(field.origin[0] + field.size[0])),
range(max(0, top - 6), top))
assert not rows, f"ink found just above a field at y={top}"
class TestClickTargetsFollowTheLayout:
def test_click_in_the_input_area_focuses(self, font, canvas):
_, draw = canvas
renderable = make_field(font, draw)
inside = (5, renderable.field_area_offset + FIELD_HEIGHT // 2)
assert renderable.handle_click(inside) is True
assert renderable._focused is True
def test_click_on_the_label_does_not_focus(self, font, canvas):
_, draw = canvas
renderable = make_field(font, draw)
on_label = (5, 2)
assert renderable.handle_click(on_label) is False
+171
View File
@@ -0,0 +1,171 @@
"""
Regression tests for backward page navigation (spec S16).
The previous page of P is the position q for which laying out forward from q ends
exactly at P. The old implementation searched for q by guessing a block index and
bisecting, with word_index pinned to 0 - so a page starting mid-paragraph was not
in the search space at all. It exhausted its ten iterations and fell back to a
position that was not the previous page, typically the start of the document.
"""
import pytest
from pyWebLayout.abstract.block import Paragraph
from pyWebLayout.abstract.inline import Word
from pyWebLayout.layout.ereader_layout import BidirectionalLayouter, RenderingPosition
from pyWebLayout.style import Font
from pyWebLayout.style.page_style import PageStyle
PAGE_SIZE = (800, 600)
@pytest.fixture
def font():
return Font(font_size=16)
def paragraph(font, count, tag):
block = Paragraph(font)
for i in range(count):
block.add_word(Word(f"{tag}{i}", font))
return block
@pytest.fixture
def long_document(font):
"""Short paragraphs around one that spans several pages."""
return [
paragraph(font, 60, "a"),
paragraph(font, 80, "b"),
paragraph(font, 1200, "long"),
paragraph(font, 70, "c"),
paragraph(font, 90, "d"),
]
@pytest.fixture
def block_document(font):
"""Many small blocks, so every page starts on a block boundary."""
return [paragraph(font, 40, f"p{i}") for i in range(40)]
def forward_chain(layouter, limit=30):
"""The page start positions a reader would visit going forward."""
starts = []
pos = RenderingPosition()
for _ in range(limit):
starts.append(pos)
_, nxt = layouter.render_page_forward(pos, 1.0)
if nxt.block_index >= len(layouter.blocks):
break
if (nxt.block_index, nxt.word_index) == (pos.block_index, pos.word_index):
pytest.fail("forward pagination made no progress")
pos = nxt
return starts
def key(position):
return (position.chapter_index, position.block_index, position.word_index)
class TestBackwardMatchesForward:
"""The defining invariant: forward from the answer lands exactly on P."""
@pytest.mark.parametrize("document", ["long_document", "block_document"])
def test_previous_page_is_the_forward_predecessor(self, document, request):
blocks = request.getfixturevalue(document)
layouter = BidirectionalLayouter(blocks, PageStyle(), PAGE_SIZE)
starts = forward_chain(layouter)
assert len(starts) > 2, "need a few pages to test against"
for i in range(1, len(starts)):
_, got = layouter.render_page_backward(starts[i], 1.0)
assert key(got) == key(starts[i - 1]), (
f"page {i}: expected to land on page {i - 1} "
f"{key(starts[i - 1])}, got {key(got)}")
def test_result_lays_out_to_the_target(self, long_document):
"""Independent of the recorded chain: replaying the answer must reach P."""
layouter = BidirectionalLayouter(long_document, PageStyle(), PAGE_SIZE)
starts = forward_chain(layouter)
for target in starts[1:]:
_, start = layouter.render_page_backward(target, 1.0)
_, end = layouter.render_page_forward(start, 1.0)
assert key(end) == key(target), (
f"a page starting at {key(start)} ends at {key(end)}, "
f"not at the requested {key(target)}")
def test_mid_paragraph_targets_are_reachable(self, long_document):
"""The specific regression: starts inside a block, not on its boundary."""
layouter = BidirectionalLayouter(long_document, PageStyle(), PAGE_SIZE)
starts = forward_chain(layouter)
mid = [s for s in starts if s.word_index > 0]
assert mid, "this document should paginate mid-paragraph"
for target in mid:
_, got = layouter.render_page_backward(target, 1.0)
assert key(got) != (0, 0, 0) or key(target) == key(starts[1]), \
"backward navigation fell back to the document start"
class TestRoundTrip:
def test_forward_then_back_returns_to_the_same_place(self, long_document):
layouter = BidirectionalLayouter(long_document, PageStyle(), PAGE_SIZE)
pos = RenderingPosition()
for _ in range(4):
_, nxt = layouter.render_page_forward(pos, 1.0)
_, back = layouter.render_page_backward(nxt, 1.0)
assert key(back) == key(pos), \
f"round trip drifted: {key(pos)} -> {key(nxt)} -> {key(back)}"
pos = nxt
class TestEdges:
def test_at_document_start_stays_there(self, long_document):
layouter = BidirectionalLayouter(long_document, PageStyle(), PAGE_SIZE)
_, got = layouter.render_page_backward(RenderingPosition(), 1.0)
assert key(got) == (0, 0, 0)
def test_second_page_goes_back_to_the_first(self, long_document):
layouter = BidirectionalLayouter(long_document, PageStyle(), PAGE_SIZE)
_, second = layouter.render_page_forward(RenderingPosition(), 1.0)
_, got = layouter.render_page_backward(second, 1.0)
assert key(got) == (0, 0, 0)
def test_empty_document_is_safe(self):
layouter = BidirectionalLayouter([], PageStyle(), PAGE_SIZE)
page, got = layouter.render_page_backward(RenderingPosition(), 1.0)
assert page is not None
assert key(got) == (0, 0, 0)
class TestCost:
def test_backward_is_not_wildly_more_expensive_than_forward(self, long_document):
"""The old path burned ten full layouts per call and still got it wrong."""
layouter = BidirectionalLayouter(long_document, PageStyle(), PAGE_SIZE)
starts = forward_chain(layouter)
calls = {"n": 0}
original = BidirectionalLayouter.render_page_forward
def counting(self, position, font_scale=1.0):
calls["n"] += 1
return original(self, position, font_scale)
BidirectionalLayouter.render_page_forward = counting
try:
worst = 0
for target in starts[1:]:
calls["n"] = 0
layouter.render_page_backward(target, 1.0)
worst = max(worst, calls["n"])
finally:
BidirectionalLayouter.render_page_forward = original
assert worst <= 10, f"backward navigation cost {worst} forward layouts"
+168
View File
@@ -0,0 +1,168 @@
"""
Tests for the highlight API on EreaderLayoutManager (R7).
core/highlight.py was fully implemented and tested but unreachable: the manager
had no highlight API, so highlighting could not be used through the library's
own interface. These tests cover the wiring, not the dataclass - that is
tests/core/test_highlight.py.
"""
import pytest
from pyWebLayout.core.highlight import Highlight, HighlightColor
from pyWebLayout.io.readers.html_extraction import parse_html_string
from pyWebLayout.layout.ereader_manager import EreaderLayoutManager
@pytest.fixture
def manager(tmp_path):
blocks = parse_html_string(
"<p>" + " ".join(f"word{i}" for i in range(300)) + "</p>")
manager = EreaderLayoutManager(blocks, page_size=(400, 600),
document_id="highlights",
bookmarks_dir=str(tmp_path))
yield manager
manager.shutdown()
def text_points(page, limit=None):
"""Points on the rendered page that land on a text object."""
found = []
for y in range(0, 120, 2):
for x in range(0, 400, 2):
result = page.query_point((x, y))
if result is not None and result.object_type == "text" and result.text:
found.append((x, y))
if limit and len(found) >= limit:
return found
return found
@pytest.fixture
def point_on_text(manager):
page = manager.get_current_page()
page.render()
return text_points(page, limit=1)[0]
class TestHighlightPoint:
def test_highlighting_a_word_returns_a_stored_highlight(self, manager, point_on_text):
highlight = manager.highlight_point(point_on_text)
assert isinstance(highlight, Highlight)
assert highlight.text
assert manager.list_highlights() == [highlight]
def test_colour_note_and_tags_are_kept(self, manager, point_on_text):
highlight = manager.highlight_point(
point_on_text, color=HighlightColor.GREEN.value,
note="a note", tags=["review"])
assert highlight.color == HighlightColor.GREEN.value
assert highlight.note == "a note"
assert highlight.tags == ["review"]
def test_highlighting_empty_space_returns_none(self, manager):
manager.get_current_page().render()
assert manager.highlight_point((399, 599)) is None
assert manager.list_highlights() == []
def test_the_originating_position_is_recorded(self, manager, point_on_text):
highlight = manager.highlight_point(point_on_text)
assert highlight.position == manager.current_position.to_dict()
class TestHighlightRange:
def test_a_selection_spans_multiple_words(self, manager):
page = manager.get_current_page()
page.render()
points = text_points(page)
highlight = manager.highlight_range(points[0], points[-1])
assert highlight is not None
assert len(highlight.text.split()) > 1
assert len(highlight.bounds) > 1
def test_a_selection_hitting_no_text_returns_none(self, manager):
manager.get_current_page().render()
assert manager.highlight_range((398, 596), (399, 599)) is None
class TestHighlightsAreScopedToTheirPage:
def test_current_page_highlights_do_not_leak_across_pages(self, manager, point_on_text):
manager.highlight_point(point_on_text)
assert len(manager.get_highlights_for_current_page()) == 1
manager.next_page()
assert manager.get_highlights_for_current_page() == []
assert len(manager.list_highlights()) == 1, "still in the document, just not here"
def test_returning_to_the_page_finds_it_again(self, manager, point_on_text):
highlight = manager.highlight_point(point_on_text)
manager.next_page()
manager.previous_page()
assert manager.get_highlights_for_current_page() == [highlight]
class TestPersistence:
def test_highlights_survive_a_restart(self, manager, point_on_text, tmp_path):
highlight = manager.highlight_point(point_on_text, note="kept")
manager.shutdown()
reopened = EreaderLayoutManager(
manager.blocks, page_size=(400, 600), document_id="highlights",
bookmarks_dir=str(tmp_path))
try:
restored = reopened.list_highlights()
assert len(restored) == 1
assert restored[0].id == highlight.id
assert restored[0].note == "kept"
assert restored[0].position == highlight.position
finally:
reopened.shutdown()
def test_highlights_share_the_bookmarks_directory_by_default(self, manager,
point_on_text, tmp_path):
manager.highlight_point(point_on_text)
assert (tmp_path / "highlights_highlights.json").exists()
def test_removing_a_highlight_persists(self, manager, point_on_text, tmp_path):
highlight = manager.highlight_point(point_on_text)
assert manager.remove_highlight(highlight.id) is True
assert manager.remove_highlight(highlight.id) is False
reopened = EreaderLayoutManager(
manager.blocks, page_size=(400, 600), document_id="highlights",
bookmarks_dir=str(tmp_path))
try:
assert reopened.list_highlights() == []
finally:
reopened.shutdown()
def test_clear_removes_everything(self, manager, point_on_text):
manager.highlight_point(point_on_text)
manager.clear_highlights()
assert manager.list_highlights() == []
def test_a_corrupt_store_does_not_stop_the_book_opening(self, tmp_path):
(tmp_path / "broken_highlights.json").write_text("{not json")
blocks = parse_html_string("<p>hello world</p>")
manager = EreaderLayoutManager(blocks, page_size=(400, 600),
document_id="broken",
bookmarks_dir=str(tmp_path))
try:
assert manager.list_highlights() == []
assert manager.get_current_page() is not None
finally:
manager.shutdown()
+129
View File
@@ -0,0 +1,129 @@
"""
Tests for pointer interaction on EreaderLayoutManager (R7).
concrete/interaction_handler.py was 310 lines reachable only from
examples/07_pressed_state_demo.py - no library code, no tests. These cover the
wiring; the press/hover state on the elements themselves lives in
tests/concrete/.
"""
import pytest
from PIL import Image
from pyWebLayout.io.readers.html_extraction import parse_html_string
from pyWebLayout.layout.ereader_manager import EreaderLayoutManager
@pytest.fixture
def manager(tmp_path):
blocks = parse_html_string(
'<p>Tap <a href="action:go">this link</a> please.</p>'
'<p>' + " ".join(f"w{i}" for i in range(400)) + '</p>')
manager = EreaderLayoutManager(blocks, page_size=(400, 600),
document_id="interaction",
bookmarks_dir=str(tmp_path))
yield manager
manager.shutdown()
@pytest.fixture
def link_point(manager):
"""A page coordinate that lands on the interactive link."""
page = manager.get_current_page()
page.render()
for y in range(0, 120, 2):
for x in range(0, 400, 2):
result = page.query_point((x, y))
if result is not None and result.is_interactive:
return (x, y)
pytest.fail("fixture document rendered no interactive element")
EMPTY_POINT = (399, 599)
class TestHover:
def test_hovering_an_element_produces_a_frame(self, manager, link_point):
assert isinstance(manager.handle_hover(link_point), Image.Image)
def test_hovering_the_same_element_again_reports_no_change(self, manager, link_point):
manager.handle_hover(link_point)
assert manager.handle_hover(link_point) is None, \
"an unchanged hover should not force the caller to redraw"
def test_moving_off_the_element_clears_the_hover(self, manager, link_point):
manager.handle_hover(link_point)
assert isinstance(manager.handle_hover(EMPTY_POINT), Image.Image)
class TestPress:
def test_pressing_an_element_produces_a_frame(self, manager, link_point):
assert isinstance(manager.handle_touch_down(link_point), Image.Image)
def test_pressing_empty_space_does_nothing(self, manager):
manager.get_current_page().render()
assert manager.handle_touch_down(EMPTY_POINT) is None
def test_release_runs_the_link_action(self, manager, link_point):
manager.handle_touch_down(link_point)
frame, result = manager.handle_touch_up(link_point)
assert isinstance(frame, Image.Image)
assert result == "action:go"
def test_release_without_a_press_is_a_no_op(self, manager):
manager.get_current_page().render()
assert manager.handle_touch_up(EMPTY_POINT) == (None, None)
def test_a_full_press_release_cycle_leaves_no_state(self, manager, link_point):
manager.handle_touch_down(link_point)
manager.handle_touch_up(link_point)
assert manager.handle_touch_up(link_point) == (None, None)
class TestStateFollowsTheDisplayedPage:
def test_navigating_rebinds_the_state_machine(self, manager, link_point):
before = manager._interaction_state()
manager.next_page()
assert manager._interaction_state() is not before, \
"press state belongs to one rendered page"
def test_state_survives_repeated_access_on_one_page(self, manager, link_point):
assert manager._interaction_state() is manager._interaction_state()
def test_reset_is_safe_before_any_interaction(self, manager):
manager.reset_interaction_state() # must not raise
def test_reset_clears_a_pending_press(self, manager, link_point):
manager.handle_touch_down(link_point)
manager.reset_interaction_state()
assert manager.handle_touch_up(link_point) == (None, None)
class TestPressedRenderingRegression:
"""
LinkText.render passed [origin, origin + size] - two numpy arrays - to
PIL's draw.rectangle, which needs a flat four-scalar box. Rendering any
hovered or pressed link raised TypeError. Nothing caught it because the
only caller was an example.
"""
def test_rendering_a_hovered_link_does_not_raise(self, manager, link_point):
manager.handle_hover(link_point)
assert isinstance(manager.get_current_page().render(), Image.Image)
def test_rendering_a_pressed_link_does_not_raise(self, manager, link_point):
manager.handle_touch_down(link_point)
assert isinstance(manager.get_current_page().render(), Image.Image)
+38 -68
View File
@@ -570,30 +570,6 @@ class TestBidirectionalLayouter:
# Should return same block # Should return same block
assert scaled == paragraph assert scaled == paragraph
def test_estimate_page_start(self):
"""Test estimation of page start position."""
layouter = BidirectionalLayouter([], PageStyle())
end_pos = RenderingPosition(chapter_index=0, block_index=20, word_index=0)
estimated = layouter._estimate_page_start(end_pos, 1.0)
# Should estimate some blocks before the end position
assert estimated.block_index < end_pos.block_index
assert estimated.block_index >= 0
def test_estimate_page_start_with_font_scale(self):
"""Test that font scale affects page start estimation."""
layouter = BidirectionalLayouter([], PageStyle())
end_pos = RenderingPosition(chapter_index=0, block_index=20, word_index=0)
est_normal = layouter._estimate_page_start(end_pos, 1.0)
est_large = layouter._estimate_page_start(end_pos, 2.0)
# Larger font should estimate fewer blocks
assert est_large.block_index >= est_normal.block_index
def test_scale_block_fonts_paragraph(self, sample_font): def test_scale_block_fonts_paragraph(self, sample_font):
"""Test scaling fonts in a paragraph block.""" """Test scaling fonts in a paragraph block."""
layouter = BidirectionalLayouter([], PageStyle()) layouter = BidirectionalLayouter([], PageStyle())
@@ -784,50 +760,6 @@ class TestBidirectionalLayouter:
# Start position should be before or at end position # Start position should be before or at end position
assert start_pos.block_index <= end_position.block_index assert start_pos.block_index <= end_position.block_index
def test_adjust_start_estimate_overshot(self):
"""Test adjustment when forward render overshoots target."""
layouter = BidirectionalLayouter([], PageStyle())
current_start = RenderingPosition(block_index=5)
target_end = RenderingPosition(block_index=10)
actual_end = RenderingPosition(block_index=12) # Overshot (went too far)
adjusted = layouter._adjust_start_estimate(
current_start, target_end, actual_end)
# Overshot means we rendered too far forward
# So we need to start EARLIER (decrease block_index) to not go as far
assert adjusted.block_index < current_start.block_index
def test_adjust_start_estimate_undershot(self):
"""Test adjustment when forward render undershoots target."""
layouter = BidirectionalLayouter([], PageStyle())
current_start = RenderingPosition(block_index=5)
target_end = RenderingPosition(block_index=10)
actual_end = RenderingPosition(block_index=8) # Undershot (didn't go far enough)
adjusted = layouter._adjust_start_estimate(
current_start, target_end, actual_end)
# Undershot means we didn't render far enough forward
# So we need to start LATER (increase block_index) to include more content
assert adjusted.block_index > current_start.block_index
def test_adjust_start_estimate_exact(self):
"""Test adjustment when forward render hits target exactly."""
layouter = BidirectionalLayouter([], PageStyle())
current_start = RenderingPosition(block_index=5)
target_end = RenderingPosition(block_index=10)
actual_end = RenderingPosition(block_index=10) # Exact
adjusted = layouter._adjust_start_estimate(
current_start, target_end, actual_end)
# Should return same or similar position
assert adjusted.block_index >= 0
def test_layout_paragraph_on_page_with_pretext( def test_layout_paragraph_on_page_with_pretext(
self, sample_font, sample_page_style): self, sample_font, sample_page_style):
"""Test paragraph layout with pretext (hyphenated word continuation).""" """Test paragraph layout with pretext (hyphenated word continuation)."""
@@ -899,5 +831,43 @@ class TestBidirectionalLayouter:
assert next_pos == position # No progress possible assert next_pos == position # No progress possible
class TestNoPageMonkeyPatching:
"""
R5: importing this module used to run _add_page_methods(), which attached
can_fit_line/available_width to Page if they were absent. They are not
absent, so it never fired - but its can_fit_line took (line_height) and
ignored descenders, while Page's takes (baseline_spacing, ascent, descent).
Had Page's ever been renamed, the import would have silently reinstated the
pre-S2 clipping bug from another package.
"""
def test_module_does_not_patch_page(self):
import pyWebLayout.layout.ereader_layout as ereader_layout
assert not hasattr(ereader_layout, '_add_page_methods')
def test_page_owns_its_geometry_methods(self):
from pyWebLayout.concrete.page import Page
assert 'can_fit_line' in vars(Page)
assert 'available_width' in vars(Page)
def test_can_fit_line_still_accounts_for_descenders(self, sample_page_style):
"""
The patched version took a single line_height and had no way to express
descent, so a descender hanging past the content box counted as fitting.
"""
from pyWebLayout.concrete.page import Page
page = Page(size=(200, 100), style=sample_page_style)
content_y, content_h = page.content_rect[1], page.content_rect[3]
available = content_y + content_h - page._current_y_offset
assert page.can_fit_line(0, ascent=available, descent=0)
assert not page.can_fit_line(0, ascent=available, descent=1), \
"a descender past the content box must not be reported as fitting"
assert page.can_fit_line(0, ascent=available - 1, descent=1)
if __name__ == "__main__": if __name__ == "__main__":
pytest.main([__file__, "-v"]) pytest.main([__file__, "-v"])
+262
View File
@@ -0,0 +1,262 @@
"""
Tests for font scaling in the ereader layout path (R3).
_scale_block_fonts rebuilds a block with scaled fonts. It used to construct a
plain Word for every word, which downgraded LinkedWord and silently discarded
every hyperlink in the document as soon as the reader changed font size. It
also handled only Paragraph and Heading, so quotes, lists and tables kept their
original size while the text around them reflowed.
"""
import tempfile
import pytest
from pyWebLayout.abstract.block import Paragraph, Heading, Quote, HList, Table
from pyWebLayout.abstract.inline import LinkedWord, Word
from pyWebLayout.concrete.functional import LinkText
from pyWebLayout.io.readers.html_extraction import parse_html_string
from pyWebLayout.layout.ereader_layout import BidirectionalLayouter
from pyWebLayout.layout.ereader_manager import EreaderLayoutManager
from pyWebLayout.style import Font
from pyWebLayout.style.page_style import PageStyle
HTML = """
<p>Go to <a href="http://example.com" title="Tooltip">this link</a> now.</p>
<blockquote><p>Quoted <a href="http://q.example">qlink</a> text.</p></blockquote>
<ul><li>item <a href="http://l.example">llink</a> one</li></ul>
<table>
<thead><tr><th>head <a href="http://h.example">hlink</a></th></tr></thead>
<tbody><tr><td>cell <a href="http://c.example">clink</a></td></tr></tbody>
</table>
"""
def collect_links(block, out=None):
"""Every LinkedWord reachable in a block, at any nesting depth."""
out = [] if out is None else out
if isinstance(block, Paragraph): # covers Heading
for _, word in block.words_iter():
if isinstance(word, LinkedWord):
out.append(word)
elif isinstance(block, Quote):
for child in block.blocks():
collect_links(child, out)
elif isinstance(block, HList):
for item in block.items():
for child in item.blocks():
collect_links(child, out)
elif isinstance(block, Table):
for rows in (block.header_rows(), block.body_rows(), block.footer_rows()):
for row in rows:
for cell in row.cells():
for child in cell.blocks():
collect_links(child, out)
return out
def collect_sizes(block, out=None):
"""Every font size reachable in a block, at any nesting depth."""
out = [] if out is None else out
if isinstance(block, Paragraph):
for _, word in block.words_iter():
out.append(word.style.font_size)
elif isinstance(block, Quote):
for child in block.blocks():
collect_sizes(child, out)
elif isinstance(block, HList):
for item in block.items():
for child in item.blocks():
collect_sizes(child, out)
elif isinstance(block, Table):
for rows in (block.header_rows(), block.body_rows(), block.footer_rows()):
for row in rows:
for cell in row.cells():
for child in cell.blocks():
collect_sizes(child, out)
return out
@pytest.fixture
def blocks():
return parse_html_string(HTML)
@pytest.fixture
def layouter(blocks):
return BidirectionalLayouter(blocks, PageStyle(), (400, 600))
# ============================================================================
# Word.with_style
# ============================================================================
class TestWithStyle:
def test_word_keeps_its_text_and_takes_the_new_font(self):
word = Word("hello", Font(font_size=16))
copy = word.with_style(Font(font_size=24))
assert copy.text == "hello"
assert copy.style.font_size == 24
assert word.style.font_size == 16, "the original must not be mutated"
def test_linked_word_stays_linked(self):
word = LinkedWord("hello", Font(font_size=16), "http://example.com",
params={"a": "1"}, title="Tooltip")
copy = word.with_style(Font(font_size=24))
assert isinstance(copy, LinkedWord)
assert copy.location == "http://example.com"
assert copy.link_type == word.link_type
assert copy.params == {"a": "1"}
assert copy.link_title == "Tooltip"
assert copy.style.font_size == 24
def test_linked_word_params_are_copied_not_shared(self):
word = LinkedWord("hello", Font(), "http://example.com", params={"a": "1"})
copy = word.with_style(Font(font_size=24))
copy.params["b"] = "2"
assert "b" not in word.params
# ============================================================================
# _scale_block_fonts
# ============================================================================
class TestScaleBlockFonts:
def test_links_survive_scaling_in_every_container(self, blocks, layouter):
before = sum(len(collect_links(b)) for b in blocks)
after = sum(len(collect_links(layouter._scale_block_fonts(b, 1.5)))
for b in blocks)
assert before == 6, "fixture should contain 6 linked words"
assert after == before, "scaling must not discard hyperlinks"
def test_link_targets_are_preserved_exactly(self, blocks, layouter):
scaled = [layouter._scale_block_fonts(b, 1.5) for b in blocks]
targets = sorted(w.location for b in scaled for w in collect_links(b))
assert targets == sorted([
"http://example.com", "http://example.com",
"http://q.example", "http://l.example",
"http://h.example", "http://c.example",
])
@pytest.mark.parametrize("index,kind", [(0, "paragraph"), (1, "quote"),
(2, "list"), (3, "table")])
def test_every_container_type_actually_scales(self, blocks, layouter, index, kind):
original = collect_sizes(blocks[index])
scaled = collect_sizes(layouter._scale_block_fonts(blocks[index], 2.0))
assert original, f"fixture {kind} should contain sized words"
assert scaled == [s * 2 for s in original], f"{kind} did not scale"
def test_table_rows_stay_in_their_section(self, blocks, layouter):
table = next(b for b in blocks if isinstance(b, Table))
scaled = layouter._scale_block_fonts(table, 1.5)
assert len(list(scaled.header_rows())) == len(list(table.header_rows()))
assert len(list(scaled.body_rows())) == len(list(table.body_rows()))
def test_unscaled_blocks_are_returned_unchanged(self, blocks, layouter):
assert layouter._scale_block_fonts(blocks[0], 1.0) is blocks[0]
def test_heading_level_is_preserved(self, layouter):
heading = parse_html_string("<h3>Title here</h3>")[0]
scaled = layouter._scale_block_fonts(heading, 1.5)
assert isinstance(scaled, Heading)
assert scaled.level == heading.level
def test_result_is_memoised(self, blocks, layouter):
"""Rebuilding a block per page render allocated on the hot path."""
first = layouter._scale_block_fonts(blocks[0], 1.5)
second = layouter._scale_block_fonts(blocks[0], 1.5)
assert first is second
def test_different_scales_are_cached_separately(self, blocks, layouter):
assert (layouter._scale_block_fonts(blocks[0], 1.5)
is not layouter._scale_block_fonts(blocks[0], 2.0))
def test_originals_are_never_mutated(self, blocks, layouter):
before = [collect_sizes(b) for b in blocks]
for b in blocks:
layouter._scale_block_fonts(b, 3.0)
assert [collect_sizes(b) for b in blocks] == before
# ============================================================================
# End to end
# ============================================================================
def rendered_link_texts(page):
"""Every LinkText on a rendered page. They live inside Line objects."""
found = []
for child in page._children:
for text_obj in getattr(child, '_text_objects', []):
if isinstance(text_obj, LinkText):
found.append(text_obj)
return found
class TestLinksRemainClickableAfterFontChange:
"""
The user-visible symptom of R3: increase the font size and links stop
responding to taps.
"""
@pytest.fixture
def manager(self):
blocks = parse_html_string(
'<p>Go to <a href="http://example.com">this link</a> now.</p>')
manager = EreaderLayoutManager(blocks, page_size=(400, 600),
bookmarks_dir=tempfile.mkdtemp())
yield manager
manager.shutdown()
def test_links_render_at_default_scale(self, manager):
page = manager.get_current_page()
page.render()
assert [t.link.location for t in rendered_link_texts(page)] == \
["http://example.com", "http://example.com"]
@pytest.mark.parametrize("scale", [0.8, 1.5, 2.0])
def test_links_survive_a_font_size_change(self, manager, scale):
manager.set_font_scale(scale)
page = manager.get_current_page()
page.render()
locations = {t.link.location for t in rendered_link_texts(page)}
assert locations == {"http://example.com"}
@pytest.mark.parametrize("scale", [1.0, 1.5])
def test_the_link_is_reachable_by_tapping(self, manager, scale):
"""
Scanned rather than probed at the LinkText's own centre: the hit region
query_point reports is offset from LinkText.origin by roughly the
ascent. That misalignment predates this fix and is tracked separately
as R9 - it reproduces identically at scale 1.0.
"""
manager.set_font_scale(scale)
page = manager.get_current_page()
page.render()
targets = set()
for y in range(0, 120, 2):
for x in range(0, 400, 2):
result = page.query_point((x, y))
if result is not None and result.object_type == "link":
targets.add(result.link_target)
assert targets == {"http://example.com"}
+237
View File
@@ -0,0 +1,237 @@
"""
Tests for the page caching layer.
Covers PageBuffer's LRU behaviour and BufferedPageRenderer's cache hits, plus
regressions for S12/R1/R2: the module must not start worker processes and must
not do blocking work in a finaliser.
"""
import subprocess
import sys
import textwrap
import pytest
from pyWebLayout.layout.page_buffer import PageBuffer, BufferedPageRenderer
from pyWebLayout.layout.ereader_layout import RenderingPosition
from pyWebLayout.abstract.block import Paragraph
from pyWebLayout.abstract.inline import Word
from pyWebLayout.style import Font
from pyWebLayout.style.page_style import PageStyle
# ============================================================================
# Fixtures
# ============================================================================
@pytest.fixture
def sample_blocks():
"""A document long enough to paginate over several pages."""
font = Font()
blocks = []
for p in range(6):
para = Paragraph(style=font)
for w in range(120):
para.add_word(Word(f"p{p}w{w}", font))
blocks.append(para)
return blocks
@pytest.fixture
def renderer(sample_blocks):
return BufferedPageRenderer(sample_blocks, PageStyle(), buffer_size=3, page_size=(800, 600))
# ============================================================================
# PageBuffer
# ============================================================================
class TestPageBuffer:
def test_get_page_misses_when_empty(self):
buf = PageBuffer(buffer_size=3)
assert buf.get_page(RenderingPosition()) is None
def test_cache_page_round_trips(self):
buf = PageBuffer(buffer_size=3)
pos, nxt = RenderingPosition(block_index=0), RenderingPosition(block_index=1)
sentinel = object()
buf.cache_page(pos, sentinel, nxt)
assert buf.get_page(pos) is sentinel
assert buf.position_map[pos] == nxt
def test_lru_evicts_oldest_and_cleans_position_map(self):
buf = PageBuffer(buffer_size=2)
positions = [RenderingPosition(block_index=i) for i in range(4)]
for i, pos in enumerate(positions):
buf.cache_page(pos, object(), RenderingPosition(block_index=i + 1))
assert buf.get_page(positions[0]) is None, "oldest should have been evicted"
assert positions[0] not in buf.position_map, "position map must not leak evicted entries"
assert buf.get_page(positions[-1]) is not None
def test_get_page_refreshes_lru_order(self):
buf = PageBuffer(buffer_size=2)
a, b, c = (RenderingPosition(block_index=i) for i in range(3))
buf.cache_page(a, object())
buf.cache_page(b, object())
buf.get_page(a) # a becomes most recently used
buf.cache_page(c, object())
assert buf.get_page(a) is not None, "recently used entry should survive"
assert buf.get_page(b) is None, "least recently used entry should be evicted"
def test_backward_pages_land_in_the_backward_buffer(self):
buf = PageBuffer(buffer_size=3)
start, end = RenderingPosition(block_index=1), RenderingPosition(block_index=2)
buf.cache_page(start, object(), end, is_backward=True)
assert start in buf.backward_buffer
assert start not in buf.forward_buffer
assert buf.reverse_position_map[end] == start
def test_font_scale_change_invalidates(self):
buf = PageBuffer(buffer_size=3)
pos = RenderingPosition()
buf.cache_page(pos, object(), RenderingPosition(block_index=1))
buf.set_font_scale(1.5)
assert buf.get_page(pos) is None
def test_same_font_scale_keeps_cache(self):
buf = PageBuffer(buffer_size=3)
pos = RenderingPosition()
buf.cache_page(pos, object(), RenderingPosition(block_index=1))
buf.set_font_scale(1.0)
assert buf.get_page(pos) is not None
def test_shutdown_is_idempotent(self):
buf = PageBuffer(buffer_size=3)
buf.cache_page(RenderingPosition(), object())
buf.shutdown()
buf.shutdown()
assert buf.get_cache_stats()['forward_buffer_size'] == 0
# ============================================================================
# BufferedPageRenderer
# ============================================================================
class TestBufferedPageRenderer:
def test_render_page_returns_a_page_and_advances(self, renderer):
page, next_pos = renderer.render_page(RenderingPosition(), 1.0)
assert page is not None
assert next_pos != RenderingPosition()
def test_second_render_of_same_position_is_served_from_cache(self, renderer):
pos = RenderingPosition()
first, first_next = renderer.render_page(pos, 1.0)
second, second_next = renderer.render_page(pos, 1.0)
assert second is first, "identical page object means it came from the cache"
assert second_next == first_next
def test_font_scale_change_forces_a_re_render(self, renderer):
pos = RenderingPosition()
first, _ = renderer.render_page(pos, 1.0)
scaled, _ = renderer.render_page(pos, 1.5)
assert scaled is not first
def test_backward_render_round_trips_to_the_original_position(self, renderer):
start = RenderingPosition()
_, second_page_pos = renderer.render_page(start, 1.0)
_, back_to = renderer.render_page_backward(second_page_pos, 1.0)
assert back_to == start
def test_shutdown_clears_the_cache(self, renderer):
renderer.render_page(RenderingPosition(), 1.0)
renderer.shutdown()
assert renderer.get_cache_stats()['forward_buffer_size'] == 0
# ============================================================================
# S12 / R1 / R2 regressions
# ============================================================================
class TestNoBackgroundProcesses:
"""
The process pool that used to live here never produced a usable page (a Page
holds a live PIL canvas and cannot be pickled), and on Python 3.14's
forkserver default it raised when driven from module-level code.
"""
def test_module_declares_no_process_pool(self):
import pyWebLayout.layout.page_buffer as page_buffer
source = page_buffer.__file__
assert not hasattr(page_buffer, '_render_page_worker')
assert not hasattr(PageBuffer(), 'executor')
with open(source, encoding='utf-8') as fh:
body = fh.read().split('"""', 2)[-1] # skip the module docstring
assert 'ProcessPoolExecutor' not in body
assert 'pickle' not in body
def test_page_buffer_has_no_finaliser(self):
"""
PageBuffer.__del__ called executor.shutdown(wait=True), which deadlocked
the interpreter at exit. Cleanup must be explicit.
"""
assert '__del__' not in vars(PageBuffer)
def test_navigation_works_without_a_main_guard(self, tmp_path):
"""
R1: EreaderLayoutManager raised RuntimeError when used from module-level
script code, because submitting to a ProcessPoolExecutor under a
non-fork start method requires an `if __name__ == "__main__"` guard.
"""
script = textwrap.dedent(f"""
from pyWebLayout.io.readers.html_extraction import parse_html_string
from pyWebLayout.layout.ereader_manager import EreaderLayoutManager
blocks = parse_html_string("<p>" + " ".join(f"w{{i}}" for i in range(2000)) + "</p>")
m = EreaderLayoutManager(blocks, page_size=(800, 600),
bookmarks_dir={str(tmp_path)!r})
m.get_current_page()
m.next_page()
m.previous_page()
m.shutdown()
print("OK")
""")
result = subprocess.run([sys.executable, "-c", script],
capture_output=True, text=True, timeout=120)
assert result.returncode == 0, result.stderr
assert "OK" in result.stdout
def test_interpreter_exits_without_explicit_shutdown(self, tmp_path):
"""
R2: a manager left to be finalised at exit must not hang. The timeout is
the assertion.
"""
script = textwrap.dedent(f"""
from pyWebLayout.io.readers.html_extraction import parse_html_string
from pyWebLayout.layout.ereader_manager import EreaderLayoutManager
blocks = parse_html_string("<p>" + " ".join(f"w{{i}}" for i in range(500)) + "</p>")
m = EreaderLayoutManager(blocks, page_size=(800, 600),
bookmarks_dir={str(tmp_path)!r})
m.get_current_page()
# deliberately no shutdown() - rely on interpreter teardown
""")
result = subprocess.run([sys.executable, "-c", script],
capture_output=True, text=True, timeout=60)
assert result.returncode == 0, result.stderr