pyWebLayout/ARCHITECTURE.md

pyWebLayout Architecture

This document describes how pyWebLayout is organised: the layers, what lives in each, and the rules that govern how they may depend on one another.

Overview

The library turns markup (HTML/EPUB) into rendered images. That pipeline is split into layers with a strict dependency direction:

    io/readers/          parse markup into document structure
         ↓
    abstract/            what the document says     ─┐
         ↓                                           ├─ built on core/ + style/
    layout/              decide where things go      │
         ↓                                           │
    concrete/            what the pixels are        ─┘
         ↓
    PIL.Image

The central distinction is abstract vs concrete:

• Abstract — the logical content and structure of a document. A Paragraph knows it contains words in an order; it does not know how wide they are.
• Concrete — the spatial realisation of that content. A Line knows exactly which glyphs sit at which pixel offsets on a specific canvas.

One abstract document produces many concrete renderings: different page sizes, font scales, and font families all re-run the concrete layer over unchanged abstract content. That is what makes ereader features like live font scaling and reflow possible.

Layers

core/ — shared foundations

Everything else is built on these. core/ depends on nothing but style/.

core/base.py defines the contracts that make a class abstract or concrete:

Contract	Kind	Meaning
Renderable	ABC	Has render(); produces or draws visual output
Queriable	ABC	Has in_object(point); can be hit-tested
Layoutable	ABC	Has layout(); arranges its own contents
Interactable	ABC	Holds a callback invoked on interaction
Geometric	mixin	origin and size as numpy arrays
Hierarchical	mixin	parent back-reference
Styleable	mixin	Carries a style object
FontRegistry	mixin	Deduplicates Font instances across a document
MetadataContainer	mixin	Key/value metadata with typed accessors
BlockContainer	mixin	Holds child Blocks
ContainerAware	mixin	Knows the container it was added to

The useful shorthand: Renderable and Geometric are the concrete markers. An abstract class that acquires either has crossed the line.

core/query.py — QueryResult and SelectionRange: the result types for mapping a pixel back to content (what was clicked, what text is selected, where it is in the document).

core/highlight.py — Highlight, HighlightColor, HighlightManager. Highlights store both pixel bounds (for drawing) and semantic bounds (word indices, for surviving a font change).

core/cache.py — UsageCache / SizedUsageCache, bounded caches keyed by (font, string) for text measurement and glyph rasterisation. Eviction is by usage count with periodic aging, not LRU; the module docstring explains why at length. This exists because a single page issues thousands of measurements over fewer than a thousand distinct pairs, on hardware where an unbounded cache is not affordable.

core/callback_registry.py — CallbackRegistry, which owns the interactive elements registered on a page and dispatches to them.

style/ — semantic styling and its resolution

This package is itself an instance of the abstract/concrete split, applied to styling:

• abstract_style.py — AbstractStyle (frozen dataclass, hashable) captures intent: FontFamily.SERIF, FontSize.LARGE, color="black". AbstractStyleRegistry interns them so a document holds one instance per distinct style.
• concrete_style.py — RenderingContext (user preferences, DPI, accessibility flags, available space) plus StyleResolver, which maps AbstractStyle + RenderingContext → ConcreteStyle (a resolved font path, a pixel size, an RGB tuple). ConcreteStyleRegistry caches the results.
• fonts.py — Font, the loaded PIL font plus its rendering attributes, and the bundled DejaVu families (BundledFont).
• page_style.py — PageStyle: borders, padding, background, default alignment.
• alignment.py — the Alignment enum.

AbstractStyle → StyleResolver → ConcreteStyle is the mechanism by which the same document renders differently for different readers.

abstract/ — document content and structure

Layout-agnostic representations of what the document contains.

abstract/block.py — block-level content, all deriving from Block:

Paragraph, Heading (with HeadingLevel), Quote, CodeBlock, HList (with ListStyle) and ListItem, Table / TableRow / TableCell, Image and LinkedImage, HorizontalRule, PageBreak.

abstract/inline.py — Word, FormattedSpan, LinkedWord, LineBreak. Word holds text, a style, and previous/next links into the document's word sequence. It can report whether it is a hyphenation candidate (possible_hyphenation(language)), but it does not perform the split — that is a measurement decision and belongs to Line.

abstract/document.py — Document, Chapter, Book, MetadataType. Book is the EPUB-shaped Document with chapters and a table of contents.

abstract/functional.py — Link, Button, Form, FormField, LinkType, FormFieldType.

abstract/interactive_image.py — InteractiveImage, an Image that is also Interactable and Queriable.

concrete/ — spatial realisation

Objects that know their position and size and can draw themselves onto a canvas.

concrete/text.py — the heart of text layout:

• Text — one renderable fragment (a whole word, or a hyphenated part). Requires an ImageDraw.Draw at construction so it can measure itself immediately.
• Line — a sequence of Text objects with resolved spacing and a baseline. Line.add_word() is where a Word becomes one or two Text objects: it measures, and if the word overflows it tries dictionary hyphenation, then brute-force splitting, before rejecting the word.
• AlignmentHandler and its subclasses (LeftAlignmentHandler, CenterRightAlignmentHandler, JustifyAlignmentHandler) — the strategy objects that turn a set of measured fragments plus an available width into concrete spacing and a start position.
• Cache management: configure_text_caches, clear_text_caches, text_cache_stats, prewarm_text_caches.

concrete/page.py — Page: a fixed-size canvas holding Renderable children, with a content rectangle derived from its PageStyle, can_fit_line() for the layouter to test against, render() returning a PIL.Image, and query_point() / query_range() for hit-testing.

concrete/box.py — Box, the Geometric + Renderable + Queriable base for positioned drawable objects.

concrete/dynamic_page.py — DynamicPage (a Page subclass) and SizeConstraints. Adds a two-phase measure-then-layout protocol, so a container such as a table can learn its content's intrinsic size before committing to a size allocation.

concrete/image.py — RenderableImage.

concrete/table.py — TableRenderer, TableRowRenderer, TableCellRenderer, TableStyle. Cells host their own nested Page, which is why Page accepts a non-zero origin.

concrete/functional.py — LinkText, ButtonText, FormFieldText: Text subclasses that are also Interactable.

concrete/interaction_handler.py — InteractionHandler, InteractionStateManager: routing taps and presses to registered elements and tracking pressed/released state.

layout/ — the abstract → concrete transformation

This is the package the pipeline diagram calls the layout engine.

layout/document_layouter.py — a set of layouter functions, one per content kind, all sharing a signature shape of (abstract element, target Page) → did it fit:

paragraph_layouter(paragraph, page, start_word=0, pretext=None, alignment_override=None)
    -> (complete: bool, failed_word_index: int | None, remaining_pretext: Text | None)

image_layouter(image, page, max_width=None, max_height=None)   -> bool
table_layouter(table, page, style=None)                        -> bool
pagebreak_layouter(page_break, page)                           -> bool
button_layouter(button, page, font=None, padding=...)          -> (bool, str)
form_field_layouter(field, page, font=None, ...)               -> ...
form_layouter(form, page, font=None, field_spacing=10)         -> (bool, list[str])

They append to a page, they do not return a list of lines. The three-part return of paragraph_layouter is what makes pagination resumable: when a paragraph runs off the bottom of a page, the caller learns which word failed and whether a hyphenated fragment is pending, and can continue on the next page from exactly there.

DocumentLayouter wraps a Page and dispatches over a list of abstract elements by type, holding the ConcreteStyleRegistry for the run.

layout/ereader_layout.py — the paginated reading model:

• RenderingPosition — a serialisable cursor expressed in abstract coordinates (chapter, block, word, table cell, list item, pending pretext). Because it names document structure rather than pixels, it survives font-size and page-size changes.
• BidirectionalLayouter — renders a page forward or backward from a position, returning (Page, next_position).
• ChapterNavigator / ChapterInfo — a table of contents built from heading structure.
• FontScaler, FontFamilyOverride — apply scale and family changes to blocks at layout time without mutating the abstract document.

layout/page_buffer.py — PageBuffer (LRU caches of rendered pages in both directions, plus position maps) and BufferedPageRenderer (background rendering).

layout/ereader_manager.py — EreaderLayoutManager, the top-level application interface (page turns, font changes, chapter jumps, progress), and BookmarkManager for persisting bookmarks and the last reading position.

layout/table_optimizer.py — column width allocation for tables.

io/readers/ — parsing

html_extraction.py — parse_html_string(html, base_font=None, document=None, base_path=None) -> List[Block]. Built from a StyleContext (a NamedTuple threaded down the tree, carrying the inherited font and styling) and a table of per-tag handlers (paragraph_handler, heading_handler, table_handler, …). This is the only place that knows about HTML.

epub_reader.py — EPUBReader and read_epub(path) -> Book: container/OPF parsing, spine and manifest, table of contents, cover handling, and image processing (with an e-ink processor available by default).

Dependency rules

The direction of dependency is what keeps the split honest:

io/readers  →  abstract  →  core, style
layout      →  abstract, concrete, core, style
concrete    →  abstract, core, style
abstract    →  core, style          ← must NOT import concrete

abstract/ importing from concrete/ is the violation to watch for. concrete/ importing from abstract/ is expected and correct: a Text may point back at the Word it came from, and a table cell renderer reads its abstract TableCell.

Where the abstract layer touches rendering today, and why:

• Abstract classes are constructed with Font objects, which carry a pixel size and a loaded font file. This is a deliberate compromise for parsing performance (HTML styling resolves to a Font once, at parse time) but it does mean the abstract layer is not fully rendering-independent. AbstractStyle is the intended replacement, and Word already accepts either.
• Word.concrete / Word.add_concete() (inline.py:42, :134) is a back-reference to the Text objects a word became. It is currently written and never read, and it cannot correctly model the relationship anyway: one Word becomes many Texts across re-layouts, and a single slot only remembers the most recent. Treat it as vestigial. When a word→pixels mapping is genuinely needed, build it on Text's source back-reference (concrete pointing at abstract), which is the safe direction — note it is currently stored as _source with no public accessor, and is itself unread today.

Worked example

from pyWebLayout.io.readers.html_extraction import parse_html_string
from pyWebLayout.concrete.page import Page
from pyWebLayout.layout.document_layouter import DocumentLayouter

# 1. Parse: markup -> abstract blocks
blocks = parse_html_string("<h1>Chapter One</h1><p>It was a dark and stormy night.</p>")
# [Heading, Paragraph]

# 2. Lay out: abstract blocks -> concrete children appended to a Page
page = Page(size=(400, 600))
DocumentLayouter(page).layout_document(blocks)

# 3. Render: Page -> PIL.Image
image = page.render()

# 4. Query: pixel -> content
result = page.query_point((50, 40))
print(result.object_type, result.text)   # e.g. "text" "Chapter"

Inspecting the intermediate concrete objects:

from pyWebLayout.concrete.text import Line

for line in (c for c in page.children if isinstance(c, Line)):
    print([t.text for t in line.text_objects])
# ['Chapter', 'One']
# ['It', 'was', 'a', 'dark', 'and', 'stormy', 'night.']

For paginated reading, drive EreaderLayoutManager instead of building pages by hand:

from pyWebLayout.layout.ereader_manager import EreaderLayoutManager

manager = EreaderLayoutManager(blocks, page_size=(800, 600), document_id="my-book")
page = manager.get_current_page()
page = manager.next_page()
manager.set_font_scale(1.25)   # re-lays out from the same RenderingPosition

Design principles

1. Abstract content is not mutated by layout. Font scaling and family overrides produce scaled copies at layout time rather than editing the document. This is what allows the same blocks list to back a buffer of pages at several font sizes.

2. Positions are expressed in abstract coordinates. RenderingPosition names a chapter, block, and word — never a pixel. A reader who changes font size stays on the same sentence.

3. Layouters report partial success. Every layouter returns whether it fit, and the text layouter also returns where it stopped. Pagination is built from this rather than from a separate page-breaking pass.

4. One abstract object may become many concrete ones. One Word → one or two Text fragments; one Paragraph → many Lines across many Pages; one Document → an unbounded sequence of Pages. Any API that assumes a one-to-one mapping will be wrong at a hyphen or a page boundary.

5. Measurement is cached, not avoided. Text width and glyph rasterisation are the hot path. core/cache.py bounds their cost; prewarm_text_caches front-loads it for a known document.

Anti-patterns

Concrete state stored on abstract objects. An abstract object that remembers its rendered width, position, or Text objects is wrong at the second rendering. If a back-reference is needed, point from concrete to abstract.

# WRONG
class Word:
    def __init__(self, text):
        self.rendered_width = None    # invalidated by any font change

# RIGHT
text = Text(word.text, font, draw, source=word)   # concrete knows its origin

Treating Word as renderable. Words are abstract; only Text draws. There is no renderable_words anywhere in the codebase, and there should not be.

Assuming a layouter returns lines. paragraph_layouter appends to a page and reports what did not fit. Code that expects List[Line] back is working from an outdated model.

Summary

• core/ — contracts and shared machinery
• style/ — semantic style, and its resolution to concrete rendering parameters
• abstract/ — what the document says
• concrete/ — where the pixels go
• layout/ — the transformation between them, and pagination on top of it
• io/readers/ — markup in