docs: rewrite ARCHITECTURE.md around layers and dependency rules

The previous version described only the abstract/concrete split and had
drifted from the code. Restructure it around the actual layering
(io → abstract → layout → concrete), state the dependency rules that
hold today, and document the layouter contract that makes pagination
resumable.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-08 12:21:02 +02:00
co-authored by Claude Opus 5
parent 737cf0771c
commit 4d596ce095
+303 -188
View File
@@ -1,233 +1,348 @@
# pyWebLayout Architecture: Abstract vs Concrete # pyWebLayout Architecture
This document explains the fundamental architectural separation between **Abstract** and **Concrete** layers in the pyWebLayout library. This document describes how pyWebLayout is organised: the layers, what lives in each,
and the rules that govern how they may depend on one another.
## Overview ## Overview
The pyWebLayout library follows a clear separation between two distinct layers: The library turns markup (HTML/EPUB) into rendered images. That pipeline is split into
layers with a strict dependency direction:
- **Abstract Layer**: Represents the logical structure and content of documents (HTML/EPUB text) ```
- **Concrete Layer**: Handles the spatial rendering and visual representation of content io/readers/ parse markup into document structure
abstract/ what the document says ─┐
↓ ├─ built on core/ + style/
layout/ decide where things go │
↓ │
concrete/ what the pixels are ─┘
PIL.Image
```
This separation provides flexibility, testability, and clean separation of concerns. The central distinction is **abstract vs concrete**:
## Abstract Layer (`pyWebLayout/abstract/`) - **Abstract** — the logical content and structure of a document. A `Paragraph` knows
it contains words in an order; it does not know how wide they are.
- **Concrete** — the spatial realisation of that content. A `Line` knows exactly which
glyphs sit at which pixel offsets on a specific canvas.
The Abstract layer deals with the **logical structure** of documents without concerning itself with how content will be visually rendered. One abstract document produces many concrete renderings: different page sizes, font
scales, and font families all re-run the concrete layer over unchanged abstract content.
That is what makes ereader features like live font scaling and reflow possible.
### Key Components ## Layers
#### `abstract/block.py` ### `core/` — shared foundations
- `Block`: Base class for all block-level content
- `Paragraph`: Represents a logical paragraph containing words
- `Heading`: Represents headings with semantic levels (H1-H6)
- `HList`: Represents ordered/unordered lists
- `Image`: Represents image references
#### `abstract/inline.py` Everything else is built on these. `core/` depends on nothing but `style/`.
- `Word`: Represents individual words with text content and styling information
- Contains methods for hyphenation and text manipulation
- Does **not** handle rendering or spatial layout
#### `abstract/document.py` **`core/base.py`** defines the contracts that make a class abstract or concrete:
- `Document`: Container for the overall document structure
- `Chapter`: Logical grouping of blocks (for books/long documents)
### Characteristics of Abstract Classes | Contract | Kind | Meaning |
|---|---|---|
| `Renderable` | ABC | Has `render()`; produces or draws visual output |
| `Queriable` | ABC | Has `in_object(point)`; can be hit-tested |
| `Layoutable` | ABC | Has `layout()`; arranges its own contents |
| `Interactable` | ABC | Holds a callback invoked on interaction |
| `Geometric` | mixin | `origin` and `size` as numpy arrays |
| `Hierarchical` | mixin | `parent` back-reference |
| `Styleable` | mixin | Carries a style object |
| `FontRegistry` | mixin | Deduplicates `Font` instances across a document |
| `MetadataContainer` | mixin | Key/value metadata with typed accessors |
| `BlockContainer` | mixin | Holds child `Block`s |
| `ContainerAware` | mixin | Knows the container it was added to |
1. **Content-focused**: Store text, structure, and semantic meaning The useful shorthand: **`Renderable` and `Geometric` are the concrete markers.** An
2. **Layout-agnostic**: No knowledge of fonts, pixels, or rendering abstract class that acquires either has crossed the line.
3. **Reusable**: Same content can be rendered in different formats/sizes
4. **Serializable**: Can be saved/loaded without rendering context
### Example: Abstract Word **`core/query.py`** — `QueryResult` and `SelectionRange`: the result types for mapping a
pixel back to content (what was clicked, what text is selected, where it is in the
document).
**`core/highlight.py`** — `Highlight`, `HighlightColor`, `HighlightManager`. Highlights
store both pixel bounds (for drawing) and semantic bounds (word indices, for surviving
a font change).
**`core/cache.py`** — `UsageCache` / `SizedUsageCache`, bounded caches keyed by
`(font, string)` for text measurement and glyph rasterisation. Eviction is by usage
count with periodic aging, not LRU; the module docstring explains why at length. This
exists because a single page issues thousands of measurements over fewer than a
thousand distinct pairs, on hardware where an unbounded cache is not affordable.
**`core/callback_registry.py`** — `CallbackRegistry`, which owns the interactive
elements registered on a page and dispatches to them.
### `style/` — semantic styling and its resolution
This package is itself an instance of the abstract/concrete split, applied to styling:
- **`abstract_style.py`** — `AbstractStyle` (frozen dataclass, hashable) captures
*intent*: `FontFamily.SERIF`, `FontSize.LARGE`, `color="black"`. `AbstractStyleRegistry`
interns them so a document holds one instance per distinct style.
- **`concrete_style.py`** — `RenderingContext` (user preferences, DPI, accessibility
flags, available space) plus `StyleResolver`, which maps `AbstractStyle` +
`RenderingContext``ConcreteStyle` (a resolved font path, a pixel size, an RGB
tuple). `ConcreteStyleRegistry` caches the results.
- **`fonts.py`** — `Font`, the loaded PIL font plus its rendering attributes, and the
bundled DejaVu families (`BundledFont`).
- **`page_style.py`** — `PageStyle`: borders, padding, background, default alignment.
- **`alignment.py`** — the `Alignment` enum.
`AbstractStyle``StyleResolver``ConcreteStyle` is the mechanism by which the same
document renders differently for different readers.
### `abstract/` — document content and structure
Layout-agnostic representations of what the document contains.
**`abstract/block.py`** — block-level content, all deriving from `Block`:
`Paragraph`, `Heading` (with `HeadingLevel`), `Quote`, `CodeBlock`, `HList` (with
`ListStyle`) and `ListItem`, `Table` / `TableRow` / `TableCell`, `Image` and
`LinkedImage`, `HorizontalRule`, `PageBreak`.
**`abstract/inline.py`** — `Word`, `FormattedSpan`, `LinkedWord`, `LineBreak`. `Word`
holds text, a style, and previous/next links into the document's word sequence. It can
report whether it is a hyphenation candidate (`possible_hyphenation(language)`), but it
does not perform the split — that is a measurement decision and belongs to `Line`.
**`abstract/document.py`** — `Document`, `Chapter`, `Book`, `MetadataType`. `Book` is
the EPUB-shaped `Document` with chapters and a table of contents.
**`abstract/functional.py`** — `Link`, `Button`, `Form`, `FormField`, `LinkType`,
`FormFieldType`.
**`abstract/interactive_image.py`** — `InteractiveImage`, an `Image` that is also
`Interactable` and `Queriable`.
### `concrete/` — spatial realisation
Objects that know their position and size and can draw themselves onto a canvas.
**`concrete/text.py`** — the heart of text layout:
- `Text` — one renderable fragment (a whole word, or a hyphenated part). Requires an
`ImageDraw.Draw` at construction so it can measure itself immediately.
- `Line` — a sequence of `Text` objects with resolved spacing and a baseline.
`Line.add_word()` is where a `Word` becomes one or two `Text` objects: it measures,
and if the word overflows it tries dictionary hyphenation, then brute-force splitting,
before rejecting the word.
- `AlignmentHandler` and its subclasses (`LeftAlignmentHandler`,
`CenterRightAlignmentHandler`, `JustifyAlignmentHandler`) — the strategy objects that
turn a set of measured fragments plus an available width into concrete spacing and a
start position.
- Cache management: `configure_text_caches`, `clear_text_caches`, `text_cache_stats`,
`prewarm_text_caches`.
**`concrete/page.py`** — `Page`: a fixed-size canvas holding `Renderable` children, with
a content rectangle derived from its `PageStyle`, `can_fit_line()` for the layouter to
test against, `render()` returning a `PIL.Image`, and `query_point()` / `query_range()`
for hit-testing.
**`concrete/box.py`** — `Box`, the `Geometric` + `Renderable` + `Queriable` base for
positioned drawable objects.
**`concrete/dynamic_page.py`** — `DynamicPage` (a `Page` subclass) and `SizeConstraints`.
Adds a two-phase measure-then-layout protocol, so a container such as a table can learn
its content's intrinsic size before committing to a size allocation.
**`concrete/image.py`** — `RenderableImage`.
**`concrete/table.py`** — `TableRenderer`, `TableRowRenderer`, `TableCellRenderer`,
`TableStyle`. Cells host their own nested `Page`, which is why `Page` accepts a
non-zero `origin`.
**`concrete/functional.py`** — `LinkText`, `ButtonText`, `FormFieldText`: `Text`
subclasses that are also `Interactable`.
**`concrete/interaction_handler.py`** — `InteractionHandler`, `InteractionStateManager`:
routing taps and presses to registered elements and tracking pressed/released state.
### `layout/` — the abstract → concrete transformation
This is the package the pipeline diagram calls the layout engine.
**`layout/document_layouter.py`** — a set of layouter functions, one per content kind,
all sharing a signature shape of *(abstract element, target `Page`) → did it fit*:
```python ```python
# An Abstract Word knows its text content and semantic properties paragraph_layouter(paragraph, page, start_word=0, pretext=None, alignment_override=None)
word = Word("supercalifragilisticexpialidocious", font_style) -> (complete: bool, failed_word_index: int | None, remaining_pretext: Text | None)
word.hyphenate() # Logical operation - finds break points
parts = word.get_hyphenated_parts() # Returns ["super-", "cali-", "fragi-", ...] image_layouter(image, page, max_width=None, max_height=None) -> bool
table_layouter(table, page, style=None) -> bool
pagebreak_layouter(page_break, page) -> bool
button_layouter(button, page, font=None, padding=...) -> (bool, str)
form_field_layouter(field, page, font=None, ...) -> ...
form_layouter(form, page, font=None, field_spacing=10) -> (bool, list[str])
``` ```
## Concrete Layer (`pyWebLayout/concrete/`) They **append to a page**, they do not return a list of lines. The three-part return of
`paragraph_layouter` is what makes pagination resumable: when a paragraph runs off the
bottom of a page, the caller learns which word failed and whether a hyphenated fragment
is pending, and can continue on the next page from exactly there.
The Concrete layer handles the **spatial representation** and actual rendering of content. `DocumentLayouter` wraps a `Page` and dispatches over a list of abstract elements by
type, holding the `ConcreteStyleRegistry` for the run.
### Key Components **`layout/ereader_layout.py`** — the paginated reading model:
- `RenderingPosition` — a serialisable cursor expressed in *abstract* coordinates
(chapter, block, word, table cell, list item, pending pretext). Because it names
document structure rather than pixels, it survives font-size and page-size changes.
- `BidirectionalLayouter` — renders a page forward or backward from a position,
returning `(Page, next_position)`.
- `ChapterNavigator` / `ChapterInfo` — a table of contents built from heading structure.
- `FontScaler`, `FontFamilyOverride` — apply scale and family changes to blocks at
layout time without mutating the abstract document.
#### `concrete/text.py` **`layout/page_buffer.py`** — `PageBuffer` (LRU caches of rendered pages in both
- `Text`: Renders a specific text fragment with precise positioning directions, plus position maps) and `BufferedPageRenderer` (background rendering).
- `Line`: Manages a line of `Text` objects with spacing and alignment
- Handles actual pixel measurements, font rendering, and positioning
#### `concrete/page.py` **`layout/ereader_manager.py`** — `EreaderLayoutManager`, the top-level application
- `Page`: Top-level container for rendered content interface (page turns, font changes, chapter jumps, progress), and `BookmarkManager`
- `Container`: Layout manager for organizing renderable objects for persisting bookmarks and the last reading position.
- Handles spatial layout, pagination, and visual composition
#### `concrete/box.py` **`layout/table_optimizer.py`** — column width allocation for tables.
- `Box`: Base class for all spatially-aware renderable objects
- Provides positioning, sizing, and rendering capabilities
### Characteristics of Concrete Classes ### `io/readers/` — parsing
1. **Rendering-focused**: Handle pixels, fonts, images, and visual output **`html_extraction.py`** — `parse_html_string(html, base_font=None, document=None,
2. **Spatially-aware**: Know exact positions, sizes, and layout constraints base_path=None) -> List[Block]`. Built from a `StyleContext` (a `NamedTuple` threaded
3. **Implementation-specific**: Tied to specific rendering technologies (PIL, etc.) down the tree, carrying the inherited font and styling) and a table of per-tag handlers
4. **Non-portable**: Rendering results are tied to specific display contexts (`paragraph_handler`, `heading_handler`, `table_handler`, …). This is the only place
that knows about HTML.
### Example: Concrete Text **`epub_reader.py`** — `EPUBReader` and `read_epub(path) -> Book`: container/OPF
parsing, spine and manifest, table of contents, cover handling, and image processing
(with an e-ink processor available by default).
## Dependency rules
The direction of dependency is what keeps the split honest:
```
io/readers → abstract → core, style
layout → abstract, concrete, core, style
concrete → abstract, core, style
abstract → core, style ← must NOT import concrete
```
`abstract/` importing from `concrete/` is the violation to watch for. `concrete/`
importing from `abstract/` is expected and correct: a `Text` may point back at the
`Word` it came from, and a table cell renderer reads its abstract `TableCell`.
**Where the abstract layer touches rendering today, and why:**
- Abstract classes are constructed with `Font` objects, which carry a pixel size and a
loaded font file. This is a deliberate compromise for parsing performance (HTML
styling resolves to a `Font` once, at parse time) but it does mean the abstract layer
is not fully rendering-independent. `AbstractStyle` is the intended replacement, and
`Word` already accepts either.
- `Word.concrete` / `Word.add_concete()` ([inline.py:42](pyWebLayout/abstract/inline.py#L42),
[:134](pyWebLayout/abstract/inline.py#L134)) is a back-reference to the `Text` objects
a word became. **It is currently written and never read**, and it cannot correctly
model the relationship anyway: one `Word` becomes many `Text`s across re-layouts, and
a single slot only remembers the most recent. Treat it as vestigial. When a
word→pixels mapping is genuinely needed, build it on `Text`'s `source` back-reference
(concrete pointing at abstract), which is the safe direction — note it is currently
stored as `_source` with no public accessor, and is itself unread today.
## Worked example
```python ```python
# A Concrete Text object handles actual rendering from pyWebLayout.io.readers.html_extraction import parse_html_string
text = Text("super-", font) # Specific text fragment from pyWebLayout.concrete.page import Page
text._calculate_dimensions() # Computes exact pixel size from pyWebLayout.layout.document_layouter import DocumentLayouter
image = text.render() # Produces actual visual output
# 1. Parse: markup -> abstract blocks
blocks = parse_html_string("<h1>Chapter One</h1><p>It was a dark and stormy night.</p>")
# [Heading, Paragraph]
# 2. Lay out: abstract blocks -> concrete children appended to a Page
page = Page(size=(400, 600))
DocumentLayouter(page).layout_document(blocks)
# 3. Render: Page -> PIL.Image
image = page.render()
# 4. Query: pixel -> content
result = page.query_point((50, 40))
print(result.object_type, result.text) # e.g. "text" "Chapter"
``` ```
## The Transformation Process Inspecting the intermediate concrete objects:
The architecture involves a clear transformation from Abstract to Concrete:
```
Abstract Document
[Parser Layer]
Abstract Blocks (Paragraph, Heading, etc.)
[Layout Engine]
Concrete Objects (Text, Line, Page)
[Rendering Engine]
Visual Output (Images, PDF, etc.)
```
### Example Transformation
```python ```python
# 1. Abstract content from pyWebLayout.concrete.text import Line
paragraph = Paragraph()
paragraph.add_word(Word("This", font))
paragraph.add_word(Word("is", font))
paragraph.add_word(Word("a", font))
paragraph.add_word(Word("test", font))
# 2. Layout transformation for line in (c for c in page.children if isinstance(c, Line)):
layout = ParagraphLayout(line_width=200, line_height=20) print([t.text for t in line.text_objects])
lines = layout.layout_paragraph(paragraph) # Returns List[Line] # ['Chapter', 'One']
# ['It', 'was', 'a', 'dark', 'and', 'stormy', 'night.']
# 3. Each Line contains concrete Text objects
for line in lines:
for text_obj in line.text_objects: # List[Text]
print(f"Text: '{text_obj.text}' at position {text_obj._origin}")
``` ```
## Key Architectural Principles For paginated reading, drive `EreaderLayoutManager` instead of building pages by hand:
### 1. **Single Responsibility**
- Abstract classes: Handle content and structure
- Concrete classes: Handle rendering and layout
### 2. **Separation of Concerns**
- Text parsing/processing ≠ Text rendering
- Document structure ≠ Page layout
- Content semantics ≠ Visual presentation
### 3. **Immutable Abstract Content**
- Abstract content remains unchanged during rendering
- Multiple concrete representations can be generated from same abstract content
- Enables pagination, different formats, responsive layouts
### 4. **One-to-Many Relationships**
- One Abstract Word → Multiple Concrete Text objects (hyphenation)
- One Abstract Paragraph → Multiple Concrete Lines
- One Abstract Document → Multiple Concrete Pages
## Common Anti-Patterns to Avoid
### ❌ **Mixing Concerns**
```python ```python
# WRONG: Abstract class knowing about pixels from pyWebLayout.layout.ereader_manager import EreaderLayoutManager
manager = EreaderLayoutManager(blocks, page_size=(800, 600), document_id="my-book")
page = manager.get_current_page()
page = manager.next_page()
manager.set_font_scale(1.25) # re-lays out from the same RenderingPosition
```
## Design principles
**1. Abstract content is not mutated by layout.** Font scaling and family overrides
produce scaled copies at layout time rather than editing the document. This is what
allows the same `blocks` list to back a buffer of pages at several font sizes.
**2. Positions are expressed in abstract coordinates.** `RenderingPosition` names a
chapter, block, and word — never a pixel. A reader who changes font size stays on the
same sentence.
**3. Layouters report partial success.** Every layouter returns whether it fit, and the
text layouter also returns where it stopped. Pagination is built from this rather than
from a separate page-breaking pass.
**4. One abstract object may become many concrete ones.** One `Word` → one or two `Text`
fragments; one `Paragraph` → many `Line`s across many `Page`s; one `Document` → an
unbounded sequence of `Page`s. Any API that assumes a one-to-one mapping will be wrong
at a hyphen or a page boundary.
**5. Measurement is cached, not avoided.** Text width and glyph rasterisation are the
hot path. `core/cache.py` bounds their cost; `prewarm_text_caches` front-loads it for a
known document.
## Anti-patterns
**Concrete state stored on abstract objects.** An abstract object that remembers its
rendered width, position, or `Text` objects is wrong at the second rendering. If a
back-reference is needed, point from concrete to abstract.
```python
# WRONG
class Word: class Word:
def __init__(self, text): def __init__(self, text):
self.text = text self.rendered_width = None # invalidated by any font change
self.rendered_width = None # ❌ Concrete concern in abstract class
# RIGHT
text = Text(word.text, font, draw, source=word) # concrete knows its origin
``` ```
### ❌ **renderable_words Concept** **Treating `Word` as renderable.** Words are abstract; only `Text` draws. There is no
```python `renderable_words` anywhere in the codebase, and there should not be.
# WRONG: Confusing abstract and concrete
line.renderable_words # ❌ This suggests Words are renderable
# Words are abstract - only Text objects render
```
### ✅ **Correct Separation** **Assuming a layouter returns lines.** `paragraph_layouter` appends to a page and
```python reports what did not fit. Code that expects `List[Line]` back is working from an
# CORRECT: Clear separation outdated model.
abstract_word = Word("test") # Abstract content
concrete_text = Text("test", font) # Concrete rendering
line.text_objects.append(concrete_text) # Concrete objects in concrete container
```
## Benefits of This Architecture ## Summary
### 1. **Flexibility** - **`core/`** — contracts and shared machinery
- Same content can be rendered at different sizes - **`style/`** — semantic style, and its resolution to concrete rendering parameters
- Multiple output formats from single source - **`abstract/`** — what the document says
- Easy to implement responsive design - **`concrete/`** — where the pixels go
- **`layout/`** — the transformation between them, and pagination on top of it
### 2. **Testability** - **`io/readers/`** — markup in
- Abstract logic can be tested without rendering
- Layout algorithms can be tested independently
- Visual rendering can be mocked
### 3. **Performance**
- Abstract content can be cached and reused
- Layout can be computed once for multiple renderings
- Incremental updates possible
### 4. **Maintainability**
- Clear boundaries between text processing and rendering
- Changes to rendering don't affect content parsing
- Easy to swap rendering backends
## File Organization
```
pyWebLayout/
├── abstract/ # Content and structure
│ ├── block.py # Document blocks (Paragraph, Heading, etc.)
│ ├── inline.py # Inline content (Word, etc.)
│ ├── document.py # Document structure
│ └── functional.py # Links, buttons, etc.
├── concrete/ # Rendering and layout
│ ├── text.py # Text and Line rendering
│ ├── page.py # Page layout and containers
│ ├── box.py # Base rendering classes
│ ├── image.py # Image rendering
│ └── functional.py # Interactive elements
├── typesetting/ # Layout algorithms
│ ├── paragraph_layout.py # Abstract → Concrete transformation
│ ├── flow.py # Text flow management
│ └── pagination.py # Page breaking logic
└── style/ # Styling and formatting
├── fonts.py # Font management
├── layout.py # Layout constants
└── alignment.py # Alignment enums
```
## Conclusion
The Abstract/Concrete separation is fundamental to pyWebLayout's design. It ensures clean separation between content processing and visual rendering, enabling flexible, maintainable, and testable document processing pipelines.
**Remember**:
- **Abstract** = What to display (content, structure, semantics)
- **Concrete** = How to display it (pixels, fonts, positioning, rendering)
This architecture enables the library to handle complex document layouts while maintaining clear, understandable code organization.