diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 09f89f0..d8f5dba 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -1,233 +1,348 @@ -# pyWebLayout Architecture: Abstract vs Concrete +# pyWebLayout Architecture -This document explains the fundamental architectural separation between **Abstract** and **Concrete** layers in the pyWebLayout library. +This document describes how pyWebLayout is organised: the layers, what lives in each, +and the rules that govern how they may depend on one another. ## Overview -The pyWebLayout library follows a clear separation between two distinct layers: +The library turns markup (HTML/EPUB) into rendered images. That pipeline is split into +layers with a strict dependency direction: -- **Abstract Layer**: Represents the logical structure and content of documents (HTML/EPUB text) -- **Concrete Layer**: Handles the spatial rendering and visual representation of content +``` + io/readers/ parse markup into document structure + ↓ + abstract/ what the document says ─┐ + ↓ ├─ built on core/ + style/ + layout/ decide where things go │ + ↓ │ + concrete/ what the pixels are ─┘ + ↓ + PIL.Image +``` -This separation provides flexibility, testability, and clean separation of concerns. +The central distinction is **abstract vs concrete**: -## Abstract Layer (`pyWebLayout/abstract/`) +- **Abstract** — the logical content and structure of a document. A `Paragraph` knows + it contains words in an order; it does not know how wide they are. +- **Concrete** — the spatial realisation of that content. A `Line` knows exactly which + glyphs sit at which pixel offsets on a specific canvas. -The Abstract layer deals with the **logical structure** of documents without concerning itself with how content will be visually rendered. +One abstract document produces many concrete renderings: different page sizes, font +scales, and font families all re-run the concrete layer over unchanged abstract content. +That is what makes ereader features like live font scaling and reflow possible. -### Key Components +## Layers -#### `abstract/block.py` -- `Block`: Base class for all block-level content -- `Paragraph`: Represents a logical paragraph containing words -- `Heading`: Represents headings with semantic levels (H1-H6) -- `HList`: Represents ordered/unordered lists -- `Image`: Represents image references +### `core/` — shared foundations -#### `abstract/inline.py` -- `Word`: Represents individual words with text content and styling information -- Contains methods for hyphenation and text manipulation -- Does **not** handle rendering or spatial layout +Everything else is built on these. `core/` depends on nothing but `style/`. -#### `abstract/document.py` -- `Document`: Container for the overall document structure -- `Chapter`: Logical grouping of blocks (for books/long documents) +**`core/base.py`** defines the contracts that make a class abstract or concrete: -### Characteristics of Abstract Classes +| Contract | Kind | Meaning | +|---|---|---| +| `Renderable` | ABC | Has `render()`; produces or draws visual output | +| `Queriable` | ABC | Has `in_object(point)`; can be hit-tested | +| `Layoutable` | ABC | Has `layout()`; arranges its own contents | +| `Interactable` | ABC | Holds a callback invoked on interaction | +| `Geometric` | mixin | `origin` and `size` as numpy arrays | +| `Hierarchical` | mixin | `parent` back-reference | +| `Styleable` | mixin | Carries a style object | +| `FontRegistry` | mixin | Deduplicates `Font` instances across a document | +| `MetadataContainer` | mixin | Key/value metadata with typed accessors | +| `BlockContainer` | mixin | Holds child `Block`s | +| `ContainerAware` | mixin | Knows the container it was added to | -1. **Content-focused**: Store text, structure, and semantic meaning -2. **Layout-agnostic**: No knowledge of fonts, pixels, or rendering -3. **Reusable**: Same content can be rendered in different formats/sizes -4. **Serializable**: Can be saved/loaded without rendering context +The useful shorthand: **`Renderable` and `Geometric` are the concrete markers.** An +abstract class that acquires either has crossed the line. -### Example: Abstract Word +**`core/query.py`** — `QueryResult` and `SelectionRange`: the result types for mapping a +pixel back to content (what was clicked, what text is selected, where it is in the +document). + +**`core/highlight.py`** — `Highlight`, `HighlightColor`, `HighlightManager`. Highlights +store both pixel bounds (for drawing) and semantic bounds (word indices, for surviving +a font change). + +**`core/cache.py`** — `UsageCache` / `SizedUsageCache`, bounded caches keyed by +`(font, string)` for text measurement and glyph rasterisation. Eviction is by usage +count with periodic aging, not LRU; the module docstring explains why at length. This +exists because a single page issues thousands of measurements over fewer than a +thousand distinct pairs, on hardware where an unbounded cache is not affordable. + +**`core/callback_registry.py`** — `CallbackRegistry`, which owns the interactive +elements registered on a page and dispatches to them. + +### `style/` — semantic styling and its resolution + +This package is itself an instance of the abstract/concrete split, applied to styling: + +- **`abstract_style.py`** — `AbstractStyle` (frozen dataclass, hashable) captures + *intent*: `FontFamily.SERIF`, `FontSize.LARGE`, `color="black"`. `AbstractStyleRegistry` + interns them so a document holds one instance per distinct style. +- **`concrete_style.py`** — `RenderingContext` (user preferences, DPI, accessibility + flags, available space) plus `StyleResolver`, which maps `AbstractStyle` + + `RenderingContext` → `ConcreteStyle` (a resolved font path, a pixel size, an RGB + tuple). `ConcreteStyleRegistry` caches the results. +- **`fonts.py`** — `Font`, the loaded PIL font plus its rendering attributes, and the + bundled DejaVu families (`BundledFont`). +- **`page_style.py`** — `PageStyle`: borders, padding, background, default alignment. +- **`alignment.py`** — the `Alignment` enum. + +`AbstractStyle` → `StyleResolver` → `ConcreteStyle` is the mechanism by which the same +document renders differently for different readers. + +### `abstract/` — document content and structure + +Layout-agnostic representations of what the document contains. + +**`abstract/block.py`** — block-level content, all deriving from `Block`: + +`Paragraph`, `Heading` (with `HeadingLevel`), `Quote`, `CodeBlock`, `HList` (with +`ListStyle`) and `ListItem`, `Table` / `TableRow` / `TableCell`, `Image` and +`LinkedImage`, `HorizontalRule`, `PageBreak`. + +**`abstract/inline.py`** — `Word`, `FormattedSpan`, `LinkedWord`, `LineBreak`. `Word` +holds text, a style, and previous/next links into the document's word sequence. It can +report whether it is a hyphenation candidate (`possible_hyphenation(language)`), but it +does not perform the split — that is a measurement decision and belongs to `Line`. + +**`abstract/document.py`** — `Document`, `Chapter`, `Book`, `MetadataType`. `Book` is +the EPUB-shaped `Document` with chapters and a table of contents. + +**`abstract/functional.py`** — `Link`, `Button`, `Form`, `FormField`, `LinkType`, +`FormFieldType`. + +**`abstract/interactive_image.py`** — `InteractiveImage`, an `Image` that is also +`Interactable` and `Queriable`. + +### `concrete/` — spatial realisation + +Objects that know their position and size and can draw themselves onto a canvas. + +**`concrete/text.py`** — the heart of text layout: +- `Text` — one renderable fragment (a whole word, or a hyphenated part). Requires an + `ImageDraw.Draw` at construction so it can measure itself immediately. +- `Line` — a sequence of `Text` objects with resolved spacing and a baseline. + `Line.add_word()` is where a `Word` becomes one or two `Text` objects: it measures, + and if the word overflows it tries dictionary hyphenation, then brute-force splitting, + before rejecting the word. +- `AlignmentHandler` and its subclasses (`LeftAlignmentHandler`, + `CenterRightAlignmentHandler`, `JustifyAlignmentHandler`) — the strategy objects that + turn a set of measured fragments plus an available width into concrete spacing and a + start position. +- Cache management: `configure_text_caches`, `clear_text_caches`, `text_cache_stats`, + `prewarm_text_caches`. + +**`concrete/page.py`** — `Page`: a fixed-size canvas holding `Renderable` children, with +a content rectangle derived from its `PageStyle`, `can_fit_line()` for the layouter to +test against, `render()` returning a `PIL.Image`, and `query_point()` / `query_range()` +for hit-testing. + +**`concrete/box.py`** — `Box`, the `Geometric` + `Renderable` + `Queriable` base for +positioned drawable objects. + +**`concrete/dynamic_page.py`** — `DynamicPage` (a `Page` subclass) and `SizeConstraints`. +Adds a two-phase measure-then-layout protocol, so a container such as a table can learn +its content's intrinsic size before committing to a size allocation. + +**`concrete/image.py`** — `RenderableImage`. + +**`concrete/table.py`** — `TableRenderer`, `TableRowRenderer`, `TableCellRenderer`, +`TableStyle`. Cells host their own nested `Page`, which is why `Page` accepts a +non-zero `origin`. + +**`concrete/functional.py`** — `LinkText`, `ButtonText`, `FormFieldText`: `Text` +subclasses that are also `Interactable`. + +**`concrete/interaction_handler.py`** — `InteractionHandler`, `InteractionStateManager`: +routing taps and presses to registered elements and tracking pressed/released state. + +### `layout/` — the abstract → concrete transformation + +This is the package the pipeline diagram calls the layout engine. + +**`layout/document_layouter.py`** — a set of layouter functions, one per content kind, +all sharing a signature shape of *(abstract element, target `Page`) → did it fit*: ```python -# An Abstract Word knows its text content and semantic properties -word = Word("supercalifragilisticexpialidocious", font_style) -word.hyphenate() # Logical operation - finds break points -parts = word.get_hyphenated_parts() # Returns ["super-", "cali-", "fragi-", ...] +paragraph_layouter(paragraph, page, start_word=0, pretext=None, alignment_override=None) + -> (complete: bool, failed_word_index: int | None, remaining_pretext: Text | None) + +image_layouter(image, page, max_width=None, max_height=None) -> bool +table_layouter(table, page, style=None) -> bool +pagebreak_layouter(page_break, page) -> bool +button_layouter(button, page, font=None, padding=...) -> (bool, str) +form_field_layouter(field, page, font=None, ...) -> ... +form_layouter(form, page, font=None, field_spacing=10) -> (bool, list[str]) ``` -## Concrete Layer (`pyWebLayout/concrete/`) +They **append to a page**, they do not return a list of lines. The three-part return of +`paragraph_layouter` is what makes pagination resumable: when a paragraph runs off the +bottom of a page, the caller learns which word failed and whether a hyphenated fragment +is pending, and can continue on the next page from exactly there. -The Concrete layer handles the **spatial representation** and actual rendering of content. +`DocumentLayouter` wraps a `Page` and dispatches over a list of abstract elements by +type, holding the `ConcreteStyleRegistry` for the run. -### Key Components +**`layout/ereader_layout.py`** — the paginated reading model: +- `RenderingPosition` — a serialisable cursor expressed in *abstract* coordinates + (chapter, block, word, table cell, list item, pending pretext). Because it names + document structure rather than pixels, it survives font-size and page-size changes. +- `BidirectionalLayouter` — renders a page forward or backward from a position, + returning `(Page, next_position)`. +- `ChapterNavigator` / `ChapterInfo` — a table of contents built from heading structure. +- `FontScaler`, `FontFamilyOverride` — apply scale and family changes to blocks at + layout time without mutating the abstract document. -#### `concrete/text.py` -- `Text`: Renders a specific text fragment with precise positioning -- `Line`: Manages a line of `Text` objects with spacing and alignment -- Handles actual pixel measurements, font rendering, and positioning +**`layout/page_buffer.py`** — `PageBuffer` (LRU caches of rendered pages in both +directions, plus position maps) and `BufferedPageRenderer` (background rendering). -#### `concrete/page.py` -- `Page`: Top-level container for rendered content -- `Container`: Layout manager for organizing renderable objects -- Handles spatial layout, pagination, and visual composition +**`layout/ereader_manager.py`** — `EreaderLayoutManager`, the top-level application +interface (page turns, font changes, chapter jumps, progress), and `BookmarkManager` +for persisting bookmarks and the last reading position. -#### `concrete/box.py` -- `Box`: Base class for all spatially-aware renderable objects -- Provides positioning, sizing, and rendering capabilities +**`layout/table_optimizer.py`** — column width allocation for tables. -### Characteristics of Concrete Classes +### `io/readers/` — parsing -1. **Rendering-focused**: Handle pixels, fonts, images, and visual output -2. **Spatially-aware**: Know exact positions, sizes, and layout constraints -3. **Implementation-specific**: Tied to specific rendering technologies (PIL, etc.) -4. **Non-portable**: Rendering results are tied to specific display contexts +**`html_extraction.py`** — `parse_html_string(html, base_font=None, document=None, +base_path=None) -> List[Block]`. Built from a `StyleContext` (a `NamedTuple` threaded +down the tree, carrying the inherited font and styling) and a table of per-tag handlers +(`paragraph_handler`, `heading_handler`, `table_handler`, …). This is the only place +that knows about HTML. -### Example: Concrete Text +**`epub_reader.py`** — `EPUBReader` and `read_epub(path) -> Book`: container/OPF +parsing, spine and manifest, table of contents, cover handling, and image processing +(with an e-ink processor available by default). + +## Dependency rules + +The direction of dependency is what keeps the split honest: + +``` +io/readers → abstract → core, style +layout → abstract, concrete, core, style +concrete → abstract, core, style +abstract → core, style ← must NOT import concrete +``` + +`abstract/` importing from `concrete/` is the violation to watch for. `concrete/` +importing from `abstract/` is expected and correct: a `Text` may point back at the +`Word` it came from, and a table cell renderer reads its abstract `TableCell`. + +**Where the abstract layer touches rendering today, and why:** + +- Abstract classes are constructed with `Font` objects, which carry a pixel size and a + loaded font file. This is a deliberate compromise for parsing performance (HTML + styling resolves to a `Font` once, at parse time) but it does mean the abstract layer + is not fully rendering-independent. `AbstractStyle` is the intended replacement, and + `Word` already accepts either. +- `Word.concrete` / `Word.add_concete()` ([inline.py:42](pyWebLayout/abstract/inline.py#L42), + [:134](pyWebLayout/abstract/inline.py#L134)) is a back-reference to the `Text` objects + a word became. **It is currently written and never read**, and it cannot correctly + model the relationship anyway: one `Word` becomes many `Text`s across re-layouts, and + a single slot only remembers the most recent. Treat it as vestigial. When a + word→pixels mapping is genuinely needed, build it on `Text`'s `source` back-reference + (concrete pointing at abstract), which is the safe direction — note it is currently + stored as `_source` with no public accessor, and is itself unread today. + +## Worked example ```python -# A Concrete Text object handles actual rendering -text = Text("super-", font) # Specific text fragment -text._calculate_dimensions() # Computes exact pixel size -image = text.render() # Produces actual visual output +from pyWebLayout.io.readers.html_extraction import parse_html_string +from pyWebLayout.concrete.page import Page +from pyWebLayout.layout.document_layouter import DocumentLayouter + +# 1. Parse: markup -> abstract blocks +blocks = parse_html_string("

Chapter One

It was a dark and stormy night.

") +# [Heading, Paragraph] + +# 2. Lay out: abstract blocks -> concrete children appended to a Page +page = Page(size=(400, 600)) +DocumentLayouter(page).layout_document(blocks) + +# 3. Render: Page -> PIL.Image +image = page.render() + +# 4. Query: pixel -> content +result = page.query_point((50, 40)) +print(result.object_type, result.text) # e.g. "text" "Chapter" ``` -## The Transformation Process - -The architecture involves a clear transformation from Abstract to Concrete: - -``` -Abstract Document - ↓ - [Parser Layer] - ↓ -Abstract Blocks (Paragraph, Heading, etc.) - ↓ - [Layout Engine] - ↓ -Concrete Objects (Text, Line, Page) - ↓ - [Rendering Engine] - ↓ -Visual Output (Images, PDF, etc.) -``` - -### Example Transformation +Inspecting the intermediate concrete objects: ```python -# 1. Abstract content -paragraph = Paragraph() -paragraph.add_word(Word("This", font)) -paragraph.add_word(Word("is", font)) -paragraph.add_word(Word("a", font)) -paragraph.add_word(Word("test", font)) +from pyWebLayout.concrete.text import Line -# 2. Layout transformation -layout = ParagraphLayout(line_width=200, line_height=20) -lines = layout.layout_paragraph(paragraph) # Returns List[Line] - -# 3. Each Line contains concrete Text objects -for line in lines: - for text_obj in line.text_objects: # List[Text] - print(f"Text: '{text_obj.text}' at position {text_obj._origin}") +for line in (c for c in page.children if isinstance(c, Line)): + print([t.text for t in line.text_objects]) +# ['Chapter', 'One'] +# ['It', 'was', 'a', 'dark', 'and', 'stormy', 'night.'] ``` -## Key Architectural Principles +For paginated reading, drive `EreaderLayoutManager` instead of building pages by hand: -### 1. **Single Responsibility** -- Abstract classes: Handle content and structure -- Concrete classes: Handle rendering and layout - -### 2. **Separation of Concerns** -- Text parsing/processing ≠ Text rendering -- Document structure ≠ Page layout -- Content semantics ≠ Visual presentation - -### 3. **Immutable Abstract Content** -- Abstract content remains unchanged during rendering -- Multiple concrete representations can be generated from same abstract content -- Enables pagination, different formats, responsive layouts - -### 4. **One-to-Many Relationships** -- One Abstract Word → Multiple Concrete Text objects (hyphenation) -- One Abstract Paragraph → Multiple Concrete Lines -- One Abstract Document → Multiple Concrete Pages - -## Common Anti-Patterns to Avoid - -### ❌ **Mixing Concerns** ```python -# WRONG: Abstract class knowing about pixels +from pyWebLayout.layout.ereader_manager import EreaderLayoutManager + +manager = EreaderLayoutManager(blocks, page_size=(800, 600), document_id="my-book") +page = manager.get_current_page() +page = manager.next_page() +manager.set_font_scale(1.25) # re-lays out from the same RenderingPosition +``` + +## Design principles + +**1. Abstract content is not mutated by layout.** Font scaling and family overrides +produce scaled copies at layout time rather than editing the document. This is what +allows the same `blocks` list to back a buffer of pages at several font sizes. + +**2. Positions are expressed in abstract coordinates.** `RenderingPosition` names a +chapter, block, and word — never a pixel. A reader who changes font size stays on the +same sentence. + +**3. Layouters report partial success.** Every layouter returns whether it fit, and the +text layouter also returns where it stopped. Pagination is built from this rather than +from a separate page-breaking pass. + +**4. One abstract object may become many concrete ones.** One `Word` → one or two `Text` +fragments; one `Paragraph` → many `Line`s across many `Page`s; one `Document` → an +unbounded sequence of `Page`s. Any API that assumes a one-to-one mapping will be wrong +at a hyphen or a page boundary. + +**5. Measurement is cached, not avoided.** Text width and glyph rasterisation are the +hot path. `core/cache.py` bounds their cost; `prewarm_text_caches` front-loads it for a +known document. + +## Anti-patterns + +**Concrete state stored on abstract objects.** An abstract object that remembers its +rendered width, position, or `Text` objects is wrong at the second rendering. If a +back-reference is needed, point from concrete to abstract. + +```python +# WRONG class Word: def __init__(self, text): - self.text = text - self.rendered_width = None # ❌ Concrete concern in abstract class + self.rendered_width = None # invalidated by any font change + +# RIGHT +text = Text(word.text, font, draw, source=word) # concrete knows its origin ``` -### ❌ **renderable_words Concept** -```python -# WRONG: Confusing abstract and concrete -line.renderable_words # ❌ This suggests Words are renderable - # Words are abstract - only Text objects render -``` +**Treating `Word` as renderable.** Words are abstract; only `Text` draws. There is no +`renderable_words` anywhere in the codebase, and there should not be. -### ✅ **Correct Separation** -```python -# CORRECT: Clear separation -abstract_word = Word("test") # Abstract content -concrete_text = Text("test", font) # Concrete rendering -line.text_objects.append(concrete_text) # Concrete objects in concrete container -``` +**Assuming a layouter returns lines.** `paragraph_layouter` appends to a page and +reports what did not fit. Code that expects `List[Line]` back is working from an +outdated model. -## Benefits of This Architecture +## Summary -### 1. **Flexibility** -- Same content can be rendered at different sizes -- Multiple output formats from single source -- Easy to implement responsive design - -### 2. **Testability** -- Abstract logic can be tested without rendering -- Layout algorithms can be tested independently -- Visual rendering can be mocked - -### 3. **Performance** -- Abstract content can be cached and reused -- Layout can be computed once for multiple renderings -- Incremental updates possible - -### 4. **Maintainability** -- Clear boundaries between text processing and rendering -- Changes to rendering don't affect content parsing -- Easy to swap rendering backends - -## File Organization - -``` -pyWebLayout/ -├── abstract/ # Content and structure -│ ├── block.py # Document blocks (Paragraph, Heading, etc.) -│ ├── inline.py # Inline content (Word, etc.) -│ ├── document.py # Document structure -│ └── functional.py # Links, buttons, etc. -│ -├── concrete/ # Rendering and layout -│ ├── text.py # Text and Line rendering -│ ├── page.py # Page layout and containers -│ ├── box.py # Base rendering classes -│ ├── image.py # Image rendering -│ └── functional.py # Interactive elements -│ -├── typesetting/ # Layout algorithms -│ ├── paragraph_layout.py # Abstract → Concrete transformation -│ ├── flow.py # Text flow management -│ └── pagination.py # Page breaking logic -│ -└── style/ # Styling and formatting - ├── fonts.py # Font management - ├── layout.py # Layout constants - └── alignment.py # Alignment enums -``` - -## Conclusion - -The Abstract/Concrete separation is fundamental to pyWebLayout's design. It ensures clean separation between content processing and visual rendering, enabling flexible, maintainable, and testable document processing pipelines. - -**Remember**: -- **Abstract** = What to display (content, structure, semantics) -- **Concrete** = How to display it (pixels, fonts, positioning, rendering) - -This architecture enables the library to handle complex document layouts while maintaining clear, understandable code organization. +- **`core/`** — contracts and shared machinery +- **`style/`** — semantic style, and its resolution to concrete rendering parameters +- **`abstract/`** — what the document says +- **`concrete/`** — where the pixels go +- **`layout/`** — the transformation between them, and pagination on top of it +- **`io/readers/`** — markup in