Compare commits

...
6 Commits
Author SHA1 Message Date
dtourolle 7bebe08432 Merge layout remediation: pagination dead-end, page padding, text alignment
Python CI / test (3.10) (push) Has been cancelled
Python CI / test (3.12) (push) Has been cancelled
Python CI / test (3.13) (push) Has been cancelled
Three fixes from the block/table rendering audit, plus the spec covering the
remaining work.

S11: a paragraph larger than one page dead-ended the reader - the resume
position was discarded, so navigation reported no progress and the book
appeared to end mid-chapter.

S2: horizontal page padding was ignored, so text started flush against the left
border and lines broke short of the right one. Page now describes its content
box directly, and gained an origin so a page can be nested inside another
surface.

S13: ragged alignments stretched their word gaps by a varying amount per line;
justified paragraphs stretched their final line and fell a pixel or two short of
the margin. Body text now defaults to justified, configurable via
PageStyle.default_alignment.
2026-08-06 22:18:32 +02:00
dtourolle 1262be6a38 fix(text): constant word space for ragged alignment, exact justification (S13)
LeftAlignmentHandler spread each line's residual space across its word gaps,
clamped to max_spacing. A line whose residual divided to under max_spacing was
stretched flush, one that exceeded it was not, so left-aligned text was
justified sometimes, by a different amount per line - which reads as a wobbling
right edge rather than as ragged-right. Centre/right did the same, and computed
their start position from a different spacing than the one they returned, so
centred lines were not centred.

Ragged alignments now use a constant word space - the font's own space advance,
clamped to the style's bounds - and report overflow instead of tightening, so
line breaking decides what fits rather than rendering squeezing it.

Justification kept two further defects:

  - the final line of a paragraph was stretched across the measure, so a
    three-word tail was spread edge to edge. Line now carries is_paragraph_end,
    set on the line holding the last word, and renders flush left. A paragraph
    continued on the next page is not marked, so it stays justified.

  - gaps were floored per gap with a truncated remainder, discarding the
    fractional part of both. Lines stopped one or two pixels short, differently
    each time. Distributing by cumulative rounding makes the gaps sum to the
    residual exactly; advance ends now land identically on every line.

Alignment is configurable rather than hardcoded: PageStyle.default_alignment,
defaulting to JUSTIFY for body text. text_align on abstract and concrete styles
defaults to None meaning "unspecified", so HTML without text-align inherits the
page default while explicit CSS still wins. Headings are never justified.
2026-08-06 22:18:04 +02:00
dtourolle f18cec2da8 fix(layout): honour horizontal padding and page origin (S2)
paragraph_layouter placed lines at page.border_size while sizing them to
available_width, which subtracts both paddings. Text therefore started flush
against the left border and the entire padding budget accumulated on the right,
so lines broke well short of the right border.

Page now describes its content box directly - content_origin, content_rect and
remaining_height - and the layouters use it instead of each recomputing the
geometry from border_size. The four block layouters had all been computing
remaining space as size[1] - y_offset - border_size, subtracting the border but
not the bottom padding, so every block type could be placed into the bottom
padding; remaining_height fixes that too.

Page also gains an origin, defaulting to (0, 0). That is inert for a top-level
page but lets a page be positioned inside another surface, which table cells
need in order to be laid out by the normal engine.

Golden images regenerated: content now sits inside the padding on all sides.
2026-08-06 21:11:28 +02:00
dtourolle a57da8011e fix(ereader): keep resume position when a block spans a page (S11)
render_page_forward discarded new_pos on the failure path, but a block that
only partially fitted has still advanced the position: paragraph_layouter
reports the word it stopped at, and _layout_paragraph_on_page packs it into
new_pos. Dropping it told the caller no progress was made, so navigation
dead-ended on any paragraph larger than a single page - the reader saw "end of
document" mid-book.

A 2877-word paragraph at 800x600 rendered 26 lines and reported the start
position back; it now paginates across 12 pages.

Also guard the navigation loop: EreaderManager.next_page treats no-progress as
end-of-document, which is only correct at the actual end. Anywhere else it now
logs the offending block index and skips that block, so a future layout bug
costs one block rather than the rest of the book.
2026-08-06 21:06:29 +02:00
dtourolle 583366ae1d docs: add layout remediation spec
Twelve specs covering the defects found auditing the block/table rendering
path, plus the pagination dead-end and the broken background renderer.

Each spec carries a reproduction of the defect against 2a543d0, a design with
concrete signatures, and acceptance criteria. Five design invariants are stated
up front so future changes can be rejected by reference rather than re-argued.
2026-08-06 21:03:12 +02:00
dtourolleandClaude Opus 5 e000068384 Cache word widths and glyph bitmaps to cut page render time ~3.5x
Rendering a page re-measured and re-rasterised the same words constantly: at
1404x1872 a page issued ~2800 textlength calls and ~2500 draw.text calls for
fewer than 1000 distinct (font, string) pairs. Profiling a page turn showed
FreeType glyph rendering at 57% of total time and width measurement at 38% of
layout.

Both are now cached. Measured on Crime and Punishment at 1404x1872, one page:

    layout  35ms -> 19ms
    render  84ms -> 41ms
    total  120ms -> 60ms

Eviction ranks by use count rather than recency. Word frequency in prose is
Zipfian and stationary, so the words worth keeping are the ones used most, and
unlike recency this lets a document's own frequencies be seeded up front --
see prewarm_caches(). Two details keep the policy from costing more than it
saves, since get() runs once per word drawn:

  - counting is O(1) with no reordering, because structures that reorder on
    every hit measured 3-5ms/page slower than the hit rate they bought;
  - eviction samples 8 entries and drops the least used of those, rather than
    maintaining a global order.

Aging (halving all counts periodically) is on by default. Without it a font
size change drove the hit rate to 0% on a real access trace: every key was new
and the previous size's entries held counts nothing could beat.

Both caches are bounded, since the glyph bitmaps reach ~19MB over a long
session and the target is a 512MB Pi Zero 2. Defaults are 4MB of bitmaps and
8192 widths; configure_text_caches() tunes them. Cache size barely affects
speed (2MB is within 13% of unbounded) because a miss costs only one ~44us
rasterisation, so the bound can be set for memory, not throughput.

EreaderLayoutManager.prewarm_caches() counts the book's word frequencies and
preloads the most common ones, seeding each with its document frequency. This
moves that rasterisation to open time and cut misses by 27%, for ~15% faster
page turns at a one-off ~300ms cost. It is opt-in; nothing calls it yet.

Rendering is no longer bit-identical. PIL positions text at sub-pixel offsets,
so the cache buckets that phase, defaulting to 2 buckets per axis. Total ink
per page is unchanged and the mean pixel difference is 3.6/255 -- a fifth of
one step of a 16-level e-ink panel. subpixel_steps=4 halves that if wanted.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 20:49:32 +02:00
28 changed files with 2804 additions and 105 deletions
File diff suppressed because it is too large Load Diff
Binary file not shown.

Before

Width:  |  Height:  |  Size: 29 KiB

After

Width:  |  Height:  |  Size: 29 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 118 KiB

After

Width:  |  Height:  |  Size: 95 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 20 KiB

After

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 87 KiB

After

Width:  |  Height:  |  Size: 88 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 109 KiB

After

Width:  |  Height:  |  Size: 109 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 59 KiB

After

Width:  |  Height:  |  Size: 59 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 31 KiB

After

Width:  |  Height:  |  Size: 31 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 89 KiB

After

Width:  |  Height:  |  Size: 76 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 28 KiB

After

Width:  |  Height:  |  Size: 26 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 124 KiB

After

Width:  |  Height:  |  Size: 123 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 62 KiB

After

Width:  |  Height:  |  Size: 62 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 61 KiB

After

Width:  |  Height:  |  Size: 61 KiB

+12 -1
View File
@@ -4,7 +4,14 @@ Concrete layer for the pyWebLayout library.
This package contains concrete implementations that can be directly rendered.
"""
from .text import Text, Line
from .text import (
Text,
Line,
configure_text_caches,
clear_text_caches,
text_cache_stats,
prewarm_text_caches,
)
from .box import Box
from .image import RenderableImage
from .page import Page
@@ -22,4 +29,8 @@ __all__ = [
'Cell',
'LinkText',
'ButtonText',
'configure_text_caches',
'clear_text_caches',
'text_cache_stats',
'prewarm_text_caches',
]
+46 -8
View File
@@ -15,15 +15,19 @@ class Page(Renderable, Queriable):
contains a given point.
"""
def __init__(self, size: Tuple[int, int], style: Optional[PageStyle] = None):
def __init__(self, size: Tuple[int, int], style: Optional[PageStyle] = None,
origin: Tuple[int, int] = (0, 0)):
"""
Initialize a new page.
Args:
size: The total size of the page (width, height) including borders
style: The PageStyle defining borders, spacing, and appearance
origin: Absolute position of the page's top-left corner. Non-zero for
a page nested inside another surface, such as a table cell.
"""
self._size = size
self._origin = origin
self._style = style if style is not None else PageStyle()
self._children: List[Renderable] = []
self._canvas: Optional[Image.Image] = None
@@ -31,7 +35,8 @@ class Page(Renderable, Queriable):
# Initialize y_offset to start of content area
# Position the first line so its baseline is close to the top boundary
# For subsequent lines, baseline-to-baseline spacing is used
self._current_y_offset = self._style.border_width + self._style.padding_top
self._current_y_offset = (self._origin[1] + self._style.border_width
+ self._style.padding_top)
self._is_first_line = True # Track if we're placing the first line
# Callback registry for managing interactable elements
self._callbacks = CallbackRegistry()
@@ -39,8 +44,12 @@ class Page(Renderable, Queriable):
self._dirty = True
def free_space(self) -> Tuple[int, int]:
"""Get the remaining space on the page"""
return (self._size[0], self._size[1] - self._current_y_offset)
"""
Get the remaining space in the content area.
Deprecated: use content_rect and remaining_height, which this delegates to.
"""
return (self.content_rect[2], self.remaining_height)
def can_fit_line(
self,
@@ -59,7 +68,8 @@ class Page(Renderable, Queriable):
True if the line fits within page boundaries
"""
# Calculate the maximum Y position allowed (bottom boundary)
max_y = self._size[1] - self._style.border_width - self._style.padding_bottom
content_y, content_h = self.content_rect[1], self.content_rect[3]
max_y = content_y + content_h
# If ascent/descent not provided, use simple check (backward compatibility)
if ascent == 0 and descent == 0:
@@ -77,6 +87,34 @@ class Page(Renderable, Queriable):
"""Get the total page size including borders"""
return self._size
@property
def origin(self) -> Tuple[int, int]:
"""Absolute position of the page's top-left corner"""
return self._origin
@property
def content_origin(self) -> Tuple[int, int]:
"""
Absolute top-left of the content box: the page origin plus its border and
top/left padding. Layout starts here.
"""
return (
self._origin[0] + self._style.border_width + self._style.padding_left,
self._origin[1] + self._style.border_width + self._style.padding_top,
)
@property
def content_rect(self) -> Tuple[int, int, int, int]:
"""(x, y, width, height) of the content box, in absolute coordinates"""
x, y = self.content_origin
return (x, y, self.content_size[0], self.content_size[1])
@property
def remaining_height(self) -> int:
"""Content-box height still available below the current layout cursor"""
_, y, _, h = self.content_rect
return max(0, y + h - self._current_y_offset)
@property
def canvas_size(self) -> Tuple[int, int]:
"""Get the canvas size (page size minus borders)"""
@@ -182,7 +220,7 @@ class Page(Renderable, Queriable):
# Clear callback registry when clearing children
self._callbacks.clear()
# Reset y_offset to start of content area (after border and padding)
self._current_y_offset = self._style.border_width + self._style.padding_top
self._current_y_offset = self.content_origin[1]
return self
@property
@@ -532,6 +570,6 @@ class Page(Renderable, Queriable):
True if the point is within the page bounds
"""
return (
0 <= point[0] < self._size[0] and
0 <= point[1] < self._size[1]
self._origin[0] <= point[0] < self._origin[0] + self._size[0] and
self._origin[1] <= point[1] < self._origin[1] + self._size[1]
)
+407 -79
View File
@@ -6,11 +6,204 @@ from pyWebLayout.style import Alignment, Font, TextDecoration
from pyWebLayout.abstract import Word
from pyWebLayout.abstract.inline import LinkedWord
from pyWebLayout.abstract.functional import Link
from PIL import ImageDraw
from typing import Tuple, List, Optional
from pyWebLayout.core.cache import UsageCache, SizedUsageCache
from PIL import ImageDraw, ImageFont
from typing import Tuple, List, Optional, Any, Dict
import logging
import math
import numpy as np
from abc import ABC, abstractmethod
logger = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
# Text rendering caches
#
# A page re-measures and re-rasterises the same words constantly: measured over a
# novel at 1404x1872, a page issues ~2800 width measurements and ~2500 glyph
# rasterisations for fewer than 1000 distinct (font, string) pairs. Caching both
# turns a ~225ms page into a ~30ms page. Both caches are bounded so that a long
# reading session cannot grow without limit on a memory-constrained device.
# ---------------------------------------------------------------------------
# Word widths are small floats; 8192 entries costs well under 1MB and comfortably
# spans the working set of several chapters at a couple of font sizes.
DEFAULT_WIDTH_CACHE_ENTRIES = 8192
# Glyph bitmaps are the expensive ones: ~700 bytes each on average at 1404x1872,
# so an unbounded cache reaches ~12MB after 40 pages. 4MB holds several pages'
# worth of distinct words while leaving headroom on a 512MB Pi Zero 2.
DEFAULT_GLYPH_CACHE_BYTES = 4 * 1024 * 1024
# PIL rasterises text at sub-pixel horizontal offsets, so a cache keyed only on
# (font, string) would quantise every word to a whole pixel. Bucketing the
# sub-pixel phase keeps that error negligible at the cost of more entries. 2 steps
# holds the mean error to ~3.6/255 -- a fifth of one step of a 16-level e-ink
# panel -- while keeping the cache four times smaller than 4 steps would.
DEFAULT_GLYPH_SUBPIXEL_STEPS = 2
def _glyph_entry_bytes(entry: Tuple[Any, Tuple[int, int]]) -> int:
"""Approximate footprint of a cached (mask, offset) pair, in bytes."""
mask = entry[0]
try:
width, height = mask.size
except (AttributeError, TypeError, ValueError):
return 0
return width * height
_width_cache: UsageCache = UsageCache(DEFAULT_WIDTH_CACHE_ENTRIES)
_glyph_cache: SizedUsageCache = SizedUsageCache(DEFAULT_GLYPH_CACHE_BYTES, _glyph_entry_bytes)
_glyph_subpixel_steps: int = DEFAULT_GLYPH_SUBPIXEL_STEPS
# Set to False the first time the fast rasterisation path is found to be
# unavailable (e.g. a PIL build without the private ImageDraw internals it uses),
# after which every Text falls back to ImageDraw.text().
_glyph_fast_path_available: bool = True
def configure_text_caches(width_entries: Optional[int] = None,
glyph_bytes: Optional[int] = None,
subpixel_steps: Optional[int] = None):
"""
Tune the text rendering caches.
Memory-constrained targets should shrink these; a desktop rendering many font
sizes may benefit from raising them.
Args:
width_entries: Maximum cached word-width measurements.
glyph_bytes: Maximum total size of cached glyph bitmaps, in bytes.
subpixel_steps: Sub-pixel phase buckets per axis. 1 disables sub-pixel
positioning entirely (smallest cache, slightly softer text).
"""
global _glyph_subpixel_steps
if width_entries is not None:
_width_cache.resize(width_entries)
if glyph_bytes is not None:
_glyph_cache.resize(glyph_bytes)
if subpixel_steps is not None:
if subpixel_steps <= 0:
raise ValueError(f"subpixel_steps must be positive, got {subpixel_steps}")
if subpixel_steps != _glyph_subpixel_steps:
# Cached entries embed the phase bucket in their key.
_glyph_cache.clear()
_glyph_subpixel_steps = subpixel_steps
def clear_text_caches():
"""Drop all cached widths and glyph bitmaps."""
_width_cache.clear()
_glyph_cache.clear()
def text_cache_stats() -> Dict[str, Any]:
"""Occupancy and hit rates for both text caches, for tuning and diagnostics."""
return {
'width': _width_cache.stats(),
'glyph': _glyph_cache.stats(),
'glyph_subpixel_steps': _glyph_subpixel_steps,
'glyph_fast_path': _glyph_fast_path_available,
}
def prewarm_text_caches(entries,
draw: Optional[ImageDraw.ImageDraw] = None,
budget_bytes: Optional[int] = None,
max_words: Optional[int] = None) -> Tuple[int, int]:
"""
Preload the caches with a document's most frequent words.
A document states its own access distribution up front: the words it uses most
are the words every page will draw. Rasterising them once at open time moves
that work off the page-turn path, and seeding each entry with its document
frequency puts it in the right place in the eviction order immediately, rather
than after the cache has learned it.
This depends on eviction ranking by use count. Under recency eviction the
preloaded entries would be discarded by the first page of unfamiliar text; under
usage ranking a word occurring 4000 times outranks anything met while scanning
and stays resident. Measured over a 50-page trace, preloading cut misses by 27%
with usage ranking against 12% with recency.
Args:
entries: Iterable of ``(font, text, colour, frequency)``, where `font` is a
PIL font object, `colour` the fill the text will be drawn in, and
`frequency` the number of times the word occurs in the document.
draw: An ImageDraw sharing the page's mode, used to resolve ink and font
mode. A scratch RGBA context is used if omitted.
budget_bytes: Cap on bytes to preload. Defaults to half the glyph budget so
that live rendering keeps room to cache what preloading missed.
max_words: Cap on distinct words to preload, before sub-pixel variants.
Returns:
Tuple of (words preloaded, bytes preloaded).
"""
if not _glyph_fast_path_available:
return 0, 0
if draw is None:
from PIL import Image
draw = ImageDraw.Draw(Image.new('RGBA', (1, 1)))
if budget_bytes is None:
budget_bytes = _glyph_cache.max_bytes // 2
budget_bytes = min(budget_bytes, _glyph_cache.max_bytes)
ranked = sorted(entries, key=lambda e: -e[3])
if max_words is not None:
ranked = ranked[:max_words]
steps = _glyph_subpixel_steps
mode = draw.fontmode
draw_mode = draw.mode
ink_cache: Dict[Any, Any] = {}
words = 0
used = 0
for font, text, colour, frequency in ranked:
if frequency <= 1 or used >= budget_bytes:
break
if not isinstance(font, ImageFont.FreeTypeFont):
continue
try:
ink = ink_cache.get(colour)
if ink is None:
ink, _ = draw._getink(colour)
if ink is None:
continue
ink_cache[colour] = ink
# Measuring is cheap and every layout pass needs it.
_width_cache.put((font, text, draw_mode),
draw.textlength(text, font=font), count=frequency)
# Words land on arbitrary sub-pixel offsets, so cover every horizontal
# phase. Baselines are whole pixels, so only phase 0 is needed
# vertically.
for x_bucket in range(steps):
entry = font.getmask2(text, mode, anchor="ls", ink=ink,
start=(x_bucket / steps, 0.0))
_glyph_cache.put((font, text, mode, ink, x_bucket, 0), entry,
count=frequency)
used += _glyph_entry_bytes(entry)
words += 1
except AttributeError:
logger.warning("Glyph cache unavailable for this Pillow build; "
"skipping prewarm.", exc_info=True)
return words, used
except (TypeError, ValueError):
continue
logger.debug("Prewarmed %d words (%.2fMB) into the text caches",
words, used / 1e6)
return words, used
class AlignmentHandler(ABC):
"""
@@ -21,7 +214,9 @@ class AlignmentHandler(ABC):
@abstractmethod
def calculate_spacing_and_position(self, text_objects: List['Text'],
available_width: int, min_spacing: int,
max_spacing: int) -> Tuple[int, int, bool]:
max_spacing: int,
natural_spacing: Optional[int] = None
) -> Tuple[int, int, bool]:
"""
Calculate the spacing between words and starting position for the line.
@@ -30,9 +225,12 @@ class AlignmentHandler(ABC):
available_width: Total width available for the line
min_spacing: Minimum spacing between words
max_spacing: Maximum spacing between words
natural_spacing: The font's own space width. Ragged alignments use it
as a constant gap; justification ignores it. Defaults to
min_spacing when not supplied.
Returns:
Tuple of (spacing_between_words, starting_x_position)
Tuple of (spacing_between_words, starting_x_position, overflow)
"""
@@ -43,16 +241,23 @@ class LeftAlignmentHandler(AlignmentHandler):
text_objects: List['Text'],
available_width: int,
min_spacing: int,
max_spacing: int) -> Tuple[int, int, bool]:
max_spacing: int,
natural_spacing: Optional[int] = None
) -> Tuple[int, int, bool]:
"""
Calculate spacing and position for left-aligned text objects.
CREngine-inspired: never allow negative spacing, always use minimum spacing for overflow.
Left-aligned text uses a constant word space and leaves whatever is left
over as a ragged right edge. It must not spread the residual space across
the gaps: that stretches each line by a different amount, which reads as
badly-set justified text rather than as ragged-right.
Args:
text_objects (List[Text]): A list of text objects to be laid out.
available_width (int): The total width available for layout.
min_spacing (int): Minimum spacing between text objects.
max_spacing (int): Maximum spacing between text objects.
natural_spacing (Optional[int]): The font's own space width.
Returns:
Tuple[int, int, bool]: Spacing, start position, and overflow flag.
@@ -61,33 +266,19 @@ class LeftAlignmentHandler(AlignmentHandler):
if len(text_objects) <= 1:
return 0, 0, False
# Calculate the total length of all text objects
text_length = sum([text.width for text in text_objects])
spacing = min_spacing if natural_spacing is None else natural_spacing
spacing = max(min_spacing, min(max_spacing, int(spacing)))
# Calculate number of gaps between texts
text_length = sum([text.width for text in text_objects])
num_gaps = len(text_objects) - 1
# Calculate minimum space needed (text + minimum gaps)
min_total_width = text_length + (min_spacing * num_gaps)
# The spacing is constant whether or not the content fits: tightening a
# full line here would make it differ from its neighbours, which is the
# variation this alignment is supposed to avoid. Report the overflow and
# let line breaking move the offending word instead.
overflow = text_length + (spacing * num_gaps) > available_width
# Check if we have overflow (CREngine pattern: always use min_spacing for
# overflow)
if min_total_width > available_width:
return min_spacing, 0, True # Overflow - but use safe minimum spacing
# Calculate residual space left after accounting for text lengths
residual_space = available_width - text_length
# Calculate ideal spacing
actual_spacing = residual_space // num_gaps
# Clamp within bounds (CREngine pattern: respect max_spacing)
if actual_spacing > max_spacing:
return max_spacing, 0, False
elif actual_spacing < min_spacing:
# Ensure we never return spacing less than min_spacing
return min_spacing, 0, False
else:
return actual_spacing, 0, False # Use calculated spacing
return spacing, 0, overflow
class CenterRightAlignmentHandler(AlignmentHandler):
@@ -98,10 +289,18 @@ class CenterRightAlignmentHandler(AlignmentHandler):
def calculate_spacing_and_position(self, text_objects: List['Text'],
available_width: int, min_spacing: int,
max_spacing: int) -> Tuple[int, int, bool]:
"""Center/right alignment uses minimum spacing with calculated start position."""
max_spacing: int,
natural_spacing: Optional[int] = None
) -> Tuple[int, int, bool]:
"""
Centre/right alignment: constant word space, line shifted as a block.
Like left alignment, the residual space must not be spread across the
gaps - it belongs in the margin. The start position is then derived from
the same spacing that will actually be used, so the line lands where it
was measured to land.
"""
word_length = sum([word.width for word in text_objects])
residual_space = available_width - word_length
# Handle single word case
if len(text_objects) <= 1:
@@ -109,23 +308,21 @@ class CenterRightAlignmentHandler(AlignmentHandler):
start_position = (available_width - word_length) // 2
else: # RIGHT
start_position = available_width - word_length
return 0, max(0, start_position), False
return 0, max(0, int(start_position)), False
actual_spacing = residual_space // (len(text_objects) - 1)
ideal_space = (min_spacing + max_spacing) / 2
if actual_spacing > 0.5 * (min_spacing + max_spacing):
actual_spacing = 0.5 * (min_spacing + max_spacing)
spacing = min_spacing if natural_spacing is None else natural_spacing
spacing = max(min_spacing, min(max_spacing, int(spacing)))
content_length = word_length + (len(text_objects) - 1) * actual_spacing
num_gaps = len(text_objects) - 1
overflow = word_length + (spacing * num_gaps) > available_width
content_length = word_length + num_gaps * spacing
if self._alignment == Alignment.CENTER:
start_position = (available_width - content_length) // 2
else:
start_position = available_width - content_length
if actual_spacing < min_spacing:
return actual_spacing, max(0, start_position), True
return ideal_space, max(0, start_position), False
return spacing, max(0, int(start_position)), overflow
class JustifyAlignmentHandler(AlignmentHandler):
@@ -137,10 +334,14 @@ class JustifyAlignmentHandler(AlignmentHandler):
def calculate_spacing_and_position(self, text_objects: List['Text'],
available_width: int, min_spacing: int,
max_spacing: int) -> Tuple[int, int, bool]:
max_spacing: int,
natural_spacing: Optional[int] = None
) -> Tuple[int, int, bool]:
"""
Justified alignment distributes space to fill the entire line width.
natural_spacing is ignored: filling the measure is the whole point.
For justified text, we ALWAYS try to fill the entire width by distributing
space between words, regardless of max_spacing constraints. The only limit
is min_spacing to ensure readability.
@@ -150,26 +351,28 @@ class JustifyAlignmentHandler(AlignmentHandler):
residual_space = available_width - word_length
num_gaps = max(1, len(text_objects) - 1)
# For justified text, calculate the actual spacing needed to fill the line
base_spacing = int(residual_space // num_gaps)
remainder = int(residual_space % num_gaps) # The extra pixels to distribute
# Check if we have enough space for minimum spacing
if base_spacing < min_spacing:
if residual_space // num_gaps < min_spacing:
# Not enough space - this is overflow
self._gap_spacings = [min_spacing] * num_gaps
return min_spacing, 0, True
# Distribute remainder pixels across the first 'remainder' gaps
# This ensures the line fills the entire width exactly
# Distribute the residual by cumulative rounding rather than by taking a
# floor per gap and scattering the remainder. Word widths are fractional,
# so flooring each gap loses part of a pixel and truncating the remainder
# loses up to another - the line then stops one or two pixels short of the
# margin, and by a different amount on each line, which is visible as a
# ragged right edge on otherwise justified text. Rounding the running
# total makes the gaps sum to the residual exactly.
total = int(round(residual_space))
self._gap_spacings = []
for i in range(num_gaps):
if i < remainder:
self._gap_spacings.append(base_spacing + 1)
else:
self._gap_spacings.append(base_spacing)
placed = 0
for i in range(1, num_gaps + 1):
cumulative = int(round(total * i / num_gaps))
self._gap_spacings.append(cumulative - placed)
placed = cumulative
return base_spacing, 0, False
return self._gap_spacings[0], 0, False
class Text(Renderable, Queriable):
@@ -205,9 +408,19 @@ class Text(Renderable, Queriable):
def _calculate_dimensions(self):
"""Calculate the width and height of the text based on the font metrics"""
# Get the size using PIL's text size functionality
# Measuring a word costs a FreeType shaping pass, and the same words recur
# constantly within a document, so results are cached per (font, string).
# The draw's image mode is part of the key because PIL derives advance
# widths differently for bilevel ("1") targets.
font = self._style.font
self._width = self._draw.textlength(self._text, font=font)
key = (font, self._text, self._draw.mode)
width = _width_cache.get(key)
if width is None:
width = self._draw.textlength(self._text, font=font)
_width_cache.put(key, width)
self._width = width
ascent, descent = font.getmetrics()
self._ascent = ascent
self._middle_y = ascent - descent / 2
@@ -343,23 +556,95 @@ class Text(Renderable, Queriable):
A PIL Image containing the rendered text
"""
style = self._style
# Draw the text background if specified
if self._style.background and self._style.background[3] > 0: # If alpha > 0
self._draw.rectangle([self._origin, self._origin +
self._size], fill=self._style.background)
if style.background and style.background[3] > 0: # If alpha > 0
self._draw.rectangle([tuple(self._origin), tuple(self._origin + self.size)],
fill=style.background)
# Draw the text using baseline as anchor point ("ls" = left-baseline)
# This ensures the origin represents the baseline, not the top-left
self._draw.text(
(self.origin[0],
self._origin[1]),
self._text,
font=self._style.font,
fill=self._style.colour,
anchor="ls")
if not self._render_from_glyph_cache(style):
self._draw.text(
(self.origin[0],
self._origin[1]),
self._text,
font=style.font,
fill=style.colour,
anchor="ls")
# Apply any text decorations with knowledge of next text
self._apply_decoration(next_text, spacing)
if style.decoration != TextDecoration.NONE:
self._apply_decoration(next_text, spacing)
def _render_from_glyph_cache(self, style) -> bool:
"""
Blit this word from the cached glyph bitmap.
Rasterising a word is the single most expensive step in drawing a page, and
the same words recur constantly, so the bitmap PIL would produce is cached
and blitted directly. This reproduces what ImageDraw.text() does internally
(getmask2 followed by draw_bitmap) minus the per-call setup.
Returns:
True if the word was drawn. False means the caller must fall back to
ImageDraw.text().
"""
global _glyph_fast_path_available
if not _glyph_fast_path_available:
return False
draw = self._draw
font = style.font
# Bitmap and other non-FreeType fonts do not expose getmask2's anchor and
# sub-pixel arguments; let PIL handle them.
if not isinstance(font, ImageFont.FreeTypeFont):
return False
try:
ink, _ = draw._getink(style.colour)
if ink is None:
return False
# floor() rather than modf() so the fraction is always in [0, 1),
# keeping bucket indices non-negative for negative coordinates.
x = float(self._origin[0])
y = float(self._origin[1])
x_whole = math.floor(x)
y_whole = math.floor(y)
steps = _glyph_subpixel_steps
x_bucket = int((x - x_whole) * steps)
y_bucket = int((y - y_whole) * steps)
mode = draw.fontmode
key = (font, self._text, mode, ink, x_bucket, y_bucket)
entry = _glyph_cache.get(key)
if entry is None:
entry = font.getmask2(
self._text, mode, anchor="ls", ink=ink,
start=(x_bucket / steps, y_bucket / steps))
_glyph_cache.put(key, entry)
mask, offset = entry
draw.draw.draw_bitmap((x_whole + offset[0], y_whole + offset[1]), mask, ink)
return True
except AttributeError:
# A PIL build without the internals this path relies on. Stop trying.
logger.warning(
"Glyph cache unavailable for this Pillow build; falling back to "
"ImageDraw.text() for all text rendering.", exc_info=True)
_glyph_fast_path_available = False
return False
except (TypeError, ValueError):
# This particular colour/mode combination is not supported by the fast
# path (e.g. an ink PIL cannot resolve). Others may still be.
return False
class Line(Box):
@@ -417,6 +702,13 @@ class Line(Box):
self._spacing_render = (spacing[0] + spacing[1]) // 2
self._position_render = 0
# The font's own space advance. Ragged alignments use this as their
# constant word gap rather than stretching to fill the measure.
try:
self._natural_spacing = int(round(self._font.font.getlength(" ")))
except (AttributeError, TypeError, ValueError):
self._natural_spacing = None
# Hyphenation configuration parameters
self._min_word_length_for_brute_force = min_word_length_for_brute_force
self._min_chars_before_hyphen = min_chars_before_hyphen
@@ -425,6 +717,34 @@ class Line(Box):
# Create the appropriate alignment handler
self._alignment_handler = self._create_alignment_handler(halign)
# Set on the final line of a paragraph. Justification stretches a line to
# fill the column, which is wrong for the last line - a three-word tail
# would be spread across the full measure. The last line takes its
# natural width instead, as in every other typesetting system.
self._is_paragraph_end = False
@property
def is_paragraph_end(self) -> bool:
"""Whether this is the final line of its paragraph"""
return self._is_paragraph_end
@is_paragraph_end.setter
def is_paragraph_end(self, value: bool):
self._is_paragraph_end = value
@property
def render_alignment_handler(self) -> AlignmentHandler:
"""
The handler used to position text when rendering.
This differs from the fitting handler only for the last line of a
justified paragraph, which is rendered flush left.
"""
if self._is_paragraph_end and isinstance(
self._alignment_handler, JustifyAlignmentHandler):
return LeftAlignmentHandler()
return self._alignment_handler
def _create_alignment_handler(self, alignment: Alignment) -> AlignmentHandler:
"""
Create the appropriate alignment handler based on the alignment type.
@@ -500,7 +820,8 @@ class Line(Box):
text = Text.from_word(word, self._draw)
self._text_objects.append(text)
spacing, position, overflow = self._alignment_handler.calculate_spacing_and_position(
self._text_objects, self._size[0], self._spacing[0], self._spacing[1])
self._text_objects, self._size[0], self._spacing[0], self._spacing[1],
self._natural_spacing)
if not overflow:
# Word fits! Add it completely
@@ -547,7 +868,8 @@ class Line(Box):
# Check if first part fits
self._text_objects.append(first_text)
spacing, position, overflow = self._alignment_handler.calculate_spacing_and_position(
self._text_objects, self._size[0], self._spacing[0], self._spacing[1])
self._text_objects, self._size[0], self._spacing[0], self._spacing[1],
self._natural_spacing)
_ = self._text_objects.pop()
if not overflow:
@@ -618,7 +940,8 @@ class Line(Box):
# Verify the first part actually fits
self._text_objects.append(first_text)
spacing, position, overflow = self._alignment_handler.calculate_spacing_and_position(
self._text_objects, self._size[0], self._spacing[0], self._spacing[1])
self._text_objects, self._size[0], self._spacing[0], self._spacing[1],
self._natural_spacing)
if not overflow:
# Brute force split works!
@@ -643,10 +966,15 @@ class Line(Box):
Returns:
A PIL Image containing the rendered line
"""
# Recalculate spacing and position for current text objects to ensure accuracy
# Recalculate spacing and position for current text objects to ensure
# accuracy. Word fitting used the paragraph's alignment; rendering uses
# render_alignment_handler, which differs only for the last line of a
# justified paragraph.
handler = self.render_alignment_handler
if len(self._text_objects) > 0:
spacing, position, overflow = self._alignment_handler.calculate_spacing_and_position(
self._text_objects, self._size[0], self._spacing[0], self._spacing[1])
spacing, position, overflow = handler.calculate_spacing_and_position(
self._text_objects, self._size[0], self._spacing[0], self._spacing[1],
self._natural_spacing)
self._spacing_render = spacing
self._position_render = position
@@ -664,10 +992,10 @@ class Line(Box):
1 < len(self._text_objects) else None
# Get the spacing for this specific gap (variable for justified text)
if isinstance(self._alignment_handler, JustifyAlignmentHandler) and \
hasattr(self._alignment_handler, '_gap_spacings') and \
i < len(self._alignment_handler._gap_spacings):
current_spacing = self._alignment_handler._gap_spacings[i]
if isinstance(handler, JustifyAlignmentHandler) and \
hasattr(handler, '_gap_spacings') and \
i < len(handler._gap_spacings):
current_spacing = handler._gap_spacings[i]
else:
current_spacing = self._spacing_render
+343
View File
@@ -0,0 +1,343 @@
"""
Bounded usage-ranked caches for the text rendering hot path.
Laying out and rasterising a page re-measures and re-draws the same words over and
over: a typical page issues ~2800 width measurements and ~2500 glyph rasterisations
for fewer than 1000 distinct (font, string) pairs. Caching both collapses that work,
but an unbounded cache is not an option on a memory-constrained target such as a
Raspberry Pi Zero 2, where the rasterised bitmaps reach ~19MB over a long session.
Eviction is by **usage count**, not recency. Word frequency in prose is Zipfian and
stationary -- a small set of words ("the", "and", "of") accounts for most tokens on
every page, and that set barely shifts as the reader advances -- so the words worth
keeping are exactly the ones used most.
Two design choices keep this from costing more than it saves, because `get` runs
once per word drawn (~2500 times per page):
* **Counting is O(1) with no reordering.** Each entry carries its own use counter,
bumped in place. Ranking structures that reorder on every hit (a frequency-bucket
LFU, or an LRU's linked-list splice) were measured 3-5ms per page slower than the
hit rate they buy is worth.
* **Eviction samples rather than sorts.** Finding the globally least-used entry
would need a heap kept current on every hit. Instead a small random sample is
drawn and the least-used member of it evicted, the same approximation Redis uses
for its LFU policy. With the default sample size the evicted entry is very
likely to be in the bottom few percent, which is all that matters here.
Both are single-threaded by design; the rendering path holds the GIL throughout and
adding locking would cost more than it protects.
"""
from __future__ import annotations
import random
from typing import Any, Callable, Dict, Generic, Hashable, List, Optional, TypeVar
K = TypeVar('K', bound=Hashable)
V = TypeVar('V')
# Entries examined per eviction. Larger samples approximate true least-frequently-used
# more closely at linear cost; 8 puts the victim in the bottom ~15% of entries, which
# is ample when the alternative is a rasterisation that costs ~60us either way.
DEFAULT_EVICTION_SAMPLE = 8
# Halving every entry's use count after this many insertions keeps the cache
# responsive to a change of working set. Without it, entries that were hot long ago
# retain counts a newly-hot entry cannot beat and are never evicted -- the classic
# failure of pure frequency eviction. Measured on a real access trace, a font-size
# change drove hit rate to 0% without aging and left it unchanged with it.
DEFAULT_AGING_INTERVAL = 10000
# Index of each field in an entry. Entries are plain lists rather than tuples or
# objects so the counter can be bumped in place, without rehashing the key.
_VALUE = 0
_COUNT = 1
_SLOT = 2
class _UsageRanked(Generic[K, V]):
"""
Shared usage-count bookkeeping for the caches below.
Entries live in a dict for lookup and, in parallel, in a flat list that makes
uniform random sampling possible. Each entry records its own index in that list
so removal can swap in the tail element and stay O(1).
Subclasses supply the bound by implementing :meth:`_over_budget` and the
accounting hooks :meth:`_record_add` / :meth:`_record_remove`.
"""
def __init__(self,
aging_interval: Optional[int] = DEFAULT_AGING_INTERVAL,
eviction_sample: int = DEFAULT_EVICTION_SAMPLE):
if aging_interval is not None and aging_interval <= 0:
raise ValueError(f"aging_interval must be positive, got {aging_interval}")
if eviction_sample <= 0:
raise ValueError(f"eviction_sample must be positive, got {eviction_sample}")
self._aging_interval = aging_interval
self._eviction_sample = eviction_sample
self._entries: Dict[K, List[Any]] = {}
self._slots: List[K] = []
self._randrange = random.randrange
self._inserts_since_aging = 0
self._hits = 0
self._misses = 0
self._evictions = 0
self._agings = 0
# -- subclass hooks ----------------------------------------------------
def _over_budget(self) -> bool:
raise NotImplementedError
def _record_add(self, key: K, value: V):
"""Account for a value entering the cache."""
def _record_remove(self, key: K):
"""Account for a value leaving the cache."""
# -- core operations ---------------------------------------------------
def get(self, key: K) -> Optional[V]:
"""Return the cached value for `key`, or None, counting the use."""
entry = self._entries.get(key)
if entry is None:
self._misses += 1
return None
entry[_COUNT] += 1
self._hits += 1
return entry[_VALUE]
def _add_new(self, key: K, value: V):
"""Insert a key not currently present."""
# New entries start at 1 rather than 0 so that a single reuse is enough to
# outrank an entry that has never been touched since the last aging pass.
self._entries[key] = [value, 1, len(self._slots)]
self._slots.append(key)
self._record_add(key, value)
def _remove(self, key: K):
"""Remove a key outright, keeping the sampling list dense."""
entry = self._entries.pop(key)
slot = entry[_SLOT]
last = self._slots.pop()
if last != key:
self._slots[slot] = last
self._entries[last][_SLOT] = slot
self._record_remove(key)
def _evict_one(self) -> bool:
"""Evict the least-used member of a random sample. False if empty."""
count = len(self._slots)
if not count:
return False
if count <= self._eviction_sample:
victim = min(self._slots, key=lambda k: self._entries[k][_COUNT])
else:
randrange = self._randrange
entries = self._entries
slots = self._slots
victim = slots[randrange(count)]
best = entries[victim][_COUNT]
for _ in range(self._eviction_sample - 1):
candidate = slots[randrange(count)]
score = entries[candidate][_COUNT]
if score < best:
victim, best = candidate, score
self._remove(victim)
self._evictions += 1
return True
def _evict_to_budget(self):
while self._over_budget():
if not self._evict_one():
break
def _maybe_age(self):
"""Halve every use count once the aging interval has elapsed."""
if self._aging_interval is None:
return
self._inserts_since_aging += 1
if self._inserts_since_aging < self._aging_interval:
return
self._inserts_since_aging = 0
self._agings += 1
for entry in self._entries.values():
entry[_COUNT] = entry[_COUNT] // 2 or 1
def clear(self):
"""Drop all entries. Counters are preserved."""
self._entries.clear()
self._slots.clear()
self._inserts_since_aging = 0
def _base_stats(self) -> Dict[str, Any]:
total = self._hits + self._misses
return {
'entries': len(self._entries),
'hits': self._hits,
'misses': self._misses,
'evictions': self._evictions,
'agings': self._agings,
'hit_rate': (self._hits / total) if total else 0.0,
}
def __len__(self) -> int:
return len(self._entries)
def __contains__(self, key: object) -> bool:
return key in self._entries
class UsageCache(_UsageRanked[K, V]):
"""
Usage-ranked cache bounded by number of entries.
Args:
max_entries: Maximum number of entries to retain. Must be positive.
aging_interval: Insertions between halving all use counts, or None to
disable aging. See :data:`DEFAULT_AGING_INTERVAL`.
eviction_sample: Entries sampled per eviction.
"""
def __init__(self, max_entries: int,
aging_interval: Optional[int] = DEFAULT_AGING_INTERVAL,
eviction_sample: int = DEFAULT_EVICTION_SAMPLE):
if max_entries <= 0:
raise ValueError(f"max_entries must be positive, got {max_entries}")
super().__init__(aging_interval, eviction_sample)
self._max_entries = max_entries
def _over_budget(self) -> bool:
return len(self._entries) > self._max_entries
def put(self, key: K, value: V, count: int = 1):
"""
Insert `value`, evicting the least-used entries past the bound.
Args:
count: Initial use count. Pass a document-derived frequency to rank a
preloaded entry ahead of words that have not been seen yet.
"""
existing = self._entries.get(key)
if existing is not None:
existing[_VALUE] = value
existing[_COUNT] += 1
return
self._add_new(key, value)
if count > 1:
self._entries[key][_COUNT] = count
self._evict_to_budget()
self._maybe_age()
@property
def max_entries(self) -> int:
return self._max_entries
def resize(self, max_entries: int):
"""Change the bound, evicting immediately if the cache now overflows."""
if max_entries <= 0:
raise ValueError(f"max_entries must be positive, got {max_entries}")
self._max_entries = max_entries
self._evict_to_budget()
def stats(self) -> Dict[str, Any]:
"""Hit/miss/eviction counters and current occupancy."""
stats = self._base_stats()
stats['max_entries'] = self._max_entries
return stats
class SizedUsageCache(_UsageRanked[K, V]):
"""
Usage-ranked cache bounded by the total size of its values.
Args:
max_bytes: Maximum total value size to retain. Must be positive.
sizer: Returns the size in bytes of a value. Called once per insertion.
aging_interval: Insertions between halving all use counts, or None to
disable aging. See :data:`DEFAULT_AGING_INTERVAL`.
eviction_sample: Entries sampled per eviction.
A value larger than `max_bytes` on its own is returned to the caller but not
retained, so that one oversized entry cannot flush the whole cache.
"""
def __init__(self, max_bytes: int, sizer: Callable[[V], int],
aging_interval: Optional[int] = DEFAULT_AGING_INTERVAL,
eviction_sample: int = DEFAULT_EVICTION_SAMPLE):
if max_bytes <= 0:
raise ValueError(f"max_bytes must be positive, got {max_bytes}")
super().__init__(aging_interval, eviction_sample)
self._max_bytes = max_bytes
self._sizer = sizer
self._sizes: Dict[K, int] = {}
self._total_bytes = 0
def _over_budget(self) -> bool:
return self._total_bytes > self._max_bytes
def _record_add(self, key: K, value: V):
size = self._sizer(value)
self._sizes[key] = size
self._total_bytes += size
def _record_remove(self, key: K):
self._total_bytes -= self._sizes.pop(key)
def put(self, key: K, value: V, count: int = 1):
"""
Insert `value`, evicting the least-used entries past the bound.
Args:
count: Initial use count. Pass a document-derived frequency to rank a
preloaded entry ahead of words that have not been seen yet.
"""
if key in self._entries:
# Re-measure: the replacement may be a different size.
self._remove(key)
if self._sizer(value) > self._max_bytes:
# Too large to ever retain; skip rather than flush everything for it.
return
self._add_new(key, value)
if count > 1:
self._entries[key][_COUNT] = count
self._evict_to_budget()
self._maybe_age()
@property
def max_bytes(self) -> int:
return self._max_bytes
@property
def total_bytes(self) -> int:
return self._total_bytes
def resize(self, max_bytes: int):
"""Change the bound, evicting immediately if the cache now overflows."""
if max_bytes <= 0:
raise ValueError(f"max_bytes must be positive, got {max_bytes}")
self._max_bytes = max_bytes
self._evict_to_budget()
def clear(self):
"""Drop all entries. Counters are preserved."""
super().clear()
self._sizes.clear()
self._total_bytes = 0
def stats(self) -> Dict[str, Any]:
"""Hit/miss/eviction counters and current occupancy."""
stats = self._base_stats()
stats['total_bytes'] = self._total_bytes
stats['max_bytes'] = self._max_bytes
return stats
+29 -13
View File
@@ -8,7 +8,7 @@ from pyWebLayout.concrete.image import RenderableImage
from pyWebLayout.concrete.functional import ButtonText, FormFieldText
from pyWebLayout.concrete.table import TableRenderer, TableStyle
from pyWebLayout.abstract import Paragraph, Word
from pyWebLayout.abstract.block import Image as AbstractImage, PageBreak, Table
from pyWebLayout.abstract.block import Image as AbstractImage, Heading, PageBreak, Table
from pyWebLayout.abstract.functional import Button, Form, FormField
from pyWebLayout.style.concrete_style import ConcreteStyleRegistry, RenderingContext, StyleResolver
from pyWebLayout.style import Font, Alignment
@@ -51,6 +51,15 @@ def paragraph_layouter(paragraph: Paragraph,
# We need to get word spacing constraints from the Font's abstract style if available
# For now, use reasonable defaults based on font size
# Alignment for text that does not specify its own. Headings are never
# justified - stretching a two-word title across the measure is always wrong -
# so they fall back to flush left.
default_alignment = getattr(page.style, 'default_alignment', None)
if not isinstance(default_alignment, Alignment):
default_alignment = Alignment.JUSTIFY
if isinstance(paragraph, Heading):
default_alignment = Alignment.LEFT
if isinstance(paragraph.style, Font):
# paragraph.style is already a Font (concrete style)
font = paragraph.style
@@ -59,7 +68,7 @@ def paragraph_layouter(paragraph: Paragraph,
min_spacing = float(font.font_size) * 0.25 # 25% of font size
max_spacing = float(font.font_size) * 0.5 # 50% of font size
word_spacing_constraints = (int(min_spacing), int(max_spacing))
text_align = Alignment.LEFT # Default alignment
text_align = default_alignment
else:
# paragraph.style is an AbstractStyle, resolve it
# Ensure font_size is an int (it could be a FontSize enum)
@@ -79,7 +88,8 @@ def paragraph_layouter(paragraph: Paragraph,
int(concrete_style.word_spacing_min),
int(concrete_style.word_spacing_max)
)
text_align = concrete_style.text_align
# text_align is None when the source did not specify one.
text_align = concrete_style.text_align or default_alignment
# Apply page-level word spacing override if specified
if hasattr(
@@ -151,7 +161,7 @@ def paragraph_layouter(paragraph: Paragraph,
y_cursor = page._current_y_offset
else:
y_cursor = page._current_y_offset
x_cursor = page.border_size
x_cursor = page.content_origin[0]
# Create a temporary Text object to calculate word width
if word:
@@ -260,7 +270,13 @@ def paragraph_layouter(paragraph: Paragraph,
else:
current_pretext = overflow_text # May be None or hyphenated remainder
# All words processed successfully
# All words processed successfully. The line holding the final word is the
# end of the paragraph, so it is rendered at its natural width rather than
# justified to the full column. A paragraph continued on the next page does
# not reach here, so its lines stay justified - which is correct.
if current_line is not None:
current_line.is_paragraph_end = True
return True, None, None
@@ -305,7 +321,7 @@ def image_layouter(image: AbstractImage, page: Page, max_width: Optional[int] =
max_width = page.available_width
# Calculate available height on page
available_height = page.size[1] - page._current_y_offset - page.border_size
available_height = page.remaining_height
# If no space available, image doesn't fit
if available_height <= 0:
@@ -325,7 +341,7 @@ def image_layouter(image: AbstractImage, page: Page, max_width: Optional[int] =
return False
# Create renderable image
x_offset = page.border_size
x_offset = page.content_origin[0]
y_offset = page._current_y_offset
# Access page.draw to ensure canvas is initialized
@@ -368,7 +384,7 @@ def table_layouter(
"""
# Calculate available space
available_width = page.available_width
x_offset = page.border_size
x_offset = page.content_origin[0]
y_offset = page._current_y_offset
# Access page.draw to ensure canvas is initialized
@@ -388,7 +404,7 @@ def table_layouter(
# Check if table fits on current page
table_height = renderer.size[1]
available_height = page.size[1] - y_offset - page.border_size
available_height = page.remaining_height
if table_height > available_height:
return False
@@ -436,7 +452,7 @@ def button_layouter(button: Button,
font = Font(font_size=14, colour=(255, 255, 255))
# Calculate available space
available_height = page.size[1] - page._current_y_offset - page.border_size
available_height = page.remaining_height
# Create ButtonText renderable
button_text = ButtonText(button, font, page.draw, padding=padding)
@@ -447,7 +463,7 @@ def button_layouter(button: Button,
return False, ""
# Position the button
x_offset = page.border_size
x_offset = page.content_origin[0]
y_offset = page._current_y_offset
button_text.set_origin(np.array([x_offset, y_offset]))
@@ -486,7 +502,7 @@ def form_field_layouter(field: FormField, page: Page, font: Optional[Font] = Non
font = Font(font_size=12, colour=(0, 0, 0))
# Calculate available space
available_height = page.size[1] - page._current_y_offset - page.border_size
available_height = page.remaining_height
# Create FormFieldText renderable
field_text = FormFieldText(field, font, page.draw, field_height=field_height)
@@ -497,7 +513,7 @@ def form_field_layouter(field: FormField, page: Page, font: Optional[Font] = Non
return False, ""
# Position the field
x_offset = page.border_size
x_offset = page.content_origin[0]
y_offset = page._current_y_offset
field_text.set_origin(np.array([x_offset, y_offset]))
+8 -1
View File
@@ -344,7 +344,14 @@ class BidirectionalLayouter:
scaled_block, page, current_pos, font_scale)
if not success:
# Block doesn't fit, we're done with this page
# The block did not fit in its entirety. It may still have been
# laid out partially - a paragraph larger than one page places as
# many lines as fit and reports the word it stopped at. Keeping
# that resume point is what allows the next page to continue;
# discarding it tells the caller no progress was made, which
# dead-ends navigation on the block forever.
if self._position_compare(new_pos, current_pos) > 0:
current_pos = new_pos
break
# Add inter-block spacing after successfully laying out a block
+79
View File
@@ -9,6 +9,7 @@ into a unified, easy-to-use API.
from __future__ import annotations
from typing import List, Dict, Optional, Tuple, Any, Callable
import json
import logging
from pathlib import Path
from .ereader_layout import RenderingPosition, ChapterNavigator, ChapterInfo
@@ -20,6 +21,8 @@ from pyWebLayout.style.page_style import PageStyle
from pyWebLayout.style.fonts import BundledFont
from pyWebLayout.layout.document_layouter import image_layouter
logger = logging.getLogger(__name__)
class BookmarkManager:
"""
@@ -219,6 +222,67 @@ class EreaderLayoutManager:
self.chapter_changed_callback: Optional[Callable[[
Optional[ChapterInfo]], None]] = None
def prewarm_caches(self, max_words: int = 2000,
budget_bytes: Optional[int] = None) -> Tuple[int, int]:
"""
Preload the text caches with this document's most frequent words.
Counts how often each word occurs in the book and rasterises the most
common ones ahead of time, so that the work lands at open time rather than
on the first page turns. Entries are seeded with their document frequency,
which is what keeps them resident under usage-ranked eviction.
Safe to call again after a font change; the fonts differ, so the new
entries simply take their place in the eviction order alongside the old.
Args:
max_words: Maximum distinct words to preload.
budget_bytes: Bytes of glyph cache to fill. Defaults to half the budget.
Returns:
Tuple of (words preloaded, bytes preloaded).
"""
from collections import Counter
from pyWebLayout.concrete.text import prewarm_text_caches
from .ereader_layout import FontScaler
override = getattr(self.renderer.layouter, 'font_family_override', None)
# Count by (style, text): the same word in a heading and in body text is a
# different rasterisation, and both are worth counting separately.
counts: Dict[Tuple[int, str], int] = Counter()
styles: Dict[int, Any] = {}
for block in self.blocks:
words = getattr(block, '_words', None)
if not words:
continue
for word in words:
style = word.style
if style is None:
continue
key = id(style)
styles.setdefault(key, style)
counts[(key, word.text)] += 1
# Resolve each distinct style once through the same scaling the layouter
# applies, so the preloaded keys match what rendering will look up.
scaled: Dict[int, Any] = {}
for key, style in styles.items():
try:
scaled[key] = FontScaler.scale_font(style, self.font_scale, override)
except Exception:
continue
entries = []
for (style_key, text), count in counts.items():
font = scaled.get(style_key)
if font is None:
continue
entries.append((font.font, text, font.colour, count))
return prewarm_text_caches(entries, budget_bytes=budget_bytes,
max_words=max_words)
def set_position_changed_callback(
self, callback: Callable[[RenderingPosition], None]):
"""Set callback for position changes"""
@@ -356,6 +420,21 @@ class EreaderLayoutManager:
self._notify_position_changed()
return self.get_current_page()
# No progress. That is the correct answer only at the end of the
# document; anywhere else a block has failed to lay out and would trap
# the reader on this page. Skipping the block costs one block, not the
# rest of the book.
if self.current_position.block_index < len(self.blocks):
logger.error(
"Block %d made no layout progress; skipping it. This is a layout "
"bug - the block placed nothing and reported no resume point.",
self.current_position.block_index)
self.current_position = RenderingPosition(
chapter_index=self.current_position.chapter_index,
block_index=self.current_position.block_index + 1)
self._notify_position_changed()
return self.get_current_page()
return None # At end of document
def previous_page(self) -> Optional[Page]:
+2 -1
View File
@@ -81,7 +81,8 @@ class AbstractStyle:
background_color: Optional[Union[str, Tuple[int, int, int, int]]] = None
# Text properties
text_align: TextAlign = TextAlign.LEFT
# None means "not specified": the page's default_alignment applies.
text_align: Optional[TextAlign] = None
line_height: Optional[Union[str, float]] = None # "normal", "1.2", 1.5, etc.
letter_spacing: Optional[Union[str, float]] = None # "normal", "0.1em", etc.
word_spacing: Optional[Union[str, float]] = None
+2 -1
View File
@@ -61,7 +61,8 @@ class ConcreteStyle:
decoration: TextDecoration = TextDecoration.NONE
# Layout properties
text_align: TextAlign = TextAlign.LEFT
# None means "not specified": the page's default_alignment applies.
text_align: Optional[TextAlign] = None
line_height: float = 1.0 # Multiplier
letter_spacing: float = 0.0 # In pixels
word_spacing: float = 0.0 # In pixels
+7 -1
View File
@@ -1,5 +1,7 @@
from typing import Tuple
from dataclasses import dataclass
from dataclasses import dataclass, field
from pyWebLayout.style.alignment import Alignment
@dataclass
@@ -8,6 +10,10 @@ class PageStyle:
Defines the styling properties for a page including borders, spacing, and layout.
"""
# Alignment applied to body text that does not specify its own. Headings are
# never justified regardless of this setting.
default_alignment: Alignment = Alignment.JUSTIFY
# Border properties
border_width: int = 0
border_color: Tuple[int, int, int] = (0, 0, 0)
+176
View File
@@ -0,0 +1,176 @@
"""
Regression tests for word spacing under each alignment (spec S13).
Only justified text stretches word gaps to fill the measure. Left, centre and
right aligned text use a natural, constant word space and leave a ragged edge;
previously they distributed the residual space across the gaps, which produced
text that looked justified but did not reach the margin, with a right edge that
wobbled by several pixels from line to line.
The final line of a justified paragraph is also not stretched.
"""
import pytest
from pyWebLayout.abstract.block import Paragraph
from pyWebLayout.abstract.inline import Word
from pyWebLayout.concrete.page import Page
from pyWebLayout.concrete.text import (
CenterRightAlignmentHandler,
JustifyAlignmentHandler,
LeftAlignmentHandler,
Line,
)
from pyWebLayout.layout.document_layouter import paragraph_layouter
from pyWebLayout.style import Alignment, Font
from pyWebLayout.style.page_style import PageStyle
PAGE = (500, 400)
PADDING = (20, 20, 20, 20)
@pytest.fixture
def font():
return Font(font_size=14)
def lay_out(font, alignment, text, size=PAGE):
page = Page(size=size, style=PageStyle(border_width=0, padding=PADDING))
paragraph = Paragraph(font)
for word in text.split():
paragraph.add_word(Word(word, font))
paragraph_layouter(paragraph, page, alignment_override=alignment)
return page
def rendered_lines(page):
lines = [c for c in page.children if isinstance(c, Line) and c._text_objects]
for line in lines:
line.render()
return lines
def gaps_of(line):
"""Observed pixel gaps between consecutive words on a rendered line."""
tos = line._text_objects
return [int(tos[i + 1]._origin[0]) - (int(tos[i]._origin[0]) + int(tos[i].width))
for i in range(len(tos) - 1)]
BODY = ("Paragraph text that is automatically laid out when this paragraph does "
"not fit on the current page the layouter will create a new page for it "
"which differs from using an explicit page break marker in the source ") * 2
class TestLeftAlignmentUsesConstantSpacing:
def test_gaps_are_uniform_within_a_line(self, font):
page = lay_out(font, Alignment.LEFT, BODY)
for line in rendered_lines(page):
gaps = gaps_of(line)
if len(gaps) > 1:
assert max(gaps) - min(gaps) <= 1, \
f"left-aligned gaps should be constant, got {gaps}"
def test_gaps_are_uniform_across_lines(self, font):
"""The regression: each line got its own stretch factor."""
page = lay_out(font, Alignment.LEFT, BODY)
all_gaps = [g for line in rendered_lines(page) for g in gaps_of(line)]
assert max(all_gaps) - min(all_gaps) <= 1, \
f"spacing must not vary line to line, got {sorted(set(all_gaps))}"
def test_lines_do_not_reach_the_right_margin(self, font):
"""Left-aligned text is ragged; a flush right edge means it was stretched."""
page = lay_out(font, Alignment.LEFT, BODY)
right = page.content_rect[0] + page.content_rect[2]
ends = [max(int(t._origin[0]) + int(t.width) for t in line._text_objects)
for line in rendered_lines(page)]
assert not all(right - e <= 1 for e in ends), \
"every line reached the margin exactly - text was justified, not left aligned"
def test_handler_returns_natural_spacing(self, font):
handler = LeftAlignmentHandler()
from pyWebLayout.concrete.text import Text
from PIL import Image, ImageDraw
draw = ImageDraw.Draw(Image.new("RGB", (10, 10)))
texts = [Text(w, font, draw) for w in ["Hello", "World"]]
spacing, position, overflow = handler.calculate_spacing_and_position(
texts, 400, 3, 7, natural_spacing=5)
assert spacing == 5, "natural spacing should be used verbatim when it fits"
assert position == 0
assert not overflow
def test_handler_clamps_natural_spacing_to_bounds(self, font):
handler = LeftAlignmentHandler()
from pyWebLayout.concrete.text import Text
from PIL import Image, ImageDraw
draw = ImageDraw.Draw(Image.new("RGB", (10, 10)))
texts = [Text(w, font, draw) for w in ["Hello", "World"]]
assert handler.calculate_spacing_and_position(
texts, 400, 3, 7, natural_spacing=99)[0] == 7
assert handler.calculate_spacing_and_position(
texts, 400, 3, 7, natural_spacing=1)[0] == 3
class TestJustifyStillFills:
def test_body_lines_reach_the_margin(self, font):
page = lay_out(font, Alignment.JUSTIFY, BODY)
lines = rendered_lines(page)
right = page.content_rect[0] + page.content_rect[2]
for line in lines:
if line.is_paragraph_end:
continue
end = max(int(t._origin[0]) + int(t.width) for t in line._text_objects)
assert right - end <= 2, f"justified line fell {right - end}px short"
def test_last_line_is_not_stretched(self, font):
page = lay_out(font, Alignment.JUSTIFY,
BODY + " and then a deliberately short tail.")
lines = rendered_lines(page)
last = [line for line in lines if line.is_paragraph_end]
assert last, "the final line of a completed paragraph must be marked"
gaps = gaps_of(last[-1])
if gaps:
assert max(gaps) <= 8, \
f"final line was justified across the measure, gaps={gaps}"
def test_continued_paragraph_keeps_justification(self, font):
"""A paragraph split across pages: its lines are not paragraph ends."""
page = lay_out(font, Alignment.JUSTIFY, BODY * 6, size=(500, 200))
lines = rendered_lines(page)
assert lines, "the page should hold some lines"
assert not any(line.is_paragraph_end for line in lines), \
"an unfinished paragraph has no final line on this page"
class TestCentreAndRight:
def test_centre_uses_constant_spacing_and_is_centred(self, font):
page = lay_out(font, Alignment.CENTER, BODY)
right = page.content_rect[0] + page.content_rect[2]
left = page.content_rect[0]
for line in rendered_lines(page):
tos = line._text_objects
# Float extents: integer truncation of each end would itself skew the
# comparison by a pixel.
start = float(tos[0]._origin[0])
end = float(tos[-1]._origin[0]) + tos[-1].width
# Equal margins either side, within rounding of the half-space.
assert abs((start - left) - (right - end)) <= 2, \
f"line not centred: left margin {start - left}, right {right - end}"
def test_right_aligned_lines_end_at_the_margin(self, font):
page = lay_out(font, Alignment.RIGHT, BODY)
right = page.content_rect[0] + page.content_rect[2]
for line in rendered_lines(page):
end = max(int(t._origin[0]) + int(t.width) for t in line._text_objects)
assert right - end <= 2, f"right-aligned line fell {right - end}px short"
+133
View File
@@ -0,0 +1,133 @@
"""
Regression tests for page content geometry (spec S2).
Content must be laid out inside the content box - the page box less its border
and padding - on all four sides. Horizontal padding was previously ignored on the
left, shifting every line left by padding_left and leaving a gutter of
padding_left + padding_right on the right, so lines appeared to break early.
"""
import pytest
from pyWebLayout.abstract.block import Paragraph
from pyWebLayout.abstract.inline import Word
from pyWebLayout.concrete.page import Page
from pyWebLayout.layout.document_layouter import DocumentLayouter
from pyWebLayout.style import Font
from pyWebLayout.style.page_style import PageStyle
PADDING = (40, 30, 40, 20) # top, right, bottom, left - deliberately asymmetric
@pytest.fixture
def font():
return Font(font_size=12)
def filled_page(size, style, font, word_count=120):
page = Page(size=size, style=style)
paragraph = Paragraph(font)
for i in range(word_count):
paragraph.add_word(Word(f"word{i}", font))
DocumentLayouter(page).layout_paragraph(paragraph)
return page
class TestContentBox:
"""content_origin / content_rect describe the box content lives in."""
def test_content_origin_includes_border_and_padding(self):
page = Page(size=(400, 300), style=PageStyle(border_width=2, padding=PADDING))
assert page.content_origin == (2 + 20, 2 + 40)
def test_content_rect_subtracts_both_paddings(self):
page = Page(size=(400, 300), style=PageStyle(border_width=2, padding=PADDING))
x, y, w, h = page.content_rect
assert (x, y) == (22, 42)
assert w == 400 - 2 * 2 - 20 - 30
assert h == 300 - 2 * 2 - 40 - 40
def test_page_origin_offsets_the_content_box(self):
"""A page placed inside another surface reports absolute coordinates."""
page = Page(size=(100, 50), style=PageStyle(border_width=1, padding=(5, 5, 5, 5)),
origin=(200, 300))
assert page.content_origin == (206, 306)
def test_remaining_height_respects_bottom_padding(self, font):
style = PageStyle(border_width=2, padding=PADDING)
page = Page(size=(400, 300), style=style)
# Nothing laid out yet: the whole content box is available.
assert page.remaining_height == page.content_rect[3]
class TestLinePlacement:
"""Lines must start after the left padding and end before the right padding."""
def test_first_line_starts_at_content_origin(self, font):
style = PageStyle(border_width=2, padding=PADDING)
page = filled_page((400, 300), style, font)
line = page.children[0]
assert int(line.origin[0]) == page.content_origin[0]
assert int(line.origin[1]) == page.content_origin[1]
def test_line_width_matches_content_width(self, font):
style = PageStyle(border_width=2, padding=PADDING)
page = filled_page((400, 300), style, font)
line = page.children[0]
assert int(line.size[0]) == page.content_rect[2]
def test_no_line_extends_past_the_right_content_edge(self, font):
style = PageStyle(border_width=2, padding=PADDING)
page = filled_page((400, 300), style, font)
right_edge = page.content_rect[0] + page.content_rect[2]
for line in page.children:
assert int(line.origin[0]) + int(line.size[0]) <= right_edge
def test_ink_stays_inside_the_content_box(self, font):
"""The rendered pixels, not just the boxes, respect the padding."""
style = PageStyle(border_width=0, padding=PADDING,
background_color=(255, 255, 255))
page = filled_page((400, 300), style, font)
image = page.render().convert("L")
pixels = image.load()
inked_x = [x for x in range(400) for y in range(300) if pixels[x, y] < 128]
assert inked_x, "the page should have text on it"
x0, _, w, _ = page.content_rect
assert min(inked_x) >= x0
assert max(inked_x) <= x0 + w
def test_right_gutter_is_not_double_width(self, font):
"""
The regression: text was shifted left by padding_left, so the right gutter
was padding_left + padding_right wide while the left gutter was zero.
"""
style = PageStyle(border_width=0, padding=(10, 30, 10, 30))
page = filled_page((400, 300), style, font, word_count=200)
image = page.render().convert("L")
pixels = image.load()
inked_x = [x for x in range(400) for y in range(300) if pixels[x, y] < 128]
left_gutter = min(inked_x)
right_gutter = 400 - max(inked_x)
# Justification means the right edge is not always exactly flush, so allow
# slack - but the two gutters must be comparable, not 0 vs 60.
assert abs(left_gutter - right_gutter) < 25, \
f"asymmetric gutters: left={left_gutter} right={right_gutter}"
class TestBlockBottomBoundary:
"""Blocks must not be placed into the bottom padding."""
def test_lines_stop_before_bottom_padding(self, font):
style = PageStyle(border_width=2, padding=PADDING)
page = filled_page((400, 300), style, font, word_count=500)
bottom_edge = page.content_rect[1] + page.content_rect[3]
for line in page.children:
assert int(line.origin[1]) <= bottom_edge
+244
View File
@@ -0,0 +1,244 @@
"""
Unit tests for the bounded usage-ranked caches.
Covers the guarantees the text rendering path depends on: that the bounds are never
exceeded, that eviction prefers the least-used entries, that aging lets a new
working set displace an old one, and that document-frequency seeding survives a
scan of unfamiliar keys.
"""
import unittest
from pyWebLayout.core.cache import (
UsageCache,
SizedUsageCache,
DEFAULT_AGING_INTERVAL,
)
class TestUsageCache(unittest.TestCase):
"""Entry-count-bounded cache."""
def test_rejects_invalid_bounds(self):
for bad in (0, -1):
with self.assertRaises(ValueError):
UsageCache(bad)
with self.assertRaises(ValueError):
UsageCache(4, aging_interval=0)
with self.assertRaises(ValueError):
UsageCache(4, eviction_sample=0)
def test_stores_and_returns_values(self):
cache = UsageCache(4)
cache.put('a', 1)
self.assertEqual(cache.get('a'), 1)
self.assertIsNone(cache.get('missing'))
self.assertIn('a', cache)
self.assertEqual(len(cache), 1)
def test_never_exceeds_max_entries(self):
cache = UsageCache(10)
for i in range(500):
cache.put(i, i)
self.assertLessEqual(len(cache), 10)
self.assertEqual(cache.stats()['entries'], 10)
def test_evicts_least_used(self):
# One hot key among many cold ones must survive a long cold scan. The
# sample is smaller than the cache, so this is probabilistic in principle;
# a hot key's count is far enough above the rest to make it reliable.
cache = UsageCache(20, eviction_sample=8)
cache.put('hot', 'value')
for _ in range(200):
cache.get('hot')
for i in range(400):
cache.put(f'cold{i}', i)
cache.get('hot')
self.assertEqual(cache.get('hot'), 'value')
def test_repeated_put_does_not_duplicate(self):
cache = UsageCache(10)
for _ in range(50):
cache.put('a', 1)
self.assertEqual(len(cache), 1)
def test_put_updates_existing_value(self):
cache = UsageCache(10)
cache.put('a', 1)
cache.put('a', 2)
self.assertEqual(cache.get('a'), 2)
def test_seeded_count_outranks_fresh_entries(self):
"""A document-frequency seed must survive a scan of unseen keys."""
cache = UsageCache(20, eviction_sample=8)
cache.put('frequent', 'value', count=5000)
for i in range(400):
cache.put(f'new{i}', i)
self.assertEqual(cache.get('frequent'), 'value')
def test_aging_lets_a_new_working_set_take_over(self):
"""Without aging, stale high counts lock the cache permanently."""
cache = UsageCache(20, aging_interval=50, eviction_sample=8)
for i in range(20):
cache.put(f'old{i}', i, count=10000)
# A completely different working set, each key used a few times.
for round_ in range(60):
for i in range(10):
key = f'new{i}'
if cache.get(key) is None:
cache.put(key, i)
survivors = sum(1 for i in range(10) if f'new{i}' in cache)
self.assertGreater(survivors, 0,
"aging should let the new working set displace the old")
self.assertGreater(cache.stats()['agings'], 0)
def test_aging_can_be_disabled(self):
cache = UsageCache(10, aging_interval=None)
for i in range(100):
cache.put(i, i)
self.assertEqual(cache.stats()['agings'], 0)
def test_resize_evicts_immediately(self):
cache = UsageCache(100)
for i in range(100):
cache.put(i, i)
cache.resize(10)
self.assertEqual(len(cache), 10)
with self.assertRaises(ValueError):
cache.resize(0)
def test_clear_empties_but_keeps_counters(self):
cache = UsageCache(10)
cache.put('a', 1)
cache.get('a')
cache.clear()
self.assertEqual(len(cache), 0)
self.assertNotIn('a', cache)
self.assertEqual(cache.stats()['hits'], 1)
def test_stats_track_hits_and_misses(self):
cache = UsageCache(10)
cache.put('a', 1)
cache.get('a')
cache.get('a')
cache.get('b')
stats = cache.stats()
self.assertEqual(stats['hits'], 2)
self.assertEqual(stats['misses'], 1)
self.assertAlmostEqual(stats['hit_rate'], 2 / 3)
self.assertEqual(stats['max_entries'], 10)
def test_internal_slot_list_stays_consistent(self):
"""Eviction swaps the tail into the freed slot; indices must stay valid."""
cache = UsageCache(8)
for i in range(300):
cache.put(i, i)
for key in list(cache._entries):
self.assertEqual(cache._slots[cache._entries[key][2]], key)
self.assertEqual(len(cache._slots), len(cache._entries))
class TestSizedUsageCache(unittest.TestCase):
"""Byte-bounded cache, as used for glyph bitmaps."""
@staticmethod
def sizer(value):
return value
def test_rejects_invalid_bounds(self):
for bad in (0, -1):
with self.assertRaises(ValueError):
SizedUsageCache(bad, self.sizer)
def test_never_exceeds_max_bytes(self):
cache = SizedUsageCache(1000, self.sizer)
for i in range(500):
cache.put(i, 100)
self.assertLessEqual(cache.total_bytes, 1000)
def test_tracks_total_bytes(self):
cache = SizedUsageCache(1000, self.sizer)
cache.put('a', 100)
cache.put('b', 250)
self.assertEqual(cache.total_bytes, 350)
def test_oversized_value_is_not_retained(self):
"""One huge entry must not flush everything else out."""
cache = SizedUsageCache(1000, self.sizer)
cache.put('small', 100)
cache.put('huge', 5000)
self.assertNotIn('huge', cache)
self.assertIn('small', cache)
self.assertEqual(cache.total_bytes, 100)
def test_replacing_a_value_remeasures_it(self):
cache = SizedUsageCache(1000, self.sizer)
cache.put('a', 100)
cache.put('a', 300)
self.assertEqual(cache.total_bytes, 300)
self.assertEqual(len(cache), 1)
def test_evicts_least_used(self):
cache = SizedUsageCache(1000, self.sizer, eviction_sample=8)
cache.put('hot', 100)
for _ in range(200):
cache.get('hot')
for i in range(400):
cache.put(f'cold{i}', 100)
cache.get('hot')
self.assertIn('hot', cache)
def test_seeded_count_outranks_fresh_entries(self):
cache = SizedUsageCache(1000, self.sizer, eviction_sample=8)
cache.put('frequent', 100, count=5000)
for i in range(400):
cache.put(f'new{i}', 100)
self.assertIn('frequent', cache)
def test_resize_evicts_immediately(self):
cache = SizedUsageCache(10000, self.sizer)
for i in range(100):
cache.put(i, 100)
cache.resize(500)
self.assertLessEqual(cache.total_bytes, 500)
def test_clear_resets_byte_accounting(self):
cache = SizedUsageCache(1000, self.sizer)
cache.put('a', 100)
cache.clear()
self.assertEqual(cache.total_bytes, 0)
self.assertEqual(len(cache), 0)
def test_stats_report_bounds(self):
cache = SizedUsageCache(1000, self.sizer)
cache.put('a', 100)
stats = cache.stats()
self.assertEqual(stats['total_bytes'], 100)
self.assertEqual(stats['max_bytes'], 1000)
self.assertEqual(stats['entries'], 1)
def test_bookkeeping_stays_consistent_under_churn(self):
"""Byte total and slot list must not drift over many evictions."""
cache = SizedUsageCache(2000, self.sizer, aging_interval=97)
for i in range(2000):
cache.put(i, (i % 7 + 1) * 50)
if i % 3 == 0:
cache.get(i)
self.assertEqual(cache.total_bytes,
sum(cache._sizes[k] for k in cache._entries))
self.assertEqual(len(cache._slots), len(cache._entries))
self.assertLessEqual(cache.total_bytes, cache.max_bytes)
class TestDefaults(unittest.TestCase):
def test_aging_is_enabled_by_default(self):
self.assertIsNotNone(DEFAULT_AGING_INTERVAL)
self.assertGreater(DEFAULT_AGING_INTERVAL, 0)
self.assertIsNotNone(UsageCache(4)._aging_interval)
if __name__ == '__main__':
unittest.main()
+123
View File
@@ -0,0 +1,123 @@
"""
Regression tests for blocks that span more than one page.
A block larger than a single page is laid out partially, and the layouter reports
where it stopped. If that resume point is discarded, the reader is told it made no
progress and navigation dead-ends on that block (spec S11).
"""
import pytest
from pyWebLayout.abstract.block import Paragraph
from pyWebLayout.abstract.inline import Word
from pyWebLayout.concrete.page import Page
from pyWebLayout.layout.ereader_layout import BidirectionalLayouter, RenderingPosition
from pyWebLayout.style import Font
from pyWebLayout.style.page_style import PageStyle
PAGE_SIZE = (800, 600)
def make_paragraph(word_count, font):
"""A paragraph of distinct words, so we can verify none are lost or repeated."""
paragraph = Paragraph(font)
for i in range(word_count):
paragraph.add_word(Word(f"word{i}", font))
return paragraph
@pytest.fixture
def font():
return Font(font_size=16)
@pytest.fixture
def huge_paragraph(font):
"""A paragraph far larger than one page - the shape that dead-ended."""
return make_paragraph(2877, font)
class TestPageSpanningParagraph:
"""A single paragraph larger than one page must paginate, not dead-end."""
def test_layouter_reports_where_it_stopped(self, huge_paragraph):
layouter = BidirectionalLayouter([huge_paragraph], PageStyle(), PAGE_SIZE)
page = Page(size=PAGE_SIZE, style=PageStyle())
success, new_pos = layouter._layout_block_on_page(
huge_paragraph, page, RenderingPosition(), 1.0)
assert not success, "a 2877-word paragraph cannot fit on one page"
assert new_pos.word_index > 0, "the resume point must be reported"
def test_first_page_advances(self, huge_paragraph):
"""The regression: next position equalled the start position."""
layouter = BidirectionalLayouter([huge_paragraph], PageStyle(), PAGE_SIZE)
start = RenderingPosition()
page, next_pos = layouter.render_page_forward(start, 1.0)
assert len(page.children) > 0, "content was placed on the page"
assert (next_pos.block_index, next_pos.word_index) > \
(start.block_index, start.word_index), \
"a page with content on it must advance the position"
def test_paginates_to_completion(self, huge_paragraph):
"""Every page advances, and the document terminates."""
layouter = BidirectionalLayouter([huge_paragraph], PageStyle(), PAGE_SIZE)
pos = RenderingPosition()
positions = [(pos.block_index, pos.word_index)]
for _ in range(100):
page, next_pos = layouter.render_page_forward(pos, 1.0)
key = (next_pos.block_index, next_pos.word_index)
if next_pos.block_index >= 1:
break # ran off the end of the (single-block) document
assert key > positions[-1], f"no progress at page {len(positions)}"
positions.append(key)
pos = next_pos
else:
pytest.fail("pagination did not terminate")
assert len(positions) > 5, "a 2877-word paragraph spans several pages"
def test_no_words_lost_or_repeated(self, huge_paragraph):
"""Word coverage across pages is exactly the paragraph, in order."""
layouter = BidirectionalLayouter([huge_paragraph], PageStyle(), PAGE_SIZE)
pos = RenderingPosition()
boundaries = [0]
for _ in range(100):
_, next_pos = layouter.render_page_forward(pos, 1.0)
if next_pos.block_index >= 1:
break
boundaries.append(next_pos.word_index)
pos = next_pos
assert boundaries == sorted(boundaries), "word indices must not go backward"
assert len(boundaries) == len(set(boundaries)), "a page must not be re-rendered"
class TestNonSpanningBlocksUnaffected:
"""The fix must not change behaviour for blocks that fit."""
def test_small_paragraphs_still_advance_by_block(self, font):
blocks = [make_paragraph(20, font) for _ in range(3)]
layouter = BidirectionalLayouter(blocks, PageStyle(), PAGE_SIZE)
_, next_pos = layouter.render_page_forward(RenderingPosition(), 1.0)
assert next_pos.block_index == 3, "all three short paragraphs fit on one page"
assert next_pos.word_index == 0
def test_empty_document_terminates(self):
layouter = BidirectionalLayouter([], PageStyle(), PAGE_SIZE)
start = RenderingPosition()
page, next_pos = layouter.render_page_forward(start, 1.0)
assert next_pos.block_index == start.block_index
assert len(page.children) == 0
+10
View File
@@ -24,6 +24,12 @@ class TestDocumentLayouter:
self.mock_page.border_size = 20
self.mock_page._current_y_offset = 50
self.mock_page.available_width = 400
# Content geometry: a 440x600 page with a 20px border and no padding, so
# the content box starts at (20, 20) and is 400 wide.
self.mock_page.size = (440, 600)
self.mock_page.content_origin = (20, 20)
self.mock_page.content_rect = (20, 20, 400, 560)
self.mock_page.remaining_height = 530 # 20 + 560 - 50
self.mock_page.draw = Mock()
self.mock_page.can_fit_line = Mock(return_value=True)
self.mock_page.add_child = Mock()
@@ -603,6 +609,10 @@ class TestTableLayouter:
self.mock_page._current_y_offset = 50
self.mock_page.available_width = 600
self.mock_page.size = (800, 1000)
# Content geometry: 800x1000 page, 20px border, no padding.
self.mock_page.content_origin = (20, 20)
self.mock_page.content_rect = (20, 20, 600, 960)
self.mock_page.remaining_height = 930 # 20 + 960 - 50
# Create mock draw and canvas
self.mock_draw = Mock()