Cache word widths and glyph bitmaps to cut page render time ~3.5x
Rendering a page re-measured and re-rasterised the same words constantly: at
1404x1872 a page issued ~2800 textlength calls and ~2500 draw.text calls for
fewer than 1000 distinct (font, string) pairs. Profiling a page turn showed
FreeType glyph rendering at 57% of total time and width measurement at 38% of
layout.
Both are now cached. Measured on Crime and Punishment at 1404x1872, one page:
layout 35ms -> 19ms
render 84ms -> 41ms
total 120ms -> 60ms
Eviction ranks by use count rather than recency. Word frequency in prose is
Zipfian and stationary, so the words worth keeping are the ones used most, and
unlike recency this lets a document's own frequencies be seeded up front --
see prewarm_caches(). Two details keep the policy from costing more than it
saves, since get() runs once per word drawn:
- counting is O(1) with no reordering, because structures that reorder on
every hit measured 3-5ms/page slower than the hit rate they bought;
- eviction samples 8 entries and drops the least used of those, rather than
maintaining a global order.
Aging (halving all counts periodically) is on by default. Without it a font
size change drove the hit rate to 0% on a real access trace: every key was new
and the previous size's entries held counts nothing could beat.
Both caches are bounded, since the glyph bitmaps reach ~19MB over a long
session and the target is a 512MB Pi Zero 2. Defaults are 4MB of bitmaps and
8192 widths; configure_text_caches() tunes them. Cache size barely affects
speed (2MB is within 13% of unbounded) because a miss costs only one ~44us
rasterisation, so the bound can be set for memory, not throughput.
EreaderLayoutManager.prewarm_caches() counts the book's word frequencies and
preloads the most common ones, seeding each with its document frequency. This
moves that rasterisation to open time and cut misses by 27%, for ~15% faster
page turns at a one-off ~300ms cost. It is opt-in; nothing calls it yet.
Rendering is no longer bit-identical. PIL positions text at sub-pixel offsets,
so the cache buckets that phase, defaulting to 2 buckets per axis. Total ink
per page is unchanged and the mean pixel difference is 3.6/255 -- a fifth of
one step of a 16-level e-ink panel. subpixel_steps=4 halves that if wanted.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5
@@ -4,7 +4,14 @@ Concrete layer for the pyWebLayout library.
|
||||
This package contains concrete implementations that can be directly rendered.
|
||||
"""
|
||||
|
||||
from .text import Text, Line
|
||||
from .text import (
|
||||
Text,
|
||||
Line,
|
||||
configure_text_caches,
|
||||
clear_text_caches,
|
||||
text_cache_stats,
|
||||
prewarm_text_caches,
|
||||
)
|
||||
from .box import Box
|
||||
from .image import RenderableImage
|
||||
from .page import Page
|
||||
@@ -22,4 +29,8 @@ __all__ = [
|
||||
'Cell',
|
||||
'LinkText',
|
||||
'ButtonText',
|
||||
'configure_text_caches',
|
||||
'clear_text_caches',
|
||||
'text_cache_stats',
|
||||
'prewarm_text_caches',
|
||||
]
|
||||
|
||||
+290
-15
@@ -6,11 +6,204 @@ from pyWebLayout.style import Alignment, Font, TextDecoration
|
||||
from pyWebLayout.abstract import Word
|
||||
from pyWebLayout.abstract.inline import LinkedWord
|
||||
from pyWebLayout.abstract.functional import Link
|
||||
from PIL import ImageDraw
|
||||
from typing import Tuple, List, Optional
|
||||
from pyWebLayout.core.cache import UsageCache, SizedUsageCache
|
||||
from PIL import ImageDraw, ImageFont
|
||||
from typing import Tuple, List, Optional, Any, Dict
|
||||
import logging
|
||||
import math
|
||||
import numpy as np
|
||||
from abc import ABC, abstractmethod
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Text rendering caches
|
||||
#
|
||||
# A page re-measures and re-rasterises the same words constantly: measured over a
|
||||
# novel at 1404x1872, a page issues ~2800 width measurements and ~2500 glyph
|
||||
# rasterisations for fewer than 1000 distinct (font, string) pairs. Caching both
|
||||
# turns a ~225ms page into a ~30ms page. Both caches are bounded so that a long
|
||||
# reading session cannot grow without limit on a memory-constrained device.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# Word widths are small floats; 8192 entries costs well under 1MB and comfortably
|
||||
# spans the working set of several chapters at a couple of font sizes.
|
||||
DEFAULT_WIDTH_CACHE_ENTRIES = 8192
|
||||
|
||||
# Glyph bitmaps are the expensive ones: ~700 bytes each on average at 1404x1872,
|
||||
# so an unbounded cache reaches ~12MB after 40 pages. 4MB holds several pages'
|
||||
# worth of distinct words while leaving headroom on a 512MB Pi Zero 2.
|
||||
DEFAULT_GLYPH_CACHE_BYTES = 4 * 1024 * 1024
|
||||
|
||||
# PIL rasterises text at sub-pixel horizontal offsets, so a cache keyed only on
|
||||
# (font, string) would quantise every word to a whole pixel. Bucketing the
|
||||
# sub-pixel phase keeps that error negligible at the cost of more entries. 2 steps
|
||||
# holds the mean error to ~3.6/255 -- a fifth of one step of a 16-level e-ink
|
||||
# panel -- while keeping the cache four times smaller than 4 steps would.
|
||||
DEFAULT_GLYPH_SUBPIXEL_STEPS = 2
|
||||
|
||||
|
||||
def _glyph_entry_bytes(entry: Tuple[Any, Tuple[int, int]]) -> int:
|
||||
"""Approximate footprint of a cached (mask, offset) pair, in bytes."""
|
||||
mask = entry[0]
|
||||
try:
|
||||
width, height = mask.size
|
||||
except (AttributeError, TypeError, ValueError):
|
||||
return 0
|
||||
return width * height
|
||||
|
||||
|
||||
_width_cache: UsageCache = UsageCache(DEFAULT_WIDTH_CACHE_ENTRIES)
|
||||
_glyph_cache: SizedUsageCache = SizedUsageCache(DEFAULT_GLYPH_CACHE_BYTES, _glyph_entry_bytes)
|
||||
_glyph_subpixel_steps: int = DEFAULT_GLYPH_SUBPIXEL_STEPS
|
||||
|
||||
# Set to False the first time the fast rasterisation path is found to be
|
||||
# unavailable (e.g. a PIL build without the private ImageDraw internals it uses),
|
||||
# after which every Text falls back to ImageDraw.text().
|
||||
_glyph_fast_path_available: bool = True
|
||||
|
||||
|
||||
def configure_text_caches(width_entries: Optional[int] = None,
|
||||
glyph_bytes: Optional[int] = None,
|
||||
subpixel_steps: Optional[int] = None):
|
||||
"""
|
||||
Tune the text rendering caches.
|
||||
|
||||
Memory-constrained targets should shrink these; a desktop rendering many font
|
||||
sizes may benefit from raising them.
|
||||
|
||||
Args:
|
||||
width_entries: Maximum cached word-width measurements.
|
||||
glyph_bytes: Maximum total size of cached glyph bitmaps, in bytes.
|
||||
subpixel_steps: Sub-pixel phase buckets per axis. 1 disables sub-pixel
|
||||
positioning entirely (smallest cache, slightly softer text).
|
||||
"""
|
||||
global _glyph_subpixel_steps
|
||||
|
||||
if width_entries is not None:
|
||||
_width_cache.resize(width_entries)
|
||||
if glyph_bytes is not None:
|
||||
_glyph_cache.resize(glyph_bytes)
|
||||
if subpixel_steps is not None:
|
||||
if subpixel_steps <= 0:
|
||||
raise ValueError(f"subpixel_steps must be positive, got {subpixel_steps}")
|
||||
if subpixel_steps != _glyph_subpixel_steps:
|
||||
# Cached entries embed the phase bucket in their key.
|
||||
_glyph_cache.clear()
|
||||
_glyph_subpixel_steps = subpixel_steps
|
||||
|
||||
|
||||
def clear_text_caches():
|
||||
"""Drop all cached widths and glyph bitmaps."""
|
||||
_width_cache.clear()
|
||||
_glyph_cache.clear()
|
||||
|
||||
|
||||
def text_cache_stats() -> Dict[str, Any]:
|
||||
"""Occupancy and hit rates for both text caches, for tuning and diagnostics."""
|
||||
return {
|
||||
'width': _width_cache.stats(),
|
||||
'glyph': _glyph_cache.stats(),
|
||||
'glyph_subpixel_steps': _glyph_subpixel_steps,
|
||||
'glyph_fast_path': _glyph_fast_path_available,
|
||||
}
|
||||
|
||||
|
||||
def prewarm_text_caches(entries,
|
||||
draw: Optional[ImageDraw.ImageDraw] = None,
|
||||
budget_bytes: Optional[int] = None,
|
||||
max_words: Optional[int] = None) -> Tuple[int, int]:
|
||||
"""
|
||||
Preload the caches with a document's most frequent words.
|
||||
|
||||
A document states its own access distribution up front: the words it uses most
|
||||
are the words every page will draw. Rasterising them once at open time moves
|
||||
that work off the page-turn path, and seeding each entry with its document
|
||||
frequency puts it in the right place in the eviction order immediately, rather
|
||||
than after the cache has learned it.
|
||||
|
||||
This depends on eviction ranking by use count. Under recency eviction the
|
||||
preloaded entries would be discarded by the first page of unfamiliar text; under
|
||||
usage ranking a word occurring 4000 times outranks anything met while scanning
|
||||
and stays resident. Measured over a 50-page trace, preloading cut misses by 27%
|
||||
with usage ranking against 12% with recency.
|
||||
|
||||
Args:
|
||||
entries: Iterable of ``(font, text, colour, frequency)``, where `font` is a
|
||||
PIL font object, `colour` the fill the text will be drawn in, and
|
||||
`frequency` the number of times the word occurs in the document.
|
||||
draw: An ImageDraw sharing the page's mode, used to resolve ink and font
|
||||
mode. A scratch RGBA context is used if omitted.
|
||||
budget_bytes: Cap on bytes to preload. Defaults to half the glyph budget so
|
||||
that live rendering keeps room to cache what preloading missed.
|
||||
max_words: Cap on distinct words to preload, before sub-pixel variants.
|
||||
|
||||
Returns:
|
||||
Tuple of (words preloaded, bytes preloaded).
|
||||
"""
|
||||
if not _glyph_fast_path_available:
|
||||
return 0, 0
|
||||
|
||||
if draw is None:
|
||||
from PIL import Image
|
||||
draw = ImageDraw.Draw(Image.new('RGBA', (1, 1)))
|
||||
|
||||
if budget_bytes is None:
|
||||
budget_bytes = _glyph_cache.max_bytes // 2
|
||||
budget_bytes = min(budget_bytes, _glyph_cache.max_bytes)
|
||||
|
||||
ranked = sorted(entries, key=lambda e: -e[3])
|
||||
if max_words is not None:
|
||||
ranked = ranked[:max_words]
|
||||
|
||||
steps = _glyph_subpixel_steps
|
||||
mode = draw.fontmode
|
||||
draw_mode = draw.mode
|
||||
ink_cache: Dict[Any, Any] = {}
|
||||
words = 0
|
||||
used = 0
|
||||
|
||||
for font, text, colour, frequency in ranked:
|
||||
if frequency <= 1 or used >= budget_bytes:
|
||||
break
|
||||
if not isinstance(font, ImageFont.FreeTypeFont):
|
||||
continue
|
||||
|
||||
try:
|
||||
ink = ink_cache.get(colour)
|
||||
if ink is None:
|
||||
ink, _ = draw._getink(colour)
|
||||
if ink is None:
|
||||
continue
|
||||
ink_cache[colour] = ink
|
||||
|
||||
# Measuring is cheap and every layout pass needs it.
|
||||
_width_cache.put((font, text, draw_mode),
|
||||
draw.textlength(text, font=font), count=frequency)
|
||||
|
||||
# Words land on arbitrary sub-pixel offsets, so cover every horizontal
|
||||
# phase. Baselines are whole pixels, so only phase 0 is needed
|
||||
# vertically.
|
||||
for x_bucket in range(steps):
|
||||
entry = font.getmask2(text, mode, anchor="ls", ink=ink,
|
||||
start=(x_bucket / steps, 0.0))
|
||||
_glyph_cache.put((font, text, mode, ink, x_bucket, 0), entry,
|
||||
count=frequency)
|
||||
used += _glyph_entry_bytes(entry)
|
||||
words += 1
|
||||
|
||||
except AttributeError:
|
||||
logger.warning("Glyph cache unavailable for this Pillow build; "
|
||||
"skipping prewarm.", exc_info=True)
|
||||
return words, used
|
||||
except (TypeError, ValueError):
|
||||
continue
|
||||
|
||||
logger.debug("Prewarmed %d words (%.2fMB) into the text caches",
|
||||
words, used / 1e6)
|
||||
return words, used
|
||||
|
||||
|
||||
class AlignmentHandler(ABC):
|
||||
"""
|
||||
@@ -205,9 +398,19 @@ class Text(Renderable, Queriable):
|
||||
|
||||
def _calculate_dimensions(self):
|
||||
"""Calculate the width and height of the text based on the font metrics"""
|
||||
# Get the size using PIL's text size functionality
|
||||
# Measuring a word costs a FreeType shaping pass, and the same words recur
|
||||
# constantly within a document, so results are cached per (font, string).
|
||||
# The draw's image mode is part of the key because PIL derives advance
|
||||
# widths differently for bilevel ("1") targets.
|
||||
font = self._style.font
|
||||
self._width = self._draw.textlength(self._text, font=font)
|
||||
key = (font, self._text, self._draw.mode)
|
||||
|
||||
width = _width_cache.get(key)
|
||||
if width is None:
|
||||
width = self._draw.textlength(self._text, font=font)
|
||||
_width_cache.put(key, width)
|
||||
self._width = width
|
||||
|
||||
ascent, descent = font.getmetrics()
|
||||
self._ascent = ascent
|
||||
self._middle_y = ascent - descent / 2
|
||||
@@ -343,23 +546,95 @@ class Text(Renderable, Queriable):
|
||||
A PIL Image containing the rendered text
|
||||
"""
|
||||
|
||||
style = self._style
|
||||
|
||||
# Draw the text background if specified
|
||||
if self._style.background and self._style.background[3] > 0: # If alpha > 0
|
||||
self._draw.rectangle([self._origin, self._origin +
|
||||
self._size], fill=self._style.background)
|
||||
if style.background and style.background[3] > 0: # If alpha > 0
|
||||
self._draw.rectangle([tuple(self._origin), tuple(self._origin + self.size)],
|
||||
fill=style.background)
|
||||
|
||||
# Draw the text using baseline as anchor point ("ls" = left-baseline)
|
||||
# This ensures the origin represents the baseline, not the top-left
|
||||
self._draw.text(
|
||||
(self.origin[0],
|
||||
self._origin[1]),
|
||||
self._text,
|
||||
font=self._style.font,
|
||||
fill=self._style.colour,
|
||||
anchor="ls")
|
||||
if not self._render_from_glyph_cache(style):
|
||||
self._draw.text(
|
||||
(self.origin[0],
|
||||
self._origin[1]),
|
||||
self._text,
|
||||
font=style.font,
|
||||
fill=style.colour,
|
||||
anchor="ls")
|
||||
|
||||
# Apply any text decorations with knowledge of next text
|
||||
self._apply_decoration(next_text, spacing)
|
||||
if style.decoration != TextDecoration.NONE:
|
||||
self._apply_decoration(next_text, spacing)
|
||||
|
||||
def _render_from_glyph_cache(self, style) -> bool:
|
||||
"""
|
||||
Blit this word from the cached glyph bitmap.
|
||||
|
||||
Rasterising a word is the single most expensive step in drawing a page, and
|
||||
the same words recur constantly, so the bitmap PIL would produce is cached
|
||||
and blitted directly. This reproduces what ImageDraw.text() does internally
|
||||
(getmask2 followed by draw_bitmap) minus the per-call setup.
|
||||
|
||||
Returns:
|
||||
True if the word was drawn. False means the caller must fall back to
|
||||
ImageDraw.text().
|
||||
"""
|
||||
global _glyph_fast_path_available
|
||||
|
||||
if not _glyph_fast_path_available:
|
||||
return False
|
||||
|
||||
draw = self._draw
|
||||
font = style.font
|
||||
|
||||
# Bitmap and other non-FreeType fonts do not expose getmask2's anchor and
|
||||
# sub-pixel arguments; let PIL handle them.
|
||||
if not isinstance(font, ImageFont.FreeTypeFont):
|
||||
return False
|
||||
|
||||
try:
|
||||
ink, _ = draw._getink(style.colour)
|
||||
if ink is None:
|
||||
return False
|
||||
|
||||
# floor() rather than modf() so the fraction is always in [0, 1),
|
||||
# keeping bucket indices non-negative for negative coordinates.
|
||||
x = float(self._origin[0])
|
||||
y = float(self._origin[1])
|
||||
x_whole = math.floor(x)
|
||||
y_whole = math.floor(y)
|
||||
|
||||
steps = _glyph_subpixel_steps
|
||||
x_bucket = int((x - x_whole) * steps)
|
||||
y_bucket = int((y - y_whole) * steps)
|
||||
|
||||
mode = draw.fontmode
|
||||
key = (font, self._text, mode, ink, x_bucket, y_bucket)
|
||||
|
||||
entry = _glyph_cache.get(key)
|
||||
if entry is None:
|
||||
entry = font.getmask2(
|
||||
self._text, mode, anchor="ls", ink=ink,
|
||||
start=(x_bucket / steps, y_bucket / steps))
|
||||
_glyph_cache.put(key, entry)
|
||||
|
||||
mask, offset = entry
|
||||
draw.draw.draw_bitmap((x_whole + offset[0], y_whole + offset[1]), mask, ink)
|
||||
return True
|
||||
|
||||
except AttributeError:
|
||||
# A PIL build without the internals this path relies on. Stop trying.
|
||||
logger.warning(
|
||||
"Glyph cache unavailable for this Pillow build; falling back to "
|
||||
"ImageDraw.text() for all text rendering.", exc_info=True)
|
||||
_glyph_fast_path_available = False
|
||||
return False
|
||||
except (TypeError, ValueError):
|
||||
# This particular colour/mode combination is not supported by the fast
|
||||
# path (e.g. an ink PIL cannot resolve). Others may still be.
|
||||
return False
|
||||
|
||||
|
||||
class Line(Box):
|
||||
|
||||
@@ -0,0 +1,343 @@
|
||||
"""
|
||||
Bounded usage-ranked caches for the text rendering hot path.
|
||||
|
||||
Laying out and rasterising a page re-measures and re-draws the same words over and
|
||||
over: a typical page issues ~2800 width measurements and ~2500 glyph rasterisations
|
||||
for fewer than 1000 distinct (font, string) pairs. Caching both collapses that work,
|
||||
but an unbounded cache is not an option on a memory-constrained target such as a
|
||||
Raspberry Pi Zero 2, where the rasterised bitmaps reach ~19MB over a long session.
|
||||
|
||||
Eviction is by **usage count**, not recency. Word frequency in prose is Zipfian and
|
||||
stationary -- a small set of words ("the", "and", "of") accounts for most tokens on
|
||||
every page, and that set barely shifts as the reader advances -- so the words worth
|
||||
keeping are exactly the ones used most.
|
||||
|
||||
Two design choices keep this from costing more than it saves, because `get` runs
|
||||
once per word drawn (~2500 times per page):
|
||||
|
||||
* **Counting is O(1) with no reordering.** Each entry carries its own use counter,
|
||||
bumped in place. Ranking structures that reorder on every hit (a frequency-bucket
|
||||
LFU, or an LRU's linked-list splice) were measured 3-5ms per page slower than the
|
||||
hit rate they buy is worth.
|
||||
* **Eviction samples rather than sorts.** Finding the globally least-used entry
|
||||
would need a heap kept current on every hit. Instead a small random sample is
|
||||
drawn and the least-used member of it evicted, the same approximation Redis uses
|
||||
for its LFU policy. With the default sample size the evicted entry is very
|
||||
likely to be in the bottom few percent, which is all that matters here.
|
||||
|
||||
Both are single-threaded by design; the rendering path holds the GIL throughout and
|
||||
adding locking would cost more than it protects.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import random
|
||||
from typing import Any, Callable, Dict, Generic, Hashable, List, Optional, TypeVar
|
||||
|
||||
K = TypeVar('K', bound=Hashable)
|
||||
V = TypeVar('V')
|
||||
|
||||
# Entries examined per eviction. Larger samples approximate true least-frequently-used
|
||||
# more closely at linear cost; 8 puts the victim in the bottom ~15% of entries, which
|
||||
# is ample when the alternative is a rasterisation that costs ~60us either way.
|
||||
DEFAULT_EVICTION_SAMPLE = 8
|
||||
|
||||
# Halving every entry's use count after this many insertions keeps the cache
|
||||
# responsive to a change of working set. Without it, entries that were hot long ago
|
||||
# retain counts a newly-hot entry cannot beat and are never evicted -- the classic
|
||||
# failure of pure frequency eviction. Measured on a real access trace, a font-size
|
||||
# change drove hit rate to 0% without aging and left it unchanged with it.
|
||||
DEFAULT_AGING_INTERVAL = 10000
|
||||
|
||||
# Index of each field in an entry. Entries are plain lists rather than tuples or
|
||||
# objects so the counter can be bumped in place, without rehashing the key.
|
||||
_VALUE = 0
|
||||
_COUNT = 1
|
||||
_SLOT = 2
|
||||
|
||||
|
||||
class _UsageRanked(Generic[K, V]):
|
||||
"""
|
||||
Shared usage-count bookkeeping for the caches below.
|
||||
|
||||
Entries live in a dict for lookup and, in parallel, in a flat list that makes
|
||||
uniform random sampling possible. Each entry records its own index in that list
|
||||
so removal can swap in the tail element and stay O(1).
|
||||
|
||||
Subclasses supply the bound by implementing :meth:`_over_budget` and the
|
||||
accounting hooks :meth:`_record_add` / :meth:`_record_remove`.
|
||||
"""
|
||||
|
||||
def __init__(self,
|
||||
aging_interval: Optional[int] = DEFAULT_AGING_INTERVAL,
|
||||
eviction_sample: int = DEFAULT_EVICTION_SAMPLE):
|
||||
if aging_interval is not None and aging_interval <= 0:
|
||||
raise ValueError(f"aging_interval must be positive, got {aging_interval}")
|
||||
if eviction_sample <= 0:
|
||||
raise ValueError(f"eviction_sample must be positive, got {eviction_sample}")
|
||||
|
||||
self._aging_interval = aging_interval
|
||||
self._eviction_sample = eviction_sample
|
||||
|
||||
self._entries: Dict[K, List[Any]] = {}
|
||||
self._slots: List[K] = []
|
||||
self._randrange = random.randrange
|
||||
|
||||
self._inserts_since_aging = 0
|
||||
self._hits = 0
|
||||
self._misses = 0
|
||||
self._evictions = 0
|
||||
self._agings = 0
|
||||
|
||||
# -- subclass hooks ----------------------------------------------------
|
||||
|
||||
def _over_budget(self) -> bool:
|
||||
raise NotImplementedError
|
||||
|
||||
def _record_add(self, key: K, value: V):
|
||||
"""Account for a value entering the cache."""
|
||||
|
||||
def _record_remove(self, key: K):
|
||||
"""Account for a value leaving the cache."""
|
||||
|
||||
# -- core operations ---------------------------------------------------
|
||||
|
||||
def get(self, key: K) -> Optional[V]:
|
||||
"""Return the cached value for `key`, or None, counting the use."""
|
||||
entry = self._entries.get(key)
|
||||
if entry is None:
|
||||
self._misses += 1
|
||||
return None
|
||||
entry[_COUNT] += 1
|
||||
self._hits += 1
|
||||
return entry[_VALUE]
|
||||
|
||||
def _add_new(self, key: K, value: V):
|
||||
"""Insert a key not currently present."""
|
||||
# New entries start at 1 rather than 0 so that a single reuse is enough to
|
||||
# outrank an entry that has never been touched since the last aging pass.
|
||||
self._entries[key] = [value, 1, len(self._slots)]
|
||||
self._slots.append(key)
|
||||
self._record_add(key, value)
|
||||
|
||||
def _remove(self, key: K):
|
||||
"""Remove a key outright, keeping the sampling list dense."""
|
||||
entry = self._entries.pop(key)
|
||||
slot = entry[_SLOT]
|
||||
last = self._slots.pop()
|
||||
if last != key:
|
||||
self._slots[slot] = last
|
||||
self._entries[last][_SLOT] = slot
|
||||
self._record_remove(key)
|
||||
|
||||
def _evict_one(self) -> bool:
|
||||
"""Evict the least-used member of a random sample. False if empty."""
|
||||
count = len(self._slots)
|
||||
if not count:
|
||||
return False
|
||||
|
||||
if count <= self._eviction_sample:
|
||||
victim = min(self._slots, key=lambda k: self._entries[k][_COUNT])
|
||||
else:
|
||||
randrange = self._randrange
|
||||
entries = self._entries
|
||||
slots = self._slots
|
||||
victim = slots[randrange(count)]
|
||||
best = entries[victim][_COUNT]
|
||||
for _ in range(self._eviction_sample - 1):
|
||||
candidate = slots[randrange(count)]
|
||||
score = entries[candidate][_COUNT]
|
||||
if score < best:
|
||||
victim, best = candidate, score
|
||||
|
||||
self._remove(victim)
|
||||
self._evictions += 1
|
||||
return True
|
||||
|
||||
def _evict_to_budget(self):
|
||||
while self._over_budget():
|
||||
if not self._evict_one():
|
||||
break
|
||||
|
||||
def _maybe_age(self):
|
||||
"""Halve every use count once the aging interval has elapsed."""
|
||||
if self._aging_interval is None:
|
||||
return
|
||||
self._inserts_since_aging += 1
|
||||
if self._inserts_since_aging < self._aging_interval:
|
||||
return
|
||||
|
||||
self._inserts_since_aging = 0
|
||||
self._agings += 1
|
||||
for entry in self._entries.values():
|
||||
entry[_COUNT] = entry[_COUNT] // 2 or 1
|
||||
|
||||
def clear(self):
|
||||
"""Drop all entries. Counters are preserved."""
|
||||
self._entries.clear()
|
||||
self._slots.clear()
|
||||
self._inserts_since_aging = 0
|
||||
|
||||
def _base_stats(self) -> Dict[str, Any]:
|
||||
total = self._hits + self._misses
|
||||
return {
|
||||
'entries': len(self._entries),
|
||||
'hits': self._hits,
|
||||
'misses': self._misses,
|
||||
'evictions': self._evictions,
|
||||
'agings': self._agings,
|
||||
'hit_rate': (self._hits / total) if total else 0.0,
|
||||
}
|
||||
|
||||
def __len__(self) -> int:
|
||||
return len(self._entries)
|
||||
|
||||
def __contains__(self, key: object) -> bool:
|
||||
return key in self._entries
|
||||
|
||||
|
||||
class UsageCache(_UsageRanked[K, V]):
|
||||
"""
|
||||
Usage-ranked cache bounded by number of entries.
|
||||
|
||||
Args:
|
||||
max_entries: Maximum number of entries to retain. Must be positive.
|
||||
aging_interval: Insertions between halving all use counts, or None to
|
||||
disable aging. See :data:`DEFAULT_AGING_INTERVAL`.
|
||||
eviction_sample: Entries sampled per eviction.
|
||||
"""
|
||||
|
||||
def __init__(self, max_entries: int,
|
||||
aging_interval: Optional[int] = DEFAULT_AGING_INTERVAL,
|
||||
eviction_sample: int = DEFAULT_EVICTION_SAMPLE):
|
||||
if max_entries <= 0:
|
||||
raise ValueError(f"max_entries must be positive, got {max_entries}")
|
||||
super().__init__(aging_interval, eviction_sample)
|
||||
self._max_entries = max_entries
|
||||
|
||||
def _over_budget(self) -> bool:
|
||||
return len(self._entries) > self._max_entries
|
||||
|
||||
def put(self, key: K, value: V, count: int = 1):
|
||||
"""
|
||||
Insert `value`, evicting the least-used entries past the bound.
|
||||
|
||||
Args:
|
||||
count: Initial use count. Pass a document-derived frequency to rank a
|
||||
preloaded entry ahead of words that have not been seen yet.
|
||||
"""
|
||||
existing = self._entries.get(key)
|
||||
if existing is not None:
|
||||
existing[_VALUE] = value
|
||||
existing[_COUNT] += 1
|
||||
return
|
||||
self._add_new(key, value)
|
||||
if count > 1:
|
||||
self._entries[key][_COUNT] = count
|
||||
self._evict_to_budget()
|
||||
self._maybe_age()
|
||||
|
||||
@property
|
||||
def max_entries(self) -> int:
|
||||
return self._max_entries
|
||||
|
||||
def resize(self, max_entries: int):
|
||||
"""Change the bound, evicting immediately if the cache now overflows."""
|
||||
if max_entries <= 0:
|
||||
raise ValueError(f"max_entries must be positive, got {max_entries}")
|
||||
self._max_entries = max_entries
|
||||
self._evict_to_budget()
|
||||
|
||||
def stats(self) -> Dict[str, Any]:
|
||||
"""Hit/miss/eviction counters and current occupancy."""
|
||||
stats = self._base_stats()
|
||||
stats['max_entries'] = self._max_entries
|
||||
return stats
|
||||
|
||||
|
||||
class SizedUsageCache(_UsageRanked[K, V]):
|
||||
"""
|
||||
Usage-ranked cache bounded by the total size of its values.
|
||||
|
||||
Args:
|
||||
max_bytes: Maximum total value size to retain. Must be positive.
|
||||
sizer: Returns the size in bytes of a value. Called once per insertion.
|
||||
aging_interval: Insertions between halving all use counts, or None to
|
||||
disable aging. See :data:`DEFAULT_AGING_INTERVAL`.
|
||||
eviction_sample: Entries sampled per eviction.
|
||||
|
||||
A value larger than `max_bytes` on its own is returned to the caller but not
|
||||
retained, so that one oversized entry cannot flush the whole cache.
|
||||
"""
|
||||
|
||||
def __init__(self, max_bytes: int, sizer: Callable[[V], int],
|
||||
aging_interval: Optional[int] = DEFAULT_AGING_INTERVAL,
|
||||
eviction_sample: int = DEFAULT_EVICTION_SAMPLE):
|
||||
if max_bytes <= 0:
|
||||
raise ValueError(f"max_bytes must be positive, got {max_bytes}")
|
||||
super().__init__(aging_interval, eviction_sample)
|
||||
self._max_bytes = max_bytes
|
||||
self._sizer = sizer
|
||||
self._sizes: Dict[K, int] = {}
|
||||
self._total_bytes = 0
|
||||
|
||||
def _over_budget(self) -> bool:
|
||||
return self._total_bytes > self._max_bytes
|
||||
|
||||
def _record_add(self, key: K, value: V):
|
||||
size = self._sizer(value)
|
||||
self._sizes[key] = size
|
||||
self._total_bytes += size
|
||||
|
||||
def _record_remove(self, key: K):
|
||||
self._total_bytes -= self._sizes.pop(key)
|
||||
|
||||
def put(self, key: K, value: V, count: int = 1):
|
||||
"""
|
||||
Insert `value`, evicting the least-used entries past the bound.
|
||||
|
||||
Args:
|
||||
count: Initial use count. Pass a document-derived frequency to rank a
|
||||
preloaded entry ahead of words that have not been seen yet.
|
||||
"""
|
||||
if key in self._entries:
|
||||
# Re-measure: the replacement may be a different size.
|
||||
self._remove(key)
|
||||
|
||||
if self._sizer(value) > self._max_bytes:
|
||||
# Too large to ever retain; skip rather than flush everything for it.
|
||||
return
|
||||
|
||||
self._add_new(key, value)
|
||||
if count > 1:
|
||||
self._entries[key][_COUNT] = count
|
||||
self._evict_to_budget()
|
||||
self._maybe_age()
|
||||
|
||||
@property
|
||||
def max_bytes(self) -> int:
|
||||
return self._max_bytes
|
||||
|
||||
@property
|
||||
def total_bytes(self) -> int:
|
||||
return self._total_bytes
|
||||
|
||||
def resize(self, max_bytes: int):
|
||||
"""Change the bound, evicting immediately if the cache now overflows."""
|
||||
if max_bytes <= 0:
|
||||
raise ValueError(f"max_bytes must be positive, got {max_bytes}")
|
||||
self._max_bytes = max_bytes
|
||||
self._evict_to_budget()
|
||||
|
||||
def clear(self):
|
||||
"""Drop all entries. Counters are preserved."""
|
||||
super().clear()
|
||||
self._sizes.clear()
|
||||
self._total_bytes = 0
|
||||
|
||||
def stats(self) -> Dict[str, Any]:
|
||||
"""Hit/miss/eviction counters and current occupancy."""
|
||||
stats = self._base_stats()
|
||||
stats['total_bytes'] = self._total_bytes
|
||||
stats['max_bytes'] = self._max_bytes
|
||||
return stats
|
||||
@@ -219,6 +219,67 @@ class EreaderLayoutManager:
|
||||
self.chapter_changed_callback: Optional[Callable[[
|
||||
Optional[ChapterInfo]], None]] = None
|
||||
|
||||
def prewarm_caches(self, max_words: int = 2000,
|
||||
budget_bytes: Optional[int] = None) -> Tuple[int, int]:
|
||||
"""
|
||||
Preload the text caches with this document's most frequent words.
|
||||
|
||||
Counts how often each word occurs in the book and rasterises the most
|
||||
common ones ahead of time, so that the work lands at open time rather than
|
||||
on the first page turns. Entries are seeded with their document frequency,
|
||||
which is what keeps them resident under usage-ranked eviction.
|
||||
|
||||
Safe to call again after a font change; the fonts differ, so the new
|
||||
entries simply take their place in the eviction order alongside the old.
|
||||
|
||||
Args:
|
||||
max_words: Maximum distinct words to preload.
|
||||
budget_bytes: Bytes of glyph cache to fill. Defaults to half the budget.
|
||||
|
||||
Returns:
|
||||
Tuple of (words preloaded, bytes preloaded).
|
||||
"""
|
||||
from collections import Counter
|
||||
from pyWebLayout.concrete.text import prewarm_text_caches
|
||||
from .ereader_layout import FontScaler
|
||||
|
||||
override = getattr(self.renderer.layouter, 'font_family_override', None)
|
||||
|
||||
# Count by (style, text): the same word in a heading and in body text is a
|
||||
# different rasterisation, and both are worth counting separately.
|
||||
counts: Dict[Tuple[int, str], int] = Counter()
|
||||
styles: Dict[int, Any] = {}
|
||||
for block in self.blocks:
|
||||
words = getattr(block, '_words', None)
|
||||
if not words:
|
||||
continue
|
||||
for word in words:
|
||||
style = word.style
|
||||
if style is None:
|
||||
continue
|
||||
key = id(style)
|
||||
styles.setdefault(key, style)
|
||||
counts[(key, word.text)] += 1
|
||||
|
||||
# Resolve each distinct style once through the same scaling the layouter
|
||||
# applies, so the preloaded keys match what rendering will look up.
|
||||
scaled: Dict[int, Any] = {}
|
||||
for key, style in styles.items():
|
||||
try:
|
||||
scaled[key] = FontScaler.scale_font(style, self.font_scale, override)
|
||||
except Exception:
|
||||
continue
|
||||
|
||||
entries = []
|
||||
for (style_key, text), count in counts.items():
|
||||
font = scaled.get(style_key)
|
||||
if font is None:
|
||||
continue
|
||||
entries.append((font.font, text, font.colour, count))
|
||||
|
||||
return prewarm_text_caches(entries, budget_bytes=budget_bytes,
|
||||
max_words=max_words)
|
||||
|
||||
def set_position_changed_callback(
|
||||
self, callback: Callable[[RenderingPosition], None]):
|
||||
"""Set callback for position changes"""
|
||||
|
||||
@@ -0,0 +1,244 @@
|
||||
"""
|
||||
Unit tests for the bounded usage-ranked caches.
|
||||
|
||||
Covers the guarantees the text rendering path depends on: that the bounds are never
|
||||
exceeded, that eviction prefers the least-used entries, that aging lets a new
|
||||
working set displace an old one, and that document-frequency seeding survives a
|
||||
scan of unfamiliar keys.
|
||||
"""
|
||||
|
||||
import unittest
|
||||
|
||||
from pyWebLayout.core.cache import (
|
||||
UsageCache,
|
||||
SizedUsageCache,
|
||||
DEFAULT_AGING_INTERVAL,
|
||||
)
|
||||
|
||||
|
||||
class TestUsageCache(unittest.TestCase):
|
||||
"""Entry-count-bounded cache."""
|
||||
|
||||
def test_rejects_invalid_bounds(self):
|
||||
for bad in (0, -1):
|
||||
with self.assertRaises(ValueError):
|
||||
UsageCache(bad)
|
||||
with self.assertRaises(ValueError):
|
||||
UsageCache(4, aging_interval=0)
|
||||
with self.assertRaises(ValueError):
|
||||
UsageCache(4, eviction_sample=0)
|
||||
|
||||
def test_stores_and_returns_values(self):
|
||||
cache = UsageCache(4)
|
||||
cache.put('a', 1)
|
||||
self.assertEqual(cache.get('a'), 1)
|
||||
self.assertIsNone(cache.get('missing'))
|
||||
self.assertIn('a', cache)
|
||||
self.assertEqual(len(cache), 1)
|
||||
|
||||
def test_never_exceeds_max_entries(self):
|
||||
cache = UsageCache(10)
|
||||
for i in range(500):
|
||||
cache.put(i, i)
|
||||
self.assertLessEqual(len(cache), 10)
|
||||
self.assertEqual(cache.stats()['entries'], 10)
|
||||
|
||||
def test_evicts_least_used(self):
|
||||
# One hot key among many cold ones must survive a long cold scan. The
|
||||
# sample is smaller than the cache, so this is probabilistic in principle;
|
||||
# a hot key's count is far enough above the rest to make it reliable.
|
||||
cache = UsageCache(20, eviction_sample=8)
|
||||
cache.put('hot', 'value')
|
||||
for _ in range(200):
|
||||
cache.get('hot')
|
||||
for i in range(400):
|
||||
cache.put(f'cold{i}', i)
|
||||
cache.get('hot')
|
||||
self.assertEqual(cache.get('hot'), 'value')
|
||||
|
||||
def test_repeated_put_does_not_duplicate(self):
|
||||
cache = UsageCache(10)
|
||||
for _ in range(50):
|
||||
cache.put('a', 1)
|
||||
self.assertEqual(len(cache), 1)
|
||||
|
||||
def test_put_updates_existing_value(self):
|
||||
cache = UsageCache(10)
|
||||
cache.put('a', 1)
|
||||
cache.put('a', 2)
|
||||
self.assertEqual(cache.get('a'), 2)
|
||||
|
||||
def test_seeded_count_outranks_fresh_entries(self):
|
||||
"""A document-frequency seed must survive a scan of unseen keys."""
|
||||
cache = UsageCache(20, eviction_sample=8)
|
||||
cache.put('frequent', 'value', count=5000)
|
||||
for i in range(400):
|
||||
cache.put(f'new{i}', i)
|
||||
self.assertEqual(cache.get('frequent'), 'value')
|
||||
|
||||
def test_aging_lets_a_new_working_set_take_over(self):
|
||||
"""Without aging, stale high counts lock the cache permanently."""
|
||||
cache = UsageCache(20, aging_interval=50, eviction_sample=8)
|
||||
for i in range(20):
|
||||
cache.put(f'old{i}', i, count=10000)
|
||||
|
||||
# A completely different working set, each key used a few times.
|
||||
for round_ in range(60):
|
||||
for i in range(10):
|
||||
key = f'new{i}'
|
||||
if cache.get(key) is None:
|
||||
cache.put(key, i)
|
||||
|
||||
survivors = sum(1 for i in range(10) if f'new{i}' in cache)
|
||||
self.assertGreater(survivors, 0,
|
||||
"aging should let the new working set displace the old")
|
||||
self.assertGreater(cache.stats()['agings'], 0)
|
||||
|
||||
def test_aging_can_be_disabled(self):
|
||||
cache = UsageCache(10, aging_interval=None)
|
||||
for i in range(100):
|
||||
cache.put(i, i)
|
||||
self.assertEqual(cache.stats()['agings'], 0)
|
||||
|
||||
def test_resize_evicts_immediately(self):
|
||||
cache = UsageCache(100)
|
||||
for i in range(100):
|
||||
cache.put(i, i)
|
||||
cache.resize(10)
|
||||
self.assertEqual(len(cache), 10)
|
||||
with self.assertRaises(ValueError):
|
||||
cache.resize(0)
|
||||
|
||||
def test_clear_empties_but_keeps_counters(self):
|
||||
cache = UsageCache(10)
|
||||
cache.put('a', 1)
|
||||
cache.get('a')
|
||||
cache.clear()
|
||||
self.assertEqual(len(cache), 0)
|
||||
self.assertNotIn('a', cache)
|
||||
self.assertEqual(cache.stats()['hits'], 1)
|
||||
|
||||
def test_stats_track_hits_and_misses(self):
|
||||
cache = UsageCache(10)
|
||||
cache.put('a', 1)
|
||||
cache.get('a')
|
||||
cache.get('a')
|
||||
cache.get('b')
|
||||
stats = cache.stats()
|
||||
self.assertEqual(stats['hits'], 2)
|
||||
self.assertEqual(stats['misses'], 1)
|
||||
self.assertAlmostEqual(stats['hit_rate'], 2 / 3)
|
||||
self.assertEqual(stats['max_entries'], 10)
|
||||
|
||||
def test_internal_slot_list_stays_consistent(self):
|
||||
"""Eviction swaps the tail into the freed slot; indices must stay valid."""
|
||||
cache = UsageCache(8)
|
||||
for i in range(300):
|
||||
cache.put(i, i)
|
||||
for key in list(cache._entries):
|
||||
self.assertEqual(cache._slots[cache._entries[key][2]], key)
|
||||
self.assertEqual(len(cache._slots), len(cache._entries))
|
||||
|
||||
|
||||
class TestSizedUsageCache(unittest.TestCase):
|
||||
"""Byte-bounded cache, as used for glyph bitmaps."""
|
||||
|
||||
@staticmethod
|
||||
def sizer(value):
|
||||
return value
|
||||
|
||||
def test_rejects_invalid_bounds(self):
|
||||
for bad in (0, -1):
|
||||
with self.assertRaises(ValueError):
|
||||
SizedUsageCache(bad, self.sizer)
|
||||
|
||||
def test_never_exceeds_max_bytes(self):
|
||||
cache = SizedUsageCache(1000, self.sizer)
|
||||
for i in range(500):
|
||||
cache.put(i, 100)
|
||||
self.assertLessEqual(cache.total_bytes, 1000)
|
||||
|
||||
def test_tracks_total_bytes(self):
|
||||
cache = SizedUsageCache(1000, self.sizer)
|
||||
cache.put('a', 100)
|
||||
cache.put('b', 250)
|
||||
self.assertEqual(cache.total_bytes, 350)
|
||||
|
||||
def test_oversized_value_is_not_retained(self):
|
||||
"""One huge entry must not flush everything else out."""
|
||||
cache = SizedUsageCache(1000, self.sizer)
|
||||
cache.put('small', 100)
|
||||
cache.put('huge', 5000)
|
||||
self.assertNotIn('huge', cache)
|
||||
self.assertIn('small', cache)
|
||||
self.assertEqual(cache.total_bytes, 100)
|
||||
|
||||
def test_replacing_a_value_remeasures_it(self):
|
||||
cache = SizedUsageCache(1000, self.sizer)
|
||||
cache.put('a', 100)
|
||||
cache.put('a', 300)
|
||||
self.assertEqual(cache.total_bytes, 300)
|
||||
self.assertEqual(len(cache), 1)
|
||||
|
||||
def test_evicts_least_used(self):
|
||||
cache = SizedUsageCache(1000, self.sizer, eviction_sample=8)
|
||||
cache.put('hot', 100)
|
||||
for _ in range(200):
|
||||
cache.get('hot')
|
||||
for i in range(400):
|
||||
cache.put(f'cold{i}', 100)
|
||||
cache.get('hot')
|
||||
self.assertIn('hot', cache)
|
||||
|
||||
def test_seeded_count_outranks_fresh_entries(self):
|
||||
cache = SizedUsageCache(1000, self.sizer, eviction_sample=8)
|
||||
cache.put('frequent', 100, count=5000)
|
||||
for i in range(400):
|
||||
cache.put(f'new{i}', 100)
|
||||
self.assertIn('frequent', cache)
|
||||
|
||||
def test_resize_evicts_immediately(self):
|
||||
cache = SizedUsageCache(10000, self.sizer)
|
||||
for i in range(100):
|
||||
cache.put(i, 100)
|
||||
cache.resize(500)
|
||||
self.assertLessEqual(cache.total_bytes, 500)
|
||||
|
||||
def test_clear_resets_byte_accounting(self):
|
||||
cache = SizedUsageCache(1000, self.sizer)
|
||||
cache.put('a', 100)
|
||||
cache.clear()
|
||||
self.assertEqual(cache.total_bytes, 0)
|
||||
self.assertEqual(len(cache), 0)
|
||||
|
||||
def test_stats_report_bounds(self):
|
||||
cache = SizedUsageCache(1000, self.sizer)
|
||||
cache.put('a', 100)
|
||||
stats = cache.stats()
|
||||
self.assertEqual(stats['total_bytes'], 100)
|
||||
self.assertEqual(stats['max_bytes'], 1000)
|
||||
self.assertEqual(stats['entries'], 1)
|
||||
|
||||
def test_bookkeeping_stays_consistent_under_churn(self):
|
||||
"""Byte total and slot list must not drift over many evictions."""
|
||||
cache = SizedUsageCache(2000, self.sizer, aging_interval=97)
|
||||
for i in range(2000):
|
||||
cache.put(i, (i % 7 + 1) * 50)
|
||||
if i % 3 == 0:
|
||||
cache.get(i)
|
||||
self.assertEqual(cache.total_bytes,
|
||||
sum(cache._sizes[k] for k in cache._entries))
|
||||
self.assertEqual(len(cache._slots), len(cache._entries))
|
||||
self.assertLessEqual(cache.total_bytes, cache.max_bytes)
|
||||
|
||||
|
||||
class TestDefaults(unittest.TestCase):
|
||||
|
||||
def test_aging_is_enabled_by_default(self):
|
||||
self.assertIsNotNone(DEFAULT_AGING_INTERVAL)
|
||||
self.assertGreater(DEFAULT_AGING_INTERVAL, 0)
|
||||
self.assertIsNotNone(UsageCache(4)._aging_interval)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
Reference in New Issue
Block a user