1304 lines
37 KiB
Markdown
1304 lines
37 KiB
Markdown
# DReader Hardware Abstraction Layer (HAL) Implementation Specification
|
|
|
|
**Version**: 1.0
|
|
**Date**: 2025-11-09
|
|
**Target**: Hardware driver developers implementing platform-specific HALs
|
|
|
|
---
|
|
|
|
## Table of Contents
|
|
|
|
1. [Overview](#1-overview)
|
|
2. [Architecture](#2-architecture)
|
|
3. [HAL Interface Specification](#3-hal-interface-specification)
|
|
4. [Touch Event & Gesture System](#4-touch-event--gesture-system)
|
|
5. [Implementation Requirements](#5-implementation-requirements)
|
|
6. [Testing & Validation](#6-testing--validation)
|
|
7. [Reference Implementation](#7-reference-implementation)
|
|
8. [Platform-Specific Considerations](#8-platform-specific-considerations)
|
|
9. [Example Implementations](#9-example-implementations)
|
|
|
|
---
|
|
|
|
## 1. Overview
|
|
|
|
### 1.1 Purpose
|
|
|
|
The DReader HAL provides a **platform-independent abstraction** for:
|
|
- **Display output**: Rendering PIL images to hardware screen
|
|
- **Touch input**: Capturing touch/mouse events and converting to gestures
|
|
- **Hardware control**: Brightness, power management, etc.
|
|
|
|
### 1.2 Design Goals
|
|
|
|
- **Portability**: Same application code runs on different hardware platforms
|
|
- **Async-first**: All operations are async for non-blocking I/O
|
|
- **Testability**: Easy desktop testing before deploying to device
|
|
- **Simplicity**: Minimal interface with clear contracts
|
|
|
|
### 1.3 Key Responsibilities
|
|
|
|
**What the HAL MUST do:**
|
|
- Display PIL Images on physical screen
|
|
- Detect touch events and classify gestures
|
|
- Return standardized `TouchEvent` objects
|
|
- Handle platform-specific initialization/cleanup
|
|
|
|
**What the HAL does NOT do:**
|
|
- Application logic (handled by `EbookReader` and `DReaderApplication`)
|
|
- Gesture routing (handled by `GestureRouter`)
|
|
- State management (handled by `StateManager`)
|
|
- Content rendering (handled by pyWebLayout)
|
|
|
|
---
|
|
|
|
## 2. Architecture
|
|
|
|
### 2.1 System Layers
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────┐
|
|
│ DReaderApplication (main.py) │
|
|
│ - State management, mode switching, event routing │
|
|
└───────────────────────┬─────────────────────────────────┘
|
|
│
|
|
│ Uses DisplayHAL interface
|
|
↓
|
|
┌─────────────────────────────────────────────────────────┐
|
|
│ DisplayHAL (Abstract Interface) │
|
|
│ - show_image(PIL.Image) │
|
|
│ - get_touch_event() -> TouchEvent │
|
|
│ - set_brightness(int) │
|
|
│ - initialize() / cleanup() │
|
|
└───────────────────────┬─────────────────────────────────┘
|
|
│
|
|
│ Implemented by
|
|
↓
|
|
┌─────────────────────────────────────────────────────────┐
|
|
│ Platform-Specific HAL Implementation │
|
|
│ │
|
|
│ Examples: │
|
|
│ • PygameDisplayHAL (Desktop/Testing) │
|
|
│ • EinkDisplayHAL (E-ink devices) │
|
|
│ • KoboDisplayHAL (Kobo readers) │
|
|
│ • RemarkableDisplayHAL (reMarkable tablets) │
|
|
│ • [Your Custom HAL] │
|
|
└───────────────────────┬─────────────────────────────────┘
|
|
│
|
|
│ Controls
|
|
↓
|
|
┌─────────────────────────────────────────────────────────┐
|
|
│ Hardware Drivers │
|
|
│ • Framebuffer / Display controller │
|
|
│ • Touch sensor / Digitizer │
|
|
│ • Backlight / Frontlight │
|
|
│ • Power management │
|
|
└─────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
### 2.2 Event Flow
|
|
|
|
```
|
|
Hardware Touch Sensor
|
|
↓
|
|
HAL Driver
|
|
↓
|
|
Touch Detection
|
|
↓
|
|
Gesture Classification ← HAL responsibility
|
|
↓
|
|
TouchEvent
|
|
↓
|
|
DReaderApplication.handle_touch()
|
|
↓
|
|
GestureRouter
|
|
↓
|
|
Gesture Handlers
|
|
↓
|
|
GestureResponse
|
|
↓
|
|
Display Update
|
|
↓
|
|
HAL.show_image()
|
|
↓
|
|
Hardware Display
|
|
```
|
|
|
|
---
|
|
|
|
## 3. HAL Interface Specification
|
|
|
|
### 3.1 Base Interface: `DisplayHAL`
|
|
|
|
**File**: `dreader/hal.py`
|
|
|
|
```python
|
|
from abc import ABC, abstractmethod
|
|
from typing import Optional
|
|
from PIL import Image
|
|
from dreader.gesture import TouchEvent
|
|
|
|
class DisplayHAL(ABC):
|
|
"""Abstract interface for display and input hardware"""
|
|
```
|
|
|
|
### 3.2 Required Methods
|
|
|
|
#### 3.2.1 `show_image(image: PIL.Image.Image)` ⭐ CRITICAL
|
|
|
|
**Purpose**: Display a PIL Image on the physical screen
|
|
|
|
**Contract**:
|
|
```python
|
|
async def show_image(self, image: Image.Image):
|
|
"""
|
|
Display a PIL Image on the screen.
|
|
|
|
Args:
|
|
image: PIL Image to display (typically RGB mode)
|
|
|
|
Requirements:
|
|
- MUST handle image format conversion if needed
|
|
- MUST handle resolution mismatch (scale/crop)
|
|
- MUST update physical display hardware
|
|
- SHOULD complete within 100ms for responsiveness
|
|
- SHOULD support e-ink optimization if applicable
|
|
|
|
Image Specifications:
|
|
- Format: Usually RGB (may be 'L' for grayscale)
|
|
- Size: Typically matches page_size config (e.g., 800x1200)
|
|
- Color depth: 8-bit per channel RGB
|
|
"""
|
|
pass
|
|
```
|
|
|
|
**Implementation Notes**:
|
|
- **E-ink devices**: Convert to grayscale, apply dithering, use appropriate refresh mode
|
|
- **Color displays**: May need RGB↔BGR conversion depending on driver
|
|
- **Framebuffer**: May need to write directly to `/dev/fb0` or use platform API
|
|
- **Performance**: Cache converted images if format conversion is expensive
|
|
|
|
**Example (Pseudocode)**:
|
|
```python
|
|
async def show_image(self, image: Image.Image):
|
|
# Convert to device format
|
|
if self.is_eink:
|
|
image = image.convert('L') # Grayscale
|
|
image = apply_dithering(image)
|
|
|
|
# Resize if needed
|
|
if image.size != (self.width, self.height):
|
|
image = image.resize((self.width, self.height), Image.LANCZOS)
|
|
|
|
# Write to framebuffer or display API
|
|
self.framebuffer.write(image)
|
|
|
|
# Refresh display
|
|
self.display_controller.refresh()
|
|
```
|
|
|
|
---
|
|
|
|
#### 3.2.2 `get_touch_event() -> Optional[TouchEvent]` ⭐ CRITICAL
|
|
|
|
**Purpose**: Wait for and return the next touch event with gesture classification
|
|
|
|
**Contract**:
|
|
```python
|
|
async def get_touch_event(self) -> Optional[TouchEvent]:
|
|
"""
|
|
Get the next touch event from hardware.
|
|
|
|
Returns:
|
|
TouchEvent if available, None if no event
|
|
|
|
Requirements:
|
|
- MUST classify gesture type (tap, swipe, long press, etc.)
|
|
- MUST return None immediately if no event (non-blocking)
|
|
- SHOULD detect gestures accurately with sensible thresholds
|
|
- MUST provide accurate pixel coordinates
|
|
- MAY implement timeouts for responsiveness
|
|
|
|
Gesture Detection Thresholds (Recommended):
|
|
- Tap: < 30px movement, < 300ms duration
|
|
- Long press: < 30px movement, >= 500ms duration
|
|
- Swipe: >= 30px movement, < 500ms duration
|
|
- Drag: >= 30px movement, sustained motion
|
|
- Pinch: Two-finger distance change
|
|
|
|
Coordinate System:
|
|
- Origin (0, 0) = top-left corner
|
|
- X increases right
|
|
- Y increases down
|
|
- Range: (0, 0) to (width-1, height-1)
|
|
"""
|
|
pass
|
|
```
|
|
|
|
**TouchEvent Format**:
|
|
```python
|
|
@dataclass
|
|
class TouchEvent:
|
|
gesture: GestureType # Required: gesture classification
|
|
x: int # Required: primary touch X
|
|
y: int # Required: primary touch Y
|
|
x2: Optional[int] = None # Optional: secondary touch X (pinch)
|
|
y2: Optional[int] = None # Optional: secondary touch Y (pinch)
|
|
timestamp_ms: float = 0 # Optional: timestamp
|
|
```
|
|
|
|
**Implementation Pattern**:
|
|
```python
|
|
async def get_touch_event(self) -> Optional[TouchEvent]:
|
|
# Read from touch hardware
|
|
touch_data = await self.read_touch_sensor()
|
|
|
|
if not touch_data:
|
|
return None
|
|
|
|
# Track touch state
|
|
if touch_data.is_down:
|
|
self._touch_start = (touch_data.x, touch_data.y)
|
|
self._touch_start_time = time.time()
|
|
|
|
elif touch_data.is_up:
|
|
# Calculate gesture
|
|
dx = touch_data.x - self._touch_start[0]
|
|
dy = touch_data.y - self._touch_start[1]
|
|
distance = (dx**2 + dy**2) ** 0.5
|
|
duration = time.time() - self._touch_start_time
|
|
|
|
# Classify gesture
|
|
if distance < 30:
|
|
if duration >= 0.5:
|
|
gesture = GestureType.LONG_PRESS
|
|
else:
|
|
gesture = GestureType.TAP
|
|
else:
|
|
# Swipe direction
|
|
if abs(dx) > abs(dy):
|
|
gesture = GestureType.SWIPE_LEFT if dx < 0 else GestureType.SWIPE_RIGHT
|
|
else:
|
|
gesture = GestureType.SWIPE_UP if dy < 0 else GestureType.SWIPE_DOWN
|
|
|
|
return TouchEvent(gesture, touch_data.x, touch_data.y)
|
|
|
|
return None
|
|
```
|
|
|
|
---
|
|
|
|
#### 3.2.3 `set_brightness(level: int)`
|
|
|
|
**Purpose**: Control display brightness/frontlight
|
|
|
|
**Contract**:
|
|
```python
|
|
async def set_brightness(self, level: int):
|
|
"""
|
|
Set display brightness.
|
|
|
|
Args:
|
|
level: Brightness level (0-10)
|
|
0 = dimmest (may be off)
|
|
10 = brightest
|
|
|
|
Requirements:
|
|
- MUST accept range 0-10
|
|
- SHOULD map to hardware's native range
|
|
- MAY be no-op if brightness control unavailable
|
|
- SHOULD persist setting if supported by hardware
|
|
"""
|
|
pass
|
|
```
|
|
|
|
**Implementation Notes**:
|
|
- Map 0-10 to hardware range (e.g., 0-255)
|
|
- E-ink frontlight: Control LED PWM or similar
|
|
- LCD backlight: Control via sysfs or platform API
|
|
- Desktop: May be no-op or control window alpha
|
|
|
|
---
|
|
|
|
#### 3.2.4 `initialize()` (Optional)
|
|
|
|
**Purpose**: One-time hardware initialization
|
|
|
|
**Contract**:
|
|
```python
|
|
async def initialize(self):
|
|
"""
|
|
Initialize the display hardware.
|
|
|
|
Called once before application starts.
|
|
|
|
Typical Tasks:
|
|
- Open framebuffer device
|
|
- Initialize touch sensor
|
|
- Configure display controller
|
|
- Set default brightness
|
|
- Calibrate touch screen
|
|
- Clear screen to white/black
|
|
"""
|
|
pass
|
|
```
|
|
|
|
---
|
|
|
|
#### 3.2.5 `cleanup()` (Optional)
|
|
|
|
**Purpose**: Hardware cleanup on shutdown
|
|
|
|
**Contract**:
|
|
```python
|
|
async def cleanup(self):
|
|
"""
|
|
Clean up display hardware resources.
|
|
|
|
Called during application shutdown.
|
|
|
|
Typical Tasks:
|
|
- Close file descriptors
|
|
- Restore default display state
|
|
- Power down display
|
|
- Release hardware locks
|
|
"""
|
|
pass
|
|
```
|
|
|
|
---
|
|
|
|
#### 3.2.6 `show_message(message: str, duration: float)` (Optional)
|
|
|
|
**Purpose**: Display simple text messages (loading screens, errors)
|
|
|
|
**Contract**:
|
|
```python
|
|
async def show_message(self, message: str, duration: float = 2.0):
|
|
"""
|
|
Display a text message.
|
|
|
|
Args:
|
|
message: Text to display
|
|
duration: How long to show (seconds)
|
|
|
|
Default implementation provided (creates PIL Image with text).
|
|
Override for platform-specific message display.
|
|
"""
|
|
pass
|
|
```
|
|
|
|
---
|
|
|
|
### 3.3 Extended Interface: `EventLoopHAL`
|
|
|
|
**Purpose**: For platforms that provide their own event loop (Pygame, Qt, Tkinter)
|
|
|
|
**File**: `dreader/hal.py`
|
|
|
|
```python
|
|
class EventLoopHAL(DisplayHAL):
|
|
"""Extended HAL that provides its own event loop"""
|
|
|
|
@abstractmethod
|
|
async def run_event_loop(self, app):
|
|
"""
|
|
Run the platform's event loop.
|
|
|
|
Args:
|
|
app: DReaderApplication instance
|
|
|
|
Typical Flow:
|
|
1. await self.initialize()
|
|
2. await app.start()
|
|
3. Enter event loop:
|
|
a. Get events
|
|
b. await app.handle_touch(event)
|
|
c. Handle quit/close
|
|
4. await app.shutdown()
|
|
5. await self.cleanup()
|
|
"""
|
|
pass
|
|
```
|
|
|
|
**Use Case**: When the platform requires control of the main loop (e.g., Pygame, Qt)
|
|
|
|
---
|
|
|
|
## 4. Touch Event & Gesture System
|
|
|
|
### 4.1 GestureType Enumeration
|
|
|
|
**File**: `dreader/gesture.py`
|
|
|
|
All gestures that HAL must be able to detect:
|
|
|
|
```python
|
|
class GestureType(Enum):
|
|
TAP = "tap" # Single finger tap
|
|
LONG_PRESS = "long_press" # Hold for 500ms+
|
|
SWIPE_LEFT = "swipe_left" # Horizontal swipe left
|
|
SWIPE_RIGHT = "swipe_right" # Horizontal swipe right
|
|
SWIPE_UP = "swipe_up" # Vertical swipe up
|
|
SWIPE_DOWN = "swipe_down" # Vertical swipe down
|
|
PINCH_IN = "pinch_in" # Two-finger pinch (zoom out)
|
|
PINCH_OUT = "pinch_out" # Two-finger spread (zoom in)
|
|
DRAG_START = "drag_start" # Start dragging
|
|
DRAG_MOVE = "drag_move" # Continue dragging
|
|
DRAG_END = "drag_end" # End dragging
|
|
```
|
|
|
|
### 4.2 Gesture Priority & Requirements
|
|
|
|
#### 4.2.1 Essential Gestures (MUST implement)
|
|
|
|
| Gesture | Usage | Detection |
|
|
|---------|-------|-----------|
|
|
| **TAP** | Link/button activation, word selection | < 30px movement, < 300ms |
|
|
| **SWIPE_LEFT** | Next page | Horizontal, dx < -30px |
|
|
| **SWIPE_RIGHT** | Previous page | Horizontal, dx > 30px |
|
|
| **SWIPE_UP** | Open navigation/TOC | Vertical, dy < -30px |
|
|
| **SWIPE_DOWN** | Open settings (from top 20%) | Vertical, dy > 30px |
|
|
|
|
#### 4.2.2 Important Gestures (SHOULD implement)
|
|
|
|
| Gesture | Usage | Detection |
|
|
|---------|-------|-----------|
|
|
| **LONG_PRESS** | Word definition, context menu | < 30px movement, >= 500ms |
|
|
| **PINCH_OUT** | Increase font size | Two-finger distance increase |
|
|
| **PINCH_IN** | Decrease font size | Two-finger distance decrease |
|
|
|
|
#### 4.2.3 Advanced Gestures (MAY implement)
|
|
|
|
| Gesture | Usage | Detection |
|
|
|---------|-------|-----------|
|
|
| **DRAG_START/MOVE/END** | Text selection | Sustained motion >= 30px |
|
|
|
|
### 4.3 Gesture Detection Algorithm
|
|
|
|
#### 4.3.1 State Machine
|
|
|
|
```
|
|
IDLE
|
|
↓ touch_down
|
|
TOUCHING
|
|
↓ timeout(500ms) → LONG_PRESS
|
|
↓ movement > 30px → MOVING
|
|
↓ touch_up → TAP
|
|
MOVING
|
|
↓ touch_up → SWIPE_*
|
|
↓ sustained → DRAG_*
|
|
```
|
|
|
|
#### 4.3.2 Recommended Thresholds
|
|
|
|
```python
|
|
# Distance thresholds
|
|
TAP_THRESHOLD = 30 # pixels
|
|
SWIPE_MIN_DISTANCE = 30 # pixels
|
|
DRAG_THRESHOLD = 30 # pixels
|
|
|
|
# Time thresholds
|
|
LONG_PRESS_DURATION = 0.5 # seconds (500ms)
|
|
TAP_MAX_DURATION = 0.3 # seconds (300ms)
|
|
SWIPE_MAX_DURATION = 0.5 # seconds
|
|
|
|
# Direction thresholds
|
|
SWIPE_ANGLE_THRESHOLD = 45 # degrees (for horizontal vs vertical)
|
|
```
|
|
|
|
#### 4.3.3 Pseudocode
|
|
|
|
```python
|
|
class GestureDetector:
|
|
def on_touch_down(self, x, y):
|
|
self.start_pos = (x, y)
|
|
self.start_time = time.now()
|
|
self.state = TOUCHING
|
|
|
|
def on_touch_move(self, x, y):
|
|
dx = x - self.start_pos[0]
|
|
dy = y - self.start_pos[1]
|
|
distance = sqrt(dx**2 + dy**2)
|
|
|
|
if distance > TAP_THRESHOLD and self.state == TOUCHING:
|
|
self.state = MOVING
|
|
# Could emit DRAG_START here
|
|
|
|
def on_touch_up(self, x, y):
|
|
dx = x - self.start_pos[0]
|
|
dy = y - self.start_pos[1]
|
|
distance = sqrt(dx**2 + dy**2)
|
|
duration = time.now() - self.start_time
|
|
|
|
if distance < TAP_THRESHOLD:
|
|
if duration >= LONG_PRESS_DURATION:
|
|
return GestureType.LONG_PRESS
|
|
else:
|
|
return GestureType.TAP
|
|
else:
|
|
# Determine swipe direction
|
|
if abs(dx) > abs(dy): # Horizontal
|
|
if dx > 0:
|
|
return GestureType.SWIPE_RIGHT
|
|
else:
|
|
return GestureType.SWIPE_LEFT
|
|
else: # Vertical
|
|
if dy > 0:
|
|
return GestureType.SWIPE_DOWN
|
|
else:
|
|
return GestureType.SWIPE_UP
|
|
```
|
|
|
|
### 4.4 TouchEvent Construction
|
|
|
|
```python
|
|
# Example: Creating a TAP event
|
|
event = TouchEvent(
|
|
gesture=GestureType.TAP,
|
|
x=450,
|
|
y=320,
|
|
timestamp_ms=time.time() * 1000
|
|
)
|
|
|
|
# Example: Creating a SWIPE event
|
|
event = TouchEvent(
|
|
gesture=GestureType.SWIPE_LEFT,
|
|
x=600, # End position
|
|
y=400,
|
|
timestamp_ms=time.time() * 1000
|
|
)
|
|
|
|
# Example: Creating a PINCH event (two fingers)
|
|
event = TouchEvent(
|
|
gesture=GestureType.PINCH_OUT,
|
|
x=400, # Finger 1
|
|
y=500,
|
|
x2=600, # Finger 2
|
|
y2=700,
|
|
timestamp_ms=time.time() * 1000
|
|
)
|
|
```
|
|
|
|
---
|
|
|
|
## 5. Implementation Requirements
|
|
|
|
### 5.1 Mandatory Features
|
|
|
|
✅ **MUST implement**:
|
|
- [ ] `show_image()` - Display PIL Images
|
|
- [ ] `get_touch_event()` - Return TouchEvents
|
|
- [ ] Basic gesture detection (TAP, SWIPE_LEFT/RIGHT/UP/DOWN)
|
|
- [ ] Correct coordinate system (0,0 = top-left)
|
|
- [ ] Async/await support for all methods
|
|
|
|
### 5.2 Recommended Features
|
|
|
|
⭐ **SHOULD implement**:
|
|
- [ ] `initialize()` and `cleanup()` - Proper lifecycle management
|
|
- [ ] `set_brightness()` - Brightness control
|
|
- [ ] LONG_PRESS gesture
|
|
- [ ] PINCH_IN/OUT gestures (if multitouch available)
|
|
- [ ] Efficient image format conversion
|
|
- [ ] E-ink optimization (partial refresh, dithering)
|
|
|
|
### 5.3 Optional Features
|
|
|
|
💡 **MAY implement**:
|
|
- [ ] `show_message()` override - Custom loading screens
|
|
- [ ] DRAG gestures - Text selection
|
|
- [ ] Touch pressure sensitivity
|
|
- [ ] Multi-touch tracking
|
|
- [ ] Hardware acceleration
|
|
- [ ] Custom refresh modes (e-ink A2/DU/GC16)
|
|
|
|
### 5.4 Performance Targets
|
|
|
|
| Metric | Target | Critical |
|
|
|--------|--------|----------|
|
|
| Image display latency | < 100ms | < 500ms |
|
|
| Touch event latency | < 50ms | < 200ms |
|
|
| Frame rate (color) | 30 FPS | 10 FPS |
|
|
| E-ink refresh (fast) | < 200ms | < 500ms |
|
|
| E-ink refresh (full) | < 1000ms | < 2000ms |
|
|
| Gesture detection accuracy | > 95% | > 80% |
|
|
|
|
---
|
|
|
|
## 6. Testing & Validation
|
|
|
|
### 6.1 Unit Tests
|
|
|
|
Test each HAL method in isolation:
|
|
|
|
```python
|
|
import pytest
|
|
from your_hal import YourDisplayHAL
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_show_image():
|
|
hal = YourDisplayHAL()
|
|
await hal.initialize()
|
|
|
|
# Create test image
|
|
img = Image.new('RGB', (800, 1200), color=(255, 0, 0))
|
|
|
|
# Should not raise
|
|
await hal.show_image(img)
|
|
|
|
await hal.cleanup()
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_gesture_detection():
|
|
hal = YourDisplayHAL()
|
|
await hal.initialize()
|
|
|
|
# Simulate tap
|
|
event = await hal.get_touch_event()
|
|
assert event.gesture == GestureType.TAP
|
|
assert 0 <= event.x < hal.width
|
|
assert 0 <= event.y < hal.height
|
|
```
|
|
|
|
### 6.2 Integration Tests
|
|
|
|
Test with actual DReaderApplication:
|
|
|
|
```python
|
|
from dreader.main import DReaderApplication, AppConfig
|
|
|
|
async def test_integration():
|
|
hal = YourDisplayHAL()
|
|
config = AppConfig(
|
|
display_hal=hal,
|
|
library_path="./test_books",
|
|
page_size=(800, 1200)
|
|
)
|
|
app = DReaderApplication(config)
|
|
|
|
await app.start()
|
|
|
|
# Simulate touch
|
|
event = TouchEvent(GestureType.TAP, 400, 600)
|
|
await app.handle_touch(event)
|
|
|
|
await app.shutdown()
|
|
```
|
|
|
|
### 6.3 Manual Test Checklist
|
|
|
|
- [ ] Display shows full-screen images without distortion
|
|
- [ ] Tap on link navigates to chapter
|
|
- [ ] Swipe left advances page
|
|
- [ ] Swipe right goes back page
|
|
- [ ] Swipe up opens TOC overlay
|
|
- [ ] Swipe down (from top) opens settings
|
|
- [ ] Long press on word shows definition
|
|
- [ ] Pinch gestures change font size (if supported)
|
|
- [ ] Brightness control works (if supported)
|
|
- [ ] No memory leaks after 100+ page turns
|
|
- [ ] No display artifacts or tearing
|
|
|
|
### 6.4 Validation Tools
|
|
|
|
**Debug Logging**:
|
|
```python
|
|
import logging
|
|
logger = logging.getLogger(__name__)
|
|
|
|
async def show_image(self, image):
|
|
logger.info(f"Displaying image: {image.size}, {image.mode}")
|
|
# ... implementation
|
|
|
|
async def get_touch_event(self):
|
|
event = # ... detect gesture
|
|
logger.info(f"Gesture detected: {event.gesture} at ({event.x}, {event.y})")
|
|
return event
|
|
```
|
|
|
|
**Touch Coordinate Visualization**:
|
|
- Draw red dot at touch coordinates
|
|
- Display gesture type as text overlay
|
|
- Show swipe trajectory lines
|
|
|
|
---
|
|
|
|
## 7. Reference Implementation
|
|
|
|
### 7.1 PygameDisplayHAL
|
|
|
|
**File**: `dreader/hal_pygame.py`
|
|
|
|
The Pygame HAL is a **complete reference implementation** for desktop testing.
|
|
|
|
**Key features**:
|
|
- Mouse → touch event conversion
|
|
- Gesture detection algorithm
|
|
- Keyboard shortcuts
|
|
- Debug logging
|
|
|
|
**Study this implementation for**:
|
|
- Gesture detection state machine
|
|
- Coordinate handling
|
|
- Event loop structure
|
|
- Error handling patterns
|
|
|
|
### 7.2 Minimal HAL Template
|
|
|
|
```python
|
|
from dreader.hal import DisplayHAL
|
|
from dreader.gesture import TouchEvent, GestureType
|
|
from PIL import Image
|
|
from typing import Optional
|
|
import asyncio
|
|
|
|
class MinimalHAL(DisplayHAL):
|
|
"""Minimal HAL implementation template"""
|
|
|
|
def __init__(self, width: int = 800, height: int = 1200):
|
|
self.width = width
|
|
self.height = height
|
|
|
|
# Gesture tracking
|
|
self._touch_start = None
|
|
self._touch_start_time = 0
|
|
|
|
async def initialize(self):
|
|
"""Initialize hardware"""
|
|
# TODO: Open framebuffer, initialize touch sensor
|
|
pass
|
|
|
|
async def cleanup(self):
|
|
"""Cleanup hardware"""
|
|
# TODO: Close file descriptors, cleanup
|
|
pass
|
|
|
|
async def show_image(self, image: Image.Image):
|
|
"""Display image on screen"""
|
|
# Convert to device format
|
|
if image.mode != 'RGB':
|
|
image = image.convert('RGB')
|
|
|
|
# Resize if needed
|
|
if image.size != (self.width, self.height):
|
|
image = image.resize(
|
|
(self.width, self.height),
|
|
Image.Resampling.LANCZOS
|
|
)
|
|
|
|
# TODO: Write to framebuffer or display API
|
|
# framebuffer.write(image)
|
|
|
|
async def get_touch_event(self) -> Optional[TouchEvent]:
|
|
"""Get next touch event"""
|
|
# TODO: Read from touch hardware
|
|
# touch_data = await read_touch_sensor()
|
|
|
|
# If no touch, return None
|
|
# if not touch_data:
|
|
# return None
|
|
|
|
# Classify gesture based on touch_down/touch_up
|
|
# See gesture detection algorithm in section 4.3
|
|
|
|
pass
|
|
|
|
async def set_brightness(self, level: int):
|
|
"""Set brightness (0-10)"""
|
|
# Map to hardware range
|
|
hardware_value = int(level * 255 / 10)
|
|
|
|
# TODO: Write to brightness control
|
|
# with open('/sys/class/backlight/.../brightness', 'w') as f:
|
|
# f.write(str(hardware_value))
|
|
```
|
|
|
|
---
|
|
|
|
## 8. Platform-Specific Considerations
|
|
|
|
### 8.1 E-Ink Devices (Kobo, Kindle, reMarkable)
|
|
|
|
#### Display Characteristics:
|
|
- Grayscale only (8-bit per pixel)
|
|
- Slow refresh (200ms - 1000ms)
|
|
- Multiple refresh modes (A2, DU, GC16, GL16)
|
|
- Requires dithering for images
|
|
|
|
#### Optimization Strategies:
|
|
|
|
**1. Partial Refresh**:
|
|
```python
|
|
async def show_image(self, image: Image.Image, partial: bool = True):
|
|
if partial:
|
|
# Use fast A2 mode for text-only updates
|
|
self.controller.set_mode('A2')
|
|
else:
|
|
# Full GC16 refresh every N pages
|
|
self.controller.set_mode('GC16')
|
|
|
|
self.framebuffer.write(image)
|
|
self.controller.refresh()
|
|
```
|
|
|
|
**2. Dithering**:
|
|
```python
|
|
from PIL import ImageDraw
|
|
|
|
def apply_dithering(image: Image.Image) -> Image.Image:
|
|
# Convert to grayscale
|
|
image = image.convert('L')
|
|
|
|
# Apply Floyd-Steinberg dithering
|
|
image = image.convert('1', dither=Image.FLOYDSTEINBERG)
|
|
image = image.convert('L')
|
|
|
|
return image
|
|
```
|
|
|
|
**3. Refresh Strategy**:
|
|
- Text pages: A2 mode (fast, ~200ms)
|
|
- Image-heavy pages: GC16 mode (slow, ~1000ms)
|
|
- Full refresh every 5-10 pages to clear ghosting
|
|
|
|
#### Touch Input:
|
|
- Often resistive touch (single-point only)
|
|
- Lower resolution than display
|
|
- May require calibration
|
|
|
|
**Example: Kobo HAL skeleton**:
|
|
```python
|
|
class KoboDisplayHAL(DisplayHAL):
|
|
def __init__(self):
|
|
self.fb = open('/dev/fb0', 'wb')
|
|
self.touch = open('/dev/input/event0', 'rb')
|
|
self.width = 1072
|
|
self.height = 1448
|
|
self.refresh_counter = 0
|
|
|
|
async def show_image(self, image: Image.Image):
|
|
# Convert to grayscale
|
|
image = image.convert('L')
|
|
|
|
# Resize
|
|
image = image.resize((self.width, self.height))
|
|
|
|
# Write to framebuffer
|
|
self.fb.write(image.tobytes())
|
|
|
|
# Decide refresh mode
|
|
self.refresh_counter += 1
|
|
if self.refresh_counter % 10 == 0:
|
|
# Full refresh every 10 pages
|
|
self._trigger_refresh('GC16')
|
|
else:
|
|
# Fast refresh
|
|
self._trigger_refresh('A2')
|
|
```
|
|
|
|
---
|
|
|
|
### 8.2 Linux Framebuffer
|
|
|
|
#### Display:
|
|
```python
|
|
async def show_image(self, image: Image.Image):
|
|
# Open framebuffer
|
|
with open('/dev/fb0', 'wb') as fb:
|
|
# Convert to RGB565 or RGB888 depending on device
|
|
if self.color_depth == 16:
|
|
# RGB565
|
|
data = image.convert('RGB').tobytes('raw', 'RGB')
|
|
# Convert to RGB565 packed format
|
|
data = self._convert_rgb888_to_rgb565(data)
|
|
else:
|
|
# RGB888
|
|
data = image.convert('RGB').tobytes()
|
|
|
|
fb.write(data)
|
|
```
|
|
|
|
#### Touch Input (evdev):
|
|
```python
|
|
import evdev
|
|
|
|
async def get_touch_event(self) -> Optional[TouchEvent]:
|
|
device = evdev.InputDevice('/dev/input/event0')
|
|
|
|
async for event in device.async_read_loop():
|
|
if event.type == evdev.ecodes.EV_ABS:
|
|
if event.code == evdev.ecodes.ABS_X:
|
|
self.current_x = event.value
|
|
elif event.code == evdev.ecodes.ABS_Y:
|
|
self.current_y = event.value
|
|
|
|
elif event.type == evdev.ecodes.EV_KEY:
|
|
if event.code == evdev.ecodes.BTN_TOUCH:
|
|
if event.value == 1: # Touch down
|
|
self._on_touch_down(self.current_x, self.current_y)
|
|
else: # Touch up
|
|
gesture = self._on_touch_up(self.current_x, self.current_y)
|
|
if gesture:
|
|
return TouchEvent(gesture, self.current_x, self.current_y)
|
|
```
|
|
|
|
---
|
|
|
|
### 8.3 Embedded Systems (Raspberry Pi, etc.)
|
|
|
|
#### Recommendations:
|
|
- Use hardware-accelerated PIL if available (Pillow-SIMD)
|
|
- Minimize memory allocations (reuse image buffers)
|
|
- Consider using mmap for framebuffer access
|
|
- Implement watchdog for crash recovery
|
|
|
|
```python
|
|
import mmap
|
|
|
|
class EmbeddedHAL(DisplayHAL):
|
|
def __init__(self):
|
|
self.fb_fd = os.open('/dev/fb0', os.O_RDWR)
|
|
self.fb_size = self.width * self.height * 4 # RGB888
|
|
self.fb_mmap = mmap.mmap(
|
|
self.fb_fd,
|
|
self.fb_size,
|
|
mmap.MAP_SHARED,
|
|
mmap.PROT_READ | mmap.PROT_WRITE
|
|
)
|
|
|
|
async def show_image(self, image: Image.Image):
|
|
# Write directly to mmap'd framebuffer
|
|
data = image.convert('RGB').tobytes()
|
|
self.fb_mmap.seek(0)
|
|
self.fb_mmap.write(data)
|
|
```
|
|
|
|
---
|
|
|
|
### 8.4 Desktop Testing (Windows/macOS/Linux)
|
|
|
|
Use **PygameDisplayHAL** (already provided) or implement similar with:
|
|
- **Tkinter**: For simple cross-platform windows
|
|
- **Qt (PyQt5/PySide6)**: For advanced GUI features
|
|
- **SDL2**: For game-like responsiveness
|
|
|
|
---
|
|
|
|
## 9. Example Implementations
|
|
|
|
### 9.1 Complete Example: Pygame HAL
|
|
|
|
See `dreader/hal_pygame.py` for full implementation.
|
|
|
|
**Key sections to study**:
|
|
- Lines 158-240: Gesture detection in `get_touch_event()`
|
|
- Lines 111-140: Image display in `show_image()`
|
|
- Lines 302-354: Event loop in `run_event_loop()`
|
|
|
|
---
|
|
|
|
### 9.2 Pseudocode: E-Ink HAL
|
|
|
|
```python
|
|
from dreader.hal import DisplayHAL
|
|
from dreader.gesture import TouchEvent, GestureType
|
|
from PIL import Image
|
|
import time
|
|
|
|
class EInkHAL(DisplayHAL):
|
|
"""E-Ink device HAL with partial refresh support"""
|
|
|
|
def __init__(self, width=1072, height=1448):
|
|
self.width = width
|
|
self.height = height
|
|
self.refresh_count = 0
|
|
|
|
# Hardware interfaces
|
|
self.framebuffer = None
|
|
self.touch_device = None
|
|
self.frontlight = None
|
|
|
|
# Gesture tracking
|
|
self._touch_down_pos = None
|
|
self._touch_down_time = 0
|
|
|
|
async def initialize(self):
|
|
"""Initialize e-ink display and touch sensor"""
|
|
# Open framebuffer
|
|
self.framebuffer = open('/dev/fb0', 'r+b')
|
|
|
|
# Open touch device
|
|
self.touch_device = evdev.InputDevice('/dev/input/event0')
|
|
|
|
# Initialize frontlight
|
|
self.frontlight = open('/sys/class/backlight/mxc_mst716/brightness', 'w')
|
|
|
|
# Clear screen
|
|
blank = Image.new('L', (self.width, self.height), 255)
|
|
await self.show_image(blank)
|
|
|
|
async def cleanup(self):
|
|
"""Cleanup hardware resources"""
|
|
if self.framebuffer:
|
|
self.framebuffer.close()
|
|
if self.frontlight:
|
|
self.frontlight.close()
|
|
|
|
async def show_image(self, image: Image.Image):
|
|
"""Display image with e-ink optimization"""
|
|
# Convert to grayscale
|
|
if image.mode != 'L':
|
|
image = image.convert('L')
|
|
|
|
# Resize if needed
|
|
if image.size != (self.width, self.height):
|
|
image = image.resize((self.width, self.height), Image.LANCZOS)
|
|
|
|
# Write to framebuffer
|
|
self.framebuffer.seek(0)
|
|
self.framebuffer.write(image.tobytes())
|
|
|
|
# Decide refresh mode
|
|
self.refresh_count += 1
|
|
if self.refresh_count % 10 == 0:
|
|
# Full refresh every 10 pages (remove ghosting)
|
|
self._trigger_refresh('FULL')
|
|
await asyncio.sleep(1.0) # Wait for full refresh
|
|
else:
|
|
# Partial refresh (fast)
|
|
self._trigger_refresh('PARTIAL')
|
|
await asyncio.sleep(0.2) # Wait for partial refresh
|
|
|
|
def _trigger_refresh(self, mode: str):
|
|
"""Trigger display refresh"""
|
|
if mode == 'FULL':
|
|
# Device-specific: trigger full GC16 refresh
|
|
# Example: ioctl(MXCFB_SEND_UPDATE, waveform=GC16)
|
|
pass
|
|
else:
|
|
# Device-specific: trigger partial A2 refresh
|
|
# Example: ioctl(MXCFB_SEND_UPDATE, waveform=A2)
|
|
pass
|
|
|
|
async def get_touch_event(self) -> Optional[TouchEvent]:
|
|
"""Read touch events from evdev"""
|
|
current_x = 0
|
|
current_y = 0
|
|
|
|
# Read events (with timeout)
|
|
try:
|
|
events = await asyncio.wait_for(
|
|
self._read_touch_events(),
|
|
timeout=0.1
|
|
)
|
|
except asyncio.TimeoutError:
|
|
return None
|
|
|
|
for event in events:
|
|
if event.type == evdev.ecodes.EV_ABS:
|
|
if event.code == evdev.ecodes.ABS_X:
|
|
current_x = event.value
|
|
elif event.code == evdev.ecodes.ABS_Y:
|
|
current_y = event.value
|
|
|
|
elif event.type == evdev.ecodes.EV_KEY:
|
|
if event.code == evdev.ecodes.BTN_TOUCH:
|
|
if event.value == 1: # Touch down
|
|
self._touch_down_pos = (current_x, current_y)
|
|
self._touch_down_time = time.time()
|
|
|
|
else: # Touch up
|
|
if not self._touch_down_pos:
|
|
continue
|
|
|
|
# Calculate gesture
|
|
dx = current_x - self._touch_down_pos[0]
|
|
dy = current_y - self._touch_down_pos[1]
|
|
distance = (dx**2 + dy**2) ** 0.5
|
|
duration = time.time() - self._touch_down_time
|
|
|
|
# Classify
|
|
if distance < 30:
|
|
if duration >= 0.5:
|
|
gesture = GestureType.LONG_PRESS
|
|
else:
|
|
gesture = GestureType.TAP
|
|
else:
|
|
if abs(dx) > abs(dy):
|
|
gesture = GestureType.SWIPE_LEFT if dx < 0 else GestureType.SWIPE_RIGHT
|
|
else:
|
|
gesture = GestureType.SWIPE_UP if dy < 0 else GestureType.SWIPE_DOWN
|
|
|
|
self._touch_down_pos = None
|
|
return TouchEvent(gesture, current_x, current_y)
|
|
|
|
return None
|
|
|
|
async def _read_touch_events(self):
|
|
"""Helper to read touch events with async"""
|
|
events = []
|
|
async for event in self.touch_device.async_read_loop():
|
|
events.append(event)
|
|
if event.type == evdev.ecodes.EV_SYN:
|
|
break # End of event batch
|
|
return events
|
|
|
|
async def set_brightness(self, level: int):
|
|
"""Set frontlight brightness"""
|
|
# Map 0-10 to hardware range (e.g., 0-100)
|
|
value = level * 10
|
|
self.frontlight.seek(0)
|
|
self.frontlight.write(str(value))
|
|
self.frontlight.flush()
|
|
```
|
|
|
|
---
|
|
|
|
### 9.3 Usage Example
|
|
|
|
```python
|
|
import asyncio
|
|
from your_hal import YourDisplayHAL
|
|
from dreader.main import DReaderApplication, AppConfig
|
|
|
|
async def main():
|
|
# Create HAL
|
|
hal = YourDisplayHAL(width=800, height=1200)
|
|
|
|
# Configure application
|
|
config = AppConfig(
|
|
display_hal=hal,
|
|
library_path="/home/user/Books",
|
|
page_size=(800, 1200),
|
|
log_level=logging.INFO
|
|
)
|
|
|
|
# Create application
|
|
app = DReaderApplication(config)
|
|
|
|
# If using EventLoopHAL:
|
|
await hal.run_event_loop(app)
|
|
|
|
# OR if using basic DisplayHAL:
|
|
# await app.start()
|
|
# while app.is_running():
|
|
# event = await hal.get_touch_event()
|
|
# if event:
|
|
# await app.handle_touch(event)
|
|
# await asyncio.sleep(0.01)
|
|
# await app.shutdown()
|
|
|
|
if __name__ == "__main__":
|
|
asyncio.run(main())
|
|
```
|
|
|
|
---
|
|
|
|
## 10. Troubleshooting
|
|
|
|
### Common Issues
|
|
|
|
**Issue**: Images not displaying
|
|
- Check framebuffer path (`/dev/fb0`)
|
|
- Verify color format (RGB vs BGR)
|
|
- Check image size matches display
|
|
- Verify write permissions
|
|
|
|
**Issue**: Touch not detected
|
|
- Check evdev device path
|
|
- Verify touch coordinates are mapped correctly
|
|
- Check coordinate system (origin, scaling)
|
|
- Test with `evtest` utility
|
|
|
|
**Issue**: Gestures not recognized
|
|
- Check thresholds (TAP_THRESHOLD, etc.)
|
|
- Add debug logging to gesture detection
|
|
- Verify touch up/down events are paired
|
|
- Check timestamp accuracy
|
|
|
|
**Issue**: Slow performance
|
|
- Profile image conversion (use cProfile)
|
|
- Cache converted images
|
|
- Use partial refresh on e-ink
|
|
- Consider hardware acceleration
|
|
|
|
---
|
|
|
|
## 11. Resources
|
|
|
|
### Documentation
|
|
- DReader Architecture: `INTEGRATION_DOCUMENTATION_INDEX.md`
|
|
- pyWebLayout Integration: `PYWEBLAYOUT_INTEGRATION_GUIDE.md`
|
|
- Async Rendering: `ASYNC_DIRTY_FLAG_RENDERING.md`
|
|
|
|
### Reference Code
|
|
- Pygame HAL: `dreader/hal_pygame.py`
|
|
- HAL Interface: `dreader/hal.py`
|
|
- Gesture Types: `dreader/gesture.py`
|
|
- Main Application: `dreader/main.py`
|
|
|
|
### External References
|
|
- Linux Framebuffer: https://www.kernel.org/doc/Documentation/fb/api.txt
|
|
- evdev Input: https://python-evdev.readthedocs.io/
|
|
- PIL/Pillow: https://pillow.readthedocs.io/
|
|
- E-Ink Controllers: Device-specific documentation (Kobo, Kindle, reMarkable)
|
|
|
|
---
|
|
|
|
## 12. Checklist for HAL Implementers
|
|
|
|
Use this checklist to track your implementation:
|
|
|
|
### Setup
|
|
- [ ] Read this specification completely
|
|
- [ ] Study PygameDisplayHAL reference implementation
|
|
- [ ] Set up development environment
|
|
- [ ] Test DReader with Pygame HAL first
|
|
|
|
### Core Implementation
|
|
- [ ] Create HAL class inheriting from DisplayHAL
|
|
- [ ] Implement `__init__()` with device configuration
|
|
- [ ] Implement `initialize()` - hardware setup
|
|
- [ ] Implement `cleanup()` - resource cleanup
|
|
- [ ] Implement `show_image()` - display rendering
|
|
- [ ] Implement `get_touch_event()` - input handling
|
|
- [ ] Implement gesture detection algorithm
|
|
- [ ] Implement `set_brightness()` (if supported)
|
|
|
|
### Testing
|
|
- [ ] Unit test each method
|
|
- [ ] Test with minimal application
|
|
- [ ] Test with full DReader application
|
|
- [ ] Test all essential gestures
|
|
- [ ] Performance profiling
|
|
- [ ] Memory leak testing
|
|
- [ ] Long-term stability testing
|
|
|
|
### Optimization
|
|
- [ ] Profile and optimize hot paths
|
|
- [ ] Implement caching where appropriate
|
|
- [ ] Add platform-specific optimizations
|
|
- [ ] Tune gesture thresholds
|
|
- [ ] Optimize display refresh strategy
|
|
|
|
### Documentation
|
|
- [ ] Document platform-specific requirements
|
|
- [ ] Document known limitations
|
|
- [ ] Create usage examples
|
|
- [ ] Write troubleshooting guide
|
|
|
|
---
|
|
|
|
## Version History
|
|
|
|
| Version | Date | Changes |
|
|
|---------|------|---------|
|
|
| 1.0 | 2025-11-09 | Initial specification |
|
|
|
|
---
|
|
|
|
**End of HAL Implementation Specification**
|
|
|
|
For questions or issues, please refer to the DReader documentation or create an issue in the project repository.
|