perf(db): reads no longer wait behind writes; pages answer from cache
A series page took about a second to show its seasons on a phone, every visit, although they were cached. Three things stacked up: - One SQLite connection behind one mutex served the whole app, so every read queued behind every write. The database now has one owner: a writer thread for writes and a pool of read-only WAL connections for reads. synchronous = NORMAL and a busy timeout on every connection. - The listing query built the set of every available item in the database before filtering to the parent (~80 ms on a desktop for a 100k-item cache), then fetched user data one row at a time. It now checks availability per row, uses the hierarchy indexes (1.5 ms on the same benchmark) and batches the user-data lookup. - A cache read that missed the 100 ms fast path was set aside until the server answered. It is now raced against the server; whichever answers first with content wins. On the Fairphone, Frasier's season and episode lists now come from cache in 34-133 ms (was 600-1030 ms waiting on the server). Fixes found on the way, each with a test that failed first: - sync_queue_mutation could return another mutation's row id: the id came from a second trip to the shared connection. insert() reads it in the same job. - save_to_cache switched foreign keys off on the shared connection across its awaits, so concurrent writes ran unchecked. The toggle now lives inside one writer job, and a page is one transaction instead of one commit per row. Also: thumbnail LRU touches no longer block the lookup; unused tokio-rusqlite dropped. Design and invariants in docs/architecture/08-database-design.md (Connection ownership, Listing query shape) and 03-data-flow.md.
This commit is contained in:
+268
-106
@@ -1,16 +1,31 @@
|
||||
//! Database service abstraction layer
|
||||
//! Database service: the single owner of the SQLite database.
|
||||
//!
|
||||
//! This module provides an async database interface that abstracts away
|
||||
//! the underlying database implementation. This makes it easy to:
|
||||
//! - Switch between sync (rusqlite) and async (tokio-rusqlite) implementations
|
||||
//! - Prevent blocking the async runtime with synchronous database calls
|
||||
//! - Test with different database backends
|
||||
//! - Migrate to other database systems in the future
|
||||
//! Every query in the app goes through [`RusqliteService`], which owns the
|
||||
//! connections and hands work to them — callers never touch a `Connection`.
|
||||
//!
|
||||
//! - **Writes** (`execute`, `insert`, `transaction`, …) are sent as jobs to one
|
||||
//! dedicated writer thread that owns the read-write connection. SQLite allows
|
||||
//! one writer at a time anyway; owning it on one thread makes that explicit,
|
||||
//! keeps connection-wide state (pragmas) out of reach of concurrent callers,
|
||||
//! and parks no tokio blocking threads on a mutex while writes queue up.
|
||||
//! - **Reads** (`query_*`) run on a small pool of read-only connections. The
|
||||
//! database is in WAL mode, so readers see the last committed state and never
|
||||
//! wait for the writer — a large catalog-cache transaction no longer stalls
|
||||
//! library pages, thumbnail lookups or settings reads.
|
||||
//!
|
||||
//! A service built with [`RusqliteService::new`] has no reader pool (in-memory
|
||||
//! databases cannot be shared between connections) and routes reads through
|
||||
//! the writer, which is the old single-connection behaviour tests rely on.
|
||||
//!
|
||||
//! See `docs/architecture/08-database-design.md` → "Connection ownership".
|
||||
|
||||
use crate::utils::lock::MutexSafe;
|
||||
use async_trait::async_trait;
|
||||
use log::{debug, error};
|
||||
use rusqlite::{params_from_iter, Connection, Result as SqliteResult, Row};
|
||||
use std::sync::{Arc, Mutex};
|
||||
use std::panic::AssertUnwindSafe;
|
||||
use std::sync::mpsc;
|
||||
use std::sync::{Arc, Condvar, Mutex};
|
||||
|
||||
/// Database query result type
|
||||
pub type DbResult<T> = Result<T, String>;
|
||||
@@ -84,8 +99,28 @@ pub trait DatabaseService: Send + Sync {
|
||||
F: FnOnce(&mut Transaction) -> DbResult<T> + Send + 'static,
|
||||
T: Send + 'static;
|
||||
|
||||
/// Get the row ID of the most recent successful INSERT
|
||||
async fn last_insert_rowid(&self) -> DbResult<i64>;
|
||||
/// Run a transaction with foreign-key enforcement switched off for its
|
||||
/// duration only.
|
||||
///
|
||||
/// `PRAGMA foreign_keys` is per connection and is a no-op inside a
|
||||
/// transaction, so it has to be flipped around the `BEGIN`/`COMMIT` — and
|
||||
/// all of that must happen as one job on the writer, or any other write
|
||||
/// that got in between would run unchecked too.
|
||||
async fn transaction_without_foreign_keys<F, T>(&self, f: F) -> DbResult<T>
|
||||
where
|
||||
F: FnOnce(&mut Transaction) -> DbResult<T> + Send + 'static,
|
||||
T: Send + 'static;
|
||||
|
||||
/// Execute an INSERT and return the rowid of the row it inserted.
|
||||
///
|
||||
/// The rowid is read in the same job as the insert. Reading it with a
|
||||
/// second call would race every other write, returning someone else's id.
|
||||
async fn insert(&self, query: Query) -> DbResult<i64>;
|
||||
|
||||
/// Queue a write without waiting for it — for best-effort bookkeeping (an
|
||||
/// LRU access time) that must not hold up the caller. It still runs in
|
||||
/// order with every other write; failures are only logged.
|
||||
fn execute_detached(&self, query: Query);
|
||||
}
|
||||
|
||||
/// Transaction handle for batching multiple operations
|
||||
@@ -110,48 +145,183 @@ impl<'a> Transaction<'a> {
|
||||
}
|
||||
}
|
||||
|
||||
/// Rusqlite-based database service implementation
|
||||
///
|
||||
/// This implementation wraps synchronous rusqlite operations in tokio::task::spawn_blocking
|
||||
/// to prevent blocking the async runtime.
|
||||
type Job = Box<dyn FnOnce(&Connection) + Send>;
|
||||
|
||||
/// The thread that owns the read-write connection. Jobs run one at a time, in
|
||||
/// the order they were sent; the thread exits when the last service handle
|
||||
/// (and so the last sender) is dropped.
|
||||
struct Writer {
|
||||
jobs: mpsc::Sender<Job>,
|
||||
}
|
||||
|
||||
impl Writer {
|
||||
fn spawn(conn: Arc<Mutex<Connection>>) -> Self {
|
||||
let (jobs, queue) = mpsc::channel::<Job>();
|
||||
std::thread::Builder::new()
|
||||
.name("db-writer".into())
|
||||
.spawn(move || {
|
||||
for job in queue {
|
||||
// The connection stays behind a mutex only so migrations and
|
||||
// tests can reach it; in the app this thread is its sole user.
|
||||
// `lock_safe` so a poisoned lock is recovered, not fatal.
|
||||
let conn = conn.lock_safe();
|
||||
// A panicking row mapper must not take the owner down with
|
||||
// it: the job's reply channel drops, its caller gets an
|
||||
// error, and the next job runs normally. The guard lives
|
||||
// outside the unwind, so the mutex is not poisoned either.
|
||||
if std::panic::catch_unwind(AssertUnwindSafe(|| job(&conn))).is_err() {
|
||||
error!("[db] a database job panicked; the writer carries on");
|
||||
// Undo whatever connection state the job was midway
|
||||
// through: an open transaction would make the next
|
||||
// job's BEGIN fail, and a job that switched foreign
|
||||
// keys off would leave them off for everyone.
|
||||
if !conn.is_autocommit() {
|
||||
let _ = conn.execute_batch("ROLLBACK");
|
||||
}
|
||||
let _ = conn.execute_batch("PRAGMA foreign_keys = ON");
|
||||
}
|
||||
}
|
||||
})
|
||||
.expect("failed to spawn the database writer thread");
|
||||
Self { jobs }
|
||||
}
|
||||
|
||||
async fn run<T, F>(&self, f: F) -> DbResult<T>
|
||||
where
|
||||
T: Send + 'static,
|
||||
F: FnOnce(&Connection) -> DbResult<T> + Send + 'static,
|
||||
{
|
||||
let (reply, result) = tokio::sync::oneshot::channel();
|
||||
self.jobs
|
||||
.send(Box::new(move |conn| {
|
||||
let _ = reply.send(f(conn));
|
||||
}))
|
||||
.map_err(|_| "database writer has stopped".to_string())?;
|
||||
result
|
||||
.await
|
||||
.map_err(|_| "database job panicked".to_string())?
|
||||
}
|
||||
|
||||
fn run_detached(&self, f: impl FnOnce(&Connection) + Send + 'static) {
|
||||
if self.jobs.send(Box::new(f)).is_err() {
|
||||
debug!("[db] writer stopped; dropped a detached write");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Read-only connections, checked out one per query. WAL gives each a
|
||||
/// snapshot of the last commit, so they never wait for the writer.
|
||||
struct ReaderPool {
|
||||
idle: Mutex<Vec<Connection>>,
|
||||
returned: Condvar,
|
||||
}
|
||||
|
||||
impl ReaderPool {
|
||||
/// Blocking: waits for a free connection. Call from `spawn_blocking`.
|
||||
fn run<T>(&self, f: impl FnOnce(&Connection) -> T) -> T {
|
||||
let conn = {
|
||||
let mut idle = self.idle.lock_safe();
|
||||
loop {
|
||||
if let Some(conn) = idle.pop() {
|
||||
break conn;
|
||||
}
|
||||
idle = self
|
||||
.returned
|
||||
.wait(idle)
|
||||
.unwrap_or_else(|poisoned| poisoned.into_inner());
|
||||
}
|
||||
};
|
||||
// Returned on drop, so a panicking mapper does not leak the connection.
|
||||
let checkout = Checkout {
|
||||
pool: self,
|
||||
conn: Some(conn),
|
||||
};
|
||||
f(checkout.conn.as_ref().expect("checked-out connection"))
|
||||
}
|
||||
}
|
||||
|
||||
struct Checkout<'a> {
|
||||
pool: &'a ReaderPool,
|
||||
conn: Option<Connection>,
|
||||
}
|
||||
|
||||
impl Drop for Checkout<'_> {
|
||||
fn drop(&mut self) {
|
||||
if let Some(conn) = self.conn.take() {
|
||||
self.pool.idle.lock_safe().push(conn);
|
||||
self.pool.returned.notify_one();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Rusqlite-based database service: a cheap, cloneable handle to the writer
|
||||
/// thread and reader pool. See the module docs.
|
||||
#[derive(Clone)]
|
||||
pub struct RusqliteService {
|
||||
conn: Arc<Mutex<Connection>>,
|
||||
writer: Arc<Writer>,
|
||||
readers: Option<Arc<ReaderPool>>,
|
||||
}
|
||||
|
||||
impl RusqliteService {
|
||||
/// A service over a single connection: writes *and* reads go through the
|
||||
/// writer thread. Used for in-memory databases, which cannot be shared
|
||||
/// between connections.
|
||||
#[cfg_attr(not(test), allow(dead_code))]
|
||||
pub fn new(conn: Arc<Mutex<Connection>>) -> Self {
|
||||
Self { conn }
|
||||
Self {
|
||||
writer: Arc::new(Writer::spawn(conn)),
|
||||
readers: None,
|
||||
}
|
||||
}
|
||||
|
||||
/// A service whose reads run on `readers` — read-only connections to the
|
||||
/// same (file-backed, WAL-mode) database — alongside the writer.
|
||||
pub fn with_readers(conn: Arc<Mutex<Connection>>, readers: Vec<Connection>) -> Self {
|
||||
let readers = (!readers.is_empty()).then(|| {
|
||||
Arc::new(ReaderPool {
|
||||
idle: Mutex::new(readers),
|
||||
returned: Condvar::new(),
|
||||
})
|
||||
});
|
||||
Self {
|
||||
writer: Arc::new(Writer::spawn(conn)),
|
||||
readers,
|
||||
}
|
||||
}
|
||||
|
||||
async fn read<T, F>(&self, f: F) -> DbResult<T>
|
||||
where
|
||||
T: Send + 'static,
|
||||
F: FnOnce(&Connection) -> DbResult<T> + Send + 'static,
|
||||
{
|
||||
match &self.readers {
|
||||
Some(pool) => {
|
||||
let pool = Arc::clone(pool);
|
||||
tokio::task::spawn_blocking(move || pool.run(f))
|
||||
.await
|
||||
.map_err(|e| format!("Task join error: {}", e))?
|
||||
}
|
||||
None => self.writer.run(f).await,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl DatabaseService for RusqliteService {
|
||||
async fn execute(&self, query: Query) -> DbResult<usize> {
|
||||
let conn = Arc::clone(&self.conn);
|
||||
tokio::task::spawn_blocking(move || {
|
||||
// `lock_safe`, not `lock`: this is the busiest lock in the app and a
|
||||
// panic under the guard would otherwise poison it, failing every
|
||||
// later query with "poisoned lock" until the process restarts.
|
||||
let conn = conn.lock_safe();
|
||||
execute_query(&conn, query)
|
||||
})
|
||||
.await
|
||||
.map_err(|e| format!("Task join error: {}", e))?
|
||||
self.writer
|
||||
.run(move |conn| execute_query(conn, query))
|
||||
.await
|
||||
}
|
||||
|
||||
async fn execute_batch(&self, sql: &str) -> DbResult<()> {
|
||||
let conn = Arc::clone(&self.conn);
|
||||
let sql = sql.to_string();
|
||||
tokio::task::spawn_blocking(move || {
|
||||
// `lock_safe`, not `lock`: this is the busiest lock in the app and a
|
||||
// panic under the guard would otherwise poison it, failing every
|
||||
// later query with "poisoned lock" until the process restarts.
|
||||
let conn = conn.lock_safe();
|
||||
conn.execute_batch(&sql)
|
||||
.map_err(|e| format!("Execute batch failed: {}", e))
|
||||
})
|
||||
.await
|
||||
.map_err(|e| format!("Task join error: {}", e))?
|
||||
self.writer
|
||||
.run(move |conn| {
|
||||
conn.execute_batch(&sql)
|
||||
.map_err(|e| format!("Execute batch failed: {}", e))
|
||||
})
|
||||
.await
|
||||
}
|
||||
|
||||
async fn query_one<T, F>(&self, query: Query, mapper: F) -> DbResult<T>
|
||||
@@ -159,16 +329,7 @@ impl DatabaseService for RusqliteService {
|
||||
T: Send + 'static,
|
||||
F: Fn(&Row) -> SqliteResult<T> + Send + 'static,
|
||||
{
|
||||
let conn = Arc::clone(&self.conn);
|
||||
tokio::task::spawn_blocking(move || {
|
||||
// `lock_safe`, not `lock`: this is the busiest lock in the app and a
|
||||
// panic under the guard would otherwise poison it, failing every
|
||||
// later query with "poisoned lock" until the process restarts.
|
||||
let conn = conn.lock_safe();
|
||||
query_one(&conn, query, mapper)
|
||||
})
|
||||
.await
|
||||
.map_err(|e| format!("Task join error: {}", e))?
|
||||
self.read(move |conn| query_one(conn, query, mapper)).await
|
||||
}
|
||||
|
||||
async fn query_optional<T, F>(&self, query: Query, mapper: F) -> DbResult<Option<T>>
|
||||
@@ -176,16 +337,8 @@ impl DatabaseService for RusqliteService {
|
||||
T: Send + 'static,
|
||||
F: Fn(&Row) -> SqliteResult<T> + Send + 'static,
|
||||
{
|
||||
let conn = Arc::clone(&self.conn);
|
||||
tokio::task::spawn_blocking(move || {
|
||||
// `lock_safe`, not `lock`: this is the busiest lock in the app and a
|
||||
// panic under the guard would otherwise poison it, failing every
|
||||
// later query with "poisoned lock" until the process restarts.
|
||||
let conn = conn.lock_safe();
|
||||
query_optional(&conn, query, mapper)
|
||||
})
|
||||
.await
|
||||
.map_err(|e| format!("Task join error: {}", e))?
|
||||
self.read(move |conn| query_optional(conn, query, mapper))
|
||||
.await
|
||||
}
|
||||
|
||||
async fn query_many<T, F>(&self, query: Query, mapper: F) -> DbResult<Vec<T>>
|
||||
@@ -193,16 +346,7 @@ impl DatabaseService for RusqliteService {
|
||||
T: Send + 'static,
|
||||
F: Fn(&Row) -> SqliteResult<T> + Send + 'static,
|
||||
{
|
||||
let conn = Arc::clone(&self.conn);
|
||||
tokio::task::spawn_blocking(move || {
|
||||
// `lock_safe`, not `lock`: this is the busiest lock in the app and a
|
||||
// panic under the guard would otherwise poison it, failing every
|
||||
// later query with "poisoned lock" until the process restarts.
|
||||
let conn = conn.lock_safe();
|
||||
query_many(&conn, query, mapper)
|
||||
})
|
||||
.await
|
||||
.map_err(|e| format!("Task join error: {}", e))?
|
||||
self.read(move |conn| query_many(conn, query, mapper)).await
|
||||
}
|
||||
|
||||
async fn transaction<F, T>(&self, f: F) -> DbResult<T>
|
||||
@@ -210,47 +354,65 @@ impl DatabaseService for RusqliteService {
|
||||
F: FnOnce(&mut Transaction) -> DbResult<T> + Send + 'static,
|
||||
T: Send + 'static,
|
||||
{
|
||||
let conn = Arc::clone(&self.conn);
|
||||
tokio::task::spawn_blocking(move || {
|
||||
// `lock_safe`, not `lock`: this is the busiest lock in the app and a
|
||||
// panic under the guard would otherwise poison it, failing every
|
||||
// later query with "poisoned lock" until the process restarts.
|
||||
let conn = conn.lock_safe();
|
||||
|
||||
conn.execute("BEGIN TRANSACTION", [])
|
||||
.map_err(|e| format!("Failed to begin transaction: {}", e))?;
|
||||
|
||||
let mut transaction = Transaction::new(&conn);
|
||||
let result = f(&mut transaction);
|
||||
|
||||
match result {
|
||||
Ok(value) => {
|
||||
conn.execute("COMMIT", [])
|
||||
.map_err(|e| format!("Failed to commit transaction: {}", e))?;
|
||||
Ok(value)
|
||||
}
|
||||
Err(e) => {
|
||||
conn.execute("ROLLBACK", [])
|
||||
.map_err(|e| format!("Failed to rollback transaction: {}", e))?;
|
||||
Err(e)
|
||||
}
|
||||
}
|
||||
})
|
||||
.await
|
||||
.map_err(|e| format!("Task join error: {}", e))?
|
||||
self.writer.run(move |conn| run_transaction(conn, f)).await
|
||||
}
|
||||
|
||||
async fn last_insert_rowid(&self) -> DbResult<i64> {
|
||||
let conn = Arc::clone(&self.conn);
|
||||
tokio::task::spawn_blocking(move || {
|
||||
// `lock_safe`, not `lock`: this is the busiest lock in the app and a
|
||||
// panic under the guard would otherwise poison it, failing every
|
||||
// later query with "poisoned lock" until the process restarts.
|
||||
let conn = conn.lock_safe();
|
||||
Ok(conn.last_insert_rowid())
|
||||
})
|
||||
.await
|
||||
.map_err(|e| format!("Task join error: {}", e))?
|
||||
async fn transaction_without_foreign_keys<F, T>(&self, f: F) -> DbResult<T>
|
||||
where
|
||||
F: FnOnce(&mut Transaction) -> DbResult<T> + Send + 'static,
|
||||
T: Send + 'static,
|
||||
{
|
||||
self.writer
|
||||
.run(move |conn| {
|
||||
conn.execute_batch("PRAGMA foreign_keys = OFF")
|
||||
.map_err(|e| format!("Failed to disable foreign keys: {}", e))?;
|
||||
let result = run_transaction(conn, f);
|
||||
// Always restored, whatever the transaction did.
|
||||
if let Err(e) = conn.execute_batch("PRAGMA foreign_keys = ON") {
|
||||
error!("[db] failed to re-enable foreign keys: {}", e);
|
||||
}
|
||||
result
|
||||
})
|
||||
.await
|
||||
}
|
||||
|
||||
async fn insert(&self, query: Query) -> DbResult<i64> {
|
||||
self.writer
|
||||
.run(move |conn| {
|
||||
execute_query(conn, query)?;
|
||||
Ok(conn.last_insert_rowid())
|
||||
})
|
||||
.await
|
||||
}
|
||||
|
||||
fn execute_detached(&self, query: Query) {
|
||||
self.writer.run_detached(move |conn| {
|
||||
if let Err(e) = execute_query(conn, query) {
|
||||
debug!("[db] detached write failed: {}", e);
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
fn run_transaction<F, T>(conn: &Connection, f: F) -> DbResult<T>
|
||||
where
|
||||
F: FnOnce(&mut Transaction) -> DbResult<T>,
|
||||
{
|
||||
conn.execute("BEGIN TRANSACTION", [])
|
||||
.map_err(|e| format!("Failed to begin transaction: {}", e))?;
|
||||
|
||||
let mut transaction = Transaction::new(conn);
|
||||
match f(&mut transaction) {
|
||||
Ok(value) => {
|
||||
conn.execute("COMMIT", [])
|
||||
.map_err(|e| format!("Failed to commit transaction: {}", e))?;
|
||||
Ok(value)
|
||||
}
|
||||
Err(e) => {
|
||||
conn.execute("ROLLBACK", [])
|
||||
.map_err(|e| format!("Failed to rollback transaction: {}", e))?;
|
||||
Err(e)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user