perf(db): reads no longer wait behind writes; pages answer from cache

A series page took about a second to show its seasons on a phone, every
visit, although they were cached. Three things stacked up:

- One SQLite connection behind one mutex served the whole app, so every
  read queued behind every write. The database now has one owner: a
  writer thread for writes and a pool of read-only WAL connections for
  reads. synchronous = NORMAL and a busy timeout on every connection.
- The listing query built the set of every available item in the
  database before filtering to the parent (~80 ms on a desktop for a
  100k-item cache), then fetched user data one row at a time. It now
  checks availability per row, uses the hierarchy indexes (1.5 ms on
  the same benchmark) and batches the user-data lookup.
- A cache read that missed the 100 ms fast path was set aside until the
  server answered. It is now raced against the server; whichever answers
  first with content wins.

On the Fairphone, Frasier's season and episode lists now come from
cache in 34-133 ms (was 600-1030 ms waiting on the server).

Fixes found on the way, each with a test that failed first:
- sync_queue_mutation could return another mutation's row id: the id
  came from a second trip to the shared connection. insert() reads it in
  the same job.
- save_to_cache switched foreign keys off on the shared connection
  across its awaits, so concurrent writes ran unchecked. The toggle now
  lives inside one writer job, and a page is one transaction instead of
  one commit per row.

Also: thumbnail LRU touches no longer block the lookup; unused
tokio-rusqlite dropped. Design and invariants in
docs/architecture/08-database-design.md (Connection ownership, Listing
query shape) and 03-data-flow.md.
This commit is contained in:
2026-09-24 03:58:04 +02:00
parent 1fb5f070c8
commit 21f24dd998
12 changed files with 1722 additions and 751 deletions
+70 -7
View File
@@ -49,6 +49,19 @@ pub async fn sync_queue_mutation(
Arc::new(database.service())
};
enqueue_mutation(&*db_service, user_id, operation, item_id, payload).await
}
/// Insert one pending mutation and return the id of *that* row.
///
/// TRACES: UR-002, UR-017 | DR-014
pub(crate) async fn enqueue_mutation<S: DatabaseService>(
db_service: &S,
user_id: String,
operation: String,
item_id: Option<String>,
payload: Option<String>,
) -> Result<i64, String> {
let query = Query::with_params(
"INSERT INTO sync_queue (user_id, operation, item_id, payload, status, created_at)
VALUES (?, ?, ?, ?, 'pending', CURRENT_TIMESTAMP)",
@@ -60,13 +73,9 @@ pub async fn sync_queue_mutation(
],
);
db_service.execute(query).await.map_err(|e| e.to_string())?;
let id = db_service
.last_insert_rowid()
.await
.map_err(|e| e.to_string())?;
Ok(id)
// `insert`, not `execute` + `last_insert_rowid`: the id must be read in
// the same job as the insert, or a concurrent write hands us its row.
db_service.insert(query).await
}
/// Get all pending sync operations for a user
@@ -387,4 +396,58 @@ mod tests {
assert!(item.retry_count == i);
}
}
/// Each queued mutation must get back the id of its *own* row.
///
/// The id used to come from a separate `last_insert_rowid()` call — a
/// second trip to the shared connection — so another insert landing in
/// between handed this mutation someone else's id, and marking it synced
/// later completed the wrong row.
///
/// TRACES: UR-002, UR-017 | DR-014 | UT-014
#[tokio::test(flavor = "multi_thread", worker_threads = 8)]
async fn concurrent_enqueues_each_get_their_own_row_id() {
let database = crate::storage::Database::open_in_memory().unwrap();
let service = Arc::new(database.service());
service
.execute(Query::new(
"INSERT INTO servers (id, name, url) VALUES ('s', 'S', 'http://s')",
))
.await
.unwrap();
service
.execute(Query::new(
"INSERT INTO users (id, server_id, username) VALUES ('u', 's', 'u')",
))
.await
.unwrap();
let tasks: Vec<_> = (0..200)
.map(|i| {
let service = Arc::clone(&service);
tokio::spawn(async move {
let op = format!("op-{i}");
let id = enqueue_mutation(&*service, "u".into(), op.clone(), None, None)
.await
.unwrap();
(op, id)
})
})
.collect();
for task in tasks {
let (op, id) = task.await.unwrap();
let stored: String = service
.query_one(
Query::with_params(
"SELECT operation FROM sync_queue WHERE id = ?",
vec![QueryParam::Int64(id)],
),
|row| row.get(0),
)
.await
.unwrap();
assert_eq!(stored, op, "mutation {op} was handed row {id}");
}
}
}