Upload a snapshot of a thumbnail shard, never the live file

Every shard is in WAL mode and every put opens its own connection, so
while thumbnails are being generated on several threads — which is when
the first sync pass runs — the log is never checkpointed and the main
file holds whatever the last quiet moment left in it. For a shard created
seconds earlier that is nothing: a zero-byte file with the schema still
in the log. The sync read that file and uploaded it, and every other
device merging it failed with "no such table: thumbs" on every pass.

Copy the shard through SQLite's backup API into scratch first, which
serialises against writers and carries the log, and upload that.
This commit is contained in:
2026-09-20 00:21:14 +02:00
parent 0fa9003e54
commit 7fba28f7d8
4 changed files with 89 additions and 15 deletions
+14 -3
View File
@@ -286,9 +286,20 @@ async fn sync_shards(
// ---- upload ----------------------------------------------------------
for shard in &local {
let path = store.shard_path(shard.id);
let Ok(bytes) = std::fs::read(&path) else {
continue;
// A snapshot, never the live file — see `ThumbStore::snapshot_shard`
// for the zero-byte upload that reading the file produced.
let snapshot = scratch.join(format!("thumb-shard-{:04}-upload.sqlite", shard.id));
let bytes = match store.snapshot_shard(shard.id, &snapshot) {
Ok(()) => std::fs::read(&snapshot),
Err(e) => Err(std::io::Error::other(e.to_string())),
};
let _ = std::fs::remove_file(&snapshot);
let bytes = match bytes {
Ok(b) => b,
Err(e) => {
log::warn!("snapshotting thumbnail shard {}: {e}", shard.id);
continue;
}
};
let name = shard_name(&client, shard.id);