Drop retired Thumbnail jobs whenever a catalog is opened
Stopping the enqueue leaves the rows already queued: 23,582 on the reference catalog, about 1 MB of table and indexes that every query over jobs pays for. A migration would be the usual tool and is the wrong one here. A schema bump makes an older build refuse the synced catalog snapshot, and the tablet is on 0.16.0. So the rows are dropped at runtime instead, by jobs::drop_retired over a new JobKind::RETIRED list, from runner::recover - which already runs exactly once per catalog open, before any worker. It runs every open rather than once because an older build sharing the catalog queues them again on its next scan. kind leads the UNIQUE(kind, subject_id) index, so with nothing left it is one index probe. Measured on a copy of the reference catalog: 23,582 rows dropped in 40 ms on the first open, 0.07 ms after. Thumbnail stays in the enum so its number is never reused for a kind that would then inherit old rows. The runner tests that call recover move to a live kind; the jobs.rs tests of queue mechanics never call it and are unchanged. Refs #73
This commit is contained in:
@@ -29,7 +29,11 @@ pub enum JobKind {
|
||||
ScanFolder = 0,
|
||||
/// Promote an image from stat-only to full EXIF.
|
||||
ExtractMetadata = 1,
|
||||
/// Build or rebuild a thumbnail.
|
||||
/// Build or rebuild a thumbnail. **Retired** — see [`JobKind::RETIRED`].
|
||||
///
|
||||
/// Kept so the number stays taken: a catalog written by 0.16.0 or earlier
|
||||
/// holds rows of kind 2, and reusing it would hand them to whatever took
|
||||
/// its place.
|
||||
Thumbnail = 2,
|
||||
/// A sidecar on disk is newer than what the catalog read.
|
||||
ReadSidecar = 3,
|
||||
@@ -69,6 +73,18 @@ impl JobKind {
|
||||
JobKind::DetectFaces,
|
||||
];
|
||||
|
||||
/// Kinds that are no longer queued by anything, whose rows are deleted on
|
||||
/// sight by [`drop_retired`].
|
||||
///
|
||||
/// `Thumbnail` is here because thumbnails are owed by the store, not by
|
||||
/// the queue. The grid's worker and the thumbnail sweep both find their
|
||||
/// work by asking `ThumbStore` what it lacks, and the store is shared
|
||||
/// between devices, so it is the only thing that can say another device
|
||||
/// already made one. Up to 0.16.0 every scan enqueued a job per
|
||||
/// photograph anyway and no handler ever claimed one: the reference
|
||||
/// catalog held 23,582 of them (#73; catalog.md §6.1).
|
||||
pub const RETIRED: [JobKind; 1] = [JobKind::Thumbnail];
|
||||
|
||||
fn from_i64(v: i64) -> Option<Self> {
|
||||
Some(match v {
|
||||
0 => JobKind::ScanFolder,
|
||||
@@ -399,7 +415,7 @@ pub fn recover_orphaned(conn: &Connection) -> Result<usize, CatalogError> {
|
||||
///
|
||||
/// Coalescing keeps the table one row per unit of work, but nothing shrinks it
|
||||
/// when the work stops existing: a library that has been culled carries a
|
||||
/// thumbnail job for every photograph deleted since the last time anything
|
||||
/// job for every photograph deleted since the last time anything
|
||||
/// looked. Each one would be claimed, run, and failed five times.
|
||||
///
|
||||
/// Only kinds whose subject really is an image ([`JobKind::subject_is_image`])
|
||||
@@ -431,6 +447,32 @@ pub fn reap_orphan_subjects(conn: &Connection) -> Result<usize, CatalogError> {
|
||||
Ok(n)
|
||||
}
|
||||
|
||||
/// Delete every row of a [`JobKind::RETIRED`] kind.
|
||||
///
|
||||
/// Not a migration, deliberately. A schema bump makes an older build refuse
|
||||
/// the synced catalog snapshot, and a device still on 0.16.0 would lose the
|
||||
/// catalog to save a megabyte. So this runs where the queue is readied —
|
||||
/// [`crate::runner::recover`], at every open — and has to be cheap when there
|
||||
/// is nothing to do: `kind` leads the `UNIQUE(kind, subject_id)` index, so an
|
||||
/// empty answer is one index probe, not a table scan.
|
||||
///
|
||||
/// Every open rather than once, because once is not enough: an older build
|
||||
/// opening the same catalog enqueues them again on its next scan.
|
||||
///
|
||||
/// Rows in any state go. Nothing claims these kinds, so none can be running,
|
||||
/// and a failed one would be a report about work nobody was going to do.
|
||||
pub fn drop_retired(conn: &Connection) -> Result<usize, CatalogError> {
|
||||
let kinds: Vec<i64> = JobKind::RETIRED.iter().map(|k| *k as i64).collect();
|
||||
let placeholders = std::iter::repeat_n("?", kinds.len())
|
||||
.collect::<Vec<_>>()
|
||||
.join(",");
|
||||
let n = conn.execute(
|
||||
&format!("DELETE FROM jobs WHERE kind IN ({placeholders})"),
|
||||
rusqlite::params_from_iter(kinds.iter()),
|
||||
)?;
|
||||
Ok(n)
|
||||
}
|
||||
|
||||
/// How much is left, by state.
|
||||
///
|
||||
/// One query rather than a listing, because the caller is a progress line: a
|
||||
|
||||
Reference in New Issue
Block a user