Ask the prefetch's cache questions on one held connection

holds_original, the prefetch worker's check that a neighbour's original
is already cached, opened the catalog for every neighbour it asked about.
Its own comment called it a row check; the open around it was four of
the five opens a develop landing made.

The worker now keeps one Catalog for the batch it is serving, opened at
the first check and reopened only if the batch names another catalog
file. The connection runs in autocommit, so each check still sees what
fetch_original committed in between. fetch_original is unchanged.

With the backfill no longer run on every open, a landing whose
neighbours are all cached goes from five opens to two, and from ~80 ms
of CPU to ~1-2 ms on a copy of the reference catalog.
This commit is contained in:
2026-09-26 14:26:10 -04:00
parent ffdd640170
commit faf52f6dbd
+19 -3
View File
@@ -890,6 +890,9 @@ pub(super) fn serve_prefetches(shared: &PrefetchShared, events: &Sender<Prefetch
(wanted.generation, conn, jobs)
};
// One catalog connection for this wish's row checks, opened at the
// first and kept for the rest, rather than one per neighbour.
let mut catalog = None;
for job in jobs {
let current = shared
.wanted
@@ -899,7 +902,7 @@ pub(super) fn serve_prefetches(shared: &PrefetchShared, events: &Sender<Prefetch
if !current {
break;
}
if holds_original(&job.cache) {
if holds_original(&job.cache, &mut catalog) {
continue;
}
@@ -926,11 +929,24 @@ pub(super) fn serve_prefetches(shared: &PrefetchShared, events: &Sender<Prefetch
/// Whether the cache already has this original — a row check, not a read,
/// so asking costs nothing and touches no `last_used`.
pub(super) fn holds_original(cache: &CacheContext) -> bool {
///
/// Asked on `held`, which the caller keeps across a batch of these: a
/// connection per question was an open per prefetched neighbour, four of the
/// five a develop landing made. Opened here on first use, and again only if
/// the batch moves to another catalog.
pub(super) fn holds_original(cache: &CacheContext, held: &mut Option<(PathBuf, Catalog)>) -> bool {
let Ok(store) = dr_catalog::Cache::open(&cache.dir, cache.budget) else {
return false;
};
let Ok(catalog) = Catalog::open(&cache.catalog_path) else {
if held
.as_ref()
.is_none_or(|(path, _)| *path != cache.catalog_path)
{
*held = Catalog::open(&cache.catalog_path)
.ok()
.map(|c| (cache.catalog_path.clone(), c));
}
let Some((_, catalog)) = held.as_ref() else {
return false;
};
store.holds_original(catalog.connection(), cache.image)