dtourolle/DarkRoom
1
0
Fork 0
Code Issues 39 Pull Requests Actions Packages Projects Releases 20 Wiki Activity

Merge master into tablet-selection

Browse Source 
Two real conflicts, both from work that landed either side of the same
lines rather than against them.

`lib.rs`: the settings controller was hoisted above the People screen's
wiring, and Android's thumbnail-tier eviction registered itself at the
same point. Independent, so both stay.

`library.rs`: manual collection ordering and burst folding each added a
clause to the same two queries. The scoped range read now carries both —
the folding matters there for one step further on than it does in the
grid, because a collapsed burst is one cell, so an ordinal counted over a
list still holding every frame names a photograph several places away
from the one the user pointed at.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
dtourolle
2026-08-29 22:52:42 +02:00
co-authored by Claude Opus 5
parent 23c74d7063 e027d34b65
commit b9891e2c04
54 changed files with 7566 additions and 166 deletions
Download Patch File Download Diff File Expand all files Collapse all files
core/dr-catalog/src/bursts.rs
+1309
View File
File diff suppressed because it is too large Load Diff
core/dr-catalog/src/lib.rs
+3 -1
View File
@@ -16,6 +16,7 @@
//! - [`collections`] — the collection tree and membership the UI edits
//! - [`keywords`] — the keyword vocabulary and what it is assigned to
//! - [`faces`] — detected faces, the people they belong to, and who said so
//! - [`bursts`] — frames that are one moment, grouped so they judge as one
//! - [`jobs`] — the durable background work queue
//! - [`trash`] — soft delete to a folder, then permanent delete
//! - [`merge`] / [`sync`] — cross-device merging of collections and keywords
@@ -33,6 +34,7 @@ use std::path::Path;
use dr_types::{Availability, ImageId};
use rusqlite::Connection;
pub mod bursts;
pub mod cache;
pub mod collections;
pub mod dedup;
@@ -63,7 +65,7 @@ pub use query::{Query, Sort};
pub use rating::{Judgement, MAX_RATING};
pub use scan::{DirAction, DirState, EntryAction, ScanOutcome};
pub use trash::{TrashedImage, TRASH_DIR};
pub use walk::{ensure_root, scan_root, RootKind, ScanProgress, ScanReport};
pub use walk::{ensure_root, mark_root_offline, scan_root, RootKind, ScanProgress, ScanReport};
/// One row of the library grid.
///
core/dr-catalog/src/schema.rs
+80 -1
View File
@@ -15,7 +15,7 @@ use rusqlite::Connection;
use crate::error::CatalogError;
/// Schema version this build writes and understands.
pub const SCHEMA_VERSION: i64 = 10;
pub const SCHEMA_VERSION: i64 = 11;
/// Apply migrations up to [`SCHEMA_VERSION`].
///
@@ -98,6 +98,13 @@ pub fn migrate(conn: &Connection) -> Result<i64, CatalogError> {
tx.commit()?;
}
if from < 11 {
let tx = conn.unchecked_transaction()?;
tx.execute_batch(V11)?;
tx.pragma_update(None, "user_version", 11)?;
tx.commit()?;
}
Ok(from)
}
@@ -205,6 +212,11 @@ pub fn v1_for_attached(schema_name: &str) -> String {
/// creating it over there would fail on columns that are not there. Nothing is
/// lost by its absence — it exists to make the *grid* page quickly, and the
/// grid never reads across an attachment.
///
/// V11 is excluded on the same grounds and for the plainer reason that a merge
/// has nothing to do with it: burst grouping is rebuilt locally from local
/// signatures, and no code reads a remote catalog's `burst_*` tables. Its
/// `ALTER TABLE` would fail here anyway, being unqualifiable by the rewrite.
pub fn for_attached(schema_name: &str) -> String {
// V10 is `ALTER TABLE`, which the textual rewrite cannot qualify, so its
// columns are spelled out. A remote genuinely older than V10 is a real
@@ -397,6 +409,73 @@ ALTER TABLE people ADD COLUMN ignored INTEGER NOT NULL DEFAULT 0;
ALTER TABLE faces ADD COLUMN crop BLOB;
"#;
/// TRACES: FR-CULL-5
/// Burst grouping: which frames are one moment, and which one stands for it.
///
/// The reasoning behind the grouping itself is in [`crate::bursts`]; what
/// belongs here is why it is stored in three pieces rather than one.
///
/// **`images.perceptual_hash` is a column, not a table**, for the same reason
/// `content_hash` is: it is one number per image, NULL until something has had
/// the pixels in hand, and every query that wants it is already reading the
/// image row. It is local derived state — a rebuilt catalog recomputes it from
/// thumbnails — which is also why it is absent from [`for_attached`], alongside
/// the shadowing and trashing columns V2 through V5 add.
///
/// **`burst_members` is rewritten whole by every pass.** No id of its own: the
/// group is named by the image id of its earliest frame, so a burst that has not
/// changed keeps its name across a regroup and the interface can remember that
/// this one is open. There is no `bursts` table to go with it because a group
/// has no properties beyond its members — inventing a row for it would create an
/// identity that survives the grouping being rebuilt, which is precisely what
/// must not happen.
///
/// **`burst_pick` is the one thing here that is not derived**, and it is a
/// separate table so that rewriting the grouping cannot erase it. A
/// representative stored on `burst_members` would be forgotten every time a
/// frame arrived; the user would be asked the same question after every scan.
/// The same argument `people.ignored` makes in V10, one subsystem over.
///
/// **`burst_expanded` is view state in the catalog**, which is unusual enough to
/// justify. The grid is a window over an ordered query — `LIMIT n OFFSET k` —
/// so what a collapsed burst hides has to be decided by the query, or the row
/// count stops agreeing with the scrollbar and the ordinals a scrub resolves to.
/// Once SQL has to see it, this is where it lives. Nothing else reads it, and it
/// is emptied of stale groups by every pass.
const V11: &str = r#"
-- A 64-bit perceptual signature. Local derived state: NULL until something has
-- decoded the image, recomputed from thumbnails if the catalog is rebuilt, and
-- comparable only to signatures produced by the same build (`bursts`).
ALTER TABLE images ADD COLUMN perceptual_hash INTEGER;
CREATE TABLE burst_members (
-- One burst at most per image: a frame belongs to the moment it was taken
-- in, and nothing else.
image_id INTEGER PRIMARY KEY REFERENCES images(id) ON DELETE CASCADE,
-- The image id of the burst's earliest frame. Not a foreign key by
-- accident: the leader is itself a member, so this genuinely references
-- images(id), and cascading its deletion is right.
burst_id INTEGER NOT NULL REFERENCES images(id) ON DELETE CASCADE,
-- The frame the group collapses to. Exactly one per burst.
representative INTEGER NOT NULL DEFAULT 0
);
-- Counting a burst's frames and listing them are what the grid asks for, once
-- per window; without this both are a scan of every grouped frame in the
-- library.
CREATE INDEX burst_members_burst ON burst_members(burst_id);
-- The user's own choice of representative. User data, never rewritten by a
-- grouping pass -- see the module doc above.
CREATE TABLE burst_pick (
image_id INTEGER PRIMARY KEY REFERENCES images(id) ON DELETE CASCADE
);
-- Bursts the grid is currently showing in full.
CREATE TABLE burst_expanded (
burst_id INTEGER PRIMARY KEY REFERENCES images(id) ON DELETE CASCADE
);
"#;
const V9: &str = r#"
-- TRACES: FR-CULL-8
-- A record that face detection has *run* on an image, distinct from what it
core/dr-catalog/src/walk.rs
+61 -3
View File
@@ -432,7 +432,7 @@ fn bump_generation(conn: &Connection, root: RootId, now: i64) -> Result<i64, Cat
Ok(next)
}
/// TRACES: FR-CAT-9
/// TRACES: FR-CAT-9 | FR-PLAT-AND-2
/// Mark every image under a root as unreachable.
///
/// The other half of FR-CAT-9's distinction: a source *proven absent* may leave
@@ -445,14 +445,38 @@ fn bump_generation(conn: &Connection, root: RootId, now: i64) -> Result<i64, Cat
/// the root now claims something about the files that is no longer known to be
/// true, and only reading the directories again can settle it. Pruning would
/// skip them all and leave a plugged-in library showing as offline forever.
fn mark_root_offline(conn: &Connection, root: RootId) -> Result<(), CatalogError> {
///
/// # Why the ETag goes with the mtime
///
/// The three columns are the same fact told by three kinds of storage: a local
/// directory proves it is unchanged with its mtime and entry count, and a
/// remote one proves it with a propagating ETag (ARCH §6.6). Clearing two of
/// them and leaving the third would disarm the re-listing on exactly the
/// libraries this is most likely to be called for — a remote scan prunes on
/// the ETag alone, so a root that came back would be walked, found unchanged
/// at every level, pruned whole, and left with every row still marked offline
/// and nothing that would ever clear the mark.
///
/// # Public, because losing a root is not only the local walk's business
///
/// This began as the private end of [`scan_root`]'s root-failure branches,
/// which is the only route a library reached through [`Storage`] can take.
/// The application does not currently take that route at all: it opens
/// libraries through `dr-sync`'s connectors, so the discovery happens in a
/// crate that cannot see this one's internals, and the correct response is
/// identical (FR-PLAT-AND-2). Exported rather than reimplemented beside the
/// caller that found out — a second copy would be a second thing to remember
/// when the ETag rule below changes.
///
/// [`Storage`]: dr_plat::Storage
pub fn mark_root_offline(conn: &Connection, root: RootId) -> Result<(), CatalogError> {
let root_id = root.0 as i64;
conn.execute(
"UPDATE images SET availability = ?1 WHERE root_id = ?2 AND availability != ?1",
rusqlite::params![availability_code(Availability::Offline), root_id],
)?;
conn.execute(
"UPDATE folders SET mtime = NULL, entry_count = NULL WHERE root_id = ?1",
"UPDATE folders SET mtime = NULL, entry_count = NULL, etag = NULL WHERE root_id = ?1",
[root_id],
)?;
Ok(())
@@ -995,6 +1019,40 @@ mod tests {
);
}
/// TRACES: FR-PLAT-AND-2 | FR-CAT-9
#[test]
fn marking_a_root_offline_forgets_the_remote_validator_too() {
// The half of the marking that only a remote library can notice, and
// the reason it has to be here rather than beside the connector: a
// remote scan prunes on the propagating ETag alone (ARCH §6.6). Clear
// the local mtime and leave the ETag standing and a library that came
// back would be walked, found unchanged at every level, pruned whole,
// and left with every row still marked offline — with nothing that
// would ever clear the mark, because clearing it is something only a
// listing can do.
//
// Written directly because this module never writes an ETag; it is
// `ui/dr-ui/src/library.rs`'s scan that does, against the same table.
let lib = Library::new("etag-forgotten");
lib.file("2026/IMG.CR3", b"raw");
lib.scan();
lib.conn()
.execute(
"UPDATE folders SET etag = 'e1' WHERE root_id = ?1",
[lib.root.0 as i64],
)
.expect("etag");
assert!(lib.count("SELECT COUNT(*) FROM folders WHERE etag IS NOT NULL") > 0);
mark_root_offline(lib.conn(), lib.root).expect("mark");
assert_eq!(
lib.count("SELECT COUNT(*) FROM folders WHERE etag IS NOT NULL"),
0,
"an unreachable library must be re-listed, not pruned as unchanged"
);
}
#[test]
fn a_root_that_comes_back_is_available_again() {
// The other half: a drive plugged back in must return the library to
core/dr-export/src/lib.rs
+1 -1
View File
@@ -1,4 +1,4 @@
//! TRACES: FR-EXP-1 | FR-EXP-2 | FR-EXP-3 | FR-EXP-4 | FR-EXP-6 | FR-EXP-9
//! TRACES: FR-EXP-1 | FR-EXP-2 | FR-EXP-3 | FR-EXP-4 | FR-EXP-6 | FR-EXP-9 | R3
//! Turning a rendered frame into a file's worth of bytes.
//!
//! # What this crate is, and is not
core/dr-gpu/src/adjust.rs
+38
View File
@@ -1026,6 +1026,44 @@ impl AdjustPass {
h
}
/// TRACES: FR-PLAT-AND-5 | NFR-RES-1
/// Give back every allocation this pass is holding only to be fast again.
///
/// What goes, and why each is safe to lose:
///
/// - **The compiled pipelines**, here and in the detail stage. A pure
/// lookup keyed by structure hash with a compile-on-miss behind it, and
/// unbounded until now — nothing ever removed an entry, so a session
/// that visited enough distinct edit structures accumulated shader
/// objects for the life of the process.
/// - **The detail intermediates**, which are viewport-sized `Rgba16Float`
/// and, as `detail.rs` says of them, grow but never shrink.
/// - **The two output textures.** Dropping these does not take the picture
/// off the screen: whatever was handed to the compositor holds its own
/// reference to the `wgpu::Texture`, so releasing ours only means the
/// *next* render allocates rather than reuses. `ensure_target` already
/// treats an empty slot as "allocate", because that is the state it
/// starts in.
///
/// **`colour_key` must be cleared with them, and this is the part that
/// would bite.** The key is the promise that slot 0 of the detail pool
/// still holds the fused colour result, and it is what lets a sharpening
/// slider skip the colour chain (FR-DEV-3d). Freeing the pool while the
/// promise stood would make the next detail-only render sample a
/// just-allocated texture with nothing in it — a silently wrong frame, not
/// a failure, and one that would only appear on a device under memory
/// pressure.
///
/// What deliberately stays: the demosaiced source is not this pass's to
/// drop, the film tables are set once by a caller that will not be asked
/// again, and the bind group layouts are bytes rather than megabytes.
pub fn release_caches(&mut self) {
self.cache.clear();
self.detail.release_caches();
self.targets = [None, None];
self.colour_key = None;
}
/// How many distinct pipelines are compiled. Exposed for tests asserting
/// that slider movement does not recompile.
pub fn cached_pipelines(&self) -> usize {
core/dr-gpu/src/detail.rs
+31
View File
@@ -157,6 +157,24 @@ impl Intermediates {
self.allocations += 1;
}
}
/// TRACES: FR-PLAT-AND-5
/// Drop the pool, leaving it as [`Intermediates::new`] left it.
///
/// The size is reset along with the slots, not merely because it is tidy:
/// [`Self::ensure`] only refills when the count is short *or* the size
/// differs, so a pool cleared while still claiming its old dimensions is
/// indistinguishable from one that never held anything — which is fine
/// here, and would stop being fine the moment `ensure` grew a fast path
/// that trusted the stored size. `allocations` deliberately keeps
/// counting: it exists so a test can see textures being made, and a
/// counter reset on eviction would hide a reallocation storm rather than
/// report one.
fn release(&mut self) {
self.slots.clear();
self.width = 0;
self.height = 0;
}
}
/// Runs the detail stage.
@@ -526,6 +544,19 @@ impl DetailRunner {
self.cache.len()
}
/// TRACES: FR-PLAT-AND-5
/// Give back everything this stage is only holding to be fast.
///
/// Both pools and the pipeline cache. Nothing here is state: a pool slot
/// is re-created by the next [`Intermediates::ensure`] and a pipeline by
/// the next compile-on-miss, so the only cost of this call is the work of
/// doing both again.
pub(crate) fn release_caches(&mut self) {
self.cache.clear();
self.pool.release();
self.reduced.release();
}
/// How many intermediate textures have been allocated since this pass was
/// created. For tests — see [`crate::MaskPass::allocations`] for the
/// regression this shape of counter exists to catch.
core/dr-gpu/src/focus.rs
+1007
View File
File diff suppressed because it is too large Load Diff
core/dr-gpu/src/lib.rs
+3
View File
@@ -1,3 +1,4 @@
//! TRACES: NFR-PORT-2
//! GPU device and compute for DarkRoom.
//!
//! In v0.1 this exists to prove one thing: a compute shader can write a
@@ -21,6 +22,7 @@ mod adjust;
mod demosaic;
mod detail;
mod error;
mod focus;
mod histogram;
mod mask;
mod readback;
@@ -33,6 +35,7 @@ pub use adjust::AdjustPass;
pub use demosaic::{DemosaicedImage, Demosaicer};
pub use detail::INTERMEDIATE_FORMAT as DETAIL_INTERMEDIATE_FORMAT;
pub use error::GpuError;
pub use focus::{FocusPeakPass, FocusPeaking, PeakColour, PeakSensitivity};
// Renamed on the way out: `BINS` says enough inside `histogram`, and nothing
// at all at a crate root shared with demosaic and segmentation.
pub use histogram::{Histogram, HistogramPass, BINS as HISTOGRAM_BINS};
core/dr-gpu/src/shaders/focus_peak.wgsl
+141
View File
@@ -0,0 +1,141 @@
// TRACES: FR-CULL-3 | NFR-P14
// Marking what is sharp, in a layer laid over the frame rather than into it.
//
// # Why the top octave, and not a gradient
//
// The obvious detector is a gradient magnitude — Sobel, or a central
// difference — and it is the wrong one, for a reason that decides whether the
// overlay is useful at all. A gradient answers "is there an edge here", and a
// defocused edge is still an edge: blur a 100-code step with a two-pixel
// Gaussian and the peak gradient is still around 20 codes per pixel, larger
// than a genuinely sharp edge across a low-contrast texture. Peaking built on
// gradients lights up the out-of-focus background of every portrait ever
// taken, which is the frame it exists to reject.
//
// What separates sharp from soft is *scale*, not amplitude. Defocus is a
// low-pass: it removes the top octave and leaves everything below it intact.
// So the detector is a high-pass — this pixel against the mean of its eight
// neighbours, a discrete Laplacian — which by construction responds only to
// the frequencies defocus destroys.
//
// The arithmetic, on a one-dimensional step of height D:
//
// | profile | abs(centre - mean of 8) |
// |--------------------------|-------------------------|
// | hard step, 1 px | 0.375 D |
// | Gaussian blur, sigma 1 | ~0.10 D |
// | Gaussian blur, sigma 2 | ~0.03 D |
// | linear ramp, any slope | 0 |
//
// The ramp row is the property being bought: the smooth luminance falloff
// across an out-of-focus highlight scores zero however bright it is.
//
// # Why luma, and why the histogram's luma
//
// One channel rather than three, because a colour edge carrying no luminance
// difference is both rare and, at the acuity an overlay is read at, invisible.
// The weights are `histogram.wgsl`'s 54/183/19 over 256 — the same Rec.709
// weighting on the same encoded values — so the two instruments in this
// application agree about what "luma" means. Two definitions of brightness in
// one panel is the kind of disagreement nobody finds until it has already
// misled someone.
//
// # Why the frame is read where it is encoded, and not in linear light
//
// This runs on the output of the display transform, on encoded values, and
// that is deliberate: a fixed difference in sRGB code values is roughly
// equally visible wherever it sits in the range, which is what a transfer
// curve is for. Measured in linear light the same detector would need a
// threshold that varied with exposure, and a shadow texture the photographer
// can plainly see would score a hundredth of the identical texture in the
// highlights. The encoding has already done the normalisation, so the
// threshold is one number.
//
// # Why this writes a layer and not the picture
//
// The frame the compositor is handed is also what the histogram counts and
// what an export renders (`app.slint`, on the region overlay: a diagnostic
// "must not reach the histogram, an export, or the texture the develop pass
// hands the compositor"). So the marks go in their own texture — transparent
// everywhere except where something is in focus — and the compositor blends
// them. Nothing about the photograph changes, and the peaking overlay cannot
// leak into a measurement or a file.
//
// Alpha is written as exactly 0 or exactly 1, never between. The importing
// compositor's convention for whether colour arrives premultiplied is not
// something this shader can see, and at those two values the two conventions
// agree — which is a cheaper guarantee than being right about which one it is.
struct Params {
width: u32,
height: u32,
// Luma difference at which a pixel is called in focus. See
// `PeakSensitivity::threshold` for where the three values come from.
threshold: f32,
// std140 rounds the scalar block up to 16 bytes before the vec4; named so
// the Rust struct's padding is visibly the same shape.
pad_0: u32,
// The mark's colour, fully saturated. Its alpha is ignored — see above.
marker: vec4<f32>,
}
@group(0) @binding(0) var frame: texture_2d<f32>;
@group(0) @binding(1) var<uniform> params: Params;
@group(0) @binding(2) var marks: texture_storage_2d<rgba8unorm, write>;
/// Rec.709 luma of an encoded triple, weighted exactly as `histogram.wgsl`
/// weights it. 54 + 183 + 19 is 256, so the weights sum to unity.
fn luma(c: vec3<f32>) -> f32 {
return dot(c, vec3<f32>(54.0, 183.0, 19.0) / 256.0);
}
/// A neighbour, with the frame edge held rather than wrapped.
///
/// Clamping duplicates the edge pixel into the missing half of the
/// neighbourhood, which pulls the mean towards the centre and so biases the
/// response *down* on the outermost row and column. That is the right
/// direction to be wrong in: the failure is a missing mark at the frame edge,
/// where nobody is judging focus, rather than a false mark produced by
/// folding the opposite side of the picture into the kernel.
fn neighbour(x: i32, y: i32) -> f32 {
let cx = clamp(x, 0i, i32(params.width) - 1i);
let cy = clamp(y, 0i, i32(params.height) - 1i);
return luma(textureLoad(frame, vec2<i32>(cx, cy), 0).rgb);
}
// 8x8, matching the detail stage's dispatch. Each texel is loaded by nine
// invocations and no workgroup-memory tile is built to avoid that: at viewport
// resolution the reads are perfectly coherent and the texture cache serves
// eight of the nine. The budget is NFR-P14's 100 ms against a dispatch
// measured in tenths of a millisecond, so there is nothing here worth the
// complexity of a tiled load.
@compute @workgroup_size(8, 8, 1)
fn main(@builtin(global_invocation_id) gid: vec3<u32>) {
if (gid.x >= params.width || gid.y >= params.height) {
return;
}
let x = i32(gid.x);
let y = i32(gid.y);
// The eight neighbours, centre excluded. Excluded rather than folded in
// because it makes the response readable: `abs(c - mean8)` is the height
// of this pixel above its surroundings in the same units as the step it
// sits on, so the threshold can be quoted as a luma difference rather than
// as eight-ninths of one.
var sum = 0.0;
for (var dy = -1; dy <= 1; dy = dy + 1) {
for (var dx = -1; dx <= 1; dx = dx + 1) {
if (dx != 0 || dy != 0) {
sum = sum + neighbour(x + dx, y + dy);
}
}
}
let centre = luma(textureLoad(frame, vec2<i32>(x, y), 0).rgb);
let response = abs(centre - sum / 8.0);
if (response >= params.threshold) {
textureStore(marks, vec2<i32>(x, y), vec4<f32>(params.marker.rgb, 1.0));
} else {
textureStore(marks, vec2<i32>(x, y), vec4<f32>(0.0, 0.0, 0.0, 0.0));
}
}
core/dr-gpu/tests/detail_stage.rs
+53
View File
@@ -413,3 +413,56 @@ fn an_empty_chain_falls_through_to_the_ordinary_render() {
assert_eq!(pass.detail_dispatches(), 0);
assert_eq!(pass.detail_allocations(), 0);
}
/// TRACES: FR-PLAT-AND-5
#[test]
fn eviction_gives_the_pools_back_without_changing_a_pixel() {
// The half of memory-pressure eviction that cannot be checked by looking
// at a counter. `release_caches` frees the detail pool, and slot 0 of that
// pool is where the fused colour result lives between frames — so the
// render after an eviction has to notice that the promise recorded in
// `colour_key` no longer holds and run the colour chain again.
//
// Leave the key standing and this test does not error: it draws. It draws
// whatever a freshly-allocated texture happens to contain, which is the
// failure worth building a test around, because on a device it would
// appear only under memory pressure and only as a wrong-looking photograph.
let Some(ctx) = ctx() else { return };
const SIZE: u32 = 48;
let source = step_edge(&ctx, SIZE);
let mut pass = AdjustPass::new(&ctx);
let mut graph = EditGraph::with_detail_probe();
graph.set_param(PROBE, RADIUS, 0.05);
let before = render(&ctx, &mut pass, &graph, &source, SIZE);
assert!(
pass.cached_pipelines() > 0,
"the colour pass compiled something"
);
assert!(
pass.cached_detail_pipelines() > 0,
"so did the detail stage"
);
let allocations = pass.detail_allocations();
assert!(allocations > 0, "and the pool holds textures");
pass.release_caches();
assert_eq!(pass.cached_pipelines(), 0);
assert_eq!(pass.cached_detail_pipelines(), 0);
// The same edit at the same size. Nothing about the picture changed, so
// nothing about the pixels may change either — only what it cost.
let after = render(&ctx, &mut pass, &graph, &source, SIZE);
assert_eq!(before.len(), after.len());
for (i, (a, b)) in before.iter().zip(&after).enumerate() {
assert!(
a.abs_diff(*b) <= 1,
"byte {i}: {a} before eviction, {b} after — the colour chain did \
not re-run, so this frame is reading an empty intermediate"
);
}
assert!(
pass.detail_allocations() > allocations,
"the pool was rebuilt, which is the evidence it was really given back"
);
}
core/dr-pipeline/src/graph.rs
+1
View File
@@ -1,3 +1,4 @@
//! TRACES: FR-DEV-1
//! The edit graph — an ordered set of operations (ARCH §3.4).
//!
//! CPU-side state, deliberately. The GPU device can be lost and rebuilt at any
core/dr-pipeline/src/lib.rs
+2 -1
View File
@@ -1,3 +1,4 @@
//! TRACES: R3
//! The develop pipeline — operations, descriptors, and shader composition.
//!
//! # What this crate is
@@ -62,7 +63,7 @@ pub use operation::{
compose, compose_with_framing, Affects, ComposedShader, Helper, Invalidation, Operation,
OutputMode, Uniform, BASE_CURVE_POINTS, BASE_CURVE_UNIFORM_OFFSET, RESERVED_UNIFORM_FIELDS,
};
pub use preset::{Preset, Scope};
pub use preset::{LibraryParseError, NameError, Preset, PresetLibrary, Scope};
pub use sidecar::{Sidecar, Version};
pub use spot::{Spot, SpotMode, SpotSet};
pub use state::{EditState, FilmRebake, FilmRef};
core/dr-pipeline/src/preset.rs
+516
View File
@@ -39,6 +39,8 @@
//! is the whole claim the action makes.
use std::collections::BTreeMap;
use std::fmt;
use std::fmt::Write as _;
use crate::descriptor::{OpId, ParamId};
use crate::graph::EditGraph;
@@ -240,6 +242,328 @@ pub(crate) fn resolve(graph: &EditGraph, op: &str, param: &str) -> Option<(OpId,
Some((cap.id, p.id))
}
// ---------------------------------------------------------------------------
// Named presets
// ---------------------------------------------------------------------------
/// TRACES: FR-DEV-6
/// Format version of a preset library file.
///
/// Present where `settings.json` has no version field, and the difference is
/// not an inconsistency. Settings are a flat bag of `#[serde(default)]`
/// fields, so an older file is *missing* keys rather than wrong about them and
/// additive change needs no version. This file has structure — blocks, and a
/// name carried in a block header — and a change to what a block *means* is
/// not something a reader can detect by noticing an absent key.
pub const LIBRARY_FORMAT_VERSION: u32 = 1;
/// The file extension for a DarkRoom preset library.
pub const LIBRARY_EXTENSION: &str = "drpl";
/// Why a name was refused.
///
/// A closed set rather than a string, so the interface can say something
/// specific about each and the message is not written here — this crate
/// depends on nothing and has no business holding user-facing prose.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum NameError {
/// Empty, or nothing but whitespace.
Empty,
/// Contains a character the block header cannot carry: `[`, `]`, or a
/// line break.
///
/// A round-trip constraint rather than a matter of taste. The header is
/// `[preset <name>]`, so a `]` inside the name would make the file parse
/// back as a *different* library, and a newline would make it parse back
/// as two.
Unrepresentable,
}
/// TRACES: FR-DEV-6
/// A set of named presets, as stored.
///
/// # Why one file rather than one file per preset
///
/// A preset per file makes the name a *path*, and every name then has to
/// survive a filesystem: a `/` becomes a directory, a name that differs only
/// in case collides on one platform and not another, and renaming becomes two
/// operations that can half-fail. Here the name is a key in a document, so
/// renaming is a map operation, deleting cannot leave an orphan, and the whole
/// library is written atomically by the same tmp-and-rename the settings store
/// uses.
///
/// The cost is that the file is rewritten whole on every change. A preset is a
/// few dozen floats and a photographer has tens of them, not thousands, so the
/// file is kilobytes; the trade would look different at a scale this is not.
///
/// # Why the sidecar's shape rather than JSON
///
/// A preset *is* the non-default half of a version (see the module note), so
/// the lines here are the lines a sidecar carries, keyed the same way. That
/// makes the two files diffable against each other and lets someone debugging
/// an edit paste a block from one into the other. It also keeps this crate
/// dependency-free, which is the property that lets it be tested without a
/// device (ARCH §6.5a).
///
/// # Ordering
///
/// By name, so the same library always writes the same bytes and a caller may
/// compare content to decide whether a write is needed — the same
/// determinism [`Sidecar::to_text`](crate::Sidecar::to_text) offers, for the
/// same reason.
#[derive(Debug, Clone, PartialEq, Default)]
pub struct PresetLibrary {
presets: BTreeMap<String, Preset>,
/// Lines inside a `[preset]` block that were not `op.param = float`.
///
/// Keyed by preset name and written back verbatim, so a build that
/// predates whatever wrote them round-trips the file without discarding
/// it. Unknown *parameters* need no such machinery: [`Preset`] holds
/// whatever keys it was given and resolves them against the descriptors
/// only at apply time, so a parameter this build has never heard of
/// survives simply by being stored.
unknown: BTreeMap<String, Vec<String>>,
}
impl PresetLibrary {
/// Whether a name is storable, and why not if it is not.
///
/// Leading and trailing whitespace is trimmed rather than refused — it is
/// almost always a stray keystroke, and refusing it would mean a dialogue
/// about a space.
pub fn check_name(name: &str) -> Result<String, NameError> {
let name = name.trim();
if name.is_empty() {
return Err(NameError::Empty);
}
if name.contains([']', '[', '\n', '\r']) {
return Err(NameError::Unrepresentable);
}
Ok(name.to_string())
}
/// Store `preset` under `name`, replacing any preset already there.
///
/// Returns whether something was replaced.
///
/// Replacing rather than refusing, with [`Self::contains`] beside it for a
/// caller that wants to ask first: whether overwriting needs a
/// confirmation is a question about the interface, and answering it here
/// would force every caller into the same answer.
///
/// An *empty* preset is stored like any other. A neutral edit is a real
/// thing to save — applying it returns an image to default, which is the
/// fastest "undo everything on these forty frames" there is — and the
/// module note above is the same argument made about the clipboard.
pub fn insert(&mut self, name: &str, preset: Preset) -> Result<bool, NameError> {
let name = Self::check_name(name)?;
let replaced = self.presets.insert(name, preset).is_some();
Ok(replaced)
}
/// The preset stored under `name`.
pub fn get(&self, name: &str) -> Option<&Preset> {
self.presets.get(name)
}
/// Whether a preset is stored under `name`.
pub fn contains(&self, name: &str) -> bool {
self.presets.contains_key(name)
}
/// Remove the preset stored under `name`, reporting whether there was one.
pub fn remove(&mut self, name: &str) -> bool {
self.unknown.remove(name);
self.presets.remove(name).is_some()
}
/// Rename `from` to `to`.
///
/// `Ok(false)` means there was nothing called `from` — not an error, since
/// the caller may be acting on a list another window has already changed.
/// Renaming onto an existing name replaces it, for the same reason
/// [`Self::insert`] does.
pub fn rename(&mut self, from: &str, to: &str) -> Result<bool, NameError> {
let to = Self::check_name(to)?;
let Some(preset) = self.presets.remove(from) else {
return Ok(false);
};
if let Some(unknown) = self.unknown.remove(from) {
self.unknown.insert(to.clone(), unknown);
}
self.presets.insert(to, preset);
Ok(true)
}
/// Every stored name, in the order they are written.
pub fn names(&self) -> impl Iterator<Item = &str> {
self.presets.keys().map(String::as_str)
}
/// Every stored preset with its name, in the order they are written.
pub fn iter(&self) -> impl Iterator<Item = (&str, &Preset)> {
self.presets.iter().map(|(n, p)| (n.as_str(), p))
}
/// How many presets are stored.
pub fn len(&self) -> usize {
self.presets.len()
}
/// Whether nothing is stored.
pub fn is_empty(&self) -> bool {
self.presets.is_empty()
}
/// Serialise to the on-disk form.
///
/// Deterministic, like the sidecar's: the same library always produces the
/// same bytes.
pub fn to_text(&self) -> String {
let mut out = format!("drpl {LIBRARY_FORMAT_VERSION}\n");
for (name, preset) in &self.presets {
let _ = write!(out, "\n[preset {name}]\n");
for ((op, param), value) in preset.params() {
let _ = writeln!(out, "{op}.{param} = {}", format_value(*value));
}
for line in self.unknown.get(name).into_iter().flatten() {
let _ = writeln!(out, "{line}");
}
}
out
}
/// Parse the on-disk form.
///
/// Tolerant on the same terms as the sidecar's parser, and for a weaker
/// version of the same reason: a preset library is not the authoritative
/// store an edit lives in, but it is still work the user did by hand, and
/// one bad line must cost that line rather than the collection. The only
/// hard failures are a file that is not a preset library at all and one
/// written by a newer build, where continuing would mean guessing.
pub fn parse(text: &str) -> Result<Self, LibraryParseError> {
let mut lines = text.lines();
let header = lines.next().unwrap_or_default().trim();
let Some(version) = header.strip_prefix("drpl ") else {
return Err(LibraryParseError::NotALibrary);
};
match version.trim().parse::<u32>() {
Ok(v) if v <= LIBRARY_FORMAT_VERSION => {}
Ok(v) => return Err(LibraryParseError::UnsupportedVersion(v)),
Err(_) => return Err(LibraryParseError::NotALibrary),
}
let mut library = Self::default();
let mut current: Option<String> = None;
let mut params: BTreeMap<(String, String), f32> = BTreeMap::new();
for line in lines {
let line = line.trim();
if line.is_empty() || line.starts_with('#') {
continue;
}
if let Some(head) = line
.strip_prefix("[preset ")
.and_then(|l| l.strip_suffix(']'))
{
if let Some(name) = current.take() {
library.presets.insert(name, Preset::from_params(params));
params = BTreeMap::new();
}
// A name the writer should never have produced is dropped
// rather than taken: accepting it would mean writing a file
// back out that no longer parses as this one.
match Self::check_name(head) {
Ok(name) => current = Some(name),
Err(_) => {
log::warn!("preset library: unusable preset name {head:?}; skipping");
current = None;
}
}
continue;
}
let Some(name) = current.clone() else {
log::warn!("preset library: line outside any preset: {line}");
continue;
};
match line.split_once('=') {
Some((key, value)) => {
let key = key.trim();
let value = value.trim();
match (key.split_once('.'), value.parse::<f32>()) {
(Some((op, param)), Ok(v)) if !op.is_empty() && !param.is_empty() => {
params.insert((op.to_string(), param.to_string()), v);
}
_ => library
.unknown
.entry(name)
.or_default()
.push(line.to_string()),
}
}
None => library
.unknown
.entry(name)
.or_default()
.push(line.to_string()),
}
}
if let Some(name) = current {
library.presets.insert(name, Preset::from_params(params));
}
// A block whose every line was unreadable still produced a preset, and
// an unknown block belonging to no preset would be written back into
// whichever one happened to sort first. Drop the orphans.
library
.unknown
.retain(|k, _| library.presets.contains_key(k));
Ok(library)
}
}
/// Format a value the way the sidecar does — no trailing `.0`, no exponent —
/// so the two files stay comparable line for line.
fn format_value(v: f32) -> String {
let mut s = format!("{v:.6}");
if s.contains('.') {
s = s.trim_end_matches('0').trim_end_matches('.').to_string();
}
if s == "-0" {
s = "0".to_string();
}
s
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum LibraryParseError {
/// The header line was missing or not a `drpl` header.
NotALibrary,
/// Written by a newer build, in a format this one cannot read.
UnsupportedVersion(u32),
}
impl fmt::Display for LibraryParseError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Self::NotALibrary => f.write_str("not a DarkRoom preset library"),
Self::UnsupportedVersion(v) => {
write!(
f,
"preset library format version {v} is newer than this build"
)
}
}
}
}
impl std::error::Error for LibraryParseError {}
#[cfg(test)]
mod tests {
use super::*;
@@ -563,4 +887,196 @@ mod tests {
.collect();
assert_eq!(excluded, vec![framing::ID.0]);
}
// -----------------------------------------------------------------------
// Named presets
// -----------------------------------------------------------------------
fn named() -> PresetLibrary {
let mut lib = PresetLibrary::default();
lib.insert("Warm portrait", Preset::capture(&edited()))
.unwrap();
lib.insert("Neutral", Preset::default()).unwrap();
lib
}
#[test]
fn a_library_round_trips_through_its_text_form() {
let lib = named();
let back = PresetLibrary::parse(&lib.to_text()).unwrap();
assert_eq!(back, lib);
}
#[test]
fn the_same_library_always_writes_the_same_bytes() {
// What lets a caller skip a write by comparing content. Built in the
// opposite order to `named()` so insertion order cannot be what makes
// this pass.
let mut other = PresetLibrary::default();
other.insert("Neutral", Preset::default()).unwrap();
other
.insert("Warm portrait", Preset::capture(&edited()))
.unwrap();
assert_eq!(other.to_text(), named().to_text());
}
#[test]
fn a_neutral_preset_is_storable_and_survives_the_round_trip() {
// The empty preset is the "clear these forty frames" action, so it has
// to be a real entry rather than an absence — and a block with no
// lines under it has to parse back as a preset rather than vanish.
let back = PresetLibrary::parse(&named().to_text()).unwrap();
assert_eq!(back.get("Neutral"), Some(&Preset::default()));
}
#[test]
fn an_unreadable_line_costs_that_line_and_not_the_library() {
let text = format!(
"drpl {LIBRARY_FORMAT_VERSION}\n\n[preset Keep]\nexposure.exposure = 0.5\n\
this line is not a setting\nsaturation.amount = not a number\n"
);
let lib = PresetLibrary::parse(&text).unwrap();
let preset = lib.get("Keep").expect("the preset survived");
assert_eq!(
preset.params().get(&("exposure".into(), "exposure".into())),
Some(&0.5)
);
}
#[test]
fn lines_this_build_cannot_read_are_written_back_untouched() {
// The sidecar's version-skew promise, applied here: a build running
// behind must not silently strip what a newer one wrote.
let text = format!(
"drpl {LIBRARY_FORMAT_VERSION}\n\n[preset Keep]\nexposure.exposure = 0.5\n\
something_new_entirely\n"
);
let out = PresetLibrary::parse(&text).unwrap().to_text();
assert!(out.contains("something_new_entirely"), "{out}");
}
#[test]
fn an_unknown_parameter_survives_without_any_machinery_for_it() {
// `Preset` stores whatever keys it is given and resolves them against
// the descriptors only at apply time, so a parameter from a newer
// build needs no preservation path of its own.
let text = format!(
"drpl {LIBRARY_FORMAT_VERSION}\n\n[preset Keep]\nnot_an_op.not_a_param = 0.25\n"
);
let out = PresetLibrary::parse(&text).unwrap().to_text();
assert!(out.contains("not_an_op.not_a_param = 0.25"), "{out}");
}
#[test]
fn a_file_from_a_newer_build_is_refused_rather_than_guessed_at() {
let text = format!("drpl {}\n", LIBRARY_FORMAT_VERSION + 1);
assert_eq!(
PresetLibrary::parse(&text),
Err(LibraryParseError::UnsupportedVersion(
LIBRARY_FORMAT_VERSION + 1
))
);
}
#[test]
fn something_that_is_not_a_preset_library_is_refused() {
assert_eq!(
PresetLibrary::parse("drsc 1\n\n[version abc]\n"),
Err(LibraryParseError::NotALibrary)
);
assert_eq!(
PresetLibrary::parse(""),
Err(LibraryParseError::NotALibrary)
);
}
#[test]
fn a_name_that_would_not_parse_back_is_refused() {
// Round-trip safety, not taste: a `]` would close the header early and
// the file would read back as a different library.
let mut lib = PresetLibrary::default();
assert_eq!(
lib.insert("bracket] inside", Preset::default()),
Err(NameError::Unrepresentable)
);
assert_eq!(
lib.insert("two\nlines", Preset::default()),
Err(NameError::Unrepresentable)
);
assert_eq!(lib.insert(" ", Preset::default()), Err(NameError::Empty));
}
#[test]
fn surrounding_whitespace_is_trimmed_rather_than_refused() {
let mut lib = PresetLibrary::default();
lib.insert(" Warm ", Preset::default()).unwrap();
assert!(lib.contains("Warm"));
}
#[test]
fn saving_over_a_name_replaces_it_and_says_so() {
let mut lib = named();
assert_eq!(lib.insert("Warm portrait", Preset::default()), Ok(true));
assert_eq!(lib.insert("Brand new", Preset::default()), Ok(false));
assert_eq!(lib.get("Warm portrait"), Some(&Preset::default()));
}
#[test]
fn renaming_moves_the_preset_and_leaves_nothing_behind() {
let mut lib = named();
let before = lib.get("Warm portrait").cloned().unwrap();
assert_eq!(lib.rename("Warm portrait", "Cool portrait"), Ok(true));
assert!(!lib.contains("Warm portrait"));
assert_eq!(lib.get("Cool portrait"), Some(&before));
}
#[test]
fn renaming_something_that_is_gone_is_not_an_error() {
// Another window may have deleted it since this list was drawn.
let mut lib = named();
assert_eq!(lib.rename("Never existed", "Whatever"), Ok(false));
}
#[test]
fn deleting_reports_whether_there_was_anything_to_delete() {
let mut lib = named();
assert!(lib.remove("Neutral"));
assert!(!lib.remove("Neutral"));
assert_eq!(lib.len(), 1);
}
#[test]
fn a_stored_preset_applies_exactly_as_a_pasted_one_does() {
// The whole point of sharing one representation: a named preset is not
// a second kind of thing with a second apply path.
let lib = named();
let stored = PresetLibrary::parse(&lib.to_text()).unwrap();
let preset = stored.get("Warm portrait").unwrap();
let mut target = EditGraph::default_chain();
preset.apply(&mut target, Scope::Adjustments);
assert_eq!(target.param(exposure::ID, exposure::EXPOSURE), Some(0.75));
// The target keeps its own framing on the default scope.
assert_eq!(target.param(framing::ID, framing::ANGLE), Some(0.0));
}
#[test]
fn a_stored_preset_amends_a_sidecar_without_a_graph() {
// The batch path: applying to forty images must not build forty
// graphs, so this is the call the library's batch apply makes.
let lib = named();
let preset = lib.get("Warm portrait").unwrap();
let mut params: BTreeMap<(String, String), f32> = BTreeMap::new();
params.insert(("framing".into(), "angle".into()), 5.0);
params.insert(("exposure".into(), "exposure".into()), -1.0);
preset.amend(&mut params, Scope::Adjustments);
assert_eq!(
params.get(&("exposure".into(), "exposure".into())),
Some(&0.75)
);
// Out of scope, so the target's own crop is untouched.
assert_eq!(params.get(&("framing".into(), "angle".into())), Some(&5.0));
}
}
core/dr-pipeline/src/sidecar.rs
+1
View File
@@ -1,3 +1,4 @@
//! TRACES: FR-DEV-1
//! Sidecar serialisation — the edit graph as durable, mergeable data.
//!
//! # Generic, for the same reason the UI is generic
core/dr-sync-folder/src/borrow.rs
+1 -1
View File
@@ -1,4 +1,4 @@
// TRACES: FR-NC-6c | FR-NC-6a
// TRACES: FR-NC-6c | FR-NC-6a | FR-NC-6d
//! Hydrating a file for as long as it is needed, and no longer.
//!
//! A pass over a library — thumbnails, face indexing — needs each photograph's
core/dr-sync-folder/src/lib.rs
+21 -6
View File
@@ -1,4 +1,4 @@
// TRACES: FR-NC-13 | FR-NC-12
// TRACES: FR-NC-13 | FR-NC-12 | FR-NC-6d
//! A library that is just a directory.
//!
//! The second [`RemoteBackend`], and the one that exists to prove the first
@@ -216,10 +216,17 @@ impl std::fmt::Debug for FolderBackend {
impl FolderBackend {
/// Open the folder at `root`.
///
/// The directory must exist now. It may stop existing later — a drive
/// unplugged, a mount dropped — and that surfaces per-operation as
/// [`RemoteError::Network`], which is what puts the app into offline mode
/// and leaves the catalog readable, exactly as a dead server does.
/// The directory must exist now, and not existing is
/// [`RemoteError::RootUnavailable`] — the library folder could not be
/// opened, which is the whole of what this knows. A drive unplugged
/// between sessions and a path typed wrongly at setup are the same
/// observation from here, and both are answered the same way: keep the
/// catalog, say which folder, and offer it again (FR-PLAT-AND-2).
///
/// A mount dropped *during* a session surfaces per-operation as
/// [`RemoteError::Network`] instead, which is what puts the app into
/// offline mode and leaves the catalog readable, exactly as a dead server
/// does.
pub fn new(root: impl Into<PathBuf>) -> Result<Self, RemoteError> {
Self::with_vfs(root, Arc::new(NoVfs))
}
@@ -232,7 +239,15 @@ impl FolderBackend {
pub fn with_vfs(root: impl Into<PathBuf>, vfs: Arc<dyn Vfs>) -> Result<Self, RemoteError> {
let root = root.into();
if !root.is_dir() {
return Err(RemoteError::Configuration(format!(
// TRACES: FR-PLAT-AND-2
// Not `Configuration`, which is where this lived while there was
// nothing better. The distinction that matters is not "was the
// account written wrongly" — which nothing here can know — but
// "can this library be opened", and a caller that knows the
// library was working yesterday can act on the second answer:
// mark what it holds as offline rather than deleting it, and ask
// for the folder again (FR-CAT-9).
return Err(RemoteError::RootUnavailable(format!(
"{} is not a folder",
root.display()
)));
core/dr-sync-folder/src/tests.rs
+13 -4
View File
@@ -48,13 +48,22 @@ fn names(entries: &[RemoteEntry]) -> Vec<String> {
// --- opening --------------------------------------------------------------
/// TRACES: FR-PLAT-AND-2
#[test]
fn a_missing_folder_is_a_configuration_error_not_a_network_one() {
// It must not put the app into offline mode: nothing was unreachable, the
// account names somewhere that is not a folder.
fn a_missing_folder_is_an_unavailable_root_not_a_network_failure() {
// Still not offline mode — nothing was unreachable over a wire, and
// reporting it as a network failure would tell the user to wait for a
// connection that is working.
//
// `RootUnavailable` rather than `Configuration`, because the caller that
// has to act on this is the one whose library worked yesterday: an
// ejected card is indistinguishable from a mistyped path here, and only
// the first of those has a catalog full of ratings to protect.
let err = FolderBackend::new("/definitely/not/here").unwrap_err();
assert!(matches!(err, RemoteError::Configuration(_)), "{err:?}");
assert!(matches!(err, RemoteError::RootUnavailable(_)), "{err:?}");
assert!(err.indicates_lost_root());
assert!(!err.indicates_offline());
assert!(err.to_string().contains("/definitely/not/here"), "{err}");
}
// --- listing --------------------------------------------------------------
core/dr-sync-folder/src/vfs.rs
+1 -1
View File
@@ -1,4 +1,4 @@
// TRACES: FR-NC-6c
// TRACES: FR-NC-6c | FR-NC-6d
//! Virtual-filesystem conventions layered over a directory.
//!
//! A sync client in virtual-files mode leaves a *placeholder* where a file is
core/dr-sync-nextcloud/src/lib.rs
+1
View File
@@ -1,3 +1,4 @@
//! TRACES: R6 | NFR-SEC-3
//! Nextcloud connector.
//!
//! One of two [`RemoteBackend`] implementations, registered through
core/dr-sync-nextcloud/src/provider.rs
+1 -1
View File
@@ -1,4 +1,4 @@
// TRACES: FR-NC-12 | FR-NC-1
// TRACES: FR-NC-12 | FR-NC-1 | NFR-SEC-3
//! Registering Nextcloud as a storage backend.
//!
//! The account model this connector used to own now lives in
core/dr-sync/src/error.rs
+62 -8
View File
@@ -61,14 +61,18 @@ pub enum RemoteError {
/// connector for.
///
/// **Not a network failure and not an auth failure**, which is why it is
/// its own variant. A folder library whose directory has been unmounted,
/// or an account naming a backend a cut-down build was not compiled with,
/// produces a request that never leaves the process — reporting either as
/// `Network` would put the app into offline mode and tell the user their
/// connection is down, and reporting them as `AuthFailed` would send them
/// to re-enter a credential that is fine. The message names what is wrong
/// with the configuration, because that is the only thing that will fix
/// it.
/// its own variant. An account naming a backend a cut-down build was not
/// compiled with, or a path that would leave the library folder, produces
/// a request that never leaves the process — reporting either as `Network`
/// would put the app into offline mode and tell the user their connection
/// is down, and reporting them as `AuthFailed` would send them to re-enter
/// a credential that is fine. The message names what is wrong with the
/// configuration, because that is the only thing that will fix it.
///
/// A folder library whose directory is not there was once reported here
/// too, and is now [`RootUnavailable`](Self::RootUnavailable): it is not
/// something wrong with the configuration, it is the library being gone,
/// and only the second of those has a catalog to protect.
#[error("account misconfigured: {0}")]
Configuration(String),
@@ -105,6 +109,43 @@ pub enum RemoteError {
#[error("operation cancelled")]
Cancelled,
/// TRACES: FR-PLAT-AND-2 | FR-CAT-9
/// The library root itself could not be opened.
///
/// **The one failure that is about the library rather than about a file in
/// it**, and it is a separate variant because every other classification
/// of it is wrong in a way that costs the user something:
///
/// - As [`NotFound`](Self::NotFound) it is indistinguishable from a folder
/// deleted between listing its parent and reaching it, which the walk
/// correctly steps over — so a whole library going away is reported as a
/// successful scan that found nothing.
/// - As [`PermissionDenied`](Self::PermissionDenied) it inherits a message
/// about Nextcloud share permissions and sidecar writes, which is
/// accurate for the case it was written for and nonsense for a tree
/// grant the user revoked in system settings.
/// - As [`Network`](Self::Network) it would claim the connection is down,
/// which is a promise that waiting will fix it.
///
/// Today this is a Nextcloud root that answers 404 or 403 — deleted, or a
/// share withdrawn — or a folder library whose directory is not there. It
/// is also, exactly, the shape a revoked Android tree permission will have
/// when the Storage Access Framework connector FR-PLAT-AND-1 asks for
/// exists: the tree URI still stored, the permission behind it gone, every
/// read failing at the root and nowhere else. **That connector is not
/// built**, so no SAF grant can be lost yet; what this variant does is put
/// the recovery FR-PLAT-AND-2 requires in the one place all three causes
/// pass through, so the third needs no new handling above it.
///
/// The response is the same for all of them and is the point of the
/// variant: mark what the catalog holds as offline, keep every row, and
/// say which library and why (FR-CAT-9).
///
/// The string is the underlying failure, not a rewrite of it. What the
/// user is told is composed where the library's name is known.
#[error("the library folder could not be opened: {0}")]
RootUnavailable(String),
}
impl RemoteError {
@@ -150,6 +191,19 @@ impl RemoteError {
pub fn indicates_offline(&self) -> bool {
matches!(self, RemoteError::Network(_))
}
/// TRACES: FR-PLAT-AND-2
/// Whether the *library* is gone, as opposed to the server or one file.
///
/// Kept beside [`Self::indicates_offline`] because the two answer the same
/// shape of question and must not be confused. Both put the app into a
/// degraded mode that keeps working from the catalog, but they differ in
/// what the user is told and in what would end it: an offline library
/// comes back when the network does, and an unavailable root comes back
/// only when someone grants access again.
pub fn indicates_lost_root(&self) -> bool {
matches!(self, RemoteError::RootUnavailable(_))
}
}
#[cfg(test)]
core/dr-sync/src/scan.rs
+134
View File
@@ -171,6 +171,37 @@ where
let entries = match backend.list(&dir, None).await {
Ok(e) => e,
// TRACES: FR-PLAT-AND-2 | FR-CAT-9
// The root is the one directory the walk may not step over, and
// `depth == 0` is the only place it can be — nothing is ever
// pushed at that depth but the root itself.
//
// Below, a directory that has gone is a directory that went away
// between its parent being listed and it being reached, and
// continuing is right. At the root the identical error means the
// *library* is gone, and continuing is catastrophic in a way that
// is completely silent: the walk ends, the scan succeeds having
// found nothing, and the app reports a healthy library with no new
// images while every path in the catalog now points nowhere.
//
// Refused rather than reclassified. Only these two causes are —
// a `Network` failure at the root is still a network failure, and
// must stay one or an unplugged network cable would present itself
// as a revoked permission and offline mode would never engage.
Err(RemoteError::NotFound(_)) if depth == 0 => {
return Err(RemoteError::RootUnavailable(format!(
"{root} is no longer there"
)));
}
Err(RemoteError::PermissionDenied) if depth == 0 => {
// Deliberately not the variant's own message, which describes
// a Nextcloud share that refuses to *update* a sidecar. At the
// root nothing has been read at all.
return Err(RemoteError::RootUnavailable(format!(
"{root} can no longer be read"
)));
}
Err(RemoteError::NotFound(_)) => {
// Deleted between listing its parent and reaching it.
log::debug!("scan: {dir} vanished during the walk");
@@ -239,6 +270,20 @@ mod tests {
caps: Capabilities,
lists: RefCell<usize>,
probes: RefCell<usize>,
/// Directories whose listing fails, and how.
///
/// An absent directory is not enough to model this: the fake answers
/// an unknown path with an empty listing, which is exactly the shape
/// the walk must *not* confuse with a library that has gone away.
deny: HashMap<String, Deny>,
}
/// The two ways a real backend refuses a directory that is still named in
/// the catalog: it is not there, or it may not be read.
#[derive(Clone, Copy)]
enum Deny {
Missing,
Forbidden,
}
// The fake is single-threaded; tests never share it across threads.
@@ -307,8 +352,15 @@ mod tests {
},
lists: RefCell::new(0),
probes: RefCell::new(0),
deny: HashMap::new(),
}
}
/// Make one directory refuse to be listed.
fn denying(mut self, path: &str, how: Deny) -> Self {
self.deny.insert(path.to_string(), how);
self
}
}
#[async_trait]
@@ -325,6 +377,11 @@ mod tests {
_since: Option<&Validator>,
) -> Result<Vec<RemoteEntry>, RemoteError> {
*self.lists.borrow_mut() += 1;
match self.deny.get(dir.as_str()) {
Some(Deny::Missing) => return Err(RemoteError::NotFound(dir.to_string())),
Some(Deny::Forbidden) => return Err(RemoteError::PermissionDenied),
None => {}
}
Ok(self.tree.get(dir.as_str()).cloned().unwrap_or_default())
}
async fn dir_validator(&self, dir: &RemotePath) -> Result<Validator, RemoteError> {
@@ -404,6 +461,83 @@ mod tests {
assert_eq!(r.progress.directories_listed, 3);
}
/// TRACES: FR-PLAT-AND-2 | FR-CAT-9
#[tokio::test]
async fn a_root_that_is_gone_is_a_failure_and_not_an_empty_library() {
// The silent one. A vanished directory below the root is stepped over,
// and before this the root was stepped over on the same terms — which
// ended the walk immediately, returned `Ok` with nothing in it, and
// let the app report a successful scan of a library that no longer
// exists. Nothing in that path is ever told the library went away, so
// nothing marks it offline and nothing tells the user.
let b =
FakeBackend::sample(ChangeDetection::PropagatingEtags).denying("Photos", Deny::Missing);
let e = scan(
&b,
&RemotePath::new("Photos"),
&FormatFilter::all(),
&HashMap::new(),
|_| {},
)
.await
.expect_err("a library that is not there is not a library with no photographs");
assert!(e.indicates_lost_root(), "got {e:?}");
assert!(!e.indicates_offline(), "waiting will not bring this back");
assert!(e.to_string().contains("Photos"), "names the library: {e}");
}
/// TRACES: FR-PLAT-AND-2
#[tokio::test]
async fn a_root_that_may_not_be_read_reports_the_root_and_not_the_share_advice() {
// A Nextcloud share withdrawn, a directory the process may no longer
// read — and the shape a revoked Android tree grant will have when one
// can be held at all. Reported as plain `PermissionDenied` it would
// have carried that variant's message, which is several lines about a
// Nextcloud share refusing to *update* an existing sidecar: advice for
// a case where reads work, offered to a user whose reads have stopped
// entirely.
let b = FakeBackend::sample(ChangeDetection::PropagatingEtags)
.denying("Photos", Deny::Forbidden);
let e = scan(
&b,
&RemotePath::new("Photos"),
&FormatFilter::all(),
&HashMap::new(),
|_| {},
)
.await
.expect_err("a root that cannot be read is a failed scan");
assert!(e.indicates_lost_root(), "got {e:?}");
assert!(
!e.to_string().contains("sidecar"),
"the share-permission advice does not belong here: {e}"
);
}
/// TRACES: FR-PLAT-AND-2
#[tokio::test]
async fn a_folder_that_goes_away_below_the_root_is_still_stepped_over() {
// The other side of the split, and the reason the root is keyed on
// depth rather than on the error. A subfolder deleted between its
// parent being listed and it being reached is ordinary, and failing
// the scan over it would abandon every photograph beside it.
let b = FakeBackend::sample(ChangeDetection::PropagatingEtags)
.denying("Photos/2025", Deny::Missing);
let r = scan(
&b,
&RemotePath::new("Photos"),
&FormatFilter::all(),
&HashMap::new(),
|_| {},
)
.await
.expect("one folder going away is not the library going away");
assert_eq!(r.images.len(), 1, "2026 was still walked");
}
/// A library with a trash folder holding a soft-deleted image.
fn with_trash() -> FakeBackend {
let mut b = FakeBackend::sample(ChangeDetection::PropagatingEtags);
core/dr-types/src/settings.rs
+1 -1
View File
@@ -1,4 +1,4 @@
//! TRACES: FR-NC-6a | FR-EXP-1 | FR-EXP-2 | FR-EXP-3 | FR-EXP-6 | FR-PLAT-LIN-1
//! TRACES: FR-NC-6a | FR-EXP-1 | FR-EXP-2 | FR-EXP-3 | FR-EXP-6 | FR-PLAT-LIN-1 | NFR-OPS-3
//! Device preferences: how much disk to spend, and what an export defaults to.
//!
//! # Why these live beside the session and not in the catalog