Files
DarkRoom/core/dr-ingest/src/layout.rs
T
dtourolleandClaude Opus 5 7d1e6f724e Import photographs from a card
Distinct from a scan, and the distinction is the whole reason the crate
exists: a scan catalogues files where they already are, where an import moves
them from a card into the library. A scan that fails halfway has read
nothing; an import that fails halfway has written something.

So the failure paths are the design. Bytes stream at 1 MiB and are hashed on
the way past, so an 80 MB RAW never sits in memory. The second destination
(FR-CAT-10's backup copy) is written from the same read rather than copied
from the primary afterwards — a backup made by re-reading the primary would
inherit a bad write rather than catch it, and re-reading the card doubles the
wear on the one copy that still exists. Verification re-reads the
destination, because hashing what is still in memory would pass on a full
disk, a dying card and a truncated write alike. Anything that fails past the
point of creating the file takes the file back, or the next scan catalogues a
truncated RAW as though it were fine.

Three things the crate refuses to know. It never deletes from the card: a
move-import records what is now redundant and a separate retire() does the
deleting, because on a syncing library "safe" means the upload was confirmed
(FR-NC-7b). It does not decode, so capture metadata arrives through a probe
and a card of unreadable files costs no demosaic. And it does not know what a
duplicate is, since that is a catalog query — FR-CAT-11's two tiers arrive as
one closure asked twice, once before any transfer and once with the digest.

Camera serial would be the stronger metadata key and is absent, because
nothing in the tree reads it yet; make and model plus capture time and the
original filename is what is available, and the digest tier covers the gap.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 14:00:47 +02:00

344 lines
13 KiB
Rust

//! TRACES: FR-CAT-10 | FR-NC-7a
//! Where an imported photograph is filed.
//!
//! One template, two destinations. A card import expands it into directories
//! under a library root (FR-CAT-10); an upload expands the *same* template
//! into folders on the server (FR-NC-7a). They must agree, because a library
//! that is `2026/2026-08-22/` locally and something else remotely is two
//! libraries that happen to hold the same files.
//!
//! # Why this returns segments rather than a path
//!
//! Neither caller can use a path. The local side creates each level through
//! [`dr_plat::WritableStorage`], which takes a parent reference plus one name
//! because a SAF document id is not composable (ARCH §6.9). The remote side
//! joins onto a `RemotePath`, which normalises separators its own way. A
//! `String` with slashes in it would be taken apart again by both, and the
//! taking-apart is where a `..` or an empty segment would slip through.
//!
//! So expansion hands back `Vec<String>`, each element already checked to be
//! one ordinary directory name.
use dr_types::civil_from_unix_at;
/// What a folder template can refer to.
///
/// Deliberately not the decoder's `Metadata`: this crate does not depend on
/// `dr-decode` (see the crate docs), and a template needs six fields out of
/// twenty. The caller fills it from whatever it read.
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct Shot {
/// When the shutter fired, Unix seconds, as EXIF recorded it.
pub captured_at: Option<i64>,
/// Minutes east of UTC where the camera recorded a zone.
pub captured_offset: Option<i32>,
pub make: Option<String>,
pub model: Option<String>,
/// The file's modification time, Unix seconds — the fallback when there is
/// no capture time, and the reason [`Shot`] always dates to *something*.
pub modified_at: i64,
}
/// Which reading the folder name came from.
///
/// FR-NC-7a requires the fallback to be visible in the import report rather
/// than silent. An image filed by mtime is filed by when it was last *copied*,
/// which for a card that has been through a card reader is often today — so a
/// user seeing a folder full of unrelated dates needs to be told why.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum DateSource {
/// EXIF capture time, in the zone the camera recorded.
Capture,
/// File modification time. Not when the photograph was taken.
Modified,
}
/// The default layout: one directory per year, one per capture day beneath it.
///
/// `2026/2026-08-22`. The day folder repeats the year rather than being bare
/// `08-22`, because a folder is often seen out of its parent's context — in a
/// file manager's recent list, in a Nextcloud share link, in a backup listing
/// — and `08-22` alone does not say which year it belongs to.
pub const DEFAULT_TEMPLATE: &str = "{yyyy}/{yyyy}-{mm}-{dd}";
/// An expanded destination.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Destination {
/// Directory names, outermost first. Each is one ordinary name: never
/// empty, never `.` or `..`, never containing a separator.
pub segments: Vec<String>,
/// Which timestamp produced the date tokens.
pub dated_from: DateSource,
}
impl Destination {
/// The layout as a display string — for a confirmation dialog and for logs.
pub fn display(&self) -> String {
self.segments.join("/")
}
}
/// Expand a folder template for one photograph.
///
/// Unknown tokens are left verbatim, matching `dr_export::name::expand`: a
/// user who typed `{yyy}` should see it in the folder name and understand what
/// happened, where a silently dropped token produces a directory whose name
/// says nothing about why it is called that.
///
/// The result is **a pure function of the template and the shot**, which
/// FR-NC-7a requires: two devices uploading the same frame must compute the
/// same destination, so nothing here may consult local library layout, the
/// current time, or a counter.
pub fn expand(template: &str, shot: &Shot) -> Destination {
let (when, dated_from) = match shot.captured_at {
Some(t) => (t, DateSource::Capture),
None => (shot.modified_at, DateSource::Modified),
};
// The offset applies only to a capture reading. An mtime is already an
// absolute instant with no camera zone attached to it, and adding the
// camera's offset to it would be arithmetic on two unrelated clocks.
let civil = match dated_from {
DateSource::Capture => civil_from_unix_at(when, shot.captured_offset),
DateSource::Modified => civil_from_unix_at(when, None),
};
let year = civil.year.to_string();
let short_year = format!("{:02}", civil.year.rem_euclid(100));
let month = format!("{:02}", civil.month);
let day = format!("{:02}", civil.day);
let mut out = String::with_capacity(template.len() + 16);
let mut rest = template;
while let Some(open) = rest.find('{') {
out.push_str(&rest[..open]);
let Some(close) = rest[open..].find('}') else {
// An unclosed brace is literal text. Consumed here rather than
// left to the tail append, which has already had everything
// before the brace taken from it.
out.push_str(&rest[open..]);
rest = "";
break;
};
let token = &rest[open + 1..open + close];
match token {
"yyyy" => out.push_str(&year),
"yy" => out.push_str(&short_year),
"mm" => out.push_str(&month),
"dd" => out.push_str(&day),
"date" => out.push_str(&format!("{year}-{month}-{day}")),
"make" => out.push_str(shot.make.as_deref().unwrap_or_default()),
"model" => out.push_str(shot.model.as_deref().unwrap_or_default()),
_ => out.push_str(&rest[open..open + close + 1]),
}
rest = &rest[open + close + 1..];
}
out.push_str(rest);
let segments: Vec<String> = out
.split('/')
.map(sanitise_segment)
// Empty segments are dropped rather than kept: a template of
// `{make}/{yyyy}` on a camera whose make is unknown must not produce
// a nameless directory level, and `//` from a typo must not either.
.filter(|s| !s.is_empty())
.collect();
// A template that expanded to nothing at all — `{make}` alone, on a file
// with no make. Falling back to the date is the one answer always
// available, and it is what the user asked for in every other case.
let segments = if segments.is_empty() {
vec![year.clone(), format!("{year}-{month}-{day}")]
} else {
segments
};
Destination {
segments,
dated_from,
}
}
/// Reduce one path segment to something every destination will accept.
///
/// The same intersection of rules as `dr_export::name::sanitise` — Linux,
/// SAF and WebDAV, plus Windows for the sync client downstream — applied to a
/// directory name rather than a filename stem. Kept apart from the exporter's
/// because a segment has one rule a stem does not: `.` and `..` are legal
/// filenames and are *not* legal directory names here, since both would move
/// the destination rather than name it.
fn sanitise_segment(seg: &str) -> String {
let mut out: String = seg
.chars()
.map(|c| match c {
'/' | '\\' | ':' | '*' | '?' | '"' | '<' | '>' | '|' => '-',
c if (c as u32) < 0x20 => '-',
c => c,
})
.collect();
// Trailing dots and spaces are legal on Linux and rejected by Windows, and
// a name ending in one is almost always a token that expanded to nothing.
while out.ends_with('.') || out.ends_with(' ') {
out.pop();
}
let out = out.trim_start().to_string();
// What is left of `.` or `..` after that pop is an empty string, which the
// caller drops. Spelled out rather than relied upon, because a segment
// that climbed out of the destination would be the one bug in this file
// that mattered.
if out == "." || out == ".." {
return String::new();
}
out
}
#[cfg(test)]
mod tests {
use super::*;
/// 2026-08-22T14:00:00Z.
const AUG_22: i64 = 1_787_407_200;
fn shot(captured_at: Option<i64>) -> Shot {
Shot {
captured_at,
modified_at: AUG_22,
..Default::default()
}
}
#[test]
fn the_default_template_is_a_year_then_a_day() {
let d = expand(DEFAULT_TEMPLATE, &shot(Some(AUG_22)));
assert_eq!(d.segments, ["2026", "2026-08-22"]);
assert_eq!(d.dated_from, DateSource::Capture);
}
#[test]
fn the_day_folder_is_zero_padded_so_a_year_sorts() {
// 2026-01-05T00:00:00Z. Unpadded, `2026-1-5` sorts after `2026-10-1`
// in every file manager there is.
let d = expand(DEFAULT_TEMPLATE, &shot(Some(1_767_571_200)));
assert_eq!(d.segments, ["2026", "2026-01-05"]);
}
#[test]
fn a_shot_is_filed_in_the_cameras_own_day_not_utcs() {
// 22:30 UTC on the 21st is already the 22nd in Tokyo. A night shoot
// must not be split across two folders at whatever hour UTC rolls.
let s = Shot {
captured_at: Some(1_787_351_400),
captured_offset: Some(540),
..shot(None)
};
assert_eq!(
expand(DEFAULT_TEMPLATE, &s).segments,
["2026", "2026-08-22"]
);
}
#[test]
fn a_file_with_no_capture_time_falls_back_visibly() {
let d = expand(DEFAULT_TEMPLATE, &shot(None));
assert_eq!(d.segments, ["2026", "2026-08-22"]);
// The requirement is not that it lands somewhere — it is that the
// report can say it was dated by mtime rather than by the shutter.
assert_eq!(d.dated_from, DateSource::Modified);
}
#[test]
fn the_cameras_zone_is_not_applied_to_a_modification_time() {
// A camera zone belongs to the shutter reading. An mtime carrying a
// +540 shift would be filed a day out for no reason at all.
let s = Shot {
captured_at: None,
captured_offset: Some(540),
modified_at: 1_787_351_400, // 21st, 22:30 UTC
..Default::default()
};
assert_eq!(
expand(DEFAULT_TEMPLATE, &s).segments,
["2026", "2026-08-21"]
);
}
#[test]
fn every_token_expands() {
let s = Shot {
captured_at: Some(AUG_22),
make: Some("Canon".into()),
model: Some("EOS R5".into()),
..shot(None)
};
let d = expand("{make}/{model}/{yy}/{date}/{yyyy}-{mm}-{dd}", &s);
assert_eq!(
d.segments,
["Canon", "EOS R5", "26", "2026-08-22", "2026-08-22"]
);
}
#[test]
fn an_unknown_token_is_left_where_the_user_can_see_it() {
let d = expand("{yyy}", &shot(Some(AUG_22)));
assert_eq!(d.segments, ["{yyy}"]);
}
#[test]
fn a_segment_cannot_climb_out_of_the_destination() {
// The one bug in this file that would matter: a template, or a camera
// model, that walks up out of the library root.
let d = expand("../../etc/{yyyy}", &shot(Some(AUG_22)));
assert_eq!(d.segments, ["etc", "2026"]);
let s = Shot {
model: Some("../..".into()),
..shot(Some(AUG_22))
};
assert_eq!(expand("{model}/{yyyy}", &s).segments, ["2026"]);
}
#[test]
fn a_leading_slash_does_not_make_it_absolute() {
let d = expand("/{yyyy}", &shot(Some(AUG_22)));
assert_eq!(d.segments, ["2026"]);
}
#[test]
fn an_empty_token_does_not_leave_a_nameless_level() {
// No make recorded. `{make}/{yyyy}` must be one level, not two with
// an unnamed parent.
let d = expand("{make}/{yyyy}", &shot(Some(AUG_22)));
assert_eq!(d.segments, ["2026"]);
}
#[test]
fn a_template_that_expands_to_nothing_falls_back_to_the_date() {
let d = expand("{make}", &shot(Some(AUG_22)));
assert_eq!(d.segments, ["2026", "2026-08-22"]);
}
#[test]
fn characters_no_destination_would_take_are_replaced() {
let s = Shot {
// Nikon writes its model with a colon on some bodies, and a colon
// is a stream separator on Windows and illegal on SAF.
model: Some("NIKON:Z 9".into()),
..shot(Some(AUG_22))
};
assert_eq!(expand("{model}", &s).segments, ["NIKON-Z 9"]);
}
#[test]
fn expansion_does_not_depend_on_where_it_runs() {
// FR-NC-7a: two devices must compute the same remote destination for
// the same frame, so the same input twice is the same answer twice.
let s = shot(Some(AUG_22));
assert_eq!(expand(DEFAULT_TEMPLATE, &s), expand(DEFAULT_TEMPLATE, &s));
}
#[test]
fn a_display_string_is_the_segments_joined() {
let d = expand(DEFAULT_TEMPLATE, &shot(Some(AUG_22)));
assert_eq!(d.display(), "2026/2026-08-22");
}
}