Add requirements traceability gate and Gitea pipelines

Ports JellyTau's traceability tooling to Rust, carrying across the bug it
was repaired for. That gate divided a traced count by frozen literal
denominators; the requirements file outgrew them and it reported 158%
coverage, so it could never fail its own threshold.

Two rules, both enforced by the extractor's own tests:

  - denominators parsed from docs/requirements.md at run time
  - coverage is |traced ∩ defined| / |defined|, never a raw traced count

The gate additionally fails hard on a misconfigured run — zero
requirements parsed or zero files scanned — rather than reporting a
plausible 0%, and on any orphan tag naming a requirement that does not
exist.

Adapted for DarkRoom: IDs are FR-CAT-1 / NFR-P13 / FR-DEV-3a shapes
rather than JellyTau's fixed three digits, and decisions (D), spikes (S),
milestone items (M) and test ids remain taggable while being excluded
from the denominator — counting them inflated it by 25.

Also adds dr-sync: the RemoteBackend trait and capability model, so the
Nextcloud connector is one implementation rather than the only shape the
engine understands. No mature Nextcloud crate exists (reqwest_dav is too
thin), so the connector will be hand-rolled over reqwest per D7.

Gitea workflows follow the same style: containerised, commented with the
reasoning, desktop and Android on every push, plus a CI check that no
core/ crate depends on the UI toolkit (ARCH §6.5a).

Coverage today: 13.3% (19/143). 50 tests passing.
This commit is contained in:
2026-08-09 08:01:32 +02:00
parent 82a5e21ec6
commit 0f202fd3f9
19 changed files with 1913 additions and 22 deletions
+139
View File
@@ -0,0 +1,139 @@
//! What a backend can do.
//!
//! Declared rather than assumed, because the operations that matter most for
//! performance are not universal (ARCH §8.1).
/// TRACES: FR-NC-4
/// How a backend reports what changed.
///
/// This is the single most consequential capability: it determines whether a
/// no-op sync over a large library costs one request or thousands.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ChangeDetection {
/// The backend maintains a change feed; we present a cursor and receive
/// what changed since. Cheapest possible.
DeltaCursor,
/// Directory ETags propagate upward, so an unchanged parent proves an
/// unchanged subtree. Nextcloud. One request proves a whole library
/// unchanged.
PropagatingEtags,
/// ETags exist per entry but do not propagate. The tree must be walked,
/// though ETags still prevent re-downloading unchanged content.
LocalEtags,
/// Modification times only. Walk and compare — vulnerable to clock skew
/// and coarse timestamp granularity.
Timestamps,
}
/// Constraints on chunked upload.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct ChunkConstraints {
pub min_chunk: u64,
pub max_chunk: u64,
/// Bodies at or below this go in a single request.
pub single_shot_below: u64,
pub max_chunks: u32,
}
/// TRACES: FR-NC-3
/// Whether the server can render thumbnails, and for what.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ServerPreviews {
/// No server-side rendering.
None,
/// Common web formats only. **This is stock Nextcloud** — no RAW preview
/// provider ships with it, so RAW thumbnails must come from local
/// embedded-preview extraction (ARCH §6.7).
CommonFormatsOnly,
/// RAW included, e.g. via the `camerarawpreviews` app or an Imaginary
/// backend. Detected per-account, never assumed.
IncludingRaw,
}
/// The full capability set.
#[derive(Debug, Clone)]
pub struct Capabilities {
pub change_detection: ChangeDetection,
/// Identity survives server-side rename and move, so a move is not
/// mistaken for delete-plus-add of a large file.
pub stable_ids: bool,
/// Byte-range reads. Without these, embedded-preview extraction is
/// impossible and remote browsing must download whole files.
pub range_reads: bool,
pub chunked_upload: Option<ChunkConstraints>,
/// Many small objects in one request.
pub bulk_upload: bool,
/// Conditional write (If-Match), for conflict-safe sidecar updates.
pub conditional_write: bool,
pub server_previews: ServerPreviews,
}
impl Capabilities {
/// The weakest backend the engine will still drive: listing and whole-file
/// transfer, nothing more. Everything degrades but stays correct.
pub fn minimal() -> Self {
Self {
change_detection: ChangeDetection::Timestamps,
stable_ids: false,
range_reads: false,
chunked_upload: None,
bulk_upload: false,
conditional_write: false,
server_previews: ServerPreviews::None,
}
}
/// Whether remote browsing can avoid downloading whole files.
///
/// Where false, the UI must not browse a remote library on a metered
/// connection without explicit consent (FR-NC-12).
pub fn can_browse_cheaply(&self) -> bool {
self.range_reads || matches!(self.server_previews, ServerPreviews::IncludingRaw)
}
/// Whether sidecar conflicts can be detected reliably.
///
/// Without conditional writes the engine falls back to comparing revision
/// counters inside the sidecar, which narrows the race but does not close
/// it — surfaced as a reduced-safety mode.
pub fn safe_concurrent_writes(&self) -> bool {
self.conditional_write
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn minimal_backend_degrades_but_stays_usable() {
let caps = Capabilities::minimal();
assert!(!caps.can_browse_cheaply());
assert!(!caps.safe_concurrent_writes());
}
#[test]
fn server_raw_previews_substitute_for_range_reads() {
// A server that renders RAW thumbnails makes browsing cheap even
// without range support.
let caps = Capabilities {
server_previews: ServerPreviews::IncludingRaw,
..Capabilities::minimal()
};
assert!(caps.can_browse_cheaply());
}
#[test]
fn common_format_previews_do_not_help_raw() {
// Stock Nextcloud: previews exist, but not for RAW, so range reads
// remain the only cheap path.
let caps = Capabilities {
server_previews: ServerPreviews::CommonFormatsOnly,
..Capabilities::minimal()
};
assert!(!caps.can_browse_cheaply());
}
}
+82
View File
@@ -0,0 +1,82 @@
/// Failures from a remote backend.
#[derive(Debug, thiserror::Error)]
pub enum RemoteError {
#[error("not authenticated")]
Unauthenticated,
#[error("authentication rejected")]
AuthFailed,
#[error("not found: {0}")]
NotFound(String),
/// The backend does not support this operation. Expected, not a bug —
/// callers check capabilities and adapt.
#[error("operation unsupported by this backend: {0}")]
Unsupported(&'static str),
/// A conditional write failed: the remote changed underneath us. Triggers
/// the sidecar merge path (ARCH §8.5).
#[error("precondition failed — remote was modified")]
PreconditionFailed,
#[error("quota exceeded")]
QuotaExceeded,
#[error("network error: {0}")]
Network(String),
#[error("unexpected server response: {status} {detail}")]
Server { status: u16, detail: String },
#[error("malformed response: {0}")]
Protocol(String),
#[error("operation cancelled")]
Cancelled,
}
impl RemoteError {
/// Whether retrying might succeed.
pub fn is_transient(&self) -> bool {
match self {
RemoteError::Network(_) => true,
RemoteError::Server { status, .. } => {
// 5xx and 429 are worth retrying; other 4xx are not.
*status >= 500 || *status == 429
}
_ => false,
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn transient_errors_are_retryable() {
assert!(RemoteError::Network("timeout".into()).is_transient());
assert!(RemoteError::Server {
status: 503,
detail: String::new()
}
.is_transient());
assert!(RemoteError::Server {
status: 429,
detail: String::new()
}
.is_transient());
}
#[test]
fn client_errors_are_not_retryable() {
assert!(!RemoteError::Server {
status: 404,
detail: String::new()
}
.is_transient());
assert!(!RemoteError::PreconditionFailed.is_transient());
assert!(!RemoteError::AuthFailed.is_transient());
}
}
+202
View File
@@ -0,0 +1,202 @@
//! Pluggable remote storage for DarkRoom.
//!
//! Defines the [`RemoteBackend`] trait and the capability model the sync
//! engine adapts to. Only the Nextcloud connector is implemented
//! (`dr-sync-nextcloud`), but the boundary is designed so other backends can
//! be added without touching the engine.
//!
//! # Why capabilities rather than a common denominator
//!
//! Nextcloud's fast path relies on a behaviour that is *not* a WebDAV
//! guarantee: directory ETags propagate up the tree, so an unchanged root
//! ETag proves nothing anywhere in the library changed. That single property
//! turns a no-op sync over 50k images into one HTTP request.
//!
//! A trait built to what every backend can do would force full enumeration
//! every time — the exact cost the design exists to avoid. So backends
//! declare what they support and the engine picks a strategy (ARCH §8.1).
use std::ops::Range;
use async_trait::async_trait;
pub mod capability;
pub mod error;
pub mod types;
pub use capability::{Capabilities, ChangeDetection, ChunkConstraints, ServerPreviews};
pub use error::RemoteError;
pub use types::{
Cursor, Identity, Precondition, RemoteChange, RemoteEntry, RemoteId, RemotePath, Validator,
};
/// TRACES: FR-NC-12
/// A remote storage backend.
///
/// Implementations are expected to be cheap to clone or to be used behind an
/// `Arc`; the engine may call them concurrently.
#[async_trait]
pub trait RemoteBackend: Send + Sync {
/// What this backend supports. Read once at connect time and used to pick
/// a sync strategy.
fn capabilities(&self) -> &Capabilities;
/// Human-readable backend name, for logs and the UI.
fn name(&self) -> &str;
// ---- discovery --------------------------------------------------------
/// List one directory level.
///
/// `since` carries the validator the caller last saw, so backends able to
/// skip unchanged entries may do so. Backends that cannot simply ignore
/// it.
async fn list(
&self,
dir: &RemotePath,
since: Option<&Validator>,
) -> Result<Vec<RemoteEntry>, RemoteError>;
/// Fetch a directory's validator without listing its contents.
///
/// The cheap probe that makes ETag pruning work: one request against the
/// root answers "did anything change?". Backends without
/// [`ChangeDetection::PropagatingEtags`] return
/// [`RemoteError::Unsupported`].
async fn dir_validator(&self, dir: &RemotePath) -> Result<Validator, RemoteError>;
/// Ask what changed since a cursor.
///
/// Only meaningful for [`ChangeDetection::DeltaCursor`] backends; others
/// return [`RemoteError::Unsupported`].
async fn delta(&self, cursor: &Cursor) -> Result<(Vec<RemoteChange>, Cursor), RemoteError>;
// ---- transfer ---------------------------------------------------------
/// Fetch an object, optionally a byte range.
///
/// The range is a hint, not a guarantee: backends without range support
/// may return the whole object, and the caller slices. Correctness holds
/// either way; [`Capabilities::range_reads`] says whether it was cheap.
async fn get(&self, id: &RemoteId, range: Option<Range<u64>>) -> Result<Vec<u8>, RemoteError>;
/// Upload, optionally guarded by a precondition.
///
/// Backends handle chunking internally based on body size — chunked
/// upload is an implementation detail, not part of this interface, since
/// exposing it would leak one server's protocol into the abstraction.
async fn put(
&self,
path: &RemotePath,
body: Vec<u8>,
precond: Option<Precondition>,
) -> Result<Validator, RemoteError>;
/// Upload many small objects.
///
/// Defaults to sequential [`put`](Self::put) calls; backends with a bulk
/// endpoint override it. Sidecars are the motivating case — hundreds of
/// a few KB each.
async fn put_many(
&self,
items: Vec<(RemotePath, Vec<u8>)>,
) -> Result<Vec<Result<Validator, RemoteError>>, RemoteError> {
let mut out = Vec::with_capacity(items.len());
for (path, body) in items {
out.push(self.put(&path, body, None).await);
}
Ok(out)
}
/// Delete an object.
async fn delete(&self, id: &RemoteId, precond: Option<Precondition>)
-> Result<(), RemoteError>;
// ---- optional ---------------------------------------------------------
/// Server-rendered thumbnail, where available.
///
/// `Ok(None)` means the server has no preview for this object — which is
/// common for RAW, since stock Nextcloud ships no RAW preview provider
/// (ARCH §6.7). Callers must have a local fallback.
async fn thumbnail(&self, _id: &RemoteId, _size: u32) -> Result<Option<Vec<u8>>, RemoteError> {
Ok(None)
}
}
/// TRACES: FR-NC-4 | FR-NC-12
/// Which sync strategy the engine should use for a backend.
///
/// Derived from capabilities at connect time. Reported to the user so a slow
/// backend is visibly slow rather than mysteriously slow (FR-NC-12).
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum SyncStrategy {
/// Ask the server what changed. Cheapest.
Delta,
/// Probe the root ETag; recurse only where it differs. One request when
/// nothing changed.
EtagPruning,
/// Enumerate the tree, using per-entry ETags to avoid re-downloading.
FullListing,
/// Enumerate and compare modification times. Degraded — clock skew and
/// second-granularity timestamps both cause misses.
TimestampCompare,
}
impl SyncStrategy {
pub fn for_capabilities(caps: &Capabilities) -> Self {
match caps.change_detection {
ChangeDetection::DeltaCursor => SyncStrategy::Delta,
ChangeDetection::PropagatingEtags => SyncStrategy::EtagPruning,
ChangeDetection::LocalEtags => SyncStrategy::FullListing,
ChangeDetection::Timestamps => SyncStrategy::TimestampCompare,
}
}
/// A short description for the UI.
pub fn describe(self) -> &'static str {
match self {
SyncStrategy::Delta => "server change feed",
SyncStrategy::EtagPruning => "incremental (ETag pruning)",
SyncStrategy::FullListing => "full listing, cached by ETag",
SyncStrategy::TimestampCompare => "full listing by timestamp (degraded)",
}
}
/// Whether this strategy can prove "nothing changed" cheaply.
///
/// Where false, every sync costs at least one request per folder, which
/// the UI should warn about on large libraries.
pub fn has_cheap_noop(self) -> bool {
matches!(self, SyncStrategy::Delta | SyncStrategy::EtagPruning)
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn strategy_follows_capability() {
let mut caps = Capabilities::minimal();
assert_eq!(
SyncStrategy::for_capabilities(&caps),
SyncStrategy::TimestampCompare
);
caps.change_detection = ChangeDetection::PropagatingEtags;
assert_eq!(
SyncStrategy::for_capabilities(&caps),
SyncStrategy::EtagPruning
);
}
#[test]
fn only_delta_and_pruning_have_cheap_noop() {
assert!(SyncStrategy::Delta.has_cheap_noop());
assert!(SyncStrategy::EtagPruning.has_cheap_noop());
// These cost at least one request per folder, every time.
assert!(!SyncStrategy::FullListing.has_cheap_noop());
assert!(!SyncStrategy::TimestampCompare.has_cheap_noop());
}
}
+198
View File
@@ -0,0 +1,198 @@
//! Vocabulary shared by every remote backend.
//!
//! Deliberately protocol-neutral: nothing here names WebDAV, HTTP, or
//! Nextcloud. A backend translates these into its own protocol, which is what
//! keeps the sync engine reusable (ARCH §8.3).
pub use dr_types::Validator;
/// A path on the remote, always `/`-separated and rooted at the account's
/// library root — never a local filesystem path.
#[derive(Debug, Clone, PartialEq, Eq, Hash, PartialOrd, Ord)]
pub struct RemotePath(String);
impl RemotePath {
pub fn new(path: impl Into<String>) -> Self {
let p: String = path.into();
// Normalise so `/a/b`, `a/b` and `a/b/` compare equal — otherwise the
// same directory can be listed twice under different spellings.
let trimmed = p.trim_matches('/');
RemotePath(trimmed.to_string())
}
pub fn root() -> Self {
RemotePath(String::new())
}
pub fn as_str(&self) -> &str {
&self.0
}
pub fn is_root(&self) -> bool {
self.0.is_empty()
}
/// Append a single path segment.
pub fn join(&self, segment: &str) -> Self {
let seg = segment.trim_matches('/');
if self.0.is_empty() {
RemotePath(seg.to_string())
} else {
RemotePath(format!("{}/{}", self.0, seg))
}
}
/// The final segment, for display.
pub fn name(&self) -> &str {
self.0.rsplit('/').next().unwrap_or(&self.0)
}
/// The containing directory, or `None` at the root.
pub fn parent(&self) -> Option<RemotePath> {
let (head, _) = self.0.rsplit_once('/')?;
Some(RemotePath(head.to_string()))
}
}
impl std::fmt::Display for RemotePath {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str(if self.0.is_empty() { "/" } else { &self.0 })
}
}
/// A backend-stable object identifier.
///
/// Where the backend supports stable ids (`Capabilities::stable_ids`) this
/// survives server-side rename and move, so a move is detected as a move
/// rather than a delete plus a re-download of a large file (FR-NC-5).
/// Otherwise it falls back to the path, and moves cost a transfer.
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub enum RemoteId {
/// A server-assigned identifier, e.g. Nextcloud's `oc:fileid`.
Stable(u64),
/// No stable identity available; the path is the identity.
Path(RemotePath),
}
/// One entry returned by a directory listing.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct RemoteEntry {
pub id: RemoteId,
pub path: RemotePath,
pub kind: EntryKind,
pub validator: Validator,
pub size: u64,
/// Unix seconds, where the backend reports one.
pub modified: Option<i64>,
/// Whether the server claims a renderable preview exists. Advisory: stock
/// Nextcloud reports none for RAW (ARCH §6.7).
pub has_preview: bool,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum EntryKind {
File,
Directory,
}
/// A change reported by a delta-capable backend.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum RemoteChange {
Upserted(RemoteEntry),
Deleted(RemoteId),
}
/// An opaque position in a backend's change feed.
///
/// Persisted between syncs. Backends may expire cursors; the engine treats an
/// expired cursor as a signal to fall back to a full listing rather than an
/// error.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Cursor(String);
impl Cursor {
pub fn new(token: impl Into<String>) -> Self {
Cursor(token.into())
}
pub fn as_str(&self) -> &str {
&self.0
}
}
/// A conditional-write guard.
///
/// `IfMatch` is optimistic concurrency for sidecar updates: a failure means
/// another device wrote first, which triggers the per-operation merge rather
/// than an overwrite (ARCH §8.5).
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum Precondition {
/// Write only if the remote still matches this validator.
IfMatch(Validator),
/// Write only if nothing exists — creation without clobbering.
IfAbsent,
}
/// Who we are connected as.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Identity {
pub user_id: String,
pub display_name: Option<String>,
pub server: String,
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn paths_normalise_so_spellings_compare_equal() {
// Otherwise the same directory is listed twice under two spellings.
assert_eq!(
RemotePath::new("/Photos/2026/"),
RemotePath::new("Photos/2026")
);
assert_eq!(
RemotePath::new("Photos/2026/"),
RemotePath::new("Photos/2026")
);
}
#[test]
fn root_is_empty_and_displays_as_slash() {
let r = RemotePath::root();
assert!(r.is_root());
assert_eq!(r.to_string(), "/");
assert_eq!(RemotePath::new("/"), r);
}
#[test]
fn join_from_root_does_not_double_separator() {
assert_eq!(RemotePath::root().join("Photos").as_str(), "Photos");
assert_eq!(
RemotePath::new("Photos").join("2026").as_str(),
"Photos/2026"
);
// A segment arriving with separators is still joined once.
assert_eq!(
RemotePath::new("Photos").join("/2026/").as_str(),
"Photos/2026"
);
}
#[test]
fn name_and_parent() {
let p = RemotePath::new("Photos/2026/IMG_0042.CR3");
assert_eq!(p.name(), "IMG_0042.CR3");
assert_eq!(p.parent().unwrap().as_str(), "Photos/2026");
assert_eq!(RemotePath::root().parent(), None);
}
#[test]
fn stable_ids_differ_from_path_ids() {
// Identity is what makes a server-side move cheap; the two forms must
// not be conflated.
let a = RemoteId::Stable(42);
let b = RemoteId::Path(RemotePath::new("Photos/x.CR3"));
assert_ne!(a, b);
}
}