Federation: replicate content, re-derive judgement (UR-008)

Implements §9a. The replication surface is four reads and no writes: a change
feed, fetch by content_id, a batch have, and a human-facing peer directory —
plus a capabilities endpoint carrying the accepted envelope versions, which
lets a client discover a schema mismatch in one request instead of a 400 per
manifest across a library sweep.

Pull, never push: a pulling server chooses what it ingests and when. Push would
let any peer inject work into the validation queue — the same abuse surface as
anonymous upload, at higher volume.

Nothing inherits a peer's judgement. A pulled manifest runs the full §6 stage 1
and 2 validation and this server's own cast check, and the fetched body must
hash to the content_id that was asked for — the check that stops an
intermediary or a misbehaving peer substituting content under a trusted id.
A peer's retraction flags for review rather than delisting, because
auto-delisting would hand every peer a remote delete primitive; only the opt-in
per-peer abuse channel delists, because a takedown propagating at the speed of
manual review is the wrong failure mode for that one case.

A test caught a real bug in the first cut: the feed cursor was a ULID, and
ULIDs are only monotonic *between* milliseconds — two generated in the same
millisecond carry independent random components and can sort opposite to write
order. A peer resuming from `seq > cursor` would then silently skip an entry:
replication losing manifests with no error anywhere. The cursor is now an
AUTOINCREMENT integer, and the test asserts strict monotonicity rather than
merely sortedness.

Peer administration is deliberately not an API. §9a requires that a peering
exist only because an operator typed a URL, so nothing a remote server returns
can establish or widen one; there_is_no_endpoint_that_creates_a_peering asserts
that absence rather than trusting it.

212 tests. Coverage 25/32 (78%).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: UR-008 | PR-006

src/api/federation.rs

@@ -0,0 +1,288 @@
+//! §9a federation endpoints.
+//!
+//! **Replicate content, re-derive judgement.** A validated manifest is immutable
+//! and content-addressable, so replication is *set reconciliation* rather than
+//! state synchronisation — no concurrent edits, no last-write-wins, no vector
+//! clocks. What must not replicate is the mutable half: `status`, `reports` and
+//! `cast_match_ratio` encode a local operator's judgement and legal position.
+//! A server that adopts a peer's `listed` flags has outsourced its liability; one
+//! that adopts their `delisted` flags has outsourced its moderation.
+//!
+//! **Pull, never push.** A pulling server chooses what it ingests and when.
+//! Push would let any peer inject work into your validation queue — the same
+//! abuse surface as anonymous upload, but at higher volume.
+
+use axum::extract::{Path, Query, State};
+use axum::http::HeaderMap;
+use axum::response::{IntoResponse, Response};
+use axum::Json;
+use serde::{Deserialize, Serialize};
+
+use crate::db::repo;
+use crate::error::{ApiError, ApiResult};
+use crate::model::JMANIFEST_VERSION;
+use crate::ratelimit::Surface;
+use crate::state::{with_quota_headers, AppState};
+
+/// §5: bounded so one peer cannot walk the whole catalogue in a single request.
+const MAX_CHANGES_LIMIT: usize = 1000;
+/// §5: `POST /federation/have`, up to 1000 ids.
+const MAX_HAVE_IDS: usize = 1000;
+
+#[derive(Debug, Deserialize)]
+pub struct ChangesParams {
+    /// The cursor a peer echoes back. Opaque on the wire — a peer must treat it
+    /// as a token, not compute with it.
+    #[serde(default)]
+    pub since: Option<i64>,
+    #[serde(default)]
+    pub limit: Option<usize>,
+}
+
+#[derive(Debug, Serialize)]
+pub struct ChangesResponse {
+    pub cursor: Option<i64>,
+    pub server_id: String,
+    pub changes: Vec<ChangeItem>,
+}
+
+#[derive(Debug, Serialize)]
+pub struct ChangeItem {
+    pub content_id: String,
+    pub op: String,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub reason: Option<String>,
+    pub origin: String,
+    pub seq: i64,
+}
+
+/// `GET /federation/changes?since={cursor}&limit=1000`
+///
+/// A monotonic, append-only feed of locally-*listed* manifests. Entries are
+/// metadata only — enough to decide whether to fetch, without transferring
+/// payloads. The cursor is opaque and monotonic, so a peer resumes from its last
+/// position and the feed is idempotent.
+///
+/// TRACES: UR-008 | PR-006
+pub async fn get_changes(
+    State(state): State<AppState>,
+    peer: crate::state::PeerIp,
+    headers: HeaderMap,
+    Query(params): Query<ChangesParams>,
+) -> ApiResult<Response> {
+    let ip = state.client_ip(&headers, peer.0);
+    let quota = state.check_limit(&ip, Surface::FederationChanges)?;
+
+    let limit = params.limit.unwrap_or(MAX_CHANGES_LIMIT).min(MAX_CHANGES_LIMIT);
+    let since = params.since;
+
+    let entries = state
+        .db
+        .read(move |conn| repo::changes_since(conn, since, limit))
+        .await
+        .map_err(ApiError::Internal)?;
+
+    let cursor = entries.last().map(|e| e.seq);
+    let changes = entries
+        .into_iter()
+        .map(|e| ChangeItem {
+            content_id: e.content_id,
+            op: e.op,
+            reason: e.reason,
+            origin: e.origin,
+            seq: e.seq,
+        })
+        .collect();
+
+    let body = ChangesResponse { cursor, server_id: state.config.server_id.clone(), changes };
+    Ok(with_quota_headers(Json(body).into_response(), quota))
+}
+
+/// `GET /federation/manifests/{content_id}`
+///
+/// Full content by hash. **The puller must verify that what comes back hashes to
+/// the id it asked for**, and reject it otherwise — that check is what makes an
+/// intermediary or a misbehaving peer unable to substitute content. This server
+/// performs it on ingest (see [`crate::federation`]).
+///
+/// TRACES: UR-008 | PR-006
+pub async fn get_manifest_by_content_id(
+    State(state): State<AppState>,
+    peer: crate::state::PeerIp,
+    headers: HeaderMap,
+    Path(content_id): Path<String>,
+) -> ApiResult<Response> {
+    let ip = state.client_ip(&headers, peer.0);
+    let quota = state.check_limit(&ip, Surface::FederationFetch)?;
+
+    let manifest = state
+        .db
+        .read(move |conn| {
+            let Some(row) = repo::manifest_by_content_id_ro(conn, &content_id)? else {
+                return Ok(None);
+            };
+            // Only served content is replicated. A `pending` manifest has not
+            // cleared this server's own checks, and a `rejected` one failed them.
+            if row.status != "listed" && row.status != "flagged" {
+                return Ok(None);
+            }
+            let title = super::fetch::title_of(conn, &row.title_id)?;
+            let kind = if title.kind == "movie" {
+                crate::model::IdentityType::Movie
+            } else {
+                crate::model::IdentityType::Episode
+            };
+            Ok(Some(super::fetch::reconstruct(conn, &row, &title, kind)?))
+        })
+        .await
+        .map_err(ApiError::Internal)?;
+
+    let manifest = manifest.ok_or(ApiError::NotFound)?;
+    Ok(with_quota_headers(Json(manifest).into_response(), quota))
+}
+
+#[derive(Debug, Deserialize)]
+#[serde(deny_unknown_fields)]
+pub struct HaveRequest {
+    pub content_ids: Vec<String>,
+}
+
+#[derive(Debug, Serialize)]
+pub struct HaveResponse {
+    pub have: Vec<String>,
+}
+
+/// `POST /federation/have`
+///
+/// Batch existence check by `content_id`, so a peer diffs its set against yours
+/// in one request before fetching anything.
+///
+/// TRACES: UR-008 | PR-006
+pub async fn post_have(
+    State(state): State<AppState>,
+    peer: crate::state::PeerIp,
+    headers: HeaderMap,
+    super::json::Json(req): super::json::Json<HaveRequest>,
+) -> ApiResult<Response> {
+    let ip = state.client_ip(&headers, peer.0);
+    let quota = state.check_limit(&ip, Surface::FederationHave)?;
+
+    if req.content_ids.len() > MAX_HAVE_IDS {
+        return Err(ApiError::BadRequest(format!("more than {MAX_HAVE_IDS} content_ids")));
+    }
+
+    let ids = req.content_ids;
+    let have = state
+        .db
+        .read(move |conn| repo::known_content_ids(conn, &ids))
+        .await
+        .map_err(ApiError::Internal)?;
+
+    Ok(with_quota_headers(Json(HaveResponse { have }).into_response(), quota))
+}
+
+#[derive(Debug, Serialize)]
+pub struct PeersResponse {
+    pub server_id: String,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub contact: Option<String>,
+    pub peers: Vec<PeerItem>,
+}
+
+#[derive(Debug, Serialize)]
+pub struct PeerItem {
+    pub url: String,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub name: Option<String>,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub since: Option<String>,
+}
+
+/// `GET /federation/peers`
+///
+/// A **human-facing directory, not a discovery mechanism** — the distinction is
+/// the whole point of §9a's peer-directory section. It publishes a list a person
+/// can read; nothing acts on it. A server never fetches a peer's peers, so there
+/// is no crawl and therefore no network-wide topology to poison.
+///
+/// Advertising is opt-in per peer, and publishing the directory at all is opt-in
+/// for the server (`PublishPeerDirectory`, default off) — a server that would
+/// rather not disclose its topology simply does not.
+///
+/// TRACES: UR-008 | PR-005
+pub async fn get_peers(
+    State(state): State<AppState>,
+    peer: crate::state::PeerIp,
+    headers: HeaderMap,
+) -> ApiResult<Response> {
+    let ip = state.client_ip(&headers, peer.0);
+    let quota = state.check_limit(&ip, Surface::FederationPeers)?;
+
+    if !state.config.publish_peer_directory {
+        return Err(ApiError::NotFound);
+    }
+
+    let peers = state.db.read(repo::advertised_peers).await.map_err(ApiError::Internal)?;
+    let body = PeersResponse {
+        server_id: state.config.server_id.clone(),
+        contact: state.config.contact.clone(),
+        peers: peers
+            .into_iter()
+            .map(|p| PeerItem { url: p.url, name: p.name, since: p.peered_since })
+            .collect(),
+    };
+    Ok(with_quota_headers(Json(body).into_response(), quota))
+}
+
+#[derive(Debug, Serialize)]
+pub struct CapabilitiesResponse {
+    pub server_id: String,
+    /// Exchange envelope versions this server accepts. A client checks this once
+    /// rather than discovering a mismatch as a `400` per manifest across a whole
+    /// library sweep.
+    pub jmanifest_versions: Vec<u32>,
+    pub federation: bool,
+    /// §3: `POST /manifests/search` is expensive and therefore optional to
+    /// implement, so it is advertised rather than assumed.
+    pub audio_search: bool,
+    pub audio_tier_matching: bool,
+}
+
+/// `GET /federation/capabilities`
+///
+/// What this server actually supports. §3 specifies this for advertising the
+/// optional audio-search endpoint; it also carries the accepted envelope
+/// versions, which is what lets a plugin discover a schema mismatch in one cheap
+/// request instead of N failed uploads.
+///
+/// TRACES: UR-008, UR-014 | SR-003
+pub async fn get_capabilities(State(state): State<AppState>) -> ApiResult<Response> {
+    let body = CapabilitiesResponse {
+        server_id: state.config.server_id.clone(),
+        jmanifest_versions: vec![JMANIFEST_VERSION],
+        federation: true,
+        // Deferred by design (§3 sequencing): signatures accumulate first.
+        audio_search: false,
+        audio_tier_matching: false,
+    };
+    Ok(Json(body).into_response())
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn capabilities_advertise_only_the_current_envelope_version() {
+        // The flag day means exactly one accepted version; advertising a range
+        // would invite a client to send something that will be rejected.
+        assert_eq!(JMANIFEST_VERSION, 2);
+    }
+
+    #[test]
+    fn have_request_rejects_unknown_fields() {
+        // §6 stage 2 applies to federation bodies too — a peer is not exempt.
+        let r = serde_json::from_str::<HaveRequest>(r#"{"content_ids":[],"extra":1}"#);
+        assert!(r.is_err());
+    }
+}

src/api/fetch.rs

@@ -238,7 +238,10 @@ pub async fn get_status(
     }
 }
 
-fn title_of(conn: &rusqlite::Connection, title_id: &str) -> anyhow::Result<repo::TitleRow> {
+pub(crate) fn title_of(
+    conn: &rusqlite::Connection,
+    title_id: &str,
+) -> anyhow::Result<repo::TitleRow> {
     let row = conn.query_row(
         "SELECT id, kind, tmdb_id, imdb_id, name, year, adult, certification
            FROM titles WHERE id = ?1",

src/api/mod.rs

@@ -1,6 +1,7 @@
 //! HTTP surface (§4). Base path `/api/v1`, JSON throughout.
 
 pub mod exists;
+pub mod federation;
 pub mod fetch;
 pub mod json;
 pub mod report;