fix(connectivity): drive reachability from real repository traffic
🏗️ Build and Test JellyTau / Run Tests (push) Successful in 3m32s
Traceability Validation / Check Requirement Traces (push) Successful in 20s
🏗️ Build and Test JellyTau / Build Android APK (push) Successful in 18m27s
Build & Release / Run Tests (push) Successful in 3m36s
Build & Release / Build Linux (push) Successful in 15m43s
Build & Release / Build Android (push) Successful in 18m40s
Build & Release / Create Release (push) Failing after 22s

The offline/online switch was janky because two independent systems decided
"online" and never communicated:

- ConnectivityMonitor owned is_server_reachable (drove the UI banner) but
  learned reachability only from a standalone /System/Info/Public ping loop
  and from auth/login calls.
- HybridRepository served all real data by racing cache-vs-server but never
  read or wrote reachability.

So the banner reflected a side-channel poller, not the system the user actually
experienced: a successful ping could read "online" while authenticated data
calls 401'd or timed out, and three different timeout regimes (5s ping / 30s
data / 100ms cache race) flapped against each other.

Unify into a single source of truth:

- Extract a cheap, cloneable ConnectivityReporter that owns all reachability
  transitions and event emission.
- OnlineRepository reports the outcome of every server request to the reporter,
  classified via RepoError: Ok/Authentication/NotFound/Server => reachable
  (the server answered), Network => offline candidate, Database/Offline =>
  ignored (not a server signal).
- Time-window debounce (OFFLINE_CONFIRM_WINDOW = 5s): flip offline only after
  sustained network failure; recover instantly on the first success.
- Demote the ping loop to an offline-only recovery probe (no online polling;
  real traffic is the signal when online).
- Frontend: navigator.onLine is now advisory (triggers a recheck instead of
  forcing offline); removed the dead markReachable/markUnreachable store methods.

Docs updated (README, 07-connectivity, 03-data-flow, 02-svelte-frontend) to
describe the new model and fix pre-existing drift (HTTP client is 30s timeout +
5s ping, not the documented 10s/base_url).

Tests: 12 connectivity tests (debounce, instant recovery, RepoError
classification through report_outcome). Full suite: 398 Rust + 384 frontend
passing, svelte-check clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-21 21:56:14 +02:00
co-authored by Claude Opus 4.8
parent 3faa595b76
commit 45aa029916
8 changed files with 645 additions and 326 deletions
+355 -229
View File
@@ -1,15 +1,23 @@
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Arc;
use std::time::Duration;
use std::time::{Duration, Instant};
use tokio::sync::RwLock;
use tauri::{AppHandle, Emitter};
use serde::{Serialize, Deserialize};
use crate::jellyfin::http_client::HttpClient;
// Adaptive polling intervals (matches TypeScript)
const AUTO_CHECK_INTERVAL_MS: u64 = 30000; // 30 seconds when online
const RETRY_CHECK_INTERVAL_MS: u64 = 5000; // 5 seconds when offline
// Offline recovery probe interval.
// Reachability while online is driven by real repository traffic, so there is
// no online polling. While offline we probe quickly to detect the server
// returning even when no user traffic is flowing.
const RETRY_CHECK_INTERVAL_MS: u64 = 5000; // 5 seconds when offline
// Time-window debounce for declaring the server offline.
// A single dropped request must not trip the banner: we only flip to offline
// once network failures have persisted continuously for this window with no
// intervening success. Recovery (online) is instant on the first success.
const OFFLINE_CONFIRM_WINDOW: Duration = Duration::from_secs(5);
/// Connectivity status
#[derive(specta::Type, Debug, Clone, Serialize, Deserialize)]
@@ -45,210 +53,115 @@ struct ConnectivityChangeEvent {
is_reachable: bool,
}
/// Connectivity monitor for tracking server reachability
pub struct ConnectivityMonitor {
server_url: Arc<RwLock<Option<String>>>,
http_client: Arc<HttpClient>,
/// Shared reachability state and transition logic.
///
/// This is the single place that mutates reachability and emits events. It is
/// cheap to clone (all fields are `Arc`/`Option`) and is shared by:
/// - the `ConnectivityMonitor` (commands, offline recovery probe), and
/// - `OnlineRepository`, which reports the outcome of every server request.
///
/// Reachability is therefore driven by real traffic; the probe only fills the
/// gap while offline.
#[derive(Clone)]
pub struct ConnectivityReporter {
status: Arc<RwLock<ConnectivityStatus>>,
is_monitoring: Arc<AtomicBool>,
/// Timestamp of the first network failure in the current failure streak.
/// Used to debounce the transition to offline (see `OFFLINE_CONFIRM_WINDOW`).
first_failure_at: Arc<RwLock<Option<Instant>>>,
app_handle: Option<AppHandle>,
}
impl ConnectivityMonitor {
/// Create a new connectivity monitor
pub fn new(http_client: HttpClient) -> Self {
impl ConnectivityReporter {
fn new(status: Arc<RwLock<ConnectivityStatus>>, app_handle: Option<AppHandle>) -> Self {
Self {
server_url: Arc::new(RwLock::new(None)),
http_client: Arc::new(http_client),
status: Arc::new(RwLock::new(ConnectivityStatus::default())),
is_monitoring: Arc::new(AtomicBool::new(false)),
app_handle: None,
status,
first_failure_at: Arc::new(RwLock::new(None)),
app_handle,
}
}
/// Set the Tauri app handle for event emission
pub fn set_app_handle(&mut self, app_handle: AppHandle) {
self.app_handle = Some(app_handle);
/// Current reachability as seen by this reporter (shared with the monitor
/// and the UI). Useful for callers that want to branch on connectivity.
pub async fn is_reachable(&self) -> bool {
self.status.read().await.is_server_reachable
}
/// Update the server URL
pub async fn set_server_url(&self, url: String) {
log::info!("[ConnectivityMonitor] Setting server URL: {}", url);
let mut server_url = self.server_url.write().await;
*server_url = Some(url.clone());
drop(server_url);
// Check new server immediately
log::info!("[ConnectivityMonitor] Checking reachability of new server...");
let is_reachable = self.check_reachability().await;
log::info!("[ConnectivityMonitor] New server is {}", if is_reachable { "REACHABLE" } else { "UNREACHABLE" });
/// Test-only: force the reporter into the offline state without going through
/// the debounce, so other modules' tests can set up an "offline" precondition.
#[cfg(test)]
pub async fn mark_unreachable_for_test(&self) {
self.apply_probe_result(false, Some("forced offline (test)".to_string()))
.await;
}
/// Get current connectivity status
pub async fn get_status(&self) -> ConnectivityStatus {
self.status.read().await.clone()
/// Report that a real server request succeeded (or that the server answered
/// at all, e.g. with 401/404/5xx). The server is up — recover instantly.
pub async fn report_success(&self) {
*self.first_failure_at.write().await = None;
self.set_reachable(true, None).await;
}
/// Check if the Jellyfin server is reachable
pub async fn check_reachability(&self) -> bool {
// Mark as checking
{
let mut status = self.status.write().await;
status.is_checking = true;
/// Report that a real server request failed with a network-level error
/// (connection refused, timeout, DNS). Subject to the time-window debounce:
/// we only flip to offline once failures have persisted for
/// `OFFLINE_CONFIRM_WINDOW` with no intervening success.
pub async fn report_network_failure(&self, error: Option<String>) {
// If already offline, nothing to debounce.
if !self.status.read().await.is_server_reachable {
return;
}
let server_url = self.server_url.read().await.clone();
if server_url.is_none() {
log::warn!("[ConnectivityMonitor] Cannot check reachability: No server URL configured");
let mut status = self.status.write().await;
status.is_server_reachable = false;
status.connection_error = Some("No server URL configured".to_string());
status.is_checking = false;
return false;
}
let url = server_url.unwrap();
let ping_url = format!("{}/System/Info/Public", url);
// Store previous reachability state
let was_reachable = {
let status = self.status.read().await;
status.is_server_reachable
let now = Instant::now();
let streak_start = {
let mut first = self.first_failure_at.write().await;
*first.get_or_insert(now)
};
log::debug!("[ConnectivityMonitor] Pinging server: {}", ping_url);
if now.duration_since(streak_start) >= OFFLINE_CONFIRM_WINDOW {
log::warn!(
"[ConnectivityMonitor] Network failures sustained for {:?}; declaring offline",
OFFLINE_CONFIRM_WINDOW
);
self.set_reachable(false, error).await;
} else {
log::debug!(
"[ConnectivityMonitor] Network failure within debounce window; not yet offline"
);
}
}
// Attempt to ping the server
let is_reachable = self.http_client.ping(&ping_url).await;
/// Apply a deliberate reachability probe result (offline recovery probe or a
/// manual check). Unlike `report_network_failure`, a probe is an explicit
/// reachability test, so its result is applied immediately without debounce.
async fn apply_probe_result(&self, is_reachable: bool, error: Option<String>) {
if is_reachable {
*self.first_failure_at.write().await = None;
}
self.set_reachable(is_reachable, error).await;
}
log::debug!(
"[ConnectivityMonitor] Ping result: {} (was: {})",
if is_reachable { "SUCCESS" } else { "FAILED" },
if was_reachable { "reachable" } else { "unreachable" }
);
// Update status
{
/// Core transition: update status and emit events only on an actual change.
async fn set_reachable(&self, is_reachable: bool, error: Option<String>) {
let was_reachable = {
let mut status = self.status.write().await;
let was = status.is_server_reachable;
status.is_server_reachable = is_reachable;
status.last_checked = Some(chrono::Utc::now().to_rfc3339());
status.connection_error = if is_reachable {
None
} else {
Some("Server unreachable".to_string())
Some(error.unwrap_or_else(|| "Server unreachable".to_string()))
};
status.is_checking = false;
}
was
};
// Emit events if reachability changed
if is_reachable != was_reachable {
self.emit_connectivity_change(is_reachable).await;
}
// Emit reconnection event
if is_reachable && !was_reachable {
self.emit_server_reconnected().await;
}
is_reachable
}
/// Mark server as reachable (called after successful API call)
pub async fn mark_reachable(&self) {
let mut status = self.status.write().await;
let was_reachable = status.is_server_reachable;
status.is_server_reachable = true;
status.last_checked = Some(chrono::Utc::now().to_rfc3339());
status.connection_error = None;
drop(status);
if !was_reachable {
log::info!("[ConnectivityMonitor] Server marked as reachable (was unreachable)");
self.emit_connectivity_change(true).await;
self.emit_server_reconnected().await;
}
}
/// Mark server as unreachable (called after failed API call)
pub async fn mark_unreachable(&self, error: Option<String>) {
let mut status = self.status.write().await;
let was_reachable = status.is_server_reachable;
status.is_server_reachable = false;
status.last_checked = Some(chrono::Utc::now().to_rfc3339());
status.connection_error = error.or_else(|| Some("Server unreachable".to_string()));
let error_msg = status.connection_error.clone().unwrap_or_default();
drop(status);
if was_reachable {
log::warn!("[ConnectivityMonitor] Server marked as unreachable (was reachable): {}", error_msg);
self.emit_connectivity_change(false).await;
}
}
/// Start monitoring connectivity with adaptive polling
pub async fn start_monitoring(&self) {
if self.is_monitoring.swap(true, Ordering::SeqCst) {
log::info!("[ConnectivityMonitor] Already monitoring");
return;
}
log::info!("[ConnectivityMonitor] Starting connectivity monitoring");
// Perform immediate check before starting background task
// This ensures we get an accurate state right away instead of assuming offline
let is_reachable = self.check_reachability().await;
log::info!("[ConnectivityMonitor] Initial connectivity check: {}", if is_reachable { "ONLINE" } else { "OFFLINE" });
// Clone Arc references for the background task
let status = Arc::clone(&self.status);
let is_monitoring = Arc::clone(&self.is_monitoring);
let server_url = Arc::clone(&self.server_url);
let http_client = Arc::clone(&self.http_client);
let self_clone = Arc::new(ConnectivityMonitorHandle {
server_url,
http_client,
status,
app_handle: self.app_handle.clone(),
});
// Spawn background monitoring task
tokio::spawn(async move {
while is_monitoring.load(Ordering::SeqCst) {
// Determine interval based on current reachability
let interval_ms = {
let status = self_clone.status.read().await;
if status.is_server_reachable {
AUTO_CHECK_INTERVAL_MS
} else {
RETRY_CHECK_INTERVAL_MS
}
};
// Wait for the interval
tokio::time::sleep(Duration::from_millis(interval_ms)).await;
// Check if still monitoring
if !is_monitoring.load(Ordering::SeqCst) {
break;
}
// Perform connectivity check
let _ = self_clone.check_reachability().await;
if is_reachable {
self.emit_server_reconnected().await;
}
log::info!("[ConnectivityMonitor] Stopped monitoring");
});
}
/// Stop monitoring connectivity
pub fn stop_monitoring(&self) {
log::info!("[ConnectivityMonitor] Stopping connectivity monitoring");
self.is_monitoring.store(false, Ordering::SeqCst);
}
}
/// Emit connectivity change event to frontend
@@ -275,74 +188,160 @@ impl ConnectivityMonitor {
}
}
/// Handle for the background monitoring task
struct ConnectivityMonitorHandle {
/// Connectivity monitor for tracking server reachability.
///
/// Reachability is driven primarily by real repository traffic via the shared
/// [`ConnectivityReporter`]. The monitor itself only runs an offline recovery
/// probe (see `start_monitoring`) and serves the connectivity Tauri commands.
pub struct ConnectivityMonitor {
server_url: Arc<RwLock<Option<String>>>,
http_client: Arc<HttpClient>,
status: Arc<RwLock<ConnectivityStatus>>,
app_handle: Option<AppHandle>,
reporter: ConnectivityReporter,
is_monitoring: Arc<AtomicBool>,
}
impl ConnectivityMonitorHandle {
async fn check_reachability(&self) -> bool {
impl ConnectivityMonitor {
/// Create a new connectivity monitor
pub fn new(http_client: HttpClient) -> Self {
let status = Arc::new(RwLock::new(ConnectivityStatus::default()));
Self {
server_url: Arc::new(RwLock::new(None)),
http_client: Arc::new(http_client),
reporter: ConnectivityReporter::new(status, None),
is_monitoring: Arc::new(AtomicBool::new(false)),
}
}
/// Set the Tauri app handle for event emission.
/// Must be called before the reporter is shared with the repository.
pub fn set_app_handle(&mut self, app_handle: AppHandle) {
self.reporter.app_handle = Some(app_handle);
}
/// Get a cheap, cloneable reporter so the repository can feed server
/// outcomes into the same reachability state the UI observes.
pub fn reporter(&self) -> ConnectivityReporter {
self.reporter.clone()
}
/// Update the server URL
pub async fn set_server_url(&self, url: String) {
log::info!("[ConnectivityMonitor] Setting server URL: {}", url);
*self.server_url.write().await = Some(url);
// Check new server immediately
log::info!("[ConnectivityMonitor] Checking reachability of new server...");
let is_reachable = self.check_reachability().await;
log::info!("[ConnectivityMonitor] New server is {}", if is_reachable { "REACHABLE" } else { "UNREACHABLE" });
}
/// Get current connectivity status
pub async fn get_status(&self) -> ConnectivityStatus {
self.reporter.status.read().await.clone()
}
/// Deliberately probe the server's reachability (manual check / recovery probe).
/// The result is applied immediately (no debounce) since this is an explicit test.
pub async fn check_reachability(&self) -> bool {
{
let mut status = self.reporter.status.write().await;
status.is_checking = true;
}
let server_url = self.server_url.read().await.clone();
if server_url.is_none() {
let Some(url) = server_url else {
log::warn!("[ConnectivityMonitor] Cannot check reachability: No server URL configured");
self.reporter
.apply_probe_result(false, Some("No server URL configured".to_string()))
.await;
return false;
}
let url = server_url.unwrap();
let ping_url = format!("{}/System/Info/Public", url);
// Store previous reachability state
let was_reachable = {
let status = self.status.read().await;
status.is_server_reachable
};
// Attempt to ping the server
let ping_url = format!("{}/System/Info/Public", url);
log::debug!("[ConnectivityMonitor] Pinging server: {}", ping_url);
let is_reachable = self.http_client.ping(&ping_url).await;
log::debug!(
"[ConnectivityMonitor] Ping result: {}",
if is_reachable { "SUCCESS" } else { "FAILED" }
);
// Update status
{
let mut status = self.status.write().await;
status.is_server_reachable = is_reachable;
status.last_checked = Some(chrono::Utc::now().to_rfc3339());
status.connection_error = if is_reachable {
None
} else {
Some("Server unreachable".to_string())
};
}
// Emit events if reachability changed
if is_reachable != was_reachable {
self.emit_connectivity_change(is_reachable).await;
}
// Emit reconnection event
if is_reachable && !was_reachable {
self.emit_server_reconnected().await;
}
self.reporter.apply_probe_result(is_reachable, None).await;
is_reachable
}
async fn emit_connectivity_change(&self, is_reachable: bool) {
if let Some(app_handle) = &self.app_handle {
let event = ConnectivityChangeEvent { is_reachable };
if let Err(e) = app_handle.emit("connectivity:changed", event) {
log::error!("[ConnectivityMonitor] Failed to emit connectivity change event: {}", e);
}
}
/// Mark server as reachable (called after successful API call / login)
pub async fn mark_reachable(&self) {
self.reporter.report_success().await;
}
async fn emit_server_reconnected(&self) {
if let Some(app_handle) = &self.app_handle {
if let Err(e) = app_handle.emit("connectivity:reconnected", ()) {
log::error!("[ConnectivityMonitor] Failed to emit reconnection event: {}", e);
}
/// Mark server as unreachable directly.
///
/// Used by deliberate signals (e.g. a failed login/connect) where the caller
/// knows the server is unreachable now. Repository traffic should prefer
/// `reporter().report_network_failure()` so the debounce applies.
pub async fn mark_unreachable(&self, error: Option<String>) {
self.reporter.apply_probe_result(false, error).await;
}
/// Start the offline recovery probe.
///
/// While **online**, reachability is kept fresh by real traffic, so the probe
/// idles. While **offline**, it polls `/System/Info/Public` every
/// `RETRY_CHECK_INTERVAL_MS` to detect the server returning even when no user
/// traffic is flowing.
pub async fn start_monitoring(&self) {
if self.is_monitoring.swap(true, Ordering::SeqCst) {
log::info!("[ConnectivityMonitor] Already monitoring");
return;
}
log::info!("[ConnectivityMonitor] Starting connectivity monitoring (offline recovery probe)");
// Perform an immediate check so startup reflects reality quickly.
let is_reachable = self.check_reachability().await;
log::info!("[ConnectivityMonitor] Initial connectivity check: {}", if is_reachable { "ONLINE" } else { "OFFLINE" });
let is_monitoring = Arc::clone(&self.is_monitoring);
let server_url = Arc::clone(&self.server_url);
let http_client = Arc::clone(&self.http_client);
let reporter = self.reporter.clone();
tokio::spawn(async move {
while is_monitoring.load(Ordering::SeqCst) {
tokio::time::sleep(Duration::from_millis(RETRY_CHECK_INTERVAL_MS)).await;
if !is_monitoring.load(Ordering::SeqCst) {
break;
}
// Only probe while offline — real traffic is the signal when online.
if reporter.status.read().await.is_server_reachable {
continue;
}
let Some(url) = server_url.read().await.clone() else {
continue;
};
let ping_url = format!("{}/System/Info/Public", url);
let is_reachable = http_client.ping(&ping_url).await;
// Probe only ever recovers us to online; a failed probe leaves us
// offline without re-emitting (no change).
if is_reachable {
reporter.apply_probe_result(true, None).await;
}
}
log::info!("[ConnectivityMonitor] Stopped monitoring");
});
}
/// Stop monitoring connectivity
pub fn stop_monitoring(&self) {
log::info!("[ConnectivityMonitor] Stopping connectivity monitoring");
self.is_monitoring.store(false, Ordering::SeqCst);
}
}
@@ -350,11 +349,22 @@ impl ConnectivityMonitorHandle {
mod tests {
use super::*;
/// Build a reporter backed by a fresh (optimistic) status, with no app handle.
/// Event emission is a no-op without a handle, which is exactly what we want
/// for unit-testing the reachability state transitions.
fn test_reporter() -> ConnectivityReporter {
ConnectivityReporter::new(Arc::new(RwLock::new(ConnectivityStatus::default())), None)
}
async fn is_reachable(reporter: &ConnectivityReporter) -> bool {
reporter.status.read().await.is_server_reachable
}
#[test]
fn test_intervals() {
// Verify intervals match TypeScript
assert_eq!(AUTO_CHECK_INTERVAL_MS, 30000);
// Offline recovery probe interval (online has no polling).
assert_eq!(RETRY_CHECK_INTERVAL_MS, 5000);
assert_eq!(OFFLINE_CONFIRM_WINDOW, Duration::from_secs(5));
}
#[tokio::test]
@@ -366,4 +376,120 @@ mod tests {
assert!(status.connection_error.is_none());
assert!(!status.is_checking);
}
/// A single (or brief) network failure must NOT flip the app offline:
/// the time-window debounce keeps us online until the failure persists.
///
/// @req-test: UR-002 - Access media when online or offline
#[tokio::test]
async fn test_single_network_failure_does_not_go_offline() {
let reporter = test_reporter();
assert!(is_reachable(&reporter).await, "starts online");
reporter
.report_network_failure(Some("timeout".to_string()))
.await;
assert!(
is_reachable(&reporter).await,
"one network failure within the debounce window stays online"
);
// But the failure streak is now being tracked.
assert!(reporter.first_failure_at.read().await.is_some());
}
/// Once failures persist past OFFLINE_CONFIRM_WINDOW, we flip offline.
/// We simulate elapsed time by backdating the streak start.
///
/// @req-test: UR-002 - Access media when online or offline
#[tokio::test]
async fn test_sustained_network_failure_goes_offline() {
let reporter = test_reporter();
// First failure starts the streak.
reporter.report_network_failure(None).await;
assert!(is_reachable(&reporter).await);
// Backdate the streak start to before the window.
{
let mut first = reporter.first_failure_at.write().await;
*first = Some(Instant::now() - OFFLINE_CONFIRM_WINDOW - Duration::from_secs(1));
}
// Next failure now exceeds the window → offline.
reporter
.report_network_failure(Some("connection refused".to_string()))
.await;
assert!(
!is_reachable(&reporter).await,
"sustained network failure flips to offline"
);
}
/// A success during a failure streak clears the streak and keeps us online —
/// recovery is instant and never trips the banner.
#[tokio::test]
async fn test_success_clears_failure_streak() {
let reporter = test_reporter();
reporter.report_network_failure(None).await;
assert!(reporter.first_failure_at.read().await.is_some());
reporter.report_success().await;
assert!(is_reachable(&reporter).await);
assert!(
reporter.first_failure_at.read().await.is_none(),
"success resets the debounce streak"
);
}
/// First success after being offline recovers instantly (no debounce on the
/// way back up).
#[tokio::test]
async fn test_recovery_is_instant() {
let reporter = test_reporter();
// Force offline.
reporter.apply_probe_result(false, Some("down".to_string())).await;
assert!(!is_reachable(&reporter).await);
// A single success brings us straight back online.
reporter.report_success().await;
assert!(is_reachable(&reporter).await);
let status = reporter.status.read().await;
assert!(status.connection_error.is_none());
}
/// Server-answered errors (401/404/5xx) are reported via report_success
/// by the repository, because the server is demonstrably reachable. This
/// test documents that contract: report_success means "server is up".
#[tokio::test]
async fn test_server_answered_error_counts_as_reachable() {
let reporter = test_reporter();
// Simulate being offline, then the server answers (even with an error).
reporter.apply_probe_result(false, None).await;
assert!(!is_reachable(&reporter).await);
// Repository maps Authentication/NotFound/Server errors to report_success.
reporter.report_success().await;
assert!(
is_reachable(&reporter).await,
"a server that answers (even with 4xx/5xx) is reachable"
);
}
/// report_network_failure is a no-op once already offline (nothing to debounce,
/// no duplicate events).
#[tokio::test]
async fn test_network_failure_noop_when_already_offline() {
let reporter = test_reporter();
reporter.apply_probe_result(false, None).await;
assert!(!is_reachable(&reporter).await);
// Should not panic or change state.
reporter.report_network_failure(Some("still down".to_string())).await;
assert!(!is_reachable(&reporter).await);
}
}