From c0399e4ebda0a064964c40677f3648ee4a67df96 Mon Sep 17 00:00:00 2001 From: Duncan Tourolle Date: Thu, 27 Aug 2026 17:56:52 +0200 Subject: [PATCH] fix(player): stop PiP dropping the video and restarting the audio behind it Watching in a picture-in-picture window would occasionally drop to audio-only, and the audio would resume from wherever the video had been when PiP was entered while the picture had carried on past it. Two independent faults, both needed to produce that. The position froze (DR-265). VideoPlayer tracks the absolute position in its own `currentTime` rather than reading `videoElement.currentTime` at the point of use, because transcoded HLS resets the element to 0 on every segment rebuild. While playing, that variable had exactly one writer: a requestAnimationFrame loop. RAF is driven by the document being rendered, and an Android activity behind a PiP window is paused, so the loop stops while the element plays on. The `timeupdate` handler that would have covered the gap was written as a fallback "for when RAF isn't running" and gated itself on `!isPlaying` -- switching itself off at precisely the moment it was the only source left. Everything downstream froze with it: the seek bar, the ten-second progress reports, the position mirrored into Rust, and the handoff. The gate is now `shouldApplyTimeUpdate` and turns only on things that genuinely own the position -- an in-flight seek, a seek-bar drag, an element below HAVE_CURRENT_DATA. Both writers producing the same derived value costs nothing. The handoff fired at all (DR-266). PiP and the background-audio handoff are alternatives -- one keeps the picture, the other throws it away -- but exclusivity was enforced from one side only: arming the toggle suppressed *auto*-PiP, while the PiP button stayed ungated, so pressing it left both armed. What then stood between them was `isInPictureInPictureMode`, sampled once inside MainActivity.onStop(). That sample is not reliable: the keyguard dismissing the window, the window being stashed, or OEM variance in when onPictureInPictureModeChanged(false) lands can all leave the activity stopped with a window still on screen and the flag reading false. Now entering PiP disarms background audio, both directions go through one BackgroundBehaviour pair, and the PiP question accepts either witness -- the native sample or the frontend's latch over jellytau-pip-entered/exited. The latch cannot report a window that has closed: both events reach the WebView through the same message queue in dispatch order. The decision itself stays in Rust; the frontend only supplies a fact it can establish more reliably than the activity can. Red first, both: the existing behaviour was extracted into pure helpers, the tests written against the correct behaviour, and both watched to fail before either was changed. --- docs/requirements.md | 10 ++- src/lib/components/player/VideoPlayer.svelte | 69 +++++++++++++++---- .../player/backgroundAudioHandoff.test.ts | 69 +++++++++++++++++++ .../player/backgroundAudioHandoff.ts | 58 ++++++++++++++++ .../components/player/timeTracking.test.ts | 45 ++++++++++++ src/lib/components/player/timeTracking.ts | 45 ++++++++++++ 6 files changed, 279 insertions(+), 17 deletions(-) create mode 100644 src/lib/components/player/timeTracking.test.ts create mode 100644 src/lib/components/player/timeTracking.ts diff --git a/docs/requirements.md b/docs/requirements.md index 5dc3e000..7a990efa 100644 --- a/docs/requirements.md +++ b/docs/requirements.md @@ -458,6 +458,8 @@ Internal architecture, components, and application logic. | DR-262 | The A-Z jump strip is bounded by the **scroller it lives in**, not by the viewport minus a guess at the bottom bars. `AlphabetScrollBar` sized itself as `window.innerHeight` minus a hardcoded `bottomGap` — 5rem, 7rem or 11rem, picked by platform and whether the mini player was showing — which dates from when the mini player and bottom nav were `position: fixed` overlays. They have been in-flow flex siblings below the scroller since BottomUi (DR-009), so the scroller's own bottom edge *is* the top of the mini player and can simply be measured. The guess was short on every device with a navigation or gesture bar, because `--safe-bottom` is padded *inside* BottomUi (DR-112) and no guess knew about it: the strip overran the scrollport by ~45px with the nav alone, ~18px with the mini player and ~50px in remote mode, burying one to three letters where they could not be tapped. The ancestor is resolved by computed `overflow-y` rather than `closest("main")`, since the root shell scrolls in a plain `
` and a miss silently fell back to the viewport — reinstating the bug on any route outside `/library`. Observing the scroller for resize is also what makes the mini player appearing re-measure, so the component no longer subscribes to player or platform stores at all | UI | UR-007 | Done | | DR-263 | Autoplay crosses the **season boundary**. `fetch_next_episode_for_item` listed the episodes of the current season and stopped dead at the last one, so the end of a season produced `AutoplayDecision::Stop`. On the Android background-audio handoff (UR-040) that is felt as playback simply pausing mid-binge with the screen locked and no UI to un-pause it — the same end that mid-season advances through in the backend. The lookup now walks the series' seasons, sorted client-side by index number because the offline repository ignores `sort_by`, and takes the first episode of the next season that has any, skipping empty ones. Specials are never rolled *into*: Jellyfin numbers them 0 so they sort ahead of season 1, but a server that leaves the index unset sorts them last, exactly where the walk would land. The lookup sits below the sleep-timer gate in `on_playback_ended`, so a timer set to end-of-episode or a remaining-episode count still stops at the boundary rather than being carried past it | Player | UR-023, UR-040 | Done | | DR-264 | The episode a viewer *just finished* is no longer offered as the one they are up to. Nothing records completion locally: the stop report writes a position through `storage_update_playback_progress` (which never sets `is_played`), and the cache mirror carried the server's flag not at all — so on a cache hit every episode read back as unwatched. Leaving the player with Back reloads the series page within a second of the stop report, inside the window where Jellyfin's Next Up still names the episode that just ended, and `pick_current_episode` handed it straight back: the season view kept the yellow ring and the "Up next" badge on the episode the viewer had just watched, and scrolled to it. Two halves. (a) `is_finished` — the played flag **or** a position at or past `MAX_PROGRESS_FRACTION` of the runtime, the same 95% threshold that already disqualifies an episode from counting as in-progress — replaces the bare `is_played` in the furthest-watched scan and the first-unwatched fallback, and screens the Next Up candidate: the server is one stop-report behind for a moment, the local position is not. (b) `OfflineRepository::mirror_user_data` carries `is_played` alongside the favourite flag and the position, under the same `pending_sync = 0` conflict rule, so watched state survives a cache write instead of being dropped — that flag was previously written by nothing but an explicit local toggle | Repository | UR-062 | Done | +| DR-265 | The player's position variable keeps advancing behind a picture-in-picture window. `VideoPlayer` tracks the absolute position in its own `currentTime` rather than reading `videoElement.currentTime` at the point of use — transcoded HLS resets the element to 0 on every segment rebuild, so only the running total is meaningful — and that variable had exactly one writer while playing: a `requestAnimationFrame` loop. RAF is driven by the document being rendered, and an Android activity behind a PiP window is paused, so the loop stops while the element plays on. The `timeupdate` handler that would have covered the gap was written as a fallback "for when RAF isn't running" and gated itself on `!isPlaying`, switching itself off at precisely the moment it was the only source left. `currentTime` therefore froze at the instant PiP was entered, and every consumer froze with it: the seek bar, the ten-second progress reports, the position mirrored into Rust through `html5Adapter`, and — the reported symptom — the background-audio handoff, which resumed the audio-only stream at the PiP-entry position while the picture carried on where it really was. The gate is now `shouldApplyTimeUpdate` and turns only on the things that genuinely own the position instead: an in-flight seek, a seek-bar drag, and an element whose `readyState` is below `HAVE_CURRENT_DATA` (which reads 0 and would rewind). Both writers producing the same derived value costs nothing — the element is the authority either way | Player | UR-004, UR-041 | Done | +| DR-266 | PiP and the background-audio handoff can no longer be armed at once, and neither can a single stale boolean end the picture. They are alternatives — one keeps the video on screen, the other throws it away — but exclusivity was enforced from one side only: arming the toggle called `setAutoEnterEnabled(false)`, while the PiP *button* stayed ungated and still worked, so pressing it left both live. What then decided between them was `isInPictureInPictureMode`, sampled once inside `MainActivity.onStop()` and passed to `background_action`. That sample is not reliable: there are orderings — the keyguard dismissing the window, the window being stashed, OEM variance in when `onPictureInPictureModeChanged(false)` lands relative to `onStop` — where the activity is stopped with a PiP window still on screen and the flag reads false. Backgrounding then meant "the app is gone" and handed a video the user was watching in the window off to audio-only. Two halves. (a) `enteringPictureInPicture` disarms background audio, because pressing PiP is an unambiguous request to keep the picture; both directions now go through one `BackgroundBehaviour` pair rather than two ad-hoc call sites. (b) `inPictureInPicture` accepts either witness — the native sample or the frontend's own latch over `jellytau-pip-entered`/`jellytau-pip-exited`. The latch cannot report a window that has closed, because both events reach the WebView through the same message queue in dispatch order, so a genuine exit is always known before the background signal that follows it. The decision itself stays in Rust; the frontend only supplies a fact it can establish more reliably than the activity can | Player | UR-040, UR-041 | Done | | DR-198 | The webview runs under a real Content-Security-Policy, and the asset protocol is scoped to the one directory it still serves. `csp` was `null`, which disables CSP entirely: any script that reached the web layer — through a future `{@html}`, a dependency, or a devtools paste — would have inherited the whole IPC surface, and with it the user's session. `script-src 'self'` (Tauri injects a nonce for SvelteKit's inline bootstrap script at build time, so no `'unsafe-inline'` is needed) plus `object-src`/`frame-src 'none'` and `base-uri 'self'` is the part that is genuinely restrictive. `img-src`/`media-src`/`connect-src` cannot be: the Jellyfin origin is typed in by the user at run time and is commonly plain `http` on a LAN, so they allow `http:`/`https:` — a wide grant for *data*, but one that still bars `file:`, `filesystem:` and scripting schemes, and leaves `script-src` untouched. `style-src` keeps `'unsafe-inline'` because Svelte compiles `style="…"` attributes (including `app.html`'s `display: contents` wrapper) into markup; this is safe only while no `