dtourolle/jellytau
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 52 Wiki Activity

refactor(player): delete the webview video path; mpv selects its own tracks

Browse Source 
DR-235 phase 3. Every video renderer is native now: mpv on Linux and
Windows, ExoPlayer on Android, all drawing behind the transparent
webview. The HTML5 <video> path is gone, not bypassed:

- Frontend: hls.js, Html5PlayerAdapter and its compatibility shim, the
  createAdapter factory, streamTransport, hlsRecovery, timeTracking,
  videoFit, the <video>/<track> markup and every element handler in
  VideoPlayer (3277 -> 2144 lines), the experimentalNativeVideo store
  and its Settings toggle, webviewVideoFallback/supportsNativeVideo, and
  the setHtml5VideoState PiP bridge call. NativePlayerAdapter is the one
  video adapter; webview audio gets its own adapter kind.
- Rust: use_html5 dropped from player_seek_video,
  player_switch_audio_track and player_set_stream_quality with the
  Html5* strategies and ReloadStream responses; use_html5_element and
  VideoBackend dropped from PlayerStatus; player_play_item always loads
  the backend (set_current_item removed); Capabilities::webview removed;
  the WebKitGTK GStreamer/VAAPI setup (and its gst-inspect spawn) removed.
- Android: the HTML5 video state in PictureInPictureManager and
  ScreenWakeManager, and the bridge method feeding it.
- CSP: connect-src loses http:/https: and worker-src loses blob: -
  both existed for hls.js; with it gone they were only an exfiltration
  channel and a blob worker for injected script. A test now keeps them
  out.

mpv takes over what the <video> element did (mpv_tracks, UT-275):
subtitles are the WebVTT list the play request carries, queued on
sub-files and selected by position in that list, starting off; audio
tracks are selected by position in the file; sid/aid are reset before
each load. Without this, Linux video had no subtitle selection and a
direct-play audio switch failed since mpv became its renderer.

Verified: Rust 948 passing, and the same 948 cross-compiled for Windows
under wine against the shipped DLL (track tests included); frontend
1111 passing; aarch64 debug APK builds. Lint warnings 158 -> 146, CI
ratchet tightened to match. Not yet seen on Windows hardware.
This commit is contained in:
dtourolle
2026-09-24 23:11:17 -04:00
parent bb3ab1edd7
commit 1677f5f299
69 changed files with 4014 additions and 7330 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src-tauri/android/src/main/java/com/dtourolle/jellytau/MainActivity.kt
+3 -20
View File
@@ -85,10 +85,9 @@ class MainActivity : TauriActivity() {
super.onWebViewCreate(webView)
android.util.Log.d("MainActivity", "onWebViewCreate - installing bridges before first page load")
mediaWebView = webView
// A new WebView means a new page, which reports no video yet. Anything the
// previous one left held would otherwise pin the screen on for the life of
// the process, since a page that goes away never sends its final
// setHtml5VideoState(false, …). (DR-202)
// A new WebView means a new page, which plays no video yet. Anything held
// for the previous one would otherwise pin the screen on for the life of
// the process. (DR-202)
ScreenWakeManager.releaseAll()
installJavascriptBridges(webView)
configureWebViewSettings(webView)
@@ -327,22 +326,6 @@ class MainActivity : TauriActivity() {
autoEnterPipEnabled = enabled
}
/**
* Report the WebView `<video>` state.
*
* Without this PiP only ever knew about the native ExoPlayer surface,
* which is behind an experimental flag that defaults to off — so in the
* shipping configuration nothing ever satisfied canEnterPip and the
* button did nothing. (DR-160)
*/
@JavascriptInterface
fun setHtml5VideoState(active: Boolean, width: Int, height: Int, playing: Boolean) {
PictureInPictureManager.setHtml5VideoState(active, width, height, playing)
// The same report is what keeps the display awake on the webview
// rendering path — the WebView takes no display wake lock of its own
// for `<video>`. (DR-202)
ScreenWakeManager.onHtml5VideoState(active, playing)
}
}, "AndroidPictureInPicture")
android.util.Log.d("MainActivity", "JavaScript interface 'AndroidPictureInPicture' added")
src-tauri/android/src/main/java/com/dtourolle/jellytau/PictureInPictureManager.kt
+21 -93
View File
@@ -46,52 +46,7 @@ object PictureInPictureManager {
private var receiver: BroadcastReceiver? = null
private var hiddenWebView: WebView? = null
/**
* State of an HTML5 `<video>` playing inside the WebView, reported by the
* frontend.
*
* PiP was written for the native ExoPlayer surface only — [canEnterPip]
* required a SurfaceView to be attached and rendering. But native video is
* behind `experimentalNativeVideo`, which defaults to **off**, so in the
* shipping configuration video plays in the WebView's `<video>` element and
* every one of those conditions is false. `enterPip` therefore always bailed
* with "no local video playing": PiP could not work at all, however the
* button was pressed.
*
* On this path the WebView *is* the video, which inverts two things: the
* WebView must stay visible in PiP rather than be hidden, and play/pause has
* to reach the element rather than ExoPlayer. Both are handled below.
*
* TRACES: UR-041 | DR-160
*/
@Volatile
private var html5VideoActive = false
@Volatile
private var html5VideoPlaying = false
@Volatile
private var html5AspectRatio: Rational? = null
/**
* Report the WebView `<video>` state from the frontend.
*
* @param active whether a video element is currently the playback surface
* @param width intrinsic video width, for the PiP window's aspect ratio
* @param height intrinsic video height
* @param playing whether it is playing right now, for the PiP play/pause action
*/
fun setHtml5VideoState(active: Boolean, width: Int, height: Int, playing: Boolean) {
html5VideoActive = active
html5VideoPlaying = playing
html5AspectRatio = if (active && width > 0 && height > 0) {
clampedRatio(width.toDouble() / height.toDouble())
} else {
null
}
}
/** True when PiP would be showing the native surface rather than the WebView. */
/** True when a native video surface is attached and playing. */
private fun isNativeVideoPath(): Boolean = try {
val player = JellyTauPlayer.getInstance()
player.isPlayingVideo() &&
@@ -120,10 +75,9 @@ object PictureInPictureManager {
*/
fun canEnterPip(activity: Activity): Boolean {
if (!isPipSupported(activity)) return false
// Either surface will do: the native one, or the WebView's `<video>`,
// which is what actually plays while experimentalNativeVideo is off.
// (DR-160)
return isNativeVideoPath() || html5VideoActive
// Video only ever renders on the native surface: the WebView `<video>`
// path is gone (DR-235). (DR-160)
return isNativeVideoPath()
}
/**
@@ -186,9 +140,7 @@ object PictureInPictureManager {
return clampedRatio(surface.width.toDouble() / surface.height.toDouble())
}
// No native surface: the WebView is the video, so use the intrinsic size
// the frontend reported. (DR-160)
return html5AspectRatio
return null
}
/**
@@ -207,16 +159,10 @@ object PictureInPictureManager {
@RequiresApi(Build.VERSION_CODES.O)
private fun buildPlayPauseAction(activity: Activity): RemoteAction {
// On the HTML5 path ExoPlayer is idle, so its `isPlaying` is always false
// and the button would be stuck showing "Play" mid-playback. (DR-160)
val isPlaying = if (isNativeVideoPath()) {
try {
JellyTauPlayer.getInstance().getExoPlayer().isPlaying
} catch (e: Exception) {
false
}
} else {
html5VideoPlaying
val isPlaying = try {
JellyTauPlayer.getInstance().getExoPlayer().isPlaying
} catch (e: Exception) {
false
}
val (iconRes, title, controlType, requestCode) = if (isPlaying) {
@@ -288,11 +234,8 @@ object PictureInPictureManager {
*/
fun onPipModeChanged(activity: Activity, isInPipMode: Boolean) {
if (isInPipMode) {
// Hiding the WebView is correct only when the video is *behind* it on
// the native surface. On the HTML5 path the WebView is the video, so
// hiding it would leave an empty black PiP window — the frontend
// instead strips its own chrome when it hears the event below.
// (DR-160)
// The video is *behind* the WebView on the native surface, so hiding
// the WebView leaves only the picture. (DR-160)
if (isNativeVideoPath()) {
hideWebView(activity)
}
@@ -315,9 +258,8 @@ object PictureInPictureManager {
/**
* Fire a DOM event into the WebView.
*
* The HTML5 PiP path is a conversation with the frontend rather than
* something native can do alone: it has to be told to strip its chrome when
* the window shrinks, and to play/pause the element. (DR-160)
* Tells the frontend the window shrank or grew, so it can strip or restore
* its chrome. (DR-160)
*/
private fun dispatchWebEvent(activity: Activity, name: String) {
val webView = findWebView(activity.window.decorView) ?: return
@@ -358,28 +300,14 @@ object PictureInPictureManager {
if (intent?.action != ACTION_MEDIA_CONTROL) return
val control = intent.getIntExtra(EXTRA_CONTROL_TYPE, 0)
if (isNativeVideoPath()) {
val player = try {
JellyTauPlayer.getInstance()
} catch (e: Exception) {
return
}
when (control) {
CONTROL_PLAY -> player.play()
CONTROL_PAUSE -> player.pause()
}
} else {
// The WebView owns playback here, so the command has to reach
// the `<video>` element. Driving ExoPlayer instead would do
// nothing at all, which is what a PiP button on the HTML5 path
// used to do. (DR-160)
val name = when (control) {
CONTROL_PLAY -> "jellytau-pip-play"
CONTROL_PAUSE -> "jellytau-pip-pause"
else -> return
}
dispatchWebEvent(activity, name)
html5VideoPlaying = control == CONTROL_PLAY
val player = try {
JellyTauPlayer.getInstance()
} catch (e: Exception) {
return
}
when (control) {
CONTROL_PLAY -> player.play()
CONTROL_PAUSE -> player.pause()
}
// Swap the button to reflect the new state.
updatePipActions(activity)
src-tauri/android/src/main/java/com/dtourolle/jellytau/ScreenWakeManager.kt
+13 -44
View File
@@ -7,14 +7,12 @@ import android.view.WindowManager
import java.lang.ref.WeakReference
/**
* Which playback paths currently want the screen kept awake.
* Whether playback currently wants the screen kept awake.
*
* Pure state, deliberately free of any Android type so it can be unit-tested —
* see ScreenWakeStateTest. Two independent holders, because video can be
* rendered by either renderer and only one of them is active at a time:
*
* - **native** — ExoPlayer drawing into the TextureView (DR-192)
* - **html5** — a `<video>` inside the WebView, reported by the frontend
* see ScreenWakeStateTest. The one holder is ExoPlayer drawing video into the
* TextureView (DR-192); the WebView `<video>` that was a second holder is gone
* (DR-235).
*
* Audio is deliberately *not* a holder. Playing music with the screen off is the
* point of the audio path; only video needs the display alive.
@@ -23,11 +21,10 @@ import java.lang.ref.WeakReference
*/
class ScreenWakeState {
private var nativeVideoPlaying = false
private var html5VideoPlaying = false
/** True while any video renderer is actively playing. */
/** True while video is actively playing. */
val keepScreenOn: Boolean
get() = nativeVideoPlaying || html5VideoPlaying
get() = nativeVideoPlaying
/**
* @param playing whether ExoPlayer is playing right now
@@ -37,18 +34,9 @@ class ScreenWakeState {
nativeVideoPlaying = playing && isVideo
}
/**
* @param active whether a webview `<video>` is the current playback surface
* @param playing whether that element is playing right now
*/
fun updateHtml5(active: Boolean, playing: Boolean) {
html5VideoPlaying = active && playing
}
/** Drop every hold (teardown, or a page that can no longer be trusted). */
fun reset() {
nativeVideoPlaying = false
html5VideoPlaying = false
}
}
@@ -66,9 +54,7 @@ class ScreenWakeState {
* appeared nowhere, and neither renderer supplies one for free — ExoPlayer's
* `setWakeMode` is a *CPU/wifi* wake lock and says nothing about the display,
* and it draws into a `TextureView` we own rather than a `PlayerView`, which is
* the media3 widget that would otherwise set `keepScreenOn` itself. The WebView
* `<video>` path does not either: the display wake lock Chrome takes for video
* lives in the browser layer, not in an embedded WebView.
* the media3 widget that would otherwise set `keepScreenOn` itself.
*
* ## Approach
*
@@ -78,15 +64,9 @@ class ScreenWakeState {
* missed release the way an explicitly acquired wake lock can. It needs no
* permission. (The manifest's `WAKE_LOCK` is the media service's, unrelated.)
*
* The two renderers report independently and are OR-ed together in
* [ScreenWakeState]:
*
* - `JellyTauPlayer.onIsPlayingChanged` and its surface teardown drive the
* native path — ExoPlayer is the authoritative source of playback state, so
* the hold follows what it reports rather than what the UI intends.
* - `MainActivity`'s `AndroidPictureInPicture.setHtml5VideoState` bridge drives
* the webview path. The frontend already reports that state on every
* play/pause and on player teardown for PiP, so no new bridge is needed.
* `JellyTauPlayer.onIsPlayingChanged` and its surface teardown drive
* [ScreenWakeState] — ExoPlayer is the authoritative source of playback state,
* so the hold follows what it reports rather than what the UI intends.
*
* The Activity reference is weak and re-set on every `onCreate`, so a
* recreation (rotation) re-applies the current hold to the new window.
@@ -126,20 +106,9 @@ object ScreenWakeManager {
}
/**
* The frontend reported the webview `<video>` state. Arrives on a WebView
* binder thread, hence the synchronization and the post to the main thread.
*/
@Synchronized
fun onHtml5VideoState(active: Boolean, playing: Boolean) {
state.updateHtml5(active, playing)
apply()
}
/**
* Drop every hold. Used when a new WebView/page load invalidates whatever the
* previous page last reported — a page that goes away without a final
* `setHtml5VideoState(false, …)` would otherwise leave the screen pinned on
* for the life of the process.
* Drop every hold. Used when a new WebView/page load starts from scratch, so
* nothing held for the previous page can pin the screen on for the life of
* the process.
*/
@Synchronized
fun releaseAll() {
src-tauri/android/src/test/java/com/dtourolle/jellytau/ScreenWakeStateTest.kt
+1 -38
View File
@@ -40,46 +40,9 @@ class ScreenWakeStateTest {
}
@Test
fun `webview video playing holds the screen on`() {
val state = ScreenWakeState()
state.updateHtml5(active = true, playing = true)
assertTrue(state.keepScreenOn)
}
@Test
fun `webview video paused releases the screen`() {
val state = ScreenWakeState()
state.updateHtml5(active = true, playing = true)
state.updateHtml5(active = true, playing = false)
assertFalse(state.keepScreenOn)
}
/** The element going away must release even if it never reported a pause. */
@Test
fun `webview video going inactive while playing releases the screen`() {
val state = ScreenWakeState()
state.updateHtml5(active = true, playing = true)
state.updateHtml5(active = false, playing = true)
assertFalse(state.keepScreenOn)
}
/** The two rendering paths are independent holders; either one is enough. */
@Test
fun `one path releasing does not release while the other still plays`() {
fun `teardown releases the hold`() {
val state = ScreenWakeState()
state.updateNative(playing = true, isVideo = true)
state.updateHtml5(active = true, playing = true)
state.updateHtml5(active = false, playing = false)
assertTrue(state.keepScreenOn)
state.updateNative(playing = false, isVideo = true)
assertFalse(state.keepScreenOn)
}
@Test
fun `teardown releases both paths`() {
val state = ScreenWakeState()
state.updateNative(playing = true, isVideo = true)
state.updateHtml5(active = true, playing = true)
state.reset()
assertFalse(state.keepScreenOn)
}
src-tauri/src/commands/player/mod.rs
+75 -256
View File
@@ -69,10 +69,6 @@ pub struct PlayerStatus {
pub muted: bool,
pub shuffle: bool,
pub repeat: RepeatMode,
/// Backend being used (native = ExoPlayer/libmpv, html5 = fallback)
pub backend: VideoBackend,
/// Whether frontend should render HTML5 video element
pub use_html5_element: bool,
// Merged fields (prefer remote session when available)
/// Media item from either local queue or remote session
@@ -156,16 +152,6 @@ pub struct QueueStatus {
pub has_previous: bool,
}
/// Backend type for video playback
#[derive(specta::Type, Debug, Serialize)]
#[serde(rename_all = "lowercase")]
pub enum VideoBackend {
/// Native backend (ExoPlayer on Android, libmpv on Linux)
Native,
/// HTML5 video element fallback
Html5,
}
/// Request to play a single video item
///
/// Simplified to video playback only. Audio playback uses player_play_tracks
@@ -217,9 +203,8 @@ pub struct PlayItemRequest {
pub series_id: Option<String>,
/// Subtitle tracks to sideload, with URLs the frontend has already resolved.
///
/// Only the native backends use these: on Android they become the
/// `MediaItem.SubtitleConfiguration`s ExoPlayer renders. The HTML5 path
/// builds its own `<track>` children instead and ignores this list.
/// On Android they become the `MediaItem.SubtitleConfiguration`s ExoPlayer
/// renders; mpv loads them as external subtitle files (`mpv_tracks`).
///
/// **Order is the contract.** `player_set_subtitle_track(n)` reaches
/// `JellyTauPlayer.setSubtitleTrack(n)`, which indexes into ExoPlayer's
@@ -328,19 +313,6 @@ pub enum VideoSeekResponse {
/// Confirmed position after seek
position: f64,
},
/// Reload stream from new position (transcoded non-HLS)
ReloadStream {
/// What to open, and how — transport included, so the frontend picks
/// its loader from a tagged enum rather than by searching the URL for
/// `.m3u8`. TRACES: UR-079 | DR-225
selection: StreamSelection,
/// `seek_offset` carries the position to RESUME AT, not a base to add to
/// the element's clock. The reloaded stream starts at the item's zero —
/// a position on an HLS playlist makes the server 400 every segment
/// behind it (DR-181) — so the adapter reaches the position by seeking
/// the element and leaves the transcode offset at zero.
seek_offset: f64,
},
}
/// Response for audio track switching operations
@@ -352,13 +324,6 @@ pub enum AudioTrackSwitchResponse {
/// Confirmation message
success: bool,
},
/// HTML5 needs to reload stream with new audio track
ReloadStream {
/// What to open, and how. TRACES: UR-079 | DR-225
selection: StreamSelection,
/// Current position to resume from
position: f64,
},
}
/// Response for a mid-playback streaming-quality change.
@@ -388,16 +353,6 @@ pub enum StreamQualityResponse {
/// Position playback resumed at.
position: f64,
},
/// HTML5 must reload its element with this selection.
ReloadStream {
/// What to open, and how — already negotiated against the requested
/// ceiling. Carries `available` too, so a picker opened after a quality
/// change still describes the source correctly.
/// TRACES: UR-070, UR-079 | DR-225, DR-227
selection: StreamSelection,
/// Position to resume from.
position: f64,
},
}
/// Helper function to create MediaItem from video request
@@ -726,36 +681,18 @@ pub async fn player_play_item(
}
let controller = player.0.lock().await;
// Who gets the stream depends on who is going to *render* it, which is a
// runtime question, not a platform constant.
//
// Historically Linux video was always the webview's (`use_html5_element`),
// so handing the file to MPV as well would only have started a redundant
// decode with no window to show it in — hence a `#[cfg(not(linux))]` guard
// and a queue-only path here. With mpv drawing the picture that inverts:
// the webview is no longer loading anything, so if this does not load the
// file, *nothing does*. The symptom is total silence — no picture and no
// audio — which reads like a broken stream rather than a stream nobody was
// given.
//
// This is the fifth place in this cycle where a renderer's capability was
// written as a compile-time platform fact. Same fix as the others: ask.
// (It said `cfg!(not(linux))`, which also loaded Windows video into the
// backend while the status sent it to the `<video>` element.)
// The backend always gets the stream: every video renderer is native (mpv,
// ExoPlayer) since the webview path was deleted (DR-235). This used to ask
// who would render — the webview's `<video>` played it itself, so the
// backend was only told about it (`set_current_item`) — and got the answer
// wrong twice: the Linux guard silenced mpv video entirely once mpv drew the
// picture, and on Windows it loaded video into the backend while the status
// sent it to the element too.
//
// TRACES: UR-080 | DR-231, DR-235, DR-237
let renders_natively = video_renders_natively();
if renders_natively {
controller
.play_item(media_item)
.map_err(|e| e.to_string())?;
} else {
// The webview will play it; keep the queue in sync for the UI and for a
// remote transfer without starting a second decode.
controller
.set_current_item(media_item)
.map_err(|e| e.to_string())?;
}
controller
.play_item(media_item)
.map_err(|e| e.to_string())?;
// Emit queue changed event
controller.emit_queue_changed();
@@ -778,8 +715,8 @@ pub async fn player_play_item(
/// `stream_url` MUST be an audio-only URL (see
/// `get_audio_only_stream_url_for_video`). The item is created as
/// `MediaType::Audio` so it starts an audio session and loads into the native
/// backend with `mediaType="audio"` — the WebView `<video>` is torn down on the
/// frontend side, so exactly one audio source is ever active.
/// backend with `mediaType="audio"`, replacing the video, so exactly one audio
/// source is ever active.
///
/// This deliberately goes through the queue-based `play_item` path (NOT a
/// side-channel) so end-of-track lands in `on_playback_ended`, which already
@@ -895,7 +832,7 @@ pub async fn player_enter_background_audio(
}
/// Exit background-audio mode: stop the native audio player and return its final
/// position so the frontend can reload the WebView `<video>` there (UR-040).
/// position so the frontend can reload the video there (UR-040).
///
/// Returns the position in seconds. The sleep timer is intentionally left
/// untouched — if it fired while backgrounded, playback is already stopped and
@@ -1190,8 +1127,9 @@ pub async fn player_stop(
let mode = playback_mode.0.get_mode();
// Stopping is a state transition worth seeing in a log. Native video is
// what made its absence matter: the webview <video> stopped implicitly when
// the component unmounted, so nothing ever had to call this — and "never
// what made its absence matter: the (since deleted) webview <video> stopped
// implicitly when the component unmounted, so nothing ever had to call
// this — and "never
// called" and "called but the backend kept playing" look identical from
// outside without it.
info!("[player_stop] called (mode: {:?})", mode);
@@ -1432,8 +1370,8 @@ pub async fn player_seek(
/// - Direct play streams: Use native seeking
/// - Transcoded non-HLS: Request new stream URL from server starting at seek position
///
/// For native (non-HTML5) backends, this command handles the entire stream reload
/// internally. For HTML5 backends, it returns the new URL for the frontend to handle.
/// The backend always handles the seek itself, including re-opening a stream,
/// since every video renderer is native (DR-235).
#[tauri::command]
#[specta::specta]
pub async fn player_seek_video(
@@ -1443,12 +1381,8 @@ pub async fn player_seek_video(
position: f64,
media_source_id: Option<String>,
audio_stream_index: Option<i32>,
use_html5: bool,
) -> Result<VideoSeekResponse, String> {
info!(
"[player_seek_video] Seeking to {} seconds (use_html5: {})",
position, use_html5
);
info!("[player_seek_video] Seeking to {} seconds", position);
// Get repository
let repository = repository_manager
@@ -1490,17 +1424,13 @@ pub async fn player_seek_video(
let controller = player.0.lock().await;
controller.capabilities().seeks_transcoded_in_place
};
let strategy = determine_video_seek_strategy(
is_local,
seeks_transcoded_in_place,
needs_transcoding,
use_html5,
);
let strategy =
determine_video_seek_strategy(is_local, seeks_transcoded_in_place, needs_transcoding);
info!(
"[player_seek_video] Stream analysis: is_local={}, seeks_transcoded_in_place={}, \
needs_transcoding={}, use_html5={}, strategy={:?}",
is_local, seeks_transcoded_in_place, needs_transcoding, use_html5, strategy
needs_transcoding={}, strategy={:?}",
is_local, seeks_transcoded_in_place, needs_transcoding, strategy
);
match strategy {
@@ -1511,35 +1441,6 @@ pub async fn player_seek_video(
controller.seek(position).map_err(|e| e.to_string())?;
Ok(VideoSeekResponse::Native { position })
}
VideoSeekStrategy::Html5NativeSeek => {
// HTML5 backend with HLS or direct play - frontend handles seeking
// We don't call backend.seek() because video is in HTML5 element, not in MPV
info!("[player_seek_video] HTML5 native seek - returning position for frontend");
Ok(VideoSeekResponse::Native { position })
}
VideoSeekStrategy::Html5ReloadStream => {
// Transcoded non-HLS with HTML5 - frontend handles stream reload
info!("[player_seek_video] HTML5 reload stream - requesting new stream URL");
let selection = repository
.get_stream_selection(
&jellyfin_item_id,
media_source_id.as_deref(),
audio_stream_index,
)
.await
.map_err(|e| format!("Failed to select a stream: {:?}", e))?;
info!(
"[player_seek_video] Selected {:?} over {:?} for position {}",
selection.playback_kind, selection.transport, position
);
Ok(VideoSeekResponse::ReloadStream {
selection,
seek_offset: position,
})
}
VideoSeekStrategy::BackendReloadStream => {
// Transcoded non-HLS with native backend - backend handles stream reload
info!("[player_seek_video] Backend reload stream - requesting new stream URL");
@@ -1610,9 +1511,6 @@ pub async fn player_seek_video(
/// carries the requested track at all** — see
/// [`determine_audio_track_switch_strategy`]:
///
/// - An HTML5 `<video>` element has no track-selection API, so the stream is
/// always re-opened at the chosen `AudioStreamIndex` and the frontend seeks
/// the reloaded element back to `position`.
/// - A native backend playing a **direct play** holds the source file with
/// every track in it, so ExoPlayer selects in place by track-group index.
/// - A native backend playing a **transcode** does not. Jellyfin builds a
@@ -1628,10 +1526,8 @@ pub async fn player_seek_video(
/// audio track index` and dropped the request — the default track just kept
/// playing, with nothing in the UI saying so.
///
/// libmpv implements neither selection nor reload here — it is the audio-only
/// backend and leaves `PlayerBackend::set_audio_track` at its
/// `not_implemented()` default, which is why IR-019 is met by these paths
/// rather than by MPV.
/// mpv selects in place the same way (`mpv_tracks::select_audio`, by position in
/// the file's audio tracks), and re-opens a transcode through the same path.
///
/// TRACES: UR-021, UR-005 | IR-019, DR-024, DR-258
#[tauri::command]
@@ -1646,12 +1542,13 @@ pub async fn player_switch_audio_track(
repository_handle: String,
stream_index: i32,
array_index: i32,
use_html5: bool,
current_position: Option<f64>,
media_source_id: Option<String>,
) -> Result<AudioTrackSwitchResponse, String> {
info!("[player_switch_audio_track] Switching to audio track - stream_index: {}, array_index: {}, use_html5: {}",
stream_index, array_index, use_html5);
info!(
"[player_switch_audio_track] Switching to audio track - stream_index: {}, array_index: {}",
stream_index, array_index
);
// Read what the engine is playing before deciding anything — including
// where it is, which has to be captured before the stop below wipes it.
@@ -1675,11 +1572,11 @@ pub async fn player_switch_audio_track(
)
};
let strategy = determine_audio_track_switch_strategy(needs_transcoding, use_html5);
let strategy = determine_audio_track_switch_strategy(needs_transcoding);
info!(
"[player_switch_audio_track] needs_transcoding={}, use_html5={}, strategy={:?}",
needs_transcoding, use_html5, strategy
"[player_switch_audio_track] needs_transcoding={}, strategy={:?}",
needs_transcoding, strategy
);
if strategy == AudioTrackSwitchStrategy::BackendSelectInPlace {
@@ -1700,7 +1597,7 @@ pub async fn player_switch_audio_track(
// Select a stream carrying the chosen audio track. It starts at zero —
// an HLS playlist cannot carry a position (DR-181) — so the position is
// restored by seeking afterwards, here or in the frontend.
// restored by seeking afterwards.
//
// Pinning a track is itself a reason the source cannot be direct-played:
// the file has one default track and the viewer asked for another, so
@@ -1721,10 +1618,6 @@ pub async fn player_switch_audio_track(
let position = crate::player::track_switch::resume_position(current_position, engine_position);
match strategy {
AudioTrackSwitchStrategy::Html5ReloadStream => Ok(AudioTrackSwitchResponse::ReloadStream {
selection,
position,
}),
AudioTrackSwitchStrategy::BackendReloadStream => {
// The native backend re-opens its own stream, the same sequence the
// transcoded seek and quality change use: stop, repoint the queue
@@ -1783,9 +1676,7 @@ pub async fn player_switch_audio_track(
/// A cap is a property of the stream the server is producing, so unlike a volume
/// change it cannot be applied to a stream already in flight — the stream has to
/// be re-opened at the new quality and resumed at the current position. That is
/// the same reload the transcoded-seek and audio-track paths use, and the same
/// two-sided split: HTML5 gets the URL back and reloads its own element, while a
/// native backend is reloaded here.
/// the same reload the transcoded-seek and audio-track paths use, done here.
///
/// The change applies to **this playback only**. The in-player picker is a
/// "this film, this connection" control and its doc has always said so, but it
@@ -1809,15 +1700,13 @@ pub async fn player_set_stream_quality(
repository_manager: State<'_, super::repository::RepositoryManagerWrapper>,
repository_handle: String,
quality: crate::settings::StreamingQuality,
use_html5: bool,
current_position: Option<f64>,
media_source_id: Option<String>,
audio_stream_index: Option<i32>,
) -> Result<StreamQualityResponse, String> {
info!(
"[player_set_stream_quality] Switching to {} (use_html5: {}, position: {:?})",
"[player_set_stream_quality] Switching to {} (position: {:?})",
quality.label(),
use_html5,
current_position
);
@@ -1882,14 +1771,7 @@ pub async fn player_set_stream_quality(
.map_err(|e| format!("Failed to select a stream: {:?}", e))?;
let new_url = selection.url.clone();
if use_html5 {
return Ok(StreamQualityResponse::ReloadStream {
selection,
position,
});
}
// Native backend (Android/ExoPlayer): stop, repoint the queue entry at the
// The native backend (mpv, ExoPlayer): stop, repoint the queue entry at the
// new URL, and reload — mirroring `VideoSeekStrategy::BackendReloadStream`.
// The re-opened stream begins at zero (an HLS playlist cannot carry a start
// position without 400ing every segment — DR-181), so it is seeked back to
@@ -1942,8 +1824,8 @@ pub async fn player_set_audio_track(
///
/// On Android this indexes ExoPlayer's *text track groups* — i.e. the position
/// of the sideloaded `MediaItem.SubtitleConfiguration`, not the Jellyfin stream
/// index. The HTML5 path never reaches here; it toggles its own `<track>`
/// children. libmpv implements neither, leaving the trait default in place.
/// index. mpv gives it the same meaning: the position in the sideloaded WebVTT
/// list, loaded as external subtitle files (`mpv_tracks`).
///
/// TRACES: UR-020 | IR-018, DR-023
#[tauri::command]
@@ -2186,17 +2068,13 @@ pub async fn player_get_queue(
#[serde(rename_all = "camelCase")]
pub struct PlaybackCapabilities {
/// True when audio is rendered by a webview `<audio>` element rather than a
/// native backend. Native audio exists on Linux (mpv) and Android
/// (ExoPlayer); everything else (Windows, future desktops) uses the webview.
/// native backend. Native audio exists on Linux and Windows (mpv) and
/// Android (ExoPlayer); only an unported desktop uses the webview.
///
/// Video has no counterpart: it is always drawn by the native backend, behind
/// the transparent webview (DR-235) — there is no webview video renderer
/// left to report.
pub uses_webview_audio: bool,
/// True when video is rendered by a native surface composited *behind* a
/// transparent webview: ExoPlayer's SurfaceView on Android, mpv's GL area on
/// Linux.
pub supports_native_video: bool,
/// True when the user may send video to the webview element instead of the
/// native renderer — the frontend offers the switch only then, and honours
/// the stored preference only then. False on every platform since DR-235.
pub webview_video_fallback: bool,
}
/// Report this platform's playback capabilities to the frontend.
@@ -2215,51 +2093,14 @@ pub async fn player_get_capabilities() -> Result<PlaybackCapabilities, String> {
Ok(PlaybackCapabilities {
uses_webview_audio: !native_audio,
// TRACES: UR-080 | DR-235
supports_native_video: video_renders_natively(),
// No platform offers one: Android since DR-293, Linux since DR-235,
// and on Windows the webview is the only video renderer, so there is
// nothing to fall back *from*. Kept on the wire until phase 3 deletes
// the frontend switch with the rest of the webview video path.
// TRACES: UR-080, UR-003 | DR-235, DR-293
webview_video_fallback: false,
})
}
/// Whether a native renderer draws video on this platform, so the backend
/// must be handed the stream and the webview must not load it.
///
/// ExoPlayer on Android, mpv on Linux; the webview `<video>` element on
/// Windows until DR-237 gives it mpv video. Asked by `player_play_item`,
/// `get_player_status` and `player_get_capabilities` — the answer drifted when
/// each spelled it out for itself.
///
/// TRACES: UR-003, UR-080 | DR-235, DR-237
pub(crate) fn video_renders_natively() -> bool {
cfg!(target_os = "android") || crate::player::native_video::enabled()
}
pub(super) fn get_player_status(controller: &PlayerController) -> PlayerStatus {
// Determine backend at compile time based on platform
let (backend, use_html5_element) = if cfg!(target_os = "android") {
// Android uses ExoPlayer native backend
(VideoBackend::Native, false)
} else if video_renders_natively() {
// mpv draws the picture on this desktop; the frontend must not also
// load it into a <video> element or the stream decodes twice and the
// two fight over the audio. TRACES: UR-080 | DR-235
(VideoBackend::Native, false)
} else {
// Windows: the webview <video> element is its only video renderer
// until mpv reaches it (DR-237).
(VideoBackend::Html5, true)
};
PlayerStatus {
state: controller.state(),
// The position on the item's timeline, whichever of the three paths is
// rendering it — the native backend answers for only one of them, and
// reads 0 for webview video and for a handoff that has not ticked yet.
// The position on the item's timeline, whichever path is rendering it —
// the native backend reads 0 for a handoff that has not ticked yet.
// TRACES: UR-005 | DR-178
position: controller.absolute_position(),
duration: controller.duration(),
@@ -2267,8 +2108,6 @@ pub(super) fn get_player_status(controller: &PlayerController) -> PlayerStatus {
muted: controller.muted(),
shuffle: controller.is_shuffle(),
repeat: controller.repeat_mode(),
backend,
use_html5_element,
// Merged fields initialized to defaults (will be set by player_get_status)
merged_media: None,
@@ -3092,61 +2931,41 @@ pub async fn player_disable_jellyfin(player: State<'_, PlayerStateWrapper>) -> R
mod tests {
use crate::utils::lock::MutexSafe;
/// The webview is not a video renderer anywhere the app ships a native one:
/// Android since DR-293, Linux since DR-235 made mpv its only video path.
/// So the frontend is never offered the switch, and a stored "native video
/// off" from before cannot send Linux video back to the `<video>` element.
///
/// TRACES: UR-080, UR-003 | DR-235, DR-293 | UT-272
#[tokio::test]
async fn test_no_platform_offers_a_webview_video_fallback() {
let caps = super::player_get_capabilities().await.unwrap();
assert!(!caps.webview_video_fallback);
if cfg!(target_os = "linux") {
assert!(caps.supports_native_video, "mpv draws video on Linux");
assert!(!caps.uses_webview_audio);
}
}
/// The three places that answer "who draws video here" give one answer:
/// `play_item` loads the backend exactly where the status tells the
/// frontend *not* to use a `<video>` element. They disagreed on Windows —
/// `play_item` loaded video into the backend while the status sent it to
/// the element — which was invisible while that backend was the webview's
/// own `<audio>`, and would play every film's soundtrack twice once mpv
/// plays Windows audio.
/// Video always goes to the backend. `player_play_item` once decided per
/// platform whether the backend or the webview's `<video>` would render,
/// and each wrong answer was silence (Linux, once mpv drew the picture) or a
/// soundtrack decoded twice (Windows). With the webview video path deleted
/// there is no second renderer to route to, and the queue-only branch is
/// gone with it.
///
/// TRACES: UR-003, UR-080 | DR-235, DR-237 | UT-273
#[tokio::test]
async fn test_video_routing_has_one_answer() {
#[test]
fn test_video_always_goes_to_the_backend() {
let src = include_str!("mod.rs");
let routing = src
.split("let renders_natively =")
let play_item = src
.split("pub async fn player_play_item(")
.nth(1)
.and_then(|rest| rest.split(';').next())
.expect("player_play_item decides renders_natively");
assert_eq!(
routing.trim(),
"video_renders_natively()",
"player_play_item must ask the same question as get_player_status"
.and_then(|rest| rest.split("\n}\n").next())
.expect("player_play_item exists");
assert!(play_item.contains(".play_item(media_item)"));
assert!(
!play_item.contains(".set_current_item("),
"player_play_item must not keep video from the backend"
);
let status = super::get_player_status(&crate::player::PlayerController::default());
assert_eq!(status.use_html5_element, !super::video_renders_natively());
let caps = super::player_get_capabilities().await.unwrap();
assert_eq!(caps.supports_native_video, super::video_renders_natively());
}
/// And the status the video page reads agrees: on Linux the frontend is told
/// the native backend renders, never to load a `<video>` element.
/// Only audio can still be the webview's, and only on a desktop with no mpv.
///
/// TRACES: UR-080 | DR-235 | UT-272
#[test]
fn test_linux_video_is_not_sent_to_the_webview() {
let controller = crate::player::PlayerController::default();
let status = super::get_player_status(&controller);
if cfg!(target_os = "linux") {
assert!(!status.use_html5_element);
/// TRACES: UR-003, UR-080 | DR-235, DR-237 | UT-272
#[tokio::test]
async fn test_every_shipped_platform_plays_audio_natively() {
let caps = super::player_get_capabilities().await.unwrap();
if cfg!(any(
target_os = "linux",
target_os = "windows",
target_os = "android"
)) {
assert!(!caps.uses_webview_audio);
}
}
src-tauri/src/commands/player/timers.rs
+10 -9
View File
@@ -138,7 +138,7 @@ pub async fn player_play_next_episode(
/// Handle playback ended event - triggers autoplay decision logic
/// This is called from:
/// - Frontend when HTML5 video ends (Linux/desktop) - passes itemId + repositoryHandle for the video
/// - Frontend when a video ends - passes itemId + repositoryHandle for the video
/// - Frontend when audio track ends via backend event - no itemId/repositoryHandle needed
/// - Android JNI callback also triggers this logic directly
///
@@ -158,7 +158,7 @@ pub async fn player_on_playback_ended(
let controller_arc = player.0.clone();
// Run autoplay decision logic
// If item_id is provided (HTML5 video case), use the video-specific path
// If item_id is provided (a video), use the video-specific path
// that bypasses the backend queue and stale end_reason
let decision = {
let controller = controller_arc.lock().await;
@@ -326,16 +326,17 @@ pub async fn player_recover_stream(player: State<'_, PlayerStateWrapper>) -> Res
}
}
// ===== HTML5 video state-report commands =====
// ===== Webview media state-report commands =====
//
// On platforms where video renders in the webview (Linux WebKitGTK HTML5
// <video>), the real player lives outside the native backend, so the frontend
// HTML5 adapter reports DOM events back through these commands. The controller
// Where media renders in the webview — the `<audio>` element of the webview
// audio backend, on a desktop with no mpv; video never does since DR-235 — the
// real player lives outside the native backend, so the frontend adapter reports
// DOM events back through these commands. The controller
// re-emits them through the same PlayerStatusEvent pipeline the native backends
// use, keeping the Rust controller the single source of truth and the frontend
// player store fed from one place (playerEvents.ts) in both modes.
/// Report an HTML5 <video> state change (playing/paused/loading/stopped/idle).
/// Report a webview media element's state change (playing/paused/loading/stopped/idle).
#[tauri::command]
#[specta::specta]
pub async fn player_report_state(
@@ -348,7 +349,7 @@ pub async fn player_report_state(
Ok(())
}
/// Report an HTML5 <video> position tick (seconds). The adapter should throttle
/// Report a webview media element's position tick (seconds). The adapter should throttle
/// these to roughly match the native backends' ~250ms cadence.
#[tauri::command]
#[specta::specta]
@@ -362,7 +363,7 @@ pub async fn player_report_position(
Ok(())
}
/// Report that the HTML5 <video> finished loading and knows its duration.
/// Report that a webview media element finished loading and knows its duration.
#[tauri::command]
#[specta::specta]
pub async fn player_report_media_loaded(
src-tauri/src/lib.rs
-87
View File
@@ -1102,86 +1102,6 @@ fn specta_builder() -> Builder<tauri::Wry> {
])
}
/// Configure GStreamer (the media backend behind WebKitGTK's HTML5 `<video>`
/// element on Linux) to prefer hardware-accelerated VAAPI decoding when the
/// host provides it, falling back to software decoding otherwise.
///
/// All variables are only set if the user has not already exported them, so an
/// explicit override (e.g. forcing software decode for debugging) is respected.
/// They must be applied before WebKitGTK builds its GStreamer pipeline, hence the
/// call at the very top of `run()`.
#[cfg(target_os = "linux")]
fn enable_linux_hardware_video_decoding() {
// Boost the rank of the modern stateless VAAPI decoders (gst-plugins-bad
// `va` plugin) so GStreamer selects them ahead of the software decoders. The
// `MAX` rank wins decoder autoplugging when the hardware/driver supports the
// codec; unsupported codecs simply fall through to software.
let rank_overrides = "vah264dec:MAX,vah265dec:MAX,vavp9dec:MAX,vaav1dec:MAX,\
vampeg2dec:MAX,vavp8dec:MAX";
set_env_if_unset("GST_PLUGIN_FEATURE_RANK", rank_overrides);
// Ensure WebKit keeps GStreamer's hardware/DMABUF video path enabled. Setting
// this to "0" would force software decoding, so only default it to "1".
set_env_if_unset("WEBKIT_GST_ENABLE_HW_VIDEO_DECODER", "1");
info!("[INIT] Linux hardware video decoding (VAAPI) enabled where supported");
log_available_vaapi_decoders();
}
/// Probe (via `gst-inspect-1.0`, which ships with GStreamer) which VAAPI hardware
/// video decoders GStreamer can actually load on this host, and log the result so
/// it is clear at startup whether hardware decoding is genuinely available or
/// whether playback will fall back to software.
#[cfg(target_os = "linux")]
fn log_available_vaapi_decoders() {
const HW_DECODERS: &[&str] = &[
"vah264dec",
"vah265dec",
"vavp9dec",
"vaav1dec",
"vampeg2dec",
"vavp8dec",
];
let available: Vec<&str> = HW_DECODERS
.iter()
.copied()
.filter(|name| {
std::process::Command::new("gst-inspect-1.0")
.arg(name)
.stdout(std::process::Stdio::null())
.stderr(std::process::Stdio::null())
.status()
.map(|s| s.success())
.unwrap_or(false)
})
.collect();
if available.is_empty() {
log::warn!(
"[INIT] No VAAPI hardware video decoders found via gst-inspect-1.0; \
video will use software decoding. Install the GStreamer 'va' plugin \
(gst-plugins-bad) and a VAAPI driver to enable hardware decoding."
);
} else {
info!(
"[INIT] VAAPI hardware video decoders available to GStreamer: {}",
available.join(", ")
);
}
}
#[cfg(target_os = "linux")]
fn set_env_if_unset(key: &str, value: &str) {
if std::env::var_os(key).is_none() {
// SAFETY: called once at startup before any threads that read the
// environment (WebKitGTK/GStreamer) are spawned.
std::env::set_var(key, value);
}
}
/// Cached thumbnails are handed to the webview as asset-protocol URLs by
/// `convertFileSrc` (`asset://localhost/…` on Linux/macOS,
/// `http://asset.localhost/…` on Windows/Android). Tauri only answers that
@@ -1261,13 +1181,6 @@ pub fn run() {
// TRACES: UR-078 | DR-218
crate::utils::diagnostics::install_panic_hook();
// On Linux, video plays through WebKitGTK's HTML5 <video> element, which uses
// GStreamer as its media backend. Enable hardware-accelerated (VAAPI) decoding
// when available so video transcoding/decoding does not fall back to the CPU.
// These must be set before WebKitGTK initializes its GStreamer pipeline.
#[cfg(target_os = "linux")]
enable_linux_hardware_video_decoding();
// NOTE: TypeScript bindings are generated by the `export_typescript_bindings`
// test (`cargo test export_typescript_bindings`), NOT at runtime. Calling
// `.export()` here would try to write `../src/lib/api/bindings.ts` at app
src-tauri/src/player/backend.rs
+6 -8
View File
@@ -98,10 +98,9 @@ pub trait PlayerBackend: Send + Sync {
/// Set the active audio track by stream index
///
/// Overridden by the Android (ExoPlayer) backend. `MpvBackend` deliberately
/// does **not** override it — MPV is the audio-only backend here, so it keeps
/// this `not_implemented()` default and the Linux video path switches track by
/// re-opening the stream instead (`player_switch_audio_track`).
/// Overridden by both video backends, ExoPlayer and `MpvBackend`; the
/// argument is a position among the file's audio tracks. A transcode is
/// re-opened instead (`player_switch_audio_track`).
///
/// TRACES: UR-021 | IR-019, DR-024
fn set_audio_track(&mut self, _stream_index: i32) -> Result<(), PlayerError> {
@@ -111,10 +110,9 @@ pub trait PlayerBackend: Send + Sync {
/// Set the active subtitle track by stream index (None to disable subtitles)
///
/// Overridden by the Android (ExoPlayer) backend. `MpvBackend` deliberately
/// does **not** override it, so it keeps this `not_implemented()` default;
/// the Linux video path renders subtitles as `<track>` children of the
/// WebKitGTK HTML5 `<video>` element and never calls this.
/// Overridden by both video backends, ExoPlayer and `MpvBackend`; the
/// argument is a position in the sideloaded subtitle list the play request
/// carried.
///
/// TRACES: UR-020 | IR-018, DR-023
fn set_subtitle_track(&mut self, _stream_index: Option<i32>) -> Result<(), PlayerError> {
src-tauri/src/player/events.rs
+9 -10
View File
@@ -85,9 +85,8 @@ pub enum PlayerStatusEvent {
remaining_seconds: u32,
},
/// Time-based sleep timer expired: playback must stop. The backend stops
/// its own (MPV/ExoPlayer) playback, but HTML5 video on Linux plays in the
/// webview outside the backend's control — the frontend pauses it on this
/// event.
/// its own (MPV/ExoPlayer) playback; the frontend pauses the active adapter
/// on this event, which reaches a webview `<audio>` element where one plays.
SleepTimerExpired,
/// Show next episode popup with countdown
ShowNextEpisodePopup {
@@ -152,9 +151,9 @@ pub enum PlayerStatusEvent {
/// media item locally), so the native side only signals intent here.
RemoteDisconnectRequested,
/// Backend-originated control command targeting the active frontend player
/// adapter (the HTML5 <video> that lives in the webview, which Rust cannot
/// drive directly). Emitted by control paths like the sleep timer, lockscreen,
/// or remote so they can pause/play/seek/stop the webview element.
/// adapter — the webview `<audio>` element, which Rust cannot drive
/// directly. Emitted by control paths like the sleep timer, lockscreen, or
/// remote so they can pause/play/seek/stop it.
/// `playerEvents.ts` routes this to the active PlayerAdapter via the facade.
ControlCommand {
/// One of: "play", "pause", "stop", "seek".
@@ -164,10 +163,10 @@ pub enum PlayerStatusEvent {
},
/// Ask the frontend webview `<audio>` element to load and play a stream.
///
/// Emitted by `WebviewAudioBackend` on platforms with no native audio
/// backend (e.g. Windows): audio-only playback is rendered by an `<audio>`
/// element in the webview, mirroring how all video already renders through
/// the webview `<video>`. The element then reports its state/position back
/// Emitted by `WebviewAudioBackend` on a desktop with no native audio
/// backend (none that ships: Linux and Windows have mpv): audio-only
/// playback is rendered by an `<audio>` element in the webview. The element
/// then reports its state/position back
/// through the `player_report_*` commands, so the Rust controller stays the
/// single source of truth. Subsequent play/pause/seek/stop reach the element
/// via `ControlCommand`.
src-tauri/src/player/fixtures/sub.vtt
+4
View File
@@ -0,0 +1,4 @@
WEBVTT
00:00.000 --> 00:02.000
fixture subtitle
src-tauri/src/player/fixtures/two-audio-tracks.mkv
BIN
View File
Binary file not shown.
src-tauri/src/player/media_player.rs
+5 -18
View File
@@ -140,7 +140,7 @@ pub struct Capabilities {
pub audio_track_switching: bool,
/// A *server-side transcode* can be seeked without re-opening the stream.
///
/// True for hls.js, which seeks within the VOD playlist it is handed and
/// True for ExoPlayer, which seeks within the VOD playlist it is handed and
/// lets the server catch up. False for mpv, whose HLS demuxer cannot make
/// the server transcode from a new offset.
///
@@ -173,9 +173,8 @@ impl Capabilities {
/// ExoPlayer.
///
/// **Can** seek a transcode in place. It is a full HLS client, so like
/// hls.js it seeks within the VOD playlist it was handed and lets the
/// server catch up. Grouping it with mpv as "a native engine" gets this
/// **Can** seek a transcode in place. It is a full HLS client, so it seeks
/// within the VOD playlist it was handed and lets the server catch up. Grouping it with mpv as "a native engine" gets this
/// exactly backwards — being native is not the property that matters here,
/// speaking HLS is, and that is the whole reason this is declared per
/// engine rather than inferred from a category.
@@ -188,18 +187,6 @@ impl Capabilities {
seeks_transcoded_in_place: true,
}
}
/// An engine that renders through the webview element, where hls.js seeks
/// within the playlist it was handed.
pub fn webview() -> Self {
Self {
video: true,
audio_settings: false,
subtitle_switching: true,
audio_track_switching: false,
seeks_transcoded_in_place: true,
}
}
}
/// A request to present an item.
@@ -242,7 +229,7 @@ impl OpenRequest {
/// Anything that can present media.
///
/// Implementations: `MpvPlayer` (Linux/Windows), `ExoPlayerPlayer` (Android),
/// `WebviewPlayer` (HTML5 element), and `FakePlayer` for tests. Every one of
/// and `FakePlayer` for tests. Every one of
/// them must pass [`super::conformance`].
pub trait MediaPlayer: Send {
/// Present `req.selection`, beginning at `req.start`.
@@ -267,7 +254,7 @@ pub trait MediaPlayer: Send {
/// Seek to an absolute position on the item's own timeline.
///
/// Whether that is an in-place seek or a re-open of the stream is the
/// engine's business: hls.js seeks within a VOD playlist, mpv's HLS demuxer
/// engine's business: ExoPlayer seeks within a VOD playlist, mpv's HLS demuxer
/// cannot make a server transcode from a new offset. Callers state the
/// destination and nothing else.
fn seek(&mut self, to: Duration) -> Result<(), PlayerError>;
src-tauri/src/player/mod.rs
+12 -39
View File
@@ -19,6 +19,8 @@ pub mod media_player;
pub mod mpv_command;
#[cfg(target_os = "linux")]
pub mod mpv_player;
#[cfg(any(target_os = "linux", target_os = "windows"))]
pub mod mpv_tracks;
pub mod queue;
pub mod seek;
pub mod session;
@@ -322,9 +324,10 @@ pub struct PlayerController {
background_audio_base: Arc<Mutex<f64>>,
// True while a background-audio handoff owns playback: the native audio
// player is the real player and the webview <video> has been torn down.
// player is the real player and the video has been replaced.
//
// The teardown is what makes this necessary. It fires a DOM `pause` that the
// The teardown is what made this necessary (when a webview <video> was
// torn down). It fires a DOM `pause` that the
// frontend reports like any other, which would otherwise leave the controller
// believing webview media is still active — aiming lockscreen transport at an
// element that no longer exists (see `is_html5_active`).
@@ -341,7 +344,8 @@ pub struct PlayerController {
// TRACES: UR-040 | DR-129
stream_resume: Arc<Mutex<stream_end::ResumeTracker>>,
// Last state reported by a webview-rendered HTML5 <video>/<audio> element.
// Last state reported by a webview-rendered `<audio>` element (the webview
// audio backend; video never renders in the webview since DR-235).
//
// Webview-rendered media is played by an element the native backend cannot
// reach, so the backend's own state() says nothing about it. Tracking the
@@ -421,7 +425,7 @@ impl PlayerController {
/// Configure the media repository used for next-episode lookups.
///
/// The Android ExoPlayer ended-callback calls `on_playback_ended` with no
/// repository handle (unlike the Linux HTML5 path, which passes one per
/// repository handle (unlike a frontend-reported end, which passes one per
/// call), so the controller needs a repository of its own or episode
/// autoplay silently decides Stop.
pub fn set_repository(&self, repo: Arc<dyn MediaRepository>) {
@@ -541,37 +545,6 @@ impl PlayerController {
Ok(())
}
/// Set the current queue item without loading it into the playback backend.
///
/// Used on platforms where video is rendered outside the native backend
/// (Linux WebKitGTK HTML5 <video>): the queue/UI state must reflect the
/// item, but MPV must not start a redundant decode for it.
///
/// Not gated to Linux. Its caller stopped being a `#[cfg]` branch and became
/// a runtime question — "does this renderer draw the picture?" — so the
/// `else` arm is compiled on every platform even where it never runs. The
/// gate outliving its caller broke the Android build outright, which went
/// unnoticed because nothing built for Android afterwards.
pub fn set_current_item(&self, item: MediaItem) -> Result<(), PlayerError> {
debug!(
"[PlayerController] set_current_item (no backend load): {}",
item.title
);
self.reset_autoplay_count();
// A different item is current; the last one's reported position must not
// be reported against it. This path is how webview-rendered video is
// queued (no backend load at all), so it is exactly where a stale
// reading would otherwise survive.
// TRACES: UR-005 | DR-178
self.clear_reported_time();
let mut queue = self.queue.lock_safe();
queue.set_queue(vec![item], 0);
Ok(())
}
/// Load and play an item without modifying the queue
/// Use this when the queue is already set up and you just want to play a specific item from it
pub fn load_and_play(&self, item: &MediaItem) -> Result<(), PlayerError> {
@@ -596,7 +569,7 @@ impl PlayerController {
// intermittent.
//
// The webview re-establishes its own authority the moment an element
// reports again, so nothing is lost on the HTML5 path: this is the same
// reports again, so nothing is lost on the webview path: this is the same
// "element is gone" semantics as the "stopped"/"idle" report, applied at
// the point where we can know it directly.
//
@@ -720,7 +693,7 @@ impl PlayerController {
Ok(())
}
/// True while webview-rendered media (HTML5 `<video>`/`<audio>`) is the real
/// True while webview-rendered media (a webview `<audio>`) is the real
/// player, so transport must be routed to it rather than the native backend.
///
/// TRACES: UR-005 | DR-097
@@ -772,7 +745,7 @@ impl PlayerController {
/// Toggle play/pause.
///
/// The decision is made HERE, from authoritative state — the reported webview
/// state for HTML5-rendered media, or the native backend's state otherwise.
/// state for webview-rendered audio, or the native backend's state otherwise.
/// The frontend must never decide this from the DOM (see DR-097).
///
/// TRACES: UR-005 | DR-097
@@ -1049,7 +1022,7 @@ impl PlayerController {
/// truncation comparison. `position()` alone answers for exactly one of the
/// three ways this app plays media, and reads 0 for the other two:
///
/// - **Webview `<video>`/`<audio>`**: nothing is loaded into the native
/// - **Webview `<audio>`**: nothing is loaded into the native
/// backend, so its position is a permanent 0. The element's own reports are
/// the only reading there is.
/// - **Background-audio handoff**: the audio-only stream's zero is the
src-tauri/src/player/mpv_backend.rs
+41 -2
View File
@@ -384,8 +384,8 @@ impl MpvBackend {
// StateChanged rather than tracking playback itself, per the
// one-directional state rule. Unobserved, the event never came and
// the button never moved. Invisible until native video shipped,
// because the webview <video> element's own DOM events drove that
// control on Linux.
// because the (since deleted) webview <video> element's own DOM
// events drove that control on Linux.
//
// TRACES: UR-005 | DR-239
ev_ctx
@@ -696,6 +696,15 @@ impl PlayerBackend for MpvBackend {
// TRACES: UR-040, UR-005 | DR-253
*self.pending_seek.lock_safe() = None;
// The item's own sideloaded subtitles, none shown, and its default audio
// track — whatever the previous item had chosen. Only video carries
// subtitles; for audio this just clears the last item's.
// TRACES: UR-020, UR-021 | DR-023, DR-024, DR-235
let subtitle_urls: Vec<&str> = media.subtitles.iter().map(|t| t.url.as_str()).collect();
super::mpv_tracks::prepare_load(&self.mpv, &subtitle_urls).map_err(|e| PlayerError {
message: format!("Failed to prepare tracks: {e}"),
})?;
// Load the media file. Through `mpv_command::command`, never
// `Mpv::command`: the URL carries server-controlled text.
// TRACES: UR-003, UR-004 | DR-298
@@ -858,6 +867,36 @@ impl PlayerBackend for MpvBackend {
state.volume
}
/// `stream_index` is a *position*: the n-th audio track of the file, the
/// same meaning ExoPlayer gives it (`player_switch_audio_track` passes the
/// array index). Only reached for a direct play/stream — a transcode carries
/// one track and is re-opened instead.
///
/// TRACES: UR-021 | IR-019, DR-024, DR-235
fn set_audio_track(&mut self, stream_index: i32) -> Result<(), PlayerError> {
let position = usize::try_from(stream_index).map_err(|_| PlayerError {
message: format!("Invalid audio track position {stream_index}"),
})?;
super::mpv_tracks::select_audio(&self.mpv, position)
.map_err(|message| PlayerError { message })
}
/// `stream_index` is the position in the sideloaded subtitle list the play
/// request carried (`nativeSubtitleArrayIndex`), `None` to hide subtitles.
///
/// TRACES: UR-020 | IR-018, DR-023, DR-235
fn set_subtitle_track(&mut self, stream_index: Option<i32>) -> Result<(), PlayerError> {
let position = stream_index
.map(|i| {
usize::try_from(i).map_err(|_| PlayerError {
message: format!("Invalid subtitle position {i}"),
})
})
.transpose()?;
super::mpv_tracks::select_subtitle(&self.mpv, position)
.map_err(|message| PlayerError { message })
}
fn set_audio_settings(&mut self, settings: &AudioSettings) -> Result<(), PlayerError> {
info!("[MpvBackend] Applying audio settings");
self.audio_settings = settings.clone();
src-tauri/src/player/mpv_tracks.rs
+257
View File
@@ -0,0 +1,257 @@
//! Subtitle and audio-track selection on mpv, by *position*, the way the
//! frontend asks for it.
//!
//! Until DR-235 mpv never drew video, so it never had to: subtitles were
//! `<track>` children of the webview `<video>` and an audio track change
//! re-opened the stream. With mpv the only desktop video renderer, it has to
//! answer the same two calls ExoPlayer does, with the same meaning:
//!
//! - **Subtitles** are the sideloaded WebVTT list the play request carries
//! (`MediaItem::subtitles`), and `set_subtitle_track(n)` selects the *n-th of
//! those* — the position the frontend computes with `nativeSubtitleArrayIndex`.
//! They reach mpv as external files queued on `sub-files` before the load, and
//! selection starts off, because the menu opens on "Off".
//! - **Audio** `set_audio_track(n)` selects the n-th audio track of the file —
//! the position in the item's audio streams, which is file order. Only a direct
//! play/stream carries every track; a transcode is re-opened instead
//! (`AudioTrackSwitchStrategy`).
//!
//! mpv's own track ids are not positions: they count every track of a type,
//! embedded before external, from 1. So a position is always resolved against
//! the live `track-list`.
//!
//! TRACES: UR-020, UR-021 | DR-023, DR-024, DR-235
use libmpv::Mpv;
use super::mpv_command;
/// One entry of mpv's `track-list`, reduced to what selection needs.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct TrackInfo {
pub id: i64,
/// "video", "audio" or "sub".
pub kind: String,
pub external: bool,
}
/// The mpv id of the `position`-th track of `kind` (optionally only external or
/// only embedded ones), in `track-list` order.
///
/// TRACES: UR-020, UR-021 | DR-023, DR-024 | UT-275
pub fn nth_track_id(
tracks: &[TrackInfo],
kind: &str,
external: Option<bool>,
position: usize,
) -> Option<i64> {
tracks
.iter()
.filter(|t| t.kind == kind && external.is_none_or(|e| t.external == e))
.nth(position)
.map(|t| t.id)
}
/// Read the current `track-list`.
pub fn track_list(mpv: &Mpv) -> Vec<TrackInfo> {
let count: i64 = mpv.get_property("track-list/count").unwrap_or(0);
(0..count)
.filter_map(|i| {
let id: i64 = mpv.get_property(&format!("track-list/{i}/id")).ok()?;
let kind: String = mpv.get_property(&format!("track-list/{i}/type")).ok()?;
let external: bool = mpv
.get_property(&format!("track-list/{i}/external"))
.unwrap_or(false);
Some(TrackInfo { id, kind, external })
})
.collect()
}
/// Prepare the next `loadfile`: these subtitle files will be loaded with it, no
/// subtitle is shown, and the file's default audio track plays.
///
/// `sid`/`aid` set while idle become the options the next file opens with, so
/// a track chosen for the previous item cannot leak into this one.
///
/// TRACES: UR-020, UR-021 | DR-023, DR-024 | UT-275
pub fn prepare_load(mpv: &Mpv, subtitle_urls: &[&str]) -> Result<(), String> {
mpv_command::command(mpv, &["change-list", "sub-files", "clr", ""])?;
for url in subtitle_urls {
// `append` adds one item without splitting on the list separator, which
// a URL's `:` would otherwise trip. Through the argv form (DR-298), so the
// URL is never parsed as command text.
mpv_command::command(mpv, &["change-list", "sub-files", "append", url])?;
}
mpv.set_property("sid", "no")
.map_err(|e| format!("could not set sid=no: {e:?}"))?;
mpv.set_property("aid", "auto")
.map_err(|e| format!("could not set aid=auto: {e:?}"))?;
Ok(())
}
/// Show the `position`-th sideloaded subtitle, or none.
///
/// TRACES: UR-020 | DR-023 | UT-275
pub fn select_subtitle(mpv: &Mpv, position: Option<usize>) -> Result<(), String> {
let Some(position) = position else {
return mpv
.set_property("sid", "no")
.map_err(|e| format!("could not set sid=no: {e:?}"));
};
let id = nth_track_id(&track_list(mpv), "sub", Some(true), position)
.ok_or_else(|| format!("no sideloaded subtitle at position {position}"))?;
mpv.set_property("sid", id)
.map_err(|e| format!("could not set sid={id}: {e:?}"))
}
/// Play the `position`-th audio track of the file.
///
/// TRACES: UR-021 | DR-024 | UT-275
pub fn select_audio(mpv: &Mpv, position: usize) -> Result<(), String> {
let id = nth_track_id(&track_list(mpv), "audio", Some(false), position)
.ok_or_else(|| format!("no audio track at position {position}"))?;
mpv.set_property("aid", id)
.map_err(|e| format!("could not set aid={id}: {e:?}"))
}
#[cfg(test)]
mod tests {
use super::*;
use std::time::{Duration, Instant};
fn fixture(name: &str) -> String {
format!("{}/src/player/fixtures/{name}", env!("CARGO_MANIFEST_DIR"))
}
fn track(id: i64, kind: &str, external: bool) -> TrackInfo {
TrackInfo {
id,
kind: kind.to_string(),
external,
}
}
/// mpv numbers each type from 1, embedded before external, so a position in
/// the sideloaded list is not an id. The file here has two embedded
/// subtitles; the first sideloaded one is mpv's sub 3.
///
/// TRACES: UR-020, UR-021 | DR-023, DR-024 | UT-275
#[test]
fn positions_resolve_to_mpv_ids_per_kind() {
let tracks = [
track(1, "video", false),
track(1, "audio", false),
track(2, "audio", false),
track(1, "sub", false),
track(2, "sub", false),
track(3, "sub", true),
track(4, "sub", true),
];
assert_eq!(nth_track_id(&tracks, "sub", Some(true), 0), Some(3));
assert_eq!(nth_track_id(&tracks, "sub", Some(true), 1), Some(4));
assert_eq!(nth_track_id(&tracks, "sub", Some(true), 2), None);
assert_eq!(nth_track_id(&tracks, "audio", Some(false), 1), Some(2));
assert_eq!(nth_track_id(&tracks, "audio", None, 0), Some(1));
}
fn loaded_mpv(subs: &[&str]) -> Mpv {
let mpv = Mpv::new().expect("libmpv must be available to run the player tests");
mpv.set_property("ao", "null").unwrap();
mpv.set_property("vo", "null").unwrap();
mpv.set_property("pause", true).unwrap();
prepare_load(&mpv, subs).unwrap();
mpv_command::command(&mpv, &["loadfile", &fixture("two-audio-tracks.mkv")]).unwrap();
// Video, two audio tracks, and one track per sideloaded subtitle.
let expected = 3 + subs.len() as i64;
let deadline = Instant::now() + Duration::from_secs(10);
while mpv.get_property::<i64>("track-list/count").unwrap_or(0) < expected {
assert!(
Instant::now() < deadline,
"the fixture never finished loading"
);
std::thread::sleep(Duration::from_millis(20));
}
mpv
}
/// Against a real file: the sideloaded subtitle arrives, starts hidden, and
/// is shown and hidden by position.
///
/// TRACES: UR-020 | DR-023 | UT-275
#[test]
fn a_sideloaded_subtitle_starts_off_and_is_selected_by_position() {
let vtt = fixture("sub.vtt");
let mpv = loaded_mpv(&[&vtt]);
assert_eq!(mpv.get_property::<String>("sid").unwrap(), "no");
select_subtitle(&mpv, Some(0)).unwrap();
let expected = nth_track_id(&track_list(&mpv), "sub", Some(true), 0).unwrap();
assert_eq!(mpv.get_property::<i64>("sid").unwrap(), expected);
select_subtitle(&mpv, None).unwrap();
assert_eq!(mpv.get_property::<String>("sid").unwrap(), "no");
assert!(select_subtitle(&mpv, Some(5)).is_err());
}
/// Against a real file with two audio tracks: position 1 is the second one.
///
/// TRACES: UR-021 | DR-024 | UT-275
#[test]
fn an_audio_track_is_selected_by_position_in_the_file() {
let mpv = loaded_mpv(&[]);
select_audio(&mpv, 1).unwrap();
assert_eq!(mpv.get_property::<i64>("aid").unwrap(), 2);
select_audio(&mpv, 0).unwrap();
assert_eq!(mpv.get_property::<i64>("aid").unwrap(), 1);
assert!(select_audio(&mpv, 2).is_err());
}
/// The next item opens with no subtitle and its own default audio, whatever
/// the last one had chosen, and with only its own subtitle files.
///
/// TRACES: UR-020, UR-021 | DR-023, DR-024 | UT-275
#[test]
fn preparing_a_load_forgets_the_previous_items_choices() {
let vtt = fixture("sub.vtt");
let mpv = loaded_mpv(&[&vtt]);
select_subtitle(&mpv, Some(0)).unwrap();
select_audio(&mpv, 1).unwrap();
// The next item: no subtitles of its own this time.
prepare_load(&mpv, &[]).unwrap();
mpv_command::command(
&mpv,
&["loadfile", &fixture("two-audio-tracks.mkv"), "replace"],
)
.unwrap();
let deadline = Instant::now() + Duration::from_secs(10);
loop {
let tracks = track_list(&mpv);
let settled = tracks.len() == 3 && mpv.get_property::<i64>("aid").is_ok();
if settled {
break;
}
assert!(
Instant::now() < deadline,
"the second load never settled: {tracks:?}"
);
std::thread::sleep(Duration::from_millis(20));
}
assert_eq!(mpv.get_property::<String>("sid").unwrap(), "no");
assert_eq!(
mpv.get_property::<i64>("aid").unwrap(),
1,
"default audio again"
);
assert_eq!(
nth_track_id(&track_list(&mpv), "sub", None, 0),
None,
"the previous item's subtitle file came along"
);
}
}
src-tauri/src/player/native_video.rs
+1 -2
View File
@@ -2,8 +2,7 @@
//!
//! Three things need this and must agree: the mpv backend (which has to be
//! configured for video *at construction*, before anything plays), the video
//! surface (which has nothing to draw otherwise), and `get_player_status`
//! (which tells the frontend whether to use a webview `<video>` element).
//! surface (which has nothing to draw otherwise), and the device profile.
//!
//! It is a function rather than three `env::var` checks for the reason this
//! codebase keeps rediscovering: a capability answered in several places is a
src-tauri/src/player/seek.rs
+40 -129
View File
@@ -5,17 +5,18 @@
//! [`VideoSeekStrategy`] into a concrete backend/frontend action.
/// Seek strategy for video playback, derived from a stream's characteristics.
///
/// Every video renderer is a native backend (mpv, ExoPlayer) since the webview
/// `<video>` path was deleted (DR-235), so the backend performs every seek; what
/// remains to decide is whether it can move the stream in place.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum VideoSeekStrategy {
/// Local file - always use native seek on backend
LocalNativeSeek,
/// HLS or direct stream with HTML5 - frontend handles seek, skip backend
Html5NativeSeek,
/// HLS or direct stream with native backend - backend handles seek
/// Seekable where it sits (direct play/stream, or a transcode the engine
/// can move) - backend seeks
BackendNativeSeek,
/// Transcoded non-HLS with HTML5 - reload stream, frontend handles
Html5ReloadStream,
/// Transcoded non-HLS with native backend - reload stream, backend handles
/// A transcode the engine cannot move - re-open the stream at the target
BackendReloadStream,
}
@@ -29,12 +30,10 @@ pub enum VideoSeekStrategy {
/// can seek a server-side transcode without re-opening it. Declared by the
/// engine via `Capabilities`, never inferred from the URL or the renderer.
/// * `needs_transcoding` - Whether the content needs transcoding
/// * `use_html5` - Whether frontend is using HTML5 video element
pub fn determine_video_seek_strategy(
is_local: bool,
seeks_transcoded_in_place: bool,
needs_transcoding: bool,
use_html5: bool,
) -> VideoSeekStrategy {
// Local files always support native seeking via backend
if is_local {
@@ -42,39 +41,20 @@ pub fn determine_video_seek_strategy(
}
// A server-side transcode is produced *from* `StartTimeTicks`, so where the
// seek lands is a property of the request, not of the stream in hand.
//
// hls.js is the exception: handed a VOD playlist it seeks within it and lets
// the server catch up segment by segment. mpv's HLS demuxer cannot make
// Jellyfin transcode from a new offset, so for the native backend a
// seek lands is a property of the request, not of the stream in hand. mpv's
// HLS demuxer cannot make Jellyfin transcode from a new offset, so for it a
// transcoded seek must re-negotiate the stream regardless of container.
//
// Before native video shipped, `use_html5` was always true for HLS and the
// native+HLS+transcode cell was unreachable, which is why `is_hls` alone
// used to be a safe proxy for "seekable in place". It no longer is: turning
// native video on routed every transcoded seek into a backend seek that
// silently does nothing, and presents as "resume does not work".
if needs_transcoding {
// Whether a transcode can be seeked in place is a property of the
// engine, and the engine states it. This used to be inferred from
// `is_hls`, which held only while hls.js was the sole HLS renderer —
// and stopped holding the moment mpv became one (DR-238).
return match (seeks_transcoded_in_place, use_html5) {
(true, true) => VideoSeekStrategy::Html5NativeSeek,
(true, false) => VideoSeekStrategy::BackendNativeSeek,
(false, true) => VideoSeekStrategy::Html5ReloadStream,
(false, false) => VideoSeekStrategy::BackendReloadStream,
};
// Whether a transcode can be seeked in place is a property of the engine,
// and the engine states it. This used to be inferred from `is_hls`, which
// held only while hls.js was the sole HLS renderer — and stopped holding the
// moment mpv became one (DR-238).
if needs_transcoding && !seeks_transcoded_in_place {
return VideoSeekStrategy::BackendReloadStream;
}
// Direct play and direct stream are seekable where they sit.
if use_html5 {
// The frontend seeks via videoElement.currentTime; calling backend.seek()
// would move a player that is not the one rendering.
VideoSeekStrategy::Html5NativeSeek
} else {
VideoSeekStrategy::BackendNativeSeek
}
// Direct play, direct stream, or a transcode the engine can move.
VideoSeekStrategy::BackendNativeSeek
}
// The four items below are consumed by the Android MediaSessionHandler; on other
@@ -224,114 +204,45 @@ mod tests {
#[test]
fn test_seek_strategy_local_file() {
// Local files always use native backend seek regardless of other flags
assert_eq!(
determine_video_seek_strategy(true, false, false, false),
VideoSeekStrategy::LocalNativeSeek
);
assert_eq!(
determine_video_seek_strategy(true, false, false, true),
VideoSeekStrategy::LocalNativeSeek
);
assert_eq!(
determine_video_seek_strategy(true, true, true, true),
VideoSeekStrategy::LocalNativeSeek
);
for (in_place, transcode) in [(false, false), (true, true), (false, true)] {
assert_eq!(
determine_video_seek_strategy(true, in_place, transcode),
VideoSeekStrategy::LocalNativeSeek
);
}
}
/// Non-transcoded streams seek in place regardless of the engine's
/// transcode ability, which only applies to transcodes.
/// Direct play and direct stream seek in place, whatever the engine's
/// transcode ability — that only applies to transcodes.
#[test]
fn test_seek_strategy_direct_stream() {
// HTML5 renders, so the frontend seeks the element
assert_eq!(
determine_video_seek_strategy(false, true, false, true),
VideoSeekStrategy::Html5NativeSeek
);
// The native engine renders, so it seeks
assert_eq!(
determine_video_seek_strategy(false, true, false, false),
VideoSeekStrategy::BackendNativeSeek
);
// A transcode an engine says it can move: seek in place
assert_eq!(
determine_video_seek_strategy(false, true, true, true),
VideoSeekStrategy::Html5NativeSeek
);
for in_place in [false, true] {
assert_eq!(
determine_video_seek_strategy(false, in_place, false),
VideoSeekStrategy::BackendNativeSeek
);
}
}
/// A server-side transcode cannot be seeked by the native backend.
/// A server-side transcode is re-opened by an engine that cannot move it,
/// and seeked in place by one that says it can.
///
/// Jellyfin produces a transcode from `StartTimeTicks`; hls.js can seek
/// within the VOD playlist it is handed, but mpv's HLS demuxer cannot make
/// the server transcode from a new offset, so the stream has to be
/// re-negotiated. Before native video existed, `use_html5` was always true
/// for HLS and this case was unreachable — turning native video on routed
/// every transcoded seek into a native seek that silently does nothing,
/// which presents as "resume does not work".
/// Jellyfin produces a transcode from `StartTimeTicks`; mpv's HLS demuxer
/// cannot make the server transcode from a new offset, so the stream has to
/// be re-negotiated. Inferring this from the container once routed every
/// transcoded seek into a native seek that silently does nothing, which
/// presents as "resume does not work".
///
/// TRACES: UR-040 | DR-238, DR-246 | UT-217
#[test]
fn test_transcoded_seek_follows_the_engines_declared_ability() {
// An engine that cannot move a server-side transcode re-opens it,
// whichever side is rendering.
assert_eq!(
determine_video_seek_strategy(false, false, true, false),
determine_video_seek_strategy(false, false, true),
VideoSeekStrategy::BackendReloadStream
);
assert_eq!(
determine_video_seek_strategy(false, false, true, true),
VideoSeekStrategy::Html5ReloadStream
);
// hls.js can, and says so, so it seeks in place.
assert_eq!(
determine_video_seek_strategy(false, true, true, true),
VideoSeekStrategy::Html5NativeSeek
);
// The container the stream arrives in no longer decides anything: the
// same declared ability gives the same answer on the native side.
assert_eq!(
determine_video_seek_strategy(false, true, true, false),
determine_video_seek_strategy(false, true, true),
VideoSeekStrategy::BackendNativeSeek
);
}
/// Test video seek strategy for direct play (non-transcoded) streams
#[test]
fn test_seek_strategy_direct_play() {
// Direct play with HTML5 - frontend handles seek
assert_eq!(
determine_video_seek_strategy(false, false, false, true),
VideoSeekStrategy::Html5NativeSeek
);
// Direct play with native backend - backend handles seek
assert_eq!(
determine_video_seek_strategy(false, false, false, false),
VideoSeekStrategy::BackendNativeSeek
);
}
/// Test video seek strategy for transcoded non-HLS streams
#[test]
fn test_seek_strategy_transcoded_non_hls() {
// Transcoded non-HLS with HTML5 - need to reload stream, frontend handles
assert_eq!(
determine_video_seek_strategy(false, false, true, true),
VideoSeekStrategy::Html5ReloadStream
);
// Transcoded non-HLS with native backend - need to reload stream, backend handles
assert_eq!(
determine_video_seek_strategy(false, false, true, false),
VideoSeekStrategy::BackendReloadStream
);
}
/// Test the specific bug fix: HLS + HTML5 should NOT call backend seek
/// This was the bug causing "Raw(-10)" errors
#[test]
fn test_hls_html5_does_not_use_backend_seek() {
let strategy = determine_video_seek_strategy(false, true, false, true);
// Should be Html5NativeSeek, NOT BackendNativeSeek
assert_eq!(strategy, VideoSeekStrategy::Html5NativeSeek);
assert_ne!(strategy, VideoSeekStrategy::BackendNativeSeek);
}
}
src-tauri/src/player/track_switch.rs
+4 -32
View File
@@ -13,10 +13,6 @@
/// How a request to change audio track has to be carried out.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum AudioTrackSwitchStrategy {
/// Re-open the stream pinned to the chosen track; the frontend reloads its
/// `<video>` element. An HTML5 element cannot select an audio track at all,
/// so this holds whether or not the current stream is a transcode.
Html5ReloadStream,
/// Re-open the stream pinned to the chosen track; the backend reloads
/// itself and restores the position.
BackendReloadStream,
@@ -29,17 +25,9 @@ pub enum AudioTrackSwitchStrategy {
/// # Arguments
/// * `needs_transcoding` - Whether the stream now playing is a server-side
/// transcode, which carries exactly the one audio track it was built around.
/// * `use_html5` - Whether the frontend `<video>` element is rendering.
///
/// TRACES: UR-021 | IR-019, DR-024, DR-258 | UT-232
pub fn determine_audio_track_switch_strategy(
needs_transcoding: bool,
use_html5: bool,
) -> AudioTrackSwitchStrategy {
if use_html5 {
return AudioTrackSwitchStrategy::Html5ReloadStream;
}
pub fn determine_audio_track_switch_strategy(needs_transcoding: bool) -> AudioTrackSwitchStrategy {
if needs_transcoding {
AudioTrackSwitchStrategy::BackendReloadStream
} else {
@@ -86,8 +74,7 @@ mod tests {
assert_eq!(resume_position(None, 1337.5), 1337.5);
}
/// The HTML5 path does have an element and its clock is the honest answer
/// there, so what the caller supplies wins.
/// A caller that does know the position is believed.
#[test]
fn a_caller_that_knows_its_position_is_believed() {
assert_eq!(resume_position(Some(42.0), 1337.5), 42.0);
@@ -123,7 +110,7 @@ mod tests {
#[test]
fn a_transcode_is_re_opened_because_it_carries_only_one_track() {
assert_eq!(
determine_audio_track_switch_strategy(true, false),
determine_audio_track_switch_strategy(true),
AudioTrackSwitchStrategy::BackendReloadStream
);
}
@@ -133,23 +120,8 @@ mod tests {
#[test]
fn a_direct_play_switches_in_place() {
assert_eq!(
determine_audio_track_switch_strategy(false, false),
determine_audio_track_switch_strategy(false),
AudioTrackSwitchStrategy::BackendSelectInPlace
);
}
/// An HTML5 `<video>` element has no track-selection API, so it reloads
/// either way. This is the path that already worked, and it must keep
/// working: the fix is about the native side only.
#[test]
fn html5_always_reloads_because_the_element_cannot_select() {
assert_eq!(
determine_audio_track_switch_strategy(true, true),
AudioTrackSwitchStrategy::Html5ReloadStream
);
assert_eq!(
determine_audio_track_switch_strategy(false, true),
AudioTrackSwitchStrategy::Html5ReloadStream
);
}
}
src-tauri/src/player/webview_audio_backend.rs
+11 -18
View File
@@ -1,25 +1,18 @@
//! Webview audio backend — audio-only playback for platforms without a native
//! audio backend (currently Windows).
//! Webview audio backend — audio-only playback for a desktop without a native
//! audio backend. No shipped platform uses it: Linux and Windows play through
//! mpv, Android through ExoPlayer.
//!
//! ## Why this exists
//! All *video* already renders through the webview HTML5 `<video>` element on
//! every platform (see `VideoPlayer.svelte`); libmpv/ExoPlayer only ever drive
//! *audio-only* (music) playback. On Windows there is no native audio backend,
//! so `create_player_backend()` used to fall back to `NullBackend` and music was
//! silent.
//!
//! This backend fills that gap without any C dependency (so it still
//! cross-compiles from Linux): instead of decoding audio itself, it hands the
//! stream URL to a frontend `<audio>` element via a `WebviewAudioLoad` event and
//! then drives play/pause/seek/stop through `ControlCommand` events — exactly the
//! round-trip the HTML5 video path already uses. The `<audio>` element reports
//! its real state/position back through the `player_report_*` commands, so the
//! Rust `PlayerController` remains the single source of truth (the controller's
//! `report_html5_*` methods fold those reports into the normal event pipeline).
//! Instead of decoding audio itself, it hands the stream URL to a frontend
//! `<audio>` element via a `WebviewAudioLoad` event and then drives
//! play/pause/seek/stop through `ControlCommand` events. The `<audio>` element
//! reports its real state/position back through the `player_report_*` commands,
//! so the Rust `PlayerController` remains the single source of truth (the
//! controller's `report_html5_*` methods fold those reports into the normal
//! event pipeline).
//!
//! Because the reported state flows through the event pipeline (not through this
//! backend's `position()`/`state()` pollers — the timer loop does not poll the
//! backend for HTML5-rendered media), this backend only needs to keep a
//! backend for webview-rendered media), this backend only needs to keep a
//! best-effort local mirror for direct `player_get_state` queries.
//!
//! TRACES: UR-003, UR-004, UR-005 | DR-004
src-tauri/tauri.conf.json
+2 -2
View File
@@ -22,8 +22,8 @@
}
],
"security": {
"csp": "default-src 'self'; script-src 'self'; style-src 'self' 'unsafe-inline'; font-src 'self' data:; img-src 'self' data: blob: asset: http://asset.localhost http: https:; media-src 'self' blob: asset: http://asset.localhost http://127.0.0.1:* http: https:; connect-src 'self' ipc: http://ipc.localhost http: https:; worker-src 'self' blob:; object-src 'none'; frame-src 'none'; base-uri 'self'; form-action 'self'; frame-ancestors 'none'",
"devCsp": "default-src 'self'; script-src 'self' 'unsafe-inline' 'unsafe-eval'; style-src 'self' 'unsafe-inline'; font-src 'self' data:; img-src 'self' data: blob: asset: http://asset.localhost http: https:; media-src 'self' blob: asset: http://asset.localhost http://127.0.0.1:* http: https:; connect-src 'self' ipc: http://ipc.localhost http: https: ws: wss:; worker-src 'self' blob:; object-src 'none'; frame-src 'none'; base-uri 'self'; form-action 'self'",
"csp": "default-src 'self'; script-src 'self'; style-src 'self' 'unsafe-inline'; font-src 'self' data:; img-src 'self' data: blob: asset: http://asset.localhost http: https:; media-src 'self' blob: asset: http://asset.localhost http://127.0.0.1:* http: https:; connect-src 'self' ipc: http://ipc.localhost; worker-src 'self'; object-src 'none'; frame-src 'none'; base-uri 'self'; form-action 'self'; frame-ancestors 'none'",
"devCsp": "default-src 'self'; script-src 'self' 'unsafe-inline' 'unsafe-eval'; style-src 'self' 'unsafe-inline'; font-src 'self' data:; img-src 'self' data: blob: asset: http://asset.localhost http: https:; media-src 'self' blob: asset: http://asset.localhost http://127.0.0.1:* http: https:; connect-src 'self' ipc: http://ipc.localhost ws: wss:; worker-src 'self'; object-src 'none'; frame-src 'none'; base-uri 'self'; form-action 'self'",
"assetProtocol": {
"enable": true,
"scope": [