dtourolle/jellytau
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 52 Wiki Activity

refactor(player): delete the webview video path; mpv selects its own tracks

Browse Source 
DR-235 phase 3. Every video renderer is native now: mpv on Linux and
Windows, ExoPlayer on Android, all drawing behind the transparent
webview. The HTML5 <video> path is gone, not bypassed:

- Frontend: hls.js, Html5PlayerAdapter and its compatibility shim, the
  createAdapter factory, streamTransport, hlsRecovery, timeTracking,
  videoFit, the <video>/<track> markup and every element handler in
  VideoPlayer (3277 -> 2144 lines), the experimentalNativeVideo store
  and its Settings toggle, webviewVideoFallback/supportsNativeVideo, and
  the setHtml5VideoState PiP bridge call. NativePlayerAdapter is the one
  video adapter; webview audio gets its own adapter kind.
- Rust: use_html5 dropped from player_seek_video,
  player_switch_audio_track and player_set_stream_quality with the
  Html5* strategies and ReloadStream responses; use_html5_element and
  VideoBackend dropped from PlayerStatus; player_play_item always loads
  the backend (set_current_item removed); Capabilities::webview removed;
  the WebKitGTK GStreamer/VAAPI setup (and its gst-inspect spawn) removed.
- Android: the HTML5 video state in PictureInPictureManager and
  ScreenWakeManager, and the bridge method feeding it.
- CSP: connect-src loses http:/https: and worker-src loses blob: -
  both existed for hls.js; with it gone they were only an exfiltration
  channel and a blob worker for injected script. A test now keeps them
  out.

mpv takes over what the <video> element did (mpv_tracks, UT-275):
subtitles are the WebVTT list the play request carries, queued on
sub-files and selected by position in that list, starting off; audio
tracks are selected by position in the file; sid/aid are reset before
each load. Without this, Linux video had no subtitle selection and a
direct-play audio switch failed since mpv became its renderer.

Verified: Rust 948 passing, and the same 948 cross-compiled for Windows
under wine against the shipped DLL (track tests included); frontend
1111 passing; aarch64 debug APK builds. Lint warnings 158 -> 146, CI
ratchet tightened to match. Not yet seen on Windows hardware.
This commit is contained in:
dtourolle
2026-09-24 23:11:17 -04:00
parent bb3ab1edd7
commit 1677f5f299
69 changed files with 4014 additions and 7330 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src-tauri/android/src/main/java/com/dtourolle/jellytau/MainActivity.kt
+3 -20
View File
@@ -85,10 +85,9 @@ class MainActivity : TauriActivity() {
super.onWebViewCreate(webView)
android.util.Log.d("MainActivity", "onWebViewCreate - installing bridges before first page load")
mediaWebView = webView
// A new WebView means a new page, which reports no video yet. Anything the
// previous one left held would otherwise pin the screen on for the life of
// the process, since a page that goes away never sends its final
// setHtml5VideoState(false, …). (DR-202)
// A new WebView means a new page, which plays no video yet. Anything held
// for the previous one would otherwise pin the screen on for the life of
// the process. (DR-202)
ScreenWakeManager.releaseAll()
installJavascriptBridges(webView)
configureWebViewSettings(webView)
@@ -327,22 +326,6 @@ class MainActivity : TauriActivity() {
autoEnterPipEnabled = enabled
}
/**
* Report the WebView `<video>` state.
*
* Without this PiP only ever knew about the native ExoPlayer surface,
* which is behind an experimental flag that defaults to off — so in the
* shipping configuration nothing ever satisfied canEnterPip and the
* button did nothing. (DR-160)
*/
@JavascriptInterface
fun setHtml5VideoState(active: Boolean, width: Int, height: Int, playing: Boolean) {
PictureInPictureManager.setHtml5VideoState(active, width, height, playing)
// The same report is what keeps the display awake on the webview
// rendering path — the WebView takes no display wake lock of its own
// for `<video>`. (DR-202)
ScreenWakeManager.onHtml5VideoState(active, playing)
}
}, "AndroidPictureInPicture")
android.util.Log.d("MainActivity", "JavaScript interface 'AndroidPictureInPicture' added")
src-tauri/android/src/main/java/com/dtourolle/jellytau/PictureInPictureManager.kt
+21 -93
View File
@@ -46,52 +46,7 @@ object PictureInPictureManager {
private var receiver: BroadcastReceiver? = null
private var hiddenWebView: WebView? = null
/**
* State of an HTML5 `<video>` playing inside the WebView, reported by the
* frontend.
*
* PiP was written for the native ExoPlayer surface only — [canEnterPip]
* required a SurfaceView to be attached and rendering. But native video is
* behind `experimentalNativeVideo`, which defaults to **off**, so in the
* shipping configuration video plays in the WebView's `<video>` element and
* every one of those conditions is false. `enterPip` therefore always bailed
* with "no local video playing": PiP could not work at all, however the
* button was pressed.
*
* On this path the WebView *is* the video, which inverts two things: the
* WebView must stay visible in PiP rather than be hidden, and play/pause has
* to reach the element rather than ExoPlayer. Both are handled below.
*
* TRACES: UR-041 | DR-160
*/
@Volatile
private var html5VideoActive = false
@Volatile
private var html5VideoPlaying = false
@Volatile
private var html5AspectRatio: Rational? = null
/**
* Report the WebView `<video>` state from the frontend.
*
* @param active whether a video element is currently the playback surface
* @param width intrinsic video width, for the PiP window's aspect ratio
* @param height intrinsic video height
* @param playing whether it is playing right now, for the PiP play/pause action
*/
fun setHtml5VideoState(active: Boolean, width: Int, height: Int, playing: Boolean) {
html5VideoActive = active
html5VideoPlaying = playing
html5AspectRatio = if (active && width > 0 && height > 0) {
clampedRatio(width.toDouble() / height.toDouble())
} else {
null
}
}
/** True when PiP would be showing the native surface rather than the WebView. */
/** True when a native video surface is attached and playing. */
private fun isNativeVideoPath(): Boolean = try {
val player = JellyTauPlayer.getInstance()
player.isPlayingVideo() &&
@@ -120,10 +75,9 @@ object PictureInPictureManager {
*/
fun canEnterPip(activity: Activity): Boolean {
if (!isPipSupported(activity)) return false
// Either surface will do: the native one, or the WebView's `<video>`,
// which is what actually plays while experimentalNativeVideo is off.
// (DR-160)
return isNativeVideoPath() || html5VideoActive
// Video only ever renders on the native surface: the WebView `<video>`
// path is gone (DR-235). (DR-160)
return isNativeVideoPath()
}
/**
@@ -186,9 +140,7 @@ object PictureInPictureManager {
return clampedRatio(surface.width.toDouble() / surface.height.toDouble())
}
// No native surface: the WebView is the video, so use the intrinsic size
// the frontend reported. (DR-160)
return html5AspectRatio
return null
}
/**
@@ -207,16 +159,10 @@ object PictureInPictureManager {
@RequiresApi(Build.VERSION_CODES.O)
private fun buildPlayPauseAction(activity: Activity): RemoteAction {
// On the HTML5 path ExoPlayer is idle, so its `isPlaying` is always false
// and the button would be stuck showing "Play" mid-playback. (DR-160)
val isPlaying = if (isNativeVideoPath()) {
try {
JellyTauPlayer.getInstance().getExoPlayer().isPlaying
} catch (e: Exception) {
false
}
} else {
html5VideoPlaying
val isPlaying = try {
JellyTauPlayer.getInstance().getExoPlayer().isPlaying
} catch (e: Exception) {
false
}
val (iconRes, title, controlType, requestCode) = if (isPlaying) {
@@ -288,11 +234,8 @@ object PictureInPictureManager {
*/
fun onPipModeChanged(activity: Activity, isInPipMode: Boolean) {
if (isInPipMode) {
// Hiding the WebView is correct only when the video is *behind* it on
// the native surface. On the HTML5 path the WebView is the video, so
// hiding it would leave an empty black PiP window — the frontend
// instead strips its own chrome when it hears the event below.
// (DR-160)
// The video is *behind* the WebView on the native surface, so hiding
// the WebView leaves only the picture. (DR-160)
if (isNativeVideoPath()) {
hideWebView(activity)
}
@@ -315,9 +258,8 @@ object PictureInPictureManager {
/**
* Fire a DOM event into the WebView.
*
* The HTML5 PiP path is a conversation with the frontend rather than
* something native can do alone: it has to be told to strip its chrome when
* the window shrinks, and to play/pause the element. (DR-160)
* Tells the frontend the window shrank or grew, so it can strip or restore
* its chrome. (DR-160)
*/
private fun dispatchWebEvent(activity: Activity, name: String) {
val webView = findWebView(activity.window.decorView) ?: return
@@ -358,28 +300,14 @@ object PictureInPictureManager {
if (intent?.action != ACTION_MEDIA_CONTROL) return
val control = intent.getIntExtra(EXTRA_CONTROL_TYPE, 0)
if (isNativeVideoPath()) {
val player = try {
JellyTauPlayer.getInstance()
} catch (e: Exception) {
return
}
when (control) {
CONTROL_PLAY -> player.play()
CONTROL_PAUSE -> player.pause()
}
} else {
// The WebView owns playback here, so the command has to reach
// the `<video>` element. Driving ExoPlayer instead would do
// nothing at all, which is what a PiP button on the HTML5 path
// used to do. (DR-160)
val name = when (control) {
CONTROL_PLAY -> "jellytau-pip-play"
CONTROL_PAUSE -> "jellytau-pip-pause"
else -> return
}
dispatchWebEvent(activity, name)
html5VideoPlaying = control == CONTROL_PLAY
val player = try {
JellyTauPlayer.getInstance()
} catch (e: Exception) {
return
}
when (control) {
CONTROL_PLAY -> player.play()
CONTROL_PAUSE -> player.pause()
}
// Swap the button to reflect the new state.
updatePipActions(activity)
src-tauri/android/src/main/java/com/dtourolle/jellytau/ScreenWakeManager.kt
+13 -44
View File
@@ -7,14 +7,12 @@ import android.view.WindowManager
import java.lang.ref.WeakReference
/**
* Which playback paths currently want the screen kept awake.
* Whether playback currently wants the screen kept awake.
*
* Pure state, deliberately free of any Android type so it can be unit-tested —
* see ScreenWakeStateTest. Two independent holders, because video can be
* rendered by either renderer and only one of them is active at a time:
*
* - **native** — ExoPlayer drawing into the TextureView (DR-192)
* - **html5** — a `<video>` inside the WebView, reported by the frontend
* see ScreenWakeStateTest. The one holder is ExoPlayer drawing video into the
* TextureView (DR-192); the WebView `<video>` that was a second holder is gone
* (DR-235).
*
* Audio is deliberately *not* a holder. Playing music with the screen off is the
* point of the audio path; only video needs the display alive.
@@ -23,11 +21,10 @@ import java.lang.ref.WeakReference
*/
class ScreenWakeState {
private var nativeVideoPlaying = false
private var html5VideoPlaying = false
/** True while any video renderer is actively playing. */
/** True while video is actively playing. */
val keepScreenOn: Boolean
get() = nativeVideoPlaying || html5VideoPlaying
get() = nativeVideoPlaying
/**
* @param playing whether ExoPlayer is playing right now
@@ -37,18 +34,9 @@ class ScreenWakeState {
nativeVideoPlaying = playing && isVideo
}
/**
* @param active whether a webview `<video>` is the current playback surface
* @param playing whether that element is playing right now
*/
fun updateHtml5(active: Boolean, playing: Boolean) {
html5VideoPlaying = active && playing
}
/** Drop every hold (teardown, or a page that can no longer be trusted). */
fun reset() {
nativeVideoPlaying = false
html5VideoPlaying = false
}
}
@@ -66,9 +54,7 @@ class ScreenWakeState {
* appeared nowhere, and neither renderer supplies one for free — ExoPlayer's
* `setWakeMode` is a *CPU/wifi* wake lock and says nothing about the display,
* and it draws into a `TextureView` we own rather than a `PlayerView`, which is
* the media3 widget that would otherwise set `keepScreenOn` itself. The WebView
* `<video>` path does not either: the display wake lock Chrome takes for video
* lives in the browser layer, not in an embedded WebView.
* the media3 widget that would otherwise set `keepScreenOn` itself.
*
* ## Approach
*
@@ -78,15 +64,9 @@ class ScreenWakeState {
* missed release the way an explicitly acquired wake lock can. It needs no
* permission. (The manifest's `WAKE_LOCK` is the media service's, unrelated.)
*
* The two renderers report independently and are OR-ed together in
* [ScreenWakeState]:
*
* - `JellyTauPlayer.onIsPlayingChanged` and its surface teardown drive the
* native path — ExoPlayer is the authoritative source of playback state, so
* the hold follows what it reports rather than what the UI intends.
* - `MainActivity`'s `AndroidPictureInPicture.setHtml5VideoState` bridge drives
* the webview path. The frontend already reports that state on every
* play/pause and on player teardown for PiP, so no new bridge is needed.
* `JellyTauPlayer.onIsPlayingChanged` and its surface teardown drive
* [ScreenWakeState] — ExoPlayer is the authoritative source of playback state,
* so the hold follows what it reports rather than what the UI intends.
*
* The Activity reference is weak and re-set on every `onCreate`, so a
* recreation (rotation) re-applies the current hold to the new window.
@@ -126,20 +106,9 @@ object ScreenWakeManager {
}
/**
* The frontend reported the webview `<video>` state. Arrives on a WebView
* binder thread, hence the synchronization and the post to the main thread.
*/
@Synchronized
fun onHtml5VideoState(active: Boolean, playing: Boolean) {
state.updateHtml5(active, playing)
apply()
}
/**
* Drop every hold. Used when a new WebView/page load invalidates whatever the
* previous page last reported — a page that goes away without a final
* `setHtml5VideoState(false, …)` would otherwise leave the screen pinned on
* for the life of the process.
* Drop every hold. Used when a new WebView/page load starts from scratch, so
* nothing held for the previous page can pin the screen on for the life of
* the process.
*/
@Synchronized
fun releaseAll() {
src-tauri/android/src/test/java/com/dtourolle/jellytau/ScreenWakeStateTest.kt
+1 -38
View File
@@ -40,46 +40,9 @@ class ScreenWakeStateTest {
}
@Test
fun `webview video playing holds the screen on`() {
val state = ScreenWakeState()
state.updateHtml5(active = true, playing = true)
assertTrue(state.keepScreenOn)
}
@Test
fun `webview video paused releases the screen`() {
val state = ScreenWakeState()
state.updateHtml5(active = true, playing = true)
state.updateHtml5(active = true, playing = false)
assertFalse(state.keepScreenOn)
}
/** The element going away must release even if it never reported a pause. */
@Test
fun `webview video going inactive while playing releases the screen`() {
val state = ScreenWakeState()
state.updateHtml5(active = true, playing = true)
state.updateHtml5(active = false, playing = true)
assertFalse(state.keepScreenOn)
}
/** The two rendering paths are independent holders; either one is enough. */
@Test
fun `one path releasing does not release while the other still plays`() {
fun `teardown releases the hold`() {
val state = ScreenWakeState()
state.updateNative(playing = true, isVideo = true)
state.updateHtml5(active = true, playing = true)
state.updateHtml5(active = false, playing = false)
assertTrue(state.keepScreenOn)
state.updateNative(playing = false, isVideo = true)
assertFalse(state.keepScreenOn)
}
@Test
fun `teardown releases both paths`() {
val state = ScreenWakeState()
state.updateNative(playing = true, isVideo = true)
state.updateHtml5(active = true, playing = true)
state.reset()
assertFalse(state.keepScreenOn)
}