feat(player): render Android video natively behind a transparent webview (DR-150, DR-151, DR-152)

Rust already reported `use_html5_element: false` on Android, but two frontend
overrides threw that answer away, so ExoPlayer's video path had never actually
run. Both are lifted behind an `experimentalNativeVideo` opt-in (default off).

The flag is a suppressor, never a promoter: off forces HTML5 even where Rust
says native, so an in-progress spike cannot ship as the default, but it can
never select native where Rust reported HTML5 — Linux cannot composite behind
WebKitGTK, and promoting there would be a black screen.

Two blockers the spec did not anticipate, both in code assumed to be merely
unreachable rather than broken:

- `JellyTauPlayer.setActivity()` had zero callers, so `currentActivity` was
  always null and `autoAttachSurface()` bailed. The SurfaceView was created and
  wired to ExoPlayer but never added to the view hierarchy — video would have
  decoded to a surface that was never on screen, whatever the webview did.
  This also revives PiP on the video path, which gated on the same flag.
- `createAdapter()` was not the real gate; it is never called in production.
  The actual override was in VideoPlayer.svelte, which forced HTML5 and stopped
  the native backend `player_play_item` had just started. Both sites now route
  through `createAdapter()`.

Compositing needs two independent opaque layers cleared, not one. Clearing only
the page leaves the WebView widget opaque — audio over a black picture, exactly
the symptom the old INTERIM comment described. `videoSurface.ts` toggles both:
the widget background and window drawable from Kotlin, the page backgrounds via
a `data-native-video` attribute keyed by app.css. Transparency lives in
`tauri.android.conf.json` so Linux keeps an opaque window, and is scoped to the
playback session so the launcher never shows through the rest of the app.

Phase 3's rect plumbing turned out to be unnecessary: video is fullscreen on the
player route, and `fitSurfaceToScreen()` already letterboxes and re-fits on
rotation. The mini-player transition remains unverified on device.

Also removes the `navigator.userAgent` sniffing in webviewAudio.ts, which was a
second copy of the Rust cfg gate free to drift from it. `player_get_capabilities`
now reports `usesWebviewAudio` and `supportsNativeVideo` from those same gates.

Tests: adapter selection covers the full matrix, including the regression guard
that the flag off beats Rust. Written first and confirmed failing (2 of 7) before
the fix.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-11 20:57:58 +02:00
co-authored by Claude Opus 5
parent 07d10dfed7
commit e144e62b31
16 changed files with 745 additions and 56 deletions
@@ -64,6 +64,20 @@ class MainActivity : TauriActivity() {
// so on devices with a tall opaque 3-button bar. (UR-066)
WindowInsetsBridge.install(this)
// Hand the player an Activity reference so it can attach its video
// SurfaceView to the content view behind the WebView.
//
// Without this, JellyTauPlayer.currentActivity stays null forever and
// autoAttachSurface() logs "Cannot attach surface - no Activity reference"
// and returns — so the SurfaceView is created, wired to ExoPlayer, and then
// never added to the view hierarchy. Native video decoded to a surface that
// was never on screen. setActivity() stores into a companion-object
// WeakReference, so calling it here (before Rust initializes the player over
// JNI) is safe and is the case it was written for.
//
// TRACES: UR-003, UR-041 | DR-151
com.dtourolle.jellytau.player.JellyTauPlayer.setActivity(this)
// Configure WebView for media playback after Tauri initialization
handler.postDelayed({
configureWebViewForMedia()
@@ -271,6 +285,46 @@ class MainActivity : TauriActivity() {
}, "AndroidNetworkType")
android.util.Log.d("MainActivity", "JavaScript interface 'AndroidNetworkType' added")
// Native video compositing: let the frontend make the WebView transparent
// so the ExoPlayer SurfaceView behind it is visible (UR-003, UR-004).
//
// Toggled rather than set once because a transparent WebView is only
// correct while a native video is on screen — every other screen needs its
// opaque background, and leaving the window transparent shows the
// launcher/wallpaper through the app.
//
// The CSS in app.css clears the *web* layer's backgrounds; this clears the
// WebView widget's own background, which CSS cannot reach. Both are
// required — an opaque WebView hides the surface no matter what the page
// paints.
//
// TRACES: UR-003, UR-004 | DR-150
webView.addJavascriptInterface(object : Any() {
/** Make the WebView background transparent (true) or opaque (false). */
@JavascriptInterface
fun setTransparent(transparent: Boolean) {
handler.post {
val color = if (transparent) {
android.graphics.Color.TRANSPARENT
} else {
android.graphics.Color.BLACK
}
mediaWebView?.setBackgroundColor(color)
// The WebView's window/surface must also stop painting opaque, or a
// hardware-accelerated WebView still composites its own background.
window.setBackgroundDrawable(
android.graphics.drawable.ColorDrawable(color)
)
android.util.Log.d("MainActivity", "WebView transparent = $transparent")
}
}
/** Whether native-video compositing is available on this platform. */
@JavascriptInterface
fun isSupported(): Boolean = true
}, "AndroidVideoSurface")
android.util.Log.d("MainActivity", "JavaScript interface 'AndroidVideoSurface' added")
// Window insets (safe areas). The push path above races the page load, so
// the frontend pulls the current values on mount through this bridge.
webView.addJavascriptInterface(WindowInsetsBridge.jsInterface(), "AndroidInsets")
+37
View File
@@ -1654,6 +1654,43 @@ pub async fn player_get_queue(
Ok(get_queue_status(&controller))
}
/// What playback facilities this platform's backend actually provides.
///
/// The frontend is presentation-only and must not re-derive backend facts from
/// `navigator.userAgent` — that sniffing was a second copy of the same platform
/// decision Rust already makes with `cfg!`, and it drifted. These flags are the
/// single source of truth; the frontend consumes them.
///
/// TRACES: UR-003, UR-005 | DR-004, DR-023, DR-024
#[derive(specta::Type, Debug, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct PlaybackCapabilities {
/// True when audio is rendered by a webview `<audio>` element rather than a
/// native backend. Native audio exists on Linux (mpv) and Android
/// (ExoPlayer); everything else (Windows, future desktops) uses the webview.
pub uses_webview_audio: bool,
/// True when video can be rendered by a native surface composited *behind*
/// a transparent webview. Android only: ExoPlayer draws into a SurfaceView
/// beneath the WebView. Linux cannot do this (WebKitGTK/Wayland
/// compositing), so it stays on the HTML5 element.
pub supports_native_video: bool,
}
/// Report this platform's playback capabilities to the frontend.
///
/// TRACES: UR-003, UR-005 | DR-004, DR-023, DR-024
#[tauri::command]
#[specta::specta]
pub async fn player_get_capabilities() -> Result<PlaybackCapabilities, String> {
// Mirrors the cfg gates the backends themselves are built under.
let native_audio = cfg!(any(target_os = "android", target_os = "linux"));
Ok(PlaybackCapabilities {
uses_webview_audio: !native_audio,
supports_native_video: cfg!(target_os = "android"),
})
}
pub(super) fn get_player_status(controller: &PlayerController) -> PlayerStatus {
// Determine backend at compile time based on platform
let (backend, use_html5_element) = if cfg!(target_os = "android") {
+2
View File
@@ -123,6 +123,7 @@ use commands::{
player_get_audio_settings,
player_get_autoplay_settings,
player_get_cache_config,
player_get_capabilities,
player_get_eq_presets,
player_get_queue,
// Session management commands
@@ -682,6 +683,7 @@ fn specta_builder() -> Builder<tauri::Wry> {
player_cycle_repeat,
player_get_status,
player_get_queue,
player_get_capabilities,
player_add_to_queue,
player_add_track_by_id,
player_add_tracks_by_ids,
+10
View File
@@ -0,0 +1,10 @@
{
"$schema": "https://schema.tauri.app/config/2",
"app": {
"windows": [
{
"transparent": true
}
]
}
}