From e144e62b3177561ef835a0e7052d3e7ba5199119 Mon Sep 17 00:00:00 2001 From: Duncan Tourolle Date: Tue, 11 Aug 2026 20:34:24 +0200 Subject: [PATCH 1/4] feat(player): render Android video natively behind a transparent webview (DR-150, DR-151, DR-152) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Rust already reported `use_html5_element: false` on Android, but two frontend overrides threw that answer away, so ExoPlayer's video path had never actually run. Both are lifted behind an `experimentalNativeVideo` opt-in (default off). The flag is a suppressor, never a promoter: off forces HTML5 even where Rust says native, so an in-progress spike cannot ship as the default, but it can never select native where Rust reported HTML5 — Linux cannot composite behind WebKitGTK, and promoting there would be a black screen. Two blockers the spec did not anticipate, both in code assumed to be merely unreachable rather than broken: - `JellyTauPlayer.setActivity()` had zero callers, so `currentActivity` was always null and `autoAttachSurface()` bailed. The SurfaceView was created and wired to ExoPlayer but never added to the view hierarchy — video would have decoded to a surface that was never on screen, whatever the webview did. This also revives PiP on the video path, which gated on the same flag. - `createAdapter()` was not the real gate; it is never called in production. The actual override was in VideoPlayer.svelte, which forced HTML5 and stopped the native backend `player_play_item` had just started. Both sites now route through `createAdapter()`. Compositing needs two independent opaque layers cleared, not one. Clearing only the page leaves the WebView widget opaque — audio over a black picture, exactly the symptom the old INTERIM comment described. `videoSurface.ts` toggles both: the widget background and window drawable from Kotlin, the page backgrounds via a `data-native-video` attribute keyed by app.css. Transparency lives in `tauri.android.conf.json` so Linux keeps an opaque window, and is scoped to the playback session so the launcher never shows through the rest of the app. Phase 3's rect plumbing turned out to be unnecessary: video is fullscreen on the player route, and `fitSurfaceToScreen()` already letterboxes and re-fits on rotation. The mini-player transition remains unverified on device. Also removes the `navigator.userAgent` sniffing in webviewAudio.ts, which was a second copy of the Rust cfg gate free to drift from it. `player_get_capabilities` now reports `usesWebviewAudio` and `supportsNativeVideo` from those same gates. Tests: adapter selection covers the full matrix, including the regression guard that the flag off beats Rust. Written first and confirmed failing (2 of 7) before the fix. Co-Authored-By: Claude Opus 5 --- docs/requirements.md | 4 + docs/specs/android-native-video-spike.md | 83 ++++++++++++++-- .../com/dtourolle/jellytau/MainActivity.kt | 54 +++++++++++ src-tauri/src/commands/player/mod.rs | 37 ++++++++ src-tauri/src/lib.rs | 2 + src-tauri/tauri.android.conf.json | 10 ++ src/app.css | 24 +++++ src/lib/api/bindings.ts | 32 +++++++ src/lib/components/player/VideoPlayer.svelte | 70 +++++++++++--- .../player/adapters/adapterSelection.test.ts | 95 +++++++++++++++++++ src/lib/player/adapters/index.ts | 58 +++++++---- src/lib/services/playbackCapabilities.ts | 70 ++++++++++++++ src/lib/services/webviewAudio.ts | 22 ++--- src/lib/stores/nativeVideo.ts | 91 ++++++++++++++++++ src/lib/utils/videoSurface.ts | 86 +++++++++++++++++ src/routes/settings/+page.svelte | 63 +++++++++++- 16 files changed, 745 insertions(+), 56 deletions(-) create mode 100644 src-tauri/tauri.android.conf.json create mode 100644 src/lib/player/adapters/adapterSelection.test.ts create mode 100644 src/lib/services/playbackCapabilities.ts create mode 100644 src/lib/stores/nativeVideo.ts create mode 100644 src/lib/utils/videoSurface.ts diff --git a/docs/requirements.md b/docs/requirements.md index cbc3df63..6a2b7bc1 100644 --- a/docs/requirements.md +++ b/docs/requirements.md @@ -312,6 +312,9 @@ Internal architecture, components, and application logic. | DR-146 | The no-audio-track fallback picks a track the renderer can actually play. When ExoPlayer selected no audio track, the recovery forced group 0 / track 0 unconditionally — but the most likely reason nothing was selected is that this very track cannot be decoded on this device, so the override reinstated the silence it was meant to fix. It now scans the groups for the first `isTrackSupported` track and overrides to that, and clears `setTrackTypeDisabled(TRACK_TYPE_AUDIO)` because audio may equally have been off at the type level, which an override alone does not undo. When no group holds a supported track the condition is logged as an error — the server was expected to transcode — rather than leaving a silent video with no explanation in the log | Playback | UR-004 | Done | | DR-148 | The video direct-play profile advertises only what the **webview** can decode. The audio codec list comes from `MediaCodecList`, which describes ExoPlayer — but video does not play through ExoPlayer on either platform: Android force-renders every video in the webview `