dtourolle/jellytau
1
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases 39 Wiki Activity

fix(player): make transport reach the player that is actually rendering

Browse Source 
Play/pause did nothing on the Android native video path — from the on-screen
tap, from the control bar, and from a direct player_toggle invocation — while
seek and skip kept working. That asymmetry was the whole clue: seek decides in
player_seek_video, transport decides in toggle_playback.

DR-195 is the cause. `html5_playing` is Rust's record of "a webview <video> is
active and in this state", and toggle_playback/play/pause all route transport to
that element whenever it is set. The player route mirrored element state into it
UNCONDITIONALLY — from handleReportStart and, fatally, from handleReportProgress,
which VideoPlayer calls on a 10-second interval. So on the native path the
frontend re-declared every ten seconds that an element was playing when none
existed, and every transport intent was emitted into the void. It also explains
the flashing: the control bar and the JRay overlay both key off isPlaying, which
was being contradicted on every tick. The mirror now lives in
mirrorElementStateToRust() in VideoPlayer, gated on useHtml5Element — the only
place that knows whether an element renders at all. The route cannot tell the
paths apart, which is exactly how it came to lie.

DR-193 hands transport authority back to the native backend when an item loads
into it. Necessary but insufficient alone: the progress interval put the flag
straight back, which is why the first device test after it still failed.

DR-192 presents native video through a TextureView instead of a SurfaceView. A
SurfaceView renders on its own layer outside the app window and punches a
transparent region through it, and everything drawn above that hole — here, the
entire Svelte UI — depends on that composition path. The overlay dropped its
incremental damage: the DOM advanced (slider 476 -> 479 across three seconds)
behind a screen showing neither, so the progress bar froze, controls would not
fade and rotation lost the transport UI, while structural DOM changes got
through, which is why the play overlay always appeared to work. It supersedes
DR-191, which forced redraws in a loop and treated the symptom.

DR-194 hides the video view across a resize and reveals it two frames later. A
TextureView retains its last frame, so between a rotation and the re-fit landing
that frame is stretched across the old rect and the previous frame flashes in
what should be the letterbox bars.

Verified on device (Honor ROD2-W09, Android 16) by driving ADB and reading the
live DOM over the devtools socket: surface tap pauses (position frozen across 12
seconds, overlay raised, transport flipped) and resumes; the control bar does
both. UT-189 drives the real 10-second interval under fake timers — an earlier
version asserted on a freshly mounted player, passed with the guard deleted, and
guarded nothing.

Still open, and deliberately not claimed: DR-192's effect on the overlay repaint
is unverified on device, DR-194's letterbox reset is untested, and the native
default (DR-188) stays off pending DR-190, the background-audio return.
This commit is contained in:
dtourolle
2026-08-16 18:03:22 +02:00
parent 95129d04a3
commit c142568230
10 changed files with 1995 additions and 1733 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src-tauri/android/src/main/java/com/dtourolle/jellytau/MainActivity.kt
-61
View File
@@ -10,12 +10,6 @@ import android.webkit.WebView
import android.view.View
import androidx.activity.enableEdgeToEdge
/**
* How often to ask the WebView overlay to redraw while a native video surface is
* behind it. Roughly one display frame — `postInvalidateOnAnimation` coalesces
* to vsync, so this only governs how often we *ask*. See [MainActivity.overlayRepaint].
*/
private const val OVERLAY_REPAINT_MS = 16L
class MainActivity : TauriActivity() {
private val handler = Handler(Looper.getMainLooper())
@@ -95,60 +89,6 @@ class MainActivity : TauriActivity() {
configureWebViewSettings(webView)
}
/**
* Keeps the WebView overlay repainting while a native video surface is behind
* it. Null when not running.
*
* With the ExoPlayer SurfaceView composited *under* a transparent WebView, the
* WebView's ordinary damage stops reaching the screen: the page kept mutating
* — the clock text changing every second, the control bar's opacity going to
* 0 — and none of it appeared, while the video underneath animated fine. The
* overlay froze on whatever frame it last managed to present, which is why the
* progress bar "stayed there and did not update" and the controls would not
* fade. It is not a state bug: reading the DOM over the devtools socket showed
* the slider value advancing (476 → 479 over three seconds) behind a screen
* showing neither.
*
* A *structural* change did force it through — injecting one element made the
* whole overlay catch up at once, jumping the displayed clock from 6:39 to
* 19:35 — so the pixels are reachable; it is the incremental damage that gets
* dropped. A CSS animation does not do it: opacity animates on the compositor
* without repainting the layer, which is exactly why that attempt changed
* nothing.
*
* So the redraw is driven from here instead. `postInvalidateOnAnimation`
* rather than a fixed-rate timer, so it rides the display's vsync and cannot
* outpace the frames it is asking for, and it runs *only* while compositing is
* on — the cost belongs to native video playback, which is already decoding.
*
* This is a workaround for platform compositing behaviour, not a fix for a
* defect of ours; the honest form of it is narrow and self-cancelling.
*
* TRACES: UR-003, UR-004 | DR-191
*/
private var overlayRepaint: Runnable? = null
private fun startOverlayRepaint() {
if (overlayRepaint != null) return
val tick = object : Runnable {
override fun run() {
val webView = mediaWebView ?: return
webView.postInvalidateOnAnimation()
// Re-post through the same field so stopOverlayRepaint() can cancel it.
overlayRepaint?.let { handler.postDelayed(it, OVERLAY_REPAINT_MS) }
}
}
overlayRepaint = tick
handler.post(tick)
android.util.Log.d("MainActivity", "Overlay repaint started")
}
private fun stopOverlayRepaint() {
overlayRepaint?.let { handler.removeCallbacks(it) }
overlayRepaint = null
android.util.Log.d("MainActivity", "Overlay repaint stopped")
}
override fun onCreate(savedInstanceState: Bundle?) {
enableEdgeToEdge()
super.onCreate(savedInstanceState)
@@ -442,7 +382,6 @@ class MainActivity : TauriActivity() {
android.graphics.drawable.ColorDrawable(color)
)
android.util.Log.d("MainActivity", "WebView transparent = $transparent")
if (transparent) startOverlayRepaint() else stopOverlayRepaint()
}
}
src-tauri/android/src/main/java/com/dtourolle/jellytau/VideoOverlayManager.kt
+8 -5
View File
@@ -1,7 +1,7 @@
package com.dtourolle.jellytau
import android.app.Activity
import android.view.SurfaceView
import android.view.TextureView
import android.view.ViewGroup
import android.widget.FrameLayout
import com.dtourolle.jellytau.player.JellyTauPlayer
@@ -14,15 +14,18 @@ import com.dtourolle.jellytau.player.JellyTauPlayer
*/
object VideoOverlayManager {
private var attachedSurfaceView: SurfaceView? = null
private var attachedSurfaceView: TextureView? = null
private var contentLayoutListener: android.view.View.OnLayoutChangeListener? = null
private var listenerContentView: ViewGroup? = null
/**
* Attach the video SurfaceView to the Activity's content view.
* Attach the video view to the Activity's content view.
*
* The SurfaceView is added at index 0 (bottom of z-order) so it renders
* behind the Tauri WebView, allowing Svelte controls to overlay on top.
* Added at index 0 (bottom of the z-order) so it renders behind the Tauri
* WebView, allowing the Svelte controls to overlay on top. Since DR-192 this
* is a TextureView, so "behind" is ordinary view z-order within one window
* rather than a separate surface punched through it — which is what makes
* the overlay above it repaint reliably.
*
* @param activity The Activity to attach the surface to
*/
src-tauri/android/src/main/java/com/dtourolle/jellytau/player/JellyTauPlayer.kt
+83 -43
View File
@@ -8,8 +8,7 @@ import android.net.Uri
import android.os.Build
import android.os.Handler
import android.os.Looper
import android.view.SurfaceHolder
import android.view.SurfaceView
import android.view.TextureView
import android.view.ViewGroup
import android.widget.FrameLayout
import androidx.annotation.OptIn
@@ -225,9 +224,8 @@ class JellyTauPlayer(private val appContext: Context) {
/** Media type enum */
enum class MediaType { AUDIO, VIDEO }
/** SurfaceView for video playback */
private var surfaceView: SurfaceView? = null
private var surfaceHolder: SurfaceHolder? = null
/** TextureView for video playback — see getOrCreateSurfaceView() for why. */
private var videoView: TextureView? = null
/** Last reported video frame size, used to fit the surface to the screen preserving aspect ratio */
private var videoWidth: Int = 0
private var videoHeight: Int = 0
@@ -1078,52 +1076,66 @@ class JellyTauPlayer(private val appContext: Context) {
}
/**
* Get or create the SurfaceView for video playback.
* Returns the view ID that can be attached to the view hierarchy.
* Get or create the video view, and hand it to ExoPlayer.
*
* Note: The surface is created but not automatically attached to the view hierarchy.
* Call attachSurfaceToActivity() or use VideoOverlayManager to attach it.
* This is a **TextureView**, not a SurfaceView, and that is the whole point.
*
* A SurfaceView renders on its own layer *outside* the app window and punches
* a transparent hole through the window to show it. Anything drawn above
* that hole — for us, the entire Svelte UI in a transparent WebView — is at
* the mercy of that composition path, and Android's own graphics
* documentation says plainly that "overlays do not currently work correctly
* with SurfaceView or TextureView". On device that showed up as the WebView
* overlay silently dropping its incremental damage: the clock text stopped
* advancing on screen while the DOM kept updating (slider 476 → 479 across
* three seconds behind a display showing neither), the control bar would not
* fade, and rotation lost the transport UI. Only *structural* DOM changes
* got through, which is why the play overlay — an `{#if}` block that is added
* and removed — always appeared to work while the progress bar never did.
*
* A TextureView is an ordinary view: its frames are drawn as a texture inside
* the window's normal rendering pass, so there is no second layer, no
* transparent region, and the WebView above composites like it would over any
* other view. This is the standard remedy for ExoPlayer overlay problems and
* is why media3 offers `surface_type="texture_view"` at all.
*
* The cost is real and accepted: TextureView uses more power and memory than
* SurfaceView and adds a frame of latency. Hardware decode through MediaCodec
* is unaffected — only presentation changes — so the reason native video
* exists survives the trade.
*
* `setVideoTextureView` installs ExoPlayer's own `SurfaceTextureListener`, so
* there is deliberately no listener of ours here; adding one would displace
* it and the video would never appear.
*
* Note: the view is created but not attached to the hierarchy. Call
* attachSurfaceToActivity() or use VideoOverlayManager to attach it.
*
* TRACES: UR-003, UR-004 | DR-192
*/
fun getOrCreateSurfaceView(): Int {
if (surfaceView == null) {
surfaceView = SurfaceView(appContext).apply {
if (videoView == null) {
videoView = TextureView(appContext).apply {
layoutParams = FrameLayout.LayoutParams(
ViewGroup.LayoutParams.MATCH_PARENT,
ViewGroup.LayoutParams.MATCH_PARENT
)
// Render BEHIND WebView - video shows through transparent areas
setZOrderMediaOverlay(false)
// Set up SurfaceHolder callbacks
holder.addCallback(object : SurfaceHolder.Callback {
override fun surfaceCreated(holder: SurfaceHolder) {
android.util.Log.d("JellyTauPlayer", "Surface created")
surfaceHolder = holder
exoPlayer.setVideoSurfaceHolder(holder)
android.util.Log.d("JellyTauPlayer", "Video surface attached to ExoPlayer")
}
override fun surfaceChanged(holder: SurfaceHolder, format: Int, width: Int, height: Int) {
android.util.Log.d("JellyTauPlayer", "Surface changed: ${width}x${height}")
}
override fun surfaceDestroyed(holder: SurfaceHolder) {
android.util.Log.d("JellyTauPlayer", "Surface destroyed")
exoPlayer.clearVideoSurfaceHolder(holder)
surfaceHolder = null
}
})
// The view is opaque where video is drawn; the WebView above it
// is what supplies transparency, exactly as before.
isOpaque = true
}
exoPlayer.setVideoTextureView(videoView)
android.util.Log.d("JellyTauPlayer", "Video TextureView created and attached to ExoPlayer")
}
return surfaceView!!.hashCode()
return videoView!!.hashCode()
}
/**
* Get the SurfaceView instance (for VideoOverlayManager).
* Returns null if no surface has been created yet.
* Get the video view instance (for VideoOverlayManager).
* Returns null if none has been created yet.
*/
fun getSurfaceView(): SurfaceView? {
return surfaceView
fun getSurfaceView(): TextureView? {
return videoView
}
/**
@@ -1138,7 +1150,7 @@ class JellyTauPlayer(private val appContext: Context) {
* This should be called from MainActivity when video playback is active.
*/
fun attachSurfaceToActivity(activity: android.app.Activity) {
if (surfaceView != null && currentMediaType == MediaType.VIDEO) {
if (videoView != null && currentMediaType == MediaType.VIDEO) {
com.dtourolle.jellytau.VideoOverlayManager.attachVideoSurface(activity)
android.util.Log.d("JellyTauPlayer", "Surface attached to Activity")
}
@@ -1184,7 +1196,7 @@ class JellyTauPlayer(private val appContext: Context) {
*/
fun fitSurfaceToScreen() {
mainHandler.post {
val view = surfaceView ?: return@post
val view = videoView ?: return@post
val parent = view.parent as? ViewGroup
// Available area: prefer the parent's measured size, fall back to the screen.
val availW = parent?.width?.takeIf { it > 0 }
@@ -1216,10 +1228,39 @@ class JellyTauPlayer(private val appContext: Context) {
if (lp is FrameLayout.LayoutParams) {
lp.gravity = android.view.Gravity.CENTER
}
// Hide the view across a resize, and reveal it once the new bounds
// hold a freshly drawn frame.
//
// A TextureView retains its last frame. Between a rotation and this
// re-fit landing, that retained frame is stretched across the OLD
// rect — which is larger than the new one along at least one axis —
// so the previous frame flashes in what should be the letterbox
// bars. Nothing is wrong with the video; it is one or two frames of
// stale texture at a stale size.
//
// Two `postOnAnimation` hops rather than one: the first runs after
// layout has been applied, the second after a frame has actually
// been drawn into the new bounds, which is the thing worth waiting
// for. Scoped to an actual size change so steady-state playback
// never touches alpha.
//
// TRACES: UR-003, UR-066 | DR-194
val sizeChanged = lp.width != targetW || lp.height != targetH
if (sizeChanged) {
view.alpha = 0f
}
lp.width = targetW
lp.height = targetH
view.layoutParams = lp
view.requestLayout()
if (sizeChanged) {
view.postOnAnimation {
view.postOnAnimation { view.alpha = 1f }
}
}
android.util.Log.d(
"JellyTauPlayer",
"Video surface fitted to ${targetW}x${targetH} (video ${videoWidth}x${videoHeight}, avail ${availW}x${availH})"
@@ -1241,11 +1282,10 @@ class JellyTauPlayer(private val appContext: Context) {
* TRACES: UR-003, UR-041 | DR-184
*/
private fun clearVideoSurface() {
surfaceView?.let {
videoView?.let {
exoPlayer.clearVideoSurface()
com.dtourolle.jellytau.VideoOverlayManager.detachVideoSurface()
surfaceView = null
surfaceHolder = null
videoView = null
android.util.Log.d("JellyTauPlayer", "Video surface cleared and detached")
}
}