fix(player): make transport reach the player that is actually rendering
Play/pause did nothing on the Android native video path — from the on-screen tap, from the control bar, and from a direct player_toggle invocation — while seek and skip kept working. That asymmetry was the whole clue: seek decides in player_seek_video, transport decides in toggle_playback. DR-195 is the cause. `html5_playing` is Rust's record of "a webview <video> is active and in this state", and toggle_playback/play/pause all route transport to that element whenever it is set. The player route mirrored element state into it UNCONDITIONALLY — from handleReportStart and, fatally, from handleReportProgress, which VideoPlayer calls on a 10-second interval. So on the native path the frontend re-declared every ten seconds that an element was playing when none existed, and every transport intent was emitted into the void. It also explains the flashing: the control bar and the JRay overlay both key off isPlaying, which was being contradicted on every tick. The mirror now lives in mirrorElementStateToRust() in VideoPlayer, gated on useHtml5Element — the only place that knows whether an element renders at all. The route cannot tell the paths apart, which is exactly how it came to lie. DR-193 hands transport authority back to the native backend when an item loads into it. Necessary but insufficient alone: the progress interval put the flag straight back, which is why the first device test after it still failed. DR-192 presents native video through a TextureView instead of a SurfaceView. A SurfaceView renders on its own layer outside the app window and punches a transparent region through it, and everything drawn above that hole — here, the entire Svelte UI — depends on that composition path. The overlay dropped its incremental damage: the DOM advanced (slider 476 -> 479 across three seconds) behind a screen showing neither, so the progress bar froze, controls would not fade and rotation lost the transport UI, while structural DOM changes got through, which is why the play overlay always appeared to work. It supersedes DR-191, which forced redraws in a loop and treated the symptom. DR-194 hides the video view across a resize and reveals it two frames later. A TextureView retains its last frame, so between a rotation and the re-fit landing that frame is stretched across the old rect and the previous frame flashes in what should be the letterbox bars. Verified on device (Honor ROD2-W09, Android 16) by driving ADB and reading the live DOM over the devtools socket: surface tap pauses (position frozen across 12 seconds, overlay raised, transport flipped) and resumes; the control bar does both. UT-189 drives the real 10-second interval under fake timers — an earlier version asserted on a freshly mounted player, passed with the guard deleted, and guarded nothing. Still open, and deliberately not claimed: DR-192's effect on the overlay repaint is unverified on device, DR-194's letterbox reset is untested, and the native default (DR-188) stays off pending DR-190, the background-audio return.
This commit is contained in:
@@ -347,7 +347,11 @@ Internal architecture, components, and application logic.
|
||||
| DR-186 | The play overlay comes down when the backend plays. `isPlaying` was assigned once from the `player_play_item` response and thereafter only by the `player://state-changed` listener — a channel the backend never emits, the same dead wire that DR-182's first fix was mistakenly hung on. On the native path the flag therefore froze at whatever the initial response said: with ExoPlayer playing, the UI still believed it was paused, so the `bg-black/30` play-button overlay stayed raised across the whole video area and the transport button kept showing ▶. The video was simultaneously dimmed and covered while it played, which reads as "the overlay never goes away" and is easily mistaken for a second compositing fault. The mirror reads the same `player` store `playerEvents.ts` feeds, which is what the architecture already says is authoritative — the player reports state, the UI consumes it — and is gated to the native path so HTML5 keeps its element-event wiring, which is authoritative there | UI | UR-003, UR-005 | Done |
|
||||
| DR-187 | The system bars go away with the player, not only with the fullscreen button. `enterImmersive()` had exactly one caller, `toggleFullscreen()`, so opening the player left the status and navigation bars painted over it until the user pressed a button most never press. On the native path this is worse than cosmetic: the SurfaceView fills the content view, so the bars sit directly on top of the video. The player is a full-screen surface by construction — `fixed inset-0 z-50` over a `MATCH_PARENT` surface — so entry is the right moment. Called synchronously in `onMount` before any `await`, per the native-mode pitfall, and paired with the `exitImmersive()` already unconditional in `onDestroy`, so a player torn down while immersive cannot leave the rest of the app without bars | UI | UR-066, UR-003 | Done |
|
||||
| DR-188 | Native Android video is **ready to be the default except for the background-audio handoff**, and the flip therefore waits. The picture defects behind DR-172 are all found, fixed and device-verified — DR-185 (the app shell painted over the surface through a CSS rule targeting an attribute nothing set), DR-182 (nothing could lift the poster card on a path with no `<video>` element), DR-183 (the JS bridges raced the page load, so `setTransparent(true)` could never arrive), DR-184 (the SurfaceView was never detached), plus DR-186 and DR-187, the two UI defects only this path could reveal. On a device logcat now carries `WebView transparent = true` and `Marking media ready` with video on screen, which is the pair DR-172 went looking for and could not find, and skip, seek and rotation were exercised by hand. Turning the default on then surfaced a *different* unverified sub-path: returning from background audio is HTML5-only (DR-190), so on the native path playback simply stays dead. Shipping it would have repeated DR-161 exactly — a verified sub-path made default over an unverified one — so the default stays off and the flip is gated on DR-190 rather than on more confidence | UI | UR-003, UR-004, UR-041 | Blocked by DR-190 |
|
||||
| DR-191 | The WebView overlay keeps repainting over the native video surface. With the ExoPlayer SurfaceView composited *under* a transparent WebView, the WebView's ordinary damage stopped reaching the screen: the page went on mutating — the clock text every second, the control bar's opacity going to 0 — while the display kept showing whatever frame the overlay last presented, over video that animated perfectly. It reads as three separate bugs (a frozen progress bar, controls that will not fade, overlays that linger) and is one. It is not a state defect: reading the live DOM over the devtools socket showed the slider advancing 476 → 479 across three seconds while the screen showed neither value. **Structural** changes do get through — injecting a single element made the whole overlay catch up at once, jumping the displayed clock from 6:39 to 19:35 — which is also why the play overlay always appeared to work: it lives in an `{#if}` block and is added and removed from the DOM, while the progress bar only changes text and the control bar only changes a class. A CSS animation does not help, because opacity animates on the compositor without repainting the layer. The redraw is therefore driven from the Activity, via `postInvalidateOnAnimation` so it rides vsync rather than outpacing the frames it asks for, started and stopped with the compositing itself so the cost belongs to native video playback, which is already decoding. This is a workaround for platform compositing behaviour rather than a fix for a defect of ours, and is deliberately narrow and self-cancelling | Android | UR-003, UR-004 | Done |
|
||||
| DR-191 | Forcing the WebView overlay to redraw from the Activity, because with the ExoPlayer **SurfaceView** beneath it the overlay's ordinary damage stopped reaching the screen: the page kept mutating — the clock text every second, the control bar's opacity going to 0 — while the display held whatever frame it last presented, over video that animated perfectly. Not a state defect; the live DOM showed the slider advancing 476 → 479 across three seconds behind a screen showing neither. Only **structural** changes got through, which is why the play overlay always appeared to work (an `{#if}` block, added and removed) while the progress bar never did, and why rotation lost the transport UI. A CSS animation cannot help, since opacity animates on the compositor without repainting the layer. **Superseded by DR-192**: this drove `postInvalidateOnAnimation` in a loop, which treats the symptom — the cause is the SurfaceView's separate layer, and removing that removes the need. Kept as the record of how the mechanism was identified | Android | UR-003, UR-004 | Superseded by DR-192 |
|
||||
| DR-195 | Play/pause works on the native path, because the frontend stops claiming a webview element is playing when there is none. `html5_playing` is Rust's record of "a webview `<video>` is active and in this state", and `toggle_playback`, `play` and `pause` all route transport to that element whenever it is set. The player route mirrored element state into it **unconditionally** — from `handleReportStart` and, fatally, from `handleReportProgress`, which VideoPlayer calls on a 10-second interval — so on the native path the frontend re-declared every ten seconds that an element was playing when none existed, and every transport intent was emitted into the void. The pause button was dead from the on-screen tap, from the control bar, and from a direct `player_toggle` invocation, while seek and skip kept working because `player_seek_video` decides elsewhere; that asymmetry is the signature. It also explains the flashing, since the control bar and the JRay overlay both key off `isPlaying`, which was being contradicted on every interval tick. DR-193 clearing the flag at load was necessary but insufficient on its own — the interval put it straight back. The mirror now lives in `mirrorElementStateToRust` in VideoPlayer, gated on `useHtml5Element`, which is the only place that knows whether an element renders at all; the route cannot tell the two paths apart, which is precisely how it came to lie. Confirmed on device by ADB: surface tap and control bar each pause (position frozen across repeated samples, transport label flipped) and resume | Playback | UR-005, UR-003 | Done |
|
||||
| DR-194 | The previous frame stops flashing in the letterbox bars on rotation. A TextureView retains its last frame, and between a rotation and `fitSurfaceToScreen()` landing that retained frame is stretched across the **old** rect — larger than the new one along at least one axis — so the previous frame appears where the bars should be. Nothing is wrong with the video: it is one or two frames of stale texture at a stale size, and it is specific to DR-192's move to a TextureView, since a SurfaceView's separate layer never showed it. The view is hidden across an actual size change and revealed after two `postOnAnimation` hops — the first lands after layout is applied, the second after a frame has been drawn into the new bounds, which is the state worth waiting for. Scoped to a real size change so steady-state playback never touches alpha. Unlike DR-191's permanent invalidation loop this is a discrete reset at a discrete event, which is the difference between a bounded workaround and a treadmill | Android | UR-003, UR-066 | Done |
|
||||
| DR-193 | Play/pause reaches the player that is actually rendering. `toggle_playback`, `play` and `pause` all route to the webview element when `is_html5_active()`, which is `html5_playing.is_some()` — a flag written **only** by the element's own state reports and cleared only when it reports "stopped"/"idle" (or on a background-audio handoff). An element that went away without that final report, or webview-rendered music earlier in the same process, therefore left the flag set, and on Android's native video path every transport intent was emitted as a `ControlCommand` at an element that no longer existed: the pause button did nothing, from the on-screen tap and from the control bar alike, while seek and skip kept working because `player_seek_video` decides elsewhere. Whether it happened at all depended on what had played before, which is exactly what made it read as flaky rather than broken. `load_and_play` — the native load path, and the one the HTML5 video path deliberately avoids via `set_current_item` — now clears the flag, because loading into the native backend *is* the statement that native renders this item. Nothing is lost on the webview path: an element re-establishes its own authority the moment it reports again, so this is the existing "element is gone" semantics applied where it can be known directly rather than inferred from a report that may never arrive | Playback | UR-005, UR-003 | Done |
|
||||
| DR-192 | Native video presents through a **TextureView**, not a SurfaceView. A SurfaceView renders on its own layer *outside* the app window and punches a transparent region through it; everything drawn above that hole — for us the entire Svelte UI in a transparent WebView — depends on that composition path, and Android's own graphics documentation states that "overlays do not currently work correctly with SurfaceView or TextureView". The consequences were four symptoms of one cause (DR-191): a frozen progress bar, controls that would not fade, rotation losing the transport UI, and overlays that lingered after the DOM removed them. A TextureView is an ordinary view whose frames are drawn as a texture in the window's normal rendering pass, so there is no second layer and no transparent region, and the WebView above composites like it would over any other view — which is why media3 offers `surface_type="texture_view"` and why it is the standard remedy for ExoPlayer overlay problems. The trade is accepted rather than hidden: TextureView costs more power and memory than SurfaceView and adds a frame of latency, but hardware decode through MediaCodec is untouched, so the reason native video exists survives it. `setVideoTextureView` installs ExoPlayer's own `SurfaceTextureListener`, so the old `SurfaceHolder.Callback` wiring is deleted rather than ported — adding a listener of ours would displace it and the video would never appear. PiP needs no change, since a TextureView is a View and the aspect-ratio probe reads its measured bounds | Android | UR-003, UR-004, UR-041 | Done |
|
||||
| DR-190 | The background-audio handoff can return to the native path. Everything that restores playback on the way back is written around the WebView `<video>`: `applyPendingForegroundSeek` returns early on `!videoElement`, the HLS re-init `$effect` returns early on `!useHtml5Element`, and `pendingForegroundSeek`/`pendingForegroundPlay` — which own the post-handoff position and play/pause — are consumed only by `handleCanPlay` and `markMediaReady`, an element event and a path that reaches the same guard. On the native path there is no element, so `exitBackgroundAudioHandoff` completes, clears `handoffState`, blanks and reassigns `currentStreamUrl` to force an effect that will not run, and nothing ever restarts ExoPlayer: the user returns from the lockscreen to a dead player. This never showed while the path was opt-in and its picture was invisible anyway. The return needs the native equivalent of the element reload — re-issue the item to the backend, seek to the position `player_exit_background_audio` reports, then honour `wasPlaying` — routed through the adapter rather than the element, so both paths restore through one contract | Playback | UR-040, UR-003 | Proposed |
|
||||
| DR-161 | Native video is the default, so picture-in-picture has a real surface. DR-160 makes PiP work on the HTML5 path, but that path can only ever shrink the *UI* into the PiP window; showing the video itself needs the SurfaceView behind the WebView, which is what `experimentalNativeVideo` gates. The flag now defaults to on when the user has never chosen, with an explicit stored choice still winning in both directions so anyone who turned it off keeps it off. This is a deliberate acceptance of risk: the flag existed because the native path was an unfinished spike, and `VideoPlayer.scrubRegression.test.ts` documents its history — a native init that flipped to HTML5 mid-lifecycle and left seeks going down one path while ExoPlayer played on another. Those tests pin the **flag-off** interim override (native response overridden to HTML5, backend stopped once), which the default no longer selects, so they now mock the flag off rather than inherit it: they still guard that path, but they no longer describe what ships. The native scrub/seek path is consequently not covered by the suite and needs device verification | UI | UR-041, UR-003 | Needs device verification |
|
||||
| DR-159 | The background-audio handoff stops leaking its relative timeline. The handoff plays the episode as a *relative* stream — the audio-only URL is built with `StartTimeTicks` = the position the screen was locked at, so ExoPlayer's zero is the handoff point — and `background_audio_base` holds the offset that turns one back into a real position. The base was a **display-only** correction, applied in exactly two places (the lockscreen scrubber and the internal truncation maths) while every other consumer worked in the relative timeline treating the number as absolute. Each crossing threw away exactly `base` seconds, which is why the jump-back distance varied with where the screen was locked and read as random. Three crossings were live: progress reporting to Jellyfin sent the relative position every 30s, so the server was told `real − base` — and since DR-155 now mirrors the server's position back and refreshes on a cache hit, that regressed value returned as the resume point (lock at 40 min, listen to 90, reopen at 50); lockscreen seeks went out absolute and came back relative, against a chunked length-less transcode that cannot honour a seek at all, so a clamped seek landed at stream zero; and media3's own `seekToDefaultPosition`/`seekBack`/`seekForward` bypassed the `ForwardingPlayer` wrapper entirely, reaching the real ExoPlayer — `Util.handlePlayButtonAction` seeking an ended player to the relative zero being the same mechanism as DR-129's truncation bug through a different door. The fix converts **once, at the boundary**: `JellyTauPlayer`'s position tick adds the base (and shifts the duration with it, since the stream's own length is only what remains) before either `nativeOnPositionUpdate` or the lockscreen sees it, so position updates, progress reports, the frontend and the truncation check all speak the episode's timeline and none needs to know a handoff happened. The base is consequently *removed* from `claim_stream_resume`, `truncated_stream_resume_position` and `player_exit_background_audio`, where adding it now double-counts, and the lockscreen's `positionOffsetMs` addition goes with it (the field remains, read-only, as the tick's input). Inbound seeks go the other way: `seek_absolute` is the new boundary for every outside seek, re-opening the stream at the requested position via `resume_stream_at` when a handoff is active — which is what `onSeekTo` had claimed for months in a comment describing code that did not exist — and an ordinary seek otherwise. `seekToDefaultPosition` is swallowed rather than forwarded, since Rust already owns what "play after the stream ended" means and the `play()` that follows reaches it. Exit reads the position *before* clearing either base, or a tick landing in between hands back a relative one | Player | UR-040, UR-005, UR-025 | Done (pending device verification) |
|
||||
@@ -370,9 +374,9 @@ Internal architecture, components, and application logic.
|
||||
|----------|-------------------------|-------------------------|
|
||||
| UR-001 | IR-001, IR-002 | - |
|
||||
| UR-002 | IR-013 | DR-003, DR-012, DR-013, DR-014 |
|
||||
| UR-003 | IR-003, IR-004, IR-011 | DR-002, DR-004, DR-010, DR-182, DR-183, DR-184, DR-185, DR-186, DR-187, DR-188, DR-190, DR-191 |
|
||||
| UR-003 | IR-003, IR-004, IR-011 | DR-002, DR-004, DR-010, DR-182, DR-183, DR-184, DR-185, DR-186, DR-187, DR-188, DR-190, DR-191, DR-192, DR-193, DR-194, DR-195 |
|
||||
| UR-004 | IR-003, IR-004, IR-008, IR-011 | DR-002, DR-004, DR-006, DR-129, DR-171, DR-176, DR-177, DR-181, DR-182, DR-183, DR-185, DR-188 |
|
||||
| UR-005 | - | DR-001, DR-005, DR-009, DR-178, DR-179, DR-186 |
|
||||
| UR-005 | - | DR-001, DR-005, DR-009, DR-178, DR-179, DR-186, DR-193, DR-195 |
|
||||
| UR-006 | IR-005, IR-006, IR-007, IR-008 | - |
|
||||
| UR-007 | IR-010 | DR-007, DR-008, DR-016 |
|
||||
| UR-008 | IR-010 | DR-007, DR-011 |
|
||||
@@ -432,7 +436,7 @@ Internal architecture, components, and application logic.
|
||||
| UR-063 | - | DR-105 |
|
||||
| UR-064 | - | DR-106 |
|
||||
| UR-065 | IR-030 | DR-108, DR-109, DR-110, DR-111 |
|
||||
| UR-066 | IR-031 | DR-112, DR-157, DR-187 |
|
||||
| UR-066 | IR-031 | DR-112, DR-157, DR-187, DR-194 |
|
||||
| UR-067 | - | DR-115, DR-116, DR-117, DR-118 |
|
||||
| UR-068 | - | DR-119 |
|
||||
| UR-069 | - | DR-113, DR-114, DR-120 |
|
||||
@@ -629,6 +633,7 @@ Internal architecture, components, and application logic.
|
||||
| UT-182 | An HLS video URL never carries `StartTimeTicks` — with a position supplied or not — while the master playlist, codec, media source and chosen audio track still ride on it | DR-181 | Done |
|
||||
| UT-183 | A reloaded stream is resumed by seeking the element to the absolute position with the transcode offset cleared to zero — never by carrying the position as an offset base, which since DR-181 would display the position while playing the item from its start — and a reload to 0:00 waits for no seek | DR-181 | Done |
|
||||
| UT-184 | The native reveal rule fires on `state === "playing"` and on a position tick carrying a position or a duration, and on nothing else — not `buffering`, `paused`, `stopped`, `ended` or `error`, not an empty tick, and not a negative position | DR-182 | Done |
|
||||
| UT-189 | On the native path the player never calls `player_report_state` — driven through the real 10-second progress interval under fake timers, which is the call site that mattered; asserting on a freshly mounted player passes with the guard deleted and guards nothing | DR-195 | Done |
|
||||
| UT-187 | On the native path the play overlay follows the backend: it clears when the backend resumes after a pause and is raised again when the backend pauses, and the system bars are hidden on player entry rather than only by the fullscreen button | DR-186, DR-187 | Done |
|
||||
| UT-186 | Every attribute the native-video compositing block in app.css targets is set somewhere in the app — `[data-app-shell]` in particular — so a selector aimed at nothing fails the suite instead of failing silently on a device | DR-185 | Done |
|
||||
| UT-185 | Mounted on the native path (backend reports native, opt-in flag on, no `<video>` element rendered and the backend not stopped), VideoPlayer keeps the poster card up until the backend reports something, drops it on a playing state or a position tick with a duration, and keeps it up through `error` and `stopped` | DR-182 | Done |
|
||||
|
||||
+1742
-1610
File diff suppressed because it is too large
Load Diff
@@ -175,8 +175,8 @@ describe("live requirements.md", () => {
|
||||
|
||||
expect(defined.UR).toBe(75);
|
||||
expect(defined.IR).toBe(32);
|
||||
expect(defined.DR).toBe(181);
|
||||
expect(defined.DR).toBe(185);
|
||||
expect(defined.JA).toBe(35);
|
||||
expect(defined.total).toBe(323);
|
||||
expect(defined.total).toBe(327);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -10,12 +10,6 @@ import android.webkit.WebView
|
||||
import android.view.View
|
||||
import androidx.activity.enableEdgeToEdge
|
||||
|
||||
/**
|
||||
* How often to ask the WebView overlay to redraw while a native video surface is
|
||||
* behind it. Roughly one display frame — `postInvalidateOnAnimation` coalesces
|
||||
* to vsync, so this only governs how often we *ask*. See [MainActivity.overlayRepaint].
|
||||
*/
|
||||
private const val OVERLAY_REPAINT_MS = 16L
|
||||
|
||||
class MainActivity : TauriActivity() {
|
||||
private val handler = Handler(Looper.getMainLooper())
|
||||
@@ -95,60 +89,6 @@ class MainActivity : TauriActivity() {
|
||||
configureWebViewSettings(webView)
|
||||
}
|
||||
|
||||
/**
|
||||
* Keeps the WebView overlay repainting while a native video surface is behind
|
||||
* it. Null when not running.
|
||||
*
|
||||
* With the ExoPlayer SurfaceView composited *under* a transparent WebView, the
|
||||
* WebView's ordinary damage stops reaching the screen: the page kept mutating
|
||||
* — the clock text changing every second, the control bar's opacity going to
|
||||
* 0 — and none of it appeared, while the video underneath animated fine. The
|
||||
* overlay froze on whatever frame it last managed to present, which is why the
|
||||
* progress bar "stayed there and did not update" and the controls would not
|
||||
* fade. It is not a state bug: reading the DOM over the devtools socket showed
|
||||
* the slider value advancing (476 → 479 over three seconds) behind a screen
|
||||
* showing neither.
|
||||
*
|
||||
* A *structural* change did force it through — injecting one element made the
|
||||
* whole overlay catch up at once, jumping the displayed clock from 6:39 to
|
||||
* 19:35 — so the pixels are reachable; it is the incremental damage that gets
|
||||
* dropped. A CSS animation does not do it: opacity animates on the compositor
|
||||
* without repainting the layer, which is exactly why that attempt changed
|
||||
* nothing.
|
||||
*
|
||||
* So the redraw is driven from here instead. `postInvalidateOnAnimation`
|
||||
* rather than a fixed-rate timer, so it rides the display's vsync and cannot
|
||||
* outpace the frames it is asking for, and it runs *only* while compositing is
|
||||
* on — the cost belongs to native video playback, which is already decoding.
|
||||
*
|
||||
* This is a workaround for platform compositing behaviour, not a fix for a
|
||||
* defect of ours; the honest form of it is narrow and self-cancelling.
|
||||
*
|
||||
* TRACES: UR-003, UR-004 | DR-191
|
||||
*/
|
||||
private var overlayRepaint: Runnable? = null
|
||||
|
||||
private fun startOverlayRepaint() {
|
||||
if (overlayRepaint != null) return
|
||||
val tick = object : Runnable {
|
||||
override fun run() {
|
||||
val webView = mediaWebView ?: return
|
||||
webView.postInvalidateOnAnimation()
|
||||
// Re-post through the same field so stopOverlayRepaint() can cancel it.
|
||||
overlayRepaint?.let { handler.postDelayed(it, OVERLAY_REPAINT_MS) }
|
||||
}
|
||||
}
|
||||
overlayRepaint = tick
|
||||
handler.post(tick)
|
||||
android.util.Log.d("MainActivity", "Overlay repaint started")
|
||||
}
|
||||
|
||||
private fun stopOverlayRepaint() {
|
||||
overlayRepaint?.let { handler.removeCallbacks(it) }
|
||||
overlayRepaint = null
|
||||
android.util.Log.d("MainActivity", "Overlay repaint stopped")
|
||||
}
|
||||
|
||||
override fun onCreate(savedInstanceState: Bundle?) {
|
||||
enableEdgeToEdge()
|
||||
super.onCreate(savedInstanceState)
|
||||
@@ -442,7 +382,6 @@ class MainActivity : TauriActivity() {
|
||||
android.graphics.drawable.ColorDrawable(color)
|
||||
)
|
||||
android.util.Log.d("MainActivity", "WebView transparent = $transparent")
|
||||
if (transparent) startOverlayRepaint() else stopOverlayRepaint()
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
package com.dtourolle.jellytau
|
||||
|
||||
import android.app.Activity
|
||||
import android.view.SurfaceView
|
||||
import android.view.TextureView
|
||||
import android.view.ViewGroup
|
||||
import android.widget.FrameLayout
|
||||
import com.dtourolle.jellytau.player.JellyTauPlayer
|
||||
@@ -14,15 +14,18 @@ import com.dtourolle.jellytau.player.JellyTauPlayer
|
||||
*/
|
||||
object VideoOverlayManager {
|
||||
|
||||
private var attachedSurfaceView: SurfaceView? = null
|
||||
private var attachedSurfaceView: TextureView? = null
|
||||
private var contentLayoutListener: android.view.View.OnLayoutChangeListener? = null
|
||||
private var listenerContentView: ViewGroup? = null
|
||||
|
||||
/**
|
||||
* Attach the video SurfaceView to the Activity's content view.
|
||||
* Attach the video view to the Activity's content view.
|
||||
*
|
||||
* The SurfaceView is added at index 0 (bottom of z-order) so it renders
|
||||
* behind the Tauri WebView, allowing Svelte controls to overlay on top.
|
||||
* Added at index 0 (bottom of the z-order) so it renders behind the Tauri
|
||||
* WebView, allowing the Svelte controls to overlay on top. Since DR-192 this
|
||||
* is a TextureView, so "behind" is ordinary view z-order within one window
|
||||
* rather than a separate surface punched through it — which is what makes
|
||||
* the overlay above it repaint reliably.
|
||||
*
|
||||
* @param activity The Activity to attach the surface to
|
||||
*/
|
||||
|
||||
@@ -8,8 +8,7 @@ import android.net.Uri
|
||||
import android.os.Build
|
||||
import android.os.Handler
|
||||
import android.os.Looper
|
||||
import android.view.SurfaceHolder
|
||||
import android.view.SurfaceView
|
||||
import android.view.TextureView
|
||||
import android.view.ViewGroup
|
||||
import android.widget.FrameLayout
|
||||
import androidx.annotation.OptIn
|
||||
@@ -225,9 +224,8 @@ class JellyTauPlayer(private val appContext: Context) {
|
||||
/** Media type enum */
|
||||
enum class MediaType { AUDIO, VIDEO }
|
||||
|
||||
/** SurfaceView for video playback */
|
||||
private var surfaceView: SurfaceView? = null
|
||||
private var surfaceHolder: SurfaceHolder? = null
|
||||
/** TextureView for video playback — see getOrCreateSurfaceView() for why. */
|
||||
private var videoView: TextureView? = null
|
||||
/** Last reported video frame size, used to fit the surface to the screen preserving aspect ratio */
|
||||
private var videoWidth: Int = 0
|
||||
private var videoHeight: Int = 0
|
||||
@@ -1078,52 +1076,66 @@ class JellyTauPlayer(private val appContext: Context) {
|
||||
}
|
||||
|
||||
/**
|
||||
* Get or create the SurfaceView for video playback.
|
||||
* Returns the view ID that can be attached to the view hierarchy.
|
||||
* Get or create the video view, and hand it to ExoPlayer.
|
||||
*
|
||||
* Note: The surface is created but not automatically attached to the view hierarchy.
|
||||
* Call attachSurfaceToActivity() or use VideoOverlayManager to attach it.
|
||||
* This is a **TextureView**, not a SurfaceView, and that is the whole point.
|
||||
*
|
||||
* A SurfaceView renders on its own layer *outside* the app window and punches
|
||||
* a transparent hole through the window to show it. Anything drawn above
|
||||
* that hole — for us, the entire Svelte UI in a transparent WebView — is at
|
||||
* the mercy of that composition path, and Android's own graphics
|
||||
* documentation says plainly that "overlays do not currently work correctly
|
||||
* with SurfaceView or TextureView". On device that showed up as the WebView
|
||||
* overlay silently dropping its incremental damage: the clock text stopped
|
||||
* advancing on screen while the DOM kept updating (slider 476 → 479 across
|
||||
* three seconds behind a display showing neither), the control bar would not
|
||||
* fade, and rotation lost the transport UI. Only *structural* DOM changes
|
||||
* got through, which is why the play overlay — an `{#if}` block that is added
|
||||
* and removed — always appeared to work while the progress bar never did.
|
||||
*
|
||||
* A TextureView is an ordinary view: its frames are drawn as a texture inside
|
||||
* the window's normal rendering pass, so there is no second layer, no
|
||||
* transparent region, and the WebView above composites like it would over any
|
||||
* other view. This is the standard remedy for ExoPlayer overlay problems and
|
||||
* is why media3 offers `surface_type="texture_view"` at all.
|
||||
*
|
||||
* The cost is real and accepted: TextureView uses more power and memory than
|
||||
* SurfaceView and adds a frame of latency. Hardware decode through MediaCodec
|
||||
* is unaffected — only presentation changes — so the reason native video
|
||||
* exists survives the trade.
|
||||
*
|
||||
* `setVideoTextureView` installs ExoPlayer's own `SurfaceTextureListener`, so
|
||||
* there is deliberately no listener of ours here; adding one would displace
|
||||
* it and the video would never appear.
|
||||
*
|
||||
* Note: the view is created but not attached to the hierarchy. Call
|
||||
* attachSurfaceToActivity() or use VideoOverlayManager to attach it.
|
||||
*
|
||||
* TRACES: UR-003, UR-004 | DR-192
|
||||
*/
|
||||
fun getOrCreateSurfaceView(): Int {
|
||||
if (surfaceView == null) {
|
||||
surfaceView = SurfaceView(appContext).apply {
|
||||
if (videoView == null) {
|
||||
videoView = TextureView(appContext).apply {
|
||||
layoutParams = FrameLayout.LayoutParams(
|
||||
ViewGroup.LayoutParams.MATCH_PARENT,
|
||||
ViewGroup.LayoutParams.MATCH_PARENT
|
||||
)
|
||||
// Render BEHIND WebView - video shows through transparent areas
|
||||
setZOrderMediaOverlay(false)
|
||||
|
||||
// Set up SurfaceHolder callbacks
|
||||
holder.addCallback(object : SurfaceHolder.Callback {
|
||||
override fun surfaceCreated(holder: SurfaceHolder) {
|
||||
android.util.Log.d("JellyTauPlayer", "Surface created")
|
||||
surfaceHolder = holder
|
||||
exoPlayer.setVideoSurfaceHolder(holder)
|
||||
android.util.Log.d("JellyTauPlayer", "Video surface attached to ExoPlayer")
|
||||
// The view is opaque where video is drawn; the WebView above it
|
||||
// is what supplies transparency, exactly as before.
|
||||
isOpaque = true
|
||||
}
|
||||
|
||||
override fun surfaceChanged(holder: SurfaceHolder, format: Int, width: Int, height: Int) {
|
||||
android.util.Log.d("JellyTauPlayer", "Surface changed: ${width}x${height}")
|
||||
exoPlayer.setVideoTextureView(videoView)
|
||||
android.util.Log.d("JellyTauPlayer", "Video TextureView created and attached to ExoPlayer")
|
||||
}
|
||||
|
||||
override fun surfaceDestroyed(holder: SurfaceHolder) {
|
||||
android.util.Log.d("JellyTauPlayer", "Surface destroyed")
|
||||
exoPlayer.clearVideoSurfaceHolder(holder)
|
||||
surfaceHolder = null
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
return surfaceView!!.hashCode()
|
||||
return videoView!!.hashCode()
|
||||
}
|
||||
|
||||
/**
|
||||
* Get the SurfaceView instance (for VideoOverlayManager).
|
||||
* Returns null if no surface has been created yet.
|
||||
* Get the video view instance (for VideoOverlayManager).
|
||||
* Returns null if none has been created yet.
|
||||
*/
|
||||
fun getSurfaceView(): SurfaceView? {
|
||||
return surfaceView
|
||||
fun getSurfaceView(): TextureView? {
|
||||
return videoView
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -1138,7 +1150,7 @@ class JellyTauPlayer(private val appContext: Context) {
|
||||
* This should be called from MainActivity when video playback is active.
|
||||
*/
|
||||
fun attachSurfaceToActivity(activity: android.app.Activity) {
|
||||
if (surfaceView != null && currentMediaType == MediaType.VIDEO) {
|
||||
if (videoView != null && currentMediaType == MediaType.VIDEO) {
|
||||
com.dtourolle.jellytau.VideoOverlayManager.attachVideoSurface(activity)
|
||||
android.util.Log.d("JellyTauPlayer", "Surface attached to Activity")
|
||||
}
|
||||
@@ -1184,7 +1196,7 @@ class JellyTauPlayer(private val appContext: Context) {
|
||||
*/
|
||||
fun fitSurfaceToScreen() {
|
||||
mainHandler.post {
|
||||
val view = surfaceView ?: return@post
|
||||
val view = videoView ?: return@post
|
||||
val parent = view.parent as? ViewGroup
|
||||
// Available area: prefer the parent's measured size, fall back to the screen.
|
||||
val availW = parent?.width?.takeIf { it > 0 }
|
||||
@@ -1216,10 +1228,39 @@ class JellyTauPlayer(private val appContext: Context) {
|
||||
if (lp is FrameLayout.LayoutParams) {
|
||||
lp.gravity = android.view.Gravity.CENTER
|
||||
}
|
||||
|
||||
// Hide the view across a resize, and reveal it once the new bounds
|
||||
// hold a freshly drawn frame.
|
||||
//
|
||||
// A TextureView retains its last frame. Between a rotation and this
|
||||
// re-fit landing, that retained frame is stretched across the OLD
|
||||
// rect — which is larger than the new one along at least one axis —
|
||||
// so the previous frame flashes in what should be the letterbox
|
||||
// bars. Nothing is wrong with the video; it is one or two frames of
|
||||
// stale texture at a stale size.
|
||||
//
|
||||
// Two `postOnAnimation` hops rather than one: the first runs after
|
||||
// layout has been applied, the second after a frame has actually
|
||||
// been drawn into the new bounds, which is the thing worth waiting
|
||||
// for. Scoped to an actual size change so steady-state playback
|
||||
// never touches alpha.
|
||||
//
|
||||
// TRACES: UR-003, UR-066 | DR-194
|
||||
val sizeChanged = lp.width != targetW || lp.height != targetH
|
||||
if (sizeChanged) {
|
||||
view.alpha = 0f
|
||||
}
|
||||
|
||||
lp.width = targetW
|
||||
lp.height = targetH
|
||||
view.layoutParams = lp
|
||||
view.requestLayout()
|
||||
|
||||
if (sizeChanged) {
|
||||
view.postOnAnimation {
|
||||
view.postOnAnimation { view.alpha = 1f }
|
||||
}
|
||||
}
|
||||
android.util.Log.d(
|
||||
"JellyTauPlayer",
|
||||
"Video surface fitted to ${targetW}x${targetH} (video ${videoWidth}x${videoHeight}, avail ${availW}x${availH})"
|
||||
@@ -1241,11 +1282,10 @@ class JellyTauPlayer(private val appContext: Context) {
|
||||
* TRACES: UR-003, UR-041 | DR-184
|
||||
*/
|
||||
private fun clearVideoSurface() {
|
||||
surfaceView?.let {
|
||||
videoView?.let {
|
||||
exoPlayer.clearVideoSurface()
|
||||
com.dtourolle.jellytau.VideoOverlayManager.detachVideoSurface()
|
||||
surfaceView = null
|
||||
surfaceHolder = null
|
||||
videoView = null
|
||||
android.util.Log.d("JellyTauPlayer", "Video surface cleared and detached")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -506,6 +506,29 @@ impl PlayerController {
|
||||
// Set end reason to NewTrackLoaded to prevent autoplay when MPV ends current track
|
||||
self.set_end_reason(EndReason::NewTrackLoaded);
|
||||
|
||||
// Loading into the native backend IS the statement that native renders
|
||||
// this item, so transport authority returns to it.
|
||||
//
|
||||
// `html5_playing` is written only by the webview element's own reports
|
||||
// and cleared only when it reports "stopped"/"idle". An element that
|
||||
// went away without that final report — or webview-rendered music
|
||||
// earlier in the same process — left `is_html5_active()` true, and then
|
||||
// every play/pause intent was emitted as a ControlCommand at an element
|
||||
// that no longer existed instead of reaching the backend. On Android's
|
||||
// native video path that is a pause button that does nothing, from the
|
||||
// surface tap and the control bar alike, while seek and skip keep
|
||||
// working because they decide elsewhere. Whether it happened at all
|
||||
// depended on what had played before, which is what made it look
|
||||
// intermittent.
|
||||
//
|
||||
// The webview re-establishes its own authority the moment an element
|
||||
// reports again, so nothing is lost on the HTML5 path: this is the same
|
||||
// "element is gone" semantics as the "stopped"/"idle" report, applied at
|
||||
// the point where we can know it directly.
|
||||
//
|
||||
// TRACES: UR-005, UR-003 | DR-193
|
||||
*self.html5_playing.lock_safe() = None;
|
||||
|
||||
let mut backend = self.backend.lock_safe();
|
||||
backend.load(item)?;
|
||||
backend.play()?;
|
||||
@@ -3200,6 +3223,59 @@ mod tests {
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_native_load_returns_transport_authority_to_the_backend() {
|
||||
// Play/pause did nothing on the Android native video path, from the
|
||||
// on-screen tap AND from the control-bar button, while seek and skip
|
||||
// worked — those take a different decision path.
|
||||
//
|
||||
// `html5_playing` is written only by the webview element's own reports
|
||||
// and cleared only when it reports "stopped"/"idle" (or on a
|
||||
// background-audio handoff). A previous element that went away without
|
||||
// that final report — or webview-rendered music earlier in the same
|
||||
// process — therefore left `is_html5_active()` true, and every transport
|
||||
// intent was emitted as a ControlCommand at an element that no longer
|
||||
// existed. Nothing reached ExoPlayer. It looked intermittent because it
|
||||
// depends entirely on what played before.
|
||||
//
|
||||
// Loading into the native backend IS the statement that native renders
|
||||
// this item, so it hands authority back — the same "element is gone"
|
||||
// semantics the "stopped"/"idle" report already has.
|
||||
//
|
||||
// TRACES: UR-005, UR-003 | DR-193
|
||||
let controller = PlayerController::default();
|
||||
let emitter = Arc::new(CapturingEmitter::new());
|
||||
controller.set_event_emitter(emitter.clone());
|
||||
|
||||
// A webview element reported itself playing and never said "stopped".
|
||||
controller.report_html5_state("playing".to_string(), Some("ep-1".to_string()));
|
||||
assert!(controller.is_html5_active());
|
||||
|
||||
// Now a native item loads — Android video through ExoPlayer.
|
||||
let item = create_test_items(1).into_iter().next().unwrap();
|
||||
controller.play_item(item).unwrap();
|
||||
|
||||
assert!(
|
||||
!controller.is_html5_active(),
|
||||
"loading into the native backend hands transport back to it"
|
||||
);
|
||||
|
||||
// The toggle must reach the backend, not be emitted at a dead element.
|
||||
controller.toggle_playback().unwrap();
|
||||
let controls: Vec<_> = emitter
|
||||
.events()
|
||||
.into_iter()
|
||||
.filter_map(|e| match e {
|
||||
PlayerStatusEvent::ControlCommand { action, .. } => Some(action),
|
||||
_ => None,
|
||||
})
|
||||
.collect();
|
||||
assert!(
|
||||
controls.is_empty(),
|
||||
"transport went to a webview element that is not rendering: {controls:?}"
|
||||
);
|
||||
}
|
||||
|
||||
// EndReason state machine tests
|
||||
#[test]
|
||||
fn test_load_and_play_sets_new_track_loaded() {
|
||||
|
||||
@@ -60,11 +60,15 @@ const playerPlayItem = vi.fn(async () => ({
|
||||
state: { kind: "playing" },
|
||||
}));
|
||||
const playerStop = vi.fn(async () => ({}));
|
||||
const playerReportState = vi.fn(async () => null);
|
||||
|
||||
vi.mock("$lib/api/bindings", () => ({
|
||||
commands: {
|
||||
playerPlayItem: (...a: any[]) => playerPlayItem(...(a as [])),
|
||||
playerStop: (...a: any[]) => playerStop(...(a as [])),
|
||||
playerReportState: (...a: any[]) => playerReportState(...(a as [])),
|
||||
playerReportPosition: vi.fn(async () => null),
|
||||
playerReportMediaLoaded: vi.fn(async () => null),
|
||||
playerSeek: vi.fn(async () => ({})),
|
||||
playerPlay: vi.fn(async () => ({})),
|
||||
playerPause: vi.fn(async () => ({})),
|
||||
@@ -294,6 +298,43 @@ describe("VideoPlayer native path reveals the video (DR-172)", () => {
|
||||
}
|
||||
});
|
||||
|
||||
it("never reports webview element state on the native path (DR-195)", async () => {
|
||||
// The report that mattered came from the 10-second progress interval, so
|
||||
// the test has to reach it: the interval needs `onReportProgress` wired and
|
||||
// `isPlaying` true, then time has to pass. Asserting on a freshly mounted
|
||||
// player proves nothing — an earlier version of this test did exactly that
|
||||
// and passed with the guard deleted.
|
||||
vi.useFakeTimers();
|
||||
try {
|
||||
const utils = render(VideoPlayer, {
|
||||
props: {
|
||||
media: makeEpisode(),
|
||||
streamUrl: "http://server/videos/ep1/master.m3u8",
|
||||
mediaSourceId: "src-1",
|
||||
needsTranscoding: false,
|
||||
onClose: vi.fn(),
|
||||
onReportProgress: vi.fn(),
|
||||
},
|
||||
});
|
||||
await vi.advanceTimersByTimeAsync(100);
|
||||
expect(utils.container.querySelector("video")).toBeNull();
|
||||
|
||||
// Backend playing, so the interval's `isPlaying` guard is satisfied.
|
||||
player.setPlaying(makeEpisode(), 5, 1440);
|
||||
await vi.advanceTimersByTimeAsync(25_000);
|
||||
|
||||
// `html5_playing` is Rust's record of "a webview element is active", and
|
||||
// `toggle_playback`/`play`/`pause` all route transport to that element
|
||||
// whenever it is set. Reporting it with no element in existence is what
|
||||
// left the pause button dead on the native path — from the surface tap,
|
||||
// the control bar, and a direct `player_toggle` invocation alike — while
|
||||
// seek and skip kept working, because they decide elsewhere.
|
||||
expect(playerReportState).not.toHaveBeenCalled();
|
||||
} finally {
|
||||
vi.useRealTimers();
|
||||
}
|
||||
});
|
||||
|
||||
it("does not clear the poster on an errored backend", async () => {
|
||||
const { container } = await mountNativePlayer();
|
||||
|
||||
|
||||
@@ -1016,6 +1016,7 @@
|
||||
progressInterval = setInterval(() => {
|
||||
if (isPlaying && !isSeeking && onReportProgress) {
|
||||
onReportProgress(currentTime, false, reportMediaId);
|
||||
mirrorElementStateToRust(false);
|
||||
}
|
||||
}, 10000);
|
||||
}
|
||||
@@ -1449,6 +1450,33 @@
|
||||
}
|
||||
});
|
||||
|
||||
/**
|
||||
* Mirror the **webview element's** play/pause and position into Rust.
|
||||
*
|
||||
* Only ever when the element is what renders. `html5_playing` is Rust's record
|
||||
* of "a webview element is active and in this state", and `toggle_playback`,
|
||||
* `play` and `pause` all route transport to that element when it is set. So
|
||||
* reporting it from the native path is not a harmless extra: it hands
|
||||
* transport authority to an element that does not exist, and every play/pause
|
||||
* intent is then emitted into the void. That is exactly what made the pause
|
||||
* button dead on the native path — from the on-screen tap, the control bar,
|
||||
* and even a direct `player_toggle` invocation — while seek and skip kept
|
||||
* working, because they decide elsewhere.
|
||||
*
|
||||
* This lived in the player route's reporting callbacks, which cannot tell the
|
||||
* two rendering paths apart and so mirrored unconditionally — including from
|
||||
* the 10-second progress interval, which is why the flag came back after
|
||||
* DR-193 cleared it at load. It belongs here, where `useHtml5Element` is
|
||||
* known.
|
||||
*
|
||||
* TRACES: UR-005, UR-003 | DR-195 | UT-189
|
||||
*/
|
||||
function mirrorElementStateToRust(paused: boolean) {
|
||||
if (!useHtml5Element) return;
|
||||
html5Adapter.reportState(paused ? "paused" : "playing", reportMediaId ?? null);
|
||||
html5Adapter.reportPosition(currentTime, duration, { force: true });
|
||||
}
|
||||
|
||||
function handlePlay() {
|
||||
isPlaying = true;
|
||||
startTimeUpdates(); // Start RAF loop for smooth time updates
|
||||
|
||||
@@ -561,11 +561,11 @@
|
||||
if (id) {
|
||||
reportPlaybackStart(id, positionSeconds, context.type, context.id);
|
||||
}
|
||||
// Mirror HTML5 <video> state into the Rust PlayerController so it is the
|
||||
// single source of truth for video playback (see html5Adapter.ts). The
|
||||
// element lives in the webview and Rust cannot observe it directly.
|
||||
html5Adapter.reportState("playing", id ?? null);
|
||||
html5Adapter.reportPosition(positionSeconds, get(playbackDuration), { force: true });
|
||||
// The element's state is mirrored into Rust by VideoPlayer, which is the
|
||||
// only place that knows whether a webview element is rendering at all.
|
||||
// Doing it here mirrored unconditionally, so on the native path it told Rust
|
||||
// a `<video>` was playing when none existed and transport was then aimed at
|
||||
// it — see mirrorElementStateToRust in VideoPlayer.svelte (DR-195).
|
||||
}
|
||||
|
||||
function handleReportProgress(positionSeconds: number, isPaused: boolean, reportId?: string) {
|
||||
@@ -573,9 +573,7 @@
|
||||
if (id) {
|
||||
reportPlaybackProgress(id, positionSeconds, isPaused);
|
||||
}
|
||||
// Feed the Rust controller the current position and play/pause state.
|
||||
html5Adapter.reportState(isPaused ? "paused" : "playing", id ?? null);
|
||||
html5Adapter.reportPosition(positionSeconds, get(playbackDuration), { force: true });
|
||||
// Element state is mirrored by VideoPlayer (DR-195) — see handleReportStart.
|
||||
}
|
||||
|
||||
function handleReportStop(positionSeconds: number, reportId?: string) {
|
||||
|
||||
Reference in New Issue
Block a user