feat(player): native video on Linux, and one contract for every player (v0.11.0)
mpv now decodes video on Linux, drawn into a framebuffer we own and blitted
into the default vbox's draw handler. Tauri's widget tree is untouched, so an
upgrade that assumes its own layout cannot invalidate this. Direct play means
the original file, hardware decoding, and no server transcode at all — where
previously every desktop video was re-encoded to h264 for the browser engine,
whatever the file actually was. Off by default: JELLYTAU_NATIVE_VIDEO=1.
That settles finding 2 of playback-backend-unification.md — "native video
cannot be composited with a Tauri webview" — by demonstration rather than
argument, on X11 and Wayland both.
Turning it on exposed nine defects, none of them mpv's. Each was the same
mistake in a different place: a capability written down as a compile-time fact
about the platform, or a state asserted instead of confirmed.
DR-238/246 a seek routed by the stream's container rather than by what the
engine could do with it - correct only while one player handled
those streams, silent the moment another did
DR-239 a property handled but never observed, so the play/pause button
waited for an event that could not arrive
DR-240 fullscreen expanding the document while the window stayed put
DR-241 a seek issued before the engine had a file, failed, and discarded
- which is why resume began at zero
DR-247 a Linux-only gate outliving the caller that made it Linux-only,
breaking the Android build outright
DR-250 a stop aimed at whichever renderer bookkeeping believed was in
charge, missing the one actually making sound
DR-251 a duration of zero believed, leaving the seek bar no scale
DR-252 a junk float converted to a Duration, panicking the backend the
instant a length-less stream appeared
So the MediaPlayer contract (DR-242 … DR-247): `open` carries a start position,
so no caller sequences load-then-seek and none can race an engine's load;
`seek` states a destination and leaves in-place-versus-re-open to the engine;
`snapshot` is one coherent read; and `Phase::Opening` names the window where
intent used to be lost. One conformance suite runs against every engine —
FakePlayer and mpv under cargo test, ExoPlayer instrumented on a device — so an
engine is either correct or visibly failing.
Two of the nine were introduced during this work and caught on hardware, not by
any suite: an over-broad capability that grouped ExoPlayer with mpv, and the
Duration panic. The suites test engines that behave. That is recorded in
docs/native-player-verification.md, which asks for the exact action sequences
that found them.
Verified: all automated gates, conformance (mpv 9/9, legacy 8/9 by design,
ExoPlayer 7/7 on device), and manual desktop and Android passes on real
hardware.
Known open and deliberately shipped: resume reads local progress and never the
server's; the background-audio handoff still declares a state swap it does not
confirm (the symptom is now impossible, the race is not); and `bun run
android:dev` builds an APK carrying the release application id, whose failure
message advises an uninstall that would destroy app data. Fix that last one
before anyone else builds for Android.
Squashed from worktree-linux-native-video, which keeps the per-defect history.
This commit is contained in:
@@ -46,6 +46,9 @@ android {
|
||||
targetSdk = 36
|
||||
versionCode = tauriProperties.getProperty("tauri.android.versionCode", "1").toInt()
|
||||
versionName = tauriProperties.getProperty("tauri.android.versionName", "1.0")
|
||||
// Required to run the on-device conformance suite
|
||||
// (src/androidTest). See docs/specs/media-player-controller.md.
|
||||
testInstrumentationRunner = "androidx.test.runner.AndroidJUnitRunner"
|
||||
}
|
||||
signingConfigs {
|
||||
create("release") {
|
||||
@@ -147,6 +150,7 @@ dependencies {
|
||||
testImplementation("junit:junit:4.13.2")
|
||||
androidTestImplementation("androidx.test.ext:junit:1.1.4")
|
||||
androidTestImplementation("androidx.test.espresso:espresso-core:3.5.0")
|
||||
androidTestImplementation("androidx.test:runner:1.5.2")
|
||||
}
|
||||
|
||||
apply(from = "tauri.build.gradle.kts")
|
||||
+252
@@ -0,0 +1,252 @@
|
||||
package com.dtourolle.jellytau.player
|
||||
|
||||
import androidx.test.ext.junit.runners.AndroidJUnit4
|
||||
import androidx.test.platform.app.InstrumentationRegistry
|
||||
import org.junit.Assert.assertTrue
|
||||
import org.junit.Assert.assertFalse
|
||||
import org.junit.Before
|
||||
import org.junit.After
|
||||
import org.junit.Test
|
||||
import org.junit.runner.RunWith
|
||||
import java.io.File
|
||||
import kotlin.math.abs
|
||||
|
||||
/**
|
||||
* The MediaPlayer conformance cases, run against ExoPlayer on a real device.
|
||||
*
|
||||
* The desktop suite (src-tauri/src/player/conformance.rs) cannot reach here:
|
||||
* ExoPlayer needs an Android Context and a Looper, so it only exists inside an
|
||||
* app process. These are the same behaviours, asserted against the engine
|
||||
* itself rather than the Rust wrapper — the layer below the contract.
|
||||
*
|
||||
* The fixture is generated rather than committed: a long silent WAV written to
|
||||
* the cache directory at setup. No binary in the repo, no `adb push` step, and
|
||||
* the duration is exact, which matters for the seek assertions.
|
||||
*
|
||||
* Run: ./gradlew :app:connectedDebugAndroidTest (from src-tauri/gen/android)
|
||||
*
|
||||
* TRACES: UR-081 | DR-247
|
||||
*/
|
||||
@RunWith(AndroidJUnit4::class)
|
||||
class PlayerConformanceTest {
|
||||
|
||||
private lateinit var player: JellyTauPlayer
|
||||
private lateinit var mediaUrl: String
|
||||
|
||||
/** Long enough to seek well past any buffer. */
|
||||
private val fixtureSeconds = 1200
|
||||
|
||||
/**
|
||||
* ExoPlayer lands on the nearest sync sample, and a `prepare` is not
|
||||
* instantaneous. Generous on purpose: a tight bound here produces a test
|
||||
* that fails on a slow device and teaches people to re-run until green.
|
||||
*/
|
||||
private val toleranceSeconds = 10.0
|
||||
|
||||
@Before
|
||||
fun setUp() {
|
||||
val context = InstrumentationRegistry.getInstrumentation().targetContext
|
||||
JellyTauPlayer.initialize(context)
|
||||
player = JellyTauPlayer.getInstance()
|
||||
|
||||
val fixture = File(context.cacheDir, "conformance-$fixtureSeconds.wav")
|
||||
if (!fixture.exists() || fixture.length() < 1024) {
|
||||
writeSilentWav(fixture, fixtureSeconds)
|
||||
}
|
||||
mediaUrl = fixture.toURI().toString()
|
||||
}
|
||||
|
||||
@After
|
||||
fun tearDown() {
|
||||
onMain { player.stop() }
|
||||
// Leave nothing playing for the next case.
|
||||
Thread.sleep(200)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- cases
|
||||
|
||||
@Test
|
||||
fun opensFromTheBeginning() {
|
||||
onMain { player.load(mediaUrl, "conformance") }
|
||||
awaitLoaded()
|
||||
|
||||
assertNear(0.0, position(), "playback should start at the beginning")
|
||||
assertTrue("duration should be known once loaded", duration() > 0)
|
||||
}
|
||||
|
||||
/**
|
||||
* DR-241. Opening at a position starts *there*, not at zero.
|
||||
*
|
||||
* `load(url, mediaId)` has no way to express a start position, so every
|
||||
* caller loads and then seeks — and a seek issued against a player that is
|
||||
* still preparing is the window resume was lost in on the desktop side.
|
||||
* This is the same defect on ExoPlayer.
|
||||
*/
|
||||
@Test
|
||||
fun opensAtAStartPosition() {
|
||||
val start = 600.0
|
||||
onMain { player.load(mediaUrl, "conformance", start) }
|
||||
awaitLoaded()
|
||||
|
||||
assertTrue(
|
||||
"opened at ${start}s but playback began at ${position()}s - " +
|
||||
"the start position was dropped",
|
||||
position() > 1.0
|
||||
)
|
||||
assertNear(start, position(), "start position")
|
||||
}
|
||||
|
||||
/** DR-241. A seek issued while still preparing is honoured, not lost. */
|
||||
@Test
|
||||
fun seekWhileOpeningIsHonoured() {
|
||||
val target = 300.0
|
||||
onMain {
|
||||
player.load(mediaUrl, "conformance")
|
||||
// Deliberately before the player is ready: this is the race,
|
||||
// expressed on purpose rather than stumbled into.
|
||||
player.seek(target)
|
||||
}
|
||||
awaitLoaded()
|
||||
|
||||
assertNear(target, position(), "seek issued while opening")
|
||||
}
|
||||
|
||||
@Test
|
||||
fun seeksAfterOpen() {
|
||||
onMain { player.load(mediaUrl, "conformance") }
|
||||
awaitLoaded()
|
||||
|
||||
val target = 420.0
|
||||
onMain { player.seek(target) }
|
||||
awaitPosition(target)
|
||||
|
||||
assertNear(target, position(), "seek after open")
|
||||
}
|
||||
|
||||
/** DR-239. Pause and play are observable, not merely accepted. */
|
||||
@Test
|
||||
fun pauseAndPlayAreObservable() {
|
||||
onMain { player.load(mediaUrl, "conformance") }
|
||||
awaitLoaded()
|
||||
|
||||
onMain { player.pause() }
|
||||
awaitPlaying(false)
|
||||
assertFalse("a paused player must not report playing", isPlaying())
|
||||
|
||||
onMain { player.play() }
|
||||
awaitPlaying(true)
|
||||
assertTrue("a resumed player must report playing", isPlaying())
|
||||
}
|
||||
|
||||
/** `stop()` releases the item, is silent, and can be called twice. */
|
||||
@Test
|
||||
fun closeIsSilentAndIdempotent() {
|
||||
onMain { player.load(mediaUrl, "conformance") }
|
||||
awaitLoaded()
|
||||
|
||||
onMain { player.stop() }
|
||||
awaitPlaying(false)
|
||||
assertFalse("a stopped player must not report playing", isPlaying())
|
||||
|
||||
onMain { player.stop() }
|
||||
assertFalse("stop must be idempotent", isPlaying())
|
||||
}
|
||||
|
||||
/**
|
||||
* An open cancelled by stop must not come back to life.
|
||||
*
|
||||
* The shape of "audio kept playing after leaving the player": a prepare
|
||||
* still in flight completed after the stop, with nothing left to tell it
|
||||
* not to.
|
||||
*/
|
||||
@Test
|
||||
fun closeDuringOpenNeverPlays() {
|
||||
onMain {
|
||||
player.load(mediaUrl, "conformance")
|
||||
player.stop()
|
||||
}
|
||||
Thread.sleep(2000)
|
||||
|
||||
assertFalse(
|
||||
"a load cancelled by stop must not start playing",
|
||||
isPlaying()
|
||||
)
|
||||
}
|
||||
|
||||
// -------------------------------------------------------------- helpers
|
||||
|
||||
private fun onMain(block: () -> Unit) {
|
||||
InstrumentationRegistry.getInstrumentation().runOnMainSync(block)
|
||||
}
|
||||
|
||||
private fun position(): Double = readOnMain { player.getPosition() }
|
||||
private fun duration(): Double = readOnMain { player.getDuration() }
|
||||
private fun isPlaying(): Boolean = readOnMain { player.getExoPlayer().isPlaying }
|
||||
|
||||
private fun <T> readOnMain(block: () -> T): T {
|
||||
var out: T? = null
|
||||
InstrumentationRegistry.getInstrumentation().runOnMainSync { out = block() }
|
||||
@Suppress("UNCHECKED_CAST")
|
||||
return out as T
|
||||
}
|
||||
|
||||
/** Poll a state the player publishes rather than sleeping a fixed time. */
|
||||
private fun await(what: String, timeoutMs: Long = 15_000, predicate: () -> Boolean) {
|
||||
val deadline = System.currentTimeMillis() + timeoutMs
|
||||
while (System.currentTimeMillis() < deadline) {
|
||||
if (predicate()) return
|
||||
Thread.sleep(50)
|
||||
}
|
||||
throw AssertionError("timed out waiting for $what")
|
||||
}
|
||||
|
||||
private fun awaitLoaded() {
|
||||
await("the player to report a duration") { duration() > 0 }
|
||||
// One more beat so a start position or a deferred seek has landed.
|
||||
Thread.sleep(500)
|
||||
}
|
||||
|
||||
private fun awaitPosition(target: Double) =
|
||||
await("position to reach ${target}s") { abs(position() - target) <= toleranceSeconds }
|
||||
|
||||
private fun awaitPlaying(expected: Boolean) =
|
||||
await("isPlaying == $expected", 5_000) { isPlaying() == expected }
|
||||
|
||||
private fun assertNear(expected: Double, actual: Double, what: String) {
|
||||
assertTrue(
|
||||
"$what: expected ~${expected}s, got ${actual}s (tolerance ${toleranceSeconds}s)",
|
||||
abs(actual - expected) <= toleranceSeconds
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* Write a silent 8 kHz mono 16-bit WAV of `seconds` length.
|
||||
*
|
||||
* Synthesised rather than committed so the repo carries no media, and so
|
||||
* the duration is exact — the seek assertions depend on it.
|
||||
*/
|
||||
private fun writeSilentWav(file: File, seconds: Int) {
|
||||
val sampleRate = 8000
|
||||
val dataBytes = sampleRate * 2 * seconds
|
||||
file.outputStream().buffered().use { out ->
|
||||
fun le32(v: Int) = out.write(
|
||||
byteArrayOf(
|
||||
(v and 0xff).toByte(),
|
||||
((v shr 8) and 0xff).toByte(),
|
||||
((v shr 16) and 0xff).toByte(),
|
||||
((v shr 24) and 0xff).toByte()
|
||||
)
|
||||
)
|
||||
fun le16(v: Int) =
|
||||
out.write(byteArrayOf((v and 0xff).toByte(), ((v shr 8) and 0xff).toByte()))
|
||||
|
||||
out.write("RIFF".toByteArray()); le32(36 + dataBytes); out.write("WAVE".toByteArray())
|
||||
out.write("fmt ".toByteArray()); le32(16); le16(1); le16(1)
|
||||
le32(sampleRate); le32(sampleRate * 2); le16(2); le16(16)
|
||||
out.write("data".toByteArray()); le32(dataBytes)
|
||||
|
||||
val chunk = ByteArray(sampleRate * 2) // one second of silence
|
||||
repeat(seconds) { out.write(chunk) }
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -556,11 +556,31 @@ class JellyTauPlayer(private val appContext: Context) {
|
||||
* @param mediaId The unique ID for this media item
|
||||
*/
|
||||
fun load(url: String, mediaId: String) {
|
||||
load(url, mediaId, 0.0)
|
||||
}
|
||||
|
||||
/**
|
||||
* Load [url] and begin at [startPositionSeconds].
|
||||
*
|
||||
* The start position is handed to ExoPlayer with the media item, not seeked
|
||||
* to afterwards. `prepare()` is asynchronous, so a seek issued straight
|
||||
* after a load targets a player that is still preparing: ExoPlayer clamps it
|
||||
* back to zero and the item plays from the beginning. That is what made
|
||||
* resume and transcoded skip start over, and it is why callers must never
|
||||
* express a start position as load-then-seek.
|
||||
*
|
||||
* TRACES: UR-081, UR-005 | DR-241, DR-247
|
||||
*/
|
||||
fun load(url: String, mediaId: String, startPositionSeconds: Double) {
|
||||
mainHandler.post {
|
||||
currentMediaId = mediaId
|
||||
endedNotified = false
|
||||
val mediaItem = MediaItem.fromUri(url)
|
||||
exoPlayer.setMediaItem(mediaItem)
|
||||
if (startPositionSeconds > 0.0) {
|
||||
exoPlayer.setMediaItem(mediaItem, (startPositionSeconds * 1000).toLong())
|
||||
} else {
|
||||
exoPlayer.setMediaItem(mediaItem)
|
||||
}
|
||||
exoPlayer.prepare()
|
||||
exoPlayer.playWhenReady = true
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user