feat(player): native video on Linux, and one contract for every player (v0.11.0)

mpv now decodes video on Linux, drawn into a framebuffer we own and blitted
into the default vbox's draw handler. Tauri's widget tree is untouched, so an
upgrade that assumes its own layout cannot invalidate this. Direct play means
the original file, hardware decoding, and no server transcode at all — where
previously every desktop video was re-encoded to h264 for the browser engine,
whatever the file actually was. Off by default: JELLYTAU_NATIVE_VIDEO=1.

That settles finding 2 of playback-backend-unification.md — "native video
cannot be composited with a Tauri webview" — by demonstration rather than
argument, on X11 and Wayland both.

Turning it on exposed nine defects, none of them mpv's. Each was the same
mistake in a different place: a capability written down as a compile-time fact
about the platform, or a state asserted instead of confirmed.

  DR-238/246  a seek routed by the stream's container rather than by what the
              engine could do with it - correct only while one player handled
              those streams, silent the moment another did
  DR-239      a property handled but never observed, so the play/pause button
              waited for an event that could not arrive
  DR-240      fullscreen expanding the document while the window stayed put
  DR-241      a seek issued before the engine had a file, failed, and discarded
              - which is why resume began at zero
  DR-247      a Linux-only gate outliving the caller that made it Linux-only,
              breaking the Android build outright
  DR-250      a stop aimed at whichever renderer bookkeeping believed was in
              charge, missing the one actually making sound
  DR-251      a duration of zero believed, leaving the seek bar no scale
  DR-252      a junk float converted to a Duration, panicking the backend the
              instant a length-less stream appeared

So the MediaPlayer contract (DR-242 … DR-247): `open` carries a start position,
so no caller sequences load-then-seek and none can race an engine's load;
`seek` states a destination and leaves in-place-versus-re-open to the engine;
`snapshot` is one coherent read; and `Phase::Opening` names the window where
intent used to be lost. One conformance suite runs against every engine —
FakePlayer and mpv under cargo test, ExoPlayer instrumented on a device — so an
engine is either correct or visibly failing.

Two of the nine were introduced during this work and caught on hardware, not by
any suite: an over-broad capability that grouped ExoPlayer with mpv, and the
Duration panic. The suites test engines that behave. That is recorded in
docs/native-player-verification.md, which asks for the exact action sequences
that found them.

Verified: all automated gates, conformance (mpv 9/9, legacy 8/9 by design,
ExoPlayer 7/7 on device), and manual desktop and Android passes on real
hardware.

Known open and deliberately shipped: resume reads local progress and never the
server's; the background-audio handoff still declares a state swap it does not
confirm (the symptom is now impossible, the race is not); and `bun run
android:dev` builds an APK carrying the release application id, whose failure
message advises an uninstall that would destroy app data. Fix that last one
before anyone else builds for Android.

Squashed from worktree-linux-native-video, which keeps the per-defect history.
This commit is contained in:
2026-08-23 10:51:45 +02:00
parent 5fede123e7
commit 11d9d760d8
87 changed files with 15968 additions and 7508 deletions
@@ -0,0 +1,252 @@
package com.dtourolle.jellytau.player
import androidx.test.ext.junit.runners.AndroidJUnit4
import androidx.test.platform.app.InstrumentationRegistry
import org.junit.Assert.assertTrue
import org.junit.Assert.assertFalse
import org.junit.Before
import org.junit.After
import org.junit.Test
import org.junit.runner.RunWith
import java.io.File
import kotlin.math.abs
/**
* The MediaPlayer conformance cases, run against ExoPlayer on a real device.
*
* The desktop suite (src-tauri/src/player/conformance.rs) cannot reach here:
* ExoPlayer needs an Android Context and a Looper, so it only exists inside an
* app process. These are the same behaviours, asserted against the engine
* itself rather than the Rust wrapper — the layer below the contract.
*
* The fixture is generated rather than committed: a long silent WAV written to
* the cache directory at setup. No binary in the repo, no `adb push` step, and
* the duration is exact, which matters for the seek assertions.
*
* Run: ./gradlew :app:connectedDebugAndroidTest (from src-tauri/gen/android)
*
* TRACES: UR-081 | DR-247
*/
@RunWith(AndroidJUnit4::class)
class PlayerConformanceTest {
private lateinit var player: JellyTauPlayer
private lateinit var mediaUrl: String
/** Long enough to seek well past any buffer. */
private val fixtureSeconds = 1200
/**
* ExoPlayer lands on the nearest sync sample, and a `prepare` is not
* instantaneous. Generous on purpose: a tight bound here produces a test
* that fails on a slow device and teaches people to re-run until green.
*/
private val toleranceSeconds = 10.0
@Before
fun setUp() {
val context = InstrumentationRegistry.getInstrumentation().targetContext
JellyTauPlayer.initialize(context)
player = JellyTauPlayer.getInstance()
val fixture = File(context.cacheDir, "conformance-$fixtureSeconds.wav")
if (!fixture.exists() || fixture.length() < 1024) {
writeSilentWav(fixture, fixtureSeconds)
}
mediaUrl = fixture.toURI().toString()
}
@After
fun tearDown() {
onMain { player.stop() }
// Leave nothing playing for the next case.
Thread.sleep(200)
}
// ---------------------------------------------------------------- cases
@Test
fun opensFromTheBeginning() {
onMain { player.load(mediaUrl, "conformance") }
awaitLoaded()
assertNear(0.0, position(), "playback should start at the beginning")
assertTrue("duration should be known once loaded", duration() > 0)
}
/**
* DR-241. Opening at a position starts *there*, not at zero.
*
* `load(url, mediaId)` has no way to express a start position, so every
* caller loads and then seeks — and a seek issued against a player that is
* still preparing is the window resume was lost in on the desktop side.
* This is the same defect on ExoPlayer.
*/
@Test
fun opensAtAStartPosition() {
val start = 600.0
onMain { player.load(mediaUrl, "conformance", start) }
awaitLoaded()
assertTrue(
"opened at ${start}s but playback began at ${position()}s - " +
"the start position was dropped",
position() > 1.0
)
assertNear(start, position(), "start position")
}
/** DR-241. A seek issued while still preparing is honoured, not lost. */
@Test
fun seekWhileOpeningIsHonoured() {
val target = 300.0
onMain {
player.load(mediaUrl, "conformance")
// Deliberately before the player is ready: this is the race,
// expressed on purpose rather than stumbled into.
player.seek(target)
}
awaitLoaded()
assertNear(target, position(), "seek issued while opening")
}
@Test
fun seeksAfterOpen() {
onMain { player.load(mediaUrl, "conformance") }
awaitLoaded()
val target = 420.0
onMain { player.seek(target) }
awaitPosition(target)
assertNear(target, position(), "seek after open")
}
/** DR-239. Pause and play are observable, not merely accepted. */
@Test
fun pauseAndPlayAreObservable() {
onMain { player.load(mediaUrl, "conformance") }
awaitLoaded()
onMain { player.pause() }
awaitPlaying(false)
assertFalse("a paused player must not report playing", isPlaying())
onMain { player.play() }
awaitPlaying(true)
assertTrue("a resumed player must report playing", isPlaying())
}
/** `stop()` releases the item, is silent, and can be called twice. */
@Test
fun closeIsSilentAndIdempotent() {
onMain { player.load(mediaUrl, "conformance") }
awaitLoaded()
onMain { player.stop() }
awaitPlaying(false)
assertFalse("a stopped player must not report playing", isPlaying())
onMain { player.stop() }
assertFalse("stop must be idempotent", isPlaying())
}
/**
* An open cancelled by stop must not come back to life.
*
* The shape of "audio kept playing after leaving the player": a prepare
* still in flight completed after the stop, with nothing left to tell it
* not to.
*/
@Test
fun closeDuringOpenNeverPlays() {
onMain {
player.load(mediaUrl, "conformance")
player.stop()
}
Thread.sleep(2000)
assertFalse(
"a load cancelled by stop must not start playing",
isPlaying()
)
}
// -------------------------------------------------------------- helpers
private fun onMain(block: () -> Unit) {
InstrumentationRegistry.getInstrumentation().runOnMainSync(block)
}
private fun position(): Double = readOnMain { player.getPosition() }
private fun duration(): Double = readOnMain { player.getDuration() }
private fun isPlaying(): Boolean = readOnMain { player.getExoPlayer().isPlaying }
private fun <T> readOnMain(block: () -> T): T {
var out: T? = null
InstrumentationRegistry.getInstrumentation().runOnMainSync { out = block() }
@Suppress("UNCHECKED_CAST")
return out as T
}
/** Poll a state the player publishes rather than sleeping a fixed time. */
private fun await(what: String, timeoutMs: Long = 15_000, predicate: () -> Boolean) {
val deadline = System.currentTimeMillis() + timeoutMs
while (System.currentTimeMillis() < deadline) {
if (predicate()) return
Thread.sleep(50)
}
throw AssertionError("timed out waiting for $what")
}
private fun awaitLoaded() {
await("the player to report a duration") { duration() > 0 }
// One more beat so a start position or a deferred seek has landed.
Thread.sleep(500)
}
private fun awaitPosition(target: Double) =
await("position to reach ${target}s") { abs(position() - target) <= toleranceSeconds }
private fun awaitPlaying(expected: Boolean) =
await("isPlaying == $expected", 5_000) { isPlaying() == expected }
private fun assertNear(expected: Double, actual: Double, what: String) {
assertTrue(
"$what: expected ~${expected}s, got ${actual}s (tolerance ${toleranceSeconds}s)",
abs(actual - expected) <= toleranceSeconds
)
}
/**
* Write a silent 8 kHz mono 16-bit WAV of `seconds` length.
*
* Synthesised rather than committed so the repo carries no media, and so
* the duration is exact — the seek assertions depend on it.
*/
private fun writeSilentWav(file: File, seconds: Int) {
val sampleRate = 8000
val dataBytes = sampleRate * 2 * seconds
file.outputStream().buffered().use { out ->
fun le32(v: Int) = out.write(
byteArrayOf(
(v and 0xff).toByte(),
((v shr 8) and 0xff).toByte(),
((v shr 16) and 0xff).toByte(),
((v shr 24) and 0xff).toByte()
)
)
fun le16(v: Int) =
out.write(byteArrayOf((v and 0xff).toByte(), ((v shr 8) and 0xff).toByte()))
out.write("RIFF".toByteArray()); le32(36 + dataBytes); out.write("WAVE".toByteArray())
out.write("fmt ".toByteArray()); le32(16); le16(1); le16(1)
le32(sampleRate); le32(sampleRate * 2); le16(2); le16(16)
out.write("data".toByteArray()); le32(dataBytes)
val chunk = ByteArray(sampleRate * 2) // one second of silence
repeat(seconds) { out.write(chunk) }
}
}
}
@@ -556,11 +556,31 @@ class JellyTauPlayer(private val appContext: Context) {
* @param mediaId The unique ID for this media item
*/
fun load(url: String, mediaId: String) {
load(url, mediaId, 0.0)
}
/**
* Load [url] and begin at [startPositionSeconds].
*
* The start position is handed to ExoPlayer with the media item, not seeked
* to afterwards. `prepare()` is asynchronous, so a seek issued straight
* after a load targets a player that is still preparing: ExoPlayer clamps it
* back to zero and the item plays from the beginning. That is what made
* resume and transcoded skip start over, and it is why callers must never
* express a start position as load-then-seek.
*
* TRACES: UR-081, UR-005 | DR-241, DR-247
*/
fun load(url: String, mediaId: String, startPositionSeconds: Double) {
mainHandler.post {
currentMediaId = mediaId
endedNotified = false
val mediaItem = MediaItem.fromUri(url)
exoPlayer.setMediaItem(mediaItem)
if (startPositionSeconds > 0.0) {
exoPlayer.setMediaItem(mediaItem, (startPositionSeconds * 1000).toLong())
} else {
exoPlayer.setMediaItem(mediaItem)
}
exoPlayer.prepare()
exoPlayer.playWhenReady = true
}