dtourolle/jellytau
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 40 Wiki Activity

feat(player): native video on Linux, and one contract for every player (v0.11.0)

Browse Source 
mpv now decodes video on Linux, drawn into a framebuffer we own and blitted
into the default vbox's draw handler. Tauri's widget tree is untouched, so an
upgrade that assumes its own layout cannot invalidate this. Direct play means
the original file, hardware decoding, and no server transcode at all — where
previously every desktop video was re-encoded to h264 for the browser engine,
whatever the file actually was. Off by default: JELLYTAU_NATIVE_VIDEO=1.

That settles finding 2 of playback-backend-unification.md — "native video
cannot be composited with a Tauri webview" — by demonstration rather than
argument, on X11 and Wayland both.

Turning it on exposed nine defects, none of them mpv's. Each was the same
mistake in a different place: a capability written down as a compile-time fact
about the platform, or a state asserted instead of confirmed.

  DR-238/246  a seek routed by the stream's container rather than by what the
              engine could do with it - correct only while one player handled
              those streams, silent the moment another did
  DR-239      a property handled but never observed, so the play/pause button
              waited for an event that could not arrive
  DR-240      fullscreen expanding the document while the window stayed put
  DR-241      a seek issued before the engine had a file, failed, and discarded
              - which is why resume began at zero
  DR-247      a Linux-only gate outliving the caller that made it Linux-only,
              breaking the Android build outright
  DR-250      a stop aimed at whichever renderer bookkeeping believed was in
              charge, missing the one actually making sound
  DR-251      a duration of zero believed, leaving the seek bar no scale
  DR-252      a junk float converted to a Duration, panicking the backend the
              instant a length-less stream appeared

So the MediaPlayer contract (DR-242 … DR-247): `open` carries a start position,
so no caller sequences load-then-seek and none can race an engine's load;
`seek` states a destination and leaves in-place-versus-re-open to the engine;
`snapshot` is one coherent read; and `Phase::Opening` names the window where
intent used to be lost. One conformance suite runs against every engine —
FakePlayer and mpv under cargo test, ExoPlayer instrumented on a device — so an
engine is either correct or visibly failing.

Two of the nine were introduced during this work and caught on hardware, not by
any suite: an over-broad capability that grouped ExoPlayer with mpv, and the
Duration panic. The suites test engines that behave. That is recorded in
docs/native-player-verification.md, which asks for the exact action sequences
that found them.

Verified: all automated gates, conformance (mpv 9/9, legacy 8/9 by design,
ExoPlayer 7/7 on device), and manual desktop and Android passes on real
hardware.

Known open and deliberately shipped: resume reads local progress and never the
server's; the background-audio handoff still declares a state swap it does not
confirm (the symptom is now impossible, the race is not); and `bun run
android:dev` builds an APK carrying the release application id, whose failure
message advises an uninstall that would destroy app data. Fix that last one
before anyone else builds for Android.

Squashed from worktree-linux-native-video, which keeps the per-defect history.
This commit is contained in:
dtourolle
2026-08-23 10:51:45 +02:00
parent 5fede123e7
commit 11d9d760d8
87 changed files with 15968 additions and 7508 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/native-player-verification.md
+191
View File
@@ -0,0 +1,191 @@
# Native player — verification plan
What to check before the `MediaPlayer` contract and Linux native video reach
`master`.
This is not a generic smoke test. Every case below exists because something
specific went wrong, and most of them were found on hardware **after** the
automated suites were green. Treat the sequences as load-bearing: several
defects only appeared in a particular order of actions, and testing the same
features in a different order missed them entirely.
Companion to [release-checklist.md](release-checklist.md), which covers the
release mechanics. This covers whether the player is fit to release at all.
## What is risky about this change
- `PlayerController` now talks to a `MediaPlayer` contract instead of
`PlayerBackend`. Every engine reaches it through an adapter that did not exist
before (DR-245).
- mpv decodes video on Linux for the first time, composited under the webview
(DR-231).
- Seek strategy is driven by an ability each engine declares rather than by a
truth table (DR-246).
- Two regressions were introduced during this work and caught only on a device:
a wrong capability for ExoPlayer (DR-246 follow-up) and a `Duration` panic
(DR-252). Both were invisible to the test suites.
The suites verify engines that behave. **The manual passes exist to catch
engines that do not.**
## 1. Automated gates
Cheap, fast, and non-negotiable. Run from the worktree.
```bash
bun run check # 0 errors, 0 warnings
bun run test # frontend
bun run test:rust # Rust
bun run format:check
bun run lint # 0 errors; warnings at or below the CI ratchet
bun run check:boundary
bun run traces:validate
bun run traces:coverage # at or above MIN_THRESHOLD
cd src-tauri && cargo fmt --check && cargo clippy --all-targets -- -D warnings
cargo clippy --all-targets --features conformance -- -D warnings
```
The eslint warning count is a **ratchet**: equal to the CI limit is a pass, one
over fails the build. Going one over is how a piece of dead state was found
during this work — do not raise the limit to get past it.
## 2. Engine conformance
```bash
bun run test:player # mpv + legacy, desktop
bun run test:player:android # ExoPlayer, on a connected device
```
Expected, and each deviation is meaningful rather than noise:
| Engine | Result | If it differs |
|---|---|---|
| `MpvPlayer` | 9/9 | A real regression. Stop. |
| `LegacyPlayer` | 8/9 | The one failure is `transport_settings_round_trip`: the old trait has no mute or rate. Any *other* failure is a regression. |
| ExoPlayer (device) | 7/7 | Two cases are absent because the Kotlin player exposes no mute or rate. |
A green conformance run is **not** sufficient evidence to ship. Both regressions
introduced during this work passed conformance.
## 3. Desktop (Linux)
Run with native video on, since that is what is new:
```bash
JELLYTAU_NATIVE_VIDEO=1 bun run tauri dev
```
- [ ] **Direct play** — a file the server does not transcode. Picture and sound.
- [ ] **Transcoded play** — something the server must re-encode (4K, HEVC, or an
audio codec the renderer cannot take).
- [ ] **Resume** — an item watched previously *on this install*. The prompt
appears and playback starts at the offered position, not at zero.
*(Resume is device-local — see "Known open".)*
- [ ] **Scrub** on a direct-play item; position lands and playback continues.
- [ ] **Scrub on a transcoded item.** Separate case on purpose: it takes a
different path, and it silently did nothing for months (DR-238).
- [ ] **Pause and resume** — the button follows the player. It stopped doing so
when a property was handled but never observed (DR-239).
- [ ] **Fullscreen** — the window really fills the display. Measure it if
unsure: the log prints `rendering WxH`, and a height short of the panel
means the document went fullscreen and the window did not (DR-240).
- [ ] **Exit the player** — audio stops. Listen; do not assume.
- [ ] **Audio-only playback** still works: mini player, queue, next/previous.
- [ ] Nothing in the log matches `PANIC` or `ERROR`.
## 4. Android
The tablet needs the *side-by-side* build. **Do not uninstall the release app**
to make an install succeed — see "Known open" for why the normal command is
currently wrong.
```bash
bun run android:build --device
./scripts/sync-android-sources.sh
cd src-tauri/gen/android && ANDROID_HOME="$HOME/Android/Sdk" ./gradlew \
:app:assembleUniversalDebug -x :app:rustBuildUniversalDebug \
-x :app:rustBuildArm64Debug -x :app:rustBuildArmDebug \
-x :app:rustBuildX86Debug -x :app:rustBuildX86_64Debug
adb install -r app/build/outputs/apk/universal/debug/app-universal-debug.apk
```
Confirm the package is `com.dtourolle.jellytau.debug` before installing:
```bash
aapt2 dump packagename <apk>
```
If it says `com.dtourolle.jellytau`, the suffix was lost — **stop**, re-sync and
re-assemble. Installing it would try to replace the real app.
Then, with `adb logcat` capturing:
- [ ] Play a video. Picture, sound, and controls.
- [ ] **Scrub.** The bar has a scale — a duration of `0.0` means the seek bar has
nothing to scrub against (DR-251).
- [ ] Transcoded seek lands rather than restarting the stream. ExoPlayer seeks a
transcode in place; declaring otherwise re-opened it (DR-246).
- [ ] PiP.
- [ ] Lockscreen: controls respond and position tracks.
- [ ] **The handoff sequence, in this exact order:**
1. play a video
2. enable background audio
3. background the app — audio continues
4. foreground the app — **video returns**
5. exit the player — **everything stops**
Steps 4 and 5 are where two separate defects lived (DR-250, DR-252). Doing
the same actions in another order finds neither.
- [ ] `grep -c 'PANIC at' <logcat>` returns 0.
## 5. Regression checks with a named cause
Each of these presented as something other than its cause, which is why they are
listed separately from the feature passes above.
| Symptom to look for | Was actually | Ref |
|---|---|---|
| Skip on a transcoded item does nothing, or jumps to zero | Seek strategy keyed on the container, not the engine | DR-238, DR-246 |
| Play/pause button does not follow the player | A property handled but never observed, so the event never arrived | DR-239 |
| Fullscreen leaves a strip of desktop | The document went fullscreen, the window did not | DR-240 |
| Resume plays from the beginning | A seek issued before the engine had a file was discarded | DR-241 |
| Scrub bar has no scale | Duration reported as `0.0` and believed | DR-251 |
| Black screen, no controls, after a background-audio round trip | A junk duration converted to a `Duration` panicked the backend | DR-252 |
| Audio still playing after leaving the player | The stop was aimed at whichever renderer bookkeeping believed was active | DR-250 |
## Known open — decide, do not discover
None of these are fixed. Each needs an explicit ship / do-not-ship call rather
than being met with surprise during testing.
- **Resume is device-local.** Progress is read from the local database and
nothing consults the server's `UserData`. A fresh install, a second device or
a reinstall offers no resume even though the server knows the position. Not a
regression — it has always been so.
- **The background-audio handoff is an unconfirmed state swap.**
`exit_background_audio` marks the video element the player again the moment it
is called, while the element has not reloaded. DR-250 makes the visible
symptom impossible; the race is intact and can still misdirect a lockscreen
command or a position read. See
[media-player-controller.md](specs/media-player-controller.md).
- 🔴 **The side-by-side debug install is broken.** `bun run android:dev`
produces an APK with the *release* application id, because the Tauri build
regenerates `gen/build.gradle.kts` after the sync drops the `.debug` suffix in.
It then fails on signatures, and its own error message advises uninstalling —
which would destroy the real app's data. **Fix this before anyone else builds
for Android.**
- **`PlayerBackend` still exists** behind `LegacyPlayer`, and the frontend still
carries some playback state. DR-248 and DR-249 are not started.
## Ship criteria
Ship when:
1. Every automated gate in §1 passes.
2. Conformance matches §2 exactly, deviations included.
3. §3 and §4 are complete, on real hardware, by a person.
4. §5 shows no symptom returning.
5. Every item in "Known open" has a recorded decision.
Do not ship on green suites alone. Both regressions introduced during this work
passed every suite and were caught by a person using the app.