feat(windows): mpv plays audio on Windows, with libmpv shipped in the installer
libmpv on Windows (DR-237): - The builder image carries zhongfly's LGPL libmpv-2.dll, pinned by asset name and sha256, plus an MSVC mpv.lib generated from the DLL's own mpv_* exports (the archive ships only a MinGW .dll.a). LGPL, not the GPL builds: no x264/x265, mpv -Dgpl=false; FFmpeg is version3, so LGPL-3.0. THIRD_PARTY_NOTICES.md records it. - build-windows-cross.sh stages both files into src-tauri/windows-libs/; build.rs links mpv.lib from there and tauri.windows.conf.json bundles the DLL beside jellytau.exe from there, with the licence texts under licenses/. One directory, so the DLL shipped is the one linked. - Workflows move to builder image 2026.09.1. Windows audio: - MpvBackend replaces WebviewAudioBackend on Windows (ao=wasapi), so volume, EQ, normalization and gapless work there as on Linux. Local files are passed to mpv as native paths, not file:// URLs. Fixes found on the way: - player_play_item decided "does the backend render video" with cfg!(not(linux)), true on Windows, while get_player_status sent Windows video to the <video> element. With mpv as the backend that would decode every film's soundtrack twice. All three callers now ask video_renders_natively() (UT-273). - confine_queued_path rebuilt paths with PathBuf::push, so on Windows a queued `downloads/x` was stored as `downloads\x`, no longer the spelling the app built. It now keeps the caller's separator (UT-205, which only ever ran on Linux, failed under Windows). Verified: jellytau.exe imports libmpv-2.dll; the full unit suite cross-compiled for Windows passes under wine against the shipped DLL (943/943, including the mpv injection and TLS tests); the NSIS installer contains the DLL and licence texts. Not yet run on real Windows hardware.
This commit is contained in:
Vendored
+29
-17
@@ -7,17 +7,31 @@ job / SMTC lockscreen), but it runs and plays media.
|
||||
|
||||
## How playback works on Windows
|
||||
|
||||
- **Video** — renders through the webview HTML5 `<video>` element (hls.js) on
|
||||
*every* platform; on Windows that is WebView2 (Chromium/Edge), which plays HLS +
|
||||
h264 fine. No Windows-specific code.
|
||||
- **Audio-only (music)** — the native audio backends are libmpv (Linux) and
|
||||
ExoPlayer (Android); neither exists on Windows. Instead
|
||||
`create_player_backend()` in [../src-tauri/src/lib.rs](../../src-tauri/src/lib.rs)
|
||||
uses `WebviewAudioBackend` on non-Linux/non-Android targets: it hands the stream
|
||||
URL to a webview `<audio>` element (see
|
||||
[../src/lib/services/webviewAudio.ts](../../src/lib/services/webviewAudio.ts)),
|
||||
which reports state back through the same `player_report_*` round-trip the video
|
||||
path uses. Pure Rust + Tauri events.
|
||||
- **Video** — renders through the webview HTML5 `<video>` element (hls.js); on
|
||||
Windows that is WebView2 (Chromium/Edge), which plays HLS + h264 fine. mpv
|
||||
takes this over in DR-237's video phase; Linux already made that move.
|
||||
- **Audio** — **libmpv**, the same `MpvBackend` Linux uses, with `ao=wasapi`.
|
||||
Volume, EQ, normalization and gapless all go through mpv's filter graph as on
|
||||
Linux. `libmpv-2.dll` ships beside `jellytau.exe` in the installer.
|
||||
|
||||
### libmpv on Windows
|
||||
|
||||
- **Which build:** zhongfly/mpv-winbuild's *LGPL* dev asset, pinned by name and
|
||||
sha256 in [Dockerfile.builder](../../Dockerfile.builder), which unpacks it to
|
||||
`/opt/libmpv-win64`. Licence terms: [THIRD_PARTY_NOTICES.md](../../THIRD_PARTY_NOTICES.md).
|
||||
The same release also carries a GPL asset (x264/x265) — do not switch to it
|
||||
casually, it moves the installer onto GPL-3.0 terms.
|
||||
- **Import library:** the archive only ships a MinGW `libmpv.dll.a`. The image
|
||||
generates an MSVC `mpv.lib` from the DLL's own `mpv_*` exports
|
||||
(`llvm-readobj --coff-exports` → `.def` → `llvm-lib /def:`), so it cannot name
|
||||
a symbol the DLL lacks.
|
||||
- **One directory:** `scripts/build-windows-cross.sh` copies both files into
|
||||
`src-tauri/windows-libs/` (gitignored) before building. `build.rs` links
|
||||
`mpv.lib` from there and `tauri.windows.conf.json` bundles the DLL from there,
|
||||
so the DLL shipped is the one linked against. Outside the image, point
|
||||
`LIBMPV_WIN_DIR` at a directory holding `libmpv-2.dll` and `mpv.lib`.
|
||||
- **Bumping it:** change the three `LIBMPV_WIN_*` values together, rebuild and
|
||||
push the image, bump the image tag the workflows pin.
|
||||
|
||||
## Cross-compiling from Linux (MSVC + cargo-xwin)
|
||||
|
||||
@@ -35,7 +49,7 @@ Tauri CLI bundle the **NSIS installer from a Linux host**.
|
||||
|
||||
The builder image ([../Dockerfile.builder](../../Dockerfile.builder)) bakes in the
|
||||
whole toolchain: the `x86_64-pc-windows-msvc` rust target, `cargo-xwin`, `lld`,
|
||||
`llvm`, and `nsis`.
|
||||
`llvm`, `nsis`, and the pinned Windows libmpv.
|
||||
|
||||
```bash
|
||||
bun run docker:build:windows # NSIS installer + .exe -> ./dist
|
||||
@@ -71,8 +85,6 @@ Outputs:
|
||||
|
||||
## Outstanding for a first-class Windows release
|
||||
|
||||
1. Gapless/crossfade + SMTC (lockscreen) — currently no-ops in the webview audio
|
||||
path.
|
||||
2. Downloaded (`Local` source) file playback needs `convertFileSrc` on the
|
||||
frontend; streaming works today.
|
||||
3. Code signing + a Windows packaging CI job.
|
||||
1. SMTC (lockscreen / media keys) — not wired on Windows.
|
||||
2. Code signing — the installer is unsigned.
|
||||
3. Video through mpv (DR-237) — needs a WebView2-side surface.
|
||||
|
||||
@@ -439,7 +439,7 @@ Internal architecture, components, and application logic.
|
||||
| DR-234 | The device profile is derived from the **renderer that will decode the stream**, not from a compile-time platform constant. `video_codecs` was `#[cfg(target_os)]`, which is correct only while a build has one video renderer; once mpv and the webview element coexist it must be runtime state. This is the change that converts the measured 7% desktop direct-play rate toward the 85% the Android profile achieves on the same library, because the two differ by nothing except which component decodes. It looks like configuration and is not — it is the input that decides whether the server re-encodes, and getting it wrong fails silently, a claimed codec the renderer cannot decode being a black picture or silence (DR-148, and DR-227's audio override). The webview's narrower *audio* set stops applying to the video path once mpv decodes it, while the multichannel bound still does, since a 5.1 track direct-played into a two-channel sink is silence or inaudible dialogue | Repository | UR-080, UR-070 | In Progress |
|
||||
| DR-235 | The webview video path is deleted, not merely bypassed. Staged, because a path cannot be removed while a shipped platform still needs it: Linux moves to mpv first, Windows follows, and only then do `hls.js`, `html5Adapter.ts`, `videoLoaderFor` and the `<video>` element go. The staging is the point — a Linux-only version would leave the fork alive permanently, taking video from three renderers to four and giving every seek strategy, track switch and lifecycle bug one more place to be got right. Android keeps ExoPlayer and keeps the webview as its documented opt-out; the background-audio `<audio>` path is untouched. With no HTML5 fallback left, a failed mpv init emits `backend-init-failed` and surfaces a real error rather than silently degrading to the transcode this work exists to stop paying for | Playback | UR-080 | In Progress |
|
||||
| DR-236 | Hardware-decode policy is decided from what mpv reports it **selected** (`hwdec-current`), never from what it was asked for. The spike established that hardware decode works through the render API at all — the load-bearing result, since it means direct play is not bought with software decoding — but also that `auto` reached for the discrete GPU in copy-back mode on a hybrid Intel+NVIDIA laptop, the least efficient hardware path, and that `vaapi` fell back to software silently because the libva driver was absent. So zero-copy VA-API on the integrated GPU is preferred where the driver is present, `auto` is a fallback rather than the default, and a missing driver is detected and logged rather than mistaken for a compositing limit | Playback | UR-080 | Proposed |
|
||||
| DR-237 | Windows reaches the same mpv path, reusing everything except the surface. The surface is genuinely different code — a native child window beneath a transparent WebView2, not GTK — but the render context, lifetime discipline, frame pacing, device profile and hwdec policy are shared, which is why none of them may be guarded on `cfg!(target_os = "linux")`. The cost is mostly build, not video: `libmpv` is currently a Linux-only dependency while Windows is cross-compiled from Linux via `x86_64-pc-windows-msvc` + `cargo-xwin`, so a Windows libmpv must reach that cross-build and its DLL must ship in the NSIS bundle, carrying the LGPL obligations DR-216 already records — dynamic linkage, licence text shipped alongside. Windows gains a native audio decoder as a side effect, which is what the long-blocked Windows audio work wants and cannot otherwise have | Playback | UR-080 | Proposed |
|
||||
| DR-237 | Windows reaches the same mpv path, reusing everything except the surface. The surface is genuinely different code — a native child window beneath a transparent WebView2, not GTK — but the render context, lifetime discipline, frame pacing, device profile and hwdec policy are shared, which is why none of them may be guarded on `cfg!(target_os = "linux")`. The cost is mostly build, not video: `libmpv` is currently a Linux-only dependency while Windows is cross-compiled from Linux via `x86_64-pc-windows-msvc` + `cargo-xwin`, so a Windows libmpv must reach that cross-build and its DLL must ship in the NSIS bundle, carrying the LGPL obligations DR-216 already records — dynamic linkage, licence text shipped alongside. Windows gains a native audio decoder as a side effect, which is what the long-blocked Windows audio work wants and cannot otherwise have | Playback | UR-080 | In Progress |
|
||||
| DR-238 | A transcoded seek re-negotiates the stream on every renderer, not just the webview. Jellyfin produces a transcode *from* `StartTimeTicks`, so where a seek lands is a property of the request rather than of the stream in hand. `determine_video_seek_strategy` treated `is_hls` as a proxy for "seekable in place", which held only because hls.js was always the HLS renderer — it seeks within the VOD playlist it is handed and lets the server catch up. mpv's HLS demuxer cannot make the server transcode from a new offset, so with native video on, every transcoded seek became a backend seek that silently did nothing and presented as "resume does not work". The rule is now written on `needs_transcoding` with hls.js as the stated exception; all four webview cells are unchanged | Player | UR-040 | Done |
|
||||
| DR-239 | Properties the mpv event loop handles are registered with `observe_property`. libmpv delivers `PropertyChange` only for observed properties, so a `match` arm for an unobserved one is unreachable code that reads as implemented — the handler is right there. `pause` was handled and never observed, so `StateChanged` was never emitted on pause or resume and the play/pause control never moved. It stayed invisible while Linux video played in the webview, because the `<video>` element's own DOM events drove that control; native video made the UI depend on the event that never came | Player | UR-005 | Done |
|
||||
| DR-240 | Fullscreen moves whatever actually owns the pixels. `requestFullscreen()` fullscreens the *document*, which sufficed while every renderer lived inside it — the HTML5 `<video>` element is part of the document, so WebKit scaled it and the OS window's real size never mattered. A native surface is drawn behind the webview at **window** size, so a document-only fullscreen expands the page and leaves the picture where it was; on WebKitGTK the result is a maximised window with decorations still holding a strip of the screen, which reads as "fullscreen is broken" rather than as a windowing problem. Android needed the same rule for the system bars (DR-157); this is its desktop half | Player | UR-066 | Done |
|
||||
@@ -868,6 +868,7 @@ Internal architecture, components, and application logic.
|
||||
| UT-270 | A hardened handle reads back `tls-verify=yes` and `ytdl=no`, and both mpv players harden the handle they create | DR-299 | Done |
|
||||
| UT-271 | Native video is on for Linux whatever `JELLYTAU_NATIVE_VIDEO` says, including unset and explicit "off" values, and off where mpv is not the video renderer | DR-235 | Done |
|
||||
| UT-272 | No platform reports a webview video fallback, Linux reports native video, and the player status on Linux never sends video to the `<video>` element | DR-235 | Done |
|
||||
| UT-273 | `player_play_item`, `get_player_status` and `player_get_capabilities` answer "does a native renderer draw video here" from one function, so the backend is loaded with video exactly where the frontend is told not to use a `<video>` element — on Windows, where mpv now plays audio, the film's soundtrack is not decoded twice | DR-237 | Done |
|
||||
### Integration Tests
|
||||
|
||||
| Test ID | Test Description | Traces To | Status |
|
||||
|
||||
@@ -7,7 +7,9 @@ webview fallback is offered). **Left:** Linux's device profile still claims only
|
||||
the direct-play gain is not yet taken; `hwdec` is unset, so mpv decodes in
|
||||
software (DR-236); a failed surface attach only logs, it does not surface an
|
||||
error; the phase 1 soak and X11/Wayland criteria are unrecorded; phases 2 and 3.
|
||||
Phase 3 also deletes the now-inert frontend switch (`experimentalNativeVideo`,
|
||||
Phase 2 has started from its build half: Windows now links and ships libmpv
|
||||
(for audio, see windows-native-audio-backend.md); its video surface is not
|
||||
written. Phase 3 also deletes the now-inert frontend switch (`experimentalNativeVideo`,
|
||||
`nativeVideoWanted`, the Settings toggle and the adapter's suppressor flag),
|
||||
which no platform reaches since `webview_video_fallback` became false everywhere.
|
||||
**Requirements:** UR-080 (new) → DR-231 … DR-237 (new); IR-033 (new)
|
||||
|
||||
@@ -1,8 +1,15 @@
|
||||
# Spec: Windows native audio backend
|
||||
|
||||
**Status:** Proposed — not started. Windows still runs on
|
||||
`WebviewAudioBackend`. Blocked on [libmpv2-migration.md](libmpv2-migration.md),
|
||||
whose crate swap has not landed either.
|
||||
**Status:** Partially implemented — code and packaging done, **unverified on
|
||||
Windows hardware**. `MpvBackend` is the Windows audio backend (`ao=wasapi`);
|
||||
the builder image carries zhongfly's pinned LGPL `libmpv-2.dll` plus a generated
|
||||
MSVC `mpv.lib`, and the NSIS installer ships the DLL beside `jellytau.exe` with
|
||||
its licence texts ([build-windows.md](../build/build-windows.md)). Done *before*
|
||||
the libmpv2 migration after all: the current `libmpv` pin cross-links fine, and
|
||||
the migration now has one call site to keep safe (`mpv_command`, DR-298) rather
|
||||
than a crate API to port twice. **Left:** every acceptance criterion that needs a
|
||||
Windows machine — audible volume/EQ/normalization/gapless, seek/queue/sleep
|
||||
timer, and the clean-VM install test.
|
||||
**Requirements:** UR-003, UR-027, UR-032, UR-033 → DR-030, DR-035, DR-036;
|
||||
⚠️ the suggested id **IR-030 has since been allocated** to the scheduled catalog
|
||||
crawl — allocate a fresh id (IR-033 or later) on implementation
|
||||
|
||||
Reference in New Issue
Block a user