docs(player): backend unification findings + correct false parity claims
Investigation into unifying the playback backends (Linux/MPV, Android/ExoPlayer, Windows/webview) onto one engine with hardware acceleration. Conclusion: video cannot be unified onto a native engine; audio can. The blocker is not mpv-specific. WebKitGTK, WebView2 and Android WebView each draw into their own compositor surface, so a native video surface sits either entirely above or entirely below the webview and cannot interleave with HTML. GStreamer and libVLC fail identically. mpv would additionally regress streaming: it has no adaptive bitrate, while the current hls.js path does. Six specs added: - playback-backend-unification: the analysis and decision, with evidence - android-audio-settings-parity: set_audio_settings on ExoPlayerBackend - android-native-video-spike: timeboxed test of SurfaceView compositing - windows-native-audio-backend: replace the webview <audio> shim with libmpv - libmpv2-migration: dead libmpv git pin -> libmpv2, plus a LICENSE file - playback-docs-corrections: the requirement-status fixes applied here Corrections to requirements.md, all verified against source: - UR-031/DR-034 claimed crossfade was "Done (Linux only)". It is implemented nowhere (mpv_backend.rs has a bare TODO) and is architecturally blocked on mpv, whose single-stream audio chain cannot feed acrossfade's two inputs. - Parity matrix listed crossfade as a Linux/Android gap; it is neither. - The matrix omitted the equalizer, which has the same Linux-only shape. - The suggested ConcatenatingMediaSource is deprecated in current Media3. nativeAdapter.ts cited tauri#10152 as an upstream blocker for native Android video. That issue is a stale feature request, dead since 2024-07-01; the capability shipped in tauri 27d01834 (2024-09-02), and the related black-screen bug was fixed in wry 0.39.4 (we ship 0.55.x). What is genuinely unproven is SurfaceView-behind-WebView compositing, which the spike now tracks.
This commit is contained in:
+26
-12
@@ -41,7 +41,7 @@ For a narrative overview of the system design, see
|
||||
| UR-028 | Navigate to artist/album by tapping names in now playing view | High | Done |
|
||||
| UR-029 | Toggle between grid and list view in library | Medium | Done |
|
||||
| UR-030 | Quick genre browsing and filtering | Medium | Done |
|
||||
| UR-031 | Crossfade between audio tracks | Low | Done (Linux only) |
|
||||
| UR-031 | Crossfade between audio tracks | Low | Not implemented (blocked — see DR-034) |
|
||||
| UR-032 | Gapless playback for seamless album listening | Medium | Done (Linux only) |
|
||||
| UR-033 | Volume normalization to prevent volume jumps between tracks | Low | Done (Linux only) |
|
||||
| UR-034 | Rich home screen with hero banners, carousels, and personalized sections | High | Done |
|
||||
@@ -193,7 +193,7 @@ Internal architecture, components, and application logic.
|
||||
| DR-031 | Clickable artist/album links in now playing view | UI | UR-028 | Done |
|
||||
| DR-032 | List view option for library browsing (albums, artists) | UI | UR-029 | Done |
|
||||
| DR-033 | Genre browsing screen with quick filters | UI | UR-030 | Done |
|
||||
| DR-034 | Crossfade engine with configurable duration (0-12s) | Player | UR-031 | Done (Linux only) |
|
||||
| DR-034 | Crossfade engine with configurable duration (0-12s) | Player | UR-031 | Not implemented (blocked on MPV: single-stream audio chain; `acrossfade` needs 2 inputs — see docs/specs/playback-backend-unification.md) |
|
||||
| DR-035 | Gapless playback between sequential tracks | Player | UR-032 | Done (Linux only) |
|
||||
| DR-036 | Volume normalization with preset levels (Loud/Normal/Quiet) | Player | UR-033 | Done (Linux only) |
|
||||
| DR-037 | Remote session browser and control UI | UI | UR-010 | Done |
|
||||
@@ -499,22 +499,36 @@ The `PlayerBackend` trait defines optional audio settings methods with default e
|
||||
| Basic playback | ✅ | ✅ | Parity |
|
||||
| Volume control | ✅ | ✅ | Parity |
|
||||
| Seek | ✅ | ✅ | Parity |
|
||||
| Crossfade | ✅ | ❌ | Gap |
|
||||
| Gapless playback | ✅ | ❌ | Gap |
|
||||
| Volume normalization | ✅ | ❌ | Gap |
|
||||
| Crossfade | ❌ | ❌ | Not implemented (blocked on MPV) |
|
||||
| Gapless playback | ✅ | ⚠️ | Implemented, pending on-device verification |
|
||||
| Volume normalization | ✅ | ⚠️ | Implemented (LoudnessEnhancer — gain stage, approximate vs MPV's dynaudnorm), pending on-device verification |
|
||||
| Equalizer (10-band) | ✅ | ⚠️ | Implemented (resampled onto device bands), pending on-device verification |
|
||||
| Position updates | 250ms | On-demand | Inconsistent |
|
||||
|
||||
**Future Fix**:
|
||||
1. Implement `set_audio_settings()` in `ExoPlayerBackend`
|
||||
2. Add Kotlin-side ExoPlayer configuration for crossfade (using `ConcatenatingMediaSource` or `DefaultMediaSourceFactory`)
|
||||
3. Implement gapless via ExoPlayer's built-in gapless support
|
||||
4. Add volume normalization via ExoPlayer's `LoudnessEnhancer` or audio processor
|
||||
5. Standardize position update frequency across platforms
|
||||
**Status** (see docs/specs/android-audio-settings-parity.md):
|
||||
1. ✅ `set_audio_settings()` implemented in `ExoPlayerBackend` (JSON over JNI)
|
||||
2. ✅ Gapless via ExoPlayer's `pauseAtEndOfMediaItems`
|
||||
3. ✅ Volume normalization via `LoudnessEnhancer`
|
||||
4. ✅ Equalizer via `android.media.audiofx.Equalizer`, canonical 10 bands
|
||||
resampled onto the device's band centres
|
||||
5. ⬜ **Not yet verified on a physical device** — the EQ/normalization effects
|
||||
depend on device-specific `AudioEffect` availability and band layouts
|
||||
6. ⬜ Flip the trait's `set_audio_settings` default from `Ok(())` to
|
||||
`Err(not_implemented())` so a backend that omits it fails loudly instead of
|
||||
silently reporting success. Deferred until (5) confirms the Android path works
|
||||
7. ⬜ Standardize position update frequency across platforms
|
||||
|
||||
Crossfade is deliberately absent: it is unimplemented on every platform and
|
||||
architecturally blocked on MPV, so building it on Android alone would invert the
|
||||
parity gap. (The previously suggested `ConcatenatingMediaSource` is also
|
||||
deprecated in current Media3.)
|
||||
|
||||
**Impact**:
|
||||
- Medium - Android users lack audio enhancement features advertised in requirements
|
||||
- User experience differs between platforms
|
||||
- UR-031 (Crossfade), UR-032 (Gapless), UR-033 (Normalization) only work on Linux
|
||||
- UR-032 (Gapless), UR-033 (Normalization) and UR-027 (Equalizer) are now
|
||||
implemented on Android as well as Linux, pending on-device verification
|
||||
- UR-031 (Crossfade) works nowhere — see DR-034
|
||||
|
||||
**Traces To**: IR-004, UR-031, UR-032, UR-033, DR-034, DR-035, DR-036
|
||||
|
||||
|
||||
@@ -0,0 +1,209 @@
|
||||
# Spec: Android audio settings parity (EQ, normalization, gapless)
|
||||
|
||||
**Status:** Proposed
|
||||
**Requirements:** UR-031, UR-032, UR-033, UR-027 → DR-034, DR-035, DR-036, DR-030; IR-004
|
||||
**UX spec:** n/a — no UI change; Settings › Audio already renders these controls
|
||||
**Supersedes / revises:** closes the audio half of the parity gap recorded in [playback-backend-unification.md](playback-backend-unification.md)
|
||||
|
||||
## Summary
|
||||
|
||||
Implement `set_audio_settings` / `audio_settings` on `ExoPlayerBackend` so the
|
||||
equalizer, volume normalization, and gapless playback settings actually take
|
||||
effect on Android. Today the Settings › Audio panel renders these controls on
|
||||
Android and they silently do nothing — `ExoPlayerBackend` is the only backend
|
||||
that does not override the trait's no-op defaults.
|
||||
|
||||
Crossfade is explicitly **not** included; see Out of scope.
|
||||
|
||||
## Motivation
|
||||
|
||||
`PlayerBackend` declares `set_audio_settings` with a default `Ok(())` body.
|
||||
`MpvBackend`, `NullBackend`, and `WebviewAudioBackend` all override it;
|
||||
`ExoPlayerBackend` does not. The settings are persisted, pushed to the backend on
|
||||
every track load, and displayed in the UI — and then dropped on the floor.
|
||||
|
||||
This is the single most user-visible platform divergence in the app: a user who
|
||||
sets a "Rock" EQ preset on Android sees the sliders move and hears no change.
|
||||
|
||||
The backend-unification investigation ruled out fixing this by swapping engines
|
||||
(video cannot be unified; see the sibling spec), so the fix is to implement the
|
||||
trait methods where they are missing.
|
||||
|
||||
## Layer assignment
|
||||
|
||||
| Logic / responsibility | Layer | Why it belongs there |
|
||||
|------------------------|-------|----------------------|
|
||||
| Band count, centre frequencies, gain range, preset→curve map | Rust (existing) | Already domain-owned in `settings.rs` per [audio-equalizer.md](audio-equalizer.md). Android must consume the same `AudioSettings`, not define its own bands. Duplicating the band layout in Kotlin would be a taxonomy leak of exactly the kind `check:boundary` guards against. |
|
||||
| Mapping `AudioSettings` → Android audio-effect parameters | Rust → JNI boundary | Platform playback detail, the direct analogue of `build_af_filter` in `mpv_backend.rs`. Belongs with the other `set_audio_settings` code. |
|
||||
| Attaching/detaching `Equalizer` and `LoudnessEnhancer` to the ExoPlayer audio session | Kotlin (`JellyTauPlayer.kt`) | Android platform API mechanics; needs the live `audioSessionId`, which only the Kotlin layer holds. |
|
||||
| Normalization preset (Loud/Normal/Quiet) → target gain | Rust (existing) | `VolumeLevel` is domain vocabulary; the same preset must mean the same loudness on every platform. |
|
||||
| Rendering sliders / preset chips | Frontend (existing) | Pure presentation; unchanged by this spec. |
|
||||
|
||||
Borderline row: attaching the effects could arguably be driven entirely from
|
||||
Rust via JNI property calls. It goes to Kotlin because `AudioEffect` construction
|
||||
requires the audio session id and must be re-attached when ExoPlayer rebuilds its
|
||||
audio sink — lifecycle state that lives in `JellyTauPlayer.kt`. Rust still owns
|
||||
*what* the values are; Kotlin owns *when* the effect objects exist.
|
||||
|
||||
## Design
|
||||
|
||||
### Rust — `ExoPlayerBackend` (`src-tauri/src/player/android/mod.rs`)
|
||||
|
||||
Override the two defaulted methods, mirroring the shape of the existing
|
||||
`set_audio_track` JNI call:
|
||||
|
||||
```rust
|
||||
fn set_audio_settings(&mut self, settings: &AudioSettings) -> Result<(), PlayerError> {
|
||||
let s = settings.clone().with_crossfade_clamped().with_equalizer_normalised();
|
||||
// Serialize as JSON — the same pattern load() already uses for subtitles,
|
||||
// avoiding a 6-arg JNI signature that has to change every time a field lands.
|
||||
let json = serde_json::to_string(&s).map_err(|e| PlayerError { message: e.to_string() })?;
|
||||
// Kotlin: fun setAudioSettings(json: String)
|
||||
self.call_player_method_string("setAudioSettings", &json)?;
|
||||
self.shared_state.lock_safe().audio_settings = s;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn audio_settings(&self) -> AudioSettings {
|
||||
self.shared_state.lock_safe().audio_settings.clone()
|
||||
}
|
||||
```
|
||||
|
||||
`ExoPlayerState` gains an `audio_settings: AudioSettings` field. Note
|
||||
`ExoPlayerBackend` currently holds no such state — `position`/`state`/`volume` are
|
||||
all pushed in by JNI callbacks — so this is the first *pull*-side field. That is
|
||||
correct: audio settings are commanded downward, never reported upward.
|
||||
|
||||
### Kotlin — `JellyTauPlayer.kt`
|
||||
|
||||
```kotlin
|
||||
fun setAudioSettings(json: String) {
|
||||
val s = JSONObject(json)
|
||||
applyEqualizer(s.getBoolean("equalizerEnabled"), s.getJSONArray("equalizerBands"))
|
||||
applyNormalization(s.getBoolean("normalizeVolume"), s.getString("volumeLevel"))
|
||||
exoPlayer.pauseAtEndOfMediaItems = !s.getBoolean("gaplessPlayback")
|
||||
}
|
||||
```
|
||||
|
||||
Three independent mechanisms:
|
||||
|
||||
- **Gapless** — nearly free. ExoPlayer is gapless by default for compatible
|
||||
formats; honouring the setting means *disabling* it when the user turns it off,
|
||||
via `pauseAtEndOfMediaItems`. Note this only applies within a loaded playlist;
|
||||
our queue loads one item at a time, so verify behaviour before claiming DR-035
|
||||
on Android (see Testing).
|
||||
- **Equalizer** — `android.media.audiofx.Equalizer` bound to
|
||||
`exoPlayer.audioSessionId`. Android's EQ exposes a device-dependent band count
|
||||
(commonly 5) at fixed centre frequencies, which will **not** match our 10-band
|
||||
ISO layout. Rust owns the canonical 10 bands; Kotlin resamples them onto the
|
||||
device's bands by nearest-centre-frequency interpolation. Gains are in
|
||||
millibels (`setBandLevel` takes mB, we store dB → ×100), clamped to the
|
||||
device's reported `getBandLevelRange()`.
|
||||
- **Normalization** — `android.media.audiofx.LoudnessEnhancer`, also bound to the
|
||||
audio session, `setTargetGain(mB)` derived from `VolumeLevel`. This is a gain
|
||||
booster, not a true EBU R128 normalizer like MPV's `dynaudnorm`; parity is
|
||||
approximate and should be documented as such rather than overclaimed.
|
||||
|
||||
Lifecycle: build the effects lazily on first use, release them in `release()`,
|
||||
and re-attach on `onAudioSessionIdChanged` — ExoPlayer can rebuild its audio sink
|
||||
(e.g. on a format change), which invalidates effects bound to the old session.
|
||||
|
||||
### Make the silent-failure mode impossible
|
||||
|
||||
The trait's default is the root cause of this whole class of bug:
|
||||
|
||||
```rust
|
||||
// backend.rs:85 — reports success while doing nothing
|
||||
fn set_audio_settings(&mut self, _settings: &AudioSettings) -> Result<(), PlayerError> {
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
Android inherits this, so every EQ/normalization change on Android returns `Ok`
|
||||
and silently does nothing — the UI ships and has no effect, with no error anywhere.
|
||||
|
||||
Once `ExoPlayerBackend` implements the methods, **change the trait default to
|
||||
`Err(PlayerError::not_implemented())`**, matching how `set_audio_track` /
|
||||
`set_subtitle_track` already behave. Any future backend that forgets to implement
|
||||
audio settings then fails loudly instead of lying.
|
||||
|
||||
Check the call sites before flipping it: `NullBackend` overrides both methods, so
|
||||
the graceful-degradation path is unaffected, but confirm nothing treats a
|
||||
`set_audio_settings` error as fatal to playback.
|
||||
|
||||
### Re-application on track load
|
||||
|
||||
`PlayerController` already re-pushes `AudioSettings` per track on the platforms
|
||||
that implement it; the Android path inherits that for free once the trait methods
|
||||
exist. No controller change.
|
||||
|
||||
### 🔴 Threading note
|
||||
|
||||
`setAudioSettings` is invoked from Rust on whatever thread the command lands on.
|
||||
`AudioEffect` construction must not happen on the ExoPlayer application thread
|
||||
from inside a player callback — that is the re-entrancy hazard CLAUDE.md warns
|
||||
about, and the same shape as the `AutoplayDecision` deadlock. Post the work to
|
||||
the player's handler rather than doing it inline in a listener.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- **Crossfade (UR-031 / DR-034).** Not implemented on *any* platform today, and
|
||||
architecturally blocked on MPV (single-stream audio chain; `acrossfade` needs
|
||||
two inputs). Implementing it on Android alone would invert the parity gap. It
|
||||
needs its own spec and probably two player instances.
|
||||
- True EBU R128 normalization. `LoudnessEnhancer` is a gain stage; matching
|
||||
`dynaudnorm` exactly is out of reach without a custom `AudioProcessor`.
|
||||
- Windows audio settings — see [windows-native-audio-backend.md](windows-native-audio-backend.md).
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] `ExoPlayerBackend` overrides `set_audio_settings` and `audio_settings`.
|
||||
- [ ] EQ preset change on Android audibly changes playback; setting persists across track changes and app restart.
|
||||
- [ ] Normalization toggle audibly changes level; the three presets are ordered Loud > Normal > Quiet.
|
||||
- [ ] Disabling gapless produces a gap between consecutive tracks; enabling it does not.
|
||||
- [ ] Effects are released on `release()` and survive an audio-session rebuild.
|
||||
- [ ] `requirements.md` parity matrix updated: EQ and normalization ✅ Android.
|
||||
- [ ] `bun run check` and `bun run test` pass.
|
||||
- [ ] `cargo fmt` clean, `cargo clippy` clean, `bun run test:rust` passes.
|
||||
- [ ] `bun run check:boundary` passes.
|
||||
- [ ] New requirement-implementing code carries `// TRACES:` comments.
|
||||
- [ ] `bindings.ts` regenerated if Rust types changed.
|
||||
|
||||
## Testing
|
||||
|
||||
**Rust** (`cargo test`): `set_audio_settings` stores the sanitized settings and
|
||||
`audio_settings()` returns them — assert clamping/normalisation is applied
|
||||
(crossfade clamped to 12s, band vector normalised to `EQ_BANDS.len()`). The JNI
|
||||
call itself is not unit-testable; extract the JSON serialization into a pure
|
||||
function and test that its shape matches what the Kotlin parser expects. That
|
||||
serialization contract is the part most likely to silently break.
|
||||
|
||||
**Kotlin**: the band-resampling function (10 canonical bands → N device bands) is
|
||||
pure arithmetic — extract it and unit-test it, including the degenerate cases of
|
||||
a 5-band device and a device reporting 10 bands.
|
||||
|
||||
**Manual, on device** (these are the ones that actually prove it):
|
||||
1. Set Bass Boost, play a track, confirm audible change.
|
||||
2. Toggle normalization mid-track; confirm level change without a playback stall.
|
||||
3. Queue two gapless-encoded tracks, toggle the setting, confirm the gap appears/disappears.
|
||||
4. Force a format change (44.1kHz → 48kHz track) and confirm the EQ still applies afterwards — this exercises the session-rebuild re-attach.
|
||||
|
||||
## TRACES
|
||||
|
||||
- `ExoPlayerBackend::set_audio_settings` → `// TRACES: UR-027, UR-032, UR-033 | DR-030, DR-035, DR-036`
|
||||
- Kotlin `setAudioSettings` / `applyEqualizer` / `applyNormalization` → same IDs
|
||||
- Band-resampling helper + its tests → `DR-030 | UT-xxx`
|
||||
|
||||
## Notes for the implementer
|
||||
|
||||
- Read [audio-equalizer.md](audio-equalizer.md) first — it defines the canonical
|
||||
band layout and the preset→curve rule this spec consumes. Do not redefine bands
|
||||
in Kotlin.
|
||||
- Android source edits go in `src-tauri/android/src` (canonical tree), then run
|
||||
`scripts/sync-android-sources.sh`. Never edit the `gen/` tree.
|
||||
- There is a **stale duplicate** `JellyTauPlayer.kt` (285 lines) at
|
||||
`src-tauri/android/app/src/main/java/com/dtourolle/jellytau/player/` alongside
|
||||
the real 1103-line file at `src-tauri/android/src/main/java/...`. Edit the
|
||||
latter. Consider deleting the former as a separate change.
|
||||
- A parallel Claude session may be active — `git diff` before "repairing"
|
||||
unexpected changes.
|
||||
@@ -0,0 +1,196 @@
|
||||
# Spec: Android native video — transparent-webview spike
|
||||
|
||||
**Status:** Proposed (spike — timeboxed, may conclude "not viable")
|
||||
**Requirements:** IR-004, UR-003, UR-004 → DR-001, DR-023, DR-024
|
||||
**UX spec:** n/a — no intended visual change; the video surface must land exactly where the `<video>` element is today
|
||||
**Supersedes / revises:** acts on finding 2 of [playback-backend-unification.md](playback-backend-unification.md)
|
||||
|
||||
## Summary
|
||||
|
||||
Test whether ExoPlayer's existing `SurfaceView` video path can be composited
|
||||
behind a transparent Tauri WebView on Android. If it works, Android regains
|
||||
hardware video decoding (MediaCodec) and libass-quality ASS/SSA subtitles, both
|
||||
of which the current webview path lacks. If it does not, we document why and
|
||||
delete the dead code.
|
||||
|
||||
This is a **spike**, not a feature commitment. The deliverable is a yes/no answer
|
||||
with evidence, plus either a working path behind a flag or a removal.
|
||||
|
||||
## Motivation
|
||||
|
||||
`createAdapter()` hardcodes `const effectiveKind = "html5"` and does
|
||||
`void backendKind`, discarding the `use_html5_element` value Rust computes in
|
||||
`get_player_status`. As a result:
|
||||
|
||||
- `NativePlayerAdapter` is dead code.
|
||||
- `JellyTauPlayer.kt`'s `getOrCreateSurfaceView()` — which already calls
|
||||
`setZOrderMediaOverlay(false)` and wires `setVideoSurfaceHolder` — is
|
||||
unreachable.
|
||||
- Android video decodes in the WebView instead of via MediaCodec, despite
|
||||
`CodecDetector.kt` going to the trouble of reporting hardware codec
|
||||
capabilities back to Rust for DeviceProfile generation.
|
||||
|
||||
The code comment in `nativeAdapter.ts:11-14` justifies this by citing
|
||||
tauri#10152 as an upstream blocker. **That justification is stale.**
|
||||
|
||||
### Why the blocker no longer holds
|
||||
|
||||
- tauri#10152 is open but **dead since 2024-07-01**, and it is a *feature
|
||||
request* ("Support transparent webviews on mobile"), not a bug report about
|
||||
compositing.
|
||||
- The capability shipped in tauri commit `27d01834` (2024-09-02) — a clippy
|
||||
cleanup that moved `transparent()` out of the desktop-gated impl block, fencing
|
||||
only the tao call behind `#[cfg(desktop)]`. Because it landed as unrelated
|
||||
cleanup, nobody closed the issue.
|
||||
- The black/white-screen reports (tauri#8381, tauri#9408) were a real but
|
||||
*different* bug: a broken JNI signature for `setBackgroundColor`, fixed in
|
||||
**wry 0.39.4** (PR #1237). We ship wry 0.55.x.
|
||||
- Current wry calls `setBackgroundColor(0)` unconditionally on Android when
|
||||
transparency is requested.
|
||||
|
||||
### The honest caveat
|
||||
|
||||
**Nobody has demonstrated SurfaceView-behind-WebView on Tauri Android.** A search
|
||||
of both `tauri-apps/tauri` and `tauri-apps/wry` issues for `surfaceview` returns
|
||||
zero results, and the one native-video Tauri plugin
|
||||
(`YeonV/tauri-plugin-videoplayer`) sidesteps compositing by launching a separate
|
||||
fullscreen Activity. Nothing upstream blocks this; nothing upstream proves it.
|
||||
Hence: spike, not feature.
|
||||
|
||||
Note this is the *Android* question only. The equivalent Linux compositing
|
||||
problem is maintainer-declared unfixable and is **not** in scope — see the
|
||||
unification spec.
|
||||
|
||||
## Layer assignment
|
||||
|
||||
| Logic / responsibility | Layer | Why it belongs there |
|
||||
|------------------------|-------|----------------------|
|
||||
| Which video backend this platform uses | Rust (existing) | `get_player_status` already computes `use_html5_element`. The frontend must *consume* it, not decide it. Restoring that is the point of the spike. |
|
||||
| Surface creation, z-ordering, `setVideoSurfaceHolder` lifecycle | Kotlin | Android platform mechanics; already written in `JellyTauPlayer.kt`. |
|
||||
| Seek/audio-track *strategy* | Rust (existing) | Already returned by `player_seek_video` / `player_switch_audio_track`; `NativePlayerAdapter` executes the chosen primitive. Unchanged — this is exactly what the `PlayerAdapter` contract was built for. |
|
||||
| Positioning the surface under the video viewport | Frontend | Pure presentation/layout. **This is the risk area** — see Design. |
|
||||
|
||||
## Design
|
||||
|
||||
### Phase 1 — prove compositing (no app changes)
|
||||
|
||||
Before touching the adapter factory, verify the primitive works at all:
|
||||
|
||||
1. Set `"transparent": true` in `tauri.conf.json` for the Android build, plus
|
||||
`html, body { background: transparent; }`.
|
||||
2. Confirm the WebView is genuinely transparent (a native view behind it is
|
||||
visible) and that the app does not regress to a black/white screen.
|
||||
|
||||
If this fails, stop — everything downstream is moot, and the finding is that
|
||||
Tauri Android transparency is still broken in practice despite the shipped fix.
|
||||
|
||||
### Phase 2 — un-hardcode the factory
|
||||
|
||||
```ts
|
||||
// src/lib/player/adapters/index.ts
|
||||
export function createAdapter({ backendKind, host, bridge }: CreateAdapterArgs): PlayerAdapter {
|
||||
return backendKind === "native"
|
||||
? new NativePlayerAdapter(host)
|
||||
: new Html5PlayerAdapter(host, bridge);
|
||||
}
|
||||
```
|
||||
|
||||
`backendKind` comes from `get_player_status` (`VideoBackend::Native` on Android).
|
||||
Gate behind a setting — `experimentalNativeVideo`, default **off** — so a broken
|
||||
spike cannot ship as a regression. Rust already owns this decision; the flag only
|
||||
suppresses it.
|
||||
|
||||
**Also in scope: remove the user-agent sniffing in
|
||||
`src/lib/services/webviewAudio.ts:30-41`.** It re-derives which audio backend the
|
||||
platform has from `navigator.userAgent` ("matching the Rust cfg gate", per its own
|
||||
comment) — the frontend deciding a backend fact it should be told. Same root cause
|
||||
as the hardcode above, same fix: consume the value Rust already computes. Fold it
|
||||
in here rather than leaving a second, subtler copy of the bug behind. If
|
||||
`get_player_status` does not currently expose enough to cover the audio case, add
|
||||
the field — that is backend work, and correct.
|
||||
|
||||
### Phase 3 — surface positioning
|
||||
|
||||
The hard part, and where this most likely fails. The webview's `<video>` element
|
||||
occupies a laid-out box; the `SurfaceView` must be positioned to match it, and
|
||||
kept matched through scroll, rotation, and mini-player transitions.
|
||||
|
||||
Approach: the video view reports its `getBoundingClientRect()` to Rust, which
|
||||
forwards the rect to Kotlin to position the `SurfaceView`. This is the same
|
||||
"faking it" technique the ecosystem uses on desktop — acceptable here *only if*
|
||||
the video is effectively fullscreen on Android, which it is in the player route.
|
||||
|
||||
**Explicit failure criterion**: if the surface cannot be kept aligned during
|
||||
rotation or the mini-player transition without visible artefacts, the spike fails
|
||||
and we keep HTML5. Do not ship a janky native path for a codec win.
|
||||
|
||||
### What we gain if it works
|
||||
|
||||
- **Hardware decode via MediaCodec** — `CodecDetector.kt` already reports
|
||||
capabilities; the DeviceProfile would finally match what actually plays.
|
||||
- **ASS/SSA subtitles** are *not* automatic. ExoPlayer cannot render them; that
|
||||
would require libmpv, which is a separate and much larger decision (see the
|
||||
unification spec's engine comparison). Scope this spike to hardware decode
|
||||
only, and do not claim subtitle improvements from it.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Linux native video. Maintainer-declared unfixable on WebKitGTK/Wayland.
|
||||
- Replacing ExoPlayer with libmpv on Android.
|
||||
- Windows native video.
|
||||
- Removing the HTML5 path. It stays as the default and the fallback.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
The spike is **complete** when one of these is true:
|
||||
|
||||
**Success path**
|
||||
- [ ] Transparent WebView confirmed working on a physical device.
|
||||
- [ ] `experimentalNativeVideo` off → behaviour byte-identical to today.
|
||||
- [ ] `webviewAudio.ts` no longer inspects `navigator.userAgent`; the platform's audio backend is read from Rust.
|
||||
- [ ] `experimentalNativeVideo` on → video plays via ExoPlayer/MediaCodec, correctly positioned, with working seek, audio-track switch, and subtitle selection through the existing `PlayerAdapter` contract.
|
||||
- [ ] No artefacts on rotation, background/foreground, or mini-player transition.
|
||||
- [ ] `adb shell dumpsys media.metrics` (or logcat) confirms a hardware decoder is in use.
|
||||
- [ ] Measured battery/thermal or CPU improvement over the HTML5 path on the same clip.
|
||||
|
||||
**Failure path**
|
||||
- [ ] The blocking behaviour is documented in this spec with evidence.
|
||||
- [ ] `NativePlayerAdapter` and the unreachable `SurfaceView` code are deleted, or explicitly retained with a *correct* comment.
|
||||
- [ ] `nativeAdapter.ts:11-14` no longer cites tauri#10152.
|
||||
|
||||
Either way:
|
||||
- [ ] `bun run check`, `bun run test`, `bun run check:boundary` pass.
|
||||
- [ ] `cargo fmt` / `cargo clippy` clean; `bun run test:rust` passes.
|
||||
|
||||
## Testing
|
||||
|
||||
Adapter-selection logic is pure and testable without a device: assert
|
||||
`createAdapter` returns `NativePlayerAdapter` for `backendKind: "native"` with
|
||||
the flag on, and `Html5PlayerAdapter` in every other combination — including that
|
||||
the flag off forces HTML5 even when Rust says native. That last case is the
|
||||
regression guard.
|
||||
|
||||
Everything else is manual on-device; there is no meaningful way to unit-test
|
||||
surface compositing. Test on at least two devices — compositing behaviour varies
|
||||
by OEM and Android version.
|
||||
|
||||
Per CLAUDE.md, if the spike turns into a bug fix (e.g. seek breaks under the
|
||||
native adapter), write the failing test first.
|
||||
|
||||
## TRACES
|
||||
|
||||
- `createAdapter` → `// TRACES: UR-003, UR-004 | DR-023, DR-024`
|
||||
- Adapter-selection tests → `UT-xxx`
|
||||
- No new requirement IDs; this spike either satisfies existing IR-004 expectations or documents why it cannot.
|
||||
|
||||
## Notes for the implementer
|
||||
|
||||
- **Do not skip Phase 1.** If transparency does not work, phases 2 and 3 are
|
||||
wasted effort.
|
||||
- `VideoPlayer.svelte` has a documented hazard: no lifecycle calls after an
|
||||
`await` in `onMount` — it flips to HTML5 mode and breaks Android seek. The
|
||||
adapter swap touches exactly this code path.
|
||||
- tauri-specta tagged responses keep Rust field names (`new_url`, not `newUrl`).
|
||||
- Android source edits go in `src-tauri/android/src`, then run
|
||||
`scripts/sync-android-sources.sh`.
|
||||
- A parallel Claude session may be active — `git diff` first.
|
||||
@@ -4,6 +4,7 @@
|
||||
**Requirements:** UR-027 → DR-030 (EQ UI), IR-020 (MPV EQ integration).
|
||||
**UX spec:** n/a (extends the Settings › Audio section, ux-flows §8.1 instant-apply).
|
||||
**Supersedes / revises:** —
|
||||
**Revised by:** [android-audio-settings-parity.md](android-audio-settings-parity.md) — lifts the "Android is a no-op" limitation below.
|
||||
|
||||
## Summary
|
||||
|
||||
@@ -115,6 +116,10 @@ fields ride along. `NullBackend`/Android inherit the trait default (no-op).
|
||||
## Out of scope
|
||||
|
||||
- Android/ExoPlayer EQ (parity gap tracked with crossfade/gapless/normalize).
|
||||
**Now specified in [android-audio-settings-parity.md](android-audio-settings-parity.md)**,
|
||||
which implements `set_audio_settings` on `ExoPlayerBackend`. The canonical band
|
||||
layout and preset→curve map defined here remain authoritative; the Android side
|
||||
resamples those bands onto the device equalizer rather than defining its own.
|
||||
- Per-track or per-library EQ profiles — one global profile only.
|
||||
- Automatic loudness/room correction; only manual bands + presets.
|
||||
- Changing the crossfade/normalize TODOs in `set_audio_settings` beyond wiring
|
||||
|
||||
@@ -0,0 +1,169 @@
|
||||
# Spec: Migrate to libmpv2 and declare the project licence
|
||||
|
||||
**Status:** Proposed
|
||||
**Requirements:** UR-003 → IR-003 (revises the MPV integration); no new user-facing behaviour
|
||||
**UX spec:** n/a
|
||||
**Supersedes / revises:** dependency and licensing housekeeping identified in [playback-backend-unification.md](playback-backend-unification.md)
|
||||
|
||||
## Summary
|
||||
|
||||
Two related pieces of housekeeping that block or complicate later work:
|
||||
|
||||
1. Replace the abandoned `libmpv` crate (pinned to a git branch) with the
|
||||
maintained `libmpv2`.
|
||||
2. Add a `LICENSE` file. The project has none, which leaves its legal status
|
||||
undefined while it links GPL-licensed libmpv.
|
||||
|
||||
Neither changes user-visible behaviour. Both are prerequisites for
|
||||
[windows-native-audio-backend.md](windows-native-audio-backend.md).
|
||||
|
||||
## Motivation
|
||||
|
||||
### The dependency is dead
|
||||
|
||||
```toml
|
||||
# src-tauri/Cargo.toml
|
||||
libmpv = { git = "https://github.com/ParadoxSpiral/libmpv-rs.git", branch = "master" }
|
||||
```
|
||||
|
||||
- crates.io `libmpv` 2.0.1 was published **2020-09-29**.
|
||||
- The upstream repo's last commit was **2023-01-08**; nothing since was released.
|
||||
- We pin a git *branch*, so builds are not reproducible — the same lockfile-less
|
||||
checkout can resolve differently over time, and CI has no protection if the
|
||||
branch moves or the repo disappears.
|
||||
|
||||
`libmpv2` (kohsine/libmpv2-rs) is a maintained fork of exactly this crate:
|
||||
6.0.0 released **2026-05-12**, ~23.5k recent downloads against the original's
|
||||
~1.1k, releases roughly quarterly since 2024.
|
||||
|
||||
### The project has no licence
|
||||
|
||||
There is no `LICENSE`/`COPYING` file and `src-tauri/Cargo.toml` has no `license`
|
||||
field. The project is open source and will never be commercial, so this is purely
|
||||
an omission — but it matters because we link libmpv, and "no licence" defaults to
|
||||
*all rights reserved*, which is incompatible with distributing a GPL-derived
|
||||
work.
|
||||
|
||||
## Design
|
||||
|
||||
### Part 1 — licence
|
||||
|
||||
**Use GPLv3.** This is forced, not chosen:
|
||||
|
||||
- mpv's default build is **GPLv2-or-later**, so the combined work must be
|
||||
GPL-compatible.
|
||||
- Apache-2.0 is **GPLv2-incompatible** (patent-termination and indemnification
|
||||
clauses) but GPLv3-compatible.
|
||||
- A scan of the dependency tree found Apache-2.0-**only** crates with no
|
||||
alternative arm — most importantly **`tao`** (Tauri's own windowing crate),
|
||||
plus `sync_wrapper`, `gethostname`, and `ring` (Apache-2.0 AND ISC).
|
||||
|
||||
`tao` is unavoidable in a Tauri app, so GPLv2 is unavailable. Exercising mpv's
|
||||
"or later" option puts the combination at **GPLv3**.
|
||||
|
||||
Actions:
|
||||
- Add `LICENSE` containing the GPLv3 text.
|
||||
- Add `license = "GPL-3.0-or-later"` to `src-tauri/Cargo.toml` and `license` to
|
||||
`package.json`.
|
||||
- Note in the README that the binary links libmpv (GPLv2+) and FFmpeg.
|
||||
|
||||
Because the project is open source, we use mpv's **default GPL build** — no
|
||||
`-Dgpl=false`, no LGPL FFmpeg build, and none of the LGPL §6 relinking analysis
|
||||
that a proprietary app would need. We keep VAAPI/VDPAU/X11 and every GPL FFmpeg
|
||||
filter.
|
||||
|
||||
🔴 Never build FFmpeg with `--enable-nonfree` — that produces a binary that is
|
||||
**unredistributable under any licence**, open source or not.
|
||||
|
||||
### Part 2 — libmpv → libmpv2
|
||||
|
||||
```toml
|
||||
# Linux (and later Windows, per the Windows audio spec)
|
||||
libmpv2 = "=6.0.0"
|
||||
```
|
||||
|
||||
Pin exactly: `libmpv2` has broken its API in **every** major release.
|
||||
|
||||
Breaking changes to expect, from the changelog:
|
||||
|
||||
| Version | Change | Impact here |
|
||||
|---|---|---|
|
||||
| 4.0.0 | Removed command helper methods — call `mpv.command(...)` directly | Low; we already use `command`/`set_property` |
|
||||
| 5.0.0 | Removed `mpv_node` support entirely (properties return strings; parse JSON yourself); `EventContext` folded into `Mpv`; `ProtocolContext` → `Protocol` | **Medium** — `start_event_loop` uses `create_event_context()`; check whether that call still exists |
|
||||
| 6.0.0 | `RenderContext::new()` → `Mpv::create_render_context()`; `'static` bound on `OpenGLInitParams`; render context now borrows `Mpv` (fixes a use-after-free) | **None** — we do not use the render API |
|
||||
|
||||
The last row matters: we run mpv audio-only (`video = no`), so the entire render
|
||||
surface is irrelevant to us. Consider disabling the default `render` feature to
|
||||
reduce build surface.
|
||||
|
||||
The main porting work is the event loop in `mpv_backend.rs` — `wait_event`,
|
||||
`disable_deprecated_events`, and the `FileLoaded` / `PlaybackRestart` /
|
||||
`PropertyChange` / `EndFile` handling, given 5.0.0 folded `EventContext` into
|
||||
`Mpv`.
|
||||
|
||||
Everything else — `set_property` calls, the `af` filter graph, the 250ms position
|
||||
thread, the seek-suppression window — should port unchanged.
|
||||
|
||||
## Layer assignment
|
||||
|
||||
No logic moves. This is a dependency swap plus a licence file; the
|
||||
`PlayerBackend` trait boundary is untouched.
|
||||
|
||||
| Logic / responsibility | Layer | Why it belongs there |
|
||||
|------------------------|-------|----------------------|
|
||||
| mpv event → `PlayerStatusEvent` mapping | Rust (unchanged) | Already correct; only the binding API beneath it changes. |
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Any behaviour change. If playback behaves differently after this, that is a bug.
|
||||
- Windows support — separate spec, but this must land first.
|
||||
- Adopting the render API. We are audio-only on mpv.
|
||||
- Re-licensing decisions beyond adding the file the project already implies.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] `LICENSE` (GPLv3) present; `license` field set in `Cargo.toml` and `package.json`.
|
||||
- [ ] A full dependency-licence audit has been run (`cargo install cargo-license && cargo license`) and confirms no GPLv3-incompatible dependency. *(The scan behind this spec resolved 441 of 575 crates from the local registry cache; the remaining 134 are unverified.)*
|
||||
- [ ] `libmpv` git dependency removed; `libmpv2` pinned to an exact version.
|
||||
- [ ] Linux audio playback works identically: play/pause/seek/volume, queue advance, gapless, EQ, normalization, sleep timer.
|
||||
- [ ] Position updates still arrive at 250ms; the 150ms post-seek suppression still prevents the jump-to-zero glitch.
|
||||
- [ ] `EndFile` still emits `PlaybackEnded` only for EOF (not STOP/QUIT/ERROR) — autoplay depends on this.
|
||||
- [ ] Builder image updated if the libmpv dev package requirement changed; **no toolchain install added to any CI step**.
|
||||
- [ ] `bun run check`, `bun run test`, `bun run check:boundary` pass.
|
||||
- [ ] `cargo fmt` clean, `cargo clippy` clean, `bun run test:rust` passes.
|
||||
|
||||
## Testing
|
||||
|
||||
The existing `mpv_backend_test.rs` plus the `build_af_filter`,
|
||||
`eq_filter_entries`, and `normalize_filter_entry` tests are the regression net —
|
||||
they must pass unchanged, since none of them touch the binding API.
|
||||
|
||||
The event loop has no unit tests and is where the risk concentrates. Verify
|
||||
manually on Linux:
|
||||
|
||||
1. Play → pause → play; confirm position does not flash to 0:00 (the known
|
||||
playing-event regression).
|
||||
2. Seek mid-track; confirm no jump-to-zero within 150ms.
|
||||
3. Let a track end naturally; confirm autoplay advances (exercises `EndFile` EOF).
|
||||
4. Press stop; confirm autoplay does **not** advance.
|
||||
5. Sleep-timer expiry; confirm it stops without triggering autoplay.
|
||||
|
||||
Cases 3–5 are the ones most likely to break silently, and each corresponds to a
|
||||
bug already fixed once in this codebase.
|
||||
|
||||
## TRACES
|
||||
|
||||
- `MpvBackend` construction / event loop → existing `// TRACES: UR-003 | IR-003`, unchanged
|
||||
- No new requirement IDs; this is a dependency migration.
|
||||
|
||||
## Notes for the implementer
|
||||
|
||||
- Do this **before** the Windows audio backend.
|
||||
- Read the 4.0/5.0/6.0 changelogs before writing code — the crate has broken API
|
||||
in every major release, most recently two months before this spec.
|
||||
- The crates.io `repository` field for `libmpv2` points at `kohsine/libmpv-rs`,
|
||||
but the repo was renamed to **`libmpv2-rs`**; the old raw URLs 404.
|
||||
- `libmpv2-sys` ships pregenerated bindings and vendored headers, so no libclang
|
||||
is needed at build time — relevant to keeping the builder image thin.
|
||||
- A parallel Claude session may be active — `git diff` before "repairing"
|
||||
unexpected changes.
|
||||
@@ -0,0 +1,226 @@
|
||||
# Spec: Playback backend unification — findings and strategy
|
||||
|
||||
**Status:** Accepted (analysis; no code changes)
|
||||
**Requirements:** IR-004, UR-031, UR-032, UR-033 — revises the "Platform Playback Backend Parity" issue in requirements.md
|
||||
**UX spec:** n/a
|
||||
**Supersedes / revises:** informs [android-native-video-spike.md](android-native-video-spike.md), [android-audio-settings-parity.md](android-audio-settings-parity.md), [windows-native-audio-backend.md](windows-native-audio-backend.md)
|
||||
|
||||
## Summary
|
||||
|
||||
This spec records the outcome of an investigation into unifying JellyTau's
|
||||
playback backends (Linux/MPV, Android/ExoPlayer, Windows/webview) onto a single
|
||||
engine with hardware acceleration everywhere. **The conclusion is that video
|
||||
cannot be unified onto a native engine, and should not be attempted.** Audio
|
||||
*can* be, and that is where the remaining specs direct effort.
|
||||
|
||||
No code changes follow from this spec directly. It exists so the decision is
|
||||
written down with its evidence, and so a future session does not re-run the same
|
||||
investigation.
|
||||
|
||||
## Motivation
|
||||
|
||||
The requirements doc carries a "Platform Playback Backend Parity" issue noting
|
||||
that audio settings work on Linux but not Android, and proposing eventual
|
||||
convergence. The natural next question — "should we just run one engine
|
||||
everywhere?" — needed answering before spending effort on per-backend patches.
|
||||
|
||||
The investigation also surfaced that several statements in requirements.md and in
|
||||
code comments are factually wrong. Those corrections are part of the deliverable.
|
||||
|
||||
## Findings
|
||||
|
||||
### 1. The current architecture is not what the docs describe
|
||||
|
||||
| Platform | Audio | Video |
|
||||
|----------|-------|-------|
|
||||
| Linux | MPV (native, **audio-only**) | webview `<video>` + hls.js |
|
||||
| Android | ExoPlayer (native) | **webview `<video>` + hls.js** |
|
||||
| Windows | webview `<audio>` | webview `<video>` + hls.js |
|
||||
|
||||
Two surprises:
|
||||
|
||||
- **MPV never decodes video.** `mpv_backend.rs` sets `video = no` and
|
||||
`audio-display = no` at construction. Linux video has always been the webview.
|
||||
Correspondingly, `player_play_item` deliberately does *not* load into MPV on
|
||||
Linux (it calls `set_current_item`, which only updates the queue).
|
||||
- **Android video is also the webview.** `createAdapter()` in
|
||||
`src/lib/player/adapters/index.ts` hardcodes `const effectiveKind = "html5"`
|
||||
and does `void backendKind`, discarding the `use_html5_element` signal that
|
||||
`get_player_status` computes in Rust. `NativePlayerAdapter` is dead code, and
|
||||
ExoPlayer's `SurfaceView` path in `JellyTauPlayer.kt` is unreachable.
|
||||
|
||||
So video is *already* unified — on HTML5, everywhere, by accident of that
|
||||
hardcode — and on the path without hardware decoding on Android.
|
||||
|
||||
### 2. Native video cannot be composited with a Tauri webview
|
||||
|
||||
This is the load-bearing finding. It is **not** an mpv limitation; it defeats
|
||||
every candidate engine identically:
|
||||
|
||||
- **mpv**: `tauri-plugin-libmpv`'s own platform table reads Linux ⚠️
|
||||
*"Experimental. Window embedding is not working."*
|
||||
- **GStreamer** (wry discussion #284, 2024): *"Gstreamer was rendering above the
|
||||
surface and covering all html elements."*
|
||||
- **libVLC** (tauri discussion #6343, 2024): *"I had to render the webview in a
|
||||
child window though because vlc kept rendering on top of it."*
|
||||
|
||||
Root cause, from Tauri maintainer amrbashir (tauri#9220, 2024-03-30):
|
||||
|
||||
> "we are limited to using Webkit2GTK on Linux and that requires a GTK window.
|
||||
> While possible to add a GTK widget as a child X11 window inside raw X11 window,
|
||||
> this is however a bit hacky and **it is not possible on Wayland at all**."
|
||||
|
||||
WebKitGTK, WebView2, and Android WebView each draw into their own compositor
|
||||
surface. A native video surface is either entirely above or entirely below the
|
||||
webview; it cannot interleave with HTML. Every working example in the ecosystem
|
||||
is the same hack — a separate child window position-synced to a
|
||||
`getBoundingClientRect()` div — which breaks on resize, scroll, and any UI drawn
|
||||
over the video. For JellyTau that means the controls, subtitle overlay, and
|
||||
mini-player.
|
||||
|
||||
The most recent comment on tauri#6343 (2026-05-23) confirms it is still unsolved:
|
||||
|
||||
> "I'm faking it and the window is not truly embedded, basically when the parent
|
||||
> moves or resizes I reset the position and size of the libmpv window to align it
|
||||
> with an HTML div."
|
||||
|
||||
**The principle to carry forward: audio can unify on a native engine; video
|
||||
cannot, because video needs a surface and the webview owns the surface.**
|
||||
|
||||
### 3. mpv would regress streaming quality
|
||||
|
||||
mpv has **no adaptive bitrate**. It delegates HLS to FFmpeg's demuxer, which
|
||||
selects one variant at open time and never adapts; mpv#3548 (2016) requested ABR
|
||||
and it never landed. `--hls-bitrate` is a static picker defaulting to `max`.
|
||||
|
||||
The webview path already has real ABR via hls.js. Moving video to mpv would be a
|
||||
**downgrade** on every platform — no graceful degradation on weak networks, and
|
||||
quality changes requiring teardown and reload.
|
||||
|
||||
### 4. Crossfade is architecturally blocked on mpv
|
||||
|
||||
mpv's audio chain is single-stream. FFmpeg's `acrossfade` is an `N→A` filter
|
||||
requiring two input streams, so there is no second input to feed it. Real
|
||||
crossfade needs **two libmpv instances** with manually ramped volumes. Upstream
|
||||
maintainer response (mpv#4512, closed three minutes after opening):
|
||||
|
||||
> "No. I also find crossfading stupid and complex, so the likeliness of that
|
||||
> happening is low."
|
||||
|
||||
GStreamer *could* do it via `audiomixer`. mpv cannot, at any reasonable cost.
|
||||
|
||||
### 5. Engine comparison summary
|
||||
|
||||
| Criterion | mpv | GStreamer | libVLC |
|
||||
|-----------|-----|-----------|--------|
|
||||
| Webview compositing | ❌ Linux broken | ❌ same wall | ❌ same wall |
|
||||
| Adaptive bitrate HLS | ❌ none | ✅ adaptivedemux2 | ✅ adaptive module |
|
||||
| Rust bindings | ⚠️ `libmpv2` active; our pin is dead | ✅ `gstreamer-rs` excellent | ❌ `vlc-rs` abandoned (2018) |
|
||||
| Windows cross-MSVC | ⚠️ prebuilt DLL | ❌ pkg-config vs cargo-xwin | ❌ no better |
|
||||
| Android packaging | ✅ Maven AAR (used by Findroid) | ⚠️ Cerbero/NDK, painful | ✅ mature AAR |
|
||||
| ASS/SSA subtitles | ✅ libass built in | ✅ libass | ✅ libass |
|
||||
| Crossfade | ❌ impossible | ✅ `audiomixer` | ⚠️ unclear |
|
||||
|
||||
Every candidate fails the first row, which is the disqualifying one.
|
||||
|
||||
### 6. Two further options ruled out
|
||||
|
||||
**Webview `<audio>`/`<video>` everywhere** (i.e. delete the native audio backends
|
||||
too) is dead on Android: `navigator.mediaSession` is *deliberately compiled out*
|
||||
of Android WebView (Chromium CL 2613133003), so lockscreen/media-notification
|
||||
control would be impossible. Chromium has also never shipped `audioTracks`. It
|
||||
remains fine for Windows *video*, which is what we already do.
|
||||
|
||||
**FFmpeg-direct / Rust-native** (`ffmpeg-next`, `rsmpeg`, Symphonia) is not
|
||||
close: the safe bindings do not expose hardware decode at all, `ffmpeg-next` is
|
||||
self-declared maintenance-only, and Symphonia lacks HE-AAC and gapless AAC. This
|
||||
is a multi-person-year path to reach parity with what we already have.
|
||||
|
||||
### 7. If libmpv is ever revisited on Android
|
||||
|
||||
Recorded so the next investigation starts from evidence rather than repeating the
|
||||
search. The `dev.jdtech.mpv:libmpv` AAR — maintained by Findroid's author, i.e.
|
||||
another Jellyfin Android client — was inspected directly:
|
||||
|
||||
- `libmpv.so` exports the full 54-function `mpv_*` C API with **zero `Java_`
|
||||
symbols**; JNI is a separate optional ~19 KB `libplayer.so`. So it is drivable
|
||||
from Rust without a Java shim. (This is precisely what disqualifies libVLC,
|
||||
whose Android video path hard-requires a Java `AWindow` jobject.)
|
||||
- ~23 MB/ABI, versus libVLC's ~46 MB/ABI.
|
||||
- 🔴 **The published AAR is built `--enable-gpl --enable-version3` — it is
|
||||
GPLv3**, not LGPL. Fine for us (see [libmpv2-migration.md](libmpv2-migration.md)),
|
||||
but it would be a hard constraint for anyone shipping closed source, and an
|
||||
LGPL rebuild would be your own build to own.
|
||||
- Top unverified risk if anyone tries this: whether `libmpv2-sys` can
|
||||
cross-compile for `aarch64-linux-android` against that prebuilt `.so`. No
|
||||
working example of `libmpv2` on Android was found.
|
||||
|
||||
None of this changes the verdict — the cost is the MediaSession/foreground-service
|
||||
rewrite, not the bindings.
|
||||
|
||||
## Decision
|
||||
|
||||
1. **Do not unify video onto a native engine.** Video stays in the webview with
|
||||
hls.js on all platforms. This is not a compromise — it is the configuration
|
||||
that falls out of the compositing constraint, and it is the only one that
|
||||
gives us ABR for free.
|
||||
2. **Android native video is worth a bounded spike anyway** — not for
|
||||
unification, but because ExoPlayer's `SurfaceView` path already exists and
|
||||
would restore hardware decode plus ASS/SSA subtitles. See
|
||||
[android-native-video-spike.md](android-native-video-spike.md).
|
||||
3. **Audio parity is the real gap** and is achievable without touching any of the
|
||||
above. See [android-audio-settings-parity.md](android-audio-settings-parity.md)
|
||||
and [windows-native-audio-backend.md](windows-native-audio-backend.md).
|
||||
4. **Migrate the dead libmpv pin** regardless of any of this. See
|
||||
[libmpv2-migration.md](libmpv2-migration.md).
|
||||
|
||||
## Corrections to existing docs
|
||||
|
||||
These are factual errors found during the investigation. Fixing them is in scope
|
||||
for this spec.
|
||||
|
||||
| Location | Says | Actually |
|
||||
|----------|------|----------|
|
||||
| `requirements.md` UR-031 (line ~44) | "Done (Linux only)" | Not implemented on any platform. |
|
||||
| `requirements.md` DR-034 (line ~196) | "Done (Linux only)" | Not implemented anywhere — `mpv_backend.rs` has a bare `// TODO: Implement crossfade via MPV audio filters if needed`. Architecturally blocked on mpv (finding 4). |
|
||||
| `requirements.md` parity matrix | Crossfade ✅ Linux / ❌ Android | ❌ / ❌ |
|
||||
| `requirements.md` parity matrix | (no EQ row) | EQ is also Linux-only — `build_af_filter`/`eq_filter_entries` exist only in `mpv_backend.rs`. Same root cause, same fix. |
|
||||
| `nativeAdapter.ts:11-14` | Native Android video "blocked upstream by tauri#10152" | tauri#10152 is a stale *feature request*, dead since 2024-07-01. The capability shipped in tauri commit `27d01834` (2024-09-02). Not a blocker. |
|
||||
|
||||
## Layer assignment
|
||||
|
||||
No new logic. The one boundary observation worth recording:
|
||||
|
||||
| Logic / responsibility | Layer | Why it belongs there |
|
||||
|------------------------|-------|----------------------|
|
||||
| Which video backend a platform uses (`use_html5_element`) | Rust | Already correctly computed in `get_player_status`. The frontend currently *discards* it — that is the bug, not the design. Restoring it means the frontend consumes a backend decision rather than making its own. |
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Any code change. This spec is analysis; the sibling specs carry the work.
|
||||
- iOS/macOS. Not current targets.
|
||||
- Replacing hls.js.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] `requirements.md` DR-034 status corrected; parity matrix updated (crossfade ❌/❌, EQ row added).
|
||||
- [ ] Stale tauri#10152 comment in `nativeAdapter.ts` corrected.
|
||||
- [ ] The four sibling specs exist and are linked from here.
|
||||
|
||||
## Testing
|
||||
|
||||
n/a — documentation only.
|
||||
|
||||
## TRACES
|
||||
|
||||
No new code. Requirement text changes only; DR-034's status line is the one
|
||||
substantive edit.
|
||||
|
||||
## Notes for the implementer
|
||||
|
||||
- The evidence above was gathered in July 2026. The compositing constraint has
|
||||
been stable since 2021 (wry#284) and is maintainer-declared unfixable, so it is
|
||||
unlikely to change soon — but if someone revisits this, tauri#6343 and wry#284
|
||||
are the threads to re-read first.
|
||||
- A parallel Claude session may be active in this repo — `git diff` before
|
||||
"repairing" unexpected changes.
|
||||
@@ -0,0 +1,198 @@
|
||||
# Spec: Playback documentation corrections
|
||||
|
||||
**Status:** Proposed
|
||||
**Requirements:** revises the status of DR-034; corrects the parity matrix in [requirements.md](../requirements.md)
|
||||
**UX spec:** n/a
|
||||
**Supersedes / revises:** implements the "Corrections to existing docs" section of [playback-backend-unification.md](playback-backend-unification.md)
|
||||
|
||||
## Summary
|
||||
|
||||
Fix four factual errors in the requirements doc and the player source comments,
|
||||
all found while investigating backend unification. Each claims something the code
|
||||
does not do. Small change, but they are actively misleading: two of them assert a
|
||||
feature is implemented when it is implemented nowhere, and one cites an upstream
|
||||
blocker that no longer exists.
|
||||
|
||||
Documentation and comments only — no behaviour change.
|
||||
|
||||
## Motivation
|
||||
|
||||
These errors compound. DR-034 reads "Done (Linux only)", so a future session
|
||||
planning Android parity would reasonably assume crossfade exists on Linux and
|
||||
only needs porting — when in fact it is unimplemented everywhere *and*
|
||||
architecturally blocked on the engine it supposedly runs on. Likewise the
|
||||
tauri#10152 comment has been discouraging work on Android native video since the
|
||||
upstream capability shipped in September 2024.
|
||||
|
||||
## The corrections
|
||||
|
||||
### 1. DR-034 status is wrong
|
||||
|
||||
`requirements.md` line ~196:
|
||||
|
||||
```
|
||||
| DR-034 | Crossfade engine with configurable duration (0-12s) | Player | UR-031 | Done (Linux only) |
|
||||
```
|
||||
|
||||
The code:
|
||||
|
||||
```rust
|
||||
// src-tauri/src/player/mpv_backend.rs, in set_audio_settings
|
||||
// TODO: Implement crossfade via MPV audio filters if needed
|
||||
```
|
||||
|
||||
That is the entire crossfade implementation. `crossfade_duration` is plumbed
|
||||
through `AudioSettings` and clamped to 0–12s, but no backend ever acts on it.
|
||||
|
||||
**Change to:** `Not implemented (blocked on MPV — see playback-backend-unification.md)`
|
||||
|
||||
Worth stating *why* in the requirements entry, because it is not a scheduling
|
||||
gap: mpv's audio chain is single-stream, and FFmpeg's `acrossfade` is an `N→A`
|
||||
filter needing two inputs. Real crossfade requires two libmpv instances with
|
||||
manually ramped volumes. Upstream declined the feature (mpv#4512).
|
||||
|
||||
### 1b. UR-031 status is wrong for the same reason
|
||||
|
||||
`requirements.md` line ~44:
|
||||
|
||||
```
|
||||
| UR-031 | Crossfade between audio tracks | Low | Done (Linux only) |
|
||||
```
|
||||
|
||||
Same error one level up: the *user* requirement is also marked done. Since no
|
||||
backend implements crossfade, UR-031 is not satisfied on any platform.
|
||||
|
||||
**Change to:** `Not implemented (blocked — see DR-034)`
|
||||
|
||||
Note line ~517 of the same file (`UR-031 (Crossfade), UR-032 (Gapless),
|
||||
UR-033 (Normalization) only work on Linux`) inherits the error — crossfade works
|
||||
nowhere, so it should read UR-032/UR-033 only.
|
||||
|
||||
### 2. Parity matrix crossfade row is wrong
|
||||
|
||||
```
|
||||
| Crossfade | ✅ | ❌ | Gap |
|
||||
```
|
||||
|
||||
**Change to** `| Crossfade | ❌ | ❌ | Not implemented |` — it is not a
|
||||
platform-parity gap, it is an unbuilt feature.
|
||||
|
||||
### 3. Parity matrix is missing the equalizer
|
||||
|
||||
The matrix lists crossfade, gapless, and normalization but omits the EQ, which
|
||||
has the same Linux-only shape and the same root cause (`ExoPlayerBackend` not
|
||||
overriding `set_audio_settings`). `build_af_filter` and `eq_filter_entries` exist
|
||||
only in `mpv_backend.rs`; there is no equalizer code in the Android tree.
|
||||
|
||||
**Add:** `| Equalizer (10-band) | ✅ | ❌ | Gap |`
|
||||
|
||||
### 4. `nativeAdapter.ts` cites a stale blocker
|
||||
|
||||
`src/lib/player/adapters/nativeAdapter.ts:11-14` states native Android video is
|
||||
blocked upstream by tauri#10152 (transparent webview / SurfaceView compositing).
|
||||
|
||||
tauri#10152 is open but **dead since 2024-07-01**, and it is a *feature request*
|
||||
that `WebviewWindowBuilder::transparent` was desktop-only — not a report that
|
||||
compositing is broken. The capability shipped in tauri commit `27d01834`
|
||||
(2024-09-02), which moved `transparent()` into the cross-platform impl block with
|
||||
only the tao call `#[cfg(desktop)]`-fenced. It landed as a clippy cleanup, so the
|
||||
issue was never closed. Separately, the black/white-screen bug (tauri#8381,
|
||||
tauri#9408) was a broken JNI signature for `setBackgroundColor`, fixed in wry
|
||||
0.39.4; we ship wry 0.55.x.
|
||||
|
||||
**Change to:** a comment stating the adapter is currently unreachable because
|
||||
`createAdapter` hardcodes the HTML5 kind, that transparency is no longer an
|
||||
upstream blocker, and that
|
||||
[android-native-video-spike.md](android-native-video-spike.md) tracks whether
|
||||
SurfaceView compositing actually works. Be explicit that *nobody has
|
||||
demonstrated* SurfaceView-behind-WebView on Tauri Android — nothing upstream
|
||||
blocks it, and nothing upstream proves it.
|
||||
|
||||
### 5. Platform capability is signalled three incompatible ways
|
||||
|
||||
Not a doc error — a real inconsistency found during the same investigation, worth
|
||||
recording here even though fixing it needs its own change.
|
||||
|
||||
Which backend a platform uses is currently expressed three ways:
|
||||
|
||||
1. Rust `#[cfg]` gates in `player/mod.rs` and `create_player_backend` — the truth.
|
||||
2. The `useHtml5Element` / `VideoBackend` value from `get_player_status` — which
|
||||
the frontend discards (see the spike spec).
|
||||
3. **Frontend user-agent sniffing** in `src/lib/services/webviewAudio.ts:30-41`:
|
||||
|
||||
```ts
|
||||
const ua = navigator.userAgent.toLowerCase();
|
||||
const isAndroid = ua.includes("android");
|
||||
const isLinux = ua.includes("linux") && !isAndroid;
|
||||
return !isAndroid && !isLinux;
|
||||
```
|
||||
|
||||
The comment says it is "matching the Rust cfg gate" — i.e. the frontend
|
||||
re-derives a backend decision from the user-agent string and hopes it stays in
|
||||
sync. That is the frontend deciding *which backend exists*, which is domain
|
||||
knowledge, not presentation. It also breaks silently the moment a new target is
|
||||
added or a webview's UA changes.
|
||||
|
||||
**This is a boundary leak of the same family the spec-review checklist exists to
|
||||
catch**, even though `check:boundary`'s tripwire (item-type arrays) does not
|
||||
match it. Rust already computes the answer; the frontend should consume it.
|
||||
|
||||
Not fixed by this spec — it is behavioural, not documentation. It should be
|
||||
folded into the spike spec's factory rework, where the same
|
||||
"consume Rust's decision instead of re-deriving it" change is already in scope.
|
||||
|
||||
### Also worth fixing while here
|
||||
|
||||
`requirements.md` IR-004 reads "In Progress (basic playback works, audio settings
|
||||
missing)". That stays accurate until
|
||||
[android-audio-settings-parity.md](android-audio-settings-parity.md) lands, but
|
||||
the "Future Fix" list in the parity issue proposes
|
||||
`ConcatenatingMediaSource` for crossfade — **deprecated in current Media3**. Drop
|
||||
that suggestion; the modern approach is a custom `AudioProcessor`.
|
||||
|
||||
## Layer assignment
|
||||
|
||||
No logic. Documentation and comments only.
|
||||
|
||||
| Logic / responsibility | Layer | Why it belongs there |
|
||||
|------------------------|-------|----------------------|
|
||||
| — | — | No logic introduced or moved by this spec. |
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Implementing crossfade. This spec only stops claiming it exists.
|
||||
- Implementing Android audio settings — see the parity spec.
|
||||
- Running the Android video spike — see that spec.
|
||||
- Rewriting the architecture docs. `docs/architecture/05-platform-backends.md`
|
||||
should be re-read for the same class of error, but that is a larger pass.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] DR-034 status corrected, with the blocking reason stated.
|
||||
- [ ] UR-031 status corrected (line ~44), and the "only work on Linux" line (~517) no longer lists crossfade.
|
||||
- [ ] Parity matrix: crossfade ❌/❌; equalizer row added.
|
||||
- [ ] `ConcatenatingMediaSource` suggestion removed from the "Future Fix" list.
|
||||
- [ ] `nativeAdapter.ts` comment corrected and pointing at the spike spec.
|
||||
- [ ] `bun run check` and `bun run test` pass (a comment change still touches TS).
|
||||
- [ ] `bun run traces:markdown` re-run if requirement text changed.
|
||||
|
||||
No Rust changes, so the `cargo` gates do not apply.
|
||||
|
||||
## Testing
|
||||
|
||||
None beyond the standard gates — no behaviour changes. Confirm
|
||||
`bun run traces:markdown` regenerates cleanly, since DR-034's row is referenced
|
||||
by the traceability matrix.
|
||||
|
||||
## TRACES
|
||||
|
||||
No code implementing requirements changes; no TRACES comments to add or update.
|
||||
The DR-034 row in `docs/traceability.md` will regenerate with the corrected text.
|
||||
|
||||
## Notes for the implementer
|
||||
|
||||
- Do **not** silently delete DR-034. The requirement (UR-031 crossfade) is still
|
||||
wanted; it is the *status* that is wrong. Keeping the row with an honest status
|
||||
and a reason is the point.
|
||||
- A parallel Claude session may be active — `git diff` before "repairing"
|
||||
unexpected changes.
|
||||
@@ -0,0 +1,209 @@
|
||||
# Spec: Windows native audio backend
|
||||
|
||||
**Status:** Proposed
|
||||
**Requirements:** UR-003, UR-027, UR-032, UR-033 → DR-030, DR-035, DR-036; new IR-030
|
||||
**UX spec:** n/a — Settings › Audio already renders the controls
|
||||
**Supersedes / revises:** acts on the "audio can unify, video cannot" conclusion in [playback-backend-unification.md](playback-backend-unification.md)
|
||||
|
||||
## Summary
|
||||
|
||||
Give Windows a real native audio backend instead of the current webview
|
||||
`<audio>` shim. Windows is the only platform where audio playback has no decoder
|
||||
of its own: `WebviewAudioBackend` hands a URL to a frontend `<audio>` element and
|
||||
relays transport commands. It cannot set volume, cannot apply any audio setting,
|
||||
and reports state only via DOM events.
|
||||
|
||||
Audio needs no rendering surface, so **none of the webview-compositing problems
|
||||
that block unified video apply here.** This is the cleanest available win.
|
||||
|
||||
## Motivation
|
||||
|
||||
`WebviewAudioBackend` was a deliberate stopgap ("audio-only playback for
|
||||
platforms without a native audio backend"), and it works — but it has a hard
|
||||
functional gap. From `webview_audio_backend.rs`:
|
||||
|
||||
```rust
|
||||
fn set_volume(&mut self, volume: f32) -> Result<(), PlayerError> {
|
||||
// ...stores locally only; there is no ControlCommand action for volume
|
||||
}
|
||||
```
|
||||
|
||||
So volume changes never reach the element; the frontend has to observe the player
|
||||
store and apply volume itself. `set_audio_settings` likewise stores values that
|
||||
nothing consumes — EQ, normalization, and gapless are all inert on Windows.
|
||||
|
||||
Meanwhile the backend-unification investigation established that a native *audio*
|
||||
engine is unproblematic on Windows specifically: `tauri-plugin-libmpv` lists
|
||||
Windows as its **fully tested** platform (in contrast to Linux, where embedding
|
||||
is broken — but that is a *video surface* problem, which audio does not have).
|
||||
|
||||
## Layer assignment
|
||||
|
||||
| Logic / responsibility | Layer | Why it belongs there |
|
||||
|------------------------|-------|----------------------|
|
||||
| Decoding and playing the audio stream | Rust | Playback is domain logic; every other platform already decodes in Rust or a native player. The webview shim is the anomaly. |
|
||||
| Applying `AudioSettings` (EQ/normalize/gapless) | Rust | Same `AudioSettings` contract as MPV/ExoPlayer; band layout and presets stay canonical in `settings.rs`. |
|
||||
| Position/state reporting | Rust | Restores the project's core principle — the player is the authoritative source of state. Today Windows inverts this: the DOM element is authoritative and Rust mirrors it. |
|
||||
| Volume | Rust | Currently broken precisely because it is split across the boundary. |
|
||||
| Rendering the player UI | Frontend | Unchanged. |
|
||||
|
||||
The strongest argument for this change is the third row. CLAUDE.md states
|
||||
playback state is one-directional with the player authoritative; on Windows that
|
||||
is currently false, and the `player_report_*` round-trip exists to paper over it.
|
||||
|
||||
## Design
|
||||
|
||||
### Engine choice
|
||||
|
||||
Two viable options; **libmpv is recommended** for consistency with the Linux
|
||||
audio backend.
|
||||
|
||||
| | libmpv | GStreamer |
|
||||
|---|---|---|
|
||||
| Windows status | ✅ `tauri-plugin-libmpv` reports fully tested | ✅ works, but… |
|
||||
| Rust bindings | `libmpv2` 6.0.0, active | `gstreamer-rs` 0.25.x, excellent |
|
||||
| Cross-MSVC from Linux | ⚠️ needs prebuilt DLL + import lib | ❌ `gstreamer-sys` uses pkg-config, fights `cargo-xwin` |
|
||||
| Code reuse | ✅ `MpvBackend` logic is directly reusable | ❌ a second engine to learn |
|
||||
| Crossfade capable | ❌ single-stream chain | ✅ `audiomixer` |
|
||||
|
||||
libmpv wins on reuse: `MpvBackend`'s `set_audio_settings` — the `af` lavfi graph
|
||||
built by `build_af_filter`, `eq_filter_entries`, `normalize_filter_entry` — is
|
||||
platform-independent and would apply unchanged.
|
||||
|
||||
The one reason to prefer GStreamer is crossfade (UR-031), which mpv structurally
|
||||
cannot do. If crossfade becomes a priority, revisit; it would then argue for
|
||||
GStreamer on *both* Linux and Windows, which is a much larger change.
|
||||
|
||||
### Structure
|
||||
|
||||
Rename the cfg gate so `MpvBackend` is no longer Linux-only:
|
||||
|
||||
```rust
|
||||
// src-tauri/src/player/mod.rs
|
||||
#[cfg(any(target_os = "linux", target_os = "windows"))]
|
||||
pub mod mpv_backend;
|
||||
```
|
||||
|
||||
`MpvBackend::new` needs one platform-specific branch: `detect_audio_system()`
|
||||
currently probes `pactl`/`pw-cli`/`/proc/asound/cards` to pick an `ao`. On
|
||||
Windows the equivalent is `wasapi` (mpv's default), so the detection is a
|
||||
`#[cfg]` returning `"wasapi"` — no probing needed.
|
||||
|
||||
Everything else — the event loop, the 250ms position thread, the seek-suppression
|
||||
window, the `af` filter graph — is unchanged.
|
||||
|
||||
`WebviewAudioBackend` stays for other targets (macOS and anything else hitting
|
||||
the `not(any(...))` arm) and as the fallback if libmpv fails to initialize. The
|
||||
existing `emit_backend_init_failed` path already handles that gracefully.
|
||||
|
||||
### Build
|
||||
|
||||
`libmpv2-sys` is well-suited to cross-compilation: no pkg-config, vendored
|
||||
headers, pregenerated bindings (no libclang). It emits `cargo:rustc-link-lib=mpv`
|
||||
unconditionally, so the build must supply a linkable import library for
|
||||
`x86_64-pc-windows-msvc`.
|
||||
|
||||
Keep the `build_libmpv` feature **off** — its Unix path shells out to mpv-build
|
||||
and explicitly rejects cross-compilation.
|
||||
|
||||
🔴 Per CLAUDE.md, the prebuilt libmpv **must be added to the builder image**
|
||||
(`Dockerfile.builder` → rebuild + push via `scripts/build-builder-image.sh`), not
|
||||
installed at CI job time. `libmpv-2.dll` must also be bundled into the NSIS
|
||||
installer via `tauri.conf.json`'s resources.
|
||||
|
||||
### Verified build mechanics
|
||||
|
||||
The cross-compile path was tested hands-on from Linux (July 2026), not inferred:
|
||||
|
||||
- Neither shinchiro nor zhongfly ships an `mpv.def` or MSVC `mpv.lib` — only a
|
||||
MinGW `libmpv.dll.a`. (Several online sources claim otherwise; they are wrong.)
|
||||
- An MSVC-style import lib can be generated locally with LLVM tools only:
|
||||
`llvm-readobj --coff-exports libmpv-2.dll` → synthesize `mpv.def` →
|
||||
`llvm-dlltool -m i386:x86-64 -d mpv.def -l mpv.lib`. `llvm-lib /def:` produces a
|
||||
byte-identical result.
|
||||
- A real `lld-link` link against that import lib **succeeds**, and the resulting
|
||||
import table resolves `mpv_client_api_version` from `libmpv-2.dll`. `lld-link`
|
||||
is the linker `cargo-xwin` uses, so this is the load-bearing step.
|
||||
- Linking directly against the shipped MinGW `libmpv.dll.a` **also** succeeds, so
|
||||
def-generation may be skippable — but that relies on lld's GNU-archive
|
||||
tolerance rather than a documented contract. Keep `llvm-dlltool` as the
|
||||
fallback.
|
||||
- MinGW origin is not an ABI problem: libmpv exports a pure C ABI, and the x86-64
|
||||
Windows calling convention is platform-defined. The upstream note that MSVC
|
||||
cannot *build* mpv is frequently misread as "MSVC cannot *link* libmpv" — that
|
||||
is not what it says.
|
||||
- 🔴 Never free/realloc across the DLL boundary — use `mpv_free`.
|
||||
|
||||
Build wiring is ordinary: `cargo:rustc-link-lib=dylib=mpv` plus
|
||||
`cargo:rustc-link-search`. Nothing about libmpv conflicts with `cargo-xwin`.
|
||||
|
||||
### Size and shipping
|
||||
|
||||
Measured uncompressed: **93 MiB** (zhongfly `mpv-dev-lgpl-x86_64`) vs **112 MiB**
|
||||
(shinchiro, full GPL build); ~26–30 MB compressed in the `.7z`.
|
||||
|
||||
**Ship the zhongfly LGPL build** — smaller, and there is no reason to pull the
|
||||
GPL variant in for an audio-only use.
|
||||
|
||||
Import-table inspection confirms **no companion DLLs are needed**: every
|
||||
dependency is a system DLL (`KERNEL32`, `USER32`, `d2d1`, `DWrite`, `OPENGL32`,
|
||||
`vulkan-1`, UCRT `api-ms-win-*`). One file to bundle.
|
||||
|
||||
93 MiB is still substantial against a Tauri app's usual few MB. Since we use mpv
|
||||
audio-only, investigate whether a pruned build (no video decoders, no libplacebo)
|
||||
is worth producing for the builder image — but treat that as an optimization,
|
||||
not a blocker.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Windows *video*. Stays in WebView2 + hls.js — it works and has ABR.
|
||||
- Crossfade (UR-031/DR-034) — not implemented anywhere; needs its own spec.
|
||||
- Replacing `WebviewAudioBackend` for macOS.
|
||||
- MPRIS/SMTC media-key integration — worth a follow-up, not this spec.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] Windows build produces a `MpvBackend`-backed player; `backend-init-failed` is emitted (not a crash) if libmpv is unavailable.
|
||||
- [ ] Volume control works from the UI — the current hard gap.
|
||||
- [ ] EQ, normalization, and gapless audibly take effect on Windows.
|
||||
- [ ] Position/state originate in Rust; the `<audio>` element is no longer in the audio path.
|
||||
- [ ] Seek, next/previous, and queue advance work; sleep timer stops playback.
|
||||
- [ ] `libmpv-2.dll` ships in the NSIS installer and the app runs on a clean Windows VM with no mpv installed.
|
||||
- [ ] Builder image carries the Windows libmpv artefacts; **no toolchain install added to any CI step**.
|
||||
- [ ] `bun run check`, `bun run test`, `bun run check:boundary` pass.
|
||||
- [ ] `cargo fmt` clean, `cargo clippy` clean, `bun run test:rust` passes.
|
||||
- [ ] New requirement-implementing code carries `// TRACES:` comments.
|
||||
|
||||
## Testing
|
||||
|
||||
**Rust**: the existing `mpv_backend_test.rs` and the `build_af_filter` /
|
||||
`normalize_filter_entry` / `eq_filter_entries` unit tests already cover the
|
||||
filter-graph logic and are platform-independent — they should pass unchanged
|
||||
under a Windows `cargo check`/test. Add a test asserting `detect_audio_system()`
|
||||
returns `wasapi` under `cfg(windows)`.
|
||||
|
||||
**Manual, on Windows**: volume, EQ preset change, normalization toggle, gapless
|
||||
between two tracks, seek, queue advance, sleep timer. Then the packaging test —
|
||||
install the NSIS output on a clean VM and confirm it launches and plays.
|
||||
|
||||
Per CLAUDE.md, the volume gap is a *bug fix*: write a failing test for
|
||||
"`set_volume` reaches the backend" before implementing.
|
||||
|
||||
## TRACES
|
||||
|
||||
- Windows `MpvBackend` construction in `create_player_backend` → `// TRACES: UR-003 | IR-030`
|
||||
- `detect_audio_system` Windows branch → `IR-030`
|
||||
- Existing `set_audio_settings` gains Windows coverage → `UR-027, UR-032, UR-033 | DR-030, DR-035, DR-036`
|
||||
- Allocate **IR-030** in `requirements.md` ("libmpv integration for Windows audio playback").
|
||||
|
||||
## Notes for the implementer
|
||||
|
||||
- Do this **after** [libmpv2-migration.md](libmpv2-migration.md) — porting the
|
||||
current dead `libmpv` git pin to a second platform would double the migration
|
||||
work.
|
||||
- `libmpv2` has broken its API in every major release (4.0 removed command
|
||||
helpers, 5.0 removed `mpv_node`, 6.0 changed `RenderContext` ownership). Pin an
|
||||
exact version.
|
||||
- Only the `render`-feature parts of `libmpv2` concern video; audio-only use does
|
||||
not need it, and disabling the default `render` feature may shrink the build.
|
||||
- A parallel Claude session may be active — `git diff` first.
|
||||
Reference in New Issue
Block a user