docs(player): backend unification findings + correct false parity claims
Investigation into unifying the playback backends (Linux/MPV, Android/ExoPlayer, Windows/webview) onto one engine with hardware acceleration. Conclusion: video cannot be unified onto a native engine; audio can. The blocker is not mpv-specific. WebKitGTK, WebView2 and Android WebView each draw into their own compositor surface, so a native video surface sits either entirely above or entirely below the webview and cannot interleave with HTML. GStreamer and libVLC fail identically. mpv would additionally regress streaming: it has no adaptive bitrate, while the current hls.js path does. Six specs added: - playback-backend-unification: the analysis and decision, with evidence - android-audio-settings-parity: set_audio_settings on ExoPlayerBackend - android-native-video-spike: timeboxed test of SurfaceView compositing - windows-native-audio-backend: replace the webview <audio> shim with libmpv - libmpv2-migration: dead libmpv git pin -> libmpv2, plus a LICENSE file - playback-docs-corrections: the requirement-status fixes applied here Corrections to requirements.md, all verified against source: - UR-031/DR-034 claimed crossfade was "Done (Linux only)". It is implemented nowhere (mpv_backend.rs has a bare TODO) and is architecturally blocked on mpv, whose single-stream audio chain cannot feed acrossfade's two inputs. - Parity matrix listed crossfade as a Linux/Android gap; it is neither. - The matrix omitted the equalizer, which has the same Linux-only shape. - The suggested ConcatenatingMediaSource is deprecated in current Media3. nativeAdapter.ts cited tauri#10152 as an upstream blocker for native Android video. That issue is a stale feature request, dead since 2024-07-01; the capability shipped in tauri 27d01834 (2024-09-02), and the related black-screen bug was fixed in wry 0.39.4 (we ship 0.55.x). What is genuinely unproven is SurfaceView-behind-WebView compositing, which the spike now tracks.
This commit is contained in:
+26
-12
@@ -41,7 +41,7 @@ For a narrative overview of the system design, see
|
||||
| UR-028 | Navigate to artist/album by tapping names in now playing view | High | Done |
|
||||
| UR-029 | Toggle between grid and list view in library | Medium | Done |
|
||||
| UR-030 | Quick genre browsing and filtering | Medium | Done |
|
||||
| UR-031 | Crossfade between audio tracks | Low | Done (Linux only) |
|
||||
| UR-031 | Crossfade between audio tracks | Low | Not implemented (blocked — see DR-034) |
|
||||
| UR-032 | Gapless playback for seamless album listening | Medium | Done (Linux only) |
|
||||
| UR-033 | Volume normalization to prevent volume jumps between tracks | Low | Done (Linux only) |
|
||||
| UR-034 | Rich home screen with hero banners, carousels, and personalized sections | High | Done |
|
||||
@@ -193,7 +193,7 @@ Internal architecture, components, and application logic.
|
||||
| DR-031 | Clickable artist/album links in now playing view | UI | UR-028 | Done |
|
||||
| DR-032 | List view option for library browsing (albums, artists) | UI | UR-029 | Done |
|
||||
| DR-033 | Genre browsing screen with quick filters | UI | UR-030 | Done |
|
||||
| DR-034 | Crossfade engine with configurable duration (0-12s) | Player | UR-031 | Done (Linux only) |
|
||||
| DR-034 | Crossfade engine with configurable duration (0-12s) | Player | UR-031 | Not implemented (blocked on MPV: single-stream audio chain; `acrossfade` needs 2 inputs — see docs/specs/playback-backend-unification.md) |
|
||||
| DR-035 | Gapless playback between sequential tracks | Player | UR-032 | Done (Linux only) |
|
||||
| DR-036 | Volume normalization with preset levels (Loud/Normal/Quiet) | Player | UR-033 | Done (Linux only) |
|
||||
| DR-037 | Remote session browser and control UI | UI | UR-010 | Done |
|
||||
@@ -499,22 +499,36 @@ The `PlayerBackend` trait defines optional audio settings methods with default e
|
||||
| Basic playback | ✅ | ✅ | Parity |
|
||||
| Volume control | ✅ | ✅ | Parity |
|
||||
| Seek | ✅ | ✅ | Parity |
|
||||
| Crossfade | ✅ | ❌ | Gap |
|
||||
| Gapless playback | ✅ | ❌ | Gap |
|
||||
| Volume normalization | ✅ | ❌ | Gap |
|
||||
| Crossfade | ❌ | ❌ | Not implemented (blocked on MPV) |
|
||||
| Gapless playback | ✅ | ⚠️ | Implemented, pending on-device verification |
|
||||
| Volume normalization | ✅ | ⚠️ | Implemented (LoudnessEnhancer — gain stage, approximate vs MPV's dynaudnorm), pending on-device verification |
|
||||
| Equalizer (10-band) | ✅ | ⚠️ | Implemented (resampled onto device bands), pending on-device verification |
|
||||
| Position updates | 250ms | On-demand | Inconsistent |
|
||||
|
||||
**Future Fix**:
|
||||
1. Implement `set_audio_settings()` in `ExoPlayerBackend`
|
||||
2. Add Kotlin-side ExoPlayer configuration for crossfade (using `ConcatenatingMediaSource` or `DefaultMediaSourceFactory`)
|
||||
3. Implement gapless via ExoPlayer's built-in gapless support
|
||||
4. Add volume normalization via ExoPlayer's `LoudnessEnhancer` or audio processor
|
||||
5. Standardize position update frequency across platforms
|
||||
**Status** (see docs/specs/android-audio-settings-parity.md):
|
||||
1. ✅ `set_audio_settings()` implemented in `ExoPlayerBackend` (JSON over JNI)
|
||||
2. ✅ Gapless via ExoPlayer's `pauseAtEndOfMediaItems`
|
||||
3. ✅ Volume normalization via `LoudnessEnhancer`
|
||||
4. ✅ Equalizer via `android.media.audiofx.Equalizer`, canonical 10 bands
|
||||
resampled onto the device's band centres
|
||||
5. ⬜ **Not yet verified on a physical device** — the EQ/normalization effects
|
||||
depend on device-specific `AudioEffect` availability and band layouts
|
||||
6. ⬜ Flip the trait's `set_audio_settings` default from `Ok(())` to
|
||||
`Err(not_implemented())` so a backend that omits it fails loudly instead of
|
||||
silently reporting success. Deferred until (5) confirms the Android path works
|
||||
7. ⬜ Standardize position update frequency across platforms
|
||||
|
||||
Crossfade is deliberately absent: it is unimplemented on every platform and
|
||||
architecturally blocked on MPV, so building it on Android alone would invert the
|
||||
parity gap. (The previously suggested `ConcatenatingMediaSource` is also
|
||||
deprecated in current Media3.)
|
||||
|
||||
**Impact**:
|
||||
- Medium - Android users lack audio enhancement features advertised in requirements
|
||||
- User experience differs between platforms
|
||||
- UR-031 (Crossfade), UR-032 (Gapless), UR-033 (Normalization) only work on Linux
|
||||
- UR-032 (Gapless), UR-033 (Normalization) and UR-027 (Equalizer) are now
|
||||
implemented on Android as well as Linux, pending on-device verification
|
||||
- UR-031 (Crossfade) works nowhere — see DR-034
|
||||
|
||||
**Traces To**: IR-004, UR-031, UR-032, UR-033, DR-034, DR-035, DR-036
|
||||
|
||||
|
||||
Reference in New Issue
Block a user