Investigation into unifying the playback backends (Linux/MPV, Android/ExoPlayer, Windows/webview) onto one engine with hardware acceleration. Conclusion: video cannot be unified onto a native engine; audio can. The blocker is not mpv-specific. WebKitGTK, WebView2 and Android WebView each draw into their own compositor surface, so a native video surface sits either entirely above or entirely below the webview and cannot interleave with HTML. GStreamer and libVLC fail identically. mpv would additionally regress streaming: it has no adaptive bitrate, while the current hls.js path does. Six specs added: - playback-backend-unification: the analysis and decision, with evidence - android-audio-settings-parity: set_audio_settings on ExoPlayerBackend - android-native-video-spike: timeboxed test of SurfaceView compositing - windows-native-audio-backend: replace the webview <audio> shim with libmpv - libmpv2-migration: dead libmpv git pin -> libmpv2, plus a LICENSE file - playback-docs-corrections: the requirement-status fixes applied here Corrections to requirements.md, all verified against source: - UR-031/DR-034 claimed crossfade was "Done (Linux only)". It is implemented nowhere (mpv_backend.rs has a bare TODO) and is architecturally blocked on mpv, whose single-stream audio chain cannot feed acrossfade's two inputs. - Parity matrix listed crossfade as a Linux/Android gap; it is neither. - The matrix omitted the equalizer, which has the same Linux-only shape. - The suggested ConcatenatingMediaSource is deprecated in current Media3. nativeAdapter.ts cited tauri#10152 as an upstream blocker for native Android video. That issue is a stale feature request, dead since 2024-07-01; the capability shipped in tauri 27d01834 (2024-09-02), and the related black-screen bug was fixed in wry 0.39.4 (we ship 0.55.x). What is genuinely unproven is SurfaceView-behind-WebView compositing, which the spike now tracks.
9.0 KiB
Spec: Audio equalizer
Status: Accepted Requirements: UR-027 → DR-030 (EQ UI), IR-020 (MPV EQ integration). UX spec: n/a (extends the Settings › Audio section, ux-flows §8.1 instant-apply). Supersedes / revises: — Revised by: android-audio-settings-parity.md — lifts the "Android is a no-op" limitation below.
Summary
Add a graphic audio equalizer to playback. Users pick a preset (Flat, Rock, Pop, Jazz, Classical, Bass Boost, Treble Boost, Vocal) or set custom per-band gains, from a new block in Settings › Audio. On Linux the gains apply live via MPV's audio-filter chain; the settings persist and re-apply on the next track and at startup, exactly like crossfade/gapless/normalize do today. Android is a no-op for now (documented parity gap, same as those three features).
Motivation
UR-027 is one of the few still-unbuilt audio features. The audio-settings
pipeline it needs already exists — AudioSettings + set_audio_settings on the
PlayerBackend trait, the player_set_audio_settings command, and the Settings
› Audio UI with instant-apply. Crossfade, gapless, and volume normalization all
ride that pipeline. The equalizer is the same shape: N more fields on
AudioSettings, an af filter on the MPV backend, one more block in the
settings panel. No new command, no new state machine.
Layer assignment
| Logic / responsibility | Layer | Why it belongs there |
|---|---|---|
| EQ band count, centre frequencies, gain range/clamping | Rust | Domain of the audio engine; the bands must match what the MPV filter expects. Changing the DSP must not require a frontend change. |
| Preset name → per-band gain curve | Rust | A preset is a domain gain curve, not a label. It changes with the audio engine's band layout, never with the UI. Placing it in the frontend would be the scoped-search taxonomy mistake again (values that look like config but are domain data). |
Translating gains → MPV af filter string |
Rust | Platform playback detail; lives with the other set_audio_settings filter code in mpv_backend.rs. |
| Persisting the chosen settings, re-pushing on load | Rust/existing | Same path crossfade/etc. already use; the controller re-applies AudioSettings per track. |
| Rendering band sliders, the preset chips, live readouts | Frontend | Pure presentation; changes only if the settings UI is redesigned. |
| Which preset chip is highlighted; instant-apply on change | Frontend | Presentation/input handling (UR-057), the same as the normalize preset picker. |
Tie-breaker note: the preset→curve map is the one tempting boundary leak. It goes in Rust because a preset is a set of band gains defined by the band layout, which is an engine property. The frontend only ever names a preset and renders the resulting gains; it never defines them.
Design
AudioSettings (Rust, settings.rs)
Add two fields (both #[serde(rename_all = "camelCase")] via the existing
struct attribute):
/// Equalizer enabled. When false, no `af` EQ filter is applied.
pub equalizer_enabled: bool,
/// Per-band gains in dB, one per FIXED band (see EQ_BANDS). Length is
/// validated/normalised to EQ_BANDS.len(); clamped to [-12, +12] dB.
pub equalizer_bands: Vec<f32>,
Fixed 10-band ISO layout (domain constant in settings.rs):
pub const EQ_BANDS: [f32; 10] =
[31.0, 62.0, 125.0, 250.0, 500.0, 1000.0, 2000.0, 4000.0, 8000.0, 16000.0];
pub const EQ_GAIN_MIN: f32 = -12.0;
pub const EQ_GAIN_MAX: f32 = 12.0;
Default:equalizer_enabled: false,equalizer_bands: vec.- New
with_equalizer_normalised(self)clamps each gain to[EQ_GAIN_MIN, EQ_GAIN_MAX]and pads/truncates the vec to 10 bands. Applied in the command alongsidewith_crossfade_clamped(add that call too — it's currently missing). - Backward compat: both fields
#[serde(default)]so old persisted JSON loads.
Presets (Rust, settings.rs)
#[derive(specta::Type, Serialize, Deserialize, Clone, Copy, PartialEq)]
#[serde(rename_all = "camelCase")]
pub enum EqPreset { Flat, Rock, Pop, Jazz, Classical, BassBoost, TrebleBoost, Vocal }
impl EqPreset {
/// The 10-band gain curve (dB) for this preset.
pub fn gains(&self) -> [f32; 10] { /* table */ }
}
Preset selection is a frontend convenience: tapping a chip sets
equalizer_bands = preset.gains() and pushes settings. The curve tables live in
Rust; the frontend reads them via a tiny player_get_eq_presets command
returning Vec<(EqPreset, Vec<f32>)> (or a map), so the frontend never encodes
the numbers. (If exposing the whole table is awkward through specta, expose
player_eq_preset_gains(preset) -> Vec<f32> instead — pick at implement time.)
MPV application (Rust, mpv_backend.rs::set_audio_settings)
Build an equalizer / anequalizer filter from the bands and set the af
property. When equalizer_enabled is false or all gains are 0, clear the EQ
filter (leave any other af entries intact). Use af add/af remove or a
rebuilt af string; keep it isolated so it doesn't stomp a future crossfade
filter. Errors map to PlayerError like the gapless code.
No new persistence table
AudioSettings is already round-tripped by the frontend settings store and
re-pushed via player_set_audio_settings on change and on load. The two new
fields ride along. NullBackend/Android inherit the trait default (no-op).
Wire summary
- Command names unchanged:
player_set_audio_settings,player_get_audio_settings(now carry the EQ fields). - New (optional) read-only command for preset curves — kebab n/a (it's a
command):
player_get_eq_presets(orplayer_eq_preset_gains). - Regenerate
bindings.tsfrom the Rust types; never hand-edit.
Out of scope
- Android/ExoPlayer EQ (parity gap tracked with crossfade/gapless/normalize).
Now specified in android-audio-settings-parity.md,
which implements
set_audio_settingsonExoPlayerBackend. The canonical band layout and preset→curve map defined here remain authoritative; the Android side resamples those bands onto the device equalizer rather than defining its own. - Per-track or per-library EQ profiles — one global profile only.
- Automatic loudness/room correction; only manual bands + presets.
- Changing the crossfade/normalize TODOs in
set_audio_settingsbeyond wiring the missingwith_crossfade_clampedcall.
Acceptance criteria
- Settings › Audio has an Equalizer block: enable toggle, preset chips, 10 band sliders with live dB readouts, instant-apply (no Save button).
- Choosing a preset sets the bands from the Rust-defined curve; editing a band switches the highlighted preset to "Custom" (frontend-only label).
- Gains clamp to [-12, +12] dB; the band vector always normalises to 10.
- On Linux, enabling EQ audibly changes output and persists across tracks and app restart; disabling clears the filter without affecting other audio.
- Old persisted settings (no EQ fields) load without error, defaulting flat.
bun run checkandbun run testpass.cargo fmtclean,cargo clippyclean,bun run test:rustpasses.bun run check:boundarypasses (no preset curve numbers in the frontend).- New requirement-implementing code carries
// TRACES:comments. bindings.tsregenerated.
Testing
- Rust (
settings.rs): default is flat + disabled;with_equalizer_normalisedclamps out-of-range gains and pads/truncates band length; serialization round-trips the camelCase fields; backward-compat load of pre-EQ JSON; each preset returns a 10-length curve; Flat is all zeros. - Rust IPC param naming for any new command (camelCase rule per CLAUDE.md).
- Frontend (
settingspage or an extracted helper): selecting a preset sets the expected band array; editing a band flips the label to Custom; enable toggle gates the sliders. Keep DSP untested on the frontend (it's Rust's).
TRACES
AudioSettingsEQ fields + normalise + presets:UR-027 | DR-030(+ unit tests)- MPV EQ filter application:
UR-027 | IR-020 - Settings EQ UI block:
UR-027 | DR-030 - Preset-curve command:
UR-027 | DR-030
Notes for the implementer
- A parallel Claude session is active in this repo (it has touched
tauri.conf.json,Dockerfile,package.json, home components, and added build scripts, and the Rust build is currently broken by itstauri.conf.jsonbundle-target change).git diffbefore "repairing" anything you didn't write; keep EQ changes isolated tosettings.rs,mpv_backend.rs,backend.rs(trait default already covers it),commands/player/settings.rs, and the settings page. - Mirror the volume-normalization block in the settings page for the toggle +
preset-picker pattern; mirror the gapless code in
set_audio_settingsfor the MPV property handling. - Confirm the exact MPV filter name available in the linked libmpv
(
equalizervsanequalizer/superequalizer) before committing the filter string; gate cleanly if unavailable.