Files
jellytau/docs/specs/audio-equalizer.md
T
dtourolle b11188e9dd docs(player): backend unification findings + correct false parity claims
Investigation into unifying the playback backends (Linux/MPV, Android/ExoPlayer,
Windows/webview) onto one engine with hardware acceleration. Conclusion: video
cannot be unified onto a native engine; audio can.

The blocker is not mpv-specific. WebKitGTK, WebView2 and Android WebView each
draw into their own compositor surface, so a native video surface sits either
entirely above or entirely below the webview and cannot interleave with HTML.
GStreamer and libVLC fail identically. mpv would additionally regress streaming:
it has no adaptive bitrate, while the current hls.js path does.

Six specs added:
- playback-backend-unification: the analysis and decision, with evidence
- android-audio-settings-parity: set_audio_settings on ExoPlayerBackend
- android-native-video-spike: timeboxed test of SurfaceView compositing
- windows-native-audio-backend: replace the webview <audio> shim with libmpv
- libmpv2-migration: dead libmpv git pin -> libmpv2, plus a LICENSE file
- playback-docs-corrections: the requirement-status fixes applied here

Corrections to requirements.md, all verified against source:
- UR-031/DR-034 claimed crossfade was "Done (Linux only)". It is implemented
  nowhere (mpv_backend.rs has a bare TODO) and is architecturally blocked on
  mpv, whose single-stream audio chain cannot feed acrossfade's two inputs.
- Parity matrix listed crossfade as a Linux/Android gap; it is neither.
- The matrix omitted the equalizer, which has the same Linux-only shape.
- The suggested ConcatenatingMediaSource is deprecated in current Media3.

nativeAdapter.ts cited tauri#10152 as an upstream blocker for native Android
video. That issue is a stale feature request, dead since 2024-07-01; the
capability shipped in tauri 27d01834 (2024-09-02), and the related
black-screen bug was fixed in wry 0.39.4 (we ship 0.55.x). What is genuinely
unproven is SurfaceView-behind-WebView compositing, which the spike now tracks.
2026-07-28 23:03:17 +02:00

9.0 KiB
Raw Permalink Blame History

Spec: Audio equalizer

Status: Accepted Requirements: UR-027 → DR-030 (EQ UI), IR-020 (MPV EQ integration). UX spec: n/a (extends the Settings Audio section, ux-flows §8.1 instant-apply). Supersedes / revises:Revised by: android-audio-settings-parity.md — lifts the "Android is a no-op" limitation below.

Summary

Add a graphic audio equalizer to playback. Users pick a preset (Flat, Rock, Pop, Jazz, Classical, Bass Boost, Treble Boost, Vocal) or set custom per-band gains, from a new block in Settings Audio. On Linux the gains apply live via MPV's audio-filter chain; the settings persist and re-apply on the next track and at startup, exactly like crossfade/gapless/normalize do today. Android is a no-op for now (documented parity gap, same as those three features).

Motivation

UR-027 is one of the few still-unbuilt audio features. The audio-settings pipeline it needs already exists — AudioSettings + set_audio_settings on the PlayerBackend trait, the player_set_audio_settings command, and the Settings Audio UI with instant-apply. Crossfade, gapless, and volume normalization all ride that pipeline. The equalizer is the same shape: N more fields on AudioSettings, an af filter on the MPV backend, one more block in the settings panel. No new command, no new state machine.

Layer assignment

Logic / responsibility Layer Why it belongs there
EQ band count, centre frequencies, gain range/clamping Rust Domain of the audio engine; the bands must match what the MPV filter expects. Changing the DSP must not require a frontend change.
Preset name → per-band gain curve Rust A preset is a domain gain curve, not a label. It changes with the audio engine's band layout, never with the UI. Placing it in the frontend would be the scoped-search taxonomy mistake again (values that look like config but are domain data).
Translating gains → MPV af filter string Rust Platform playback detail; lives with the other set_audio_settings filter code in mpv_backend.rs.
Persisting the chosen settings, re-pushing on load Rust/existing Same path crossfade/etc. already use; the controller re-applies AudioSettings per track.
Rendering band sliders, the preset chips, live readouts Frontend Pure presentation; changes only if the settings UI is redesigned.
Which preset chip is highlighted; instant-apply on change Frontend Presentation/input handling (UR-057), the same as the normalize preset picker.

Tie-breaker note: the preset→curve map is the one tempting boundary leak. It goes in Rust because a preset is a set of band gains defined by the band layout, which is an engine property. The frontend only ever names a preset and renders the resulting gains; it never defines them.

Design

AudioSettings (Rust, settings.rs)

Add two fields (both #[serde(rename_all = "camelCase")] via the existing struct attribute):

/// Equalizer enabled. When false, no `af` EQ filter is applied.
pub equalizer_enabled: bool,
/// Per-band gains in dB, one per FIXED band (see EQ_BANDS). Length is
/// validated/normalised to EQ_BANDS.len(); clamped to [-12, +12] dB.
pub equalizer_bands: Vec<f32>,

Fixed 10-band ISO layout (domain constant in settings.rs):

pub const EQ_BANDS: [f32; 10] =
    [31.0, 62.0, 125.0, 250.0, 500.0, 1000.0, 2000.0, 4000.0, 8000.0, 16000.0];
pub const EQ_GAIN_MIN: f32 = -12.0;
pub const EQ_GAIN_MAX: f32 = 12.0;
  • Default: equalizer_enabled: false, equalizer_bands: vec![0.0; 10] (flat).
  • New with_equalizer_normalised(self) clamps each gain to [EQ_GAIN_MIN, EQ_GAIN_MAX] and pads/truncates the vec to 10 bands. Applied in the command alongside with_crossfade_clamped (add that call too — it's currently missing).
  • Backward compat: both fields #[serde(default)] so old persisted JSON loads.

Presets (Rust, settings.rs)

#[derive(specta::Type, Serialize, Deserialize, Clone, Copy, PartialEq)]
#[serde(rename_all = "camelCase")]
pub enum EqPreset { Flat, Rock, Pop, Jazz, Classical, BassBoost, TrebleBoost, Vocal }

impl EqPreset {
    /// The 10-band gain curve (dB) for this preset.
    pub fn gains(&self) -> [f32; 10] { /* table */ }
}

Preset selection is a frontend convenience: tapping a chip sets equalizer_bands = preset.gains() and pushes settings. The curve tables live in Rust; the frontend reads them via a tiny player_get_eq_presets command returning Vec<(EqPreset, Vec<f32>)> (or a map), so the frontend never encodes the numbers. (If exposing the whole table is awkward through specta, expose player_eq_preset_gains(preset) -> Vec<f32> instead — pick at implement time.)

MPV application (Rust, mpv_backend.rs::set_audio_settings)

Build an equalizer / anequalizer filter from the bands and set the af property. When equalizer_enabled is false or all gains are 0, clear the EQ filter (leave any other af entries intact). Use af add/af remove or a rebuilt af string; keep it isolated so it doesn't stomp a future crossfade filter. Errors map to PlayerError like the gapless code.

No new persistence table

AudioSettings is already round-tripped by the frontend settings store and re-pushed via player_set_audio_settings on change and on load. The two new fields ride along. NullBackend/Android inherit the trait default (no-op).

Wire summary

  • Command names unchanged: player_set_audio_settings, player_get_audio_settings (now carry the EQ fields).
  • New (optional) read-only command for preset curves — kebab n/a (it's a command): player_get_eq_presets (or player_eq_preset_gains).
  • Regenerate bindings.ts from the Rust types; never hand-edit.

Out of scope

  • Android/ExoPlayer EQ (parity gap tracked with crossfade/gapless/normalize). Now specified in android-audio-settings-parity.md, which implements set_audio_settings on ExoPlayerBackend. The canonical band layout and preset→curve map defined here remain authoritative; the Android side resamples those bands onto the device equalizer rather than defining its own.
  • Per-track or per-library EQ profiles — one global profile only.
  • Automatic loudness/room correction; only manual bands + presets.
  • Changing the crossfade/normalize TODOs in set_audio_settings beyond wiring the missing with_crossfade_clamped call.

Acceptance criteria

  • Settings Audio has an Equalizer block: enable toggle, preset chips, 10 band sliders with live dB readouts, instant-apply (no Save button).
  • Choosing a preset sets the bands from the Rust-defined curve; editing a band switches the highlighted preset to "Custom" (frontend-only label).
  • Gains clamp to [-12, +12] dB; the band vector always normalises to 10.
  • On Linux, enabling EQ audibly changes output and persists across tracks and app restart; disabling clears the filter without affecting other audio.
  • Old persisted settings (no EQ fields) load without error, defaulting flat.
  • bun run check and bun run test pass.
  • cargo fmt clean, cargo clippy clean, bun run test:rust passes.
  • bun run check:boundary passes (no preset curve numbers in the frontend).
  • New requirement-implementing code carries // TRACES: comments.
  • bindings.ts regenerated.

Testing

  • Rust (settings.rs): default is flat + disabled; with_equalizer_normalised clamps out-of-range gains and pads/truncates band length; serialization round-trips the camelCase fields; backward-compat load of pre-EQ JSON; each preset returns a 10-length curve; Flat is all zeros.
  • Rust IPC param naming for any new command (camelCase rule per CLAUDE.md).
  • Frontend (settings page or an extracted helper): selecting a preset sets the expected band array; editing a band flips the label to Custom; enable toggle gates the sliders. Keep DSP untested on the frontend (it's Rust's).

TRACES

  • AudioSettings EQ fields + normalise + presets: UR-027 | DR-030 (+ unit tests)
  • MPV EQ filter application: UR-027 | IR-020
  • Settings EQ UI block: UR-027 | DR-030
  • Preset-curve command: UR-027 | DR-030

Notes for the implementer

  • A parallel Claude session is active in this repo (it has touched tauri.conf.json, Dockerfile, package.json, home components, and added build scripts, and the Rust build is currently broken by its tauri.conf.json bundle-target change). git diff before "repairing" anything you didn't write; keep EQ changes isolated to settings.rs, mpv_backend.rs, backend.rs (trait default already covers it), commands/player/settings.rs, and the settings page.
  • Mirror the volume-normalization block in the settings page for the toggle + preset-picker pattern; mirror the gapless code in set_audio_settings for the MPV property handling.
  • Confirm the exact MPV filter name available in the linked libmpv (equalizer vs anequalizer/superequalizer) before committing the filter string; gate cleanly if unavailable.