Files
jellytau/docs/specs/audio-equalizer.md
T
dtourolle b11188e9dd docs(player): backend unification findings + correct false parity claims
Investigation into unifying the playback backends (Linux/MPV, Android/ExoPlayer,
Windows/webview) onto one engine with hardware acceleration. Conclusion: video
cannot be unified onto a native engine; audio can.

The blocker is not mpv-specific. WebKitGTK, WebView2 and Android WebView each
draw into their own compositor surface, so a native video surface sits either
entirely above or entirely below the webview and cannot interleave with HTML.
GStreamer and libVLC fail identically. mpv would additionally regress streaming:
it has no adaptive bitrate, while the current hls.js path does.

Six specs added:
- playback-backend-unification: the analysis and decision, with evidence
- android-audio-settings-parity: set_audio_settings on ExoPlayerBackend
- android-native-video-spike: timeboxed test of SurfaceView compositing
- windows-native-audio-backend: replace the webview <audio> shim with libmpv
- libmpv2-migration: dead libmpv git pin -> libmpv2, plus a LICENSE file
- playback-docs-corrections: the requirement-status fixes applied here

Corrections to requirements.md, all verified against source:
- UR-031/DR-034 claimed crossfade was "Done (Linux only)". It is implemented
  nowhere (mpv_backend.rs has a bare TODO) and is architecturally blocked on
  mpv, whose single-stream audio chain cannot feed acrossfade's two inputs.
- Parity matrix listed crossfade as a Linux/Android gap; it is neither.
- The matrix omitted the equalizer, which has the same Linux-only shape.
- The suggested ConcatenatingMediaSource is deprecated in current Media3.

nativeAdapter.ts cited tauri#10152 as an upstream blocker for native Android
video. That issue is a stale feature request, dead since 2024-07-01; the
capability shipped in tauri 27d01834 (2024-09-02), and the related
black-screen bug was fixed in wry 0.39.4 (we ship 0.55.x). What is genuinely
unproven is SurfaceView-behind-WebView compositing, which the spike now tracks.
2026-07-28 23:03:17 +02:00

177 lines
9.0 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Spec: Audio equalizer
**Status:** Accepted
**Requirements:** UR-027 → DR-030 (EQ UI), IR-020 (MPV EQ integration).
**UX spec:** n/a (extends the Settings Audio section, ux-flows §8.1 instant-apply).
**Supersedes / revises:**
**Revised by:** [android-audio-settings-parity.md](android-audio-settings-parity.md) — lifts the "Android is a no-op" limitation below.
## Summary
Add a graphic audio equalizer to playback. Users pick a preset (Flat, Rock,
Pop, Jazz, Classical, Bass Boost, Treble Boost, Vocal) or set custom per-band
gains, from a new block in Settings Audio. On Linux the gains apply live via
MPV's audio-filter chain; the settings persist and re-apply on the next track
and at startup, exactly like crossfade/gapless/normalize do today. Android is a
no-op for now (documented parity gap, same as those three features).
## Motivation
UR-027 is one of the few still-unbuilt audio features. The audio-settings
pipeline it needs already exists — `AudioSettings` + `set_audio_settings` on the
`PlayerBackend` trait, the `player_set_audio_settings` command, and the Settings
Audio UI with instant-apply. Crossfade, gapless, and volume normalization all
ride that pipeline. The equalizer is the same shape: N more fields on
`AudioSettings`, an `af` filter on the MPV backend, one more block in the
settings panel. No new command, no new state machine.
## Layer assignment
| Logic / responsibility | Layer | Why it belongs there |
|------------------------|-------|----------------------|
| EQ band count, centre frequencies, gain range/clamping | Rust | Domain of the audio engine; the bands must match what the MPV filter expects. Changing the DSP must not require a frontend change. |
| Preset name → per-band gain curve | Rust | A preset *is* a domain gain curve, not a label. It changes with the audio engine's band layout, never with the UI. Placing it in the frontend would be the scoped-search taxonomy mistake again (values that look like config but are domain data). |
| Translating gains → MPV `af` filter string | Rust | Platform playback detail; lives with the other `set_audio_settings` filter code in `mpv_backend.rs`. |
| Persisting the chosen settings, re-pushing on load | Rust/existing | Same path crossfade/etc. already use; the controller re-applies `AudioSettings` per track. |
| Rendering band sliders, the preset chips, live readouts | Frontend | Pure presentation; changes only if the settings UI is redesigned. |
| Which preset chip is highlighted; instant-apply on change | Frontend | Presentation/input handling (UR-057), the same as the normalize preset picker. |
Tie-breaker note: the preset→curve map is the one tempting boundary leak. It goes
in Rust because a preset is a set of band gains defined *by the band layout*,
which is an engine property. The frontend only ever names a preset and renders
the resulting gains; it never defines them.
## Design
### `AudioSettings` (Rust, `settings.rs`)
Add two fields (both `#[serde(rename_all = "camelCase")]` via the existing
struct attribute):
```rust
/// Equalizer enabled. When false, no `af` EQ filter is applied.
pub equalizer_enabled: bool,
/// Per-band gains in dB, one per FIXED band (see EQ_BANDS). Length is
/// validated/normalised to EQ_BANDS.len(); clamped to [-12, +12] dB.
pub equalizer_bands: Vec<f32>,
```
Fixed 10-band ISO layout (domain constant in `settings.rs`):
```rust
pub const EQ_BANDS: [f32; 10] =
[31.0, 62.0, 125.0, 250.0, 500.0, 1000.0, 2000.0, 4000.0, 8000.0, 16000.0];
pub const EQ_GAIN_MIN: f32 = -12.0;
pub const EQ_GAIN_MAX: f32 = 12.0;
```
- `Default`: `equalizer_enabled: false`, `equalizer_bands: vec![0.0; 10]` (flat).
- New `with_equalizer_normalised(self)` clamps each gain to `[EQ_GAIN_MIN,
EQ_GAIN_MAX]` and pads/truncates the vec to 10 bands. Applied in the command
alongside `with_crossfade_clamped` (add that call too — it's currently missing).
- Backward compat: both fields `#[serde(default)]` so old persisted JSON loads.
### Presets (Rust, `settings.rs`)
```rust
#[derive(specta::Type, Serialize, Deserialize, Clone, Copy, PartialEq)]
#[serde(rename_all = "camelCase")]
pub enum EqPreset { Flat, Rock, Pop, Jazz, Classical, BassBoost, TrebleBoost, Vocal }
impl EqPreset {
/// The 10-band gain curve (dB) for this preset.
pub fn gains(&self) -> [f32; 10] { /* table */ }
}
```
Preset selection is a *frontend* convenience: tapping a chip sets
`equalizer_bands = preset.gains()` and pushes settings. The curve tables live in
Rust; the frontend reads them via a tiny `player_get_eq_presets` command
returning `Vec<(EqPreset, Vec<f32>)>` (or a map), so the frontend never encodes
the numbers. (If exposing the whole table is awkward through specta, expose
`player_eq_preset_gains(preset) -> Vec<f32>` instead — pick at implement time.)
### MPV application (Rust, `mpv_backend.rs::set_audio_settings`)
Build an `equalizer` / `anequalizer` filter from the bands and set the `af`
property. When `equalizer_enabled` is false or all gains are 0, clear the EQ
filter (leave any other `af` entries intact). Use `af add`/`af remove` or a
rebuilt `af` string; keep it isolated so it doesn't stomp a future crossfade
filter. Errors map to `PlayerError` like the gapless code.
### No new persistence table
`AudioSettings` is already round-tripped by the frontend settings store and
re-pushed via `player_set_audio_settings` on change and on load. The two new
fields ride along. `NullBackend`/Android inherit the trait default (no-op).
### Wire summary
- Command names unchanged: `player_set_audio_settings`,
`player_get_audio_settings` (now carry the EQ fields).
- New (optional) read-only command for preset curves — kebab n/a (it's a
command): `player_get_eq_presets` (or `player_eq_preset_gains`).
- Regenerate `bindings.ts` from the Rust types; never hand-edit.
## Out of scope
- Android/ExoPlayer EQ (parity gap tracked with crossfade/gapless/normalize).
**Now specified in [android-audio-settings-parity.md](android-audio-settings-parity.md)**,
which implements `set_audio_settings` on `ExoPlayerBackend`. The canonical band
layout and preset→curve map defined here remain authoritative; the Android side
resamples those bands onto the device equalizer rather than defining its own.
- Per-track or per-library EQ profiles — one global profile only.
- Automatic loudness/room correction; only manual bands + presets.
- Changing the crossfade/normalize TODOs in `set_audio_settings` beyond wiring
the missing `with_crossfade_clamped` call.
## Acceptance criteria
- [ ] Settings Audio has an Equalizer block: enable toggle, preset chips, 10
band sliders with live dB readouts, instant-apply (no Save button).
- [ ] Choosing a preset sets the bands from the Rust-defined curve; editing a
band switches the highlighted preset to "Custom" (frontend-only label).
- [ ] Gains clamp to [-12, +12] dB; the band vector always normalises to 10.
- [ ] On Linux, enabling EQ audibly changes output and persists across tracks
and app restart; disabling clears the filter without affecting other audio.
- [ ] Old persisted settings (no EQ fields) load without error, defaulting flat.
- [ ] `bun run check` and `bun run test` pass.
- [ ] `cargo fmt` clean, `cargo clippy` clean, `bun run test:rust` passes.
- [ ] `bun run check:boundary` passes (no preset curve numbers in the frontend).
- [ ] New requirement-implementing code carries `// TRACES:` comments.
- [ ] `bindings.ts` regenerated.
## Testing
- Rust (`settings.rs`): default is flat + disabled; `with_equalizer_normalised`
clamps out-of-range gains and pads/truncates band length; serialization
round-trips the camelCase fields; backward-compat load of pre-EQ JSON; each
preset returns a 10-length curve; Flat is all zeros.
- Rust IPC param naming for any new command (camelCase rule per CLAUDE.md).
- Frontend (`settings` page or an extracted helper): selecting a preset sets the
expected band array; editing a band flips the label to Custom; enable toggle
gates the sliders. Keep DSP untested on the frontend (it's Rust's).
## TRACES
- `AudioSettings` EQ fields + normalise + presets: `UR-027 | DR-030` (+ unit tests)
- MPV EQ filter application: `UR-027 | IR-020`
- Settings EQ UI block: `UR-027 | DR-030`
- Preset-curve command: `UR-027 | DR-030`
## Notes for the implementer
- A parallel Claude session is active in this repo (it has touched
`tauri.conf.json`, `Dockerfile`, `package.json`, home components, and added
build scripts, and the Rust build is currently broken by its
`tauri.conf.json` bundle-target change). `git diff` before "repairing"
anything you didn't write; keep EQ changes isolated to `settings.rs`,
`mpv_backend.rs`, `backend.rs` (trait default already covers it),
`commands/player/settings.rs`, and the settings page.
- Mirror the volume-normalization block in the settings page for the toggle +
preset-picker pattern; mirror the gapless code in `set_audio_settings` for the
MPV property handling.
- Confirm the exact MPV filter name available in the linked libmpv
(`equalizer` vs `anequalizer`/`superequalizer`) before committing the filter
string; gate cleanly if unavailable.