feat(android): implement audio settings (EQ, normalization, gapless)
🏗️ Build and Test JellyTau / Android Compile Check (push) Has been cancelled
Publish Documentation / Build & publish docs to gitea-pages (push) Has been cancelled
Traceability Validation / Check Requirement Traces (push) Has been cancelled
🏗️ Build and Test JellyTau / Run Tests (push) Has been cancelled

ExoPlayerBackend was the only backend not overriding the PlayerBackend trait's
set_audio_settings/audio_settings defaults, so the Settings > Audio controls
rendered on Android and silently did nothing — the default returns Ok(()) while
applying nothing, so the failure was invisible.

Rust owns what the values are (canonical 10-band ISO layout, preset curves,
normalization presets); Kotlin owns when the AudioEffect objects exist, since
that needs the live audio session id.

- settings.rs: audio_settings_jni_payload() sanitises (crossfade clamped, band
  vector normalised) before serialising, so a malformed vector cannot reach the
  Kotlin parser. JSON rather than a wide JNI signature, matching how load()
  already passes subtitles — adding a field will not change the signature.
- ExoPlayerBackend: set_audio_settings/audio_settings over JNI; ExoPlayerState
  gains the first command-side field (settings are pushed out, never reported).
- JellyTauPlayer.kt: Equalizer, LoudnessEnhancer, and gapless via
  pauseAtEndOfMediaItems.

Three details that are easy to get wrong:
- Effects re-attach on onAudioSessionIdChanged. ExoPlayer rebuilds its audio
  sink on a format change, which invalidates effects bound to the old session;
  without this the EQ silently stops applying mid-queue.
- All effect work is posted to mainHandler rather than run inline. AudioEffect
  construction from a player callback can re-enter the player and deadlock —
  the same shape as the AutoplayDecision lock-scrutinee bug.
- Device equalizers expose a device-dependent band count (commonly 5) at fixed
  centres, so the canonical 10 bands are resampled by nearest centre frequency.
  resampleBands() is a pure @JvmStatic function so that mapping is testable
  without a device.

Normalization is approximate, not parity: LoudnessEnhancer is a gain stage, not
a true EBU R128 normalizer like MPV's dynaudnorm. Recorded as such rather than
claimed as equivalent.

Crossfade is deliberately excluded — unimplemented on every platform and
blocked on mpv, so building it on Android alone would invert the parity gap.

Tests written first and observed failing (cannot find function
audio_settings_jni_payload) before the implementation: the payload contract is
pinned by tests because a serde rename would otherwise silently break the
Kotlin parser.

Not yet verified on a physical device — AudioEffect availability and band
layouts are device-specific. Requirements matrix marks these rows accordingly,
and flipping the trait default to Err(not_implemented()) is deferred until that
verification lands.
This commit is contained in:
2026-07-28 23:03:31 +02:00
parent b11188e9dd
commit cb79a376b3
4 changed files with 1099 additions and 691 deletions
+781 -691
View File
File diff suppressed because it is too large Load Diff
@@ -36,6 +36,53 @@ class JellyTauPlayer(private val appContext: Context) {
/** Position update interval in milliseconds */ /** Position update interval in milliseconds */
private const val POSITION_UPDATE_INTERVAL_MS = 250L private const val POSITION_UPDATE_INTERVAL_MS = 250L
/** AudioEffect priority. Positive = higher priority than the default. */
private const val EFFECT_PRIORITY = 1000
/**
* Canonical 10-band ISO centre frequencies (Hz), mirroring EQ_BANDS in
* settings.rs. Kept in sync deliberately: Rust owns the band layout, this
* is only the lookup table used to map those gains onto whatever bands
* the device's equalizer actually has.
*/
private val CANONICAL_BAND_CENTRES_HZ =
intArrayOf(31, 62, 125, 250, 500, 1000, 2000, 4000, 8000, 16000)
/**
* Map canonical band gains onto a device's band centres by nearest
* centre frequency.
*
* Pure function so it can be unit-tested without a device device band
* counts vary (commonly 5) and getting this wrong silently mis-shapes the
* EQ curve rather than failing.
*
* TRACES: UR-027 | DR-030
*/
@JvmStatic
fun resampleBands(
canonicalGains: FloatArray,
canonicalCentresHz: IntArray,
deviceCentresHz: IntArray
): FloatArray {
if (canonicalGains.isEmpty() || deviceCentresHz.isEmpty()) {
return FloatArray(deviceCentresHz.size)
}
val usable = minOf(canonicalGains.size, canonicalCentresHz.size)
return FloatArray(deviceCentresHz.size) { d ->
val target = deviceCentresHz[d]
var nearest = 0
var bestDelta = Int.MAX_VALUE
for (c in 0 until usable) {
val delta = kotlin.math.abs(canonicalCentresHz[c] - target)
if (delta < bestDelta) {
bestDelta = delta
nearest = c
}
}
canonicalGains[nearest]
}
}
/** Singleton instance for JNI access */ /** Singleton instance for JNI access */
@Volatile @Volatile
private var instance: JellyTauPlayer? = null private var instance: JellyTauPlayer? = null
@@ -135,6 +182,18 @@ class JellyTauPlayer(private val appContext: Context) {
private val coroutineScope = CoroutineScope(Dispatchers.Main + SupervisorJob()) private val coroutineScope = CoroutineScope(Dispatchers.Main + SupervisorJob())
private var positionUpdateJob: Job? = null private var positionUpdateJob: Job? = null
/** Graphic EQ bound to the current audio session, or null if not attached. */
private var equalizer: android.media.audiofx.Equalizer? = null
/** Loudness/normalization effect bound to the current audio session. */
private var loudnessEnhancer: android.media.audiofx.LoudnessEnhancer? = null
/**
* Last settings pushed from Rust, replayed when the audio session is rebuilt.
* Held as the raw payload so re-application needs no second parse contract.
*/
private var lastAudioSettings: org.json.JSONObject? = null
/** Current media ID being played */ /** Current media ID being played */
private var currentMediaId: String? = null private var currentMediaId: String? = null
@@ -334,6 +393,11 @@ class JellyTauPlayer(private val appContext: Context) {
override fun onAudioSessionIdChanged(audioSessionId: Int) { override fun onAudioSessionIdChanged(audioSessionId: Int) {
android.util.Log.d("JellyTauPlayer", "▶▶▶ AUDIO SESSION ID CHANGED: $audioSessionId") android.util.Log.d("JellyTauPlayer", "▶▶▶ AUDIO SESSION ID CHANGED: $audioSessionId")
// ExoPlayer rebuilt its audio sink (e.g. on a format change), so
// effects bound to the old session are dead. Re-attach, or the EQ
// silently stops applying mid-queue.
releaseAudioEffects()
lastAudioSettings?.let { applyAudioEffects(it) }
} }
}) })
} }
@@ -416,6 +480,133 @@ class JellyTauPlayer(private val appContext: Context) {
} }
} }
/**
* Apply audio settings pushed from Rust as JSON.
*
* Rust owns *what* the values are (band layout, preset curves, normalization
* presets); this owns *when* the Android AudioEffect objects exist, since
* that needs the live audio session id and must survive a sink rebuild.
*
* Posted to the main handler rather than run inline: AudioEffect construction
* from a player callback can re-enter the player and deadlock.
*
* TRACES: UR-027, UR-032, UR-033 | DR-030, DR-035, DR-036
*/
fun setAudioSettings(json: String) {
mainHandler.post {
try {
val settings = org.json.JSONObject(json)
lastAudioSettings = settings
// Gapless: ExoPlayer is gapless by default for compatible
// formats, so honouring the setting means disabling it when off.
exoPlayer.pauseAtEndOfMediaItems = !settings.optBoolean("gaplessPlayback", true)
applyAudioEffects(settings)
} catch (e: Exception) {
android.util.Log.e("JellyTauPlayer", "Failed to apply audio settings", e)
}
}
}
/** Attach/update the EQ and loudness effects for the current audio session. */
private fun applyAudioEffects(settings: org.json.JSONObject) {
val sessionId = exoPlayer.audioSessionId
if (sessionId == C.AUDIO_SESSION_ID_UNSET) {
// No sink yet; onAudioSessionIdChanged will re-drive this.
return
}
try {
applyEqualizer(sessionId, settings)
} catch (e: Exception) {
android.util.Log.e("JellyTauPlayer", "Equalizer unavailable on this device", e)
}
try {
applyNormalization(sessionId, settings)
} catch (e: Exception) {
android.util.Log.e("JellyTauPlayer", "LoudnessEnhancer unavailable on this device", e)
}
}
private fun applyEqualizer(sessionId: Int, settings: org.json.JSONObject) {
val enabled = settings.optBoolean("equalizerEnabled", false)
if (!enabled) {
equalizer?.enabled = false
return
}
val eq = equalizer ?: android.media.audiofx.Equalizer(EFFECT_PRIORITY, sessionId).also {
equalizer = it
}
val bandsJson = settings.optJSONArray("equalizerBands")
val canonicalGains = FloatArray(bandsJson?.length() ?: 0) { i ->
bandsJson!!.optDouble(i, 0.0).toFloat()
}
if (canonicalGains.isEmpty()) {
eq.enabled = false
return
}
// The device's band count/centres are device-dependent (commonly 5) and
// will not match our canonical 10-band ISO layout, so resample.
val deviceBandCount = eq.numberOfBands.toInt()
val deviceCentresHz = IntArray(deviceBandCount) { i ->
eq.getCenterFreq(i.toShort()) / 1000 // device reports milliHertz
}
val levelRange = eq.bandLevelRange // millibels, [min, max]
val resampled = resampleBands(canonicalGains, CANONICAL_BAND_CENTRES_HZ, deviceCentresHz)
for (i in 0 until deviceBandCount) {
val millibels = (resampled[i] * 100f)
.coerceIn(levelRange[0].toFloat(), levelRange[1].toFloat())
eq.setBandLevel(i.toShort(), millibels.toInt().toShort())
}
eq.enabled = true
}
private fun applyNormalization(sessionId: Int, settings: org.json.JSONObject) {
val enabled = settings.optBoolean("normalizeVolume", false)
if (!enabled) {
loudnessEnhancer?.enabled = false
return
}
val enhancer = loudnessEnhancer
?: android.media.audiofx.LoudnessEnhancer(sessionId).also { loudnessEnhancer = it }
// Approximate parity with the Linux dynaudnorm path: LoudnessEnhancer is
// a gain stage, not a true EBU R128 normalizer, so these are relative
// offsets preserving the Loud > Normal > Quiet ordering.
val targetGainMb = when (settings.optString("volumeLevel", "normal")) {
"loud" -> 600
"quiet" -> -600
else -> 0
}
enhancer.setTargetGain(targetGainMb)
enhancer.enabled = true
}
private fun releaseAudioEffects() {
try {
equalizer?.release()
} catch (e: Exception) {
android.util.Log.w("JellyTauPlayer", "Equalizer release failed", e)
}
try {
loudnessEnhancer?.release()
} catch (e: Exception) {
android.util.Log.w("JellyTauPlayer", "LoudnessEnhancer release failed", e)
}
equalizer = null
loudnessEnhancer = null
}
/** /**
* Get the current playback position in seconds. * Get the current playback position in seconds.
*/ */
@@ -756,6 +947,7 @@ class JellyTauPlayer(private val appContext: Context) {
mainHandler.post { mainHandler.post {
stopPositionUpdates() stopPositionUpdates()
coroutineScope.cancel() coroutineScope.cancel()
releaseAudioEffects()
exoPlayer.release() exoPlayer.release()
instance = null instance = null
} }
+56
View File
@@ -18,6 +18,7 @@ use super::events::{PlayerStatusEvent, SharedEventEmitter};
use super::media::{MediaItem, MediaType}; use super::media::{MediaItem, MediaType};
use super::state::PlayerState; use super::state::PlayerState;
use crate::playback_reporting::{EventThrottler, PlaybackOperation, PlaybackReporter}; use crate::playback_reporting::{EventThrottler, PlaybackOperation, PlaybackReporter};
use crate::settings::{audio_settings_jni_payload, AudioSettings};
use crate::utils::conversions::seconds_to_ticks; use crate::utils::conversions::seconds_to_ticks;
/// Global reference to the JavaVM for JNI callbacks /// Global reference to the JavaVM for JNI callbacks
@@ -148,6 +149,10 @@ struct ExoPlayerState {
volume: f32, volume: f32,
is_loaded: bool, is_loaded: bool,
current_media: Option<MediaItem>, current_media: Option<MediaItem>,
/// Last applied audio settings. Unlike the fields above (which JNI callbacks
/// push *in*), this is commanded *out* — audio settings are never reported
/// by the player, so this is the authoritative copy for `audio_settings()`.
audio_settings: AudioSettings,
} }
impl ExoPlayerState { impl ExoPlayerState {
@@ -159,6 +164,7 @@ impl ExoPlayerState {
volume: 1.0, volume: 1.0,
is_loaded: false, is_loaded: false,
current_media: None, current_media: None,
audio_settings: AudioSettings::default(),
} }
} }
} }
@@ -529,6 +535,56 @@ impl PlayerBackend for ExoPlayerBackend {
self.shared_state.lock_safe().volume self.shared_state.lock_safe().volume
} }
/// Apply audio settings to ExoPlayer (equalizer, normalization, gapless).
///
/// Sent as JSON rather than a wide JNI signature so new fields do not change
/// the method signature — the same approach `load()` uses for subtitles. The
/// Kotlin side owns the *mechanics* (attaching AudioEffects to the audio
/// session); the canonical band layout and preset curves stay in Rust.
///
/// TRACES: UR-027, UR-032, UR-033 | DR-030, DR-035, DR-036
fn set_audio_settings(&mut self, settings: &AudioSettings) -> Result<(), PlayerError> {
let json = audio_settings_jni_payload(settings).map_err(|e| {
PlayerError::playback_failed(format!("Failed to serialize audio settings: {}", e))
})?;
let vm = JAVA_VM
.get()
.ok_or_else(|| PlayerError::playback_failed("JavaVM not initialized"))?;
let mut env = vm
.attach_current_thread()
.map_err(|e| PlayerError::playback_failed(format!("Failed to attach thread: {}", e)))?;
let json_jstring = env.new_string(&json).map_err(|e| {
PlayerError::playback_failed(format!("Failed to create settings string: {}", e))
})?;
env.call_method(
&self.player_ref,
"setAudioSettings",
"(Ljava/lang/String;)V",
&[JValue::Object(&json_jstring)],
)
.map_err(|e| {
PlayerError::playback_failed(format!("Failed to call setAudioSettings: {}", e))
})?;
// Store the sanitised form so audio_settings() reflects what was applied,
// not what was requested.
self.shared_state.lock_safe().audio_settings = settings
.clone()
.with_crossfade_clamped()
.with_equalizer_normalised();
Ok(())
}
/// TRACES: UR-027, UR-032, UR-033 | DR-030, DR-035, DR-036
fn audio_settings(&self) -> AudioSettings {
self.shared_state.lock_safe().audio_settings.clone()
}
fn set_audio_track(&mut self, stream_index: i32) -> Result<(), PlayerError> { fn set_audio_track(&mut self, stream_index: i32) -> Result<(), PlayerError> {
let vm = JAVA_VM let vm = JAVA_VM
.get() .get()
+70
View File
@@ -179,10 +179,80 @@ impl VideoSettings {
} }
} }
/// Serialise `AudioSettings` into the JSON payload handed to the Android player
/// over JNI.
///
/// Sanitises first (crossfade clamped, band vector normalised) so a malformed
/// vector can never reach the Kotlin parser. JSON is used rather than a wide JNI
/// signature so that adding a field does not change the method signature — the
/// same approach `load()` already uses for subtitles.
///
/// The emitted keys are camelCase (serde) and `volumeLevel` is lowercase; the
/// Kotlin side matches on those literals. Both are pinned by tests.
///
/// TRACES: UR-027, UR-032, UR-033 | DR-030, DR-035, DR-036
pub fn audio_settings_jni_payload(settings: &AudioSettings) -> Result<String, serde_json::Error> {
let sanitised = settings
.clone()
.with_crossfade_clamped()
.with_equalizer_normalised();
serde_json::to_string(&sanitised)
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
/// The JNI payload must sanitise before serialising: an over-long crossfade
/// is clamped and a wrong-length band vector is normalised to EQ_BANDS.len().
/// Sending raw values would let a malformed vector reach the Kotlin parser.
///
/// TRACES: UR-027, UR-032, UR-033 | DR-030, DR-035, DR-036 | UT-AUDIO-JNI-1
#[test]
fn test_audio_settings_jni_payload_is_sanitised() {
let settings = AudioSettings {
crossfade_duration: 30.0,
equalizer_bands: vec![20.0, -30.0],
..AudioSettings::default()
};
let json = audio_settings_jni_payload(&settings).expect("serialises");
let v: serde_json::Value = serde_json::from_str(&json).expect("valid JSON");
assert_eq!(v["crossfadeDuration"], 12.0, "crossfade clamped to 12s");
let bands = v["equalizerBands"].as_array().expect("bands array");
assert_eq!(bands.len(), EQ_BANDS.len(), "band vector normalised to 10");
assert_eq!(bands[0], EQ_GAIN_MAX as f64, "gain clamped to +12dB");
assert_eq!(bands[1], EQ_GAIN_MIN as f64, "gain clamped to -12dB");
}
/// The Kotlin side parses these exact keys. camelCase is what serde emits
/// for AudioSettings; a rename here silently breaks the Android parser,
/// which is why the contract is pinned by a test rather than by convention.
///
/// TRACES: UR-027, UR-032, UR-033 | DR-030, DR-035, DR-036 | UT-AUDIO-JNI-2
#[test]
fn test_audio_settings_jni_payload_key_contract() {
let json = audio_settings_jni_payload(&AudioSettings::default()).expect("serialises");
let v: serde_json::Value = serde_json::from_str(&json).expect("valid JSON");
for key in [
"crossfadeDuration",
"gaplessPlayback",
"normalizeVolume",
"volumeLevel",
"equalizerEnabled",
"equalizerBands",
] {
assert!(v.get(key).is_some(), "JNI payload must carry `{key}`");
}
// VolumeLevel is #[serde(rename_all = "lowercase")]; Kotlin matches on
// these literals.
assert_eq!(v["volumeLevel"], "normal");
}
#[test] #[test]
fn test_default_settings() { fn test_default_settings() {
let settings = AudioSettings::default(); let settings = AudioSettings::default();