feat(android): play the original file — decode Dolby/DTS audio with FFmpeg

Android ships no AC-3, E-AC-3, DTS or TrueHD decoders; they are licensed
codecs, present only where a vendor paid for them. The ROD2-W09 tablet has a
vendor DTS decoder and no AC-3/E-AC-3 at all. So every film with Dolby audio
was re-encoded by the server, for streaming and for download alike, and a
transcoded download has no Content-Length and ignores Range: ~1 MB/s,
restarting from byte zero on every network blip.

ExoPlayer now carries Jellyfin's media3 FFmpeg audio decoder in extension
mode ON (platform decoders first, FFmpeg for what they lack), and
CodecDetector reports its codecs so the device profile and the download
policy agree with what actually decodes. The download policy judges audio
against the renderer that will play the file (renderer_can_decode_audio)
instead of the webview's list, so Android downloads are always the direct
copy — a 910 MB E-AC-3 5.1 episode downloaded in 94 s and played offline.

The webview video path is removed on Android: it decodes none of these
codecs, so a stored "native video off" would play every original-file
download silent. Rust reports webview_video_fallback (false on Android, true
only beside mpv native video on Linux); Settings offers the switch and the
player honours it only then. Linux keeps the fallback and, with it, the
server transcode for undecodable audio.

The decoder is GPL-3.0; the distributed APK carries its terms and the source
stays MIT (THIRD_PARTY_NOTICES.md). The on-device remux spec this replaces is
folded into 05-platform-backends.md and deleted.

DR-293, UT-259, UT-262.
This commit is contained in:
2026-09-22 22:22:05 -04:00
parent bb7d5dc01a
commit bed1030443
19 changed files with 339 additions and 382 deletions
+47 -2
View File
@@ -95,8 +95,9 @@ flowchart LR
**Location**: `src/lib/player/html5Adapter.ts`, `src/lib/player/index.ts`, report commands in
`src-tauri/src/commands/player/timers.rs`
Video on desktop (Linux WebKitGTK) — and, per current interim behavior, Android — is rendered by an
HTML5 `<video>`/HLS element **inside the webview**. libmpv is initialized audio-only (`vo=null`,
Video on desktop (Linux WebKitGTK) is rendered by an HTML5 `<video>`/HLS element **inside the
webview**. Android no longer uses this path for video — see *The webview is not a video renderer on
Android* below. libmpv is initialized audio-only (`vo=null`,
`video=false`), so the native backend cannot render or observe this element. The `<video>` is therefore
the real player, living outside Rust's reach.
@@ -287,6 +288,50 @@ and the trait default is still a silent `Ok(())` rather than an error, so a back
that omits the method still reports success. Flipping that default waits on the
device verification.
### Licensed audio codecs: the FFmpeg extension
**TRACES**: UR-004, UR-071 | DR-293
Android does not ship AC-3, E-AC-3, DTS or TrueHD decoders — they are licensed
codecs, present only where a vendor paid for them. The ROD2-W09 test tablet has a
vendor DTS decoder and no AC-3/E-AC-3 at all. ExoPlayer has no decoders of its
own, so on such a device those tracks are undecodable, and before this every film
with Dolby audio was re-encoded by the server — for streaming *and* for download.
`JellyTauPlayer` builds ExoPlayer with `DefaultRenderersFactory` in
`EXTENSION_RENDERER_MODE_ON`: the platform's decoders are tried first (a vendor DTS
decoder stays in charge where there is one) and the FFmpeg audio renderer takes
what they cannot decode. `CodecDetector` reports the extension's codecs beside the
`MediaCodecList` ones, asking `FfmpegLibrary.supportsFormat` per MIME type rather
than assuming, so a build whose native library failed to load reports only what
the platform decodes. Rust's device profile and download policy read that list,
which is what keeps "what we tell the server" and "what actually decodes" in step.
The decoder is `org.jellyfin.media3:media3-ffmpeg-decoder` — Jellyfin's build of
media3's FFmpeg extension, versioned `<media3 version>+N`. **Bump it in the same
commit as media3.** It is GPL-3.0: the distributed APK carries those terms, the
source stays MIT (see `THIRD_PARTY_NOTICES.md`). Its JNI methods are covered by the
AAR's own consumer rules and by `-keep class androidx.media3.** { *; }` in
`proguard-jellytau.pro`, which also keeps the renderer ExoPlayer loads reflectively.
**Rejected:** re-encoding a download's audio on the device after it lands (a
remux). It costs minutes of CPU and twice the disk per film, needs a pipeline
state of its own, and does nothing for streaming. Decoding at playback fixes both
paths with no extra step.
### The webview is not a video renderer on Android
ExoPlayer is Android's only video renderer. The HTML5 path used to be reachable
through the `experimentalNativeVideo` setting (a *suppressor* of Rust's native
choice), but the webview decodes none of the codecs above — so with the original
file now downloaded as-is (DR-293), turning native video off would play every such
download as a silent film. Rust reports `webview_video_fallback` in
`PlaybackCapabilities`: **false on Android**, true only beside mpv native video on
Linux, where the webview is still the tested fallback. The frontend offers the
switch and honours a stored "off" only when it is true (`nativeVideoWanted` in
`stores/nativeVideo.ts`), so a user who once switched it off on Android is not
stranded on the silent path.
### The equalizer, and where its vocabulary lives
**TRACES**: UR-027 | DR-030, IR-020