feat(android): play the original file — decode Dolby/DTS audio with FFmpeg
Android ships no AC-3, E-AC-3, DTS or TrueHD decoders; they are licensed codecs, present only where a vendor paid for them. The ROD2-W09 tablet has a vendor DTS decoder and no AC-3/E-AC-3 at all. So every film with Dolby audio was re-encoded by the server, for streaming and for download alike, and a transcoded download has no Content-Length and ignores Range: ~1 MB/s, restarting from byte zero on every network blip. ExoPlayer now carries Jellyfin's media3 FFmpeg audio decoder in extension mode ON (platform decoders first, FFmpeg for what they lack), and CodecDetector reports its codecs so the device profile and the download policy agree with what actually decodes. The download policy judges audio against the renderer that will play the file (renderer_can_decode_audio) instead of the webview's list, so Android downloads are always the direct copy — a 910 MB E-AC-3 5.1 episode downloaded in 94 s and played offline. The webview video path is removed on Android: it decodes none of these codecs, so a stored "native video off" would play every original-file download silent. Rust reports webview_video_fallback (false on Android, true only beside mpv native video on Linux); Settings offers the switch and the player honours it only then. Linux keeps the fallback and, with it, the server transcode for undecodable audio. The decoder is GPL-3.0; the distributed APK carries its terms and the source stays MIT (THIRD_PARTY_NOTICES.md). The on-device remux spec this replaces is folded into 05-platform-backends.md and deleted. DR-293, UT-259, UT-262.
This commit is contained in:
@@ -0,0 +1,21 @@
|
||||
# Third-party notices
|
||||
|
||||
JellyTau's own source code is licensed under the MIT License (see `LICENSE`).
|
||||
Some builds bundle third-party components under other licences, listed here.
|
||||
|
||||
## Android: FFmpeg audio decoder (GPL-3.0)
|
||||
|
||||
The Android app bundles **`org.jellyfin.media3:media3-ffmpeg-decoder`**, the
|
||||
Jellyfin project's build of the media3 FFmpeg extension, which contains FFmpeg.
|
||||
It lets the player decode AC-3, E-AC-3, DTS and TrueHD audio, which Android does
|
||||
not ship.
|
||||
|
||||
- Licence: **GNU General Public License v3.0**
|
||||
- Source: <https://github.com/jellyfin/jellyfin-androidx-media> (build of
|
||||
<https://github.com/androidx/media>), with FFmpeg from <https://ffmpeg.org>
|
||||
|
||||
Because this component is GPL-3.0, **the Android APK as distributed is subject
|
||||
to the terms of the GPL-3.0**. The complete corresponding source for JellyTau is
|
||||
available in this repository; JellyTau's own code remains available under MIT.
|
||||
|
||||
Desktop builds do not include this component.
|
||||
@@ -95,8 +95,9 @@ flowchart LR
|
||||
**Location**: `src/lib/player/html5Adapter.ts`, `src/lib/player/index.ts`, report commands in
|
||||
`src-tauri/src/commands/player/timers.rs`
|
||||
|
||||
Video on desktop (Linux WebKitGTK) — and, per current interim behavior, Android — is rendered by an
|
||||
HTML5 `<video>`/HLS element **inside the webview**. libmpv is initialized audio-only (`vo=null`,
|
||||
Video on desktop (Linux WebKitGTK) is rendered by an HTML5 `<video>`/HLS element **inside the
|
||||
webview**. Android no longer uses this path for video — see *The webview is not a video renderer on
|
||||
Android* below. libmpv is initialized audio-only (`vo=null`,
|
||||
`video=false`), so the native backend cannot render or observe this element. The `<video>` is therefore
|
||||
the real player, living outside Rust's reach.
|
||||
|
||||
@@ -287,6 +288,50 @@ and the trait default is still a silent `Ok(())` rather than an error, so a back
|
||||
that omits the method still reports success. Flipping that default waits on the
|
||||
device verification.
|
||||
|
||||
### Licensed audio codecs: the FFmpeg extension
|
||||
|
||||
**TRACES**: UR-004, UR-071 | DR-293
|
||||
|
||||
Android does not ship AC-3, E-AC-3, DTS or TrueHD decoders — they are licensed
|
||||
codecs, present only where a vendor paid for them. The ROD2-W09 test tablet has a
|
||||
vendor DTS decoder and no AC-3/E-AC-3 at all. ExoPlayer has no decoders of its
|
||||
own, so on such a device those tracks are undecodable, and before this every film
|
||||
with Dolby audio was re-encoded by the server — for streaming *and* for download.
|
||||
|
||||
`JellyTauPlayer` builds ExoPlayer with `DefaultRenderersFactory` in
|
||||
`EXTENSION_RENDERER_MODE_ON`: the platform's decoders are tried first (a vendor DTS
|
||||
decoder stays in charge where there is one) and the FFmpeg audio renderer takes
|
||||
what they cannot decode. `CodecDetector` reports the extension's codecs beside the
|
||||
`MediaCodecList` ones, asking `FfmpegLibrary.supportsFormat` per MIME type rather
|
||||
than assuming, so a build whose native library failed to load reports only what
|
||||
the platform decodes. Rust's device profile and download policy read that list,
|
||||
which is what keeps "what we tell the server" and "what actually decodes" in step.
|
||||
|
||||
The decoder is `org.jellyfin.media3:media3-ffmpeg-decoder` — Jellyfin's build of
|
||||
media3's FFmpeg extension, versioned `<media3 version>+N`. **Bump it in the same
|
||||
commit as media3.** It is GPL-3.0: the distributed APK carries those terms, the
|
||||
source stays MIT (see `THIRD_PARTY_NOTICES.md`). Its JNI methods are covered by the
|
||||
AAR's own consumer rules and by `-keep class androidx.media3.** { *; }` in
|
||||
`proguard-jellytau.pro`, which also keeps the renderer ExoPlayer loads reflectively.
|
||||
|
||||
**Rejected:** re-encoding a download's audio on the device after it lands (a
|
||||
remux). It costs minutes of CPU and twice the disk per film, needs a pipeline
|
||||
state of its own, and does nothing for streaming. Decoding at playback fixes both
|
||||
paths with no extra step.
|
||||
|
||||
### The webview is not a video renderer on Android
|
||||
|
||||
ExoPlayer is Android's only video renderer. The HTML5 path used to be reachable
|
||||
through the `experimentalNativeVideo` setting (a *suppressor* of Rust's native
|
||||
choice), but the webview decodes none of the codecs above — so with the original
|
||||
file now downloaded as-is (DR-293), turning native video off would play every such
|
||||
download as a silent film. Rust reports `webview_video_fallback` in
|
||||
`PlaybackCapabilities`: **false on Android**, true only beside mpv native video on
|
||||
Linux, where the webview is still the tested fallback. The frontend offers the
|
||||
switch and honours a stored "off" only when it is true (`nativeVideoWanted` in
|
||||
`stores/nativeVideo.ts`), so a user who once switched it off on Android is not
|
||||
stranded on the silent path.
|
||||
|
||||
### The equalizer, and where its vocabulary lives
|
||||
|
||||
**TRACES**: UR-027 | DR-030, IR-020
|
||||
|
||||
+10
-3
@@ -494,6 +494,8 @@ Internal architecture, components, and application logic.
|
||||
| DR-290 | A download whose response states no length still reports progress against a predicted total. A transcode is produced as it is sent — chunked, no `Content-Length` — and the worker reported `progress: 0.0` for its whole duration: an empty bar reading "0%" while the byte count climbed for an hour, which is the case every film whose audio must be re-encoded lands in. The backend already fetches the item to decide the audio policy, and that item carries what a prediction needs: the source's size (an `original` download copies the picture, so the output is the source give or take the audio track — and exactly the source when nothing is re-encoded) and its runtime (a preset re-encodes at fixed rates, so the size is rate × runtime, from the same preset table the URL is built from so the two cannot drift). The prediction is made where the URL is resolved and persisted as the row's `file_size`; the worker uses it **only** when the response has no length, the server's figure always wins, an estimated bar is capped at 99% so a low prediction never shows a finished download still running, and the `Completed` event carries the bytes actually written so neither side persists the prediction as the real size. With no prediction the bar is indeterminate, which is honest and was the status quo. The single-video button joins the series/season buttons on the enqueue path so all three resolve — and predict — in one place | Downloads | UR-071 | Done |
|
||||
| DR-291 | The offline banner stays off the full-screen player. Every other shell rule in `layoutShell.ts` already treats `/player/*` as immersive; the amber "You're offline" strip was the one piece of chrome still rendered above it. On the native Android video path that is not cosmetic: VideoPlayer makes itself transparent so the ExoPlayer SurfaceView behind the WebView is visible (DR-185), so a shell child that still paints shows *through* the picture as a stripe across the top of the film. Offline is also precisely when a downloaded video plays, so the banner appeared when it was most in the way, and it offers the viewer nothing to act on — local playback needs no server. The rule moves into the pure module as `showOfflineBanner({ pathname, isAuthenticated, isConnected })` rather than staying an inline `{#if}` in the shell, so the immersive-route contract is stated in one tested place | UI | UR-003, UR-043 | Done |
|
||||
| DR-292 | The offline catalog reveal is one rule, applied by both library views. Two defects, one cause — "server only" was a private `$derived` inside `MediaCard`. (1) The list view (`LibraryListView`, what `LibraryGrid` renders when the stored view preference is `list`) had no notion of it at all, so a library browsed as a list offline showed every revealed item as an ordinary tappable row that plays nothing, with no way to queue it. (2) The rule asked the downloads store whether *this item id* was downloaded, but only a playable leaf (Audio, Movie, Episode) ever has a download row — an album's tracks carry them, the album does not — so a fully downloaded album greyed itself out and offered to queue what was already on the device, which is what "my downloaded music is greyed out" was. The rule moves to the pure `$lib/utils/serverOnly`, both views call it, and the container half is answered by the backend: `get_download_disk_usage().sizes` already carries container subtotals beside leaf sizes (DR-085), so `deviceContentIds` is membership in a Rust-computed map rather than a frontend guess at which item types are containers. That map was loaded only by the Downloads page, so the shell now primes it at startup and re-reads it whenever the offline gate settles (the DR-143 signal). Queueing is shared too (`queueOfflineDownload`), since the list view had no copy to diverge from | UI | UR-052, UR-055 | Done |
|
||||
| DR-293 | Android plays the original file: ExoPlayer decodes AC-3, E-AC-3, DTS and TrueHD in software through the FFmpeg extension, so neither a download nor a stream needs the server to re-encode its audio. These are licensed codecs that Android does not ship — the ROD2-W09 tablet has a vendor DTS decoder and no AC-3/E-AC-3 at all — so the download policy (DR-171) judged audio against the webview's list and turned most films into a server transcode: generated as it is sent, no `Content-Length`, `Range` ignored, measured at ~1 MB/s and restarting from zero on every network blip, against a direct copy that moved a 910 MB episode in 94 s. The renderer is `DefaultRenderersFactory` in `EXTENSION_RENDERER_MODE_ON` (platform decoders first, FFmpeg for what they lack), and `CodecDetector` reports the extension's codecs beside `MediaCodecList`'s, so the device profile and the download policy — now `renderer_can_decode_audio`, DR-234's per-platform answer, instead of the webview's list — agree with what actually decodes. The webview video path is gone on Android: it decodes none of those codecs, so an original-file download would play there as a silent film; `webview_video_fallback` (Rust) is false on Android and the frontend neither offers the switch nor honours a stored "off". Linux keeps the webview fallback beside mpv native video, and with it the server transcode for undecodable audio. Rejected: re-encoding audio on the device after download — minutes of CPU and twice the disk per film, and it would not have helped streaming. The decoder is Jellyfin's `media3-ffmpeg-decoder` build (GPL-3.0; the distributed APK carries its terms, the source stays MIT) and must be versioned in step with media3 | Playback | UR-004, UR-071 | Done |
|
||||
| DR-294 | A download plays with no network. Playing a downloaded item asked the server for its `PlaybackInfo` — only to read the media-source id that subtitle URLs are keyed by — and `HybridRepository::get_playback_info` went to the server alone, so offline the call retried for seven seconds, failed, and the file on disk was never opened. A completed download for the current user now answers playback info from its download row, first and regardless of reachability: the local path, direct play, and the item id as media-source id (a download names no source, so the server served its default, which carries the item's id). Next Up had the same shape — server-only — and the TV landing page loads it in one `Promise.all` with its other rows, so offline that single failure blanked the whole page with Continue Watching and Latest sitting in the cache; it now falls back to the cache when the server cannot answer. And a slow cache read is waited for, never discarded: the cache is one SQLite connection behind one mutex, so any write in progress (the catalog sync at every launch, a download finishing) pushes a read past the 100 ms fast path, and `get_items`, the library list, genres and playlist items discarded such a read, waited on the server, and offline returned its error over data on disk — "More info" on a downloaded show failed exactly so. They keep the read running (`cache_try`) and wait for it when the server fails (`settle`); the cache-only reads (search, favourites) simply await the cache | Repository | UR-002, UR-071 | Done |
|
||||
|
||||
---
|
||||
|
||||
@@ -504,9 +506,9 @@ Internal architecture, components, and application logic.
|
||||
| User Req | Integration Requirements | Development Requirements |
|
||||
|----------|-------------------------|-------------------------|
|
||||
| UR-001 | IR-001, IR-002 | - |
|
||||
| UR-002 | IR-013 | DR-003, DR-012, DR-013, DR-014 |
|
||||
| UR-002 | IR-013 | DR-003, DR-012, DR-013, DR-014, DR-294 |
|
||||
| UR-003 | IR-003, IR-004, IR-011 | DR-002, DR-004, DR-010, DR-182, DR-183, DR-184, DR-185, DR-186, DR-187, DR-188, DR-190, DR-191, DR-192, DR-193, DR-194, DR-195, DR-196, DR-291 |
|
||||
| UR-004 | IR-003, IR-004, IR-008, IR-011 | DR-002, DR-004, DR-006, DR-129, DR-171, DR-176, DR-177, DR-181, DR-182, DR-183, DR-185, DR-188, DR-203, DR-265 |
|
||||
| UR-004 | IR-003, IR-004, IR-008, IR-011 | DR-002, DR-004, DR-006, DR-129, DR-171, DR-176, DR-177, DR-181, DR-182, DR-183, DR-185, DR-188, DR-203, DR-265, DR-293 |
|
||||
| UR-005 | - | DR-001, DR-005, DR-009, DR-178, DR-179, DR-186, DR-193, DR-195 |
|
||||
| UR-006 | IR-005, IR-006, IR-007, IR-008 | DR-200, DR-201 |
|
||||
| UR-007 | IR-010 | DR-007, DR-008, DR-016, DR-257, DR-262, DR-277, DR-278 |
|
||||
@@ -572,7 +574,7 @@ Internal architecture, components, and application logic.
|
||||
| UR-068 | - | DR-119 |
|
||||
| UR-069 | - | DR-113, DR-114, DR-120 |
|
||||
| UR-070 | - | DR-121, DR-122 |
|
||||
| UR-071 | IR-032 | DR-123, DR-124, DR-125, DR-126, DR-127, DR-128, DR-133, DR-134, DR-135, DR-136, DR-137, DR-138, DR-170, DR-171, DR-180, DR-198, DR-199, DR-289, DR-290 |
|
||||
| UR-071 | IR-032 | DR-123, DR-124, DR-125, DR-126, DR-127, DR-128, DR-133, DR-134, DR-135, DR-136, DR-137, DR-138, DR-170, DR-171, DR-180, DR-198, DR-199, DR-289, DR-290, DR-293, DR-294 |
|
||||
| UR-072 | - | DR-156 |
|
||||
| UR-073 | - | DR-158 |
|
||||
| UR-074 | - | DR-162, DR-177, DR-181 |
|
||||
@@ -847,6 +849,11 @@ Internal architecture, components, and application logic.
|
||||
| UT-255 | The offline banner shows while offline on ordinary routes and never on `/player/*`, and stays off while connected or signed out | DR-291 | Done |
|
||||
| UT-257 | The server-only rule: true only offline with the reveal on and nothing on the device; never for a library tile; and not for a container whose children are downloaded (the greyed-album regression) | DR-292 | Done |
|
||||
| UT-258 | The list view greys a server-only row, makes it inert to tap, offers the queue button (and the Queued badge once pending), and leaves downloaded rows and containers with device content alone | DR-292 | Done |
|
||||
| UT-259 | The user may send video to the webview only beside mpv native video on Linux: never on Android, where ExoPlayer is the only video renderer, and not where the webview is the only renderer | DR-293 | Done |
|
||||
| UT-260 | A downloaded item gets playback info with the server unreachable — immediately, from its download row (local path, direct play, item id as media source) — while an unfinished download, another user's, or an item never downloaded is left to the server | DR-294 | Done |
|
||||
| UT-261 | Next Up answers from the cache, rather than failing, when the server is unreachable | DR-294 | Done |
|
||||
| UT-262 | The Android webview fallback is neither offered in Settings nor honoured by the player unless Rust reports it, so a stored "native video off" cannot route video to a renderer that plays the original file silent | DR-293 | Done |
|
||||
| UT-263 | With the database held past the 100 ms fast path and the server unreachable, `get_items`, the library list, a cache-only search and cache-only favourites all answer from the cache instead of failing | DR-294 | Done |
|
||||
### Integration Tests
|
||||
|
||||
| Test ID | Test Description | Traces To | Status |
|
||||
|
||||
@@ -28,8 +28,10 @@ know how something *works*, read
|
||||
|
||||
**Next free requirement ids** (always re-check
|
||||
[requirements.md](../requirements.md) before allocating): **UR-086**,
|
||||
**IR-036**, **JA-038**, **DR-291** (DR-289/290 went to the download stall
|
||||
timeout and the transcode progress estimate in v0.12.2). Three specs below suggested ids that have
|
||||
**IR-036**, **JA-038**, **DR-295**, **UT-264** (DR-289/290 went to the v0.12.2
|
||||
download fixes; DR-291/292 and UT-255/257/258 to the offline-banner and
|
||||
server-only-reveal work; DR-293/294 and UT-259-263 to Android's FFmpeg decoding
|
||||
and offline-without-network). Three specs below suggested ids that have
|
||||
since been taken by other work; each carries a ⚠️ note at the top — this line
|
||||
was itself stale by five, two and forty-seven until 2026-09-08, which is why the
|
||||
re-check is not optional.
|
||||
@@ -50,7 +52,6 @@ re-check is not optional.
|
||||
|---|---|
|
||||
| [desktop-native-video.md](desktop-native-video.md) | mpv draws video on every desktop platform, then the webview `<video>` path and hls.js are deleted. Converts a measured 7% direct-play rate toward Android's 85%. Stacked on backend-owned stream selection. |
|
||||
| [backend-owned-stream-selection.md](backend-owned-stream-selection.md) | Rust owns direct-play-vs-transcode, transport and quality; players consume one `StreamSelection`. Partly built — `StreamSelection`, `Transport` and the `.m3u8` sniff removal have landed. |
|
||||
| [on-device-audio-remux.md](on-device-audio-remux.md) | Downloads fetch the original (`Static=true`, resumable — measured 14× faster) and a bundled FFmpeg re-encodes only the audio on device, replacing DR-171's server-side transcode. HEVC side effect accepted (Android-first; the webview video path is being retired). Android APK size increase still to be measured. |
|
||||
| [build-provenance.md](build-provenance.md) | `build.rs` is still bare. ⚠️ suggested id DR-093 is taken. |
|
||||
| [player-facade-enforcement.md](player-facade-enforcement.md) | ~60 `commands.player*` sites still outside the facade; no lint rule. ⚠️ suggested id DR-095 is taken. |
|
||||
| [windows-native-audio-backend.md](windows-native-audio-backend.md) | Blocked on the libmpv2 swap. ⚠️ suggested id IR-030 is taken. |
|
||||
@@ -84,4 +85,5 @@ Where to look for each:
|
||||
| Video background audio | [05-platform-backends.md](../architecture/05-platform-backends.md) — Background Audio Handoff |
|
||||
| Traceability gate repair | [traceability-ci.md](../traceability-ci.md) |
|
||||
| Boundary tripwire hardening | `scripts/check-frontend-boundary.sh` (its header is the spec) |
|
||||
| Original-file downloads & Android FFmpeg decoding (was on-device-audio-remux) | [05-platform-backends.md](../architecture/05-platform-backends.md) — Licensed audio codecs; [06-downloads-and-offline.md](../architecture/06-downloads-and-offline.md) — What a Video Download Fetches, Offline Means No Network |
|
||||
| Playback docs corrections · req-coverage script removal | Nothing to document — both were corrections that have been applied |
|
||||
|
||||
@@ -1,327 +0,0 @@
|
||||
# Spec: downloads fetch the original file and fix the audio on device
|
||||
|
||||
**Status:** Proposed
|
||||
**Requirements:** UR-071, UR-004 → DR-291 (new). Revises **DR-171**, which keeps
|
||||
its diagnosis and loses its remedy — see [Relationship to DR-171](#relationship-to-dr-171).
|
||||
**UX spec:** n/a — one new row state in the existing transfers list.
|
||||
**Destination on completion:** [06-downloads-and-offline.md](../architecture/06-downloads-and-offline.md)
|
||||
— a new "Post-download processing" section after the download worker, plus a
|
||||
rewrite of the audio-policy paragraph that currently describes DR-171's
|
||||
server-side remedy.
|
||||
|
||||
## Summary
|
||||
|
||||
A video download always fetches the server's original bytes (`Static=true`) and,
|
||||
when the source carries audio this app's renderers cannot decode, re-encodes
|
||||
that audio **on the device** with a bundled FFmpeg instead of asking the server
|
||||
to transcode the whole film on the fly. Downloads become byte-range resumable
|
||||
again — the property that makes them reliable — and the server stops spending a
|
||||
CPU-hour per saved film.
|
||||
|
||||
## Motivation
|
||||
|
||||
A Jellyfin transcode is generated as it is sent: chunked, no `Content-Length`,
|
||||
and `Range` ignored. Every interruption therefore restarts it from byte zero,
|
||||
and with the queue at three concurrent downloads it is also three FFmpeg jobs on
|
||||
the server.
|
||||
|
||||
Measured on the ROD2-W09 tablet against the production server (2026-09-22):
|
||||
|
||||
| Download kind | Throughput | Retries | Resumable |
|
||||
|---|---|---|---|
|
||||
| `Static=true` direct copy | **2.06 GB in 142 s** (~14.5 MB/s) | 0 | yes (HTTP 206) |
|
||||
| Server transcode | ~1 MB/s | restarts from 0 on any blip | no (HTTP 200) |
|
||||
|
||||
That is a ~14× throughput difference, and the transcode's failure mode is
|
||||
unbounded: three retries × a full restart each is the whole file fetched four
|
||||
times, which is what DR-289 was masking and DR-290 was papering over.
|
||||
|
||||
The transcode is only ever requested because of the audio track. Roughly every
|
||||
AC-3/E-AC-3 film hits it on this device (no `audio/ac3` or `audio/eac3` decoder
|
||||
in `MediaCodecList`), so on a typical library the slow, non-resumable path is
|
||||
the common one, not the exception.
|
||||
|
||||
### Relationship to DR-171
|
||||
|
||||
DR-171 is **right about the defect and wrong about the remedy**, and this spec
|
||||
keeps the first half intact.
|
||||
|
||||
The defect: `Static=true` hands back the source untouched, E-AC-3 track
|
||||
included, and a downloaded film played as picture in silence while the same film
|
||||
had sound when streamed. Offline, the download is the only source a video has,
|
||||
so there is no working path to fall back to.
|
||||
|
||||
DR-171's remedy was to ask the server for a transcode, and it explicitly
|
||||
accepted the cost: *"the transcode costs the byte-range resumability
|
||||
`Static=true` gives the download worker"*. That cost is now measured, and it is
|
||||
the dominant one.
|
||||
|
||||
DR-171 also chose to judge the codec against the **webview's** list rather than
|
||||
the device's, reasoning that *"a downloaded file outlives whatever
|
||||
`experimentalNativeVideo` was set to when it arrived"*. **That reasoning
|
||||
survives this spec and constrains it**: whatever lands on disk must play on
|
||||
either renderer, on either platform, years after the setting that was active
|
||||
when it arrived. It is the reason this spec fixes the bytes on disk rather than
|
||||
teaching one renderer to cope (see [Rejected alternatives](#rejected-alternatives)).
|
||||
|
||||
## Layer assignment
|
||||
|
||||
| Logic / responsibility | Layer | Why it belongs there |
|
||||
|---|---|---|
|
||||
| Whether a source's audio needs re-encoding at all | **Rust** | Domain: it compares a Jellyfin-reported codec against renderer capability. Changes when Jellyfin's stream metadata or a platform's decoder set changes. Already Rust (`served_audio_codec` + `webview_can_decode_audio`); unchanged by this spec. |
|
||||
| Which download URL to request (`Static=true` vs a transcode) | **Rust** | Domain: a Jellyfin route with Jellyfin query semantics. This spec changes the *answer* (always static) but not the owner. |
|
||||
| Target codec/bitrate/channel layout of the local re-encode | **Rust** | Domain: it must satisfy the same renderer-capability rule as the streaming profile. A frontend-chosen value would be a second, drifting copy of DR-149's policy. |
|
||||
| Running FFmpeg, and the remux invariants (copy video, re-encode audio only, preserve subtitle/chapter streams) | **Rust** | Backend owns downloads end to end (CLAUDE.md). It is also the only side that can see the file. |
|
||||
| Deciding a row is not yet available offline while processing | **Rust** | Business rule about download state. Falls out for free: every offline query already gates on `status = 'completed'`. |
|
||||
| Re-encode progress as a fraction | **Rust** | Computed from FFmpeg's reported position against the item runtime — domain data. Emitted like download progress. |
|
||||
| Rendering the `processing` state (label, spinner, bar) | **Frontend** | Pure presentation; changes only if the transfers list is redesigned. |
|
||||
| Whether to show the transfers list sorted with processing rows first | **Frontend** | View ordering preference. |
|
||||
|
||||
Borderline row, stated with its tie-breaker: **"is FFmpeg available on this
|
||||
build?"** could read as an environment/presentation concern. It is placed in
|
||||
**Rust** because the answer changes what URL is requested (a fallback build must
|
||||
go back to asking the server to transcode), and that is a domain decision, not a
|
||||
capability the UI reasons about. The frontend never sees the flag.
|
||||
|
||||
## Design
|
||||
|
||||
### Pipeline
|
||||
|
||||
```
|
||||
resolve → download (Static=true, resumable) → [needs audio fix?] → play
|
||||
│ yes
|
||||
▼
|
||||
remux on device (FFmpeg)
|
||||
status = 'processing'
|
||||
│
|
||||
▼
|
||||
replace file, status = 'completed'
|
||||
```
|
||||
|
||||
1. `resolve_video_download` keeps judging the served audio codec exactly as
|
||||
today, but the verdict no longer selects a URL. The URL is always
|
||||
`Static=true` (plus `mediaSourceId`). The verdict is **persisted on the row**
|
||||
as `needs_audio_remux` so the worker does not have to re-fetch the item, and
|
||||
so a resumed download decided before a server change keeps its own verdict.
|
||||
2. The worker downloads as it does now. It gets a real `Content-Length` and
|
||||
HTTP 206 on resume, so DR-290's estimate machinery becomes a fallback rather
|
||||
than the normal case.
|
||||
3. On completion, a row with `needs_audio_remux = 1` moves to `processing`
|
||||
rather than `completed`, and a new `download::remux` module runs FFmpeg:
|
||||
`-c:v copy -c:s copy -c:a aac -b:a 384k -ac <min(source, 6)>`, writing to
|
||||
`<target>.remux.mp4` and renaming over the original only on success.
|
||||
4. The row goes `completed` with `file_size` set to the **remuxed** file's size.
|
||||
|
||||
The video stream is copied, so this is I/O plus an audio encode — minutes on a
|
||||
feature film, not the hours a video re-encode would take. Peak disk is ~2× the
|
||||
film for the duration of the remux; the spec's acceptance criteria include
|
||||
refusing to start a remux without that headroom.
|
||||
|
||||
### Why `processing` is a status and not a flag
|
||||
|
||||
Every offline query already reads `status = 'completed'` (`offline.rs:36`,
|
||||
`:65`, `:107`; `repository/offline.rs:321`, `:330`, `:970`, `:979`, `:1150`).
|
||||
A new status that is simply *not* `completed` therefore excludes a
|
||||
mid-remux file from offline browsing with **no query changes** — a file whose
|
||||
audio has not been fixed yet is exactly as unplayable as one still downloading,
|
||||
and the existing gate already says so.
|
||||
|
||||
### Wire shapes
|
||||
|
||||
Status vocabulary gains one value, on both sides:
|
||||
|
||||
```rust
|
||||
// schema.rs comment + DownloadInfo
|
||||
status TEXT DEFAULT 'pending', -- pending, downloading, processing, completed, failed, paused
|
||||
```
|
||||
|
||||
```rust
|
||||
// download/events.rs — the existing kebab-case "download-event" channel
|
||||
#[serde(rename_all = "camelCase")]
|
||||
Processing {
|
||||
download_id: i64,
|
||||
item_id: String,
|
||||
/// 0.0..=1.0, from FFmpeg's position against the item runtime.
|
||||
progress: f64,
|
||||
},
|
||||
```
|
||||
|
||||
```typescript
|
||||
// stores/downloads.ts
|
||||
status: "pending" | "downloading" | "processing" | "completed" | "failed" | "paused";
|
||||
type: … | "processing";
|
||||
```
|
||||
|
||||
No new command is needed: the frontend already subscribes to `download-event`
|
||||
and renders whatever status the row carries. `bindings.ts` is regenerated from
|
||||
Rust (`cargo test export_typescript_bindings`), not hand-edited.
|
||||
|
||||
**The status arrives twice and both payloads carry the new value**: once as
|
||||
`DownloadInfo.status` from `get_downloads` (the refresh path, e.g. after an app
|
||||
restart mid-remux) and once as the `Processing` event. A build that taught only
|
||||
the event about `processing` would show a correct live remux and a row stuck on
|
||||
"Downloading" after any refresh.
|
||||
|
||||
New row column:
|
||||
|
||||
```sql
|
||||
ALTER TABLE downloads ADD COLUMN needs_audio_remux INTEGER DEFAULT 0;
|
||||
```
|
||||
|
||||
### Packaging FFmpeg
|
||||
|
||||
Both platforms link **libav\*** from Rust rather than shelling out to an
|
||||
`ffmpeg` binary, so the remux is one code path with one set of tests.
|
||||
|
||||
- **Android**: build a minimal FFmpeg for `arm64-v8a` (plus the other ABIs the
|
||||
universal APK carries) enabling only what the job needs — decoders
|
||||
`ac3,eac3,dts,truehd`, encoder `aac`, demuxers/muxers `mov,mp4,matroska`,
|
||||
and the `copy` bitstream path. **The size of that build is an open number this
|
||||
spec requires measuring before acceptance**, not an estimate to design
|
||||
around; if it exceeds a budget the maintainer sets, drop TrueHD and DTS-HD
|
||||
first (rarest, largest tables).
|
||||
- **Linux**: the AppImage already ships an FFmpeg stack behind libmpv, but those
|
||||
are libmpv's private libraries and must not be linked against directly. Build
|
||||
or link the same minimal set as Android.
|
||||
- **CI**: per CLAUDE.md, the FFmpeg toolchain and the prebuilt libraries live in
|
||||
the builder image (`Dockerfile.builder`, rebuilt and pushed via
|
||||
`scripts/build-builder-image.sh`). Nothing is fetched or compiled at job time.
|
||||
|
||||
### Fallback, so this ships incrementally
|
||||
|
||||
If the build has no FFmpeg, or the remux fails for any reason, the item falls
|
||||
back to **today's behaviour**: request the server-side transcode URL. This means
|
||||
the change can land platform by platform, a remux bug degrades to the current
|
||||
(working, slow) path rather than to a silent film, and the DR-171 defect cannot
|
||||
reappear. The fallback is logged at `warn` so it is visible rather than silent.
|
||||
|
||||
## Rejected alternatives
|
||||
|
||||
Recorded because each is the obvious next idea and each has a specific reason it
|
||||
fails — this is the half that gets folded into the architecture doc.
|
||||
|
||||
- **Teach ExoPlayer to decode AC-3 via media3's FFmpeg decoder extension.**
|
||||
Cheaper-looking: no file is rewritten, and it would fix streaming direct-play
|
||||
for the same codecs. It fails DR-171's surviving constraint — a downloaded
|
||||
file outlives the `experimentalNativeVideo` setting, and on Linux the webview
|
||||
draws video, so the webview path would still be silent. It fixes one renderer;
|
||||
the bytes on disk have to satisfy all of them. Worth doing **separately** as a
|
||||
streaming optimisation, where the constraint does not apply.
|
||||
- **Use the device's own MediaCodec to decode AC-3.** The test device reports no
|
||||
`audio/ac3` or `audio/eac3` decoder at all, which is the reason the transcode
|
||||
is requested there in the first place. It is not a decoder we can assume.
|
||||
- **A pure-Rust decoder (Symphonia).** Symphonia does not implement AC-3, E-AC-3
|
||||
or DTS. There is no pure-Rust path for the codecs that actually matter here.
|
||||
- **Download the server's HLS transcode segment by segment and remux locally.**
|
||||
Segments are individually addressable, so this would restore resumability at
|
||||
segment granularity without decoding anything on device. Rejected because it
|
||||
still spends the server CPU this spec is trying to stop spending, still needs
|
||||
an mp4 muxer on device (most of the same dependency), and HLS transcode
|
||||
sessions expire — a download paused overnight would find its segments gone.
|
||||
- **Do nothing and raise the retry budget.** The transcode restarts from byte
|
||||
zero, so each extra retry is another full-file fetch. More retries buys a
|
||||
linear increase in bytes moved for a fixed, low probability of finishing.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- **Files already downloaded stay as they are.** DR-171's closing note applies
|
||||
unchanged: the bytes on disk are the wrong bytes, and only a re-download (or a
|
||||
future one-off "repair downloads" pass) replaces them. If that pass is wanted,
|
||||
it belongs beside the remux module as a command that re-runs it over existing
|
||||
`completed` rows — noted there, not left to this file, which will be deleted.
|
||||
- **Streaming.** DR-149's server-side transcode for undecodable audio is
|
||||
untouched; a stream has no file to fix and no resumability to lose. The media3
|
||||
decoder extension above is the lever there.
|
||||
- **Video re-encoding on device.** HEVC sources that the *webview* cannot render
|
||||
are currently converted to h264 as a side effect of the server transcode.
|
||||
Under this spec an HEVC source with fine audio is copied verbatim and plays on
|
||||
ExoPlayer but not in the webview.
|
||||
|
||||
**Decided 2026-09-22: accept it.** The project is Android-first until video
|
||||
can be 100% mpv, and both ends of that road decode HEVC — ExoPlayer does today,
|
||||
mpv will on the desktop ([desktop-native-video.md](desktop-native-video.md)).
|
||||
The webview `<video>` element is the renderer being retired, so spending a
|
||||
full server transcode of every HEVC film to keep a path alive that is on its
|
||||
way out is the wrong trade. The exposure while it lasts: an HEVC film
|
||||
downloaded on Android, then played with `experimentalNativeVideo` turned off,
|
||||
shows no picture. Unlike DR-171's silent audio this is **recoverable without
|
||||
re-downloading** — turn the setting back on — which is what makes it
|
||||
acceptable where the audio case was not.
|
||||
- Quality presets (`high`/`medium`/`low`) keep asking the server to transcode —
|
||||
they are a deliberate request for smaller files, the server does it better,
|
||||
and their non-resumability is a known cost of a choice the user made.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- [ ] A film whose audio needs fixing downloads over `Static=true`, reports a
|
||||
real `Content-Length`, and resumes from a kill -9 with HTTP 206.
|
||||
- [ ] That film, played offline afterwards, has sound on both ExoPlayer and the
|
||||
webview `<video>` element.
|
||||
- [ ] Its row passes through `processing` and is absent from offline browsing
|
||||
until the remux finishes.
|
||||
- [ ] A remux failure leaves the row `failed` with the partial output removed,
|
||||
never `completed` over a half-written file.
|
||||
- [ ] A build without FFmpeg still downloads the same film correctly via the
|
||||
server transcode, with a `warn` naming the fallback.
|
||||
- [ ] The Android APK size increase is measured and recorded in the PR.
|
||||
- [ ] `bun run check` and `bun run test` pass.
|
||||
- [ ] `cargo fmt` clean, `cargo clippy --all-targets -- -D warnings` clean,
|
||||
`bun run test:rust` passes.
|
||||
- [ ] `bun run check:boundary` passes.
|
||||
- [ ] New requirement-implementing code carries `// TRACES:` comments and
|
||||
`bun run traces:validate` passes.
|
||||
- [ ] `bindings.ts` regenerated from Rust.
|
||||
|
||||
## Testing
|
||||
|
||||
**Rust (`cargo test`)**
|
||||
|
||||
- The URL builder returns `Static=true` for every quality-`original` case,
|
||||
including the codecs that previously forced a transcode — the inverse of
|
||||
today's `test_video_download_url_original_transcodes_undecodable_audio`, which
|
||||
this spec **rewrites rather than deletes** (its `Static=true` assertion for
|
||||
playable codecs stays).
|
||||
- `needs_audio_remux` is persisted from the same verdict the old URL choice used,
|
||||
at all three resolution sites.
|
||||
- Remux argument construction is a pure function over (source codec, channel
|
||||
count) and is tested without invoking FFmpeg: video is copied, audio is `aac`,
|
||||
subtitles survive, no bitrate or scale filter appears.
|
||||
- A row in `processing` is excluded by `offline_is_available`.
|
||||
- Disk-headroom refusal: a remux is not started when free space is under the
|
||||
output estimate.
|
||||
|
||||
**Frontend (vitest)**
|
||||
|
||||
- `describeProgress` and the transfers row render `processing` distinctly from
|
||||
`downloading` — extend `downloadProgress.test.ts`.
|
||||
- The store carries `processing` through and does not treat it as terminal —
|
||||
extend `downloads.test.ts`.
|
||||
|
||||
**On device** — the acceptance criteria above are the manual pass; the tablet
|
||||
is the right target because it is the one with no AC-3 decoder.
|
||||
|
||||
## TRACES
|
||||
|
||||
| Piece | Tag |
|
||||
|---|---|
|
||||
| Always-static download URL + persisted remux verdict | `// TRACES: UR-071, UR-004 \| DR-291 \| UT-255` |
|
||||
| `download::remux` module (argument construction, invariants) | `// TRACES: UR-071 \| DR-291 \| UT-256` |
|
||||
| `processing` status + `Processing` event | `// TRACES: UR-071 \| DR-291 \| UT-257` |
|
||||
| Frontend `processing` rendering | `// TRACES: UR-071 \| DR-291 \| UT-258` |
|
||||
|
||||
Allocate **DR-291** in [requirements.md](../requirements.md) (DR-290 is taken by
|
||||
the progress-estimate work) and **UT-255**–**UT-258**. Re-check the maxima
|
||||
before allocating: the specs index has been stale about this before.
|
||||
|
||||
## Notes for the implementer
|
||||
|
||||
- **A parallel Claude session may be active in this repo.** Run `git diff`
|
||||
before "repairing" unexpected changes — see the CLAUDE.md gotchas.
|
||||
- DR-290's estimate code is not dead after this lands: quality presets still
|
||||
transcode server-side, and the FFmpeg-less fallback still exists. Leave it.
|
||||
- The remux must not run on the tokio runtime's I/O threads — it is a long CPU
|
||||
job. Use a blocking task, and honour the existing `download::stop` flag so a
|
||||
cancel during processing is not ignored.
|
||||
- Editing Android sources means editing `src-tauri/android/src` and running
|
||||
`scripts/sync-android-sources.sh`; never edit the `gen/` tree.
|
||||
- Do not run `./gradlew` directly in `gen/android` — build through `scripts/`.
|
||||
@@ -172,6 +172,16 @@ dependencies {
|
||||
// itself: without a view to hand them to, a selected subtitle track renders
|
||||
// nowhere. See JellyTauPlayer.onCues. (DR-260)
|
||||
implementation("androidx.media3:media3-ui:1.5.0")
|
||||
// Software audio decoders for what Android does not ship: AC-3, E-AC-3,
|
||||
// DTS and TrueHD are licensed codecs, present only where a vendor paid for
|
||||
// them (the ROD2-W09 tablet has DTS but no AC-3/E-AC-3 at all). With this,
|
||||
// ExoPlayer plays the source file as-is, so neither a download nor a stream
|
||||
// needs the server to re-encode its audio (DR-293). Jellyfin's own build of
|
||||
// the media3 FFmpeg extension, versioned to match media3 above — keep the
|
||||
// two in step. Licence: GPL-3.0 — the distributed APK carries its terms,
|
||||
// the source stays MIT; see THIRD_PARTY_NOTICES.md and
|
||||
// docs/architecture/05-platform-backends.md.
|
||||
implementation("org.jellyfin.media3:media3-ffmpeg-decoder:1.5.0+1")
|
||||
implementation("com.google.guava:guava:33.0.0-android")
|
||||
|
||||
// Media library for VolumeProviderCompat (remote volume control)
|
||||
|
||||
@@ -3,8 +3,11 @@ package com.dtourolle.jellytau.player
|
||||
import android.content.Context
|
||||
import android.media.MediaCodecList
|
||||
import android.util.Log
|
||||
import androidx.annotation.OptIn
|
||||
import androidx.media3.common.AudioAttributes
|
||||
import androidx.media3.common.MimeTypes
|
||||
import androidx.media3.common.util.UnstableApi
|
||||
import androidx.media3.decoder.ffmpeg.FfmpegLibrary
|
||||
import androidx.media3.exoplayer.audio.AudioCapabilities
|
||||
|
||||
/**
|
||||
@@ -13,9 +16,24 @@ import androidx.media3.exoplayer.audio.AudioCapabilities
|
||||
* This class queries the device's media codec capabilities and reports
|
||||
* them to the Rust backend via JNI for accurate DeviceProfile generation.
|
||||
*/
|
||||
@OptIn(UnstableApi::class) // FfmpegLibrary and the licensed-codec MimeTypes
|
||||
object CodecDetector {
|
||||
private const val TAG = "CodecDetector"
|
||||
|
||||
/**
|
||||
* Formats the FFmpeg extension can decode, as Jellyfin codec names. The
|
||||
* platform decodes the rest itself; these are the licensed codecs a device
|
||||
* often lacks.
|
||||
*/
|
||||
private val FFMPEG_AUDIO_FORMATS = listOf(
|
||||
MimeTypes.AUDIO_AC3 to "ac3",
|
||||
MimeTypes.AUDIO_E_AC3 to "eac3",
|
||||
MimeTypes.AUDIO_E_AC3_JOC to "eac3",
|
||||
MimeTypes.AUDIO_DTS to "dts",
|
||||
MimeTypes.AUDIO_DTS_HD to "dts",
|
||||
MimeTypes.AUDIO_TRUEHD to "truehd",
|
||||
)
|
||||
|
||||
/**
|
||||
* Data class to hold detected codec capabilities.
|
||||
*/
|
||||
@@ -67,6 +85,25 @@ object CodecDetector {
|
||||
}
|
||||
}
|
||||
|
||||
// The FFmpeg extension decodes in software what the platform lacks.
|
||||
// ExoPlayer uses it for playback (JellyTauPlayer's renderers factory),
|
||||
// so it belongs in the same list: Rust judges both the streaming
|
||||
// profile and the download policy against this set, and a codec
|
||||
// missing here is re-encoded by the server for nothing. Asked per
|
||||
// format rather than assumed, so a build whose native library failed
|
||||
// to load reports only what the platform itself decodes.
|
||||
// TRACES: UR-004, UR-071 | DR-293
|
||||
if (FfmpegLibrary.isAvailable()) {
|
||||
for ((mime, codec) in FFMPEG_AUDIO_FORMATS) {
|
||||
if (FfmpegLibrary.supportsFormat(mime)) {
|
||||
audioCodecs.add(codec)
|
||||
Log.d(TAG, "Audio codec: $codec (MIME: $mime, FFmpeg extension)")
|
||||
}
|
||||
}
|
||||
} else {
|
||||
Log.w(TAG, "FFmpeg extension unavailable; reporting platform decoders only")
|
||||
}
|
||||
|
||||
Log.i(TAG, "Detected ${videoCodecs.size} video codecs: ${videoCodecs.sorted()}")
|
||||
Log.i(TAG, "Detected ${audioCodecs.size} audio codecs: ${audioCodecs.sorted()}")
|
||||
} catch (e: Exception) {
|
||||
@@ -148,7 +185,12 @@ object CodecDetector {
|
||||
"audio/eac3" -> "eac3"
|
||||
"audio/eac3-joc" -> "eac3"
|
||||
"audio/dts" -> "dts"
|
||||
// The platform's own spelling — what MediaCodecList reports on the
|
||||
// ROD2-W09. Only the `.hd` variant was listed, so plain DTS was
|
||||
// detected by luck, through the HD decoder advertising both.
|
||||
"audio/vnd.dts" -> "dts"
|
||||
"audio/vnd.dts.hd" -> "dts"
|
||||
"audio/true-hd" -> "truehd"
|
||||
"audio/x-ms-wma" -> "wma"
|
||||
"audio/amr-nb" -> "amrnb"
|
||||
"audio/amr-wb" -> "amrwb"
|
||||
|
||||
@@ -19,6 +19,7 @@ import androidx.media3.common.MediaMetadata
|
||||
import androidx.media3.common.PlaybackException
|
||||
import androidx.media3.common.Player
|
||||
import androidx.media3.common.util.UnstableApi
|
||||
import androidx.media3.exoplayer.DefaultRenderersFactory
|
||||
import androidx.media3.exoplayer.ExoPlayer
|
||||
import androidx.media3.exoplayer.source.DefaultMediaSourceFactory
|
||||
import androidx.media3.exoplayer.upstream.DefaultLoadErrorHandlingPolicy
|
||||
@@ -332,6 +333,18 @@ class JellyTauPlayer(private val appContext: Context) {
|
||||
//
|
||||
// TRACES: UR-004, UR-006 | IR-008
|
||||
exoPlayer = ExoPlayer.Builder(appContext)
|
||||
// Extension renderers ON: the device's own decoders are tried first
|
||||
// (a vendor DTS decoder stays in charge where there is one), and the
|
||||
// FFmpeg audio renderer takes any format they cannot decode — AC-3,
|
||||
// E-AC-3, TrueHD on a device without Dolby licensing. This is what
|
||||
// lets the untouched source file play, instead of a server transcode.
|
||||
// CodecDetector reports the same codecs to Rust, so the device
|
||||
// profile and the download policy agree with what actually decodes.
|
||||
// TRACES: UR-004, UR-071 | DR-293
|
||||
.setRenderersFactory(
|
||||
DefaultRenderersFactory(appContext)
|
||||
.setExtensionRendererMode(DefaultRenderersFactory.EXTENSION_RENDERER_MODE_ON)
|
||||
)
|
||||
// Decline the player's own load-error retry for a stream it could
|
||||
// only restart (DR-203). Every other source keeps the default
|
||||
// behaviour, which resumes the failed load where it stopped.
|
||||
|
||||
@@ -2187,6 +2187,25 @@ pub struct PlaybackCapabilities {
|
||||
/// beneath the WebView. Linux cannot do this (WebKitGTK/Wayland
|
||||
/// compositing), so it stays on the HTML5 element.
|
||||
pub supports_native_video: bool,
|
||||
/// True when the user may send video to the webview element instead of the
|
||||
/// native renderer — the frontend offers the switch only then, and honours
|
||||
/// the stored preference only then. See [`webview_video_fallback`].
|
||||
pub webview_video_fallback: bool,
|
||||
}
|
||||
|
||||
/// Whether the user may send video to the webview `<video>` element instead of
|
||||
/// the native renderer.
|
||||
///
|
||||
/// Never on Android: ExoPlayer is its only video renderer. Downloads there are
|
||||
/// the untouched source file (DR-293), and the webview decodes none of the
|
||||
/// AC-3/E-AC-3/DTS/TrueHD that ExoPlayer plays through the FFmpeg extension, so
|
||||
/// the fallback would be a silent film. Beside mpv's native video on Linux the
|
||||
/// webview is still the tested fallback; everywhere else it is the only
|
||||
/// renderer and there is nothing to switch.
|
||||
///
|
||||
/// TRACES: UR-003, UR-071 | DR-293 | UT-259
|
||||
pub fn webview_video_fallback(is_android: bool, native_video_enabled: bool) -> bool {
|
||||
!is_android && native_video_enabled
|
||||
}
|
||||
|
||||
/// Report this platform's playback capabilities to the frontend.
|
||||
@@ -2203,6 +2222,11 @@ pub async fn player_get_capabilities() -> Result<PlaybackCapabilities, String> {
|
||||
// TRACES: UR-080 | DR-235
|
||||
supports_native_video: cfg!(target_os = "android")
|
||||
|| crate::player::native_video::enabled(),
|
||||
// TRACES: UR-003, UR-071 | DR-293
|
||||
webview_video_fallback: webview_video_fallback(
|
||||
cfg!(target_os = "android"),
|
||||
crate::player::native_video::enabled(),
|
||||
),
|
||||
})
|
||||
}
|
||||
|
||||
@@ -3058,6 +3082,37 @@ pub async fn player_disable_jellyfin(player: State<'_, PlayerStateWrapper>) -> R
|
||||
mod tests {
|
||||
use crate::utils::lock::MutexSafe;
|
||||
|
||||
/// Android has one video renderer, ExoPlayer. The webview element could only
|
||||
/// be reached by the user switching native video off, and a file downloaded
|
||||
/// as the untouched original — AC-3 audio included — plays silent there,
|
||||
/// so the switch is gone on Android (DR-293). Where mpv draws video on Linux
|
||||
/// the webview is still the tested fallback, so the switch stays there;
|
||||
/// everywhere else the webview is the only renderer and there is nothing to
|
||||
/// switch.
|
||||
///
|
||||
/// TRACES: UR-003, UR-071 | DR-293 | UT-259
|
||||
#[test]
|
||||
fn test_webview_video_fallback_is_offered_only_beside_mpv_native_video() {
|
||||
use super::webview_video_fallback;
|
||||
|
||||
assert!(
|
||||
!webview_video_fallback(true, false),
|
||||
"Android: ExoPlayer is the only video renderer"
|
||||
);
|
||||
assert!(
|
||||
!webview_video_fallback(true, true),
|
||||
"Android never falls back, whatever else is switched on"
|
||||
);
|
||||
assert!(
|
||||
webview_video_fallback(false, true),
|
||||
"Linux with mpv native video: the webview is the fallback"
|
||||
);
|
||||
assert!(
|
||||
!webview_video_fallback(false, false),
|
||||
"the webview is the only renderer; nothing to fall back from"
|
||||
);
|
||||
}
|
||||
|
||||
/// UT-206 — the volume the command hands on is always a real number in
|
||||
/// 0.0..=1.0.
|
||||
///
|
||||
|
||||
@@ -195,17 +195,15 @@ pub fn subtitle_supports_external_delivery(codec: Option<&str>) -> bool {
|
||||
/// the raw list makes Jellyfin direct-play a track the webview cannot decode, and
|
||||
/// the user gets picture with no sound.
|
||||
///
|
||||
/// Which renderer gets it is not fixed: Linux is always the element, and Android
|
||||
/// follows `experimentalNativeVideo`, which took ExoPlayer as its default in
|
||||
/// DR-161 but is a user setting either way. So the *narrow* list is the only one
|
||||
/// that holds on both sides of that switch. The cost is a Dolby-licensed Android
|
||||
/// device transcoding an E-AC-3 track its ExoPlayer could have direct-played;
|
||||
/// the alternative is silence for everyone the switch lands the other way, which
|
||||
/// is the bug this exists to prevent.
|
||||
///
|
||||
/// The gap is widest on devices whose vendor licenses Dolby: a phone with
|
||||
/// `c2.dolby.eac3.decoder` reports `eac3`, so it — and only it — gets a silent
|
||||
/// direct play where a leaner device is transcoded to AAC and plays fine.
|
||||
/// Which renderer gets it depends on the platform. Linux draws video in the
|
||||
/// element (unless mpv native video is switched on), so it gets the narrow list.
|
||||
/// Android draws video only in ExoPlayer: it used to follow the
|
||||
/// `experimentalNativeVideo` setting, which could send video to the webview, and
|
||||
/// while that switch existed the narrow list was the only one true on both sides
|
||||
/// of it. DR-293 removed the webview video path on Android, so there the
|
||||
/// platform list is the whole answer — it includes the FFmpeg extension's
|
||||
/// AC-3/E-AC-3/DTS/TrueHD, which `CodecDetector` reports alongside the
|
||||
/// `MediaCodecList` decoders.
|
||||
///
|
||||
/// This applies to the *video* direct-play profile only. Audio-only playback
|
||||
/// really is ExoPlayer's, so its profile keeps the full platform list.
|
||||
@@ -323,6 +321,9 @@ pub fn renderer_can_decode_audio(codec: &str) -> bool {
|
||||
/// Whether the webview `<video>` element can decode this audio codec.
|
||||
///
|
||||
/// TRACES: UR-004 | DR-149 | UT-148
|
||||
// Unreachable on Android since DR-293: video renders only in ExoPlayer there,
|
||||
// so every caller goes through `renderer_can_decode_audio`'s device-list arm.
|
||||
#[cfg_attr(target_os = "android", allow(dead_code))]
|
||||
pub fn webview_can_decode_audio(codec: &str) -> bool {
|
||||
WEBVIEW_AUDIO_CODECS
|
||||
.iter()
|
||||
|
||||
@@ -2530,28 +2530,32 @@ impl MediaRepository for OnlineRepository {
|
||||
params.push("allowVideoStreamCopy=false".to_string());
|
||||
}
|
||||
// "original" (and any unknown value) → direct, resumable copy —
|
||||
// unless the audio in that copy is undecodable where the file will
|
||||
// be played back. A download is watched with no server in reach, so
|
||||
// it has to satisfy the same constraint DR-149 applies to streams:
|
||||
// the webview `<video>` element renders video on both platforms and
|
||||
// decodes none of AC-3/E-AC-3/DTS/TrueHD. Copying those bytes to
|
||||
// disk is what made a downloaded film play offline as picture with
|
||||
// no sound while the same film had sound when streamed.
|
||||
// unless the audio in that copy is undecodable by the renderer that
|
||||
// will play the file. A download is watched with no server in reach,
|
||||
// so there is nothing to fall back to: copying a track the renderer
|
||||
// cannot decode is what made a downloaded film play offline as
|
||||
// picture with no sound while the same film had sound when streamed
|
||||
// (DR-171).
|
||||
//
|
||||
// Only the *audio* is re-encoded. `allowVideoStreamCopy` keeps an
|
||||
// h264 source's picture byte-for-byte, so "original" still means
|
||||
// original quality, and no bitrate or resolution cap is added. A
|
||||
// source the webview could not have rendered anyway (HEVC) is
|
||||
// re-encoded to h264 as a side effect, which is the only form of it
|
||||
// that would have played.
|
||||
// "The renderer" is DR-234's per-platform answer, not the webview's
|
||||
// list. On Android that is ExoPlayer — the only video renderer there
|
||||
// since DR-293 removed the webview path — which decodes the device's
|
||||
// own codecs plus AC-3/E-AC-3/DTS/TrueHD through the FFmpeg
|
||||
// extension. So on Android every `original` download is a
|
||||
// `Static=true` copy: fast, resumable (HTTP 206), and the real file.
|
||||
// Judging against the webview's list instead turned most films into a
|
||||
// server transcode — generated as it is sent, no `Content-Length`,
|
||||
// `Range` ignored — measured at ~1 MB/s against 14.5 MB/s for the
|
||||
// copy, and restarting from zero on every network blip.
|
||||
//
|
||||
// The cost of the transcode is that the response is no longer
|
||||
// range-resumable, which is exactly why this is decided per item
|
||||
// rather than applied to every `original` download.
|
||||
// On Linux the webview still draws video, so the renderer's list *is*
|
||||
// the webview's and the transcode below still applies there. Only the
|
||||
// *audio* is re-encoded: `allowVideoStreamCopy` keeps an h264 source's
|
||||
// picture byte-for-byte, so "original" still means original quality.
|
||||
//
|
||||
// TRACES: UR-071, UR-004 | DR-171 | UT-166
|
||||
// TRACES: UR-071, UR-004 | DR-171, DR-293 | UT-166
|
||||
None => match source_audio_codec {
|
||||
Some(codec) if !super::device_profile::webview_can_decode_audio(codec) => {
|
||||
Some(codec) if !super::device_profile::renderer_can_decode_audio(codec) => {
|
||||
params.push("videoCodec=h264".to_string());
|
||||
params.push("allowVideoStreamCopy=true".to_string());
|
||||
params.push("audioCodec=aac".to_string());
|
||||
@@ -3731,13 +3735,17 @@ mod tests {
|
||||
/// holds audio this device cannot decode.
|
||||
///
|
||||
/// `Static=true` hands back the source bytes untouched, E-AC-3/AC-3/DTS
|
||||
/// track included, and video plays through the webview `<video>` element on
|
||||
/// both platforms — which decodes none of them. Streaming already knows this
|
||||
/// (DR-149 forces a transcode over the server's own direct-play offer); the
|
||||
/// download path did not, so a downloaded film played offline as picture with
|
||||
/// no sound while the very same film had sound when streamed.
|
||||
/// track included. Where the webview `<video>` element renders video —
|
||||
/// Linux, which is where this test runs — none of them decode. Streaming
|
||||
/// already knew this (DR-149); the download path did not, so a downloaded
|
||||
/// film played offline as picture with no sound.
|
||||
///
|
||||
/// TRACES: UR-071, UR-004 | DR-171 | UT-166
|
||||
/// On Android the renderer is ExoPlayer with the FFmpeg extension, which
|
||||
/// decodes all of these, so the same call there yields a `Static=true` copy
|
||||
/// (DR-293). The policy is `renderer_can_decode_audio`; this test pins its
|
||||
/// webview half.
|
||||
///
|
||||
/// TRACES: UR-071, UR-004 | DR-171, DR-293 | UT-166
|
||||
#[test]
|
||||
fn test_video_download_url_original_transcodes_undecodable_audio() {
|
||||
let repo = create_test_repository();
|
||||
|
||||
@@ -2886,7 +2886,13 @@ usesWebviewAudio: boolean;
|
||||
* beneath the WebView. Linux cannot do this (WebKitGTK/Wayland
|
||||
* compositing), so it stays on the HTML5 element.
|
||||
*/
|
||||
supportsNativeVideo: boolean }
|
||||
supportsNativeVideo: boolean;
|
||||
/**
|
||||
* True when the user may send video to the webview element instead of the
|
||||
* native renderer — the frontend offers the switch only then, and honours
|
||||
* the stored preference only then. See [`webview_video_fallback`].
|
||||
*/
|
||||
webviewVideoFallback: boolean }
|
||||
/**
|
||||
* Playback information
|
||||
*/
|
||||
|
||||
@@ -44,6 +44,18 @@ vi.mock("$lib/stores/nativeVideo", async (importOriginal) => {
|
||||
};
|
||||
});
|
||||
|
||||
// The webview path only exists where Rust offers a fallback from the native
|
||||
// renderer — beside mpv native video on Linux; never on Android since DR-293,
|
||||
// where a stored "off" is ignored. These tests guard that path's scrubbing, so
|
||||
// they declare a platform that has it.
|
||||
vi.mock("$lib/services/playbackCapabilities", () => ({
|
||||
getPlaybackCapabilities: async () => ({
|
||||
usesWebviewAudio: false,
|
||||
supportsNativeVideo: true,
|
||||
webviewVideoFallback: true,
|
||||
}),
|
||||
}));
|
||||
|
||||
vi.mock("@tauri-apps/api/event", () => ({
|
||||
listen: vi.fn(async (channel: string, handler: any) => {
|
||||
channelHandlers[channel] = handler;
|
||||
|
||||
@@ -41,7 +41,8 @@
|
||||
type Html5ElementBridge,
|
||||
} from "$lib/player/adapters";
|
||||
import { createRustReportHost } from "$lib/player/adapters/rustReportHost";
|
||||
import { experimentalNativeVideo } from "$lib/stores/nativeVideo";
|
||||
import { experimentalNativeVideo, nativeVideoWanted } from "$lib/stores/nativeVideo";
|
||||
import { getPlaybackCapabilities } from "$lib/services/playbackCapabilities";
|
||||
import {
|
||||
enableNativeVideoCompositing,
|
||||
disableNativeVideoCompositing,
|
||||
@@ -1020,7 +1021,12 @@
|
||||
// it is off we must also stop the native backend that player_play_item
|
||||
// just started, or ExoPlayer and the <video> element both decode the
|
||||
// same stream and the audio doubles.
|
||||
if (!useHtml5Element && !$experimentalNativeVideo) {
|
||||
// The stored choice only counts where Rust says a webview fallback
|
||||
// exists — never on Android, where the webview would play an
|
||||
// original-file download silent (DR-293).
|
||||
const { webviewVideoFallback } = await getPlaybackCapabilities();
|
||||
const wantNative = nativeVideoWanted($experimentalNativeVideo, webviewVideoFallback);
|
||||
if (!useHtml5Element && !wantNative) {
|
||||
log.debug("Native backend available but experimentalNativeVideo is off - using HTML5");
|
||||
useHtml5Element = true;
|
||||
try {
|
||||
@@ -1077,7 +1083,7 @@
|
||||
bridge: adapterBridge,
|
||||
// useHtml5Element is already the resolved decision above, so the
|
||||
// flag has had its say; pass it through for the invariant check.
|
||||
experimentalNativeVideo: $experimentalNativeVideo,
|
||||
experimentalNativeVideo: wantNative,
|
||||
});
|
||||
// No-op for the native adapter, which owns no DOM element.
|
||||
playerAdapter.attach(videoElement);
|
||||
|
||||
@@ -47,6 +47,18 @@ vi.mock("$lib/stores/nativeVideo", async (importOriginal) => {
|
||||
};
|
||||
});
|
||||
|
||||
// The webview path only exists where Rust offers a fallback from the native
|
||||
// renderer — beside mpv native video on Linux; never on Android since DR-293,
|
||||
// where a stored "off" is ignored. These tests guard that path's scrubbing, so
|
||||
// they declare a platform that has it.
|
||||
vi.mock("$lib/services/playbackCapabilities", () => ({
|
||||
getPlaybackCapabilities: async () => ({
|
||||
usesWebviewAudio: false,
|
||||
supportsNativeVideo: true,
|
||||
webviewVideoFallback: true,
|
||||
}),
|
||||
}));
|
||||
|
||||
vi.mock("@tauri-apps/api/event", () => ({
|
||||
listen: vi.fn(async (channel: string, handler: any) => {
|
||||
channelHandlers[channel] = handler;
|
||||
|
||||
@@ -22,6 +22,12 @@ export interface PlaybackCapabilities {
|
||||
usesWebviewAudio: boolean;
|
||||
/** Video can render on a native surface behind a transparent webview. */
|
||||
supportsNativeVideo: boolean;
|
||||
/**
|
||||
* The user may send video to the webview element instead of the native
|
||||
* renderer. False on Android, where ExoPlayer is the only video renderer
|
||||
* (DR-293). Rust decides; see `webview_video_fallback`.
|
||||
*/
|
||||
webviewVideoFallback: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -33,6 +39,7 @@ export interface PlaybackCapabilities {
|
||||
const FALLBACK: PlaybackCapabilities = {
|
||||
usesWebviewAudio: false,
|
||||
supportsNativeVideo: false,
|
||||
webviewVideoFallback: false,
|
||||
};
|
||||
|
||||
let cached: PlaybackCapabilities | null = null;
|
||||
@@ -52,6 +59,7 @@ export async function getPlaybackCapabilities(): Promise<PlaybackCapabilities> {
|
||||
cached = {
|
||||
usesWebviewAudio: !!caps?.usesWebviewAudio,
|
||||
supportsNativeVideo: !!caps?.supportsNativeVideo,
|
||||
webviewVideoFallback: !!caps?.webviewVideoFallback,
|
||||
};
|
||||
return cached;
|
||||
} catch (err) {
|
||||
|
||||
@@ -116,6 +116,23 @@ function createExperimentalNativeVideoStore() {
|
||||
*/
|
||||
export const experimentalNativeVideo = createExperimentalNativeVideoStore();
|
||||
|
||||
/**
|
||||
* Whether video should take the native path, given the user's stored choice
|
||||
* and whether this platform lets the user choose at all.
|
||||
*
|
||||
* On Android the answer is always native: ExoPlayer is the only video renderer
|
||||
* there, and the webview element decodes none of the AC-3/E-AC-3/DTS/TrueHD
|
||||
* that ExoPlayer plays through the FFmpeg extension — so a stored "off" would
|
||||
* turn every original-file download into a silent film (DR-293). Rust reports
|
||||
* whether a fallback exists (`webviewVideoFallback`); only then does the
|
||||
* stored choice count.
|
||||
*
|
||||
* TRACES: UR-003, UR-071 | DR-293 | UT-262
|
||||
*/
|
||||
export function nativeVideoWanted(storedChoice: boolean, webviewVideoFallback: boolean): boolean {
|
||||
return webviewVideoFallback ? storedChoice : true;
|
||||
}
|
||||
|
||||
function createNativeVideoActiveStore() {
|
||||
const { subscribe, set } = writable<boolean>(false);
|
||||
|
||||
|
||||
@@ -0,0 +1,17 @@
|
||||
import { describe, it, expect } from "vitest";
|
||||
import { nativeVideoWanted } from "./nativeVideo";
|
||||
|
||||
// TRACES: UR-003, UR-071 | DR-293 | UT-262
|
||||
describe("nativeVideoWanted", () => {
|
||||
it("ignores a stored 'off' where there is no webview fallback (Android)", () => {
|
||||
// Someone who once switched native video off on Android must not be
|
||||
// routed to the webview, which plays original-file downloads silent.
|
||||
expect(nativeVideoWanted(false, false)).toBe(true);
|
||||
expect(nativeVideoWanted(true, false)).toBe(true);
|
||||
});
|
||||
|
||||
it("honours the stored choice where a fallback exists (Linux beside mpv)", () => {
|
||||
expect(nativeVideoWanted(false, true)).toBe(false);
|
||||
expect(nativeVideoWanted(true, true)).toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -131,10 +131,12 @@
|
||||
{ label: "Unlimited", bytes: 0 },
|
||||
];
|
||||
|
||||
// Native-video opt-in (Android). `supportsNativeVideo` comes from Rust, which
|
||||
// owns the "does this platform have a native video surface" decision; the
|
||||
// toggle is hidden entirely where it cannot apply.
|
||||
let supportsNativeVideo = $state(false);
|
||||
// Native-video switch. Shown only where Rust reports a webview fallback —
|
||||
// beside mpv native video on Linux. Never on Android: ExoPlayer is the only
|
||||
// video renderer there, and the webview would play original-file downloads
|
||||
// silent (DR-293). Rust owns the decision; the toggle is hidden where it
|
||||
// cannot apply.
|
||||
let offerNativeVideoSwitch = $state(false);
|
||||
let nativeVideoEnabled = $state(false);
|
||||
|
||||
const unsubscribeNativeVideo = experimentalNativeVideo.subscribe((v) => {
|
||||
@@ -156,7 +158,7 @@
|
||||
onMount(async () => {
|
||||
await loadSettings();
|
||||
askOnStart = await commands.profilesGetAskOnStart();
|
||||
supportsNativeVideo = (await getPlaybackCapabilities()).supportsNativeVideo;
|
||||
offerNativeVideoSwitch = (await getPlaybackCapabilities()).webviewVideoFallback;
|
||||
|
||||
// Which update story this platform gets. Android cannot install its own
|
||||
// APK, so it is offered the releases page instead of an install button.
|
||||
@@ -952,9 +954,9 @@
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<!-- Native video (experimental). Only rendered where the platform's Rust
|
||||
backend actually has a native video surface (Android). -->
|
||||
{#if supportsNativeVideo}
|
||||
<!-- Native video. Only rendered where Rust reports a webview fallback
|
||||
(Linux beside mpv native video); never on Android (DR-293). -->
|
||||
{#if offerNativeVideoSwitch}
|
||||
<div class="bg-[var(--color-surface)] rounded-lg p-6 mt-4">
|
||||
<div class="flex items-center justify-between">
|
||||
<div class="pr-4">
|
||||
|
||||
Reference in New Issue
Block a user