Compare commits

..
5 Commits
Author SHA1 Message Date
dtourolle 07d10dfed7 docs(traceability): land the DR-149 requirement rows and settle a UT collision
🏗️ Build and Test JellyTau / Run Tests (push) Successful in 19m31s
Publish Documentation / Build & publish docs to gitea-pages (push) Successful in 5m28s
Traceability Validation / Check Requirement Traces (push) Successful in 22s
Build & Release / Run Tests (push) Successful in 6m47s
🏗️ Build and Test JellyTau / Android Compile Check (push) Successful in 10m6s
Build & Release / Build Linux (push) Successful in 19m57s
Build & Release / Build Windows (push) Successful in 14m14s
Build & Release / Build Android (push) Successful in 30m26s
Build & Release / Create Release (push) Successful in 19s
The DR-149 row lost an index race with a parallel session's edit of the same
file, so the previous commit carried the count assertion (DR 144, total 282)
without the requirement it counts — a clean checkout of that commit failed
`bun run test` against its own requirements.md.

The parallel session also reached UT-143 and UT-147 for subtitle work, which
collided with the UT-143 used for the client-side transcode tests. Those move
to UT-148, in the table and in the device_profile TRACES comments, so no two
requirements share an ID.
2026-08-11 20:11:22 +02:00
dtourolle acddcdd6fa fix(playback): force a transcode when the webview cannot decode the audio (DR-149, 0.4.8)
Advertising a webview-shaped profile (DR-148) was necessary but not
sufficient. Probing the server directly showed Jellyfin 10.11.5 enforces a
DirectPlayProfile's Container and VideoCodec — excluding either returns
SupportsDirectPlay:false with TranscodeReasons=ContainerNotSupported /
VideoCodecNotSupported — but ignores its AudioCodec entirely: an E-AC-3
track is still offered for direct play against a profile listing only
aac,flac,mp3,opus,vorbis. Neither a VideoAudio CodecProfile forbidding the
codec nor MaxAudioChannels:2 against a 6-channel track changes the answer,
so no profile the client can send fixes this and the picture plays silent.

The client therefore stops delegating a question it can answer itself. The
negotiated source's audio is checked against what the webview decodes, and
an undecodable track forces the existing h264/aac HLS transcode regardless
of the server calling direct play fine; direct_play and needs_transcoding
are corrected to match so the frontend and the reporting path agree with
the URL actually used. The track judged is the one that would be served —
the default, else the first — since a supported track further down is not
the one that plays. A source with no audio, or a codec the server did not
name, is left alone rather than transcoded on a guess.

Test-first: the new tests failed against the old behaviour before the
decision existed. Verified on a motorola edge 30 by the audio HAL, not by
ear — the same E-AC-3 episode logged isMusicActive=true once and 58
ACDB-LOADER lines under this build, against 0 and 0 on 0.4.6, where an AAC
file in the same session produced 16 and 116. No FATAL EXCEPTION, so R8 on
the signed release build is unaffected.

Also carries in-flight subtitle-track work authored in a parallel session
(subtitleTracks, VideoPlayer, player/media, bindings) at the user's
request, so the tag matches the APK verified on device.
2026-08-11 20:07:11 +02:00
dtourolle 6a712c46cb fix(player): send subtitle tracks to ExoPlayer on Android (UR-020)
Selecting a subtitle on Android did nothing. The Kotlin side has been
complete for a long time — JellyTauPlayer.load() parses a subtitles JSON
array into MediaItem.SubtitleConfigurations and setSubtitleTrack() drives a
TrackSelectionOverride — but nothing ever reached it.

VideoPlayer built the list and then threw it away: it resolved every
subtitle stream's URL into a subtitleTracks array and the
commands.playerPlayItem({...}) call two lines below passed only streamUrl,
title, id, videoCodec and needsTranscoding. PlayItemRequest had no subtitle
field to put them in, so create_media_item hardcoded subtitles: vec![],
android/mod.rs serialized "[]" across JNI, and every MediaItem reached
ExoPlayer with zero SubtitleConfigurations. A later set_subtitle_track then
found no text track groups and logged "Invalid subtitle track index".

PlayItemRequest now carries the tracks (defaulted, so the background-audio
handoff and next-episode callers are unchanged) and create_media_item
threads them onto the MediaItem.

Serialization: SubtitleTrack is reused verbatim rather than given an
IPC-specific twin, and deliberately keeps snake_case. The same struct feeds
two consumers that both spell mime_type — the JNI JSON that
JellyTauPlayer.load() reads with optString("mime_type"), and the generated
binding the frontend types against. camelCasing it would not fail the build
or the IPC; Kotlin would silently fall back to its default MIME type for
every track. UT-146 asserts the exact serialized keys so a future
rename_all cannot pass unnoticed.

The index mapping was NOT already correct. setSubtitleTrack(n) indexes
ExoPlayer's filtered text track groups, i.e. the position of the sideloaded
configuration — but the menu passed its own {#each} row number, which counts
every subtitle *stream*, including ones whose URL failed to resolve and were
therefore never sideloaded. One failed URL and every track below it selected
the wrong subtitle. The position is now looked up in the exact array that
was sent (nativeSubtitleArrayIndex), and a stream that was never sent maps
to "off" rather than to a guessed position.

The resolution loop also reuses resolveSubtitleTracks() from the Linux fix
instead of duplicating it, which fans the URL requests out in parallel
rather than awaiting them one per stream before playback can start. The
awaits are safe where they sit: the native-mode pitfall is about Svelte
lifecycle calls after an await, and nothing is registered here — the
background-audio subscriptions above still run synchronously.

No Kotlin change was needed.

Tests (UT-145, UT-146, UT-147) were written first and failed: PlayItemRequest
had no subtitles field to compile against, nativeSubtitleTracks and
nativeSubtitleArrayIndex did not exist, and the playerPlayItem call carried
no subtitles key.

TRACES: UR-020 | IR-016, JA-008 | UT-145, UT-146, UT-147
2026-08-11 20:03:19 +02:00
dtourolle 211792947d fix(player): render subtitle tracks on the Linux HTML5 path (UR-020)
Selecting a subtitle on Linux did nothing. VideoPlayer rendered no <track>
children at all — the block was commented out as "temporarily disabled to
debug playback issues" (it has been that way since the POC) — so
Html5PlayerAdapter.selectSubtitle() walked an empty textTracks list and the
menu, which is built from media.mediaStreams, was purely decorative.

The reason it had to be disabled is still visible in the dead markup:
getSubtitleUrl() is async, so src={getSubtitleUrl(track.index)} bound a
Promise to the attribute and every track pointed at "[object Promise]" — an
unloadable resource hanging off the media element.

Subtitle URLs are now resolved off the render path into component state
(subtitleTracks.ts), and only streams whose URL actually resolved are
rendered; a per-track failure drops that track instead of emitting a dead
src. data-stream-index is kept, since that is what the adapter matches on.

Subtitles stay OFF unless the user asks for them: the server's isDefault flag
is shown in the menu but is never promoted to a selection, and the `default`
attribute is deliberately not emitted. A <track default> auto-shows, so the
menu would open on "Off" while subtitles were burned over the picture, and
every user who never wanted subtitles would suddenly get them. That matches
the existing initial state (selectedSubtitleIndex = null).

Selection and rendered tracks are reconciled whenever the list changes: a
selection that no longer resolves collapses to "Off", and a surviving one is
re-applied after the new <track> elements exist. "Off" disables every text
track, as before.

Cross-origin text-track fetches use the media element's CORS setting, so the
element opts in with crossorigin="anonymous" — but only for an http(s)
stream, never for a local/offline file:/asset: source, where forcing CORS
onto the video fetch could break playback. It is keyed on the subtitle stream
count, known at first render, so the attribute cannot flip under an in-flight
media load.

Android/native is untouched: the ExoPlayer branch still goes through
player_set_subtitle_track.

Tests (UT-143, UT-144) were written first and failed against the old markup:
the commented-out block, the Promise bound to src, and the default attribute.

TRACES: UR-020 | DR-023 | UT-143, UT-144
2026-08-11 19:25:39 +02:00
dtourolle 2c3955914e fix(playback): advertise only webview-decodable audio for video (DR-148, 0.4.7)
The audio codec list sent to Jellyfin comes from MediaCodecList, which
describes ExoPlayer — but video does not play through ExoPlayer. Android
force-renders every video in the webview <video> element (the interim
override in VideoPlayer.svelte) and Linux always has, and Chromium/WebKit
decode a far narrower set than the platform does.

A motorola edge 30 ships /vendor/etc/media_codecs_dolby_audio.xml, so it
reported ac3,eac3; the server direct-played an E-AC-3 track with
static=true and the webview built a video decoder and no audio decoder at
all — full picture, no sound. The defect is triggered by capability rather
than the lack of it, which is why a Fairphone and an Honor tablet play the
same file on the same build: without the Dolby decoder they never claim the
codec, so the server transcodes to AAC. Confirmed by A/B on the failing
device — hevc+eac3 silent, hevc+aac audible, same session, same profile,
same direct-play path, audio codec the only variable.

video_audio_codecs narrows the platform list to the webview-decodable set
for the video direct-play profile only. Audio-only playback really is the
native player's, so that profile keeps the full list rather than
transcoding music that plays perfectly well. A list with nothing decodable
still claims aac, since a profile claiming nothing invites the server to
give up instead of transcoding. The video codec list is deliberately
untouched: HEVC direct-plays through the webview correctly, so the
constraint is specific to audio.

Test-first: the tests failed against the old behaviour before the filter
existed, including the case built from the phone's real codec list. The
requirement-count assertion in extract-traces.test.ts moves 280 -> 281 for
the added DR, which is the deliberate edit that test exists to force.

Not yet verified on device — the 0.4.7 APK was still building.
2026-08-11 19:13:46 +02:00
17 changed files with 2402 additions and 1022 deletions
+30
View File
@@ -6,6 +6,36 @@ Entries are grouped by the capability they change, not by commit. Requirement
IDs in parentheses point at [docs/requirements.md](docs/requirements.md); the IDs in parentheses point at [docs/requirements.md](docs/requirements.md); the
generated trace matrix lives in [docs/traceability.md](docs/traceability.md). generated trace matrix lives in [docs/traceability.md](docs/traceability.md).
## v0.4.8
### 🐛 Fixes
- **Video with an undecodable soundtrack now transcodes instead of playing
silent.** Advertising a webview-shaped profile (v0.4.7) turned out not to be
enough: Jellyfin 10.11.5 enforces a direct-play profile's container and video
codec but ignores its audio codec, offering an E-AC-3 track for direct play
against a profile listing only AAC — and no `CodecProfile` or channel limit
changes that. The client now checks the track it would actually be served
against what its renderer can decode and forces the h264/AAC HLS transcode
when it cannot, rather than trusting the negotiation.
(UR-004 → DR-149)
## v0.4.7
### 🐛 Fixes
- **Video plays with sound on devices that ship a Dolby decoder.** The audio
codec list sent to Jellyfin came from `MediaCodecList`, which describes
ExoPlayer — but video does not play through ExoPlayer: it renders in the
webview `<video>` element, which decodes far less. A phone whose vendor
licenses Dolby therefore advertised `ac3`/`eac3`, got a direct play, and
showed full picture with no audio, while a leaner device claimed neither
codec, received an AAC transcode, and played the same file correctly. The
video direct-play profile is now narrowed to what the webview can decode;
audio-only playback is genuinely the native player's and keeps the full list,
so music is not transcoded needlessly.
(UR-004 → DR-148)
## v0.4.1 ## v0.4.1
### 🐛 Fixes ### 🐛 Fixes
+9
View File
@@ -310,6 +310,8 @@ Internal architecture, components, and application logic.
| DR-141 | The device profile states how many channels the audio route can actually voice. `MediaCodecList` answers "can this device *decode* 5.1", which is not the question that decides whether the user hears anything — a phone decodes an AC-3 5.1 track happily and still has two channels to play it out of. With no `MaxAudioChannels` in the profile, Jellyfin was free to direct-play the multichannel track, and the result is device dependent: a failed `AudioSink` configuration (silence) or dialogue folded into surround channels that go nowhere. media3's `AudioCapabilities.maxChannelCount` for the current route is reported over JNI alongside the codec lists, and bounds both the direct-play profile and the transcoding profiles, so the server downmixes rather than shipping channels the sink cannot take. Codecs are never removed from the profile — a device with genuine surround output keeps direct-playing it. A missing or zero reading means "route not yet established", not "no audio", and falls back to stereo, the one capability every sink has | Playback | UR-004 | Done | | DR-141 | The device profile states how many channels the audio route can actually voice. `MediaCodecList` answers "can this device *decode* 5.1", which is not the question that decides whether the user hears anything — a phone decodes an AC-3 5.1 track happily and still has two channels to play it out of. With no `MaxAudioChannels` in the profile, Jellyfin was free to direct-play the multichannel track, and the result is device dependent: a failed `AudioSink` configuration (silence) or dialogue folded into surround channels that go nowhere. media3's `AudioCapabilities.maxChannelCount` for the current route is reported over JNI alongside the codec lists, and bounds both the direct-play profile and the transcoding profiles, so the server downmixes rather than shipping channels the sink cannot take. Codecs are never removed from the profile — a device with genuine surround output keeps direct-playing it. A missing or zero reading means "route not yet established", not "no audio", and falls back to stereo, the one capability every sink has | Playback | UR-004 | Done |
| DR-145 | Video playback starts only once the app actually holds audio focus. Video manages focus by hand (`handleAudioFocus=false`, because ExoPlayer's automatic handling is reserved for the audio path), and the request's three outcomes were all treated as success: `AUDIOFOCUS_REQUEST_DELAYED` — which `setAcceptsDelayedFocusGain(true)` explicitly invites, and which means the system is *withholding our audio* until it calls back — and an outright `REQUEST_FAILED` were logged and then followed by `playWhenReady = true`. The picture rolled with no sound, indistinguishable to the user from a broken stream. Playback is now held when focus is not granted and started from the `AUDIOFOCUS_GAIN` callback; an explicit `play()` re-requests focus rather than resuming into a stream the system is still muting, guarded by a held-focus flag so repeated plays do not leak focus requests. A `LOSS` clears the pending flag, so an unrelated later `GAIN` cannot start playback the user never asked for | Playback | UR-004 | Done | | DR-145 | Video playback starts only once the app actually holds audio focus. Video manages focus by hand (`handleAudioFocus=false`, because ExoPlayer's automatic handling is reserved for the audio path), and the request's three outcomes were all treated as success: `AUDIOFOCUS_REQUEST_DELAYED` — which `setAcceptsDelayedFocusGain(true)` explicitly invites, and which means the system is *withholding our audio* until it calls back — and an outright `REQUEST_FAILED` were logged and then followed by `playWhenReady = true`. The picture rolled with no sound, indistinguishable to the user from a broken stream. Playback is now held when focus is not granted and started from the `AUDIOFOCUS_GAIN` callback; an explicit `play()` re-requests focus rather than resuming into a stream the system is still muting, guarded by a held-focus flag so repeated plays do not leak focus requests. A `LOSS` clears the pending flag, so an unrelated later `GAIN` cannot start playback the user never asked for | Playback | UR-004 | Done |
| DR-146 | The no-audio-track fallback picks a track the renderer can actually play. When ExoPlayer selected no audio track, the recovery forced group 0 / track 0 unconditionally — but the most likely reason nothing was selected is that this very track cannot be decoded on this device, so the override reinstated the silence it was meant to fix. It now scans the groups for the first `isTrackSupported` track and overrides to that, and clears `setTrackTypeDisabled(TRACK_TYPE_AUDIO)` because audio may equally have been off at the type level, which an override alone does not undo. When no group holds a supported track the condition is logged as an error — the server was expected to transcode — rather than leaving a silent video with no explanation in the log | Playback | UR-004 | Done | | DR-146 | The no-audio-track fallback picks a track the renderer can actually play. When ExoPlayer selected no audio track, the recovery forced group 0 / track 0 unconditionally — but the most likely reason nothing was selected is that this very track cannot be decoded on this device, so the override reinstated the silence it was meant to fix. It now scans the groups for the first `isTrackSupported` track and overrides to that, and clears `setTrackTypeDisabled(TRACK_TYPE_AUDIO)` because audio may equally have been off at the type level, which an override alone does not undo. When no group holds a supported track the condition is logged as an error — the server was expected to transcode — rather than leaving a silent video with no explanation in the log | Playback | UR-004 | Done |
| DR-148 | The video direct-play profile advertises only what the **webview** can decode. The audio codec list comes from `MediaCodecList`, which describes ExoPlayer — but video does not play through ExoPlayer on either platform: Android force-renders every video in the webview `<video>` element (the interim override in `VideoPlayer.svelte`, because the native SurfaceView sits behind an opaque webview) and Linux always has. Chromium and WebKit decode a far narrower set than the platform does, and the gap is widest on devices whose vendor licenses Dolby: a phone shipping `/vendor/etc/media_codecs_dolby_audio.xml` reports `ac3,eac3`, so Jellyfin direct-played an E-AC-3 track with `static=true` and the webview built a video decoder and no audio decoder at all — full picture, no sound. The defect is triggered by *capability*, not the lack of it, which is why it reproduced on one Motorola while a Fairphone and an Honor tablet played the same file on the same build: a device without the Dolby decoder never claims the codec, so the server transcodes to AAC and it plays. `video_audio_codecs` narrows the platform list to the webview-decodable set (`aac,mp3,opus,vorbis,flac`) for the video direct-play profile *only* — the audio-only profile keeps the full list, since that playback really is the native player's and narrowing it would transcode music that plays perfectly well. A list with nothing decodable still claims `aac` rather than going out empty, because a profile that claims nothing invites the server to give up instead of transcoding. The video codec list is deliberately untouched: HEVC direct-plays through the webview correctly, so the constraint is specific to audio | Playback | UR-004 | Done |
| DR-149 | The client decides whether its own renderer can decode the audio, rather than trusting the server's negotiation. Advertising a webview-shaped profile (DR-148) turned out to be necessary but not sufficient: Jellyfin 10.11.5 enforces a `DirectPlayProfile`'s `Container` and `VideoCodec` — excluding either returns `SupportsDirectPlay: false` with `TranscodeReasons=ContainerNotSupported` / `VideoCodecNotSupported` — but **ignores its `AudioCodec`**, offering an E-AC-3 track for direct play against a profile listing only `aac,flac,mp3,opus,vorbis`. Neither a `VideoAudio` `CodecProfile` forbidding the codec nor a `MaxAudioChannels: 2` against a 6-channel track changes the answer, so no profile the client can send fixes it and the picture plays silent. The negotiated source's audio is therefore checked locally against what the webview decodes, and an undecodable track forces the existing h264/aac HLS transcode URL regardless of the server saying direct play is fine — `direct_play` and `needs_transcoding` are corrected to match, so the frontend and the reporting path agree with the URL actually used. The track judged is the one the server would serve: the default, or the first when nothing is marked default, since a supported track further down the list is not the one that plays. A source with no audio streams, or a stream whose codec the server did not name, is left alone — forcing a transcode on a guess spends server CPU on files that already play | Playback | UR-004 | Done |
| DR-143 | Flipping the offline downloaded-only gate actually re-queries the listing. The gate (DR-078) is a process-wide flag in Rust consulted only *while a query runs*, but no library surface re-queried when its inputs changed: `useServerReachabilityReload` fires only on the offline → **online** transition, and `GenericMediaListPage`, `GenericGenreBrowser` and the favourites page never even called its `checkServerReachability`. So going offline left the full server catalog on screen under a now-closed gate, and toggling "Show all server media" only greyed cards — `MediaCard.isServerOnly` is a pure frontend derivation that updates instantly — without adding or removing a single row. The filter therefore read as "shows everything until I filter, then greys some of it" while the backend gate was correct and simply never exercised. `catalogFilterVersion` is the refetch signal: `pushCatalogVisibility` now awaits `set_show_server_catalog` and bumps the version only **after** the backend accepts the new flag, since a reload racing the push would re-query under the old gate and undo itself. A failed push clears `lastIncludeCatalog` instead of latching it, so the next identical transition is retried rather than skipped as a no-op and left permanently disagreeing with the backend. `useOfflineFilterReload` subscribes pages to that signal, skipping the value they already loaded under; it is wired into both generic list components and the movies/music/tv/favourites landing pages and the `/library/[id]` detail page | UI | UR-052 | Done | | DR-143 | Flipping the offline downloaded-only gate actually re-queries the listing. The gate (DR-078) is a process-wide flag in Rust consulted only *while a query runs*, but no library surface re-queried when its inputs changed: `useServerReachabilityReload` fires only on the offline → **online** transition, and `GenericMediaListPage`, `GenericGenreBrowser` and the favourites page never even called its `checkServerReachability`. So going offline left the full server catalog on screen under a now-closed gate, and toggling "Show all server media" only greyed cards — `MediaCard.isServerOnly` is a pure frontend derivation that updates instantly — without adding or removing a single row. The filter therefore read as "shows everything until I filter, then greys some of it" while the backend gate was correct and simply never exercised. `catalogFilterVersion` is the refetch signal: `pushCatalogVisibility` now awaits `set_show_server_catalog` and bumps the version only **after** the backend accepts the new flag, since a reload racing the push would re-query under the old gate and undo itself. A failed push clears `lastIncludeCatalog` instead of latching it, so the next identical transition is retried rather than skipped as a no-op and left permanently disagreeing with the backend. `useOfflineFilterReload` subscribes pages to that signal, skipping the value they already loaded under; it is wired into both generic list components and the movies/music/tv/favourites landing pages and the `/library/[id]` detail page | UI | UR-052 | Done |
| DR-135 | A download's media type comes from the item, not a default. `download_item` — the path a media card uses to queue an item while offline — never records `media_type`, and the reconnect resolver read that NULL as `'audio'`, so a **movie** queued from a card had its URL resolved by `get_audio_stream_url`. The file that landed on disk was an audio-only transcode, which is why a "downloaded" film could never play offline no matter how the path or protocol was fixed. The resolver now falls back to the item's own `item_type` (`VIDEO_ITEM_TYPES` in Rust, so the frontend never learns which types are video) and only defaults to audio when the item is not cached locally. An explicit `media_type` on the row still wins | Downloads | UR-071, UR-052 | Done | | DR-135 | A download's media type comes from the item, not a default. `download_item` — the path a media card uses to queue an item while offline — never records `media_type`, and the reconnect resolver read that NULL as `'audio'`, so a **movie** queued from a card had its URL resolved by `get_audio_stream_url`. The file that landed on disk was an audio-only transcode, which is why a "downloaded" film could never play offline no matter how the path or protocol was fixed. The resolver now falls back to the item's own `item_type` (`VIDEO_ITEM_TYPES` in Rust, so the frontend never learns which types are video) and only defaults to audio when the item is not cached locally. An explicit `media_type` on the row still wins | Downloads | UR-071, UR-052 | Done |
| DR-136 | Rows already downloaded under the audio default are repaired, not just prevented. They are identifiable after the fact — no `media_type`, but a video item — so on reconnect they are reset to `pending` with their audio URL cleared and re-resolved by DR-135's corrected logic, overwriting the audio file in place. Without this the fix is invisible to anyone who had already queued a film: the row still reads "downloaded" and still fails to play. Rows carrying an explicit `media_type` and genuine audio downloads are left untouched | Downloads | UR-071 | Done | | DR-136 | Rows already downloaded under the audio default are repaired, not just prevented. They are identifiable after the fact — no `media_type`, but a video item — so on reconnect they are reset to `pending` with their audio URL cleared and re-resolved by DR-135's corrected logic, overwriting the audio file in place. Without this the fix is invisible to anyone who had already queued a film: the row still reads "downloaded" and still fails to play. Rows carrying an explicit `media_type` and genuine audio downloads are left untouched | Downloads | UR-071 | Done |
@@ -540,6 +542,13 @@ Internal architecture, components, and application logic.
| UT-139 | A failed visibility push is retried on the next identical transition rather than latched | DR-143 | Done | | UT-139 | A failed visibility push is retried on the next identical transition rather than latched | DR-143 | Done |
| UT-140 | `useOfflineFilterReload` skips the value a page already loaded under and reloads on each later change | DR-143 | Done | | UT-140 | `useOfflineFilterReload` skips the value a page already loaded under and reloads on each later change | DR-143 | Done |
| UT-141 | The advertised channel cap: an unknown or zero reading falls back to stereo, a real route keeps its channels, an absurd driver reading is capped at 7.1, and mono is taken at its word | DR-141 | Done | | UT-141 | The advertised channel cap: an unknown or zero reading falls back to stereo, a real route keeps its channels, an absurd driver reading is capped at 7.1, and mono is taken at its word | DR-141 | Done |
| UT-148 | Forcing a transcode from the client: an undecodable default track forces one, a decodable track does not, the default track decides rather than the first, the first decides when nothing is marked default, and neither an audio-less source nor an unnamed codec is second-guessed | DR-149 | Done |
| UT-142 | The audio codecs offered for video direct play: a Dolby device's real `MediaCodecList` output drops `ac3`/`eac3`, AMR and raw PCM are dropped too, a fully-supported list is passed through untouched, a list with nothing decodable still claims `aac`, and stray spacing or casing does not decide whether the user gets sound | DR-148 | Done |
| UT-143 | Subtitle URLs resolve to plain strings before they reach the markup (never a Promise), unresolvable tracks are dropped, a stale selection collapses to "Off", and a server-default track is never auto-selected | UR-020, DR-023 | Done |
| UT-144 | VideoPlayer actually renders `<track kind="subtitles">` children carrying `data-stream-index`, with no `default` attribute and no async `getSubtitleUrl()` bound to `src` | UR-020, DR-023 | Done |
| UT-145 | The frontend's subtitle payload survives the IPC hop: a camelCase `PlayItemRequest` carrying `subtitles` deserializes, `create_media_item` lands them on `MediaItem.subtitles` in the order sent, and a request without the field still defaults to empty | UR-020, IR-016 | Done |
| UT-146 | The subtitle JSON serialized across the JNI boundary uses the keys `JellyTauPlayer.load()` reads — `url`, `language`, `label` and `mime_type`, never `mimeType` | UR-020, IR-016, JA-008 | Done |
| UT-147 | The native subtitle payload and the track-selection index come from the same resolved list: the wire shape keeps `mime_type` and stream order, `playerPlayItem` actually sends it, and the index is a position in the sent list (so a track whose URL failed to resolve cannot shift the others) rather than the menu's row number | UR-020, IR-016 | Done |
### Integration Tests ### Integration Tests
+1234 -942
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -1,6 +1,6 @@
{ {
"name": "jellytau", "name": "jellytau",
"version": "0.4.6", "version": "0.4.8",
"description": "", "description": "",
"type": "module", "type": "module",
"packageManager": "bun@1.3.5", "packageManager": "bun@1.3.5",
+2 -2
View File
@@ -175,8 +175,8 @@ describe("live requirements.md", () => {
expect(defined.UR).toBe(71); expect(defined.UR).toBe(71);
expect(defined.IR).toBe(32); expect(defined.IR).toBe(32);
expect(defined.DR).toBe(142); expect(defined.DR).toBe(144);
expect(defined.JA).toBe(35); expect(defined.JA).toBe(35);
expect(defined.total).toBe(280); expect(defined.total).toBe(282);
}); });
}); });
+1 -1
View File
@@ -2018,7 +2018,7 @@ dependencies = [
[[package]] [[package]]
name = "jellytau" name = "jellytau"
version = "0.4.6" version = "0.4.8"
dependencies = [ dependencies = [
"aes-gcm", "aes-gcm",
"async-trait", "async-trait",
+1 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "jellytau" name = "jellytau"
version = "0.4.6" version = "0.4.8"
description = "A Tauri App" description = "A Tauri App"
authors = ["you"] authors = ["you"]
edition = "2021" edition = "2021"
+160 -1
View File
@@ -202,6 +202,27 @@ pub struct PlayItemRequest {
/// look up the next episode when a background-audio track ends. /// look up the next episode when a background-audio track ends.
#[serde(default)] #[serde(default)]
pub series_id: Option<String>, pub series_id: Option<String>,
/// Subtitle tracks to sideload, with URLs the frontend has already resolved.
///
/// Only the native backends use these: on Android they become the
/// `MediaItem.SubtitleConfiguration`s ExoPlayer renders. The HTML5 path
/// builds its own `<track>` children instead and ignores this list.
///
/// **Order is the contract.** `player_set_subtitle_track(n)` reaches
/// `JellyTauPlayer.setSubtitleTrack(n)`, which indexes into ExoPlayer's
/// *text track groups* — i.e. the position of the sideloaded configuration,
/// not the Jellyfin stream index (which is kept on each entry for the UI's
/// benefit). So `n` must be a position in this very array, and the array
/// must not be reordered or filtered between building it and sending it.
/// `nativeSubtitleArrayIndex()` on the frontend computes `n` from the same
/// list that is sent here, for exactly this reason.
///
/// Defaulted so the background-audio handoff and the autoplay/next-episode
/// callers, which have no subtitles to offer, need not send the field.
///
/// TRACES: UR-020 | IR-016, JA-008 | UT-145
#[serde(default)]
pub subtitles: Vec<crate::player::SubtitleTrack>,
} }
/// Queue context for remote transfer - what type of queue is this? /// Queue context for remote transfer - what type of queue is this?
@@ -373,7 +394,10 @@ pub(super) async fn create_media_item(
needs_transcoding: req.needs_transcoding, needs_transcoding: req.needs_transcoding,
video_width: None, // Not available from video-only request video_width: None, // Not available from video-only request
video_height: None, // Not available from video-only request video_height: None, // Not available from video-only request
subtitles: vec![], // Sideloaded subtitles, in the order the frontend sent them — that order
// is what `player_set_subtitle_track(n)` indexes into on Android.
// TRACES: UR-020 | IR-016 | UT-145
subtitles: req.subtitles,
series_id: None, // Not available from video-only request series_id: None, // Not available from video-only request
server_id: None, // Not available from video-only request server_id: None, // Not available from video-only request
}) })
@@ -2459,6 +2483,141 @@ pub async fn player_disable_jellyfin(player: State<'_, PlayerStateWrapper>) -> R
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
/// The subtitle list the frontend resolved must survive the IPC hop and end
/// up on the `MediaItem` the native backend loads.
///
/// The bug: `VideoPlayer.svelte` built a fully-resolved subtitle array and
/// then dropped it on the floor — `PlayItemRequest` had no field to put it
/// in — so `create_media_item` always produced `subtitles: vec![]`,
/// `android/mod.rs` serialized `[]` across JNI, and ExoPlayer was handed a
/// `MediaItem` with zero `SubtitleConfiguration`s. Every later
/// `setSubtitleTrack(n)` then found no text track groups and logged
/// "Invalid subtitle track index".
///
/// The payload below is exactly what the frontend sends: camelCase for the
/// top-level command params (Tauri v2 converts them), and the subtitle
/// entries in the casing of `SubtitleTrack` itself — note `mime_type`.
///
/// TRACES: UR-020 | IR-016 | UT-145
#[tokio::test]
async fn test_play_item_request_carries_subtitles_into_media_item() {
use super::{create_media_item, PlayItemRequest};
let payload = serde_json::json!({
"id": "ep-1",
"title": "Pilot",
"streamUrl": "https://jelly.example/Videos/ep-1/master.m3u8",
"videoCodec": "h264",
"needsTranscoding": false,
"subtitles": [
{
"index": 2,
"url": "https://jelly.example/Videos/ep-1/2/Subtitles/subtitles.vtt",
"language": "eng",
"label": "English (SRT)",
"mime_type": "text/vtt"
},
{
"index": 3,
"url": "https://jelly.example/Videos/ep-1/3/Subtitles/subtitles.vtt",
"language": null,
"label": null,
"mime_type": "text/vtt"
}
]
});
let req: PlayItemRequest =
serde_json::from_value(payload).expect("frontend payload must deserialize");
assert_eq!(
req.subtitles.len(),
2,
"PlayItemRequest must carry the subtitle tracks, not silently ignore them"
);
let media = create_media_item(req, None).await.unwrap();
assert_eq!(
media.subtitles.len(),
2,
"create_media_item must thread the tracks onto the MediaItem the backend loads"
);
assert_eq!(media.subtitles[0].index, 2);
assert_eq!(media.subtitles[0].language.as_deref(), Some("eng"));
assert_eq!(media.subtitles[0].label.as_deref(), Some("English (SRT)"));
assert_eq!(media.subtitles[0].mime_type, "text/vtt");
// Order is the contract: `player_set_subtitle_track(n)` is a position in
// this list (see the note on `PlayItemRequest::subtitles`).
assert_eq!(media.subtitles[1].index, 3);
assert!(media.subtitles[1].language.is_none());
}
/// A request without subtitles must still deserialize — the field is
/// defaulted so the background-audio handoff and the autoplay/next-episode
/// callers keep compiling and sending what they always sent.
///
/// TRACES: UR-020 | IR-016 | UT-145
#[tokio::test]
async fn test_play_item_request_without_subtitles_defaults_to_empty() {
use super::{create_media_item, PlayItemRequest};
let req: PlayItemRequest = serde_json::from_value(serde_json::json!({
"id": "movie-1",
"title": "Movie",
"streamUrl": "https://jelly.example/Videos/movie-1/stream.mp4",
"videoCodec": "h264",
"needsTranscoding": false
}))
.expect("a subtitle-less payload must still deserialize");
assert!(req.subtitles.is_empty());
assert!(create_media_item(req, None)
.await
.unwrap()
.subtitles
.is_empty());
}
/// The JSON handed to Kotlin over JNI must use the keys
/// `JellyTauPlayer.load()` actually reads.
///
/// `MediaItem` is `rename_all = "camelCase"`, and the instinct (and the
/// house IPC rule) is to camelCase nested structs too — but
/// `JellyTauPlayer.kt` reads `subtitle.optString("mime_type", …)`. Renaming
/// the field would not fail to compile or fail the IPC; it would silently
/// fall back to the default MIME type for every track, so this is asserted
/// on the exact bytes `android/mod.rs` sends.
///
/// TRACES: UR-020 | IR-016, JA-008 | UT-146
#[test]
fn test_subtitle_json_for_jni_uses_the_keys_kotlin_reads() {
use crate::player::media::SubtitleTrack;
let subtitles = vec![SubtitleTrack {
index: 2,
url: "https://jelly.example/subs.vtt".to_string(),
language: Some("eng".to_string()),
label: Some("English".to_string()),
mime_type: "text/vtt".to_string(),
}];
// Exactly what player/android/mod.rs passes to loadWithMetadata.
let json = serde_json::to_string(&subtitles).unwrap();
let parsed: Vec<serde_json::Value> = serde_json::from_str(&json).unwrap();
let obj = parsed[0].as_object().unwrap();
for key in ["url", "language", "label", "mime_type"] {
assert!(
obj.contains_key(key),
"JellyTauPlayer.load() reads `{key}`; serialized keys were {:?}",
obj.keys().collect::<Vec<_>>()
);
}
assert!(
!obj.contains_key("mimeType"),
"camelCasing mime_type silently drops every track's MIME type on Android"
);
}
/// The audio-only handoff must play a downloaded file when there is one, /// The audio-only handoff must play a downloaded file when there is one,
/// rather than fetching an audio-only stream for media already on disk. /// rather than fetching an audio-only stream for media already on disk.
/// ///
+19 -1
View File
@@ -23,6 +23,23 @@ pub enum QueueContext {
} }
/// Represents a subtitle track /// Represents a subtitle track
///
/// 🔴 **Do not add `#[serde(rename_all = "camelCase")]` here.** This is the one
/// struct in the player that deliberately keeps snake_case on the wire, because
/// the *same* serialization feeds two consumers that both spell `mime_type`:
///
/// * the JNI boundary — `player/android/mod.rs` serializes `MediaItem::subtitles`
/// with `serde_json` and hands the string to `JellyTauPlayer.loadWithMetadata`,
/// whose parser reads `url`, `language`, `label` and `optString("mime_type")`;
/// * the IPC boundary — `PlayItemRequest::subtitles` deserializes this same type
/// from the frontend, and the generated binding (`SubtitleTrack` in
/// `bindings.ts`) therefore also declares `mime_type`.
///
/// Renaming would not break the build and would not fail the IPC: Kotlin's
/// `optString` would just fall back to its default MIME type for every track, so
/// the failure would be silent. UT-146 asserts the serialized keys.
///
/// TRACES: UR-020 | IR-016, JA-008 | UT-146
#[derive(specta::Type, Debug, Clone, Serialize, Deserialize, PartialEq)] #[derive(specta::Type, Debug, Clone, Serialize, Deserialize, PartialEq)]
pub struct SubtitleTrack { pub struct SubtitleTrack {
/// Stream index in the media source /// Stream index in the media source
@@ -33,7 +50,8 @@ pub struct SubtitleTrack {
pub language: Option<String>, pub language: Option<String>,
/// Display title /// Display title
pub label: Option<String>, pub label: Option<String>,
/// MIME type (e.g., "text/vtt", "application/x-subrip") /// MIME type (e.g., "text/vtt", "application/x-subrip").
/// Snake_case on purpose — see the note on the struct.
pub mime_type: String, pub mime_type: String,
} }
+1 -1
View File
@@ -32,7 +32,7 @@ pub mod webview_audio_backend;
pub use autoplay::{AutoplayDecision, AutoplaySettings}; pub use autoplay::{AutoplayDecision, AutoplaySettings};
pub use backend::{NullBackend, PlayerBackend, PlayerError}; pub use backend::{NullBackend, PlayerBackend, PlayerError};
pub use events::{PlayerEventEmitter, PlayerStatusEvent, TauriEventEmitter}; pub use events::{PlayerEventEmitter, PlayerStatusEvent, TauriEventEmitter};
pub use media::{MediaItem, MediaSource, MediaType, QueueContext}; pub use media::{MediaItem, MediaSource, MediaType, QueueContext, SubtitleTrack};
pub use queue::{QueueManager, RepeatMode}; pub use queue::{QueueManager, RepeatMode};
pub use seek::{determine_video_seek_strategy, VideoSeekStrategy}; pub use seek::{determine_video_seek_strategy, VideoSeekStrategy};
pub use session::{MediaSessionManager, MediaSessionType}; pub use session::{MediaSessionManager, MediaSessionType};
+179
View File
@@ -50,10 +50,189 @@ pub fn max_audio_channels() -> u32 {
clamp_max_audio_channels(reported) clamp_max_audio_channels(reported)
} }
/// Audio codecs the webview's `<video>` element can decode.
///
/// Deliberately narrower than what the platform reports: see
/// [`video_audio_codecs`].
const WEBVIEW_AUDIO_CODECS: &[&str] = &["aac", "mp3", "opus", "vorbis", "flac"];
/// The codec claimed when a device reports nothing we can use. Every renderer
/// decodes AAC, and claiming *something* is what makes the server transcode to
/// it rather than give up.
const FALLBACK_AUDIO_CODEC: &str = "aac";
/// Narrow a detected audio-codec list to what the renderer that will actually
/// play the **video** can decode.
///
/// The platform list comes from `MediaCodecList`, which describes ExoPlayer —
/// but video does not play through ExoPlayer. Both Android and Linux render it
/// in a webview `<video>` element, and Chromium/WebKit decode a much smaller set
/// than the platform does. Advertising the raw list makes Jellyfin direct-play a
/// track the webview cannot decode, and the user gets picture with no sound.
///
/// The gap is widest on devices whose vendor licenses Dolby: a phone with
/// `c2.dolby.eac3.decoder` reports `eac3`, so it — and only it — gets a silent
/// direct play where a leaner device is transcoded to AAC and plays fine.
///
/// This applies to the *video* direct-play profile only. Audio-only playback
/// really is ExoPlayer's, so its profile keeps the full platform list.
///
/// TRACES: UR-004 | DR-148 | UT-142
pub fn video_audio_codecs(detected: &str) -> String {
let kept: Vec<&str> = detected
.split(',')
.filter_map(|codec| {
let codec = codec.trim();
// Match case-insensitively but emit our own spelling: the platform
// list is assembled from MIME strings and its casing is not ours to
// forward to the server.
WEBVIEW_AUDIO_CODECS
.iter()
.copied()
.find(|supported| supported.eq_ignore_ascii_case(codec))
})
.collect();
if kept.is_empty() {
FALLBACK_AUDIO_CODEC.to_string()
} else {
kept.join(",")
}
}
/// Whether the webview `<video>` element can decode this audio codec.
///
/// TRACES: UR-004 | DR-149 | UT-148
pub fn webview_can_decode_audio(codec: &str) -> bool {
WEBVIEW_AUDIO_CODECS
.iter()
.any(|supported| supported.eq_ignore_ascii_case(codec.trim()))
}
/// Decide whether we must transcode *regardless of what the server negotiated*,
/// given the source's audio streams as `(codec, is_default)` in source order.
///
/// Advertising a narrow profile ([`video_audio_codecs`]) is necessary but not
/// sufficient: Jellyfin 10.11.5 enforces a `DirectPlayProfile`'s container and
/// video codec but **ignores its audio codec** — an E-AC-3 track is offered for
/// direct play even when the profile lists only AAC, and neither a `VideoAudio`
/// `CodecProfile` nor `MaxAudioChannels` changes that. So the client cannot
/// delegate this decision; it knows what its own renderer can decode and must
/// apply that itself.
///
/// The track that matters is the one the server will actually serve: the
/// default, or the first when none is marked. An unknown codec is left alone —
/// forcing a transcode on a guess would burn server CPU for files that play.
///
/// TRACES: UR-004 | DR-149 | UT-148
pub fn audio_forces_transcode(streams: &[(Option<&str>, bool)]) -> bool {
let served = streams
.iter()
.find(|(_, is_default)| *is_default)
.or_else(|| streams.first());
match served {
Some((Some(codec), _)) => !webview_can_decode_audio(codec),
// No audio at all, or a codec the server did not name: leave it alone.
Some((None, _)) | None => false,
}
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
#[test]
fn an_undecodable_default_track_forces_a_transcode() {
// The reported bug: one E-AC-3 track, which the webview cannot decode.
assert!(audio_forces_transcode(&[(Some("eac3"), false)]));
assert!(audio_forces_transcode(&[(Some("ac3"), true)]));
}
#[test]
fn a_decodable_track_is_left_to_direct_play() {
// Never spend server CPU on a file that already plays.
assert!(!audio_forces_transcode(&[(Some("aac"), true)]));
assert!(!audio_forces_transcode(&[(Some("mp3"), false)]));
}
#[test]
fn the_default_track_decides_not_the_first() {
// The webview plays the default track, so that is the one that has to be
// decodable — a supported track further down does not save us.
assert!(audio_forces_transcode(&[
(Some("aac"), false),
(Some("eac3"), true)
]));
assert!(!audio_forces_transcode(&[
(Some("eac3"), false),
(Some("aac"), true)
]));
}
#[test]
fn with_no_default_marked_the_first_track_decides() {
// Jellyfin leaves IsDefault false on every stream for some files; the
// server then serves the first, so judge that one.
assert!(audio_forces_transcode(&[
(Some("eac3"), false),
(Some("aac"), false)
]));
}
#[test]
fn a_source_with_no_audio_is_not_transcoded() {
// Nothing to rescue, and a transcode would not create audio.
assert!(!audio_forces_transcode(&[]));
}
#[test]
fn an_unknown_codec_is_not_second_guessed() {
// The server did not tell us the codec; assuming the worst would
// transcode files that play perfectly.
assert!(!audio_forces_transcode(&[(None, true)]));
}
#[test]
fn a_dolby_device_does_not_advertise_dolby_for_video() {
// The bug: a Motorola reporting c2.dolby.eac3.decoder direct-played
// E-AC-3 into a webview that cannot decode it — silent video, on that
// device only.
let codecs = video_audio_codecs("aac,ac3,amrnb,amrwb,eac3,flac,mp3,opus,pcm,vorbis");
assert_eq!(codecs, "aac,flac,mp3,opus,vorbis");
}
#[test]
fn codecs_the_webview_cannot_decode_are_dropped() {
// AMR and raw PCM come from the AOSP set, so this is not a Dolby-only
// problem — it is just rarer content.
assert_eq!(video_audio_codecs("amrnb,amrwb,pcm,aac"), "aac");
assert_eq!(video_audio_codecs("dts,truehd,mp3"), "mp3");
}
#[test]
fn a_list_the_webview_fully_supports_is_untouched() {
assert_eq!(
video_audio_codecs("aac,mp3,opus,vorbis,flac"),
"aac,mp3,opus,vorbis,flac"
);
}
#[test]
fn nothing_decodable_still_claims_aac() {
// Claiming an empty list invites the server to give up rather than
// transcode. AAC is universally decodable, so ask for it.
assert_eq!(video_audio_codecs("eac3,dts"), "aac");
assert_eq!(video_audio_codecs(""), "aac");
}
#[test]
fn spacing_and_case_in_the_platform_list_are_tolerated() {
// The list is assembled from MediaCodecList strings; do not let
// whitespace decide whether the user gets sound.
assert_eq!(video_audio_codecs("aac, EAC3 , Mp3"), "aac,mp3");
}
#[test] #[test]
fn an_unknown_route_falls_back_to_stereo() { fn an_unknown_route_falls_back_to_stereo() {
// Codec detection has not run yet, or the platform has no answer. Never // Codec detection has not run yet, or the platform has no answer. Never
+46 -8
View File
@@ -1,9 +1,7 @@
//! TRACES: UR-002, UR-007 | DR-013 | IR-010 //! TRACES: UR-002, UR-007 | DR-013 | IR-010
use async_trait::async_trait; use async_trait::async_trait;
#[cfg(target_os = "android")] use log::{debug, error, info, warn};
use log::warn;
use log::{debug, error, info};
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use std::sync::Arc; use std::sync::Arc;
@@ -1342,6 +1340,9 @@ impl MediaRepository for OnlineRepository {
index: i32, index: i32,
#[serde(default)] #[serde(default)]
codec: Option<String>, codec: Option<String>,
/// The track the server serves when the client pins none.
#[serde(default)]
is_default: bool,
} }
// Get detected codecs from Android MediaCodecList or use platform defaults // Get detected codecs from Android MediaCodecList or use platform defaults
@@ -1356,8 +1357,9 @@ impl MediaRepository for OnlineRepository {
// Linux desktop plays video through the WebKitGTK HTML5 <video> element, // Linux desktop plays video through the WebKitGTK HTML5 <video> element,
// which cannot reliably decode HEVC/AV1/VP9. Advertise only codecs the // which cannot reliably decode HEVC/AV1/VP9. Advertise only codecs the
// WebView can decode so Jellyfin transcodes anything else to h264 HLS. // WebView can decode so Jellyfin transcodes anything else to h264 HLS.
// (Audio-only files still direct-play via MPV, but the PlaybackInfo // (Audio-only files still direct-play via MPV; these codecs are what
// profile is shared, so we keep the broadly-supported audio codecs.) // both renderers handle, and the audio profile keeps them in full while
// the video profile is narrowed below.)
#[cfg(all(not(target_os = "android"), target_os = "linux"))] #[cfg(all(not(target_os = "android"), target_os = "linux"))]
let (video_codecs, audio_codecs) = let (video_codecs, audio_codecs) =
("h264".to_string(), "aac,mp3,opus,vorbis,flac".to_string()); ("h264".to_string(), "aac,mp3,opus,vorbis,flac".to_string());
@@ -1368,8 +1370,19 @@ impl MediaRepository for OnlineRepository {
"aac,mp3,opus,vorbis,flac".to_string(), "aac,mp3,opus,vorbis,flac".to_string(),
); );
// Video plays in a webview <video> element on every platform, which
// decodes a narrower audio set than the platform does — so the video
// profile must claim less than the audio-only profile. Without this a
// Dolby-licensed device advertises eac3, gets a direct play, and shows
// picture with no sound.
let video_audio_codecs = super::device_profile::video_audio_codecs(&audio_codecs);
info!("[DeviceProfile] Using video codecs: {}", video_codecs); info!("[DeviceProfile] Using video codecs: {}", video_codecs);
info!("[DeviceProfile] Using audio codecs: {}", audio_codecs); info!("[DeviceProfile] Using audio codecs: {}", audio_codecs);
info!(
"[DeviceProfile] Audio codecs for video direct play: {}",
video_audio_codecs
);
// Bound every profile by what the audio route can actually voice, so a // Bound every profile by what the audio route can actually voice, so a
// multichannel track is downmixed by the server rather than direct-played // multichannel track is downmixed by the server rather than direct-played
@@ -1388,12 +1401,16 @@ impl MediaRepository for OnlineRepository {
profile_type: "Video".to_string(), profile_type: "Video".to_string(),
container: "mp4,mkv,avi,mov,flv,ts,m2ts,webm,ogv,3gp".to_string(), container: "mp4,mkv,avi,mov,flv,ts,m2ts,webm,ogv,3gp".to_string(),
video_codec: Some(video_codecs.clone()), video_codec: Some(video_codecs.clone()),
audio_codec: audio_codecs.clone(), // The webview decodes this stream, not ExoPlayer/MPV.
audio_codec: video_audio_codecs.clone(),
}, },
DirectPlayProfile { DirectPlayProfile {
profile_type: "Audio".to_string(), profile_type: "Audio".to_string(),
container: "mp3,aac,flac,alac,wav,ogg,wma,opus".to_string(), container: "mp3,aac,flac,alac,wav,ogg,wma,opus".to_string(),
video_codec: None, video_codec: None,
// Audio-only really is the native player's, so it keeps the
// full platform list — narrowing it would transcode music
// that plays perfectly well.
audio_codec: audio_codecs.clone(), audio_codec: audio_codecs.clone(),
}, },
], ],
@@ -1459,9 +1476,29 @@ impl MediaRepository for OnlineRepository {
); );
} }
// Jellyfin 10.11.5 honours a DirectPlayProfile's container and video codec
// but ignores its audio codec, so it offers an E-AC-3 track for direct
// play even though DR-148 advertises only AAC — and the webview renders
// the picture in silence. Judge the track we would actually be served
// against what the webview can decode, and override the server's answer.
let audio_streams: Vec<(Option<&str>, bool)> = source
.media_streams
.iter()
.filter(|stream| stream.stream_type == "Audio")
.map(|stream| (stream.codec.as_deref(), stream.is_default))
.collect();
let audio_forces_transcode = super::device_profile::audio_forces_transcode(&audio_streams);
// Use TranscodingUrl from response if available (Streamyfin pattern) // Use TranscodingUrl from response if available (Streamyfin pattern)
let stream_url = if let Some(transcoding_url) = &source.transcoding_url { let stream_url = if let Some(transcoding_url) = &source.transcoding_url {
format!("{}{}", self.server_url, transcoding_url) format!("{}{}", self.server_url, transcoding_url)
} else if audio_forces_transcode {
warn!(
"[PlaybackInfo] Server offered direct play for audio the webview cannot decode ({:?}) — forcing an HLS transcode",
audio_streams.first().and_then(|(codec, _)| *codec)
);
self.get_video_stream_url(item_id, Some(&source.id), None, None)
.await?
} else { } else {
// Fall back to direct stream URL. No audioStreamIndex: static=true // Fall back to direct stream URL. No audioStreamIndex: static=true
// serves the original file untouched, and pinning index 0 (the video // serves the original file untouched, and pinning index 0 (the video
@@ -1482,8 +1519,9 @@ impl MediaRepository for OnlineRepository {
media_source_id: source.id.clone(), media_source_id: source.id.clone(),
play_session_id: response.play_session_id, play_session_id: response.play_session_id,
stream_url, stream_url,
direct_play: source.supports_direct_play, direct_play: source.supports_direct_play && !audio_forces_transcode,
needs_transcoding: !source.supports_direct_play && source.supports_transcoding, needs_transcoding: audio_forces_transcode
|| (!source.supports_direct_play && source.supports_transcoding),
}) })
} }
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"$schema": "https://schema.tauri.app/config/2", "$schema": "https://schema.tauri.app/config/2",
"productName": "jellytau", "productName": "jellytau",
"version": "0.4.6", "version": "0.4.8",
"identifier": "com.dtourolle.jellytau", "identifier": "com.dtourolle.jellytau",
"build": { "build": {
"beforeDevCommand": "bun run dev", "beforeDevCommand": "bun run dev",
+42 -2
View File
@@ -2231,7 +2231,29 @@ itemType?: string | null;
* Series ID for TV episodes. Needed alongside `item_type` so the backend can * Series ID for TV episodes. Needed alongside `item_type` so the backend can
* look up the next episode when a background-audio track ends. * look up the next episode when a background-audio track ends.
*/ */
seriesId?: string | null } seriesId?: string | null;
/**
* Subtitle tracks to sideload, with URLs the frontend has already resolved.
*
* Only the native backends use these: on Android they become the
* `MediaItem.SubtitleConfiguration`s ExoPlayer renders. The HTML5 path
* builds its own `<track>` children instead and ignores this list.
*
* **Order is the contract.** `player_set_subtitle_track(n)` reaches
* `JellyTauPlayer.setSubtitleTrack(n)`, which indexes into ExoPlayer's
* *text track groups* i.e. the position of the sideloaded configuration,
* not the Jellyfin stream index (which is kept on each entry for the UI's
* benefit). So `n` must be a position in this very array, and the array
* must not be reordered or filtered between building it and sending it.
* `nativeSubtitleArrayIndex()` on the frontend computes `n` from the same
* list that is sent here, for exactly this reason.
*
* Defaulted so the background-audio handoff and the autoplay/next-episode
* callers, which have no subtitles to offer, need not send the field.
*
* TRACES: UR-020 | IR-016, JA-008 | UT-145
*/
subtitles?: SubtitleTrack[] }
/** /**
* Queue context for remote transfer - what type of queue is this? * Queue context for remote transfer - what type of queue is this?
*/ */
@@ -2769,6 +2791,23 @@ export type StreamKind = "audio" | "video" | "subtitle" |
"other" "other"
/** /**
* Represents a subtitle track * Represents a subtitle track
*
* 🔴 **Do not add `#[serde(rename_all = "camelCase")]` here.** This is the one
* struct in the player that deliberately keeps snake_case on the wire, because
* the *same* serialization feeds two consumers that both spell `mime_type`:
*
* * the JNI boundary `player/android/mod.rs` serializes `MediaItem::subtitles`
* with `serde_json` and hands the string to `JellyTauPlayer.loadWithMetadata`,
* whose parser reads `url`, `language`, `label` and `optString("mime_type")`;
* * the IPC boundary `PlayItemRequest::subtitles` deserializes this same type
* from the frontend, and the generated binding (`SubtitleTrack` in
* `bindings.ts`) therefore also declares `mime_type`.
*
* Renaming would not break the build and would not fail the IPC: Kotlin's
* `optString` would just fall back to its default MIME type for every track, so
* the failure would be silent. UT-146 asserts the serialized keys.
*
* TRACES: UR-020 | IR-016, JA-008 | UT-146
*/ */
export type SubtitleTrack = { export type SubtitleTrack = {
/** /**
@@ -2788,7 +2827,8 @@ language: string | null;
*/ */
label: string | null; label: string | null;
/** /**
* MIME type (e.g., "text/vtt", "application/x-subrip") * MIME type (e.g., "text/vtt", "application/x-subrip").
* Snake_case on purpose see the note on the struct.
*/ */
mime_type: string } mime_type: string }
/** /**
+146 -61
View File
@@ -1,6 +1,6 @@
<!-- TRACES: UR-003, UR-005, UR-020, UR-021, UR-026, UR-040, UR-061 | DR-010, DR-023, DR-024, DR-051, DR-052, DR-092, DR-098, DR-099 --> <!-- TRACES: UR-003, UR-005, UR-020, UR-021, UR-026, UR-040, UR-061 | DR-010, DR-023, DR-024, DR-051, DR-052, DR-092, DR-098, DR-099 -->
<script lang="ts"> <script lang="ts">
import { onMount, onDestroy, untrack } from "svelte"; import { onMount, onDestroy, tick, untrack } from "svelte";
import { get } from "svelte/store"; import { get } from "svelte/store";
import { goto } from "$app/navigation"; import { goto } from "$app/navigation";
import { commands } from "$lib/api/bindings"; import { commands } from "$lib/api/bindings";
@@ -14,6 +14,14 @@
import SleepTimerIndicator from "./SleepTimerIndicator.svelte"; import SleepTimerIndicator from "./SleepTimerIndicator.svelte";
import CachedImage from "../common/CachedImage.svelte"; import CachedImage from "../common/CachedImage.svelte";
import { videoFitClass } from "./videoFit"; import { videoFitClass } from "./videoFit";
import {
resolveSubtitleTracks,
reconcileSelectedSubtitle,
videoCrossOriginMode,
nativeSubtitleTracks,
nativeSubtitleArrayIndex,
type RenderableSubtitleTrack,
} from "./subtitleTracks";
import { sleepTimerActive, sleepTimerExpiredSignal } from "$lib/stores/sleepTimer"; import { sleepTimerActive, sleepTimerExpiredSignal } from "$lib/stores/sleepTimer";
import { playbackPosition, playerState } from "$lib/stores/player"; import { playbackPosition, playerState } from "$lib/stores/player";
import * as html5Adapter from "$lib/player/html5Adapter"; import * as html5Adapter from "$lib/player/html5Adapter";
@@ -275,6 +283,61 @@
return tracks; return tracks;
}); });
// ===== Subtitle <track> sources for the HTML5 element (Linux/WebKitGTK) =====
// Resolved asynchronously into state and only then rendered. The URLs come
// from an async command, so they must never be bound to `src` directly — the
// original markup did exactly that and put "[object Promise]" on every track,
// which is why the whole block ended up commented out (and why selecting a
// subtitle did nothing: with no <track> children the element has no
// textTracks for the adapter to switch on).
// TRACES: UR-020 | DR-023 | UT-143, UT-144
let renderedSubtitleTracks = $state<RenderableSubtitleTrack[]>([]);
// The subtitle list actually handed to the native backend at load time
// (Android/ExoPlayer). Kept because `player_set_subtitle_track` takes a
// *position in this list*, not a Jellyfin stream index — see
// nativeSubtitleArrayIndex. It is written once, from onMount, before the
// play request; it is not derived, because the request is what fixed the
// backend's idea of the track order.
// TRACES: UR-020 | IR-016 | UT-147
let sentSubtitleTracks = $state<RenderableSubtitleTrack[]>([]);
// Cross-origin <track> fetches use the media element's CORS setting; see
// videoCrossOriginMode for why this is opt-in and same-origin-only.
const videoCrossOrigin = $derived(
videoCrossOriginMode(currentStreamUrl, subtitleTracks().length)
);
$effect(() => {
const streams = media?.mediaStreams ?? null;
const itemId = media?.id;
const sourceId = mediaSourceId;
// Native (ExoPlayer) mode renders subtitles itself; the element has none.
if (!useHtml5Element || !itemId || !sourceId) {
renderedSubtitleTracks = [];
return;
}
let cancelled = false;
void (async () => {
const tracks = await resolveSubtitleTracks(streams, (index) => getSubtitleUrl(index));
if (cancelled) return;
renderedSubtitleTracks = tracks;
// Keep the menu's checkmark and the element's text tracks in agreement:
// a selection that no longer resolves collapses to "Off".
const selected = reconcileSelectedSubtitle(tracks, untrack(() => selectedSubtitleIndex));
selectedSubtitleIndex = selected;
// The <track> children were just (re)created, so re-apply the selection to
// the new TextTrack objects — otherwise a surviving selection shows nothing.
await tick();
if (!cancelled) applySubtitleToElement(selected);
})();
return () => {
cancelled = true;
};
});
// Track the last prop value to detect when parent changes the URL (vs internal seeks) // Track the last prop value to detect when parent changes the URL (vs internal seeks)
let lastStreamUrlProp = $state(""); let lastStreamUrlProp = $state("");
@@ -544,28 +607,23 @@
console.log("[VideoPlayer] Initializing player for:", media.name); console.log("[VideoPlayer] Initializing player for:", media.name);
console.log("[VideoPlayer] Stream URL:", currentStreamUrl); console.log("[VideoPlayer] Stream URL:", currentStreamUrl);
// Build subtitle tracks for native player // Resolve subtitle URLs for the native (ExoPlayer) path. These must be
const subtitleTracks = []; // in hand *before* the play request: ExoPlayer sideloads subtitles as
if (media.mediaStreams && mediaSourceId) { // MediaItem.SubtitleConfigurations, which have to exist before
const subtitles = media.mediaStreams.filter(s => s.kind === "subtitle"); // prepare() — there is no way to add one to a loaded item afterwards.
for (const sub of subtitles) { //
try { // Awaiting here is safe despite the native-mode pitfall: that rule is
const url = await getSubtitleUrl(sub.index); // about Svelte *lifecycle* calls (onMount/onDestroy) after an await,
if (url) { // which throw lifecycle_outside_component and used to be misread as an
subtitleTracks.push({ // init failure. Nothing is registered here, and the background-audio
index: sub.index, // subscriptions above already ran synchronously. resolveSubtitleTracks
url: url, // fans the requests out in parallel, so this costs one round trip, not
language: sub.language || null, // one per subtitle stream as the old serial loop did.
label: sub.displayTitle || sub.language || `Track ${sub.index}`, // TRACES: UR-020 | IR-016, JA-008 | UT-147
mime_type: "text/vtt" // Jellyfin converts to WebVTT sentSubtitleTracks = mediaSourceId
}); ? await resolveSubtitleTracks(media.mediaStreams, (index) => getSubtitleUrl(index))
} : [];
} catch (err) { console.log(`[VideoPlayer] Sending ${sentSubtitleTracks.length} subtitle tracks to the backend`);
console.warn(`[VideoPlayer] Failed to build subtitle URL for track ${sub.index}:`, err);
}
}
console.log(`[VideoPlayer] Built ${subtitleTracks.length} subtitle tracks for native player`);
}
// Call Rust backend to start playback // Call Rust backend to start playback
// Rust will choose ExoPlayer (Android), libmpv (Linux), or tell us to use HTML5 // Rust will choose ExoPlayer (Android), libmpv (Linux), or tell us to use HTML5
@@ -576,6 +634,10 @@
id: media.id, id: media.id,
videoCodec: needsTranscoding ? "hevc" : "h264", videoCodec: needsTranscoding ? "hevc" : "h264",
needsTranscoding: needsTranscoding, needsTranscoding: needsTranscoding,
// Order matters: player_set_subtitle_track(n) is a position in this
// array. Previously this array was built and then dropped, so
// ExoPlayer got a MediaItem with no subtitles at all.
subtitles: nativeSubtitleTracks(sentSubtitleTracks),
}); });
// Rust tells us which backend it's using // Rust tells us which backend it's using
@@ -1670,41 +1732,59 @@
showSubtitleMenu = !showSubtitleMenu; showSubtitleMenu = !showSubtitleMenu;
} }
async function selectSubtitle(streamIndex: number | null, arrayIndex?: number) { /**
console.log("[VideoPlayer] Selecting subtitle - streamIndex:", streamIndex, "arrayIndex:", arrayIndex); * Show exactly one (or no) text track on the HTML5 element. `null` disables
* every track, which is what the menu's "Off" entry means.
*
* TRACES: UR-020 | DR-023
*/
function applySubtitleToElement(streamIndex: number | null) {
if (!useHtml5Element || !videoElement || !videoElement.textTracks) return;
// Disable all text tracks first, so "Off" genuinely turns subtitles off.
for (let i = 0; i < videoElement.textTracks.length; i++) {
videoElement.textTracks[i].mode = "disabled";
}
if (streamIndex === null) return;
// Find the corresponding track element by stream index.
videoElement.querySelectorAll("track").forEach((track) => {
const trackStreamIndex = parseInt(track.getAttribute("data-stream-index") || "-1");
if (trackStreamIndex === streamIndex && track.track) {
track.track.mode = "showing";
console.log("[VideoPlayer] Enabled subtitle track:", streamIndex);
}
});
}
/**
* Apply the menu's choice. `streamIndex` is always the Jellyfin media-stream
* index (or `null` for "Off") — the UI speaks stream indices throughout.
*
* The native backend does not: `player_set_subtitle_track(n)` reaches
* `JellyTauPlayer.setSubtitleTrack(n)`, which indexes ExoPlayer's text track
* groups, i.e. the position of the sideloaded subtitle configuration. That
* position is derived from `sentSubtitleTracks` — the exact array sent with
* the play request — and not from the menu's row number, which counts every
* subtitle *stream* including ones whose URL never resolved and so were never
* sideloaded.
*
* TRACES: UR-020 | DR-023, IR-016 | UT-147
*/
async function selectSubtitle(streamIndex: number | null) {
console.log("[VideoPlayer] Selecting subtitle - streamIndex:", streamIndex);
selectedSubtitleIndex = streamIndex; selectedSubtitleIndex = streamIndex;
showSubtitleMenu = false; showSubtitleMenu = false;
// For HTML5 video element, update the text tracks // For HTML5 video element, update the text tracks
if (useHtml5Element && videoElement && videoElement.textTracks) { if (useHtml5Element) {
// Disable all text tracks first applySubtitleToElement(streamIndex);
for (let i = 0; i < videoElement.textTracks.length; i++) { } else {
videoElement.textTracks[i].mode = "disabled";
}
// Enable the selected track if not null
if (streamIndex !== null) {
// Find the corresponding track element by stream index
const tracks = videoElement.querySelectorAll("track");
tracks.forEach((track) => {
const trackStreamIndex = parseInt(track.getAttribute("data-stream-index") || "-1");
if (trackStreamIndex === streamIndex) {
const textTrack = track.track;
if (textTrack) {
textTrack.mode = "showing";
console.log("[VideoPlayer] Enabled subtitle track:", streamIndex);
}
}
});
}
} else if (!useHtml5Element) {
// For native backend (Android), send command to change subtitle track // For native backend (Android), send command to change subtitle track
try { try {
// Use array index for ExoPlayer (0-based position in subtitle tracks array) const indexToUse = nativeSubtitleArrayIndex(sentSubtitleTracks, streamIndex);
// If streamIndex is null (disable), pass null; otherwise use arrayIndex
const indexToUse = streamIndex === null ? null : (arrayIndex !== undefined ? arrayIndex : streamIndex);
await commands.playerSetSubtitleTrack(indexToUse); await commands.playerSetSubtitleTrack(indexToUse);
console.log("[VideoPlayer] Native backend subtitle track changed - arrayIndex:", arrayIndex, "used:", indexToUse); console.log("[VideoPlayer] Native backend subtitle track changed - streamIndex:", streamIndex, "position:", indexToUse);
} catch (error) { } catch (error) {
console.error("[VideoPlayer] Failed to set subtitle track:", error); console.error("[VideoPlayer] Failed to set subtitle track:", error);
} }
@@ -1743,6 +1823,7 @@
<video <video
bind:this={videoElement} bind:this={videoElement}
src={currentStreamUrl.includes('.m3u8') && Hls.isSupported() ? '' : currentStreamUrl} src={currentStreamUrl.includes('.m3u8') && Hls.isSupported() ? '' : currentStreamUrl}
crossorigin={videoCrossOrigin}
class={videoFitClass()} class={videoFitClass()}
class:invisible={!isMediaReady} class:invisible={!isMediaReady}
style="filter: brightness({brightness})" style="filter: brightness({brightness})"
@@ -1761,18 +1842,22 @@
onloadstart={handleLoadStart} onloadstart={handleLoadStart}
onclick={handleSurfaceClick} onclick={handleSurfaceClick}
> >
<!-- Temporarily disabled to debug playback issues <!--
{#each subtitleTracks() as track} Subtitles for the HTML5 path. `src` is a resolved string (see
renderedSubtitleTracks); `data-stream-index` is what
Html5PlayerAdapter.selectSubtitle() matches on. No `default`
attribute: a default track auto-shows, which would contradict the
menu opening on "Off".
-->
{#each renderedSubtitleTracks as track (track.streamIndex)}
<track <track
kind="subtitles" kind="subtitles"
src={getSubtitleUrl(track.index)} src={track.url}
srclang={track.language || "unknown"} srclang={track.srclang}
label={track.displayTitle || track.language || `Track ${track.index}`} label={track.label}
data-stream-index={track.index} data-stream-index={track.streamIndex}
default={track.isDefault}
/> />
{/each} {/each}
-->
</video> </video>
{:else} {:else}
<!-- Android ExoPlayer - video rendered natively in SurfaceView behind WebView --> <!-- Android ExoPlayer - video rendered natively in SurfaceView behind WebView -->
@@ -2072,9 +2157,9 @@
{/if} {/if}
</button> </button>
<!-- Subtitle tracks --> <!-- Subtitle tracks -->
{#each subtitleTracks() as track, i} {#each subtitleTracks() as track}
<button <button
onclick={() => selectSubtitle(track.index, i)} onclick={() => selectSubtitle(track.index)}
class="w-full text-left px-3 py-2 text-white hover:bg-white/10 rounded transition-colors flex items-center justify-between {selectedSubtitleIndex === track.index ? 'bg-white/20' : ''}" class="w-full text-left px-3 py-2 text-white hover:bg-white/10 rounded transition-colors flex items-center justify-between {selectedSubtitleIndex === track.index ? 'bg-white/20' : ''}"
> >
<div class="flex flex-col"> <div class="flex flex-col">
@@ -0,0 +1,303 @@
import { describe, it, expect } from "vitest";
import { readFileSync } from "node:fs";
import { resolve } from "node:path";
import {
subtitleStreamsOf,
subtitleTrackLabel,
resolveSubtitleTracks,
reconcileSelectedSubtitle,
videoCrossOriginMode,
nativeSubtitleTracks,
nativeSubtitleArrayIndex,
type SubtitleStreamLike,
} from "./subtitleTracks";
/**
* Subtitles on the Linux / WebKitGTK HTML5 `<video>` path.
*
* TRACES: UR-020 | DR-023 | UT-143, UT-144
*
* The bug this guards: VideoPlayer rendered no `<track>` children at all (the
* block was commented out "to debug playback issues"), so
* `Html5PlayerAdapter.selectSubtitle()` walked an empty `textTracks` list and
* the subtitle menu was inert on Linux. The reason it had to be disabled is
* visible in the original markup `src={getSubtitleUrl(track.index)}` bound the
* *Promise* returned by an async function to the attribute, so every track's src
* stringified to "[object Promise]", an unloadable resource hanging off the
* media element.
*
* So the fix has two halves and both are tested here: URLs must be resolved into
* plain strings *before* they reach the markup, and the markup must actually
* render the tracks (with the `data-stream-index` the adapter matches on).
*/
const SUBS: SubtitleStreamLike[] = [
{ index: 2, kind: "subtitle", language: "eng", displayTitle: "English (SRT)", isDefault: true },
{ index: 3, kind: "subtitle", language: "fre", displayTitle: "French", isDefault: false },
];
const STREAMS: SubtitleStreamLike[] = [
{ index: 0, kind: "video", language: null, displayTitle: "1080p" },
{ index: 1, kind: "audio", language: "eng", displayTitle: "English AAC" },
...SUBS,
];
const url = (i: number) => `http://jelly.example/Videos/x/Subtitles/${i}/0/subtitles.vtt?api_key=k`;
describe("subtitleStreamsOf", () => {
it("keeps only subtitle streams, in stream order", () => {
expect(subtitleStreamsOf(STREAMS).map((s) => s.index)).toEqual([2, 3]);
});
it("tolerates missing media streams", () => {
expect(subtitleStreamsOf(null)).toEqual([]);
expect(subtitleStreamsOf(undefined)).toEqual([]);
});
});
describe("subtitleTrackLabel", () => {
it("prefers the display title, then language, then the index", () => {
expect(subtitleTrackLabel({ index: 2, displayTitle: "English (SRT)", language: "eng" })).toBe("English (SRT)");
expect(subtitleTrackLabel({ index: 2, displayTitle: null, language: "eng" })).toBe("eng");
expect(subtitleTrackLabel({ index: 2 })).toBe("Track 2");
});
});
describe("resolveSubtitleTracks", () => {
it("resolves real string URLs — never a Promise — for every subtitle stream", async () => {
const tracks = await resolveSubtitleTracks(STREAMS, async (i) => url(i));
expect(tracks).toHaveLength(2);
for (const track of tracks) {
expect(typeof track.url).toBe("string");
// The exact regression: a Promise bound to src stringifies to this.
expect(String(track.url)).not.toContain("[object Promise]");
expect(track.url).toContain("subtitles.vtt");
}
// The adapter matches <track> elements by data-stream-index, so the stream
// index has to survive resolution.
expect(tracks.map((t) => t.streamIndex)).toEqual([2, 3]);
expect(tracks.map((t) => t.label)).toEqual(["English (SRT)", "French"]);
expect(tracks.map((t) => t.srclang)).toEqual(["eng", "fre"]);
expect(tracks[0].isDefault).toBe(true);
});
it("drops tracks whose URL cannot be built instead of rendering a dead src", async () => {
const tracks = await resolveSubtitleTracks(SUBS, async (i) => {
if (i === 2) throw new Error("no repository");
return url(i);
});
expect(tracks.map((t) => t.streamIndex)).toEqual([3]);
});
it("drops empty and non-string URLs", async () => {
const tracks = await resolveSubtitleTracks(SUBS, async (i) =>
i === 2 ? " " : (undefined as unknown as string),
);
expect(tracks).toEqual([]);
});
it("returns nothing when there are no subtitle streams", async () => {
expect(await resolveSubtitleTracks([STREAMS[0]], async (i) => url(i))).toEqual([]);
expect(await resolveSubtitleTracks(null, async (i) => url(i))).toEqual([]);
});
});
describe("reconcileSelectedSubtitle", () => {
it("starts off (null) and keeps 'off' selectable", async () => {
const tracks = await resolveSubtitleTracks(SUBS, async (i) => url(i));
expect(reconcileSelectedSubtitle(tracks, null)).toBeNull();
});
it("keeps a selection that is still renderable", async () => {
const tracks = await resolveSubtitleTracks(SUBS, async (i) => url(i));
expect(reconcileSelectedSubtitle(tracks, 3)).toBe(3);
});
it("falls back to off when the selected track is gone (new item / failed URL)", async () => {
const tracks = await resolveSubtitleTracks(SUBS, async (i) => url(i));
expect(reconcileSelectedSubtitle(tracks, 9)).toBeNull();
expect(reconcileSelectedSubtitle([], 3)).toBeNull();
});
it("never auto-selects the server's default track", async () => {
// The menu opens on "Off" and a <track default> would auto-show, so the UI
// would claim subtitles are off while they are burned over the picture.
const tracks = await resolveSubtitleTracks(SUBS, async (i) => url(i));
expect(tracks[0].isDefault).toBe(true);
expect(reconcileSelectedSubtitle(tracks, null)).toBeNull();
});
});
describe("videoCrossOriginMode", () => {
it("opts into CORS for a server stream that has subtitles", () => {
expect(videoCrossOriginMode("http://jelly.example/Videos/x/master.m3u8", 2)).toBe("anonymous");
expect(videoCrossOriginMode("https://jelly.example/Videos/x/stream.mp4", 1)).toBe("anonymous");
});
it("leaves a local/offline source alone so playback cannot regress", () => {
expect(videoCrossOriginMode("asset://localhost/movie.mkv", 2)).toBeUndefined();
expect(videoCrossOriginMode("file:///home/u/movie.mkv", 2)).toBeUndefined();
});
it("stays out of the way when there is nothing to load", () => {
expect(videoCrossOriginMode("http://jelly.example/x.m3u8", 0)).toBeUndefined();
expect(videoCrossOriginMode("", 0)).toBeUndefined();
});
it("is decided by inputs known at first render, so it cannot flip mid-load", () => {
// Same answer before and after the async URL resolution completes.
const before = videoCrossOriginMode("http://jelly.example/x.m3u8", SUBS.length);
const after = videoCrossOriginMode("http://jelly.example/x.m3u8", SUBS.length);
expect(before).toBe(after);
});
});
/**
* Subtitles on the Android / ExoPlayer native path.
*
* TRACES: UR-020 | IR-016, JA-008 | UT-147
*
* The bug this guards: VideoPlayer built a fully-resolved subtitle array in
* onMount and then never sent it `commands.playerPlayItem({...})` passed only
* streamUrl/title/id/videoCodec/needsTranscoding so every MediaItem reached
* ExoPlayer with zero SubtitleConfigurations and `setSubtitleTrack(n)` logged
* "Invalid subtitle track index".
*
* And the second half: `setSubtitleTrack(n)` indexes ExoPlayer's *text track
* groups*, i.e. the position of the sideloaded configuration not the Jellyfin
* stream index. The menu used to pass its own row position, which is a position
* in the *unresolved* stream list; the moment one subtitle URL failed to
* resolve, the two lists diverged and every track below the gap selected the
* wrong subtitle.
*/
describe("nativeSubtitleTracks", () => {
it("maps to the wire shape Rust deserializes and Kotlin parses", async () => {
const resolved = await resolveSubtitleTracks(SUBS, async (i) => url(i));
const payload = nativeSubtitleTracks(resolved);
expect(payload).toHaveLength(2);
// Kotlin reads url/language/label/mime_type; Rust's SubtitleTrack keeps
// snake_case for exactly that reason, and so does the generated binding.
for (const track of payload) {
expect(Object.keys(track).sort()).toEqual(
["index", "label", "language", "mime_type", "url"].sort(),
);
expect(track).not.toHaveProperty("mimeType");
expect(track.mime_type).toBe("text/vtt");
}
// Jellyfin serves every subtitle stream as WebVTT here, and the stream index
// rides along so the UI can keep talking in stream indices.
expect(payload.map((t) => t.index)).toEqual([2, 3]);
expect(payload[0].url).toContain("subtitles.vtt");
expect(payload[0].language).toBe("eng");
expect(payload[0].label).toBe("English (SRT)");
});
it("preserves stream order, because that order is the selection index", async () => {
const resolved = await resolveSubtitleTracks(STREAMS, async (i) => url(i));
expect(nativeSubtitleTracks(resolved).map((t) => t.index)).toEqual(
resolved.map((t) => t.streamIndex),
);
});
it("has nothing to send when no subtitle URL resolved", async () => {
expect(nativeSubtitleTracks(await resolveSubtitleTracks(SUBS, async () => ""))).toEqual([]);
expect(nativeSubtitleTracks([])).toEqual([]);
});
it("carries a null language/label through rather than inventing one", () => {
const payload = nativeSubtitleTracks([
{ streamIndex: 5, url: "u.vtt", srclang: "und", label: "Track 5", isDefault: false },
]);
expect(payload[0].language).toBeNull();
expect(payload[0].label).toBe("Track 5");
});
});
describe("nativeSubtitleArrayIndex", () => {
it("returns the position in the list that was actually sent, not the stream index", async () => {
const resolved = await resolveSubtitleTracks(SUBS, async (i) => url(i));
expect(nativeSubtitleArrayIndex(resolved, 2)).toBe(0);
expect(nativeSubtitleArrayIndex(resolved, 3)).toBe(1);
});
it("stays aligned when a subtitle URL failed to resolve (the mis-selection bug)", async () => {
// Stream 2 has no URL, so it is not among the sideloaded configurations.
// The menu's own row for stream 3 is position 1, but ExoPlayer only has one
// text track group — position 0. Sending 1 would select nothing.
const resolved = await resolveSubtitleTracks(SUBS, async (i) => {
if (i === 2) throw new Error("no repository");
return url(i);
});
expect(resolved).toHaveLength(1);
expect(nativeSubtitleArrayIndex(resolved, 3)).toBe(0);
});
it("maps 'Off' to null so the backend disables text instead of selecting track 0", async () => {
const resolved = await resolveSubtitleTracks(SUBS, async (i) => url(i));
expect(nativeSubtitleArrayIndex(resolved, null)).toBeNull();
});
it("maps a track that was never sent to null rather than to a wrong position", async () => {
const resolved = await resolveSubtitleTracks(SUBS, async (i) => url(i));
expect(nativeSubtitleArrayIndex(resolved, 99)).toBeNull();
expect(nativeSubtitleArrayIndex([], 3)).toBeNull();
});
});
describe("VideoPlayer markup (the regression that made the menu inert)", () => {
const source = readFileSync(
resolve(__dirname, "VideoPlayer.svelte"),
"utf-8",
);
it("renders <track> elements instead of leaving them commented out", () => {
expect(source).not.toContain("Temporarily disabled to debug playback issues");
expect(source).toMatch(/<track\b/);
expect(source).toContain('kind="subtitles"');
});
/** The rendered element, not a `<track>` mentioned in prose. */
const trackElement = source.slice(source.search(/<track\s/), source.search(/<track\s/) + 400);
it("keeps data-stream-index — Html5PlayerAdapter.selectSubtitle matches on it", () => {
expect(trackElement).toContain("data-stream-index");
});
it("never binds the async getSubtitleUrl() Promise to src", () => {
expect(source).not.toMatch(/src=\{\s*getSubtitleUrl\(/);
});
it("does not mark any track default (a default track auto-shows)", () => {
expect(trackElement).not.toMatch(/\bdefault=/);
});
});
/**
* The half of the Android fix that lives in the component: the resolved list has
* to actually be handed to `playerPlayItem`, and the index sent to the backend
* has to be computed from that same list.
*
* TRACES: UR-020 | IR-016 | UT-147
*/
describe("VideoPlayer -> playerPlayItem (the tracks that were built and thrown away)", () => {
const source = readFileSync(resolve(__dirname, "VideoPlayer.svelte"), "utf-8");
/** The playerPlayItem({...}) argument object. */
const playItemCall = (() => {
const start = source.indexOf("commands.playerPlayItem(");
expect(start).toBeGreaterThan(-1);
return source.slice(start, source.indexOf("});", start) + 3);
})();
it("sends the subtitle tracks it resolved", () => {
expect(playItemCall).toMatch(/\bsubtitles:/);
});
it("selects by position in the sent list, not by the menu's row number", () => {
expect(source).toContain("nativeSubtitleArrayIndex");
// The old code forwarded the `{#each}` index straight to the backend.
expect(source).not.toMatch(/playerSetSubtitleTrack\(\s*arrayIndex\s*\)/);
});
});
+227
View File
@@ -0,0 +1,227 @@
// Subtitle plumbing for the Linux / WebKitGTK HTML5 `<video>` playback path.
//
// Extracted from VideoPlayer.svelte so it is unit-testable, and because the
// original inline version hid a fatal mistake in plain sight: `getSubtitleUrl()`
// is async, so `src={getSubtitleUrl(track.index)}` bound a *Promise* to the
// attribute and every `<track>` pointed at "[object Promise]". The whole block
// was commented out rather than fixed, which left `<video>` with no text tracks
// at all — `Html5PlayerAdapter.selectSubtitle()` then iterated an empty
// `textTracks` list and the subtitle menu silently did nothing.
//
// The rule this module enforces: URLs are resolved to plain strings *here*, off
// the render path, and only tracks that actually resolved are handed to the
// markup.
//
// The Android / ExoPlayer native path shares this module (see
// nativeSubtitleTracks / nativeSubtitleArrayIndex at the bottom): it needs the
// exact same "resolve the URLs first, keep only what resolved" list, just handed
// to Rust instead of to `<track>` elements.
//
// TRACES: UR-020 | DR-023, IR-016 | UT-143, UT-144, UT-147
import type { SubtitleTrack } from "$lib/api/bindings";
/**
* The subset of `MediaStream` (from the generated bindings) this module needs.
* Kept structural so tests do not have to build full binding objects.
*/
export interface SubtitleStreamLike {
index: number;
kind?: string | null;
language?: string | null;
displayTitle?: string | null;
isDefault?: boolean;
isForced?: boolean;
}
/** A subtitle stream whose URL resolved — i.e. one we can actually render. */
export interface RenderableSubtitleTrack {
/** Jellyfin media-stream index; the adapter matches `data-stream-index`. */
streamIndex: number;
/** Fully resolved WebVTT URL. Always a string, never a Promise. */
url: string;
srclang: string;
label: string;
/** Server's "default" flag — shown in the menu, never auto-enabled. */
isDefault: boolean;
}
/** Subtitle streams of a media item, in stream order. */
export function subtitleStreamsOf(
streams: readonly SubtitleStreamLike[] | null | undefined,
): SubtitleStreamLike[] {
if (!streams) return [];
return streams.filter((s) => s.kind === "subtitle");
}
/** Human label for a subtitle stream, matching the menu's own fallback chain. */
export function subtitleTrackLabel(stream: SubtitleStreamLike): string {
return stream.displayTitle || stream.language || `Track ${stream.index}`;
}
/** A src we are willing to put on a `<track>`: a non-blank plain string. */
function isRenderableUrl(url: unknown): url is string {
return typeof url === "string" && url.trim().length > 0;
}
/**
* Resolve every subtitle stream's URL and return only the tracks that can be
* rendered. `resolveUrl` failures are swallowed per track: one unavailable
* subtitle must not cost the user the others, and a dead `src` on a media
* element is exactly what made this block get disabled in the first place.
*/
export async function resolveSubtitleTracks(
streams: readonly SubtitleStreamLike[] | null | undefined,
resolveUrl: (streamIndex: number) => Promise<string>,
): Promise<RenderableSubtitleTrack[]> {
const subtitles = subtitleStreamsOf(streams);
if (subtitles.length === 0) return [];
const resolved = await Promise.all(
subtitles.map(async (stream) => {
try {
const url = await resolveUrl(stream.index);
if (!isRenderableUrl(url)) return null;
return {
streamIndex: stream.index,
url,
srclang: stream.language || "und",
label: subtitleTrackLabel(stream),
isDefault: stream.isDefault === true,
} satisfies RenderableSubtitleTrack;
} catch {
return null;
}
}),
);
return resolved.filter((t): t is RenderableSubtitleTrack => t !== null);
}
/**
* The selection to keep once the rendered track list changes.
*
* Subtitles are OFF unless the user turns them on: `null` in, `null` out. The
* server's `isDefault` flag is deliberately NOT promoted to a selection (and the
* markup deliberately omits the `default` attribute, which would auto-show the
* track) the menu opens on "Off", so auto-enabling would make the UI lie about
* what is on screen, and it would change behaviour for every user who has never
* asked for subtitles.
*
* A selection that is no longer renderable (new item, or a URL that failed to
* resolve) collapses to off, so the menu's checkmark can never point at a track
* that does not exist on the element.
*/
export function reconcileSelectedSubtitle(
tracks: readonly RenderableSubtitleTrack[],
selected: number | null,
): number | null {
if (selected === null) return null;
return tracks.some((t) => t.streamIndex === selected) ? selected : null;
}
function originOf(url: string): string | null {
try {
const parsed = new URL(url);
if (parsed.protocol !== "http:" && parsed.protocol !== "https:") return null;
return parsed.origin;
} catch {
return null;
}
}
/**
* The `crossorigin` value for the `<video>` element, or undefined for none.
*
* Text-track fetches are CORS-enabled per the HTML spec and use the *media
* element's* CORS setting, so a cross-origin `<track>` never loads unless the
* element opts in. The webview page's origin is `tauri://localhost`, so every
* subtitle served by Jellyfin is cross-origin.
*
* Opting in is only safe when the media itself comes from an http(s) server
* the same Jellyfin that already answers hls.js' cross-origin XHRs, so we know
* it sends the headers. For a local/offline source (`file:`/`asset:`) we leave
* the attribute off: subtitles staying dark there is the status quo, whereas
* forcing CORS onto the video fetch could break playback outright.
*
* Deliberately keyed on the *count of subtitle streams* rather than on the
* resolved tracks: both inputs are known at first render, so the attribute is
* decided before the element starts loading and never flips underneath an
* in-flight media fetch.
*/
export function videoCrossOriginMode(
streamUrl: string,
subtitleStreamCount: number,
): "anonymous" | undefined {
if (subtitleStreamCount <= 0) return undefined;
return originOf(streamUrl) ? "anonymous" : undefined;
}
// ===== Native (Android / ExoPlayer) path ====================================
//
// The HTML5 element gets `<track>` children; the native backend instead gets the
// list *up front*, as part of the play request, because ExoPlayer sideloads
// subtitles as `MediaItem.SubtitleConfiguration`s that must exist before
// `prepare()`. There is no "add a subtitle later" — a track absent from the
// MediaItem simply does not exist as far as the player is concerned.
/**
* Map resolved tracks onto the wire shape `PlayItemRequest.subtitles` carries.
*
* The element type is the *generated* `SubtitleTrack` binding on purpose, so
* `bun run check` fails if the Rust struct's field names ever move. In
* particular `mime_type` is snake_case and must stay that way: the very same
* bytes are re-serialized across JNI in `player/android/mod.rs`, and
* `JellyTauPlayer.load()` reads `optString("mime_type")`. Renaming it to
* `mimeType` would not error anywhere Kotlin would just silently fall back to
* its default MIME type for every track.
*
* Jellyfin is asked for every subtitle stream as WebVTT (see
* `getSubtitleUrl(..., "vtt")`), so the MIME type is fixed rather than derived
* from the source subtitle codec.
*
* TRACES: UR-020 | IR-016, JA-008 | UT-147
*/
export function nativeSubtitleTracks(
tracks: readonly RenderableSubtitleTrack[],
): SubtitleTrack[] {
return tracks.map((track) => ({
index: track.streamIndex,
url: track.url,
// `srclang` carries "und" for a stream with no language, which is the right
// value for a `<track>` but is not a language the native side should claim.
language: track.srclang === "und" ? null : track.srclang,
label: track.label,
mime_type: "text/vtt",
}));
}
/**
* The argument for `player_set_subtitle_track` on the native backend.
*
* 🔴 This is **not** the Jellyfin stream index.
* `JellyTauPlayer.setSubtitleTrack(n)` filters ExoPlayer's track groups down to
* `C.TRACK_TYPE_TEXT` and indexes that list with `n`, so `n` is the *position of
* the sideloaded subtitle configuration* which is the position in the array
* that `nativeSubtitleTracks()` produced and `playerPlayItem` sent.
*
* The menu's own row number is not that position: the menu lists every subtitle
* *stream*, while only the streams whose URL resolved are sent. One failed URL
* and everything below it selects the wrong subtitle. So the index is looked up
* in the sent list instead of being passed down from the `{#each}`.
*
* `null` (the menu's "Off") stays `null`, which the backend turns into -1 and
* Kotlin turns into "disable text tracks". A stream that was never sent also
* maps to `null`: disabling subtitles is a truthful outcome, whereas guessing a
* position would show the user a different language than the one they clicked.
*
* TRACES: UR-020 | IR-016 | UT-147
*/
export function nativeSubtitleArrayIndex(
tracks: readonly RenderableSubtitleTrack[],
streamIndex: number | null,
): number | null {
if (streamIndex === null) return null;
const position = tracks.findIndex((t) => t.streamIndex === streamIndex);
return position === -1 ? null : position;
}