feat(playback): let Rust decide what stream to play, and say so
Playing a video meant asking the server to re-encode it, always. That
decision was made nowhere and written down nowhere, so whoever needed it
re-derived it downstream — the player worked out whether it had been handed
a playlist by looking for ".m3u8" in the URL, in two places. A viewer paid
for a transcode of a file their device could have played untouched, and the
app could not tell them which it was.
One negotiation now produces one self-describing StreamSelection — direct
play, remux or transcode; over a playlist, a plain HTTP file, or a local one
— and every renderer consumes that same answer.
Measured against the development server (Jellyfin 10.11.5), 400 items
sampled for codec mix and 40 put through a real PlaybackInfo negotiation
per profile:
Linux / WebKitGTK (h264 only, 2ch) 3/40 — 7% direct play
Android / ExoPlayer (hevc, ac3/eac3, 6ch) 34/40 — 85% direct play
The library is ~80% hevc, which is why the two diverge so hard. The payoff
is overwhelmingly Android, where 85% of plays were starting a transcode
nobody needed. Linux stays near 7% until libmpv decodes the picture — the
h264-only profile is a WebKitGTK constraint, not a JellyTau choice.
DR-219 StreamSelection: url + tagged Transport (hls/progressive/localFile)
+ PlaybackKind (directPlay/directStream/transcode) + the negotiated
rendition + this source's ladder + a needs_transcoding flag derived
in Rust so the rule is answered once. Both enums are serde-tagged
so the frontend matches a discriminant, not a substring. The paths
that never negotiate get the same shape from Rust rather than
assembling one — media_local_selection for a downloaded file,
LiveStreamInfo.transport for a live channel — so there is no second
place where a transport is decided.
DR-220 The ceiling becomes two levels: a durable device default (Settings,
persisted) and a per-playback override the in-player picker sets.
The picker had called itself a "this film, this connection" control
since it was written but wrote the process-wide default, so dropping
one awkward film to 2 Mbps silently capped every video played
afterwards for the rest of the process, with Settings still showing
the old value. The override is cleared whenever playback moves to a
new item, which stops it surviving into an autoplayed next episode.
effective_streaming_quality() is the single resolution point.
DR-221 The quality picker is filled from what this media source can offer.
Rust marks a rung exceeds_source when its ceiling is at or above the
source's own bitrate — such a rung is another way to spell Original
— and the frontend does not draw those. Original is never marked; a
source whose bitrate the server does not report marks nothing, which
keeps every rung offered.
DR-222 Direct play and direct stream are negotiated, with two client-side
overrides on top because the server's answer is right about the file
and wrong about what this app will do with it: undecodable audio
(Jellyfin 10.11.5 honours a DirectPlayProfile's container and video
codec but ignores its audio codec, so it offers direct play for an
E-AC-3 track the webview renders in silence) and a viewer-pinned
audio track the file does not default to. A direct stream is a remux
and is deliberately not counted as transcoding.
DR-223 Dropped on measurement, not deferred. A master playlist from this
server carries exactly one EXT-X-STREAM-INF: Jellyfin builds it from
the single rendition the request asked for rather than publishing a
ladder. So there is no adaptation for hls.js to be preserving and
none mpv would lose — the claim that there was, in
playback-backend-unification.md, does not hold. Recorded rather than
deleted because it is a measurement: a server that does publish a
ladder would change the answer.
DR-224 Every backend consumes the same selection. The queue item carries
the transport, so player_seek_video picks its seek strategy from the
backend's decision instead of the last stream_url.contains(".m3u8")
in the codebase. Items queued by a path that never negotiated carry
None and fall back to needs_transcoding, which is exact rather than
a guess because every transcode this app requests is HLS (DR-140).
The frontend loader decision moves to streamTransport.ts so it can be
tested: the two cases that pin it are the ones that failed against the old
implementation — a progressive stream whose URL contains ".m3u8" must not
get an HLS loader, and an HLS stream whose URL contains none must.
Also verified the URL the direct-play branch builds actually serves playable
bytes: 206, video/mp4, valid ISO-BMFF, and a mid-file range works, so
seeking a direct play works.
The spec is folded into docs/architecture/{01,02,03} and deleted, per the
rule that docs/specs holds only work that has not shipped. DR-121 leaves
read-through-media-cache.md with a pointer; that spec keeps its capture half.
Not verified: real playback on a device. Direct play changes what actually
gets played, and neither fixtures nor curl prove the WebKitGTK and ExoPlayer
paths render it.
This commit is contained in:
+253
-12
@@ -120,7 +120,7 @@ async playerSeek(position: number) : Promise<PlayerStatus> {
|
||||
*
|
||||
* This command analyzes the current video stream and automatically chooses
|
||||
* the best seeking strategy:
|
||||
* - HLS streams (.m3u8): Use native seeking
|
||||
* - HLS streams: Use native seeking
|
||||
* - Direct play streams: Use native seeking
|
||||
* - Transcoded non-HLS: Request new stream URL from server starting at seek position
|
||||
*
|
||||
@@ -263,13 +263,17 @@ async playerGetStreamingQualities() : Promise<([StreamingQuality, string, string
|
||||
* two-sided split: HTML5 gets the URL back and reloads its own element, while a
|
||||
* native backend is reloaded here.
|
||||
*
|
||||
* The change applies to this playback *and* to everything started afterwards
|
||||
* (it sets the process-wide ceiling), but it is deliberately **not** persisted:
|
||||
* the in-player picker is a "this film, this connection" control, and the
|
||||
* durable default belongs to Settings. `player_set_video_settings` is the one
|
||||
* that writes to the database.
|
||||
* The change applies to **this playback only**. The in-player picker is a
|
||||
* "this film, this connection" control and its doc has always said so, but it
|
||||
* used to be implemented by writing the process-wide ceiling — so choosing
|
||||
* 2 Mbps to get one awkward film moving silently capped every video played
|
||||
* afterwards for the rest of the process, with the Settings screen still
|
||||
* showing the old value and nothing in the UI admitting the change. It now
|
||||
* sets a per-playback override that the next item clears; the durable default
|
||||
* belongs to Settings, and `player_set_video_settings` is the one that writes
|
||||
* to the database.
|
||||
*
|
||||
* TRACES: UR-074 | DR-162
|
||||
* TRACES: UR-074, UR-079 | DR-162, DR-225
|
||||
*/
|
||||
async playerSetStreamQuality(repositoryHandle: string, quality: StreamingQuality, useHtml5: boolean, currentPosition: number | null, mediaSourceId: string | null, audioStreamIndex: number | null) : Promise<StreamQualityResponse> {
|
||||
return await TAURI_INVOKE("player_set_stream_quality", { repositoryHandle, quality, useHtml5, currentPosition, mediaSourceId, audioStreamIndex });
|
||||
@@ -1033,6 +1037,22 @@ async markDownloadFailed(downloadId: number, errorMessage: string) : Promise<nul
|
||||
async mediaLocalUrl(path: string) : Promise<string> {
|
||||
return await TAURI_INVOKE("media_local_url", { path });
|
||||
},
|
||||
/**
|
||||
* The stream selection for a downloaded file.
|
||||
*
|
||||
* The local-playback counterpart to `repository_get_stream_selection`. A file
|
||||
* on disk needs no negotiation — it is a direct play over a local transport,
|
||||
* with no quality ladder, because nothing about it can be re-negotiated — but
|
||||
* the *frontend must not be the one to say so*. It gets the same
|
||||
* [`StreamSelection`] shape as a streamed source so the player has one contract
|
||||
* to consume rather than two, and so no caller has to infer a transport from a
|
||||
* loopback URL.
|
||||
*
|
||||
* TRACES: UR-071, UR-079 | DR-224
|
||||
*/
|
||||
async mediaLocalSelection(path: string) : Promise<StreamSelection> {
|
||||
return await TAURI_INVOKE("media_local_selection", { path });
|
||||
},
|
||||
/**
|
||||
* Start downloading a file immediately
|
||||
* This command actually downloads the file using the worker
|
||||
@@ -1618,6 +1638,24 @@ async repositoryGetPlaybackInfo(handle: string, itemId: string) : Promise<Playba
|
||||
async repositoryGetVideoStreamUrl(handle: string, itemId: string, mediaSourceId: string | null, audioStreamIndex: number | null) : Promise<string> {
|
||||
return await TAURI_INVOKE("repository_get_video_stream_url", { handle, itemId, mediaSourceId, audioStreamIndex });
|
||||
},
|
||||
/**
|
||||
* Decide what stream to play for a video, and describe it.
|
||||
*
|
||||
* Replaces `repository_get_video_stream_url` for playback. The returned
|
||||
* [`StreamSelection`] carries the transport explicitly, so the frontend picks
|
||||
* its loader from a tagged enum instead of testing the URL for `.m3u8`; and it
|
||||
* carries the quality ladder as it applies to *this* source, so the picker can
|
||||
* stop offering rungs that produce the same bytes as Original.
|
||||
*
|
||||
* No start-position parameter, for the same reason as the URL builder: a
|
||||
* position on an HLS playlist is copied onto every segment URI and the server
|
||||
* rejects each with `400` (DR-181). Callers resume by seeking after load.
|
||||
*
|
||||
* TRACES: UR-070, UR-079 | DR-224, DR-226, DR-227 | UT-212
|
||||
*/
|
||||
async repositoryGetStreamSelection(handle: string, itemId: string, mediaSourceId: string | null, audioStreamIndex: number | null) : Promise<StreamSelection> {
|
||||
return await TAURI_INVOKE("repository_get_stream_selection", { handle, itemId, mediaSourceId, audioStreamIndex });
|
||||
},
|
||||
/**
|
||||
* Get audio stream URL for a track
|
||||
*/
|
||||
@@ -1965,7 +2003,7 @@ export type AudioTrackSwitchResponse =
|
||||
/**
|
||||
* HTML5 needs to reload stream with new audio track
|
||||
*/
|
||||
{ strategy: "reloadStream"; new_url: string; position: number }
|
||||
{ strategy: "reloadStream"; selection: StreamSelection; position: number }
|
||||
/**
|
||||
* Authentication result
|
||||
*/
|
||||
@@ -2329,7 +2367,18 @@ excludedItemIds?: string[] }
|
||||
* streamed; the server returns a transcoding URL (already absolute) plus a
|
||||
* `live_stream_id` that can later be used to close the stream.
|
||||
*/
|
||||
export type LiveStreamInfo = { streamUrl: string; playSessionId: string | null; liveStreamId: string | null; mediaSourceId: string | null }
|
||||
export type LiveStreamInfo = { streamUrl: string; playSessionId: string | null; liveStreamId: string | null; mediaSourceId: string | null;
|
||||
/**
|
||||
* How to open `stream_url`.
|
||||
*
|
||||
* A live channel is always an HLS transcode — the server has to repackage a
|
||||
* broadcast mux into something a browser can play, and there is no static
|
||||
* file to direct-play. Saying so here means the player page never has to
|
||||
* work it out from the URL, which is the whole of DR-224.
|
||||
*
|
||||
* TRACES: UR-079 | DR-224
|
||||
*/
|
||||
transport: Transport }
|
||||
/**
|
||||
* An LMS multi-room sync group, as returned by JellyLMS `/JellyLms/SyncGroups`.
|
||||
*
|
||||
@@ -2568,6 +2617,18 @@ videoCodec: string;
|
||||
* Whether the video requires server-side transcoding
|
||||
*/
|
||||
needsTranscoding: boolean;
|
||||
/**
|
||||
* How this item's stream is fetched, as the backend decided it.
|
||||
*
|
||||
* Carried on the queue item so a later seek/reload does not have to guess.
|
||||
* `None` for items queued by a path that never negotiated (audio tracks,
|
||||
* direct URLs) and for anything queued before this field existed, where the
|
||||
* caller falls back to `needs_transcoding` — every transcode this app
|
||||
* requests is HLS (DR-140), so that fallback is exact rather than a guess.
|
||||
*
|
||||
* TRACES: UR-003, UR-004, UR-079 | DR-224, DR-229
|
||||
*/
|
||||
transport?: Transport | null;
|
||||
/**
|
||||
* Optional now-playing metadata. Used by the background-audio handoff so the
|
||||
* lockscreen/miniplayer show the item (title/subtitle/artwork). Defaulted so
|
||||
@@ -2680,6 +2741,32 @@ supportsNativeVideo: boolean }
|
||||
* Playback information
|
||||
*/
|
||||
export type PlaybackInfo = { mediaSourceId: string; playSessionId: string; streamUrl: string; directPlay: boolean; needsTranscoding: boolean }
|
||||
/**
|
||||
* What the server is doing to the source to produce this stream.
|
||||
*
|
||||
* Distinct from [`Transport`] because the two are genuinely independent: a
|
||||
* direct-streamed remux and a transcode can both arrive over HLS, and a direct
|
||||
* play can arrive progressively or as a local file. Keeping them apart is what
|
||||
* lets the UI say "this is not costing the server anything" without inferring
|
||||
* it from a URL shape.
|
||||
*
|
||||
* TRACES: UR-079 | DR-227
|
||||
*/
|
||||
export type PlaybackKind =
|
||||
/**
|
||||
* The source file is served untouched. No server CPU, no quality loss.
|
||||
*/
|
||||
{ type: "directPlay" } |
|
||||
/**
|
||||
* The container is repackaged but the codecs are copied — cheap, and
|
||||
* visually identical to the source.
|
||||
*/
|
||||
{ type: "directStream" } |
|
||||
/**
|
||||
* The server is re-encoding. The only case where a bitrate ceiling can
|
||||
* actually be honoured, and the only one that costs the server real work.
|
||||
*/
|
||||
{ type: "transcode" }
|
||||
/**
|
||||
* Playback mode - local device, remote session, or idle
|
||||
*/
|
||||
@@ -2779,6 +2866,18 @@ videoCodec?: string | null;
|
||||
* Whether the video requires server-side transcoding
|
||||
*/
|
||||
needsTranscoding?: boolean;
|
||||
/**
|
||||
* How this item's stream is fetched, as the backend decided it.
|
||||
*
|
||||
* Carried on the queue item so a later seek/reload does not have to guess.
|
||||
* `None` for items queued by a path that never negotiated (audio tracks,
|
||||
* direct URLs) and for anything queued before this field existed, where the
|
||||
* caller falls back to `needs_transcoding` — every transcode this app
|
||||
* requests is HLS (DR-140), so that fallback is exact rather than a guess.
|
||||
*
|
||||
* TRACES: UR-003, UR-004, UR-079 | DR-224, DR-229
|
||||
*/
|
||||
transport?: Transport | null;
|
||||
/**
|
||||
* Video width in pixels
|
||||
*/
|
||||
@@ -3061,6 +3160,38 @@ alreadyDownloaded: number;
|
||||
* Number of tracks skipped (no jellyfin ID or other reasons)
|
||||
*/
|
||||
skipped: number }
|
||||
/**
|
||||
* One rung of the quality picker, as it applies to *this* media source.
|
||||
*
|
||||
* The picker used to be filled from the fixed [`StreamingQuality::ALL`] ladder,
|
||||
* which meant offering "20 Mbps" for a 1.1 Mbps podcast — eight rungs, six of
|
||||
* them indistinguishable from Original. `exceeds_source` is what lets the
|
||||
* frontend render that honestly without knowing anything about bitrates.
|
||||
*
|
||||
* TRACES: UR-070, UR-079 | DR-226, DR-121
|
||||
*/
|
||||
export type QualityOption = { quality: StreamingQuality;
|
||||
/**
|
||||
* Human label ("8 Mbps"). Lives in Rust beside the number it describes.
|
||||
*/
|
||||
label: string;
|
||||
/**
|
||||
* Secondary line ("1080p").
|
||||
*/
|
||||
detail: string;
|
||||
/**
|
||||
* True when this rung's ceiling is at or above what the source itself
|
||||
* carries, so selecting it yields the same stream as `Original`.
|
||||
*
|
||||
* The frontend renders these differently (or hides them); it does not
|
||||
* decide which they are.
|
||||
*/
|
||||
exceedsSource: boolean;
|
||||
/**
|
||||
* The source's own bitrate, when the server reported one. Presentation
|
||||
* only — the picker shows "Original (6.7 Mbps)" rather than a bare word.
|
||||
*/
|
||||
sourceBitrate: number | null }
|
||||
/**
|
||||
* Response for queue queries
|
||||
*/
|
||||
@@ -3069,6 +3200,36 @@ export type QueueStatus = { items: PlayerMediaItem[]; currentIndex: number | nul
|
||||
* Remote session status for UI updates
|
||||
*/
|
||||
export type RemoteSessionStatus = { position: number; duration: number | null; isPlaying: boolean; nowPlayingItem: NowPlayingItem | null }
|
||||
/**
|
||||
* The rendition actually negotiated — what the viewer is receiving right now.
|
||||
*
|
||||
* `None` on a [`StreamSelection`] when the source is being direct-played as-is:
|
||||
* there is no *chosen* rendition in that case, only the file itself, and
|
||||
* reporting the ceiling that happened to be set would misdescribe it.
|
||||
*
|
||||
* TRACES: UR-079 | DR-224, DR-225
|
||||
*/
|
||||
export type Rendition = {
|
||||
/**
|
||||
* The rung of the ladder this stream was built against.
|
||||
*/
|
||||
quality: StreamingQuality;
|
||||
/**
|
||||
* Total bits per second the stream may use, when a ceiling applies.
|
||||
*/
|
||||
maxBitrate: number | null;
|
||||
/**
|
||||
* Resolution ceiling, when one applies. `None` preserves the source's.
|
||||
*/
|
||||
maxHeight: number | null;
|
||||
/**
|
||||
* Video codec the server was asked to produce.
|
||||
*/
|
||||
videoCodec: string | null;
|
||||
/**
|
||||
* Audio codec the server was asked to produce.
|
||||
*/
|
||||
audioCodec: string | null }
|
||||
/**
|
||||
* Repeat mode for the queue
|
||||
*
|
||||
@@ -3186,9 +3347,59 @@ export type StreamQualityResponse =
|
||||
*/
|
||||
{ strategy: "native"; position: number } |
|
||||
/**
|
||||
* HTML5 must reload its element with this URL.
|
||||
* HTML5 must reload its element with this selection.
|
||||
*/
|
||||
{ strategy: "reloadStream"; new_url: string; position: number }
|
||||
{ strategy: "reloadStream"; selection: StreamSelection; position: number }
|
||||
/**
|
||||
* Everything a player backend needs to open a stream, and everything the UI
|
||||
* needs to describe it.
|
||||
*
|
||||
* Replaces the bare `String` URL that `get_video_stream_url` used to return.
|
||||
*
|
||||
* TRACES: UR-079 | DR-224, DR-226, DR-227
|
||||
*/
|
||||
export type StreamSelection = {
|
||||
/**
|
||||
* The URL (or loopback URL) to open.
|
||||
*/
|
||||
url: string;
|
||||
/**
|
||||
* How to fetch it. Replaces the `.m3u8` substring check.
|
||||
*/
|
||||
transport: Transport;
|
||||
/**
|
||||
* What the server is doing to the source to produce it.
|
||||
*/
|
||||
playbackKind: PlaybackKind;
|
||||
/**
|
||||
* The negotiated rendition; `None` when direct-playing the source as-is.
|
||||
*/
|
||||
rendition: Rendition | null;
|
||||
/**
|
||||
* What this media source can offer, for the quality picker (DR-226).
|
||||
*/
|
||||
available: QualityOption[];
|
||||
/**
|
||||
* The media source this selection is for, so a later re-open (quality
|
||||
* change, audio-track switch, transcoded seek) targets the same one.
|
||||
*/
|
||||
mediaSourceId: string | null;
|
||||
/**
|
||||
* The transcode identity the server keyed this job by, when there is one.
|
||||
*/
|
||||
playSessionId: string | null;
|
||||
/**
|
||||
* Whether the server is spending encoder time on this stream.
|
||||
*
|
||||
* Derived from [`playback_kind`](Self::playback_kind) rather than left for
|
||||
* the frontend to compute: "which kinds count as transcoding" is a domain
|
||||
* rule, and a direct *stream* is a remux that must not be counted. The
|
||||
* queue's long-standing `needs_transcoding` flag and the seek strategy both
|
||||
* read this, so there is one answer rather than three.
|
||||
*
|
||||
* TRACES: UR-079 | DR-224, DR-227
|
||||
*/
|
||||
needsTranscoding: boolean }
|
||||
/**
|
||||
* A ceiling on how much bandwidth a *video* stream may consume.
|
||||
*
|
||||
@@ -3269,6 +3480,36 @@ itemName: string | null }
|
||||
* Statistics about the thumbnail cache
|
||||
*/
|
||||
export type ThumbnailCacheStats = { totalSizeBytes: number; itemCount: number; limitBytes: number }
|
||||
/**
|
||||
* How the bytes of a chosen stream are fetched.
|
||||
*
|
||||
* This field exists to delete a substring search. The frontend previously
|
||||
* decided which loader to attach by testing `url.contains(".m3u8")`, which is a
|
||||
* domain fact reconstructed in the presentation layer — the same class of leak
|
||||
* as the item-type taxonomy that `check:boundary` guards, and one that breaks
|
||||
* silently the moment a server serves a playlist from a path that does not end
|
||||
* in `.m3u8`, or serves a progressive file from one that does.
|
||||
*
|
||||
* Tagged (`{"type":"hls"}`) rather than a bare string so the frontend matches a
|
||||
* discriminant instead of comparing text.
|
||||
*
|
||||
* TRACES: UR-079 | DR-224
|
||||
*/
|
||||
export type Transport =
|
||||
/**
|
||||
* An HLS playlist. The webview attaches hls.js (or Safari's native loader);
|
||||
* ExoPlayer uses its HLS media source.
|
||||
*/
|
||||
{ type: "hls" } |
|
||||
/**
|
||||
* A single progressive HTTP resource, seekable by byte range.
|
||||
*/
|
||||
{ type: "progressive" } |
|
||||
/**
|
||||
* A file already on disk — a completed download, or the loopback media
|
||||
* server standing in front of one.
|
||||
*/
|
||||
{ type: "localFile" }
|
||||
/**
|
||||
* User information
|
||||
*/
|
||||
@@ -3316,7 +3557,7 @@ export type VideoSeekResponse =
|
||||
/**
|
||||
* Reload stream from new position (transcoded non-HLS)
|
||||
*/
|
||||
{ strategy: "reloadStream"; new_url: string; seek_offset: number }
|
||||
{ strategy: "reloadStream"; selection: StreamSelection; seek_offset: number }
|
||||
/**
|
||||
* Video playback settings
|
||||
*/
|
||||
|
||||
Reference in New Issue
Block a user