feat(playback): let Rust decide what stream to play, and say so
Playing a video meant asking the server to re-encode it, always. That
decision was made nowhere and written down nowhere, so whoever needed it
re-derived it downstream — the player worked out whether it had been handed
a playlist by looking for ".m3u8" in the URL, in two places. A viewer paid
for a transcode of a file their device could have played untouched, and the
app could not tell them which it was.
One negotiation now produces one self-describing StreamSelection — direct
play, remux or transcode; over a playlist, a plain HTTP file, or a local one
— and every renderer consumes that same answer.
Measured against the development server (Jellyfin 10.11.5), 400 items
sampled for codec mix and 40 put through a real PlaybackInfo negotiation
per profile:
Linux / WebKitGTK (h264 only, 2ch) 3/40 — 7% direct play
Android / ExoPlayer (hevc, ac3/eac3, 6ch) 34/40 — 85% direct play
The library is ~80% hevc, which is why the two diverge so hard. The payoff
is overwhelmingly Android, where 85% of plays were starting a transcode
nobody needed. Linux stays near 7% until libmpv decodes the picture — the
h264-only profile is a WebKitGTK constraint, not a JellyTau choice.
DR-219 StreamSelection: url + tagged Transport (hls/progressive/localFile)
+ PlaybackKind (directPlay/directStream/transcode) + the negotiated
rendition + this source's ladder + a needs_transcoding flag derived
in Rust so the rule is answered once. Both enums are serde-tagged
so the frontend matches a discriminant, not a substring. The paths
that never negotiate get the same shape from Rust rather than
assembling one — media_local_selection for a downloaded file,
LiveStreamInfo.transport for a live channel — so there is no second
place where a transport is decided.
DR-220 The ceiling becomes two levels: a durable device default (Settings,
persisted) and a per-playback override the in-player picker sets.
The picker had called itself a "this film, this connection" control
since it was written but wrote the process-wide default, so dropping
one awkward film to 2 Mbps silently capped every video played
afterwards for the rest of the process, with Settings still showing
the old value. The override is cleared whenever playback moves to a
new item, which stops it surviving into an autoplayed next episode.
effective_streaming_quality() is the single resolution point.
DR-221 The quality picker is filled from what this media source can offer.
Rust marks a rung exceeds_source when its ceiling is at or above the
source's own bitrate — such a rung is another way to spell Original
— and the frontend does not draw those. Original is never marked; a
source whose bitrate the server does not report marks nothing, which
keeps every rung offered.
DR-222 Direct play and direct stream are negotiated, with two client-side
overrides on top because the server's answer is right about the file
and wrong about what this app will do with it: undecodable audio
(Jellyfin 10.11.5 honours a DirectPlayProfile's container and video
codec but ignores its audio codec, so it offers direct play for an
E-AC-3 track the webview renders in silence) and a viewer-pinned
audio track the file does not default to. A direct stream is a remux
and is deliberately not counted as transcoding.
DR-223 Dropped on measurement, not deferred. A master playlist from this
server carries exactly one EXT-X-STREAM-INF: Jellyfin builds it from
the single rendition the request asked for rather than publishing a
ladder. So there is no adaptation for hls.js to be preserving and
none mpv would lose — the claim that there was, in
playback-backend-unification.md, does not hold. Recorded rather than
deleted because it is a measurement: a server that does publish a
ladder would change the answer.
DR-224 Every backend consumes the same selection. The queue item carries
the transport, so player_seek_video picks its seek strategy from the
backend's decision instead of the last stream_url.contains(".m3u8")
in the codebase. Items queued by a path that never negotiated carry
None and fall back to needs_transcoding, which is exact rather than
a guess because every transcode this app requests is HLS (DR-140).
The frontend loader decision moves to streamTransport.ts so it can be
tested: the two cases that pin it are the ones that failed against the old
implementation — a progressive stream whose URL contains ".m3u8" must not
get an HLS loader, and an HLS stream whose URL contains none must.
Also verified the URL the direct-play branch builds actually serves playable
bytes: 206, video/mp4, valid ISO-BMFF, and a mid-file range works, so
seeking a direct play works.
The spec is folded into docs/architecture/{01,02,03} and deleted, per the
rule that docs/specs holds only work that has not shipped. DR-121 leaves
read-through-media-cache.md with a pointer; that spec keeps its capture half.
Not verified: real playback on a device. Direct play changes what actually
gets played, and neither fixtures nor curl prove the WebKitGTK and ExoPlayer
paths render it.
This commit is contained in:
@@ -3,8 +3,8 @@
|
||||
import { page } from "$app/stores";
|
||||
import { goto } from "$app/navigation";
|
||||
import { commands } from "$lib/api/bindings";
|
||||
import { downloadedFilePath, resolveVideoSource } from "$lib/player/localSource";
|
||||
import type { PlayQueueRequest } from "$lib/api/bindings";
|
||||
import { downloadedFilePath } from "$lib/player/localSource";
|
||||
import type { PlayQueueRequest, StreamSelection } from "$lib/api/bindings";
|
||||
import type { MediaItem, MediaKind } from "$lib/api/types";
|
||||
import { auth } from "$lib/stores/auth";
|
||||
import { library } from "$lib/stores/library";
|
||||
@@ -76,7 +76,15 @@
|
||||
const hasNext = $derived($hasNextStore);
|
||||
const hasPrevious = $derived($hasPreviousStore);
|
||||
let currentMedia = $state<MediaItem | null>(null);
|
||||
let streamUrl = $state<string | null>(null);
|
||||
/**
|
||||
* What to play, as the backend decided it. Null while still resolving.
|
||||
*
|
||||
* Replaces a bare URL string: the transport travels with it, so neither this
|
||||
* page nor VideoPlayer has to work out whether the URL is a playlist.
|
||||
*
|
||||
* TRACES: UR-079 | DR-224
|
||||
*/
|
||||
let selection = $state<StreamSelection | null>(null);
|
||||
let mediaSourceId = $state<string | null>(null);
|
||||
let isVideo = $state(false);
|
||||
let isLive = $state(false); // Whether this is a live stream (Live TV channel) - no seek/resume
|
||||
@@ -94,7 +102,7 @@
|
||||
|
||||
// Which player component to render. Video without a stream URL is "pending"
|
||||
// (still resolving), never audio — see playerSurface.ts.
|
||||
const surface = $derived(resolvePlayerSurface({ isVideo, streamUrl }));
|
||||
const surface = $derived(resolvePlayerSurface({ isVideo, streamUrl: selection?.url ?? null }));
|
||||
|
||||
onMount(() => {
|
||||
// Start position polling (only for audio via MPV backend)
|
||||
@@ -308,17 +316,17 @@
|
||||
const fullPath = downloadedFilePath(storagePath, localDownload.filePath);
|
||||
log.debug("loadAndPlay: Full local path:", fullPath);
|
||||
|
||||
// Serve the file over the loopback media server rather than the asset
|
||||
// protocol: the asset protocol answers a range-less request with the
|
||||
// entire file, so a downloaded film never finished loading. Rust mints
|
||||
// the URL (it holds the port and the per-session token).
|
||||
// TRACES: UR-071 | DR-137
|
||||
const localUrl = await commands.mediaLocalUrl(fullPath);
|
||||
log.debug("loadAndPlay: Local media URL resolved");
|
||||
|
||||
if (isVideo) {
|
||||
// Local video files don't need transcoding and support native seeking
|
||||
streamUrl = localUrl;
|
||||
// Served over the loopback media server rather than the asset
|
||||
// protocol: the asset protocol answers a range-less request with the
|
||||
// entire file, so a downloaded film never finished loading. Rust mints
|
||||
// the URL (it holds the port and the per-session token) and states the
|
||||
// transport with it.
|
||||
//
|
||||
// A downloaded file is a direct play over a local transport, and Rust
|
||||
// says so rather than this page assuming it.
|
||||
// TRACES: UR-071 | DR-137, DR-224
|
||||
selection = await commands.mediaLocalSelection(fullPath);
|
||||
videoNeedsTranscoding = false;
|
||||
// Use explicit startPosition, or fall back to retrieved progress from database
|
||||
const effectivePosition = startPosition ?? retrievedProgressSeconds ?? 0;
|
||||
@@ -355,7 +363,19 @@
|
||||
const liveInfo = await repo.openLiveStream(id);
|
||||
log.debug("loadAndPlay: Live stream URL:", liveInfo.streamUrl);
|
||||
mediaSourceId = liveInfo.mediaSourceId;
|
||||
streamUrl = liveInfo.streamUrl;
|
||||
selection = {
|
||||
url: liveInfo.streamUrl,
|
||||
// Rust's verdict, not a guess from the URL.
|
||||
transport: liveInfo.transport,
|
||||
playbackKind: { type: "transcode" },
|
||||
rendition: null,
|
||||
// A live channel has no ladder to offer: there is no source file to
|
||||
// measure and no rendition to re-negotiate against.
|
||||
available: [],
|
||||
mediaSourceId: liveInfo.mediaSourceId,
|
||||
playSessionId: liveInfo.playSessionId,
|
||||
needsTranscoding: true,
|
||||
};
|
||||
videoNeedsTranscoding = true;
|
||||
videoInitialPosition = 0;
|
||||
isPlaying = true;
|
||||
@@ -363,46 +383,46 @@
|
||||
return;
|
||||
}
|
||||
|
||||
log.debug("loadAndPlay: Getting playback info");
|
||||
const playbackInfo = await repo.getPlaybackInfo(id);
|
||||
log.debug("loadAndPlay: Got playback info, mediaSourceId:", playbackInfo.mediaSourceId);
|
||||
|
||||
if (isVideo) {
|
||||
// Playback API now detects HEVC/10-bit and returns transcoded URL when needed
|
||||
log.debug(
|
||||
"loadAndPlay: Using video stream, directPlay:",
|
||||
playbackInfo.directPlay,
|
||||
"needsTranscoding:",
|
||||
playbackInfo.needsTranscoding,
|
||||
);
|
||||
mediaSourceId = playbackInfo.mediaSourceId;
|
||||
|
||||
// Prefer a completed download over streaming. Audio has done this
|
||||
// since the queue is built; video previously always streamed, so a
|
||||
// downloaded film re-spent bandwidth already spent and would not play
|
||||
// at all offline. Rust returns null when nothing is downloaded or the
|
||||
// file has gone, so this falls back to the server on its own.
|
||||
// TRACES: UR-071 | DR-123
|
||||
// A downloaded file is served over the loopback media server, not the
|
||||
// asset protocol — see DR-137. The URL is minted up front because
|
||||
// resolveVideoSource stays pure/synchronous.
|
||||
// TRACES: UR-071 | DR-123, DR-137
|
||||
//
|
||||
// Checked *first* so the streaming path below negotiates exactly once:
|
||||
// asking for a `PlaybackInfo` and then a stream selection meant two
|
||||
// negotiations per load, and each one claims a transcode identity and
|
||||
// retires the previous — so the server started a job only to be told
|
||||
// to stop it a moment later. Observed in the log as a pair of
|
||||
// `[StreamSelection]` lines for one play.
|
||||
//
|
||||
// TRACES: UR-071 | DR-123, DR-137, DR-224
|
||||
const localPath = await commands.playerLocalMediaPath(id);
|
||||
const localUrl = localPath ? await commands.mediaLocalUrl(localPath) : null;
|
||||
const source = resolveVideoSource({
|
||||
localPath,
|
||||
remoteUrl: playbackInfo.streamUrl,
|
||||
remoteNeedsTranscoding: playbackInfo.needsTranscoding,
|
||||
toAssetUrl: () => localUrl ?? "",
|
||||
});
|
||||
|
||||
streamUrl = source.url;
|
||||
videoNeedsTranscoding = source.needsTranscoding;
|
||||
log.debug(
|
||||
source.isLocal
|
||||
? "loadAndPlay: Playing downloaded file from disk"
|
||||
: `loadAndPlay: Using stream URL: ${streamUrl}`,
|
||||
);
|
||||
if (localPath) {
|
||||
// A downloaded file is a direct play over a local transport, served
|
||||
// by the loopback media server rather than the asset protocol
|
||||
// (DR-137). Its media-source id still comes from the server, since
|
||||
// that is what subtitle URLs are keyed by.
|
||||
selection = await commands.mediaLocalSelection(localPath);
|
||||
videoNeedsTranscoding = false;
|
||||
mediaSourceId = (await repo.getPlaybackInfo(id)).mediaSourceId;
|
||||
log.debug("loadAndPlay: Playing downloaded file from disk");
|
||||
} else {
|
||||
// Rust negotiates direct play vs direct stream vs transcode against
|
||||
// the device profile and the ceiling in force, and returns the
|
||||
// transport and the media-source id with it. This page no longer
|
||||
// decides — or separately asks for — any of that.
|
||||
// TRACES: UR-070, UR-079 | DR-224, DR-226, DR-227
|
||||
selection = await repo.getStreamSelection(id, null, null);
|
||||
mediaSourceId = selection.mediaSourceId;
|
||||
// Rust's own verdict — "which kinds count as transcoding" is a
|
||||
// domain rule, and a direct *stream* is a remux that does not.
|
||||
videoNeedsTranscoding = selection.needsTranscoding;
|
||||
log.debug(
|
||||
`loadAndPlay: ${selection.playbackKind.type} over ${selection.transport.type}`,
|
||||
);
|
||||
}
|
||||
|
||||
// Set initial position for the video player to seek to after load.
|
||||
// Use explicit startPosition, or fall back to retrieved progress.
|
||||
@@ -847,10 +867,10 @@
|
||||
class="w-8 h-8 border-2 border-[var(--color-jellyfin)] border-t-transparent rounded-full animate-spin"
|
||||
></div>
|
||||
</div>
|
||||
{:else if surface === "video" && streamUrl}
|
||||
{:else if surface === "video" && selection}
|
||||
<VideoPlayer
|
||||
media={currentMedia}
|
||||
{streamUrl}
|
||||
{selection}
|
||||
mediaSourceId={mediaSourceId ?? undefined}
|
||||
initialPosition={videoInitialPosition}
|
||||
needsTranscoding={videoNeedsTranscoding}
|
||||
|
||||
Reference in New Issue
Block a user