dtourolle/jellytau
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases 38 Wiki Activity

feat(playback): let Rust decide what stream to play, and say so

Browse Source 
Playing a video meant asking the server to re-encode it, always. That
decision was made nowhere and written down nowhere, so whoever needed it
re-derived it downstream — the player worked out whether it had been handed
a playlist by looking for ".m3u8" in the URL, in two places. A viewer paid
for a transcode of a file their device could have played untouched, and the
app could not tell them which it was.

One negotiation now produces one self-describing StreamSelection — direct
play, remux or transcode; over a playlist, a plain HTTP file, or a local one
— and every renderer consumes that same answer.

Measured against the development server (Jellyfin 10.11.5), 400 items
sampled for codec mix and 40 put through a real PlaybackInfo negotiation
per profile:

  Linux / WebKitGTK (h264 only, 2ch)          3/40 —  7% direct play
  Android / ExoPlayer (hevc, ac3/eac3, 6ch)  34/40 — 85% direct play

The library is ~80% hevc, which is why the two diverge so hard. The payoff
is overwhelmingly Android, where 85% of plays were starting a transcode
nobody needed. Linux stays near 7% until libmpv decodes the picture — the
h264-only profile is a WebKitGTK constraint, not a JellyTau choice.

DR-219  StreamSelection: url + tagged Transport (hls/progressive/localFile)
        + PlaybackKind (directPlay/directStream/transcode) + the negotiated
        rendition + this source's ladder + a needs_transcoding flag derived
        in Rust so the rule is answered once. Both enums are serde-tagged
        so the frontend matches a discriminant, not a substring. The paths
        that never negotiate get the same shape from Rust rather than
        assembling one — media_local_selection for a downloaded file,
        LiveStreamInfo.transport for a live channel — so there is no second
        place where a transport is decided.

DR-220  The ceiling becomes two levels: a durable device default (Settings,
        persisted) and a per-playback override the in-player picker sets.
        The picker had called itself a "this film, this connection" control
        since it was written but wrote the process-wide default, so dropping
        one awkward film to 2 Mbps silently capped every video played
        afterwards for the rest of the process, with Settings still showing
        the old value. The override is cleared whenever playback moves to a
        new item, which stops it surviving into an autoplayed next episode.
        effective_streaming_quality() is the single resolution point.

DR-221  The quality picker is filled from what this media source can offer.
        Rust marks a rung exceeds_source when its ceiling is at or above the
        source's own bitrate — such a rung is another way to spell Original
        — and the frontend does not draw those. Original is never marked; a
        source whose bitrate the server does not report marks nothing, which
        keeps every rung offered.

DR-222  Direct play and direct stream are negotiated, with two client-side
        overrides on top because the server's answer is right about the file
        and wrong about what this app will do with it: undecodable audio
        (Jellyfin 10.11.5 honours a DirectPlayProfile's container and video
        codec but ignores its audio codec, so it offers direct play for an
        E-AC-3 track the webview renders in silence) and a viewer-pinned
        audio track the file does not default to. A direct stream is a remux
        and is deliberately not counted as transcoding.

DR-223  Dropped on measurement, not deferred. A master playlist from this
        server carries exactly one EXT-X-STREAM-INF: Jellyfin builds it from
        the single rendition the request asked for rather than publishing a
        ladder. So there is no adaptation for hls.js to be preserving and
        none mpv would lose — the claim that there was, in
        playback-backend-unification.md, does not hold. Recorded rather than
        deleted because it is a measurement: a server that does publish a
        ladder would change the answer.

DR-224  Every backend consumes the same selection. The queue item carries
        the transport, so player_seek_video picks its seek strategy from the
        backend's decision instead of the last stream_url.contains(".m3u8")
        in the codebase. Items queued by a path that never negotiated carry
        None and fall back to needs_transcoding, which is exact rather than
        a guess because every transcode this app requests is HLS (DR-140).

The frontend loader decision moves to streamTransport.ts so it can be
tested: the two cases that pin it are the ones that failed against the old
implementation — a progressive stream whose URL contains ".m3u8" must not
get an HLS loader, and an HLS stream whose URL contains none must.

Also verified the URL the direct-play branch builds actually serves playable
bytes: 206, video/mp4, valid ISO-BMFF, and a mid-file range works, so
seeking a direct play works.

The spec is folded into docs/architecture/{01,02,03} and deleted, per the
rule that docs/specs holds only work that has not shipped. DR-121 leaves
read-through-media-cache.md with a pointer; that spec keeps its capture half.

Not verified: real playback on a device. Direct play changes what actually
gets played, and neither fixtures nor curl prove the WebKitGTK and ExoPlayer
paths render it.
This commit is contained in:
dtourolle
2026-08-22 13:45:03 +02:00
parent 5fede123e7
commit 109700b949
45 changed files with 9744 additions and 6581 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/lib/api/bindings.ts
+253 -12
View File
@@ -120,7 +120,7 @@ async playerSeek(position: number) : Promise<PlayerStatus> {
*
* This command analyzes the current video stream and automatically chooses
* the best seeking strategy:
* - HLS streams (.m3u8): Use native seeking
* - HLS streams: Use native seeking
* - Direct play streams: Use native seeking
* - Transcoded non-HLS: Request new stream URL from server starting at seek position
*
@@ -263,13 +263,17 @@ async playerGetStreamingQualities() : Promise<([StreamingQuality, string, string
* two-sided split: HTML5 gets the URL back and reloads its own element, while a
* native backend is reloaded here.
*
* The change applies to this playback *and* to everything started afterwards
* (it sets the process-wide ceiling), but it is deliberately **not** persisted:
* the in-player picker is a "this film, this connection" control, and the
* durable default belongs to Settings. `player_set_video_settings` is the one
* that writes to the database.
* The change applies to **this playback only**. The in-player picker is a
* "this film, this connection" control and its doc has always said so, but it
* used to be implemented by writing the process-wide ceiling — so choosing
* 2 Mbps to get one awkward film moving silently capped every video played
* afterwards for the rest of the process, with the Settings screen still
* showing the old value and nothing in the UI admitting the change. It now
* sets a per-playback override that the next item clears; the durable default
* belongs to Settings, and `player_set_video_settings` is the one that writes
* to the database.
*
* TRACES: UR-074 | DR-162
* TRACES: UR-074, UR-079 | DR-162, DR-225
*/
async playerSetStreamQuality(repositoryHandle: string, quality: StreamingQuality, useHtml5: boolean, currentPosition: number | null, mediaSourceId: string | null, audioStreamIndex: number | null) : Promise<StreamQualityResponse> {
return await TAURI_INVOKE("player_set_stream_quality", { repositoryHandle, quality, useHtml5, currentPosition, mediaSourceId, audioStreamIndex });
@@ -1033,6 +1037,22 @@ async markDownloadFailed(downloadId: number, errorMessage: string) : Promise<nul
async mediaLocalUrl(path: string) : Promise<string> {
return await TAURI_INVOKE("media_local_url", { path });
},
/**
* The stream selection for a downloaded file.
*
* The local-playback counterpart to `repository_get_stream_selection`. A file
* on disk needs no negotiation — it is a direct play over a local transport,
* with no quality ladder, because nothing about it can be re-negotiated — but
* the *frontend must not be the one to say so*. It gets the same
* [`StreamSelection`] shape as a streamed source so the player has one contract
* to consume rather than two, and so no caller has to infer a transport from a
* loopback URL.
*
* TRACES: UR-071, UR-079 | DR-224
*/
async mediaLocalSelection(path: string) : Promise<StreamSelection> {
return await TAURI_INVOKE("media_local_selection", { path });
},
/**
* Start downloading a file immediately
* This command actually downloads the file using the worker
@@ -1618,6 +1638,24 @@ async repositoryGetPlaybackInfo(handle: string, itemId: string) : Promise<Playba
async repositoryGetVideoStreamUrl(handle: string, itemId: string, mediaSourceId: string | null, audioStreamIndex: number | null) : Promise<string> {
return await TAURI_INVOKE("repository_get_video_stream_url", { handle, itemId, mediaSourceId, audioStreamIndex });
},
/**
* Decide what stream to play for a video, and describe it.
*
* Replaces `repository_get_video_stream_url` for playback. The returned
* [`StreamSelection`] carries the transport explicitly, so the frontend picks
* its loader from a tagged enum instead of testing the URL for `.m3u8`; and it
* carries the quality ladder as it applies to *this* source, so the picker can
* stop offering rungs that produce the same bytes as Original.
*
* No start-position parameter, for the same reason as the URL builder: a
* position on an HLS playlist is copied onto every segment URI and the server
* rejects each with `400` (DR-181). Callers resume by seeking after load.
*
* TRACES: UR-070, UR-079 | DR-224, DR-226, DR-227 | UT-212
*/
async repositoryGetStreamSelection(handle: string, itemId: string, mediaSourceId: string | null, audioStreamIndex: number | null) : Promise<StreamSelection> {
return await TAURI_INVOKE("repository_get_stream_selection", { handle, itemId, mediaSourceId, audioStreamIndex });
},
/**
* Get audio stream URL for a track
*/
@@ -1965,7 +2003,7 @@ export type AudioTrackSwitchResponse =
/**
* HTML5 needs to reload stream with new audio track
*/
{ strategy: "reloadStream"; new_url: string; position: number }
{ strategy: "reloadStream"; selection: StreamSelection; position: number }
/**
* Authentication result
*/
@@ -2329,7 +2367,18 @@ excludedItemIds?: string[] }
* streamed; the server returns a transcoding URL (already absolute) plus a
* `live_stream_id` that can later be used to close the stream.
*/
export type LiveStreamInfo = { streamUrl: string; playSessionId: string | null; liveStreamId: string | null; mediaSourceId: string | null }
export type LiveStreamInfo = { streamUrl: string; playSessionId: string | null; liveStreamId: string | null; mediaSourceId: string | null;
/**
* How to open `stream_url`.
*
* A live channel is always an HLS transcode — the server has to repackage a
* broadcast mux into something a browser can play, and there is no static
* file to direct-play. Saying so here means the player page never has to
* work it out from the URL, which is the whole of DR-224.
*
* TRACES: UR-079 | DR-224
*/
transport: Transport }
/**
* An LMS multi-room sync group, as returned by JellyLMS `/JellyLms/SyncGroups`.
*
@@ -2568,6 +2617,18 @@ videoCodec: string;
* Whether the video requires server-side transcoding
*/
needsTranscoding: boolean;
/**
* How this item's stream is fetched, as the backend decided it.
*
* Carried on the queue item so a later seek/reload does not have to guess.
* `None` for items queued by a path that never negotiated (audio tracks,
* direct URLs) and for anything queued before this field existed, where the
* caller falls back to `needs_transcoding` — every transcode this app
* requests is HLS (DR-140), so that fallback is exact rather than a guess.
*
* TRACES: UR-003, UR-004, UR-079 | DR-224, DR-229
*/
transport?: Transport | null;
/**
* Optional now-playing metadata. Used by the background-audio handoff so the
* lockscreen/miniplayer show the item (title/subtitle/artwork). Defaulted so
@@ -2680,6 +2741,32 @@ supportsNativeVideo: boolean }
* Playback information
*/
export type PlaybackInfo = { mediaSourceId: string; playSessionId: string; streamUrl: string; directPlay: boolean; needsTranscoding: boolean }
/**
* What the server is doing to the source to produce this stream.
*
* Distinct from [`Transport`] because the two are genuinely independent: a
* direct-streamed remux and a transcode can both arrive over HLS, and a direct
* play can arrive progressively or as a local file. Keeping them apart is what
* lets the UI say "this is not costing the server anything" without inferring
* it from a URL shape.
*
* TRACES: UR-079 | DR-227
*/
export type PlaybackKind =
/**
* The source file is served untouched. No server CPU, no quality loss.
*/
{ type: "directPlay" } |
/**
* The container is repackaged but the codecs are copied — cheap, and
* visually identical to the source.
*/
{ type: "directStream" } |
/**
* The server is re-encoding. The only case where a bitrate ceiling can
* actually be honoured, and the only one that costs the server real work.
*/
{ type: "transcode" }
/**
* Playback mode - local device, remote session, or idle
*/
@@ -2779,6 +2866,18 @@ videoCodec?: string | null;
* Whether the video requires server-side transcoding
*/
needsTranscoding?: boolean;
/**
* How this item's stream is fetched, as the backend decided it.
*
* Carried on the queue item so a later seek/reload does not have to guess.
* `None` for items queued by a path that never negotiated (audio tracks,
* direct URLs) and for anything queued before this field existed, where the
* caller falls back to `needs_transcoding` — every transcode this app
* requests is HLS (DR-140), so that fallback is exact rather than a guess.
*
* TRACES: UR-003, UR-004, UR-079 | DR-224, DR-229
*/
transport?: Transport | null;
/**
* Video width in pixels
*/
@@ -3061,6 +3160,38 @@ alreadyDownloaded: number;
* Number of tracks skipped (no jellyfin ID or other reasons)
*/
skipped: number }
/**
* One rung of the quality picker, as it applies to *this* media source.
*
* The picker used to be filled from the fixed [`StreamingQuality::ALL`] ladder,
* which meant offering "20 Mbps" for a 1.1 Mbps podcast — eight rungs, six of
* them indistinguishable from Original. `exceeds_source` is what lets the
* frontend render that honestly without knowing anything about bitrates.
*
* TRACES: UR-070, UR-079 | DR-226, DR-121
*/
export type QualityOption = { quality: StreamingQuality;
/**
* Human label ("8 Mbps"). Lives in Rust beside the number it describes.
*/
label: string;
/**
* Secondary line ("1080p").
*/
detail: string;
/**
* True when this rung's ceiling is at or above what the source itself
* carries, so selecting it yields the same stream as `Original`.
*
* The frontend renders these differently (or hides them); it does not
* decide which they are.
*/
exceedsSource: boolean;
/**
* The source's own bitrate, when the server reported one. Presentation
* only — the picker shows "Original (6.7 Mbps)" rather than a bare word.
*/
sourceBitrate: number | null }
/**
* Response for queue queries
*/
@@ -3069,6 +3200,36 @@ export type QueueStatus = { items: PlayerMediaItem[]; currentIndex: number | nul
* Remote session status for UI updates
*/
export type RemoteSessionStatus = { position: number; duration: number | null; isPlaying: boolean; nowPlayingItem: NowPlayingItem | null }
/**
* The rendition actually negotiated — what the viewer is receiving right now.
*
* `None` on a [`StreamSelection`] when the source is being direct-played as-is:
* there is no *chosen* rendition in that case, only the file itself, and
* reporting the ceiling that happened to be set would misdescribe it.
*
* TRACES: UR-079 | DR-224, DR-225
*/
export type Rendition = {
/**
* The rung of the ladder this stream was built against.
*/
quality: StreamingQuality;
/**
* Total bits per second the stream may use, when a ceiling applies.
*/
maxBitrate: number | null;
/**
* Resolution ceiling, when one applies. `None` preserves the source's.
*/
maxHeight: number | null;
/**
* Video codec the server was asked to produce.
*/
videoCodec: string | null;
/**
* Audio codec the server was asked to produce.
*/
audioCodec: string | null }
/**
* Repeat mode for the queue
*
@@ -3186,9 +3347,59 @@ export type StreamQualityResponse =
*/
{ strategy: "native"; position: number } |
/**
* HTML5 must reload its element with this URL.
* HTML5 must reload its element with this selection.
*/
{ strategy: "reloadStream"; new_url: string; position: number }
{ strategy: "reloadStream"; selection: StreamSelection; position: number }
/**
* Everything a player backend needs to open a stream, and everything the UI
* needs to describe it.
*
* Replaces the bare `String` URL that `get_video_stream_url` used to return.
*
* TRACES: UR-079 | DR-224, DR-226, DR-227
*/
export type StreamSelection = {
/**
* The URL (or loopback URL) to open.
*/
url: string;
/**
* How to fetch it. Replaces the `.m3u8` substring check.
*/
transport: Transport;
/**
* What the server is doing to the source to produce it.
*/
playbackKind: PlaybackKind;
/**
* The negotiated rendition; `None` when direct-playing the source as-is.
*/
rendition: Rendition | null;
/**
* What this media source can offer, for the quality picker (DR-226).
*/
available: QualityOption[];
/**
* The media source this selection is for, so a later re-open (quality
* change, audio-track switch, transcoded seek) targets the same one.
*/
mediaSourceId: string | null;
/**
* The transcode identity the server keyed this job by, when there is one.
*/
playSessionId: string | null;
/**
* Whether the server is spending encoder time on this stream.
*
* Derived from [`playback_kind`](Self::playback_kind) rather than left for
* the frontend to compute: "which kinds count as transcoding" is a domain
* rule, and a direct *stream* is a remux that must not be counted. The
* queue's long-standing `needs_transcoding` flag and the seek strategy both
* read this, so there is one answer rather than three.
*
* TRACES: UR-079 | DR-224, DR-227
*/
needsTranscoding: boolean }
/**
* A ceiling on how much bandwidth a *video* stream may consume.
*
@@ -3269,6 +3480,36 @@ itemName: string | null }
* Statistics about the thumbnail cache
*/
export type ThumbnailCacheStats = { totalSizeBytes: number; itemCount: number; limitBytes: number }
/**
* How the bytes of a chosen stream are fetched.
*
* This field exists to delete a substring search. The frontend previously
* decided which loader to attach by testing `url.contains(".m3u8")`, which is a
* domain fact reconstructed in the presentation layer — the same class of leak
* as the item-type taxonomy that `check:boundary` guards, and one that breaks
* silently the moment a server serves a playlist from a path that does not end
* in `.m3u8`, or serves a progressive file from one that does.
*
* Tagged (`{"type":"hls"}`) rather than a bare string so the frontend matches a
* discriminant instead of comparing text.
*
* TRACES: UR-079 | DR-224
*/
export type Transport =
/**
* An HLS playlist. The webview attaches hls.js (or Safari's native loader);
* ExoPlayer uses its HLS media source.
*/
{ type: "hls" } |
/**
* A single progressive HTTP resource, seekable by byte range.
*/
{ type: "progressive" } |
/**
* A file already on disk — a completed download, or the loopback media
* server standing in front of one.
*/
{ type: "localFile" }
/**
* User information
*/
@@ -3316,7 +3557,7 @@ export type VideoSeekResponse =
/**
* Reload stream from new position (transcoded non-HLS)
*/
{ strategy: "reloadStream"; new_url: string; seek_offset: number }
{ strategy: "reloadStream"; selection: StreamSelection; seek_offset: number }
/**
* Video playback settings
*/
src/lib/api/repository-client.ts
+29 -1
View File
@@ -3,7 +3,7 @@
// NO direct HTTP calls - everything routes through Rust backend
import { commands } from "./bindings";
import type { JRayActor, DownloadDiskUsage, SearchScope } from "./bindings";
import type { DownloadDiskUsage, JRayActor, SearchScope, StreamSelection } from "./bindings";
import type { QualityPreset } from "./quality-presets";
import type {
Library,
@@ -247,6 +247,34 @@ export class RepositoryClient {
);
}
/**
* Decide what stream to play, and describe it.
*
* The playback counterpart to {@link getVideoStreamUrl}, which returns only a
* URL and therefore forces its caller to work out the rest. This returns the
* transport (so the player picks a loader from a tagged enum rather than by
* searching the URL for `.m3u8`), the playback kind (direct play / direct
* stream / transcode), and the quality ladder as it applies to this source.
*
* No position parameter, for the same reason as {@link getVideoStreamUrl}: a
* start position on an HLS playlist makes Jellyfin reject every segment behind
* it with `400` (DR-181). Resume by seeking once loaded.
*
* TRACES: UR-070, UR-079 | DR-224, DR-226, DR-227 | UT-212
*/
async getStreamSelection(
itemId: string,
mediaSourceId?: string | null,
audioStreamIndex?: number | null,
): Promise<StreamSelection> {
return commands.repositoryGetStreamSelection(
this.ensureHandle(),
itemId,
mediaSourceId ?? null,
audioStreamIndex ?? null,
);
}
/**
* Audio-only stream URL for a video item, for the background-audio handoff.
* The server extracts just the audio track — no video is decoded on-device.
src/lib/components/player/VideoPlayer.nativeReveal.test.ts
+20 -3
View File
@@ -123,6 +123,23 @@ import VideoPlayer from "./VideoPlayer.svelte";
import { player } from "$lib/stores/player";
import type { MediaItem } from "$lib/api/types";
/**
* A `StreamSelection` for tests that only care about the URL. Transcoded HLS is
* what these paths exercised before the contract carried a transport.
*/
function testSelection(url: string, transport: "hls" | "progressive" | "localFile" = "hls") {
return {
url,
transport: { type: transport },
playbackKind: { type: transport === "hls" ? "transcode" : "directPlay" },
rendition: null,
available: [],
mediaSourceId: null,
playSessionId: null,
needsTranscoding: transport === "hls",
} as import("$lib/api/bindings").StreamSelection;
}
function makeEpisode(): MediaItem {
return {
id: "ep1",
@@ -136,7 +153,7 @@ async function mountNativePlayer() {
const utils = render(VideoPlayer, {
props: {
media: makeEpisode(),
streamUrl: "http://server/videos/ep1/master.m3u8",
selection: testSelection("http://server/videos/ep1/master.m3u8"),
mediaSourceId: "src-1",
needsTranscoding: false,
onClose: vi.fn(),
@@ -261,7 +278,7 @@ describe("VideoPlayer native path reveals the video (DR-172)", () => {
const utils = render(VideoPlayer, {
props: {
media: makeEpisode(),
streamUrl: "http://server/videos/ep1/master.m3u8",
selection: testSelection("http://server/videos/ep1/master.m3u8"),
mediaSourceId: "src-1",
needsTranscoding: false,
onClose: vi.fn(),
@@ -303,7 +320,7 @@ describe("VideoPlayer native path reveals the video (DR-172)", () => {
const utils = render(VideoPlayer, {
props: {
media: makeEpisode(),
streamUrl: "http://server/videos/ep1/master.m3u8",
selection: testSelection("http://server/videos/ep1/master.m3u8"),
mediaSourceId: "src-1",
needsTranscoding: false,
onClose: vi.fn(),
src/lib/components/player/VideoPlayer.scrubRegression.test.ts
+18 -1
View File
@@ -118,6 +118,23 @@ import VideoPlayer from "./VideoPlayer.svelte";
import { sleepTimer, sleepTimerExpiredSignal } from "$lib/stores/sleepTimer";
import type { MediaItem } from "$lib/api/types";
/**
* A `StreamSelection` for tests that only care about the URL. Transcoded HLS is
* what these paths exercised before the contract carried a transport.
*/
function testSelection(url: string, transport: "hls" | "progressive" | "localFile" = "hls") {
return {
url,
transport: { type: transport },
playbackKind: { type: transport === "hls" ? "transcode" : "directPlay" },
rendition: null,
available: [],
mediaSourceId: null,
playSessionId: null,
needsTranscoding: transport === "hls",
} as import("$lib/api/bindings").StreamSelection;
}
function makeEpisode(): MediaItem {
return {
id: "ep1",
@@ -139,7 +156,7 @@ async function mountAndroidPlayer() {
const utils = render(VideoPlayer, {
props: {
media: makeEpisode(),
streamUrl: "http://server/videos/ep1/master.m3u8",
selection: testSelection("http://server/videos/ep1/master.m3u8"),
mediaSourceId: "src-1",
needsTranscoding: false,
onClose: vi.fn(),
src/lib/components/player/VideoPlayer.svelte
+172 -56
View File
@@ -4,7 +4,12 @@
import { get } from "svelte/store";
import { goto } from "$app/navigation";
import { commands } from "$lib/api/bindings";
import type { JRayActor, StreamingQuality, BackgroundAction } from "$lib/api/bindings";
import type {
JRayActor,
StreamingQuality,
BackgroundAction,
StreamSelection,
} from "$lib/api/bindings";
import { listen } from "@tauri-apps/api/event";
import Hls from "hls.js";
import type { MediaItem } from "$lib/api/types";
@@ -77,12 +82,21 @@
type BackgroundAudioState,
} from "./backgroundAudioHandoff";
import { createLogger } from "$lib/utils/logger";
import { elementSrcFor, videoLoaderFor } from "$lib/player/streamTransport";
const log = createLogger("VideoPlayer");
interface Props {
media: MediaItem | null;
streamUrl: string;
/**
* What to play, as the backend decided it: URL, transport, playback kind and
* the quality ladder for this source. Replaces the bare `streamUrl` string,
* which forced this component to re-derive the transport by searching for
* `.m3u8`.
*
* TRACES: UR-079 | DR-224, DR-226
*/
selection: StreamSelection;
mediaSourceId?: string; // Media source ID for subtitle URLs
initialPosition?: number; // Position in seconds to seek to after load (for resume)
needsTranscoding?: boolean; // Whether content needs transcoding (HEVC/10-bit) - affects seeking behavior
@@ -103,7 +117,7 @@
let {
media,
streamUrl,
selection,
mediaSourceId,
initialPosition,
needsTranscoding = false,
@@ -179,7 +193,12 @@
// Capture only the initial streamUrl prop; later prop changes are applied via
// the $effect below (untrack keeps this a one-time snapshot, matching
// reportMediaId above and silencing state_referenced_locally).
let currentStreamUrl = $state(untrack(() => streamUrl));
// The selection currently loaded. Starts from the prop and is replaced
// wholesale by a reload (quality change, audio-track switch, transcoded seek)
// so transport and URL can never disagree.
// TRACES: UR-079 | DR-224
let currentSelection = $state<StreamSelection>(untrack(() => selection));
const currentStreamUrl = $derived(currentSelection.url);
let hasReportedStart = $state(false);
let progressInterval: ReturnType<typeof setInterval> | null = null;
let isMediaReady = $state(false); // Track if media is ready to play (implements Loading state from DR-001)
@@ -250,14 +269,31 @@
}
}
/**
* A selection identical to the one loaded, but pointing at a different URL.
*
* Used by the paths that swap the stream without re-negotiating — the
* background-audio handoff and its return. Each states the transport it is
* moving to rather than letting it be inferred, which is the whole point of
* DR-224: the audio handoff really is a progressive mp3, and the rebuilt
* video stream really is an HLS transcode, and neither is knowable from the
* URL text.
*
* TRACES: UR-040, UR-079 | DR-224
*/
function selectionAt(url: string, transport: StreamSelection["transport"]): StreamSelection {
// A re-opened stream is a new transcode job; the old session id is stale.
return { ...currentSelection, url, transport, playSessionId: null };
}
const adapterBridge: Html5ElementBridge = {
getElement: () => videoElement,
getSeekOffset: () => seekOffset,
setSeekOffset: (o) => {
seekOffset = o;
},
setStreamUrl: (u) => {
currentStreamUrl = u;
setStreamSelection: (sel) => {
currentSelection = sel;
},
destroyHls: tearDownHls,
getMediaSourceId: () => mediaSourceId ?? null,
@@ -275,9 +311,46 @@
// Rust — the frontend never encodes what a step means.
// TRACES: UR-074 | DR-162
let showQualityMenu = $state(false);
let streamingQualities = $state<[StreamingQuality, string, string][]>([]);
let selectedQuality = $state<StreamingQuality>("original");
let changingQuality = $state(false);
/**
* The device's durable default, shown when the stream is a direct play and so
* has no rendition of its own to report. Read once from Settings.
*/
let defaultQuality = $state<StreamingQuality>("original");
/**
* The rungs to offer for the stream that is playing, straight from the
* backend (DR-226). Rungs whose ceiling is at or above the source bitrate are
* dropped: they produce the same bytes as Original, so listing five of them is
* five ways to spell one choice. Rust decides which those are — this only
* decides not to draw them.
*
* `Original` is always kept; it is the source, never redundant with it.
*
* TRACES: UR-070, UR-079 | DR-226, DR-121
*/
const qualityOptions = $derived(
currentSelection.available.filter((o) => !o.exceedsSource || o.quality === "original"),
);
/**
* The rung in force. A transcode reports the rendition it was built against;
* a direct play has none, because it *is* the source — so it reads as
* Original rather than as whatever ceiling happens to be set.
*/
const selectedQuality = $derived<StreamingQuality>(
currentSelection.rendition?.quality ??
(currentSelection.playbackKind.type === "transcode" ? defaultQuality : "original"),
);
/** Human line for what the server is doing with this stream. */
const playbackKindLabel = $derived(
currentSelection.playbackKind.type === "directPlay"
? "Direct play — the original file"
: currentSelection.playbackKind.type === "directStream"
? "Direct stream — repackaged, not re-encoded"
: "Transcoding on the server",
);
// Track duration from video element (for when media item doesn't have runTimeTicks)
let videoDuration = $state(0);
@@ -450,9 +523,9 @@
// Update stream URL when prop changes (from parent component, not from internal seeks)
$effect(() => {
// Only reset when the streamUrl prop actually changes from parent
if (streamUrl !== lastStreamUrlProp) {
lastStreamUrlProp = streamUrl;
currentStreamUrl = streamUrl;
if (selection.url !== lastStreamUrlProp) {
lastStreamUrlProp = selection.url;
currentSelection = selection;
seekOffset = 0;
isMediaReady = false; // Reset to loading state when stream URL changes
hasPerformedInitialSeek = false; // Reset so new video can seek to initial position
@@ -567,9 +640,14 @@
return;
}
const isHlsStream = currentStreamUrl.includes(".m3u8");
// The loader comes from the backend's tagged transport, never from the URL.
// TRACES: UR-079 | DR-224 | UT-213
const loader = videoLoaderFor(currentSelection, {
hlsJsSupported: Hls.isSupported(),
nativeHlsSupported: !!videoElement.canPlayType("application/vnd.apple.mpegurl"),
});
if (isHlsStream && Hls.isSupported()) {
if (loader === "hlsjs") {
// Clean up existing HLS instance if any - CRITICAL for preventing dual audio
if (hls) {
log.debug("Cleaning up existing HLS instance");
@@ -724,13 +802,13 @@
videoElement.pause();
}
};
} else if (isHlsStream && videoElement.canPlayType("application/vnd.apple.mpegurl")) {
// Native HLS support (Safari)
} else if (loader === "nativeHls") {
// The element parses the playlist itself (Safari/WebKit).
log.debug("Using native HLS support");
videoElement.src = currentStreamUrl;
} else {
// Not an HLS stream, use regular video element
log.debug("Using regular video element for non-HLS stream");
// Progressive or local: the element loads the URL directly.
log.debug("Using regular video element", currentSelection.transport.type);
}
});
@@ -801,21 +879,25 @@
});
});
// Populate the quality menu. Deliberately its own *synchronous* onMount that
// fires the load without awaiting it: an await inside the main onMount below
// flips the component into HTML5 mode and breaks native seeking, and nothing
// about playback waits on this list.
// The quality *ladder* now arrives with the stream selection (DR-226), so all
// this still needs is the device default, for the case where the stream is a
// direct play and has no rendition of its own.
//
// TRACES: UR-074 | DR-162
// Deliberately its own *synchronous* onMount that fires the load without
// awaiting it: an await inside the main onMount below flips the component into
// HTML5 mode and breaks native seeking, and nothing about playback waits on
// this value.
//
// TRACES: UR-074, UR-079 | DR-162, DR-226
onMount(() => {
Promise.all([commands.playerGetStreamingQualities(), commands.playerGetVideoSettings()])
.then(([qualities, settings]) => {
streamingQualities = qualities;
commands
.playerGetVideoSettings()
.then((settings) => {
// Optional on the wire (serde default) — absent means uncapped.
selectedQuality = settings.streamingQuality ?? "original";
defaultQuality = settings.streamingQuality ?? "original";
})
.catch((err) => {
log.warn("Failed to load streaming qualities:", err);
log.warn("Failed to load the default streaming quality:", err);
});
});
@@ -878,6 +960,9 @@
id: media.id,
videoCodec: needsTranscoding ? "hevc" : "h264",
needsTranscoding: needsTranscoding,
// Carry the negotiated transport onto the queue item so a later seek
// reads it instead of falling back. TRACES: UR-079 | DR-229
transport: currentSelection.transport,
// Order matters: player_set_subtitle_track(n) is a position in this
// array. Previously this array was built and then dropped, so
// ExoPlayer got a MediaItem with no subtitles at all.
@@ -944,7 +1029,9 @@
const host = createRustReportHost(media.id, {
onEnded: () => notifyEnded(),
onStreamUrlChanged: (u) => {
currentStreamUrl = u;
// Rust re-opened the same stream (a transcoded seek): the
// transport is unchanged, only the job behind it.
currentSelection = selectionAt(u, currentSelection.transport);
},
});
playerAdapter = createAdapter({
@@ -1885,8 +1972,9 @@
pendingForegroundPlay = plan.shouldPlay;
// Determine the target URL + how the element/offset should be positioned.
let targetUrl: string;
// Determine the target stream + how the element/offset should be
// positioned.
let targetSelection: StreamSelection;
if (needsTranscoding && onSeek) {
// Transcoded HLS is rebuilt rather than seeked in place, but the rebuilt
// stream starts at the BEGINNING of the item, not at `pos`: a start
@@ -1897,13 +1985,16 @@
// that really did start there; leaving it would now display `pos` while
// playing the opening titles.
// TRACES: UR-040, UR-004 | DR-181
targetUrl = await onSeek(pos, selectedAudioTrackIndex ?? undefined);
// Every transcode this app requests is HLS (DR-140).
targetSelection = selectionAt(await onSeek(pos, selectedAudioTrackIndex ?? undefined), {
type: "hls",
});
seekOffset = 0;
currentTime = pos;
pendingForegroundSeek = pos;
} else {
// Direct stream: reload the original URL and seek the element to pos.
targetUrl = streamUrl;
// Direct stream: reload the original selection and seek to pos.
targetSelection = selection;
seekOffset = 0;
pendingForegroundSeek = pos;
}
@@ -1923,18 +2014,21 @@
// than re-fetched.
//
// TRACES: UR-040, UR-003 | DR-196
currentStreamUrl = targetUrl;
currentSelection = targetSelection;
await commands.playerPlayItem({
streamUrl: targetUrl,
streamUrl: targetSelection.url,
title: media.name,
id: media.id,
videoCodec: needsTranscoding ? "hevc" : "h264",
needsTranscoding,
// TRACES: UR-079 | DR-229
transport: targetSelection.transport,
subtitles: nativeSubtitleTracks(sentSubtitleTracks),
});
didStartNativePlayback = true;
await playerAdapter?.load(targetUrl, {
await playerAdapter?.load(targetSelection.url, {
mediaId: media.id,
selection: targetSelection,
mediaSourceId: mediaSourceId ?? null,
needsTranscoding,
initialPosition: plan.position,
@@ -1961,9 +2055,9 @@
// blank it first, then set it on the next microtask so Svelte sees a real
// transition. Without this, assigning the same value is a no-op and the
// player stays stuck on the loading spinner (HLS never re-initialises).
currentStreamUrl = "";
currentSelection = selectionAt("", targetSelection.transport);
await Promise.resolve();
currentStreamUrl = targetUrl;
currentSelection = targetSelection;
} catch (err) {
log.error("Background-audio return failed:", err);
}
@@ -2285,33 +2379,41 @@
*
* The backend owns everything about how that happens — it decides whether the
* caller reloads (HTML5) or it reloads the native backend itself — so this
* only supplies the position to resume at and reverts the selection if the
* switch fails.
* only supplies the position to resume at.
*
* TRACES: UR-074 | DR-162
* The change applies to this playback alone; the durable Settings default is
* untouched (DR-225). Nothing is optimistically assigned here: what the picker
* shows comes from the selection the backend hands back, because what you get
* is not always what you asked for — a ceiling above the source bitrate is the
* source, and claiming otherwise is the kind of lie the old picker told.
*
* TRACES: UR-074, UR-079 | DR-162, DR-225, DR-226
*/
async function selectQuality(quality: StreamingQuality) {
showQualityMenu = false;
if (quality === selectedQuality || changingQuality) return;
const previous = selectedQuality;
selectedQuality = quality;
changingQuality = true;
try {
stopTimeUpdates();
await playerController.setStreamQuality(
const negotiated = await playerController.setStreamQuality(
quality,
videoElement ? videoElement.currentTime + seekOffset : null,
mediaSourceId ?? null,
selectedAudioTrackIndex,
);
// The HTML5 path reloads through the adapter, which already set the new
// selection via the bridge. The native path reloads inside Rust and
// returns nothing, so record what was asked for as the ceiling in force.
if (!negotiated) {
defaultQuality = quality;
}
if (videoElement && !videoElement.paused) {
startTimeUpdates();
}
log.debug("Streaming quality changed:", quality);
} catch (err) {
log.error("Failed to change streaming quality:", err);
selectedQuality = previous;
} finally {
changingQuality = false;
}
@@ -2416,7 +2518,10 @@
<!-- HTML5 video for desktop/non-Android platforms -->
<video
bind:this={videoElement}
src={currentStreamUrl.includes(".m3u8") && Hls.isSupported() ? "" : currentStreamUrl}
src={elementSrcFor(currentSelection, {
hlsJsSupported: Hls.isSupported(),
nativeHlsSupported: true,
})}
crossorigin={videoCrossOrigin}
class={videoFitClass()}
class:invisible={!isMediaReady}
@@ -2756,8 +2861,11 @@
</div>
{/if}
<!-- Streaming quality (bandwidth ceiling). TRACES: UR-074 | DR-162 -->
{#if streamingQualities.length > 0}
<!--
Streaming quality (bandwidth ceiling), populated from what this media
source can actually offer. TRACES: UR-070, UR-074 | DR-162, DR-226
-->
{#if qualityOptions.length > 1}
<div class="relative">
<button
onclick={toggleQualityMenu}
@@ -2778,22 +2886,30 @@
class="absolute bottom-full right-0 mb-2 bg-black/90 backdrop-blur-sm rounded-lg shadow-xl min-w-[220px] max-h-[300px] overflow-y-auto"
>
<div class="p-2">
<div class="text-white text-sm font-semibold px-3 py-2 border-b border-white/20">
Quality
<div class="px-3 py-2 border-b border-white/20">
<div class="text-white text-sm font-semibold">Quality</div>
<!--
What the server is actually doing. Only knowable now that
the backend reports it. TRACES: UR-079 | DR-227
-->
<div class="text-xs text-gray-400 mt-0.5">{playbackKindLabel}</div>
</div>
{#each streamingQualities as [quality, label, detail]}
{#each qualityOptions as option (option.quality)}
<button
onclick={() => selectQuality(quality)}
onclick={() => selectQuality(option.quality)}
class="w-full text-left px-3 py-2 text-white hover:bg-white/10 rounded transition-colors flex items-center justify-between {selectedQuality ===
quality
option.quality
? 'bg-white/20'
: ''}"
>
<div class="flex flex-col">
<span class="text-sm">{label}</span>
<span class="text-xs text-gray-400">{detail}</span>
<span class="text-sm">{option.label}</span>
<span class="text-xs text-gray-400">
{option.detail}{#if option.quality === "original" && option.sourceBitrate}
&middot; {(option.sourceBitrate / 1_000_000).toFixed(1)} Mbps{/if}
</span>
</div>
{#if selectedQuality === quality}
{#if selectedQuality === option.quality}
<svg
class="w-4 h-4 text-[var(--color-jellyfin)]"
fill="currentColor"
src/lib/components/player/VideoPlayer.tapSurface.test.ts
+18 -1
View File
@@ -36,6 +36,23 @@ import { invoke } from "@tauri-apps/api/core";
import VideoPlayer from "./VideoPlayer.svelte";
import { SEEK_FORWARD_SECONDS } from "./tapGestures";
/**
* A `StreamSelection` for tests that only care about the URL. Transcoded HLS is
* what these paths exercised before the contract carried a transport.
*/
function testSelection(url: string, transport: "hls" | "progressive" | "localFile" = "hls") {
return {
url,
transport: { type: transport },
playbackKind: { type: transport === "hls" ? "transcode" : "directPlay" },
rendition: null,
available: [],
mediaSourceId: null,
playSessionId: null,
needsTranscoding: transport === "hls",
} as import("$lib/api/bindings").StreamSelection;
}
// --- Mocks: everything VideoPlayer reaches for that is not the tap surface. ---
const toggleSpy = vi.fn();
@@ -122,7 +139,7 @@ function touchAt(el: Element, x: number) {
function renderPlayer() {
return render(VideoPlayer, {
props: { media: MEDIA, streamUrl: "http://x/master.m3u8", onClose: vi.fn() },
props: { media: MEDIA, selection: testSelection("http://x/master.m3u8"), onClose: vi.fn() },
});
}
src/lib/components/player/VideoPlayer.touchScrub.test.ts
+18 -1
View File
@@ -116,6 +116,23 @@ import { tick } from "svelte";
import VideoPlayer from "./VideoPlayer.svelte";
import type { MediaItem } from "$lib/api/types";
/**
* A `StreamSelection` for tests that only care about the URL. Transcoded HLS is
* what these paths exercised before the contract carried a transport.
*/
function testSelection(url: string, transport: "hls" | "progressive" | "localFile" = "hls") {
return {
url,
transport: { type: transport },
playbackKind: { type: transport === "hls" ? "transcode" : "directPlay" },
rendition: null,
available: [],
mediaSourceId: null,
playSessionId: null,
needsTranscoding: transport === "hls",
} as import("$lib/api/bindings").StreamSelection;
}
function makeEpisode(): MediaItem {
return {
id: "ep1",
@@ -129,7 +146,7 @@ async function mountAndroidPlayer() {
const utils = render(VideoPlayer, {
props: {
media: makeEpisode(),
streamUrl: "http://server/videos/ep1/master.m3u8",
selection: testSelection("http://server/videos/ep1/master.m3u8"),
mediaSourceId: "src-1",
needsTranscoding: false,
onClose: vi.fn(),
src/lib/player/adapters/html5Adapter.test.ts
+26 -7
View File
@@ -10,6 +10,23 @@ import { describe, it, expect, vi, beforeEach } from "vitest";
import { Html5PlayerAdapter, type Html5ElementBridge } from "./html5Adapter";
import type { AdapterHost } from "./types";
/**
* A `StreamSelection` for tests that only care about the URL. Transcoded HLS is
* what these paths exercised before the contract carried a transport.
*/
function testSelection(url: string, transport: "hls" | "progressive" | "localFile" = "hls") {
return {
url,
transport: { type: transport },
playbackKind: { type: transport === "hls" ? "transcode" : "directPlay" },
rendition: null,
available: [],
mediaSourceId: null,
playSessionId: null,
needsTranscoding: transport === "hls",
} as import("$lib/api/bindings").StreamSelection;
}
/** A minimal fake <video> element that records mutations and fires events. */
function makeFakeVideo() {
const listeners: Record<string, Array<() => void>> = {};
@@ -54,7 +71,7 @@ function makeBridge(overrides: Partial<Html5ElementBridge> = {}): Html5ElementBr
setSeekOffset: vi.fn((o: number) => {
offset = o;
}),
setStreamUrl: vi.fn(),
setStreamSelection: vi.fn(),
destroyHls: vi.fn(),
getMediaSourceId: () => "msid-1",
...overrides,
@@ -184,7 +201,7 @@ describe("Html5PlayerAdapter", () => {
it("reloadSource() runs the invariant teardown->swap->resume sequence", async () => {
video.paused = false; // was playing → should resume
const p = adapter.reloadSource("http://new/master.m3u8", 120);
const p = adapter.reloadSource(testSelection("http://new/master.m3u8"), 120);
// Teardown happened synchronously before the awaited canplay wait.
expect(video.pause).toHaveBeenCalled();
@@ -194,7 +211,9 @@ describe("Html5PlayerAdapter", () => {
// Allow the internal 100ms settle delay, then fire canplay to resume.
await new Promise((r) => setTimeout(r, 110));
expect(bridge.setStreamUrl).toHaveBeenCalledWith("http://new/master.m3u8");
expect(bridge.setStreamSelection).toHaveBeenCalledWith(
expect.objectContaining({ url: "http://new/master.m3u8", transport: { type: "hls" } }),
);
video._fire("canplay");
video._fire("seeked");
await p;
@@ -217,7 +236,7 @@ describe("Html5PlayerAdapter", () => {
*/
it("reloadSource() seeks to the position and clears the transcode offset", async () => {
video.paused = false;
const p = adapter.reloadSource("http://new/master.m3u8", 1200);
const p = adapter.reloadSource(testSelection("http://new/master.m3u8"), 1200);
await new Promise((r) => setTimeout(r, 110));
expect(bridge.setSeekOffset).toHaveBeenCalledWith(0);
@@ -238,7 +257,7 @@ describe("Html5PlayerAdapter", () => {
/** A reload to the very start has nothing to seek to; it must not stall. */
it("reloadSource() at position 0 does not wait for a seek", async () => {
video.paused = false;
const p = adapter.reloadSource("http://new/master.m3u8", 0);
const p = adapter.reloadSource(testSelection("http://new/master.m3u8"), 0);
await new Promise((r) => setTimeout(r, 110));
video._fire("canplay");
await p; // resolves without any "seeked" event
@@ -258,7 +277,7 @@ describe("Html5PlayerAdapter", () => {
vi.useFakeTimers();
try {
video.paused = false;
const p = adapter.reloadSource("http://new/master.m3u8", 120);
const p = adapter.reloadSource(testSelection("http://new/master.m3u8"), 120);
const assertion = expect(p).rejects.toThrow(/canplay/i);
await vi.advanceTimersByTimeAsync(11_000); // past the 10s readiness budget
await assertion;
@@ -270,7 +289,7 @@ describe("Html5PlayerAdapter", () => {
it("reloadSource() does not resume when it was paused", async () => {
video.paused = true;
const p = adapter.reloadSource("http://new/master.m3u8", 30);
const p = adapter.reloadSource(testSelection("http://new/master.m3u8"), 30);
await new Promise((r) => setTimeout(r, 110));
video._fire("canplay");
video._fire("seeked");
src/lib/player/adapters/html5Adapter.ts
+53 -10
View File
@@ -1,3 +1,4 @@
import type { StreamSelection } from "$lib/api/bindings";
/**
* Html5PlayerAdapter — the Linux/desktop (and interim Android) PlayerAdapter
* implementation. It owns the high-level control surface for an HTML5 `<video>`
@@ -24,6 +25,39 @@ import { createLogger } from "$lib/utils/logger";
const log = createLogger("Html5PlayerAdapter");
/**
* The selection for a plain `load(url)` call.
*
* `PlayerLoadOptions` carries the backend's selection when the caller has one.
* When it does not — a local file, a live stream, a direct URL — the transport
* is inferred *once, here*, from what the caller already knows rather than from
* the URL text: a local path is a local file, and anything the backend flagged
* as transcoded is HLS, because every transcode this app requests is HLS.
*
* This is the one place a fallback is tolerable, and it is explicitly a
* fallback: the negotiated path never reaches it.
*
* TRACES: UR-079 | DR-224
*/
function selectionForLoad(streamUrl: string, options: PlayerLoadOptions): StreamSelection {
if (options.selection) return options.selection;
const transport: StreamSelection["transport"] = options.isLocalFile
? { type: "localFile" }
: options.needsTranscoding
? { type: "hls" }
: { type: "progressive" };
return {
url: streamUrl,
transport,
playbackKind: options.needsTranscoding ? { type: "transcode" } : { type: "directPlay" },
rendition: null,
available: [],
mediaSourceId: options.mediaSourceId ?? null,
playSessionId: null,
needsTranscoding: options.needsTranscoding,
};
}
/**
* Narrow seam the owning component provides so the adapter can execute the
* element/HLS-coupled parts of a control action without re-implementing the
@@ -36,8 +70,16 @@ export interface Html5ElementBridge {
/** Current seek offset (seconds) for transcoded streams. */
getSeekOffset(): number;
setSeekOffset(offset: number): void;
/** Update the stream URL the component renders (triggers its HLS $effect). */
setStreamUrl(url: string): void;
/**
* Update the stream the component renders (triggers its HLS $effect).
*
* Carries the whole [`StreamSelection`], not just the URL: the component's
* effect has to know the transport to choose a loader, and deriving that from
* the URL is the substring check DR-224 removes.
*
* TRACES: UR-079 | DR-224
*/
setStreamSelection(selection: StreamSelection): void;
/** Tear down the component-owned hls.js instance (dual-audio prevention). */
destroyHls(): void;
/** Media source id for seek/audio-track URLs. */
@@ -86,12 +128,13 @@ export class Html5PlayerAdapter implements PlayerAdapter {
this.attachedElement = element;
}
async load(streamUrl: string, _options: PlayerLoadOptions): Promise<void> {
async load(streamUrl: string, options: PlayerLoadOptions): Promise<void> {
// The component's reactive HLS $effect performs the actual attach/load when
// the stream URL is set; loading is therefore driven by setStreamUrl. The
// component's canplay/frag-buffered path reports readiness through the host.
// the selection is set; loading is therefore driven by setStreamSelection.
// The component's canplay/frag-buffered path reports readiness through the
// host.
this.bridge.setSeekOffset(0);
this.bridge.setStreamUrl(streamUrl);
this.bridge.setStreamSelection(selectionForLoad(streamUrl, options));
this.host.onState("loading");
}
@@ -171,12 +214,12 @@ export class Html5PlayerAdapter implements PlayerAdapter {
*
* TRACES: UR-004, UR-005 | DR-181 | UT-183
*/
async reloadSource(url: string, positionSeconds: number): Promise<void> {
async reloadSource(selection: StreamSelection, positionSeconds: number): Promise<void> {
const el = this.element;
if (!el) {
// Still update the stream URL so the component's HLS $effect can pick it up.
// Still update the selection so the component's HLS $effect can pick it up.
this.bridge.setSeekOffset(0);
this.bridge.setStreamUrl(url);
this.bridge.setStreamSelection(selection);
return;
}
const wasPlaying = !el.paused;
@@ -189,7 +232,7 @@ export class Html5PlayerAdapter implements PlayerAdapter {
await new Promise((r) => setTimeout(r, 100));
// The reloaded stream begins at the item's zero, so there is no base to add.
this.bridge.setSeekOffset(0);
this.bridge.setStreamUrl(url);
this.bridge.setStreamSelection(selection);
// A source that never becomes playable is a failed reload, not a slow one:
// the caller (quality switch, transcoded seek) has to know so it can revert
// its selection and surface the error instead of leaving the UI claiming a
src/lib/player/adapters/nativeAdapter.test.ts
+18 -1
View File
@@ -28,6 +28,23 @@ vi.mock("$lib/api/bindings", () => ({
import { NativePlayerAdapter } from "./nativeAdapter";
import type { AdapterHost } from "./types";
/**
* A `StreamSelection` for tests that only care about the URL. Transcoded HLS is
* what these paths exercised before the contract carried a transport.
*/
function testSelection(url: string, transport: "hls" | "progressive" | "localFile" = "hls") {
return {
url,
transport: { type: transport },
playbackKind: { type: transport === "hls" ? "transcode" : "directPlay" },
rendition: null,
available: [],
mediaSourceId: null,
playSessionId: null,
needsTranscoding: transport === "hls",
} as import("$lib/api/bindings").StreamSelection;
}
function makeHost(): AdapterHost {
return {
onState: vi.fn(),
@@ -67,7 +84,7 @@ describe("NativePlayerAdapter", () => {
it("records position on seek/reload primitives (backend does the real work)", async () => {
await adapter.seekElement(55, 0);
expect(adapter.getPosition()).toBe(55);
await adapter.reloadSource("ignored", 200);
await adapter.reloadSource(testSelection("ignored"), 200);
expect(adapter.getPosition()).toBe(200);
});
src/lib/player/adapters/nativeAdapter.ts
+2 -1
View File
@@ -1,3 +1,4 @@
import type { StreamSelection } from "$lib/api/bindings";
/**
* NativePlayerAdapter — the Android/ExoPlayer PlayerAdapter implementation.
*
@@ -89,7 +90,7 @@ export class NativePlayerAdapter implements PlayerAdapter {
* performed the reload+seek internally as part of the seek decision; nothing
* to do on the frontend beyond recording position.
*/
async reloadSource(_url: string, offset: number): Promise<void> {
async reloadSource(_selection: StreamSelection, offset: number): Promise<void> {
this.position = offset;
}
src/lib/player/adapters/types.ts
+23 -5
View File
@@ -1,3 +1,4 @@
import type { StreamSelection } from "$lib/api/bindings";
/**
* PlayerAdapter contract — the decoupled boundary between the UI/backend and a
* concrete video player implementation (Linux HTML5+hls.js, or Android native).
@@ -42,6 +43,19 @@ export interface PlayerLoadOptions {
knownDuration: number;
/** Subtitle tracks available for this media. */
subtitleTracks: SubtitleTrackInput[];
/**
* The backend's decision about this stream, when it made one.
*
* Present for anything negotiated through `repository_get_stream_selection`.
* Null for the paths that never negotiate — a local file, a live channel, a
* plugin's direct URL — where the adapter falls back to what the other
* options already say rather than to reading the URL.
*
* TRACES: UR-079 | DR-224
*/
selection?: StreamSelection | null;
/** The source is a file on disk (or the loopback server in front of one). */
isLocalFile?: boolean;
}
/**
@@ -106,12 +120,16 @@ export interface PlayerAdapter {
seekElement(positionSeconds: number, offset: number): Promise<void>;
/**
* Compound reload: swap to `url` and resume at `offset` seconds. Runs the
* invariant mechanical sequence for this platform (html5: pause → hls teardown
* → clear src → set new url → wait ready → resume; native: ExoPlayer setMediaItem
* + seekTo). No decision is made here — the backend already decided to reload.
* Compound reload: swap to `selection` and resume at `offset` seconds. Runs
* the invariant mechanical sequence for this platform (html5: pause → hls
* teardown → clear src → set new selection → wait ready → resume; native:
* ExoPlayer setMediaItem + seekTo). No decision is made here — the backend
* already decided to reload, and `selection.transport` says how to open it, so
* no adapter has to infer that from the URL.
*
* TRACES: UR-079 | DR-224
*/
reloadSource(url: string, offset: number): Promise<void>;
reloadSource(selection: StreamSelection, offset: number): Promise<void>;
setVolume(volume: number): void; // 0..1
setMuted(muted: boolean): void;
src/lib/player/adapters/webviewAudioAdapter.ts
+3 -2
View File
@@ -1,3 +1,4 @@
import type { StreamSelection } from "$lib/api/bindings";
/**
* Webview audio adapter — plays audio-only media through a hidden `<audio>`
* element on platforms with no native audio backend (currently Windows).
@@ -104,8 +105,8 @@ export class WebviewAudioAdapter implements PlayerAdapter {
}
/** No transcode-reload concept for direct audio; treat as a fresh load. */
async reloadSource(url: string, offset: number): Promise<void> {
await this.load(url, {
async reloadSource(selection: StreamSelection, offset: number): Promise<void> {
await this.load(selection.url, {
mediaId: "",
mediaSourceId: null,
needsTranscoding: false,
src/lib/player/index.ts
+19 -10
View File
@@ -22,6 +22,7 @@ import type {
PlayAlbumTrackRequest,
PlayItemRequest,
StreamingQuality,
StreamSelection,
} from "$lib/api/bindings";
import { auth } from "$lib/stores/auth";
import type { PlayerAdapter } from "./adapters/types";
@@ -150,12 +151,12 @@ async function seekVideo(
audioTrackIndex,
adapter.kind === "html5",
)) as any;
// Serde keeps these snake_case (only the "strategy" tag is camelCase).
// Serde keeps `seek_offset` snake_case (only the "strategy" tag is camelCase).
if (response.strategy === "reloadStream") {
// `seek_offset` is the ABSOLUTE position to resume at, not a base to add to
// the element's clock: the reloaded stream starts at the item's zero since
// DR-181, so reloadSource seeks there. (The name is the wire field's.)
await adapter.reloadSource(response.new_url ?? "", response.seek_offset ?? positionSeconds);
await adapter.reloadSource(response.selection, response.seek_offset ?? positionSeconds);
} else {
await adapter.seekElement(response.position ?? positionSeconds, 0);
}
@@ -182,26 +183,32 @@ async function switchAudioTrack(
mediaSourceId,
)) as any;
if (response.strategy === "reloadStream") {
await adapter.reloadSource(response.new_url!, response.position!);
await adapter.reloadSource(response.selection, response.position!);
}
}
/**
* Change the bandwidth ceiling of the video playing now. The backend re-opens
* the stream at the new quality and decides who reloads: it handles a native
* backend itself, and hands HTML5 a URL for the same `reloadSource` primitive
* the audio-track switch uses. Requires an active video adapter.
* backend itself, and hands HTML5 a selection for the same `reloadSource`
* primitive the audio-track switch uses. Requires an active video adapter.
*
* TRACES: UR-074 | DR-162
* The change applies to **this playback only** — the backend sets a per-playback
* override that the next item clears, leaving the durable Settings default
* alone. Returns the negotiated selection so the caller can show what it
* actually got, which is not always what was asked for: a ceiling above the
* source bitrate is the source.
*
* TRACES: UR-074, UR-079 | DR-162, DR-225
*/
async function setStreamQuality(
quality: StreamingQuality,
currentPosition: number | null,
mediaSourceId: string | null,
audioTrackIndex: number | null,
): Promise<void> {
): Promise<StreamSelection | null> {
const adapter = activeAdapter;
if (!adapter) return;
if (!adapter) return null;
const response = (await commands.playerSetStreamQuality(
requireHandle(),
quality,
@@ -210,10 +217,12 @@ async function setStreamQuality(
mediaSourceId,
audioTrackIndex,
)) as any;
// Serde keeps these snake_case (only the "strategy" tag is camelCase).
if (response.strategy === "reloadStream") {
await adapter.reloadSource(response.new_url ?? "", response.position ?? currentPosition ?? 0);
await adapter.reloadSource(response.selection, response.position ?? currentPosition ?? 0);
return response.selection;
}
// A native backend reloaded itself; there is no selection on that branch.
return null;
}
async function next() {
src/lib/player/streamTransport.test.ts
+93
View File
@@ -0,0 +1,93 @@
/**
* The loader is chosen from the backend's `transport` tag, never from the URL.
*
* TRACES: UR-079 | DR-224 | UT-213
*/
import { describe, expect, it } from "vitest";
import { elementSrcFor, videoLoaderFor, type LoaderCapabilities } from "./streamTransport";
import type { StreamSelection, Transport } from "$lib/api/bindings";
const MODERN: LoaderCapabilities = { hlsJsSupported: true, nativeHlsSupported: false };
const SAFARI: LoaderCapabilities = { hlsJsSupported: false, nativeHlsSupported: true };
const NEITHER: LoaderCapabilities = { hlsJsSupported: false, nativeHlsSupported: false };
function selection(transport: Transport, url: string): Pick<StreamSelection, "url" | "transport"> {
return { url, transport };
}
describe("videoLoaderFor", () => {
it("attaches hls.js when the backend says HLS and hls.js is available", () => {
expect(videoLoaderFor(selection({ type: "hls" }, "https://s/master.m3u8"), MODERN)).toBe(
"hlsjs",
);
});
it("falls back to the element's own HLS loader when hls.js is unavailable", () => {
expect(videoLoaderFor(selection({ type: "hls" }, "https://s/master.m3u8"), SAFARI)).toBe(
"nativeHls",
);
});
it("loads a progressive stream directly", () => {
expect(
videoLoaderFor(
selection({ type: "progressive" }, "https://s/Videos/1/stream?static=true"),
MODERN,
),
).toBe("direct");
});
it("loads a local file directly", () => {
expect(
videoLoaderFor(selection({ type: "localFile" }, "http://127.0.0.1:9/media/x.mkv"), MODERN),
).toBe("direct");
});
// ---------------------------------------------------------------------
// The two cases the `.m3u8` substring check gets wrong. These are the
// reason the field exists; both fail against a URL-sniffing implementation.
// ---------------------------------------------------------------------
it("does NOT attach hls.js to a progressive stream whose URL happens to end .m3u8", () => {
// A direct play served from a path containing the substring — nothing stops
// a server, a proxy, or a local cache from producing this.
expect(
videoLoaderFor(selection({ type: "progressive" }, "https://s/files/movie.m3u8.mp4"), MODERN),
).toBe("direct");
expect(
videoLoaderFor(selection({ type: "progressive" }, "https://s/x?name=master.m3u8"), MODERN),
).toBe("direct");
});
it("DOES attach hls.js to an HLS stream whose URL does not contain .m3u8", () => {
// Jellyfin's own transcoding URLs are not required to end in `.m3u8`, and a
// DASH or query-routed playlist endpoint never would.
expect(videoLoaderFor(selection({ type: "hls" }, "https://s/Videos/1/hls"), MODERN)).toBe(
"hlsjs",
);
expect(
videoLoaderFor(selection({ type: "hls" }, "https://s/stream?format=playlist"), SAFARI),
).toBe("nativeHls");
});
it("falls back to direct when HLS is requested but nothing can play it", () => {
expect(videoLoaderFor(selection({ type: "hls" }, "https://s/master.m3u8"), NEITHER)).toBe(
"direct",
);
});
});
describe("elementSrcFor", () => {
it("empties the element's src only when hls.js drives it", () => {
expect(elementSrcFor(selection({ type: "hls" }, "https://s/master.m3u8"), MODERN)).toBe("");
expect(elementSrcFor(selection({ type: "hls" }, "https://s/master.m3u8"), SAFARI)).toBe(
"https://s/master.m3u8",
);
});
it("keeps the src for a progressive stream that looks like a playlist", () => {
const s = selection({ type: "progressive" }, "https://s/files/movie.m3u8.mp4");
expect(elementSrcFor(s, MODERN)).toBe("https://s/files/movie.m3u8.mp4");
});
});
src/lib/player/streamTransport.ts
+68
View File
@@ -0,0 +1,68 @@
/**
* Which loader opens a stream in the webview `<video>` element.
*
* Extracted from `VideoPlayer.svelte` so the decision can be unit-tested — the
* same pattern as `episodeStrip.ts` and `TrackList.logic.test.ts`.
*
* TRACES: UR-079 | DR-224 | UT-213
*/
import type { StreamSelection, Transport } from "$lib/api/bindings";
/** How the element should be fed. */
export type VideoLoader =
/** hls.js drives a MediaSource; the element's own `src` stays empty. */
| "hlsjs"
/** The element loads the playlist itself (Safari/WebKit native HLS). */
| "nativeHls"
/** The element loads the URL directly — a progressive file or a local one. */
| "direct";
/** What the running browser can do, passed in so the decision stays pure. */
export interface LoaderCapabilities {
/** `Hls.isSupported()` */
hlsJsSupported: boolean;
/** `video.canPlayType("application/vnd.apple.mpegurl")` was non-empty */
nativeHlsSupported: boolean;
}
/**
* Pick the loader from the backend's tagged `transport`.
*
* This used to read `url.includes(".m3u8")`, in two places in
* `VideoPlayer.svelte`. Rust *builds* that URL and knows exactly what it is;
* re-deriving the answer here by substring match is a domain fact reconstructed
* in the presentation layer — the same error as leaking item-type taxonomy, and
* one that fails silently in both directions: a progressive file served from a
* path containing `.m3u8` gets an HLS loader, and a playlist served from a path
* without it does not.
*
* The transport is the *stream's* property; whether a given loader exists is the
* *browser's*. Only the second is decided here.
*/
export function videoLoaderFor(
selection: Pick<StreamSelection, "url" | "transport">,
capabilities: LoaderCapabilities,
): VideoLoader {
if (selection.transport.type !== "hls") {
// Progressive and local files are what the element loads natively. No
// MediaSource, no playlist parsing.
return "direct";
}
if (capabilities.hlsJsSupported) return "hlsjs";
if (capabilities.nativeHlsSupported) return "nativeHls";
// Nothing here can parse a playlist. Handing the URL to the element is very
// likely to fail, but it is the only remaining move and it surfaces a real
// media error rather than silently doing nothing.
return "direct";
}
/** Convenience for the template: does the element's `src` stay empty? */
export function elementSrcFor(
selection: Pick<StreamSelection, "url" | "transport">,
capabilities: LoaderCapabilities,
): string {
return videoLoaderFor(selection, capabilities) === "hlsjs" ? "" : selection.url;
}
export type { Transport };
src/routes/player/[id]/+page.svelte
+70 -50
View File
@@ -3,8 +3,8 @@
import { page } from "$app/stores";
import { goto } from "$app/navigation";
import { commands } from "$lib/api/bindings";
import { downloadedFilePath, resolveVideoSource } from "$lib/player/localSource";
import type { PlayQueueRequest } from "$lib/api/bindings";
import { downloadedFilePath } from "$lib/player/localSource";
import type { PlayQueueRequest, StreamSelection } from "$lib/api/bindings";
import type { MediaItem, MediaKind } from "$lib/api/types";
import { auth } from "$lib/stores/auth";
import { library } from "$lib/stores/library";
@@ -76,7 +76,15 @@
const hasNext = $derived($hasNextStore);
const hasPrevious = $derived($hasPreviousStore);
let currentMedia = $state<MediaItem | null>(null);
let streamUrl = $state<string | null>(null);
/**
* What to play, as the backend decided it. Null while still resolving.
*
* Replaces a bare URL string: the transport travels with it, so neither this
* page nor VideoPlayer has to work out whether the URL is a playlist.
*
* TRACES: UR-079 | DR-224
*/
let selection = $state<StreamSelection | null>(null);
let mediaSourceId = $state<string | null>(null);
let isVideo = $state(false);
let isLive = $state(false); // Whether this is a live stream (Live TV channel) - no seek/resume
@@ -94,7 +102,7 @@
// Which player component to render. Video without a stream URL is "pending"
// (still resolving), never audio — see playerSurface.ts.
const surface = $derived(resolvePlayerSurface({ isVideo, streamUrl }));
const surface = $derived(resolvePlayerSurface({ isVideo, streamUrl: selection?.url ?? null }));
onMount(() => {
// Start position polling (only for audio via MPV backend)
@@ -308,17 +316,17 @@
const fullPath = downloadedFilePath(storagePath, localDownload.filePath);
log.debug("loadAndPlay: Full local path:", fullPath);
// Serve the file over the loopback media server rather than the asset
// protocol: the asset protocol answers a range-less request with the
// entire file, so a downloaded film never finished loading. Rust mints
// the URL (it holds the port and the per-session token).
// TRACES: UR-071 | DR-137
const localUrl = await commands.mediaLocalUrl(fullPath);
log.debug("loadAndPlay: Local media URL resolved");
if (isVideo) {
// Local video files don't need transcoding and support native seeking
streamUrl = localUrl;
// Served over the loopback media server rather than the asset
// protocol: the asset protocol answers a range-less request with the
// entire file, so a downloaded film never finished loading. Rust mints
// the URL (it holds the port and the per-session token) and states the
// transport with it.
//
// A downloaded file is a direct play over a local transport, and Rust
// says so rather than this page assuming it.
// TRACES: UR-071 | DR-137, DR-224
selection = await commands.mediaLocalSelection(fullPath);
videoNeedsTranscoding = false;
// Use explicit startPosition, or fall back to retrieved progress from database
const effectivePosition = startPosition ?? retrievedProgressSeconds ?? 0;
@@ -355,7 +363,19 @@
const liveInfo = await repo.openLiveStream(id);
log.debug("loadAndPlay: Live stream URL:", liveInfo.streamUrl);
mediaSourceId = liveInfo.mediaSourceId;
streamUrl = liveInfo.streamUrl;
selection = {
url: liveInfo.streamUrl,
// Rust's verdict, not a guess from the URL.
transport: liveInfo.transport,
playbackKind: { type: "transcode" },
rendition: null,
// A live channel has no ladder to offer: there is no source file to
// measure and no rendition to re-negotiate against.
available: [],
mediaSourceId: liveInfo.mediaSourceId,
playSessionId: liveInfo.playSessionId,
needsTranscoding: true,
};
videoNeedsTranscoding = true;
videoInitialPosition = 0;
isPlaying = true;
@@ -363,46 +383,46 @@
return;
}
log.debug("loadAndPlay: Getting playback info");
const playbackInfo = await repo.getPlaybackInfo(id);
log.debug("loadAndPlay: Got playback info, mediaSourceId:", playbackInfo.mediaSourceId);
if (isVideo) {
// Playback API now detects HEVC/10-bit and returns transcoded URL when needed
log.debug(
"loadAndPlay: Using video stream, directPlay:",
playbackInfo.directPlay,
"needsTranscoding:",
playbackInfo.needsTranscoding,
);
mediaSourceId = playbackInfo.mediaSourceId;
// Prefer a completed download over streaming. Audio has done this
// since the queue is built; video previously always streamed, so a
// downloaded film re-spent bandwidth already spent and would not play
// at all offline. Rust returns null when nothing is downloaded or the
// file has gone, so this falls back to the server on its own.
// TRACES: UR-071 | DR-123
// A downloaded file is served over the loopback media server, not the
// asset protocol — see DR-137. The URL is minted up front because
// resolveVideoSource stays pure/synchronous.
// TRACES: UR-071 | DR-123, DR-137
//
// Checked *first* so the streaming path below negotiates exactly once:
// asking for a `PlaybackInfo` and then a stream selection meant two
// negotiations per load, and each one claims a transcode identity and
// retires the previous — so the server started a job only to be told
// to stop it a moment later. Observed in the log as a pair of
// `[StreamSelection]` lines for one play.
//
// TRACES: UR-071 | DR-123, DR-137, DR-224
const localPath = await commands.playerLocalMediaPath(id);
const localUrl = localPath ? await commands.mediaLocalUrl(localPath) : null;
const source = resolveVideoSource({
localPath,
remoteUrl: playbackInfo.streamUrl,
remoteNeedsTranscoding: playbackInfo.needsTranscoding,
toAssetUrl: () => localUrl ?? "",
});
streamUrl = source.url;
videoNeedsTranscoding = source.needsTranscoding;
log.debug(
source.isLocal
? "loadAndPlay: Playing downloaded file from disk"
: `loadAndPlay: Using stream URL: ${streamUrl}`,
);
if (localPath) {
// A downloaded file is a direct play over a local transport, served
// by the loopback media server rather than the asset protocol
// (DR-137). Its media-source id still comes from the server, since
// that is what subtitle URLs are keyed by.
selection = await commands.mediaLocalSelection(localPath);
videoNeedsTranscoding = false;
mediaSourceId = (await repo.getPlaybackInfo(id)).mediaSourceId;
log.debug("loadAndPlay: Playing downloaded file from disk");
} else {
// Rust negotiates direct play vs direct stream vs transcode against
// the device profile and the ceiling in force, and returns the
// transport and the media-source id with it. This page no longer
// decides — or separately asks for — any of that.
// TRACES: UR-070, UR-079 | DR-224, DR-226, DR-227
selection = await repo.getStreamSelection(id, null, null);
mediaSourceId = selection.mediaSourceId;
// Rust's own verdict — "which kinds count as transcoding" is a
// domain rule, and a direct *stream* is a remux that does not.
videoNeedsTranscoding = selection.needsTranscoding;
log.debug(
`loadAndPlay: ${selection.playbackKind.type} over ${selection.transport.type}`,
);
}
// Set initial position for the video player to seek to after load.
// Use explicit startPosition, or fall back to retrieved progress.
@@ -847,10 +867,10 @@
class="w-8 h-8 border-2 border-[var(--color-jellyfin)] border-t-transparent rounded-full animate-spin"
></div>
</div>
{:else if surface === "video" && streamUrl}
{:else if surface === "video" && selection}
<VideoPlayer
media={currentMedia}
{streamUrl}
{selection}
mediaSourceId={mediaSourceId ?? undefined}
initialPosition={videoInitialPosition}
needsTranscoding={videoNeedsTranscoding}