Files
jellytau/src/lib/player/adapters/html5Adapter.ts
T
dtourolle 109700b949 feat(playback): let Rust decide what stream to play, and say so
Playing a video meant asking the server to re-encode it, always. That
decision was made nowhere and written down nowhere, so whoever needed it
re-derived it downstream — the player worked out whether it had been handed
a playlist by looking for ".m3u8" in the URL, in two places. A viewer paid
for a transcode of a file their device could have played untouched, and the
app could not tell them which it was.

One negotiation now produces one self-describing StreamSelection — direct
play, remux or transcode; over a playlist, a plain HTTP file, or a local one
— and every renderer consumes that same answer.

Measured against the development server (Jellyfin 10.11.5), 400 items
sampled for codec mix and 40 put through a real PlaybackInfo negotiation
per profile:

  Linux / WebKitGTK (h264 only, 2ch)          3/40 —  7% direct play
  Android / ExoPlayer (hevc, ac3/eac3, 6ch)  34/40 — 85% direct play

The library is ~80% hevc, which is why the two diverge so hard. The payoff
is overwhelmingly Android, where 85% of plays were starting a transcode
nobody needed. Linux stays near 7% until libmpv decodes the picture — the
h264-only profile is a WebKitGTK constraint, not a JellyTau choice.

DR-219  StreamSelection: url + tagged Transport (hls/progressive/localFile)
        + PlaybackKind (directPlay/directStream/transcode) + the negotiated
        rendition + this source's ladder + a needs_transcoding flag derived
        in Rust so the rule is answered once. Both enums are serde-tagged
        so the frontend matches a discriminant, not a substring. The paths
        that never negotiate get the same shape from Rust rather than
        assembling one — media_local_selection for a downloaded file,
        LiveStreamInfo.transport for a live channel — so there is no second
        place where a transport is decided.

DR-220  The ceiling becomes two levels: a durable device default (Settings,
        persisted) and a per-playback override the in-player picker sets.
        The picker had called itself a "this film, this connection" control
        since it was written but wrote the process-wide default, so dropping
        one awkward film to 2 Mbps silently capped every video played
        afterwards for the rest of the process, with Settings still showing
        the old value. The override is cleared whenever playback moves to a
        new item, which stops it surviving into an autoplayed next episode.
        effective_streaming_quality() is the single resolution point.

DR-221  The quality picker is filled from what this media source can offer.
        Rust marks a rung exceeds_source when its ceiling is at or above the
        source's own bitrate — such a rung is another way to spell Original
        — and the frontend does not draw those. Original is never marked; a
        source whose bitrate the server does not report marks nothing, which
        keeps every rung offered.

DR-222  Direct play and direct stream are negotiated, with two client-side
        overrides on top because the server's answer is right about the file
        and wrong about what this app will do with it: undecodable audio
        (Jellyfin 10.11.5 honours a DirectPlayProfile's container and video
        codec but ignores its audio codec, so it offers direct play for an
        E-AC-3 track the webview renders in silence) and a viewer-pinned
        audio track the file does not default to. A direct stream is a remux
        and is deliberately not counted as transcoding.

DR-223  Dropped on measurement, not deferred. A master playlist from this
        server carries exactly one EXT-X-STREAM-INF: Jellyfin builds it from
        the single rendition the request asked for rather than publishing a
        ladder. So there is no adaptation for hls.js to be preserving and
        none mpv would lose — the claim that there was, in
        playback-backend-unification.md, does not hold. Recorded rather than
        deleted because it is a measurement: a server that does publish a
        ladder would change the answer.

DR-224  Every backend consumes the same selection. The queue item carries
        the transport, so player_seek_video picks its seek strategy from the
        backend's decision instead of the last stream_url.contains(".m3u8")
        in the codebase. Items queued by a path that never negotiated carry
        None and fall back to needs_transcoding, which is exact rather than
        a guess because every transcode this app requests is HLS (DR-140).

The frontend loader decision moves to streamTransport.ts so it can be
tested: the two cases that pin it are the ones that failed against the old
implementation — a progressive stream whose URL contains ".m3u8" must not
get an HLS loader, and an HLS stream whose URL contains none must.

Also verified the URL the direct-play branch builds actually serves playable
bytes: 206, video/mp4, valid ISO-BMFF, and a mid-file range works, so
seeking a direct play works.

The spec is folded into docs/architecture/{01,02,03} and deleted, per the
rule that docs/specs holds only work that has not shipped. DR-121 leaves
read-through-media-cache.md with a pointer; that spec keeps its capture half.

Not verified: real playback on a device. Direct play changes what actually
gets played, and neither fixtures nor curl prove the WebKitGTK and ExoPlayer
paths render it.
2026-08-22 13:45:03 +02:00

318 lines
12 KiB
TypeScript

import type { StreamSelection } from "$lib/api/bindings";
/**
* Html5PlayerAdapter — the Linux/desktop (and interim Android) PlayerAdapter
* implementation. It owns the high-level control surface for an HTML5 `<video>`
* element and reports the element's lifecycle back into Rust via its
* {@link AdapterHost}.
*
* Design note on the split with VideoPlayer.svelte:
* The delicate, timing-sensitive parts (hls.js instance lifecycle, the transcode
* "reload stream" seek/audio-track dance with its dual-audio teardown and
* canplay waits) are inherently coupled to Svelte reactive state and the DOM
* element. Rather than relocate that reactive machinery wholesale (high
* regression risk), the adapter receives an {@link Html5ElementBridge} of narrow
* callbacks the owning component supplies. The adapter is the single OWNER of the
* control contract (play/pause/seek/track/volume) and of reporting; the bridge is
* the seam to the component's element/HLS/reactive state. This keeps all control
* intents flowing through the PlayerAdapter interface while preserving the
* hard-won element behavior verbatim.
*
* TRACES: UR-003, UR-005, UR-020, UR-021 | DR-001, DR-023, DR-024, DR-028, DR-096
*/
import type { AdapterHost, PlayerAdapter, PlayerLoadOptions } from "./types";
import { createLogger } from "$lib/utils/logger";
const log = createLogger("Html5PlayerAdapter");
/**
* The selection for a plain `load(url)` call.
*
* `PlayerLoadOptions` carries the backend's selection when the caller has one.
* When it does not — a local file, a live stream, a direct URL — the transport
* is inferred *once, here*, from what the caller already knows rather than from
* the URL text: a local path is a local file, and anything the backend flagged
* as transcoded is HLS, because every transcode this app requests is HLS.
*
* This is the one place a fallback is tolerable, and it is explicitly a
* fallback: the negotiated path never reaches it.
*
* TRACES: UR-079 | DR-224
*/
function selectionForLoad(streamUrl: string, options: PlayerLoadOptions): StreamSelection {
if (options.selection) return options.selection;
const transport: StreamSelection["transport"] = options.isLocalFile
? { type: "localFile" }
: options.needsTranscoding
? { type: "hls" }
: { type: "progressive" };
return {
url: streamUrl,
transport,
playbackKind: options.needsTranscoding ? { type: "transcode" } : { type: "directPlay" },
rendition: null,
available: [],
mediaSourceId: options.mediaSourceId ?? null,
playSessionId: null,
needsTranscoding: options.needsTranscoding,
};
}
/**
* Narrow seam the owning component provides so the adapter can execute the
* element/HLS-coupled parts of a control action without re-implementing the
* component's reactive HLS lifecycle. Every function here is a thin wrapper over
* work the component already does.
*/
export interface Html5ElementBridge {
/** The bound <video> element, or null before mount / after teardown. */
getElement(): HTMLVideoElement | null;
/** Current seek offset (seconds) for transcoded streams. */
getSeekOffset(): number;
setSeekOffset(offset: number): void;
/**
* Update the stream the component renders (triggers its HLS $effect).
*
* Carries the whole [`StreamSelection`], not just the URL: the component's
* effect has to know the transport to choose a loader, and deriving that from
* the URL is the substring check DR-224 removes.
*
* TRACES: UR-079 | DR-224
*/
setStreamSelection(selection: StreamSelection): void;
/** Tear down the component-owned hls.js instance (dual-audio prevention). */
destroyHls(): void;
/** Media source id for seek/audio-track URLs. */
getMediaSourceId(): string | null;
}
/**
* True for the `AbortError` the browser raises when a pending `play()` promise is
* cancelled by a `pause()` (or a source/seek change). It signals "that specific
* play attempt was superseded", not "playback failed" — hls.js' stall recovery
* produces it routinely, so it must not reach the player's error channel.
*/
function isPlayInterruptedError(err: unknown): boolean {
if (!err || typeof err !== "object") return false;
const { name, message } = err as { name?: string; message?: string };
return name === "AbortError" || (message ?? "").includes("interrupted");
}
export class Html5PlayerAdapter implements PlayerAdapter {
readonly kind = "html5" as const;
private attachedElement: HTMLVideoElement | null = null;
/** In-flight play() attempt, so concurrent callers share one element.play(). */
private pendingPlay: Promise<void> | null = null;
private host: AdapterHost;
private bridge: Html5ElementBridge;
constructor(host: AdapterHost, bridge: Html5ElementBridge) {
this.host = host;
this.bridge = bridge;
}
/**
* Resolve the LIVE <video> element. The bridge's `getElement()` returns the
* component's current reactive `videoElement`, which is authoritative: the
* element can be re-bound when the {#if} block re-renders, so a value captured
* once in `attach()` may go stale (this caused play/pause to silently no-op).
* Falls back to the attach()-captured element for unit tests whose bridge
* returns null.
*/
private get element(): HTMLVideoElement | null {
return this.bridge.getElement() ?? this.attachedElement;
}
attach(element: HTMLVideoElement | null): void {
this.attachedElement = element;
}
async load(streamUrl: string, options: PlayerLoadOptions): Promise<void> {
// The component's reactive HLS $effect performs the actual attach/load when
// the selection is set; loading is therefore driven by setStreamSelection.
// The component's canplay/frag-buffered path reports readiness through the
// host.
this.bridge.setSeekOffset(0);
this.bridge.setStreamSelection(selectionForLoad(streamUrl, options));
this.host.onState("loading");
}
async play(): Promise<void> {
const el = this.element;
if (!el) return;
// Coalesce concurrent attempts. While an HLS stream stalls, the UI and the
// gap-controller recovery path can both ask to play; stacking element.play()
// calls is what turns one stall into an AbortError storm.
if (this.pendingPlay) return this.pendingPlay;
this.pendingPlay = (async () => {
try {
await el.play();
// handlePlay on the element reports "playing"; no double-report here.
} catch (err) {
// A play() aborted by a pause() is transient, not a failure: hls.js
// nudges the element to recover from a stall, which cancels the pending
// play promise while the element keeps trying. Surfacing it would report
// an error roughly once a second for the duration of the stall.
if (isPlayInterruptedError(err)) {
log.debug("play() interrupted by pause (stall recovery)");
} else {
this.host.onError(`play() failed: ${err}`);
}
} finally {
this.pendingPlay = null;
}
})();
return this.pendingPlay;
}
async pause(): Promise<void> {
this.element?.pause();
}
async toggle(): Promise<boolean> {
const el = this.element;
if (!el) return false;
if (el.paused) {
await this.play();
return true;
}
await this.pause();
return false;
}
/**
* PRIMITIVE: in-place element seek (no reload). The backend already decided
* this seek does not need a transcode reload.
*/
async seekElement(positionSeconds: number, offset: number): Promise<void> {
const el = this.element;
if (!el) return;
el.currentTime = positionSeconds;
this.bridge.setSeekOffset(offset);
await this.waitForEvent(el, "seeked", 2000);
}
/**
* PRIMITIVE: compound reload — swap the source and resume at
* `positionSeconds`, an **absolute** position on the item's own timeline.
* Contains NO strategy decision; the backend already decided to reload and
* supplied the url/position. Preserves the hard-won dual-audio teardown and
* canplay wait.
*
* The position is reached by *seeking the element*, and the transcode offset
* is cleared to zero. It used to be the other way round — the offset was set
* to the position and nothing seeked — which was correct only while the
* reloaded URL itself began there, via `StartTimeTicks`. DR-181 removes that
* parameter (on an HLS playlist it makes the server reject every segment with
* `400`), so a reloaded stream now always starts at the beginning of the item.
* Leaving the old arithmetic in place would have left `currentTime` reading
* `offset + 0` — the scrubber showing 20:00 while the opening titles play, and
* no seek ever happening.
*
* TRACES: UR-004, UR-005 | DR-181 | UT-183
*/
async reloadSource(selection: StreamSelection, positionSeconds: number): Promise<void> {
const el = this.element;
if (!el) {
// Still update the selection so the component's HLS $effect can pick it up.
this.bridge.setSeekOffset(0);
this.bridge.setStreamSelection(selection);
return;
}
const wasPlaying = !el.paused;
el.pause();
this.bridge.destroyHls();
if (el.src) {
el.removeAttribute("src");
el.load();
}
await new Promise((r) => setTimeout(r, 100));
// The reloaded stream begins at the item's zero, so there is no base to add.
this.bridge.setSeekOffset(0);
this.bridge.setStreamSelection(selection);
// A source that never becomes playable is a failed reload, not a slow one:
// the caller (quality switch, transcoded seek) has to know so it can revert
// its selection and surface the error instead of leaving the UI claiming a
// stream that is not playing.
const ready = await this.waitForEvent(el, "canplay", 10000);
if (!ready) {
throw new Error(`Reloaded stream never fired "canplay" within 10000ms`);
}
// Now that the new source is playable, put it where the caller asked for.
// Seeking before `canplay` is dropped by the element, which is why this
// follows the wait rather than riding along with the URL swap.
if (positionSeconds > 0) {
el.currentTime = positionSeconds;
await this.waitForEvent(el, "seeked", 2000);
}
if (wasPlaying) await el.play();
}
setVolume(volume: number): void {
if (this.element) this.element.volume = Math.max(0, Math.min(1, volume));
}
setMuted(muted: boolean): void {
if (this.element) this.element.muted = muted;
}
/** Subtitle selection: HTML5 toggles textTracks on the element directly. */
async selectSubtitle(streamIndex: number | null, _arrayIndex?: number): Promise<void> {
const el = this.element;
if (!el || !el.textTracks) return;
for (let i = 0; i < el.textTracks.length; i++) {
el.textTracks[i].mode = "disabled";
}
if (streamIndex !== null) {
const tracks = el.querySelectorAll("track");
tracks.forEach((track) => {
const idx = parseInt(track.getAttribute("data-stream-index") || "-1");
if (idx === streamIndex && track.track) {
track.track.mode = "showing";
}
});
}
}
getPosition(): number {
const el = this.element;
if (!el) return 0;
return el.currentTime + this.bridge.getSeekOffset();
}
async dispose(): Promise<void> {
this.bridge.destroyHls();
const el = this.element;
if (el) {
el.pause();
el.removeAttribute("src");
el.load();
}
this.attachedElement = null;
}
/** Resolve when `event` fires on `el`, or after `timeoutMs` as a fallback. */
/**
* Resolves `true` when the event fires, `false` if the budget runs out. The
* distinction is the caller's to act on: a missing `seeked` is cosmetic, a
* missing `canplay` means the reload failed.
*/
private waitForEvent(el: HTMLVideoElement, event: string, timeoutMs: number): Promise<boolean> {
return new Promise<boolean>((resolve) => {
const done = (fired: boolean) => {
el.removeEventListener(event, listener);
clearTimeout(timer);
resolve(fired);
};
const listener = () => done(true);
el.addEventListener(event, listener);
// `done` closes over `timer`, but can only run once the listener fires or
// the timeout elapses — both strictly after this assignment.
const timer: ReturnType<typeof setTimeout> = setTimeout(() => done(false), timeoutMs);
});
}
}