feat(playback): let Rust decide what stream to play, and say so

Playing a video meant asking the server to re-encode it, always. That decision was made nowhere and written down nowhere, so whoever needed it re-derived it downstream — the player worked out whether it had been handed a playlist by looking for ".m3u8" in the URL, in two places. A viewer paid for a transcode of a file their device could have played untouched, and the app could not tell them which it was. One negotiation now produces one self-describing StreamSelection — direct play, remux or transcode; over a playlist, a plain HTTP file, or a local one — and every renderer consumes that same answer. Measured against the development server (Jellyfin 10.11.5), 400 items sampled for codec mix and 40 put through a real PlaybackInfo negotiation per profile: Linux / WebKitGTK (h264 only, 2ch) 3/40 — 7% direct play Android / ExoPlayer (hevc, ac3/eac3, 6ch) 34/40 — 85% direct play The library is ~80% hevc, which is why the two diverge so hard. The payoff is overwhelmingly Android, where 85% of plays were starting a transcode nobody needed. Linux stays near 7% until libmpv decodes the picture — the h264-only profile is a WebKitGTK constraint, not a JellyTau choice. DR-219 StreamSelection: url + tagged Transport (hls/progressive/localFile) + PlaybackKind (directPlay/directStream/transcode) + the negotiated rendition + this source's ladder + a needs_transcoding flag derived in Rust so the rule is answered once. Both enums are serde-tagged so the frontend matches a discriminant, not a substring. The paths that never negotiate get the same shape from Rust rather than assembling one — media_local_selection for a downloaded file, LiveStreamInfo.transport for a live channel — so there is no second place where a transport is decided. DR-220 The ceiling becomes two levels: a durable device default (Settings, persisted) and a per-playback override the in-player picker sets. The picker had called itself a "this film, this connection" control since it was written but wrote the process-wide default, so dropping one awkward film to 2 Mbps silently capped every video played afterwards for the rest of the process, with Settings still showing the old value. The override is cleared whenever playback moves to a new item, which stops it surviving into an autoplayed next episode. effective_streaming_quality() is the single resolution point. DR-221 The quality picker is filled from what this media source can offer. Rust marks a rung exceeds_source when its ceiling is at or above the source's own bitrate — such a rung is another way to spell Original — and the frontend does not draw those. Original is never marked; a source whose bitrate the server does not report marks nothing, which keeps every rung offered. DR-222 Direct play and direct stream are negotiated, with two client-side overrides on top because the server's answer is right about the file and wrong about what this app will do with it: undecodable audio (Jellyfin 10.11.5 honours a DirectPlayProfile's container and video codec but ignores its audio codec, so it offers direct play for an E-AC-3 track the webview renders in silence) and a viewer-pinned audio track the file does not default to. A direct stream is a remux and is deliberately not counted as transcoding. DR-223 Dropped on measurement, not deferred. A master playlist from this server carries exactly one EXT-X-STREAM-INF: Jellyfin builds it from the single rendition the request asked for rather than publishing a ladder. So there is no adaptation for hls.js to be preserving and none mpv would lose — the claim that there was, in playback-backend-unification.md, does not hold. Recorded rather than deleted because it is a measurement: a server that does publish a ladder would change the answer. DR-224 Every backend consumes the same selection. The queue item carries the transport, so player_seek_video picks its seek strategy from the backend's decision instead of the last stream_url.contains(".m3u8") in the codebase. Items queued by a path that never negotiated carry None and fall back to needs_transcoding, which is exact rather than a guess because every transcode this app requests is HLS (DR-140). The frontend loader decision moves to streamTransport.ts so it can be tested: the two cases that pin it are the ones that failed against the old implementation — a progressive stream whose URL contains ".m3u8" must not get an HLS loader, and an HLS stream whose URL contains none must. Also verified the URL the direct-play branch builds actually serves playable bytes: 206, video/mp4, valid ISO-BMFF, and a mid-file range works, so seeking a direct play works. The spec is folded into docs/architecture/{01,02,03} and deleted, per the rule that docs/specs holds only work that has not shipped. DR-121 leaves read-through-media-cache.md with a pointer; that spec keeps its capture half. Not verified: real playback on a device. Direct play changes what actually gets played, and neither fixtures nor curl prove the WebKitGTK and ExoPlayer paths render it.
2026-08-22 13:45:03 +02:00
parent 5fede123e7
commit 109700b949
45 changed files with 9744 additions and 6581 deletions
@@ -10,6 +10,23 @@ import { describe, it, expect, vi, beforeEach } from "vitest";
 import { Html5PlayerAdapter, type Html5ElementBridge } from "./html5Adapter";
 import type { AdapterHost } from "./types";

+/**
+ * A `StreamSelection` for tests that only care about the URL. Transcoded HLS is
+ * what these paths exercised before the contract carried a transport.
+ */
+function testSelection(url: string, transport: "hls" | "progressive" | "localFile" = "hls") {
+  return {
+    url,
+    transport: { type: transport },
+    playbackKind: { type: transport === "hls" ? "transcode" : "directPlay" },
+    rendition: null,
+    available: [],
+    mediaSourceId: null,
+    playSessionId: null,
+    needsTranscoding: transport === "hls",
+  } as import("$lib/api/bindings").StreamSelection;
+}
+
 /** A minimal fake <video> element that records mutations and fires events. */
 function makeFakeVideo() {
  const listeners: Record<string, Array<() => void>> = {};
@@ -54,7 +71,7 @@ function makeBridge(overrides: Partial<Html5ElementBridge> = {}): Html5ElementBr
    setSeekOffset: vi.fn((o: number) => {
      offset = o;
    }),
-    setStreamUrl: vi.fn(),
+    setStreamSelection: vi.fn(),
    destroyHls: vi.fn(),
    getMediaSourceId: () => "msid-1",
    ...overrides,
@@ -184,7 +201,7 @@ describe("Html5PlayerAdapter", () => {

  it("reloadSource() runs the invariant teardown->swap->resume sequence", async () => {
    video.paused = false; // was playing → should resume
-    const p = adapter.reloadSource("http://new/master.m3u8", 120);
+    const p = adapter.reloadSource(testSelection("http://new/master.m3u8"), 120);

    // Teardown happened synchronously before the awaited canplay wait.
    expect(video.pause).toHaveBeenCalled();
@@ -194,7 +211,9 @@ describe("Html5PlayerAdapter", () => {

    // Allow the internal 100ms settle delay, then fire canplay to resume.
    await new Promise((r) => setTimeout(r, 110));
-    expect(bridge.setStreamUrl).toHaveBeenCalledWith("http://new/master.m3u8");
+    expect(bridge.setStreamSelection).toHaveBeenCalledWith(
+      expect.objectContaining({ url: "http://new/master.m3u8", transport: { type: "hls" } }),
+    );
    video._fire("canplay");
    video._fire("seeked");
    await p;
@@ -217,7 +236,7 @@ describe("Html5PlayerAdapter", () => {
   */
  it("reloadSource() seeks to the position and clears the transcode offset", async () => {
    video.paused = false;
-    const p = adapter.reloadSource("http://new/master.m3u8", 1200);
+    const p = adapter.reloadSource(testSelection("http://new/master.m3u8"), 1200);

    await new Promise((r) => setTimeout(r, 110));
    expect(bridge.setSeekOffset).toHaveBeenCalledWith(0);
@@ -238,7 +257,7 @@ describe("Html5PlayerAdapter", () => {
  /** A reload to the very start has nothing to seek to; it must not stall. */
  it("reloadSource() at position 0 does not wait for a seek", async () => {
    video.paused = false;
-    const p = adapter.reloadSource("http://new/master.m3u8", 0);
+    const p = adapter.reloadSource(testSelection("http://new/master.m3u8"), 0);
    await new Promise((r) => setTimeout(r, 110));
    video._fire("canplay");
    await p; // resolves without any "seeked" event
@@ -258,7 +277,7 @@ describe("Html5PlayerAdapter", () => {
    vi.useFakeTimers();
    try {
      video.paused = false;
-      const p = adapter.reloadSource("http://new/master.m3u8", 120);
+      const p = adapter.reloadSource(testSelection("http://new/master.m3u8"), 120);
      const assertion = expect(p).rejects.toThrow(/canplay/i);
      await vi.advanceTimersByTimeAsync(11_000); // past the 10s readiness budget
      await assertion;
@@ -270,7 +289,7 @@ describe("Html5PlayerAdapter", () => {

  it("reloadSource() does not resume when it was paused", async () => {
    video.paused = true;
-    const p = adapter.reloadSource("http://new/master.m3u8", 30);
+    const p = adapter.reloadSource(testSelection("http://new/master.m3u8"), 30);
    await new Promise((r) => setTimeout(r, 110));
    video._fire("canplay");
    video._fire("seeked");
@@ -1,3 +1,4 @@
+import type { StreamSelection } from "$lib/api/bindings";
 /**
 * Html5PlayerAdapter — the Linux/desktop (and interim Android) PlayerAdapter
 * implementation. It owns the high-level control surface for an HTML5 `<video>`
@@ -24,6 +25,39 @@ import { createLogger } from "$lib/utils/logger";

 const log = createLogger("Html5PlayerAdapter");

+/**
+ * The selection for a plain `load(url)` call.
+ *
+ * `PlayerLoadOptions` carries the backend's selection when the caller has one.
+ * When it does not — a local file, a live stream, a direct URL — the transport
+ * is inferred *once, here*, from what the caller already knows rather than from
+ * the URL text: a local path is a local file, and anything the backend flagged
+ * as transcoded is HLS, because every transcode this app requests is HLS.
+ *
+ * This is the one place a fallback is tolerable, and it is explicitly a
+ * fallback: the negotiated path never reaches it.
+ *
+ * TRACES: UR-079 | DR-224
+ */
+function selectionForLoad(streamUrl: string, options: PlayerLoadOptions): StreamSelection {
+  if (options.selection) return options.selection;
+  const transport: StreamSelection["transport"] = options.isLocalFile
+    ? { type: "localFile" }
+    : options.needsTranscoding
+      ? { type: "hls" }
+      : { type: "progressive" };
+  return {
+    url: streamUrl,
+    transport,
+    playbackKind: options.needsTranscoding ? { type: "transcode" } : { type: "directPlay" },
+    rendition: null,
+    available: [],
+    mediaSourceId: options.mediaSourceId ?? null,
+    playSessionId: null,
+    needsTranscoding: options.needsTranscoding,
+  };
+}
+
 /**
 * Narrow seam the owning component provides so the adapter can execute the
 * element/HLS-coupled parts of a control action without re-implementing the
@@ -36,8 +70,16 @@ export interface Html5ElementBridge {
  /** Current seek offset (seconds) for transcoded streams. */
  getSeekOffset(): number;
  setSeekOffset(offset: number): void;
-  /** Update the stream URL the component renders (triggers its HLS $effect). */
-  setStreamUrl(url: string): void;
+  /**
+   * Update the stream the component renders (triggers its HLS $effect).
+   *
+   * Carries the whole [`StreamSelection`], not just the URL: the component's
+   * effect has to know the transport to choose a loader, and deriving that from
+   * the URL is the substring check DR-224 removes.
+   *
+   * TRACES: UR-079 | DR-224
+   */
+  setStreamSelection(selection: StreamSelection): void;
  /** Tear down the component-owned hls.js instance (dual-audio prevention). */
  destroyHls(): void;
  /** Media source id for seek/audio-track URLs. */
@@ -86,12 +128,13 @@ export class Html5PlayerAdapter implements PlayerAdapter {
    this.attachedElement = element;
  }

-  async load(streamUrl: string, _options: PlayerLoadOptions): Promise<void> {
+  async load(streamUrl: string, options: PlayerLoadOptions): Promise<void> {
    // The component's reactive HLS $effect performs the actual attach/load when
-    // the stream URL is set; loading is therefore driven by setStreamUrl. The
-    // component's canplay/frag-buffered path reports readiness through the host.
+    // the selection is set; loading is therefore driven by setStreamSelection.
+    // The component's canplay/frag-buffered path reports readiness through the
+    // host.
    this.bridge.setSeekOffset(0);
-    this.bridge.setStreamUrl(streamUrl);
+    this.bridge.setStreamSelection(selectionForLoad(streamUrl, options));
    this.host.onState("loading");
  }

@@ -171,12 +214,12 @@ export class Html5PlayerAdapter implements PlayerAdapter {
   *
   * TRACES: UR-004, UR-005 | DR-181 | UT-183
   */
-  async reloadSource(url: string, positionSeconds: number): Promise<void> {
+  async reloadSource(selection: StreamSelection, positionSeconds: number): Promise<void> {
    const el = this.element;
    if (!el) {
-      // Still update the stream URL so the component's HLS $effect can pick it up.
+      // Still update the selection so the component's HLS $effect can pick it up.
      this.bridge.setSeekOffset(0);
-      this.bridge.setStreamUrl(url);
+      this.bridge.setStreamSelection(selection);
      return;
    }
    const wasPlaying = !el.paused;
@@ -189,7 +232,7 @@ export class Html5PlayerAdapter implements PlayerAdapter {
    await new Promise((r) => setTimeout(r, 100));
    // The reloaded stream begins at the item's zero, so there is no base to add.
    this.bridge.setSeekOffset(0);
-    this.bridge.setStreamUrl(url);
+    this.bridge.setStreamSelection(selection);
    // A source that never becomes playable is a failed reload, not a slow one:
    // the caller (quality switch, transcoded seek) has to know so it can revert
    // its selection and surface the error instead of leaving the UI claiming a
@@ -28,6 +28,23 @@ vi.mock("$lib/api/bindings", () => ({
 import { NativePlayerAdapter } from "./nativeAdapter";
 import type { AdapterHost } from "./types";

+/**
+ * A `StreamSelection` for tests that only care about the URL. Transcoded HLS is
+ * what these paths exercised before the contract carried a transport.
+ */
+function testSelection(url: string, transport: "hls" | "progressive" | "localFile" = "hls") {
+  return {
+    url,
+    transport: { type: transport },
+    playbackKind: { type: transport === "hls" ? "transcode" : "directPlay" },
+    rendition: null,
+    available: [],
+    mediaSourceId: null,
+    playSessionId: null,
+    needsTranscoding: transport === "hls",
+  } as import("$lib/api/bindings").StreamSelection;
+}
+
 function makeHost(): AdapterHost {
  return {
    onState: vi.fn(),
@@ -67,7 +84,7 @@ describe("NativePlayerAdapter", () => {
  it("records position on seek/reload primitives (backend does the real work)", async () => {
    await adapter.seekElement(55, 0);
    expect(adapter.getPosition()).toBe(55);
-    await adapter.reloadSource("ignored", 200);
+    await adapter.reloadSource(testSelection("ignored"), 200);
    expect(adapter.getPosition()).toBe(200);
  });

@@ -1,3 +1,4 @@
+import type { StreamSelection } from "$lib/api/bindings";
 /**
 * NativePlayerAdapter — the Android/ExoPlayer PlayerAdapter implementation.
 *
@@ -89,7 +90,7 @@ export class NativePlayerAdapter implements PlayerAdapter {
   * performed the reload+seek internally as part of the seek decision; nothing
   * to do on the frontend beyond recording position.
   */
-  async reloadSource(_url: string, offset: number): Promise<void> {
+  async reloadSource(_selection: StreamSelection, offset: number): Promise<void> {
    this.position = offset;
  }

@@ -1,3 +1,4 @@
+import type { StreamSelection } from "$lib/api/bindings";
 /**
 * PlayerAdapter contract — the decoupled boundary between the UI/backend and a
 * concrete video player implementation (Linux HTML5+hls.js, or Android native).
@@ -42,6 +43,19 @@ export interface PlayerLoadOptions {
  knownDuration: number;
  /** Subtitle tracks available for this media. */
  subtitleTracks: SubtitleTrackInput[];
+  /**
+   * The backend's decision about this stream, when it made one.
+   *
+   * Present for anything negotiated through `repository_get_stream_selection`.
+   * Null for the paths that never negotiate — a local file, a live channel, a
+   * plugin's direct URL — where the adapter falls back to what the other
+   * options already say rather than to reading the URL.
+   *
+   * TRACES: UR-079 | DR-224
+   */
+  selection?: StreamSelection | null;
+  /** The source is a file on disk (or the loopback server in front of one). */
+  isLocalFile?: boolean;
 }

 /**
@@ -106,12 +120,16 @@ export interface PlayerAdapter {
  seekElement(positionSeconds: number, offset: number): Promise<void>;

  /**
-   * Compound reload: swap to `url` and resume at `offset` seconds. Runs the
-   * invariant mechanical sequence for this platform (html5: pause → hls teardown
-   * → clear src → set new url → wait ready → resume; native: ExoPlayer setMediaItem
-   * + seekTo). No decision is made here — the backend already decided to reload.
+   * Compound reload: swap to `selection` and resume at `offset` seconds. Runs
+   * the invariant mechanical sequence for this platform (html5: pause → hls
+   * teardown → clear src → set new selection → wait ready → resume; native:
+   * ExoPlayer setMediaItem + seekTo). No decision is made here — the backend
+   * already decided to reload, and `selection.transport` says how to open it, so
+   * no adapter has to infer that from the URL.
+   *
+   * TRACES: UR-079 | DR-224
   */
-  reloadSource(url: string, offset: number): Promise<void>;
+  reloadSource(selection: StreamSelection, offset: number): Promise<void>;

  setVolume(volume: number): void; // 0..1
  setMuted(muted: boolean): void;
@@ -1,3 +1,4 @@
+import type { StreamSelection } from "$lib/api/bindings";
 /**
 * Webview audio adapter — plays audio-only media through a hidden `<audio>`
 * element on platforms with no native audio backend (currently Windows).
@@ -104,8 +105,8 @@ export class WebviewAudioAdapter implements PlayerAdapter {
  }

  /** No transcode-reload concept for direct audio; treat as a fresh load. */
-  async reloadSource(url: string, offset: number): Promise<void> {
-    await this.load(url, {
+  async reloadSource(selection: StreamSelection, offset: number): Promise<void> {
+    await this.load(selection.url, {
      mediaId: "",
      mediaSourceId: null,
      needsTranscoding: false,