Files
jellytau/src/lib/components/player/VideoPlayer.nativeReveal.test.ts
T
dtourolle 109700b949 feat(playback): let Rust decide what stream to play, and say so
Playing a video meant asking the server to re-encode it, always. That
decision was made nowhere and written down nowhere, so whoever needed it
re-derived it downstream — the player worked out whether it had been handed
a playlist by looking for ".m3u8" in the URL, in two places. A viewer paid
for a transcode of a file their device could have played untouched, and the
app could not tell them which it was.

One negotiation now produces one self-describing StreamSelection — direct
play, remux or transcode; over a playlist, a plain HTTP file, or a local one
— and every renderer consumes that same answer.

Measured against the development server (Jellyfin 10.11.5), 400 items
sampled for codec mix and 40 put through a real PlaybackInfo negotiation
per profile:

  Linux / WebKitGTK (h264 only, 2ch)          3/40 —  7% direct play
  Android / ExoPlayer (hevc, ac3/eac3, 6ch)  34/40 — 85% direct play

The library is ~80% hevc, which is why the two diverge so hard. The payoff
is overwhelmingly Android, where 85% of plays were starting a transcode
nobody needed. Linux stays near 7% until libmpv decodes the picture — the
h264-only profile is a WebKitGTK constraint, not a JellyTau choice.

DR-219  StreamSelection: url + tagged Transport (hls/progressive/localFile)
        + PlaybackKind (directPlay/directStream/transcode) + the negotiated
        rendition + this source's ladder + a needs_transcoding flag derived
        in Rust so the rule is answered once. Both enums are serde-tagged
        so the frontend matches a discriminant, not a substring. The paths
        that never negotiate get the same shape from Rust rather than
        assembling one — media_local_selection for a downloaded file,
        LiveStreamInfo.transport for a live channel — so there is no second
        place where a transport is decided.

DR-220  The ceiling becomes two levels: a durable device default (Settings,
        persisted) and a per-playback override the in-player picker sets.
        The picker had called itself a "this film, this connection" control
        since it was written but wrote the process-wide default, so dropping
        one awkward film to 2 Mbps silently capped every video played
        afterwards for the rest of the process, with Settings still showing
        the old value. The override is cleared whenever playback moves to a
        new item, which stops it surviving into an autoplayed next episode.
        effective_streaming_quality() is the single resolution point.

DR-221  The quality picker is filled from what this media source can offer.
        Rust marks a rung exceeds_source when its ceiling is at or above the
        source's own bitrate — such a rung is another way to spell Original
        — and the frontend does not draw those. Original is never marked; a
        source whose bitrate the server does not report marks nothing, which
        keeps every rung offered.

DR-222  Direct play and direct stream are negotiated, with two client-side
        overrides on top because the server's answer is right about the file
        and wrong about what this app will do with it: undecodable audio
        (Jellyfin 10.11.5 honours a DirectPlayProfile's container and video
        codec but ignores its audio codec, so it offers direct play for an
        E-AC-3 track the webview renders in silence) and a viewer-pinned
        audio track the file does not default to. A direct stream is a remux
        and is deliberately not counted as transcoding.

DR-223  Dropped on measurement, not deferred. A master playlist from this
        server carries exactly one EXT-X-STREAM-INF: Jellyfin builds it from
        the single rendition the request asked for rather than publishing a
        ladder. So there is no adaptation for hls.js to be preserving and
        none mpv would lose — the claim that there was, in
        playback-backend-unification.md, does not hold. Recorded rather than
        deleted because it is a measurement: a server that does publish a
        ladder would change the answer.

DR-224  Every backend consumes the same selection. The queue item carries
        the transport, so player_seek_video picks its seek strategy from the
        backend's decision instead of the last stream_url.contains(".m3u8")
        in the codebase. Items queued by a path that never negotiated carry
        None and fall back to needs_transcoding, which is exact rather than
        a guess because every transcode this app requests is HLS (DR-140).

The frontend loader decision moves to streamTransport.ts so it can be
tested: the two cases that pin it are the ones that failed against the old
implementation — a progressive stream whose URL contains ".m3u8" must not
get an HLS loader, and an HLS stream whose URL contains none must.

Also verified the URL the direct-play branch builds actually serves playable
bytes: 206, video/mp4, valid ISO-BMFF, and a mid-file range works, so
seeking a direct play works.

The spec is folded into docs/architecture/{01,02,03} and deleted, per the
rule that docs/specs holds only work that has not shipped. DR-121 leaves
read-through-media-cache.md with a pointer; that spec keeps its capture half.

Not verified: real playback on a device. Direct play changes what actually
gets played, and neither fixtures nor curl prove the WebKitGTK and ExoPlayer
paths render it.
2026-08-22 13:45:03 +02:00

359 lines
13 KiB
TypeScript

/**
* VideoPlayer native-path reveal tests (Android / ExoPlayer)
*
* Reproduces "native video plays as audio with no picture" (DR-172).
*
* The poster/title card is an opaque `bg-black` overlay drawn while
* `isMediaReady` is false. Every signal that clears it — `canplay`,
* `loadedmetadata`, hls.js `FRAG_BUFFERED`, the `playing` event and two
* `readyState` timeouts — comes from the HTML5 `<video>` element. On the native
* path there is no such element, so nothing ever cleared it: ExoPlayer decoded
* and fed its SurfaceView correctly the whole time, behind a black div.
*
* These tests pin the **flag-on** path: the backend reports native, the user
* opted in, and the video area must be revealed by the *backend's* own signals.
*
* TRACES: UR-003, UR-004, UR-041 | DR-182 | UT-185
*/
import { describe, it, expect, vi, beforeEach } from "vitest";
// ---- Mocks (must precede component import) --------------------------------
const channelHandlers: Record<string, (event: any) => void> = {};
// The native path is what these tests guard, so the opt-in flag is mocked ON.
// Stated explicitly rather than inherited: the default has moved twice
// (DR-161 on, DR-172 off) and a test that inherits it silently changes meaning.
vi.mock("$lib/stores/nativeVideo", async (importOriginal) => {
const actual = await importOriginal<typeof import("$lib/stores/nativeVideo")>();
return {
...actual,
experimentalNativeVideo: {
subscribe: (run: (v: boolean) => void) => {
run(true);
return () => {};
},
set: () => {},
current: () => true,
},
};
});
vi.mock("@tauri-apps/api/event", () => ({
listen: vi.fn(async (channel: string, handler: any) => {
channelHandlers[channel] = handler;
return () => {
delete channelHandlers[channel];
};
}),
}));
vi.mock("@tauri-apps/api/core", () => ({
invoke: vi.fn(),
}));
const playerPlayItem = vi.fn(async () => ({
// What Android reports: native ExoPlayer backend, no HTML5 element.
useHtml5Element: false,
backend: "exoplayer",
state: { kind: "playing" },
}));
const playerStop = vi.fn(async () => ({}));
const playerReportState = vi.fn(async () => null);
vi.mock("$lib/api/bindings", () => ({
commands: {
playerPlayItem: (...a: any[]) => playerPlayItem(...(a as [])),
playerStop: (...a: any[]) => playerStop(...(a as [])),
playerReportState: (...a: any[]) => playerReportState(...(a as [])),
playerReportPosition: vi.fn(async () => null),
playerReportMediaLoaded: vi.fn(async () => null),
playerSeek: vi.fn(async () => ({})),
playerPlay: vi.fn(async () => ({})),
playerPause: vi.fn(async () => ({})),
playerToggle: vi.fn(async () => ({ state: "playing" })),
playerSeekVideo: vi.fn(async (_h: string, position: number) => ({
strategy: "native",
position,
})),
playerSetSubtitleTrack: vi.fn(async () => ({})),
playerSwitchAudioTrack: vi.fn(async () => ({})),
playerSetSleepTimer: vi.fn(async (mode: any) => ({ mode, remainingSeconds: 0 })),
playerCancelSleepTimer: vi.fn(async () => ({
mode: { kind: "off" },
remainingSeconds: 0,
})),
playerGetStreamingQualities: vi.fn(async () => []),
playerGetVideoSettings: vi.fn(async () => ({ streamingQuality: "original" })),
storageGetSeriesAudioPreference: vi.fn(async () => null),
storageSaveSeriesAudioPreference: vi.fn(async () => ({})),
},
events: {
playerStatusEvent: { listen: vi.fn(async () => () => {}) },
},
}));
vi.mock("$lib/stores/auth", () => ({
auth: {
getUserId: () => "user-1",
getRepository: () => ({
getHandle: () => "repo-1",
getSubtitleUrl: async () => "",
jrayActorsAt: async () => [],
}),
},
}));
vi.mock("$app/navigation", () => ({
goto: vi.fn(),
}));
// The immersive bridge is native-only; assert the call rather than its effect.
const enterImmersive = vi.fn();
vi.mock("$lib/utils/immersive", () => ({
enterImmersive: (...a: any[]) => enterImmersive(...a),
exitImmersive: vi.fn(),
isImmersiveSupported: () => true,
}));
import { render, waitFor } from "@testing-library/svelte";
import { tick } from "svelte";
import VideoPlayer from "./VideoPlayer.svelte";
import { player } from "$lib/stores/player";
import type { MediaItem } from "$lib/api/types";
/**
* A `StreamSelection` for tests that only care about the URL. Transcoded HLS is
* what these paths exercised before the contract carried a transport.
*/
function testSelection(url: string, transport: "hls" | "progressive" | "localFile" = "hls") {
return {
url,
transport: { type: transport },
playbackKind: { type: transport === "hls" ? "transcode" : "directPlay" },
rendition: null,
available: [],
mediaSourceId: null,
playSessionId: null,
needsTranscoding: transport === "hls",
} as import("$lib/api/bindings").StreamSelection;
}
function makeEpisode(): MediaItem {
return {
id: "ep1",
name: "Episode 1",
kind: "episode",
durationMs: 24 * 60 * 1000,
} as MediaItem;
}
async function mountNativePlayer() {
const utils = render(VideoPlayer, {
props: {
media: makeEpisode(),
selection: testSelection("http://server/videos/ep1/master.m3u8"),
mediaSourceId: "src-1",
needsTranscoding: false,
onClose: vi.fn(),
},
});
await waitFor(() => expect(playerPlayItem).toHaveBeenCalled());
// The native path must NOT be overridden to HTML5 and must NOT be stopped —
// if it were, these tests would be guarding the HTML5 path by accident.
await waitFor(() => expect(utils.container.querySelector("video")).toBeNull());
expect(playerStop).not.toHaveBeenCalled();
return utils;
}
/** The opaque poster/title card drawn while the media is not yet revealed. */
function poster(container: HTMLElement): HTMLElement | null {
return container.querySelector('[data-testid="video-poster"]');
}
/**
* Report backend playback state the way the app actually does.
*
* NOT via `player://position-update` / `player://state-changed`: those channels
* are **never emitted by the backend**, which is exactly the trap this test
* exists to avoid. An earlier version of it fired those handlers by hand, went
* green, and guarded nothing — on the device the poster stayed up while
* ExoPlayer played behind it. `playerEvents.ts` feeds the `player` store, and
* the store is what the component must read.
*/
async function backendReports(kind: "playing" | "paused" | "error", position = 0, duration = 0) {
const media = makeEpisode();
if (kind === "playing") player.setPlaying(media, position, duration);
else if (kind === "paused") player.setPaused(media, position, duration);
else player.setError("Decoder failed", media);
await tick();
}
describe("VideoPlayer native path reveals the video (DR-172)", () => {
beforeEach(() => {
vi.clearAllMocks();
for (const key of Object.keys(channelHandlers)) delete channelHandlers[key];
player.setIdle();
});
it("keeps the poster up until the backend reports something", async () => {
const { container } = await mountNativePlayer();
// Nothing has been heard from ExoPlayer yet, so the title card is correct.
expect(poster(container)).not.toBeNull();
});
it("clears the poster when the backend reports playing", async () => {
const { container } = await mountNativePlayer();
await backendReports("playing", 0, 1440);
// The surface is rendering behind the webview; an opaque overlay over it is
// exactly the "audio with no picture" defect.
await waitFor(() => expect(poster(container)).toBeNull());
});
it("clears the poster when the backend reports a paused position with a duration", async () => {
const { container } = await mountNativePlayer();
// Backstop for a backend that starts paused: a position carrying a real
// duration means the media is loaded and the surface has content,
// mirroring the HTML5 readyState fallback.
await backendReports("paused", 12, 1440);
await waitFor(() => expect(poster(container)).toBeNull());
});
it("clears the play overlay when the backend resumes after a pause (DR-186)", async () => {
const { container } = await mountNativePlayer();
await backendReports("paused", 5, 1440);
await waitFor(() =>
expect(container.querySelector('[data-testid="play-overlay"]')).not.toBeNull(),
);
await backendReports("playing", 6, 1440);
// This overlay is `bg-black/30` across the whole video area: left up, it
// both dims and covers the ExoPlayer surface while it plays. Before the
// mirror, nothing after init could take it down, because the only other
// writer was the never-emitted `player://state-changed` channel.
await waitFor(() => expect(container.querySelector('[data-testid="play-overlay"]')).toBeNull());
});
it("raises the play overlay again when the backend reports paused (DR-186)", async () => {
const { container } = await mountNativePlayer();
await backendReports("playing", 5, 1440);
await waitFor(() => expect(container.querySelector('[data-testid="play-overlay"]')).toBeNull());
await backendReports("paused", 6, 1440);
// The mirror has to work in both directions, or pausing leaves no affordance
// to resume.
await waitFor(() =>
expect(container.querySelector('[data-testid="play-overlay"]')).not.toBeNull(),
);
});
it("hides the system bars on entry, not only on the fullscreen button (DR-187)", async () => {
await mountNativePlayer();
// The player owns the whole screen; on the native path the system bars would
// otherwise sit directly on top of the ExoPlayer surface.
expect(enterImmersive).toHaveBeenCalled();
});
it("hides the control bar once playback starts, however late (DR-189)", async () => {
// Reproduce the device sequence: the backend is still starting when the
// player mounts, so playback begins *after* the first countdown window.
playerPlayItem.mockResolvedValueOnce({
useHtml5Element: false,
backend: "exoplayer",
state: { kind: "loading" },
} as any);
vi.useFakeTimers();
try {
const utils = render(VideoPlayer, {
props: {
media: makeEpisode(),
selection: testSelection("http://server/videos/ep1/master.m3u8"),
mediaSourceId: "src-1",
needsTranscoding: false,
onClose: vi.fn(),
},
});
await vi.advanceTimersByTimeAsync(50);
// The three seconds after entry elapse while the backend is still
// starting, so the bar correctly stays up. This is the exact window that
// defeated the first attempt: a one-shot timer armed on entry fired here,
// declined, and was never re-armed.
await vi.advanceTimersByTimeAsync(3500);
expect(utils.container.querySelector("[data-player-controls]")?.className).not.toContain(
"opacity-0",
);
// Playback starts late; the countdown has to restart on its own.
player.setPlaying(makeEpisode(), 5, 1440);
await vi.advanceTimersByTimeAsync(3500);
await vi.waitFor(() =>
expect(utils.container.querySelector("[data-player-controls]")?.className).toContain(
"opacity-0",
),
);
} finally {
vi.useRealTimers();
}
});
it("never reports webview element state on the native path (DR-195)", async () => {
// The report that mattered came from the 10-second progress interval, so
// the test has to reach it: the interval needs `onReportProgress` wired and
// `isPlaying` true, then time has to pass. Asserting on a freshly mounted
// player proves nothing — an earlier version of this test did exactly that
// and passed with the guard deleted.
vi.useFakeTimers();
try {
const utils = render(VideoPlayer, {
props: {
media: makeEpisode(),
selection: testSelection("http://server/videos/ep1/master.m3u8"),
mediaSourceId: "src-1",
needsTranscoding: false,
onClose: vi.fn(),
onReportProgress: vi.fn(),
},
});
await vi.advanceTimersByTimeAsync(100);
expect(utils.container.querySelector("video")).toBeNull();
// Backend playing, so the interval's `isPlaying` guard is satisfied.
player.setPlaying(makeEpisode(), 5, 1440);
await vi.advanceTimersByTimeAsync(25_000);
// `html5_playing` is Rust's record of "a webview element is active", and
// `toggle_playback`/`play`/`pause` all route transport to that element
// whenever it is set. Reporting it with no element in existence is what
// left the pause button dead on the native path — from the surface tap,
// the control bar, and a direct `player_toggle` invocation alike — while
// seek and skip kept working, because they decide elsewhere.
expect(playerReportState).not.toHaveBeenCalled();
} finally {
vi.useRealTimers();
}
});
it("does not clear the poster on an errored backend", async () => {
const { container } = await mountNativePlayer();
await backendReports("error");
// Revealing here would replace the title card with a transparent hole
// showing the launcher through the app.
expect(poster(container)).not.toBeNull();
});
});