mpv now decodes video on Linux, drawn into a framebuffer we own and blitted
into the default vbox's draw handler. Tauri's widget tree is untouched, so an
upgrade that assumes its own layout cannot invalidate this. Direct play means
the original file, hardware decoding, and no server transcode at all — where
previously every desktop video was re-encoded to h264 for the browser engine,
whatever the file actually was. Off by default: JELLYTAU_NATIVE_VIDEO=1.
That settles finding 2 of playback-backend-unification.md — "native video
cannot be composited with a Tauri webview" — by demonstration rather than
argument, on X11 and Wayland both.
Turning it on exposed nine defects, none of them mpv's. Each was the same
mistake in a different place: a capability written down as a compile-time fact
about the platform, or a state asserted instead of confirmed.
DR-238/246 a seek routed by the stream's container rather than by what the
engine could do with it - correct only while one player handled
those streams, silent the moment another did
DR-239 a property handled but never observed, so the play/pause button
waited for an event that could not arrive
DR-240 fullscreen expanding the document while the window stayed put
DR-241 a seek issued before the engine had a file, failed, and discarded
- which is why resume began at zero
DR-247 a Linux-only gate outliving the caller that made it Linux-only,
breaking the Android build outright
DR-250 a stop aimed at whichever renderer bookkeeping believed was in
charge, missing the one actually making sound
DR-251 a duration of zero believed, leaving the seek bar no scale
DR-252 a junk float converted to a Duration, panicking the backend the
instant a length-less stream appeared
So the MediaPlayer contract (DR-242 … DR-247): `open` carries a start position,
so no caller sequences load-then-seek and none can race an engine's load;
`seek` states a destination and leaves in-place-versus-re-open to the engine;
`snapshot` is one coherent read; and `Phase::Opening` names the window where
intent used to be lost. One conformance suite runs against every engine —
FakePlayer and mpv under cargo test, ExoPlayer instrumented on a device — so an
engine is either correct or visibly failing.
Two of the nine were introduced during this work and caught on hardware, not by
any suite: an over-broad capability that grouped ExoPlayer with mpv, and the
Duration panic. The suites test engines that behave. That is recorded in
docs/native-player-verification.md, which asks for the exact action sequences
that found them.
Verified: all automated gates, conformance (mpv 9/9, legacy 8/9 by design,
ExoPlayer 7/7 on device), and manual desktop and Android passes on real
hardware.
Known open and deliberately shipped: resume reads local progress and never the
server's; the background-audio handoff still declares a state swap it does not
confirm (the symptom is now impossible, the race is not); and `bun run
android:dev` builds an APK carrying the release application id, whose failure
message advises an uninstall that would destroy app data. Fix that last one
before anyone else builds for Android.
Squashed from worktree-linux-native-video, which keeps the per-defect history.
359 lines
13 KiB
TypeScript
359 lines
13 KiB
TypeScript
/**
|
|
* VideoPlayer native-path reveal tests (Android / ExoPlayer)
|
|
*
|
|
* Reproduces "native video plays as audio with no picture" (DR-172).
|
|
*
|
|
* The poster/title card is an opaque `bg-black` overlay drawn while
|
|
* `isMediaReady` is false. Every signal that clears it — `canplay`,
|
|
* `loadedmetadata`, hls.js `FRAG_BUFFERED`, the `playing` event and two
|
|
* `readyState` timeouts — comes from the HTML5 `<video>` element. On the native
|
|
* path there is no such element, so nothing ever cleared it: ExoPlayer decoded
|
|
* and fed its SurfaceView correctly the whole time, behind a black div.
|
|
*
|
|
* These tests pin the **flag-on** path: the backend reports native, the user
|
|
* opted in, and the video area must be revealed by the *backend's* own signals.
|
|
*
|
|
* TRACES: UR-003, UR-004, UR-041 | DR-182 | UT-185
|
|
*/
|
|
|
|
import { describe, it, expect, vi, beforeEach } from "vitest";
|
|
|
|
// ---- Mocks (must precede component import) --------------------------------
|
|
|
|
const channelHandlers: Record<string, (event: any) => void> = {};
|
|
|
|
// The native path is what these tests guard, so the opt-in flag is mocked ON.
|
|
// Stated explicitly rather than inherited: the default has moved twice
|
|
// (DR-161 on, DR-172 off) and a test that inherits it silently changes meaning.
|
|
vi.mock("$lib/stores/nativeVideo", async (importOriginal) => {
|
|
const actual = await importOriginal<typeof import("$lib/stores/nativeVideo")>();
|
|
return {
|
|
...actual,
|
|
experimentalNativeVideo: {
|
|
subscribe: (run: (v: boolean) => void) => {
|
|
run(true);
|
|
return () => {};
|
|
},
|
|
set: () => {},
|
|
current: () => true,
|
|
},
|
|
};
|
|
});
|
|
|
|
vi.mock("@tauri-apps/api/event", () => ({
|
|
listen: vi.fn(async (channel: string, handler: any) => {
|
|
channelHandlers[channel] = handler;
|
|
return () => {
|
|
delete channelHandlers[channel];
|
|
};
|
|
}),
|
|
}));
|
|
|
|
vi.mock("@tauri-apps/api/core", () => ({
|
|
invoke: vi.fn(),
|
|
}));
|
|
|
|
const playerPlayItem = vi.fn(async () => ({
|
|
// What Android reports: native ExoPlayer backend, no HTML5 element.
|
|
useHtml5Element: false,
|
|
backend: "exoplayer",
|
|
state: { kind: "playing" },
|
|
}));
|
|
const playerStop = vi.fn(async () => ({}));
|
|
const playerReportState = vi.fn(async () => null);
|
|
|
|
vi.mock("$lib/api/bindings", () => ({
|
|
commands: {
|
|
playerPlayItem: (...a: any[]) => playerPlayItem(...(a as [])),
|
|
playerStop: (...a: any[]) => playerStop(...(a as [])),
|
|
playerReportState: (...a: any[]) => playerReportState(...(a as [])),
|
|
playerReportPosition: vi.fn(async () => null),
|
|
playerReportMediaLoaded: vi.fn(async () => null),
|
|
playerSeek: vi.fn(async () => ({})),
|
|
playerPlay: vi.fn(async () => ({})),
|
|
playerPause: vi.fn(async () => ({})),
|
|
playerToggle: vi.fn(async () => ({ state: "playing" })),
|
|
playerSeekVideo: vi.fn(async (_h: string, position: number) => ({
|
|
strategy: "native",
|
|
position,
|
|
})),
|
|
playerSetSubtitleTrack: vi.fn(async () => ({})),
|
|
playerSwitchAudioTrack: vi.fn(async () => ({})),
|
|
playerSetSleepTimer: vi.fn(async (mode: any) => ({ mode, remainingSeconds: 0 })),
|
|
playerCancelSleepTimer: vi.fn(async () => ({
|
|
mode: { kind: "off" },
|
|
remainingSeconds: 0,
|
|
})),
|
|
playerGetStreamingQualities: vi.fn(async () => []),
|
|
playerGetVideoSettings: vi.fn(async () => ({ streamingQuality: "original" })),
|
|
storageGetSeriesAudioPreference: vi.fn(async () => null),
|
|
storageSaveSeriesAudioPreference: vi.fn(async () => ({})),
|
|
},
|
|
events: {
|
|
playerStatusEvent: { listen: vi.fn(async () => () => {}) },
|
|
},
|
|
}));
|
|
|
|
vi.mock("$lib/stores/auth", () => ({
|
|
auth: {
|
|
getUserId: () => "user-1",
|
|
getRepository: () => ({
|
|
getHandle: () => "repo-1",
|
|
getSubtitleUrl: async () => "",
|
|
jrayActorsAt: async () => [],
|
|
}),
|
|
},
|
|
}));
|
|
|
|
vi.mock("$app/navigation", () => ({
|
|
goto: vi.fn(),
|
|
}));
|
|
|
|
// The immersive bridge is native-only; assert the call rather than its effect.
|
|
const enterImmersive = vi.fn();
|
|
vi.mock("$lib/utils/immersive", () => ({
|
|
enterImmersive: (...a: any[]) => enterImmersive(...a),
|
|
exitImmersive: vi.fn(),
|
|
isImmersiveSupported: () => true,
|
|
}));
|
|
|
|
import { render, waitFor } from "@testing-library/svelte";
|
|
import { tick } from "svelte";
|
|
import VideoPlayer from "./VideoPlayer.svelte";
|
|
import { player } from "$lib/stores/player";
|
|
import type { MediaItem } from "$lib/api/types";
|
|
|
|
/**
|
|
* A `StreamSelection` for tests that only care about the URL. Transcoded HLS is
|
|
* what these paths exercised before the contract carried a transport.
|
|
*/
|
|
function testSelection(url: string, transport: "hls" | "progressive" | "localFile" = "hls") {
|
|
return {
|
|
url,
|
|
transport: { type: transport },
|
|
playbackKind: { type: transport === "hls" ? "transcode" : "directPlay" },
|
|
rendition: null,
|
|
available: [],
|
|
mediaSourceId: null,
|
|
playSessionId: null,
|
|
needsTranscoding: transport === "hls",
|
|
} as import("$lib/api/bindings").StreamSelection;
|
|
}
|
|
|
|
function makeEpisode(): MediaItem {
|
|
return {
|
|
id: "ep1",
|
|
name: "Episode 1",
|
|
kind: "episode",
|
|
durationMs: 24 * 60 * 1000,
|
|
} as MediaItem;
|
|
}
|
|
|
|
async function mountNativePlayer() {
|
|
const utils = render(VideoPlayer, {
|
|
props: {
|
|
media: makeEpisode(),
|
|
selection: testSelection("http://server/videos/ep1/master.m3u8"),
|
|
mediaSourceId: "src-1",
|
|
needsTranscoding: false,
|
|
onClose: vi.fn(),
|
|
},
|
|
});
|
|
|
|
await waitFor(() => expect(playerPlayItem).toHaveBeenCalled());
|
|
// The native path must NOT be overridden to HTML5 and must NOT be stopped —
|
|
// if it were, these tests would be guarding the HTML5 path by accident.
|
|
await waitFor(() => expect(utils.container.querySelector("video")).toBeNull());
|
|
expect(playerStop).not.toHaveBeenCalled();
|
|
return utils;
|
|
}
|
|
|
|
/** The opaque poster/title card drawn while the media is not yet revealed. */
|
|
function poster(container: HTMLElement): HTMLElement | null {
|
|
return container.querySelector('[data-testid="video-poster"]');
|
|
}
|
|
|
|
/**
|
|
* Report backend playback state the way the app actually does.
|
|
*
|
|
* NOT via `player://position-update` / `player://state-changed`: those channels
|
|
* are **never emitted by the backend**, which is exactly the trap this test
|
|
* exists to avoid. An earlier version of it fired those handlers by hand, went
|
|
* green, and guarded nothing — on the device the poster stayed up while
|
|
* ExoPlayer played behind it. `playerEvents.ts` feeds the `player` store, and
|
|
* the store is what the component must read.
|
|
*/
|
|
async function backendReports(kind: "playing" | "paused" | "error", position = 0, duration = 0) {
|
|
const media = makeEpisode();
|
|
if (kind === "playing") player.setPlaying(media, position, duration);
|
|
else if (kind === "paused") player.setPaused(media, position, duration);
|
|
else player.setError("Decoder failed", media);
|
|
await tick();
|
|
}
|
|
|
|
describe("VideoPlayer native path reveals the video (DR-172)", () => {
|
|
beforeEach(() => {
|
|
vi.clearAllMocks();
|
|
for (const key of Object.keys(channelHandlers)) delete channelHandlers[key];
|
|
player.setIdle();
|
|
});
|
|
|
|
it("keeps the poster up until the backend reports something", async () => {
|
|
const { container } = await mountNativePlayer();
|
|
// Nothing has been heard from ExoPlayer yet, so the title card is correct.
|
|
expect(poster(container)).not.toBeNull();
|
|
});
|
|
|
|
it("clears the poster when the backend reports playing", async () => {
|
|
const { container } = await mountNativePlayer();
|
|
|
|
await backendReports("playing", 0, 1440);
|
|
|
|
// The surface is rendering behind the webview; an opaque overlay over it is
|
|
// exactly the "audio with no picture" defect.
|
|
await waitFor(() => expect(poster(container)).toBeNull());
|
|
});
|
|
|
|
it("clears the poster when the backend reports a paused position with a duration", async () => {
|
|
const { container } = await mountNativePlayer();
|
|
|
|
// Backstop for a backend that starts paused: a position carrying a real
|
|
// duration means the media is loaded and the surface has content,
|
|
// mirroring the HTML5 readyState fallback.
|
|
await backendReports("paused", 12, 1440);
|
|
|
|
await waitFor(() => expect(poster(container)).toBeNull());
|
|
});
|
|
|
|
it("clears the play overlay when the backend resumes after a pause (DR-186)", async () => {
|
|
const { container } = await mountNativePlayer();
|
|
|
|
await backendReports("paused", 5, 1440);
|
|
await waitFor(() =>
|
|
expect(container.querySelector('[data-testid="play-overlay"]')).not.toBeNull(),
|
|
);
|
|
|
|
await backendReports("playing", 6, 1440);
|
|
|
|
// This overlay is `bg-black/30` across the whole video area: left up, it
|
|
// both dims and covers the ExoPlayer surface while it plays. Before the
|
|
// mirror, nothing after init could take it down, because the only other
|
|
// writer was the never-emitted `player://state-changed` channel.
|
|
await waitFor(() => expect(container.querySelector('[data-testid="play-overlay"]')).toBeNull());
|
|
});
|
|
|
|
it("raises the play overlay again when the backend reports paused (DR-186)", async () => {
|
|
const { container } = await mountNativePlayer();
|
|
|
|
await backendReports("playing", 5, 1440);
|
|
await waitFor(() => expect(container.querySelector('[data-testid="play-overlay"]')).toBeNull());
|
|
|
|
await backendReports("paused", 6, 1440);
|
|
|
|
// The mirror has to work in both directions, or pausing leaves no affordance
|
|
// to resume.
|
|
await waitFor(() =>
|
|
expect(container.querySelector('[data-testid="play-overlay"]')).not.toBeNull(),
|
|
);
|
|
});
|
|
|
|
it("hides the system bars on entry, not only on the fullscreen button (DR-187)", async () => {
|
|
await mountNativePlayer();
|
|
|
|
// The player owns the whole screen; on the native path the system bars would
|
|
// otherwise sit directly on top of the ExoPlayer surface.
|
|
expect(enterImmersive).toHaveBeenCalled();
|
|
});
|
|
|
|
it("hides the control bar once playback starts, however late (DR-189)", async () => {
|
|
// Reproduce the device sequence: the backend is still starting when the
|
|
// player mounts, so playback begins *after* the first countdown window.
|
|
playerPlayItem.mockResolvedValueOnce({
|
|
useHtml5Element: false,
|
|
backend: "exoplayer",
|
|
state: { kind: "loading" },
|
|
} as any);
|
|
vi.useFakeTimers();
|
|
try {
|
|
const utils = render(VideoPlayer, {
|
|
props: {
|
|
media: makeEpisode(),
|
|
selection: testSelection("http://server/videos/ep1/master.m3u8"),
|
|
mediaSourceId: "src-1",
|
|
needsTranscoding: false,
|
|
onClose: vi.fn(),
|
|
},
|
|
});
|
|
await vi.advanceTimersByTimeAsync(50);
|
|
|
|
// The three seconds after entry elapse while the backend is still
|
|
// starting, so the bar correctly stays up. This is the exact window that
|
|
// defeated the first attempt: a one-shot timer armed on entry fired here,
|
|
// declined, and was never re-armed.
|
|
await vi.advanceTimersByTimeAsync(3500);
|
|
expect(utils.container.querySelector("[data-player-controls]")?.className).not.toContain(
|
|
"opacity-0",
|
|
);
|
|
|
|
// Playback starts late; the countdown has to restart on its own.
|
|
player.setPlaying(makeEpisode(), 5, 1440);
|
|
await vi.advanceTimersByTimeAsync(3500);
|
|
|
|
await vi.waitFor(() =>
|
|
expect(utils.container.querySelector("[data-player-controls]")?.className).toContain(
|
|
"opacity-0",
|
|
),
|
|
);
|
|
} finally {
|
|
vi.useRealTimers();
|
|
}
|
|
});
|
|
|
|
it("never reports webview element state on the native path (DR-195)", async () => {
|
|
// The report that mattered came from the 10-second progress interval, so
|
|
// the test has to reach it: the interval needs `onReportProgress` wired and
|
|
// `isPlaying` true, then time has to pass. Asserting on a freshly mounted
|
|
// player proves nothing — an earlier version of this test did exactly that
|
|
// and passed with the guard deleted.
|
|
vi.useFakeTimers();
|
|
try {
|
|
const utils = render(VideoPlayer, {
|
|
props: {
|
|
media: makeEpisode(),
|
|
selection: testSelection("http://server/videos/ep1/master.m3u8"),
|
|
mediaSourceId: "src-1",
|
|
needsTranscoding: false,
|
|
onClose: vi.fn(),
|
|
onReportProgress: vi.fn(),
|
|
},
|
|
});
|
|
await vi.advanceTimersByTimeAsync(100);
|
|
expect(utils.container.querySelector("video")).toBeNull();
|
|
|
|
// Backend playing, so the interval's `isPlaying` guard is satisfied.
|
|
player.setPlaying(makeEpisode(), 5, 1440);
|
|
await vi.advanceTimersByTimeAsync(25_000);
|
|
|
|
// `html5_playing` is Rust's record of "a webview element is active", and
|
|
// `toggle_playback`/`play`/`pause` all route transport to that element
|
|
// whenever it is set. Reporting it with no element in existence is what
|
|
// left the pause button dead on the native path — from the surface tap,
|
|
// the control bar, and a direct `player_toggle` invocation alike — while
|
|
// seek and skip kept working, because they decide elsewhere.
|
|
expect(playerReportState).not.toHaveBeenCalled();
|
|
} finally {
|
|
vi.useRealTimers();
|
|
}
|
|
});
|
|
|
|
it("does not clear the poster on an errored backend", async () => {
|
|
const { container } = await mountNativePlayer();
|
|
|
|
await backendReports("error");
|
|
|
|
// Revealing here would replace the title card with a transparent hole
|
|
// showing the launcher through the app.
|
|
expect(poster(container)).not.toBeNull();
|
|
});
|
|
});
|