Files
jellytau/src/lib/components/player/VideoPlayer.tapSurface.test.ts
T
dtourolle 109700b949 feat(playback): let Rust decide what stream to play, and say so
Playing a video meant asking the server to re-encode it, always. That
decision was made nowhere and written down nowhere, so whoever needed it
re-derived it downstream — the player worked out whether it had been handed
a playlist by looking for ".m3u8" in the URL, in two places. A viewer paid
for a transcode of a file their device could have played untouched, and the
app could not tell them which it was.

One negotiation now produces one self-describing StreamSelection — direct
play, remux or transcode; over a playlist, a plain HTTP file, or a local one
— and every renderer consumes that same answer.

Measured against the development server (Jellyfin 10.11.5), 400 items
sampled for codec mix and 40 put through a real PlaybackInfo negotiation
per profile:

  Linux / WebKitGTK (h264 only, 2ch)          3/40 —  7% direct play
  Android / ExoPlayer (hevc, ac3/eac3, 6ch)  34/40 — 85% direct play

The library is ~80% hevc, which is why the two diverge so hard. The payoff
is overwhelmingly Android, where 85% of plays were starting a transcode
nobody needed. Linux stays near 7% until libmpv decodes the picture — the
h264-only profile is a WebKitGTK constraint, not a JellyTau choice.

DR-219  StreamSelection: url + tagged Transport (hls/progressive/localFile)
        + PlaybackKind (directPlay/directStream/transcode) + the negotiated
        rendition + this source's ladder + a needs_transcoding flag derived
        in Rust so the rule is answered once. Both enums are serde-tagged
        so the frontend matches a discriminant, not a substring. The paths
        that never negotiate get the same shape from Rust rather than
        assembling one — media_local_selection for a downloaded file,
        LiveStreamInfo.transport for a live channel — so there is no second
        place where a transport is decided.

DR-220  The ceiling becomes two levels: a durable device default (Settings,
        persisted) and a per-playback override the in-player picker sets.
        The picker had called itself a "this film, this connection" control
        since it was written but wrote the process-wide default, so dropping
        one awkward film to 2 Mbps silently capped every video played
        afterwards for the rest of the process, with Settings still showing
        the old value. The override is cleared whenever playback moves to a
        new item, which stops it surviving into an autoplayed next episode.
        effective_streaming_quality() is the single resolution point.

DR-221  The quality picker is filled from what this media source can offer.
        Rust marks a rung exceeds_source when its ceiling is at or above the
        source's own bitrate — such a rung is another way to spell Original
        — and the frontend does not draw those. Original is never marked; a
        source whose bitrate the server does not report marks nothing, which
        keeps every rung offered.

DR-222  Direct play and direct stream are negotiated, with two client-side
        overrides on top because the server's answer is right about the file
        and wrong about what this app will do with it: undecodable audio
        (Jellyfin 10.11.5 honours a DirectPlayProfile's container and video
        codec but ignores its audio codec, so it offers direct play for an
        E-AC-3 track the webview renders in silence) and a viewer-pinned
        audio track the file does not default to. A direct stream is a remux
        and is deliberately not counted as transcoding.

DR-223  Dropped on measurement, not deferred. A master playlist from this
        server carries exactly one EXT-X-STREAM-INF: Jellyfin builds it from
        the single rendition the request asked for rather than publishing a
        ladder. So there is no adaptation for hls.js to be preserving and
        none mpv would lose — the claim that there was, in
        playback-backend-unification.md, does not hold. Recorded rather than
        deleted because it is a measurement: a server that does publish a
        ladder would change the answer.

DR-224  Every backend consumes the same selection. The queue item carries
        the transport, so player_seek_video picks its seek strategy from the
        backend's decision instead of the last stream_url.contains(".m3u8")
        in the codebase. Items queued by a path that never negotiated carry
        None and fall back to needs_transcoding, which is exact rather than
        a guess because every transcode this app requests is HLS (DR-140).

The frontend loader decision moves to streamTransport.ts so it can be
tested: the two cases that pin it are the ones that failed against the old
implementation — a progressive stream whose URL contains ".m3u8" must not
get an HLS loader, and an HLS stream whose URL contains none must.

Also verified the URL the direct-play branch builds actually serves playable
bytes: 206, video/mp4, valid ISO-BMFF, and a mid-file range works, so
seeking a direct play works.

The spec is folded into docs/architecture/{01,02,03} and deleted, per the
rule that docs/specs holds only work that has not shipped. DR-121 leaves
read-through-media-cache.md with a pointer; that spec keeps its capture half.

Not verified: real playback on a device. Direct play changes what actually
gets played, and neither fixtures nor curl prove the WebKitGTK and ExoPlayer
paths render it.
2026-08-22 13:45:03 +02:00

248 lines
8.6 KiB
TypeScript

/**
* Behavioural regression tests for the video tap surface — rendered against the
* REAL component, not a hand-modelled DOM.
*
* TRACES: UR-005, UR-061 | DR-098 | UT-092
*
* Why this file exists:
*
* `tapGestures.test.ts` tests `registerTap` / `isControlSurfaceTouch` /
* `isSynthesizedTouchClick` as isolated pure functions. Every one of those tests
* passed while, on the device, in sequence: the player pause-looped, then
* pausing became impossible, then the bottom controls went dead, then
* double-tap-to-seek stopped working. The helpers were each behaving exactly as
* specified — the bugs were all in the *composition*: which element actually
* receives a tap once Svelte has re-rendered.
*
* Testing my own helpers could not catch that, and modelling the DOM by hand in
* a test just re-encodes the same wrong assumption. So these tests render
* VideoPlayer and dispatch real touch/click events at whatever element is
* genuinely on top, asserting user-visible outcomes ("a double tap seeks")
* rather than internals.
*
* The specific traps encoded here, each a bug that shipped:
* - pausing renders a full-screen <button> play overlay OVER the video, so the
* second tap of a double tap lands on a button, not the video;
* - the browser synthesizes a `click` after a touch tap, which must not toggle
* a second time, on ANY layered target;
* - the bottom controls bar must drive its own buttons and NOT the container's
* tap gestures.
*/
import { describe, it, expect, vi, beforeEach } from "vitest";
import { render } from "@testing-library/svelte";
import { tick } from "svelte";
import { invoke } from "@tauri-apps/api/core";
import VideoPlayer from "./VideoPlayer.svelte";
import { SEEK_FORWARD_SECONDS } from "./tapGestures";
/**
* A `StreamSelection` for tests that only care about the URL. Transcoded HLS is
* what these paths exercised before the contract carried a transport.
*/
function testSelection(url: string, transport: "hls" | "progressive" | "localFile" = "hls") {
return {
url,
transport: { type: transport },
playbackKind: { type: transport === "hls" ? "transcode" : "directPlay" },
rendition: null,
available: [],
mediaSourceId: null,
playSessionId: null,
needsTranscoding: transport === "hls",
} as import("$lib/api/bindings").StreamSelection;
}
// --- Mocks: everything VideoPlayer reaches for that is not the tap surface. ---
const toggleSpy = vi.fn();
const seekVideoSpy = vi.fn();
const seekSpy = vi.fn();
vi.mock("$app/navigation", () => ({ goto: vi.fn() }));
vi.mock("$lib/player", () => ({
playerController: {
toggle: (...a: unknown[]) => {
toggleSpy(...a);
return Promise.resolve();
},
seekVideo: (...a: unknown[]) => {
seekVideoSpy(...a);
return Promise.resolve();
},
seek: (...a: unknown[]) => {
seekSpy(...a);
return Promise.resolve();
},
setActiveAdapter: vi.fn(),
clearActiveAdapter: vi.fn(),
getActiveAdapter: vi.fn(() => null),
},
}));
vi.mock("$lib/player/adapters/rustReportHost", () => ({
createRustReportHost: () => ({
onState: vi.fn(),
onPosition: vi.fn(),
onMediaLoaded: vi.fn(),
onEnded: vi.fn(),
onError: vi.fn(),
onStreamUrlChanged: vi.fn(),
onBuffering: vi.fn(),
onReady: vi.fn(),
}),
}));
vi.mock("$lib/player/html5Adapter", () => ({
reportState: vi.fn(),
reportPosition: vi.fn(),
reportMediaLoaded: vi.fn(),
resetReporting: vi.fn(),
}));
vi.mock("$lib/utils/pictureInPicture", () => ({
isPipSupported: () => false,
enterPip: vi.fn(),
setAutoEnterEnabled: vi.fn(),
setHtml5VideoState: vi.fn(),
}));
vi.mock("$lib/stores/auth", () => ({
auth: {
getRepository: () => ({ getHandle: () => "h", jrayActorsAt: async () => [] }),
subscribe: (fn: (v: unknown) => void) => {
fn({ isAuthenticated: true });
return () => {};
},
},
}));
const MEDIA = {
id: "item-1",
name: "Test Episode",
type: "Episode",
runTimeTicks: 6_000_000_000, // 600s
} as any;
/** Dispatch a touch at (x, y) on whatever element is topmost there. */
function touchAt(el: Element, x: number) {
const touch = { clientX: x, clientY: 300 } as Touch;
el.dispatchEvent(
new TouchEvent("touchstart", {
bubbles: true,
cancelable: true,
touches: [touch] as unknown as Touch[],
}),
);
}
function renderPlayer() {
return render(VideoPlayer, {
props: { media: MEDIA, selection: testSelection("http://x/master.m3u8"), onClose: vi.fn() },
});
}
describe("VideoPlayer tap surface (real component)", () => {
beforeEach(() => {
vi.clearAllMocks();
// This file deliberately does NOT mock `$lib/api/bindings` — it renders the
// real component against the real bindings, which bottom out in the globally
// mocked `invoke`. That mock resolves `undefined` for every command, so the
// commands whose results are *rendered* have to be answered here: the
// quality picker assigns the result straight to state and then does
// `streamingQualities.length` in the template, which throws (asynchronously,
// outside any test) on undefined and fails the run with an unhandled error.
vi.mocked(invoke).mockImplementation(async (cmd: string) => {
switch (cmd) {
case "player_get_streaming_qualities":
return [];
case "player_get_video_settings":
return { streamingQuality: "original" };
default:
return undefined;
}
});
});
it("a single tap on the video toggles play/pause exactly once", async () => {
const { container } = renderPlayer();
const video = container.querySelector("video");
expect(video).toBeTruthy();
touchAt(video!, 900);
expect(toggleSpy).toHaveBeenCalledTimes(1);
});
it("the synthesized click after a tap does not toggle a second time", async () => {
const { container } = renderPlayer();
const video = container.querySelector("video")!;
touchAt(video, 900);
// The compatibility click the browser fires after a touch tap. detail=0 is
// how engines mark it; a late real-detail click is covered by the recency
// guard, which this exercises too since it lands immediately.
video.dispatchEvent(new MouseEvent("click", { bubbles: true, detail: 0 }));
expect(toggleSpy).toHaveBeenCalledTimes(1);
});
it("a double tap seeks even though the first tap raised the play overlay", async () => {
// THE regression this file exists for. On device the first tap pauses, which
// makes Svelte render a full-screen <button> play overlay over the video —
// so the SECOND tap lands on a button, not the video. A control-surface
// guard that does not know about that overlay discards it and seeking dies.
//
// Reproducing it requires the overlay to actually render, which means
// driving `isPlaying` the way the real element does: via its `pause` event.
vi.useFakeTimers();
try {
const { container } = renderPlayer();
const video = container.querySelector("video")!;
// Tap 1 on the video.
touchAt(video, 900);
// The element reports it paused → isPlaying=false → overlay renders.
video.dispatchEvent(new Event("pause"));
await Promise.resolve();
await tick();
const overlay = container.querySelector("[data-player-surface]");
expect(overlay, "the play overlay should be covering the video").toBeTruthy();
vi.advanceTimersByTime(120); // inside DOUBLE_TAP_WINDOW_MS
// Tap 2 lands on the OVERLAY, exactly as on device.
touchAt(overlay!, 900);
// Either seek route is acceptable — which one runs depends on whether a
// video adapter is registered. What must hold is that a seek happened, to
// roughly the forward-skip target.
const calls = [...seekVideoSpy.mock.calls, ...seekSpy.mock.calls];
expect(calls.length).toBe(1);
const [position] = calls[0];
expect(position).toBeGreaterThan(0);
expect(position).toBeLessThanOrEqual(SEEK_FORWARD_SECONDS);
} finally {
vi.useRealTimers();
}
});
it("tapping the bottom play/pause button toggles once, not twice", async () => {
const { container } = renderPlayer();
const controls = container.querySelector("[data-player-controls]");
expect(controls).toBeTruthy();
const playBtn = controls!.querySelector("button");
expect(playBtn).toBeTruthy();
// A real press: touchstart bubbles to the container's gesture handler, then
// the button's own click fires. Only ONE toggle may result.
touchAt(playBtn!, 40);
playBtn!.dispatchEvent(new MouseEvent("click", { bubbles: true, detail: 1 }));
expect(toggleSpy).toHaveBeenCalledTimes(1);
});
});