dtourolle/jellytau
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 40 Wiki Activity

feat(player): native video on Linux, and one contract for every player (v0.11.0)

Browse Source 
mpv now decodes video on Linux, drawn into a framebuffer we own and blitted
into the default vbox's draw handler. Tauri's widget tree is untouched, so an
upgrade that assumes its own layout cannot invalidate this. Direct play means
the original file, hardware decoding, and no server transcode at all — where
previously every desktop video was re-encoded to h264 for the browser engine,
whatever the file actually was. Off by default: JELLYTAU_NATIVE_VIDEO=1.

That settles finding 2 of playback-backend-unification.md — "native video
cannot be composited with a Tauri webview" — by demonstration rather than
argument, on X11 and Wayland both.

Turning it on exposed nine defects, none of them mpv's. Each was the same
mistake in a different place: a capability written down as a compile-time fact
about the platform, or a state asserted instead of confirmed.

  DR-238/246  a seek routed by the stream's container rather than by what the
              engine could do with it - correct only while one player handled
              those streams, silent the moment another did
  DR-239      a property handled but never observed, so the play/pause button
              waited for an event that could not arrive
  DR-240      fullscreen expanding the document while the window stayed put
  DR-241      a seek issued before the engine had a file, failed, and discarded
              - which is why resume began at zero
  DR-247      a Linux-only gate outliving the caller that made it Linux-only,
              breaking the Android build outright
  DR-250      a stop aimed at whichever renderer bookkeeping believed was in
              charge, missing the one actually making sound
  DR-251      a duration of zero believed, leaving the seek bar no scale
  DR-252      a junk float converted to a Duration, panicking the backend the
              instant a length-less stream appeared

So the MediaPlayer contract (DR-242 … DR-247): `open` carries a start position,
so no caller sequences load-then-seek and none can race an engine's load;
`seek` states a destination and leaves in-place-versus-re-open to the engine;
`snapshot` is one coherent read; and `Phase::Opening` names the window where
intent used to be lost. One conformance suite runs against every engine —
FakePlayer and mpv under cargo test, ExoPlayer instrumented on a device — so an
engine is either correct or visibly failing.

Two of the nine were introduced during this work and caught on hardware, not by
any suite: an over-broad capability that grouped ExoPlayer with mpv, and the
Duration panic. The suites test engines that behave. That is recorded in
docs/native-player-verification.md, which asks for the exact action sequences
that found them.

Verified: all automated gates, conformance (mpv 9/9, legacy 8/9 by design,
ExoPlayer 7/7 on device), and manual desktop and Android passes on real
hardware.

Known open and deliberately shipped: resume reads local progress and never the
server's; the background-audio handoff still declares a state swap it does not
confirm (the symptom is now impossible, the race is not); and `bun run
android:dev` builds an APK carrying the release application id, whose failure
message advises an uninstall that would destroy app data. Fix that last one
before anyone else builds for Android.

Squashed from worktree-linux-native-video, which keeps the per-defect history.
This commit is contained in:
dtourolle
2026-08-23 10:51:45 +02:00
parent 5fede123e7
commit 11d9d760d8
87 changed files with 15968 additions and 7508 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/routes/player/[id]/+page.svelte
+76 -51
View File
@@ -3,8 +3,8 @@
import { page } from "$app/stores";
import { goto } from "$app/navigation";
import { commands } from "$lib/api/bindings";
import { downloadedFilePath, resolveVideoSource } from "$lib/player/localSource";
import type { PlayQueueRequest } from "$lib/api/bindings";
import { downloadedFilePath } from "$lib/player/localSource";
import type { PlayQueueRequest, StreamSelection } from "$lib/api/bindings";
import type { MediaItem, MediaKind } from "$lib/api/types";
import { auth } from "$lib/stores/auth";
import { library } from "$lib/stores/library";
@@ -76,7 +76,15 @@
const hasNext = $derived($hasNextStore);
const hasPrevious = $derived($hasPreviousStore);
let currentMedia = $state<MediaItem | null>(null);
let streamUrl = $state<string | null>(null);
/**
* What to play, as the backend decided it. Null while still resolving.
*
* Replaces a bare URL string: the transport travels with it, so neither this
* page nor VideoPlayer has to work out whether the URL is a playlist.
*
* TRACES: UR-079 | DR-225
*/
let selection = $state<StreamSelection | null>(null);
let mediaSourceId = $state<string | null>(null);
let isVideo = $state(false);
let isLive = $state(false); // Whether this is a live stream (Live TV channel) - no seek/resume
@@ -94,7 +102,7 @@
// Which player component to render. Video without a stream URL is "pending"
// (still resolving), never audio — see playerSurface.ts.
const surface = $derived(resolvePlayerSurface({ isVideo, streamUrl }));
const surface = $derived(resolvePlayerSurface({ isVideo, streamUrl: selection?.url ?? null }));
onMount(() => {
// Start position polling (only for audio via MPV backend)
@@ -308,17 +316,17 @@
const fullPath = downloadedFilePath(storagePath, localDownload.filePath);
log.debug("loadAndPlay: Full local path:", fullPath);
// Serve the file over the loopback media server rather than the asset
// protocol: the asset protocol answers a range-less request with the
// entire file, so a downloaded film never finished loading. Rust mints
// the URL (it holds the port and the per-session token).
// TRACES: UR-071 | DR-137
const localUrl = await commands.mediaLocalUrl(fullPath);
log.debug("loadAndPlay: Local media URL resolved");
if (isVideo) {
// Local video files don't need transcoding and support native seeking
streamUrl = localUrl;
// Served over the loopback media server rather than the asset
// protocol: the asset protocol answers a range-less request with the
// entire file, so a downloaded film never finished loading. Rust mints
// the URL (it holds the port and the per-session token) and states the
// transport with it.
//
// A downloaded file is a direct play over a local transport, and Rust
// says so rather than this page assuming it.
// TRACES: UR-071 | DR-137, DR-225
selection = await commands.mediaLocalSelection(fullPath);
videoNeedsTranscoding = false;
// Use explicit startPosition, or fall back to retrieved progress from database
const effectivePosition = startPosition ?? retrievedProgressSeconds ?? 0;
@@ -346,7 +354,12 @@
} else {
// Online playback - get playback info from server
isOfflinePlayback = false;
const repo = auth.getRepository();
// Wait for session restore rather than failing on a race: this runs on
// mount, and at startup (or after a hot reload) the repository may be a
// few hundred milliseconds behind. Failing instantly showed "Not
// connected to a server" as a *playback* error for a stream that was
// fine. TRACES: UR-002, UR-004 | DR-013
const repo = await auth.waitForRepository();
if (isLive) {
// Live TV channels must be "opened" before streaming; the server returns
@@ -355,7 +368,19 @@
const liveInfo = await repo.openLiveStream(id);
log.debug("loadAndPlay: Live stream URL:", liveInfo.streamUrl);
mediaSourceId = liveInfo.mediaSourceId;
streamUrl = liveInfo.streamUrl;
selection = {
url: liveInfo.streamUrl,
// Rust's verdict, not a guess from the URL.
transport: liveInfo.transport,
playbackKind: { type: "transcode" },
rendition: null,
// A live channel has no ladder to offer: there is no source file to
// measure and no rendition to re-negotiate against.
available: [],
mediaSourceId: liveInfo.mediaSourceId,
playSessionId: liveInfo.playSessionId,
needsTranscoding: true,
};
videoNeedsTranscoding = true;
videoInitialPosition = 0;
isPlaying = true;
@@ -363,46 +388,46 @@
return;
}
log.debug("loadAndPlay: Getting playback info");
const playbackInfo = await repo.getPlaybackInfo(id);
log.debug("loadAndPlay: Got playback info, mediaSourceId:", playbackInfo.mediaSourceId);
if (isVideo) {
// Playback API now detects HEVC/10-bit and returns transcoded URL when needed
log.debug(
"loadAndPlay: Using video stream, directPlay:",
playbackInfo.directPlay,
"needsTranscoding:",
playbackInfo.needsTranscoding,
);
mediaSourceId = playbackInfo.mediaSourceId;
// Prefer a completed download over streaming. Audio has done this
// since the queue is built; video previously always streamed, so a
// downloaded film re-spent bandwidth already spent and would not play
// at all offline. Rust returns null when nothing is downloaded or the
// file has gone, so this falls back to the server on its own.
// TRACES: UR-071 | DR-123
// A downloaded file is served over the loopback media server, not the
// asset protocol — see DR-137. The URL is minted up front because
// resolveVideoSource stays pure/synchronous.
// TRACES: UR-071 | DR-123, DR-137
//
// Checked *first* so the streaming path below negotiates exactly once:
// asking for a `PlaybackInfo` and then a stream selection meant two
// negotiations per load, and each one claims a transcode identity and
// retires the previous — so the server started a job only to be told
// to stop it a moment later. Observed in the log as a pair of
// `[StreamSelection]` lines for one play.
//
// TRACES: UR-071 | DR-123, DR-137, DR-225
const localPath = await commands.playerLocalMediaPath(id);
const localUrl = localPath ? await commands.mediaLocalUrl(localPath) : null;
const source = resolveVideoSource({
localPath,
remoteUrl: playbackInfo.streamUrl,
remoteNeedsTranscoding: playbackInfo.needsTranscoding,
toAssetUrl: () => localUrl ?? "",
});
streamUrl = source.url;
videoNeedsTranscoding = source.needsTranscoding;
log.debug(
source.isLocal
? "loadAndPlay: Playing downloaded file from disk"
: `loadAndPlay: Using stream URL: ${streamUrl}`,
);
if (localPath) {
// A downloaded file is a direct play over a local transport, served
// by the loopback media server rather than the asset protocol
// (DR-137). Its media-source id still comes from the server, since
// that is what subtitle URLs are keyed by.
selection = await commands.mediaLocalSelection(localPath);
videoNeedsTranscoding = false;
mediaSourceId = (await repo.getPlaybackInfo(id)).mediaSourceId;
log.debug("loadAndPlay: Playing downloaded file from disk");
} else {
// Rust negotiates direct play vs direct stream vs transcode against
// the device profile and the ceiling in force, and returns the
// transport and the media-source id with it. This page no longer
// decides — or separately asks for — any of that.
// TRACES: UR-070, UR-079 | DR-225, DR-227, DR-228
selection = await repo.getStreamSelection(id, null, null);
mediaSourceId = selection.mediaSourceId;
// Rust's own verdict — "which kinds count as transcoding" is a
// domain rule, and a direct *stream* is a remux that does not.
videoNeedsTranscoding = selection.needsTranscoding;
log.debug(
`loadAndPlay: ${selection.playbackKind.type} over ${selection.transport.type}`,
);
}
// Set initial position for the video player to seek to after load.
// Use explicit startPosition, or fall back to retrieved progress.
@@ -847,10 +872,10 @@
class="w-8 h-8 border-2 border-[var(--color-jellyfin)] border-t-transparent rounded-full animate-spin"
></div>
</div>
{:else if surface === "video" && streamUrl}
{:else if surface === "video" && selection}
<VideoPlayer
media={currentMedia}
{streamUrl}
{selection}
mediaSourceId={mediaSourceId ?? undefined}
initialPosition={videoInitialPosition}
needsTranscoding={videoNeedsTranscoding}