mpv now decodes video on Linux, drawn into a framebuffer we own and blitted
into the default vbox's draw handler. Tauri's widget tree is untouched, so an
upgrade that assumes its own layout cannot invalidate this. Direct play means
the original file, hardware decoding, and no server transcode at all — where
previously every desktop video was re-encoded to h264 for the browser engine,
whatever the file actually was. Off by default: JELLYTAU_NATIVE_VIDEO=1.
That settles finding 2 of playback-backend-unification.md — "native video
cannot be composited with a Tauri webview" — by demonstration rather than
argument, on X11 and Wayland both.
Turning it on exposed nine defects, none of them mpv's. Each was the same
mistake in a different place: a capability written down as a compile-time fact
about the platform, or a state asserted instead of confirmed.
DR-238/246 a seek routed by the stream's container rather than by what the
engine could do with it - correct only while one player handled
those streams, silent the moment another did
DR-239 a property handled but never observed, so the play/pause button
waited for an event that could not arrive
DR-240 fullscreen expanding the document while the window stayed put
DR-241 a seek issued before the engine had a file, failed, and discarded
- which is why resume began at zero
DR-247 a Linux-only gate outliving the caller that made it Linux-only,
breaking the Android build outright
DR-250 a stop aimed at whichever renderer bookkeeping believed was in
charge, missing the one actually making sound
DR-251 a duration of zero believed, leaving the seek bar no scale
DR-252 a junk float converted to a Duration, panicking the backend the
instant a length-less stream appeared
So the MediaPlayer contract (DR-242 … DR-247): `open` carries a start position,
so no caller sequences load-then-seek and none can race an engine's load;
`seek` states a destination and leaves in-place-versus-re-open to the engine;
`snapshot` is one coherent read; and `Phase::Opening` names the window where
intent used to be lost. One conformance suite runs against every engine —
FakePlayer and mpv under cargo test, ExoPlayer instrumented on a device — so an
engine is either correct or visibly failing.
Two of the nine were introduced during this work and caught on hardware, not by
any suite: an over-broad capability that grouped ExoPlayer with mpv, and the
Duration panic. The suites test engines that behave. That is recorded in
docs/native-player-verification.md, which asks for the exact action sequences
that found them.
Verified: all automated gates, conformance (mpv 9/9, legacy 8/9 by design,
ExoPlayer 7/7 on device), and manual desktop and Android passes on real
hardware.
Known open and deliberately shipped: resume reads local progress and never the
server's; the background-audio handoff still declares a state swap it does not
confirm (the symptom is now impossible, the race is not); and `bun run
android:dev` builds an APK carrying the release application id, whose failure
message advises an uninstall that would destroy app data. Fix that last one
before anyone else builds for Android.
Squashed from worktree-linux-native-video, which keeps the per-defect history.
388 lines
13 KiB
TypeScript
388 lines
13 KiB
TypeScript
/**
|
|
* Unified frontend player API (the boundary).
|
|
*
|
|
* This is the single write-side entry point for playback. Every UI component
|
|
* that wants to *control* the player calls a method here; nothing else should
|
|
* invoke `commands.player*` directly. The Rust `PlayerController` remains the
|
|
* single source of truth — these methods only send intent-level commands and
|
|
* let state flow back through `PlayerStatusEvent` → `playerEvents.ts` → the
|
|
* `player`/`queue` stores.
|
|
*
|
|
* Reads stay on the established stores: this module re-exports the read-only
|
|
* derived + merged (remote-session-aware) stores so UI can import state and
|
|
* actions from one place, in both local and remote modes.
|
|
*
|
|
* TRACES: UR-005 | DR-001, DR-009, DR-097 | UT-091
|
|
*/
|
|
|
|
import { get } from "svelte/store";
|
|
import { commands } from "$lib/api/bindings";
|
|
import type {
|
|
PlayTracksContext,
|
|
PlayAlbumTrackRequest,
|
|
PlayItemRequest,
|
|
StreamingQuality,
|
|
StreamSelection,
|
|
} from "$lib/api/bindings";
|
|
import { auth } from "$lib/stores/auth";
|
|
import type { PlayerAdapter } from "./adapters/types";
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// Active player adapter registry
|
|
//
|
|
// When a video is playing, VideoPlayer registers its PlayerAdapter here so that
|
|
// control intents — whether from UI or routed from a backend control event
|
|
// (lockscreen/remote/sleep-timer) — reach the actual player element/surface.
|
|
// When no adapter is registered (audio-only playback), control falls through to
|
|
// the queue-level backend commands, which is the correct behavior there.
|
|
// ---------------------------------------------------------------------------
|
|
let activeAdapter: PlayerAdapter | null = null;
|
|
|
|
function setActiveAdapter(adapter: PlayerAdapter): void {
|
|
activeAdapter = adapter;
|
|
}
|
|
|
|
function clearActiveAdapter(adapter?: PlayerAdapter): void {
|
|
// Only clear if it's still the one we think is active (guards against a newly
|
|
// mounted player's adapter being cleared by the outgoing player's teardown).
|
|
if (!adapter || activeAdapter === adapter) {
|
|
activeAdapter = null;
|
|
}
|
|
}
|
|
|
|
function getActiveAdapter(): PlayerAdapter | null {
|
|
return activeAdapter;
|
|
}
|
|
|
|
/**
|
|
* Resolve the current repository handle, throwing a clear error if the user is
|
|
* not authenticated. Centralizes the `auth.getRepository().getHandle()` dance
|
|
* that was previously duplicated across every context-play call site.
|
|
*/
|
|
function requireHandle(): string {
|
|
// The repository is the source of truth for the handle. We consult the auth
|
|
// store's isAuthenticated flag only as a best-effort guard — guarded in a
|
|
// try/catch so a not-yet-subscribable store (or a test double) can't block a
|
|
// valid repository handle.
|
|
try {
|
|
const authState = get(auth);
|
|
if (authState && authState.isAuthenticated === false) {
|
|
throw new Error("User not authenticated");
|
|
}
|
|
} catch (err) {
|
|
// get(auth) failed (e.g. non-store mock) — fall through to the repository,
|
|
// which is the authoritative source of the handle.
|
|
if (err instanceof Error && err.message === "User not authenticated") throw err;
|
|
}
|
|
const repo = auth.getRepository();
|
|
if (!repo) {
|
|
throw new Error("No repository available");
|
|
}
|
|
return repo.getHandle();
|
|
}
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// Transport controls (no repository handle required)
|
|
// ---------------------------------------------------------------------------
|
|
|
|
// Transport intents ALWAYS go to the backend, in both native and HTML5 modes.
|
|
//
|
|
// These used to short-circuit into the active video adapter, which made the
|
|
// webview the decider: `adapter.toggle()` read `el.paused` off the DOM and
|
|
// flipped the element, so Rust never saw the intent. `el.paused` flips
|
|
// transiently while an element buffers or settles a seek, so two intents
|
|
// ~150ms apart could read different values and take opposing actions — a
|
|
// self-sustaining play/pause loop.
|
|
//
|
|
// Now Rust decides from PlayerController state and drives the element back
|
|
// through a `ControlCommand` event (handled in playerEvents.ts), the same
|
|
// "backend decides, adapter executes the primitive" split used by
|
|
// player_seek_video. Do NOT reintroduce an adapter short-circuit here.
|
|
|
|
async function play() {
|
|
await commands.playerPlay();
|
|
}
|
|
|
|
async function pause() {
|
|
await commands.playerPause();
|
|
}
|
|
|
|
async function toggle() {
|
|
await commands.playerToggle();
|
|
}
|
|
|
|
async function stop() {
|
|
// Stop is a queue/session-level action (clears playback); always go to backend.
|
|
// The adapter is disposed by VideoPlayer's own teardown.
|
|
await commands.playerStop();
|
|
}
|
|
|
|
async function seek(positionSeconds: number) {
|
|
// Audio path: backend seeks the native backend directly.
|
|
if (!activeAdapter) {
|
|
await commands.playerSeek(positionSeconds);
|
|
return;
|
|
}
|
|
// Video path: ask the backend to DECIDE the strategy (in-place vs reload), then
|
|
// execute the matching adapter primitive. The decision logic stays in Rust
|
|
// (player_seek_video); the adapter only runs the chosen mechanical primitive.
|
|
await seekVideo(positionSeconds, null, null);
|
|
}
|
|
|
|
/**
|
|
* Video seek: backend decides strategy, facade dispatches the chosen adapter
|
|
* primitive. `mediaSourceId`/`audioTrackIndex` come from the video view (they are
|
|
* needed for the transcode reload URL). Requires an active video adapter.
|
|
*/
|
|
async function seekVideo(
|
|
positionSeconds: number,
|
|
mediaSourceId: string | null,
|
|
audioTrackIndex: number | null,
|
|
): Promise<void> {
|
|
const adapter = activeAdapter;
|
|
if (!adapter) {
|
|
await commands.playerSeek(positionSeconds);
|
|
return;
|
|
}
|
|
const response = (await commands.playerSeekVideo(
|
|
requireHandle(),
|
|
positionSeconds,
|
|
mediaSourceId,
|
|
audioTrackIndex,
|
|
adapter.kind === "html5",
|
|
)) as any;
|
|
// Serde keeps `seek_offset` snake_case (only the "strategy" tag is camelCase).
|
|
if (response.strategy === "reloadStream") {
|
|
// `seek_offset` is the ABSOLUTE position to resume at, not a base to add to
|
|
// the element's clock: the reloaded stream starts at the item's zero since
|
|
// DR-181, so reloadSource seeks there. (The name is the wire field's.)
|
|
await adapter.reloadSource(response.selection, response.seek_offset ?? positionSeconds);
|
|
} else {
|
|
await adapter.seekElement(response.position ?? positionSeconds, 0);
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Switch audio track: backend decides (may reload the stream), facade dispatches
|
|
* the resulting primitive. Requires an active video adapter.
|
|
*/
|
|
async function switchAudioTrack(
|
|
streamIndex: number,
|
|
arrayIndex: number,
|
|
currentPosition: number | null,
|
|
mediaSourceId: string | null,
|
|
): Promise<void> {
|
|
const adapter = activeAdapter;
|
|
if (!adapter) return;
|
|
const response = (await commands.playerSwitchAudioTrack(
|
|
requireHandle(),
|
|
streamIndex,
|
|
arrayIndex,
|
|
adapter.kind === "html5",
|
|
currentPosition,
|
|
mediaSourceId,
|
|
)) as any;
|
|
if (response.strategy === "reloadStream") {
|
|
await adapter.reloadSource(response.selection, response.position!);
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Change the bandwidth ceiling of the video playing now. The backend re-opens
|
|
* the stream at the new quality and decides who reloads: it handles a native
|
|
* backend itself, and hands HTML5 a selection for the same `reloadSource`
|
|
* primitive the audio-track switch uses. Requires an active video adapter.
|
|
*
|
|
* The change applies to **this playback only** — the backend sets a per-playback
|
|
* override that the next item clears, leaving the durable Settings default
|
|
* alone. Returns the negotiated selection so the caller can show what it
|
|
* actually got, which is not always what was asked for: a ceiling above the
|
|
* source bitrate is the source.
|
|
*
|
|
* TRACES: UR-074, UR-079 | DR-162, DR-226
|
|
*/
|
|
async function setStreamQuality(
|
|
quality: StreamingQuality,
|
|
currentPosition: number | null,
|
|
mediaSourceId: string | null,
|
|
audioTrackIndex: number | null,
|
|
): Promise<StreamSelection | null> {
|
|
const adapter = activeAdapter;
|
|
if (!adapter) return null;
|
|
const response = (await commands.playerSetStreamQuality(
|
|
requireHandle(),
|
|
quality,
|
|
adapter.kind === "html5",
|
|
currentPosition,
|
|
mediaSourceId,
|
|
audioTrackIndex,
|
|
)) as any;
|
|
if (response.strategy === "reloadStream") {
|
|
await adapter.reloadSource(response.selection, response.position ?? currentPosition ?? 0);
|
|
return response.selection;
|
|
}
|
|
// The native backend reloaded itself, but still reports what it opened — the
|
|
// caller needs it to show the rung actually in force.
|
|
return response.selection ?? null;
|
|
}
|
|
|
|
async function next() {
|
|
await commands.playerNext();
|
|
}
|
|
|
|
async function previous() {
|
|
await commands.playerPrevious();
|
|
}
|
|
|
|
async function skipTo(index: number) {
|
|
await commands.playerSkipTo(index);
|
|
}
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// Queue mode controls
|
|
// ---------------------------------------------------------------------------
|
|
|
|
async function toggleShuffle() {
|
|
await commands.playerToggleShuffle();
|
|
}
|
|
|
|
async function cycleRepeat() {
|
|
await commands.playerCycleRepeat();
|
|
}
|
|
|
|
async function removeFromQueue(index: number) {
|
|
await commands.playerRemoveFromQueue(index);
|
|
}
|
|
|
|
async function moveInQueue(fromIndex: number, toIndex: number) {
|
|
await commands.playerMoveInQueue(fromIndex, toIndex);
|
|
}
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// Volume
|
|
// ---------------------------------------------------------------------------
|
|
|
|
async function setVolume(volume: number) {
|
|
if (activeAdapter) activeAdapter.setVolume(volume);
|
|
await commands.playerSetVolume(volume);
|
|
}
|
|
|
|
async function toggleMute() {
|
|
await commands.playerToggleMute();
|
|
}
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// Track selection (video) — dispatch to the active video adapter when present
|
|
// ---------------------------------------------------------------------------
|
|
|
|
async function setSubtitleTrack(streamIndex: number | null) {
|
|
if (activeAdapter) return void (await activeAdapter.selectSubtitle(streamIndex));
|
|
await commands.playerSetSubtitleTrack(streamIndex);
|
|
}
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// Context-aware playback (repository handle required — resolved internally)
|
|
// ---------------------------------------------------------------------------
|
|
|
|
/**
|
|
* Play a set of tracks by ID with an explicit queue context. The backend
|
|
* fetches all metadata and builds the queue; the frontend queue store updates
|
|
* from the resulting `queue_changed` event.
|
|
*/
|
|
async function playTracks(request: {
|
|
trackIds: string[];
|
|
startIndex: number;
|
|
shuffle: boolean;
|
|
context: PlayTracksContext;
|
|
startPosition?: number;
|
|
}) {
|
|
await commands.playerPlayTracks(requireHandle(), request);
|
|
}
|
|
|
|
/** Play a single track within its album context (more efficient than playTracks). */
|
|
async function playAlbumTrack(request: PlayAlbumTrackRequest) {
|
|
await commands.playerPlayAlbumTrack(requireHandle(), request);
|
|
}
|
|
|
|
/** Play a single explicit media item (used by the video path). */
|
|
async function playItem(request: PlayItemRequest) {
|
|
return commands.playerPlayItem(request);
|
|
}
|
|
|
|
/** Add a single track to the queue by ID. */
|
|
async function addTrackById(trackId: string, position: "next" | "end" = "end") {
|
|
await commands.playerAddTrackById(requireHandle(), { trackId, position });
|
|
}
|
|
|
|
/** Add multiple tracks to the queue by ID. */
|
|
async function addTracksByIds(trackIds: string[], position: "next" | "end" = "end") {
|
|
await commands.playerAddTracksByIds(requireHandle(), { trackIds, position });
|
|
}
|
|
|
|
/**
|
|
* The unified player facade. Import this and call its methods instead of
|
|
* reaching for `commands.player*` in UI code.
|
|
*/
|
|
export const playerController = {
|
|
play,
|
|
pause,
|
|
toggle,
|
|
stop,
|
|
seek,
|
|
next,
|
|
previous,
|
|
skipTo,
|
|
toggleShuffle,
|
|
cycleRepeat,
|
|
removeFromQueue,
|
|
moveInQueue,
|
|
setVolume,
|
|
toggleMute,
|
|
setSubtitleTrack,
|
|
seekVideo,
|
|
switchAudioTrack,
|
|
setStreamQuality,
|
|
playTracks,
|
|
playAlbumTrack,
|
|
playItem,
|
|
addTrackById,
|
|
addTracksByIds,
|
|
// Active-adapter registry (used by VideoPlayer to register its element adapter
|
|
// and by playerEvents.ts to route backend control commands to it).
|
|
setActiveAdapter,
|
|
clearActiveAdapter,
|
|
getActiveAdapter,
|
|
};
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// Read-side re-exports: UI reads state from ONE place, in both local & remote
|
|
// modes. These remain the single source of truth fed by playerEvents.ts.
|
|
// ---------------------------------------------------------------------------
|
|
|
|
export {
|
|
playerState,
|
|
currentMedia,
|
|
isPlaying,
|
|
isPaused,
|
|
isLoading,
|
|
playbackPosition,
|
|
playbackDuration,
|
|
volume,
|
|
isMuted,
|
|
mergedMedia,
|
|
mergedIsPlaying,
|
|
mergedPosition,
|
|
mergedDuration,
|
|
mergedVolume,
|
|
} from "$lib/stores/player";
|
|
|
|
export {
|
|
queueItems,
|
|
currentQueueIndex,
|
|
currentQueueItem,
|
|
isShuffle,
|
|
repeatMode,
|
|
hasNext,
|
|
hasPrevious,
|
|
} from "$lib/stores/queue";
|