diff --git a/.gitea/workflows/build-and-test.yml b/.gitea/workflows/build-and-test.yml index e25cbecc..c5356e7a 100644 --- a/.gitea/workflows/build-and-test.yml +++ b/.gitea/workflows/build-and-test.yml @@ -109,7 +109,7 @@ jobs: # at "warn" until its class is cleared and it can be promoted to "error". # Lower this as you clear them. Never raise it to make a build pass. - name: Lint - run: bun run lint -- --max-warnings=159 + run: bun run lint -- --max-warnings=158 - name: Check TypeScript run: | diff --git a/CHANGELOG.md b/CHANGELOG.md index 757e83bf..45461db8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,97 @@ generated trace matrix lives in [docs/traceability.md](docs/traceability.md). For how long each fixed defect had been shipping before it was found, see [docs/defect-windows.md](docs/defect-windows.md). +## v0.11.0 + +Video can play through the native renderer on Linux, and the machinery every +platform's playback goes through was rebuilt around one contract. Nine defects +fell out of doing it — each one a capability the code had written down as a +fact about the platform rather than asking the thing that would know. + +### ✨ Changes + +- **Video can decode natively on Linux, without the server re-encoding it.** + Until now every video played on the desktop was transcoded by Jellyfin to + h264 and handed to the browser engine, whatever the file actually was — so the + server burned CPU on every play, and quality was capped by that conversion. + mpv can now draw the picture directly, composited beneath the interface so the + controls, subtitles and overlays still sit on top of it. Direct play means the + original file, hardware decoding, and no server work at all. This is off by + default while it settles: set `JELLYTAU_NATIVE_VIDEO=1` to try it. The browser + path is untouched and remains what you get otherwise. (UR-080 → DR-231 … + DR-237) + +- **Playback speaks one language across every player.** Linux, Android and + Windows each drove their engine through a different set of calls, and a rule + learned on one did not reach the others — which is why several of the fixes + below existed on one platform and not another. All three now go through a + single contract, and one suite of behaviours runs against every engine, + including ExoPlayer on a real device. An engine is either correct or visibly + failing. Nothing about this is visible while it works, which is the point. + (UR-081 → DR-242 … DR-247) + +### 🐛 Fixes + +- **Resuming a film starts where you left it, instead of at the beginning.** + Asking a player to open a file and asking it to start at a position were two + separate steps, and the second was issued before the first had finished — so + it failed, was discarded, and playback began at zero. It affected resume and + any skip on a stream the server was converting. The position is now part of + opening the file, so there is no gap for it to fall into. (DR-241) + +- **Skipping works on films the server is converting.** A skip was routed by the + *shape* of the stream rather than by what the player could do with it. That + happened to be right while one particular player handled those streams and + became wrong the moment another did — after which skipping simply did nothing, + silently. Players now say what they can do and are asked. (DR-238, DR-246) + +- **The play and pause button follows the player again.** The code that reacted + to pausing was never subscribed to the event it was waiting for, so the button + stayed where it was while playback did something else. (DR-239) + +- **Fullscreen fills the screen.** It expanded the page rather than the window, + which was invisible while the picture was drawn inside the page and obvious as + soon as it was not. (DR-240) + +- **The seek bar knows how long the film is.** A player that had not yet worked + out the duration reported zero, and zero was believed — leaving the bar with + no scale and nothing to drag against, even though the length had been known + since the library listed it. (DR-251) + +- **Leaving the player stops the sound.** The stop was aimed at whichever + renderer the app believed was in charge. Enabling background audio hands over + to a different one, so afterwards the app stopped something that was no longer + playing and the film carried on as an audio track in the mini player. Closing + now stops everything, regardless of who was in charge. (DR-250) + +- **Coming back from background audio no longer leaves a black screen.** The + stream that plays while the app is hidden has no fixed length, and the value a + player uses to say so is a very large negative number. Converting it crashed + the playback engine outright, which looked like a dead player with no + controls. (DR-252) + +- **Android builds again.** A rule that only applied to Linux stayed attached to + code that had stopped being Linux-only, and the Android build had not compiled + since. (DR-247) + +### 🧹 Under the hood + +- The conformance suite can be run on its own: `bun run test:player` for the + desktop engines, `bun run test:player:android` for ExoPlayer on a connected + device. Both build a test fixture rather than carrying media in the + repository. + +- [docs/native-player-verification.md](docs/native-player-verification.md) + records what to check before a release, including the exact sequences that + found two of the defects above — both of which passed every automated test. + +### Known limitations + +- Resume reads progress saved on the device, not from the server, so a fresh + install or a second device will not offer to resume something watched + elsewhere. +- Native video on Linux is opt-in and is not yet the default. + ## v0.10.1 A single fix, for something that had been quietly overriding a choice you made. diff --git a/docs-site/SUMMARY.md b/docs-site/SUMMARY.md index 45a207be..bdaebfa1 100644 --- a/docs-site/SUMMARY.md +++ b/docs-site/SUMMARY.md @@ -33,7 +33,6 @@ - [Spec Review Checklist](specs/SPEC-REVIEW-CHECKLIST.md) - [Playback Backend Unification](specs/playback-backend-unification.md) - [Linux Native Video Spike](specs/linux-native-video-spike.md) -- [Backend-Owned Stream Selection](specs/backend-owned-stream-selection.md) - [Player Facade Enforcement](specs/player-facade-enforcement.md) - [Windows Native Audio Backend](specs/windows-native-audio-backend.md) - [libmpv2 Migration](specs/libmpv2-migration.md) @@ -42,12 +41,14 @@ - [Scoped Search Boundary](specs/scoped-search-boundary.md) - [Scoped Search Boundary — Implementation](specs/scoped-search-boundary-implementation.md) - [Frontend Domain Model](specs/frontend-domain-model.md) +- [Desktop Native Video](specs/desktop-native-video.md) - [Build Provenance](specs/build-provenance.md) # Build & Release - [Build & Release](build/build-release.md) - [Release Checklist](release-checklist.md) +- [Native Player Verification](native-player-verification.md) - [Desktop Packaging](build/build-desktop-packages.md) - [Windows Build](build/build-windows.md) - [Defect Windows](defect-windows.md) diff --git a/docs/architecture/01-rust-backend.md b/docs/architecture/01-rust-backend.md index 8486aeb0..cbd7b93b 100644 --- a/docs/architecture/01-rust-backend.md +++ b/docs/architecture/01-rust-backend.md @@ -673,8 +673,161 @@ device profile. Sending it there — not just on the transcode URL — is what m the cap real: a stream the server decides to *direct play* is served at the source file's own bitrate, and no URL parameter afterwards can reduce it. +#### Two levels of ceiling + +**Location**: `src-tauri/src/repository/online.rs` (TRACES: UR-074, UR-079 | DR-226) + +There are two, and they are not the same thing: + +| | Set by | Lives until | Read via | +|---|---|---|---| +| **Device default** | Settings (`player_set_video_settings`) | Persisted; restored at startup | `streaming_quality()` | +| **Per-playback override** | The in-player picker (`player_set_stream_quality`) | The next item starts playing | `playback_quality_override()` | + +`effective_streaming_quality()` resolves the pair — override first, else default — +and **is the only thing stream construction may read**. Every URL builder and the +`PlaybackInfo` negotiation go through it, for the reason the process-wide static +existed in the first place: if the negotiation and the URL builder disagree, the +cap leaks — the negotiation authorises a direct play the builder then never gets +to constrain, or the reverse. + +> The override exists because a single global cannot express "this 4K remux needs +> a ceiling, that podcast does not". The picker had documented itself as a "this +> film, this connection" control since it was written, but was implemented by +> writing the *default* — so dropping one awkward film to 2 Mbps silently capped +> every video played afterwards for the rest of the process, with Settings still +> showing the old value. It is cleared on every `player_play_item` / +> `player_play_queue` / `player_play_tracks`, which is what stops it surviving +> into an autoplayed next episode where nobody would reopen the picker. + +### Stream selection + +**Location**: `src-tauri/src/repository/stream_selection.rs`, +`OnlineRepository::get_stream_selection` (TRACES: UR-070, UR-079 | DR-225, DR-227, DR-228) + +**Rust decides *what stream*. The player decides *how to deliver it*.** That line +is the whole design. A backend with genuine adaptive selection (ExoPlayer over a +multi-variant playlist) is left to do it; Rust chooses what to request and never +paces bytes. + +`get_stream_selection` returns one self-describing `StreamSelection` in place of +the bare URL `get_video_stream_url` used to hand out: + +| Field | Carries | +|---|---| +| `url` | What to open | +| `transport` | `Hls` / `Progressive` / `LocalFile` — how to fetch it | +| `playback_kind` | `DirectPlay` / `DirectStream` / `Transcode` — what the server is doing to the source | +| `rendition` | The negotiated ceiling and codecs; `None` for a direct play, which *is* the source | +| `available` | The quality ladder as it applies to this media source (DR-227) | +| `needs_transcoding` | Derived from `playback_kind`, so the rule is answered once | + +Both enums are serde-tagged (`{"type":"hls"}`) so the frontend matches a +discriminant rather than comparing text. + +> **Why `transport` exists.** `VideoPlayer.svelte` chose its loader with +> `url.includes(".m3u8")`, in two places. Rust *built* that URL and knows exactly +> what it is; re-deriving it downstream by substring match is a domain fact +> reconstructed in the presentation layer — the same class of error as leaking +> item-type taxonomy, and one that fails silently in **both** directions: a +> progressive file served from a path containing the substring gets an HLS +> loader, and a playlist served from a path without it does not. +> +> The paths that never negotiate get the same shape from Rust rather than letting +> a caller assemble one — `media_local_selection` for a downloaded file, +> `LiveStreamInfo.transport` for a live channel — so there is no second place +> where a transport is decided. + +#### The playback-kind decision + +`decide_playback_kind` is a free function and pure, so every branch is testable +from `PlaybackInfo` fixtures without a server. Order matters — the two +client-side overrides come first, because each describes a case where the +server's answer is right about the *file* and wrong about what this app will do +with it: + +1. **Undecodable audio → `Transcode`.** Jellyfin 10.11.5 honours a + DirectPlayProfile's container and video codec but *ignores its audio codec*, + so it offers direct play for an E-AC-3 track the webview renders in silence. + A silent direct play is worse than a transcode. +2. **A pinned audio track → `Transcode`.** Not a defect in the server's answer, a + different question: the file has one default track and the viewer asked for + another. +3. Otherwise `supports_direct_play` → `DirectPlay`, else `supports_direct_stream` + → `DirectStream`, else `Transcode`. + +A direct **stream** is a remux — codecs copied, container repackaged. It is cheap +and is deliberately *not* counted as transcoding; conflating the two would report +a free passthrough as a server-side re-encode. + +> **What this is worth, measured.** Against the development server (Jellyfin +> 10.11.5), 400 items sampled for codec mix and 40 put through a real negotiation +> per profile: +> +> | Profile | Direct play | +> |---|---| +> | Linux / WebKitGTK (`h264` only, 2ch) | 3/40 — **7%** | +> | Android / ExoPlayer (`h264,hevc,vp8,vp9,av1,mpeg4` + `ac3,eac3`, 6ch) | 34/40 — **85%** | +> +> The library is ~80% hevc (`hevc+eac3` alone is a third of it), which is why the +> two diverge so hard. +> +> **Read that 85% as a ceiling, not a result.** It was measured with a profile +> containing `ac3,eac3`. The Android device this was later run on reports neither +> in its `MediaCodecList` — no Dolby licence, which is normal for a tablet — so +> eac3 content, about a third of the sampled library, correctly transcodes there. +> What any given device achieves depends on its own codec list, and on the +> profile being derived from the renderer at all (DR-234), which it was not when +> the figure was taken. +> +> **The payoff is still overwhelmingly Android**, because that is where a real +> decoder is already doing the work. Linux stays near 7% until libmpv decodes the +> picture — the h264-only profile is a WebKitGTK constraint, not a JellyTau +> choice, and is what `linux-native-video-spike.md` exists to remove. A reviewer +> should not expect this code to fix Linux on its own. + +#### The quality ladder per source + +`quality_options_for_source(source_bitrate)` returns every rung, each marked with +`exceeds_source`: true when that rung's ceiling is at or above what the source +itself carries, so selecting it produces the same bytes as `Original`. The +frontend draws the list and drops the redundant rungs; it does not decide which +they are. + +- `Original` is never marked — it *is* the source. +- An unreported source bitrate (some containers have none; the sampled library + has `avi` files with no bitrate at all) marks **nothing** redundant, keeping + every rung offered. That is the safe direction: the viewer keeps every choice. + +#### No adaptive ladder to preserve + +**TRACES: UR-079 | DR-229 (Won't Do)** + +Mid-playback re-negotiation on throughput was scoped and dropped on measurement. +A master playlist from this server carries exactly **one** `EXT-X-STREAM-INF`: +Jellyfin builds it from the single rendition the request asked for rather than +publishing a ladder. So there is no adaptation for hls.js to be preserving and +none that mpv would lose — the claim that there was is recorded in +`playback-backend-unification.md` and does not hold. "Adapt mid-stream" collapses +into "pick well at open", which is what the two levels of ceiling and the +per-source ladder already are. + +Kept here because it is a measurement, not an opinion: a server that *does* +publish a ladder would change the answer, and the re-negotiation path below is +the hook that work would build on. + +#### Re-negotiation + +One mechanism, not two. `player_seek_video`, `player_switch_audio_track` and +`player_set_stream_quality` all return a tagged `strategy` saying who reloads — +the backend handles a native backend itself and hands the webview a +`StreamSelection` for `reloadSource`. Note the wire wart: tauri-specta keeps +these response fields snake_case (`seek_offset`), while the `strategy` tag itself +is camelCase. + The frontend names a variant and nothing else; the labels the picker shows are -served over IPC by `player_get_streaming_qualities`. +served over IPC — from `available` on the selection, or +`player_get_streaming_qualities` for the Settings list. ## Background workers diff --git a/docs/architecture/02-svelte-frontend.md b/docs/architecture/02-svelte-frontend.md index 539253bb..93d2a8e0 100644 --- a/docs/architecture/02-svelte-frontend.md +++ b/docs/architecture/02-svelte-frontend.md @@ -802,6 +802,45 @@ by exactly the inset. Unlike `addJavascriptInterface`, the inset push only writes CSS properties, so it can safely be re-sent on resume. +## Stream Transport + +**Location**: `src/lib/player/streamTransport.ts` +**TRACES**: UR-079 | DR-225 | UT-214 + +`videoLoaderFor(selection, capabilities)` picks the loader for the webview +`