fix(player): stop the server burning subtitles into the picture

A transcoded episode stalled every few seconds and seeking took five to
nine seconds to produce a frame. Neither was a seek bug: both seeks in the
capture landed correctly. The stream itself could not keep up.

The episode was HEVC video, E-AC-3 audio, and a PGSSUB subtitle track.
Only the audio needed transcoding — the device profile supports HEVC and
the server would have remuxed the video untouched. But the PlaybackInfo
request omitted SubtitleStreamIndex, and omitting it does not mean "no
subtitles": the server then honours the source's default/forced flag and
picks a track itself. It picked the PGS one. PGS is a bitmap, and the
profile advertised only srt/vtt as External, so it could not go out as a
sidecar — leaving SubtitleMethod=Encode, burn-in.

Burn-in is a video cost, not a subtitle cost. Compositing rules out
remuxing, so the whole HEVC stream was re-encoded to h264 frame by frame.
The server could not sustain that in real time: the buffer never grew past
one segment and playback ran waiting -> HLS error -> canplay -> three
seconds of picture, indefinitely, while each seek restarted the encoder
from scratch. TranscodeReasons named it — SubtitleCodecNotSupported — but
nothing in the log connected that to the stall, so the diagnostic now says
which track it is declining and why.

Ask for SubtitleStreamIndex=-1 explicitly, and advertise every text format
we can render (srt/subrip/ass/ssa/vtt) as External so a subtitle can only
ever arrive as a sidecar. Nothing is lost: the app already fetches subtitle
tracks itself and draws them over the video (UR-020), so the server's
composited copy was always redundant. Image-based tracks are consequently
not offered, which is honest rather than a regression — the renderer cannot
composite a bitmap, and the previous behaviour paid for them by making the
stream unwatchable.

The policy lives beside the other device-profile rules in Rust, where it is
testable without a device.

TRACES: UR-020, UR-004 | DR-176 | UT-168
This commit is contained in:
2026-08-16 09:23:39 +02:00
parent 1a9805f0f3
commit 041969f446
3 changed files with 134 additions and 13 deletions
+4 -2
View File
@@ -334,6 +334,7 @@ Internal architecture, components, and application logic.
| DR-162 | Video streams are opened against a **bandwidth ceiling the user chose**, instead of a fixed allowance nobody could change. Every video URL carried `MaxStreamingBitrate=20000000`/`VideoBitrate=18000000`, `PlaybackInfo` negotiated at 20 Mbps, and the device profile advertised `999999999` — so on a metered or slow connection the only lever was not watching. `StreamingQuality` is a ladder of ceilings (Original, 20/10/8/4/2/1 Mbps, 720 kbps) in which a step is not a label but a bundle of transcode parameters: the total ceiling, the audio share of it, and the resolution that budget can carry. It lives in Rust because those numbers are Jellyfin encoding vocabulary — the frontend names a variant and reads labels back over `player_get_streaming_qualities`, the same arrangement as the EQ preset curves. The video bitrate is the total *minus* the audio share, so the two together honour the cap rather than overshooting it by the size of the audio track, and `MaxHeight` falls with the ladder so a small budget is not spent on pixels it cannot afford. The cap has to reach the **negotiation**, not only the transcode URL: `max_static_bitrate` in the device profile is what makes the server refuse to direct-play a source fatter than the ceiling, and without it a 30 Mbps remux is handed over untouched and every URL parameter downstream is moot — which is why it is applied at all four places that decide bandwidth (the HLS builder, `PlaybackInfo`, `open_live_stream`, and the background-audio handoff, which takes the lower of the cap and its own 384 kbps). The ceiling is process-wide rather than a field on `OnlineRepository`, mirroring `INCLUDE_CATALOG_BROWSE`: it is a preference about *this device's connection*, it must survive a repository rebuilt on re-login, and every builder plus the negotiation have to agree on it or the cap leaks. Settings owns the durable default and is the only writer to `app_settings` — persisted unlike the rest of `VideoSettings`, because a limit set for a metered connection that silently reverts to uncapped on the next launch spends the user's data with no changed setting to show for it — and it is restored at startup from the async runtime, defaulting to uncapped if the read fails so a database problem degrades to the old behaviour rather than to an arbitrary limit. The in-player menu is the per-video override: a cap is a property of the stream the server is producing, so it cannot be applied to one already in flight — `player_set_stream_quality` re-opens the stream at the new quality and resumes at the current position, reloading a native backend itself and handing HTML5 a URL for the same `reloadSource` primitive the audio-track switch uses, so no strategy branch lives in the UI. It deliberately does not persist. This gives UR-070 its resume-at-the-same-point mechanism; the server-offered per-item rendition list that requirement also asks for remains proposed | Playback | UR-074, UR-070 | Done |
| DR-174 | Tiles of mixed shapes are laid out **justified** rather than gridded. A CSS grid gives every cell one box, so on a page holding square music covers, 16:9 library backdrops and 2:3 posters at once, everything that is not the chosen shape is cropped to it — the home shortcut strip was explicitly forcing `aspect="video"` on music libraries for exactly this reason, which lined the row up by cutting the covers down. `layoutMosaic` packs tiles into rows of a **shared height** and gives each its own width from its own aspect ratio: it adds tiles to a row until the height needed to fill the container has fallen to the target, closes the row there (so rows land at or below the target, never above), and justifies the row to the container width by absorbing the rounding remainder into its widest tile, where a pixel is least visible. The last row is deliberately *not* justified — with one tile left over, filling the width would inflate it to a banner — so it sits at the target height, left-aligned. Ratios are clamped to a band, which costs a crop on genuine outliers and stops one panorama owning a row or one very tall image shrinking to a sliver. It is a pure module with no DOM: the component supplies only the two things the DOM knows — the measured container width, and the artwork's *decoded* aspect ratio, reported by `CachedImage` so the layout uses the shape an image actually has rather than the one its item type implies. Those measurements are committed in one debounced batch rather than per image, because artwork arrives over several hundred milliseconds and re-packing on each arrival would shuffle the grid under the pointer repeatedly. Labels are drawn *over* the bottom of each tile rather than beneath it: a caption below sits outside the computed box, and one that wraps to two lines would break the row alignment the layout exists to provide | UI | UR-075 | Done |
| DR-175 | A library knows which favourites category it belongs to, and the frontend does not work it out. The mosaic offers a favourites tile per category beside its library, which needs a collection-type → category answer; deriving it in Svelte would have re-created the exact leak `SearchScope::item_types` was extracted to close (docs/specs/scoped-search-boundary.md) — one table of Jellyfin vocabulary, differing only in which vocabulary. `SearchScope::for_collection_type` maps `movies`/`tvshows`/`music` and returns `None` for everything else, so a Live TV or books library gets no tile at all rather than one opening an unfiltered list; `All` is never derived from a library, being the cross-library entry offered beside them rather than a property of one. `Library::new` stamps the result onto every library at construction — a constructor rather than a struct literal precisely so a derived field cannot be forgotten at one of the four sites — and it rides to the frontend as an optional `favoritesScope`, absent rather than null when there is none. The UI's remaining share is presentation only: what to call the tile, where to put it, and showing a category's tile **once** however many libraries share it, since two movie libraries have one favourites list between them | UI | UR-075, UR-067 | Done |
| DR-176 | The server is never asked to burn a subtitle into the picture. `PlaybackInfo` omitted `SubtitleStreamIndex`, which does not mean "none" — the server then honours the source's default/forced flag and picks a track itself. On a source whose default subtitle is image-based (PGS/DVD/DVB) that track cannot go out as a sidecar, so the server falls back to `SubtitleMethod=Encode` and composites it into the video. The cost lands on the *video*, not the subtitle: burn-in rules out remuxing, so an HEVC stream the device could have taken untouched is re-encoded frame by frame. Observed on an HEVC + E-AC-3 + PGSSUB episode, where only the audio actually needed transcoding: the server could not sustain the re-encode in real time, the buffer never grew past a single segment, and playback stalled every few seconds — taking seeking with it, since each seek restarted the encoder and cost seconds before the first frame. The fix is to request `SubtitleStreamIndex=-1` explicitly and to advertise every *text* format we can render (`srt`/`subrip`/`ass`/`ssa`/`vtt`) as `External`, so a subtitle can only ever arrive as a sidecar. Nothing is lost, because the app already fetches subtitle tracks itself and draws them over the video (UR-020) — the server's composited copy was always redundant. Image-based tracks are consequently not offered, which is honest rather than a regression: the renderer cannot composite a bitmap, and the previous behaviour paid for them by making the whole stream unwatchable | Playback | UR-020, UR-004 | Done |
| DR-161 | Native video is the default, so picture-in-picture has a real surface. DR-160 makes PiP work on the HTML5 path, but that path can only ever shrink the *UI* into the PiP window; showing the video itself needs the SurfaceView behind the WebView, which is what `experimentalNativeVideo` gates. The flag now defaults to on when the user has never chosen, with an explicit stored choice still winning in both directions so anyone who turned it off keeps it off. This is a deliberate acceptance of risk: the flag existed because the native path was an unfinished spike, and `VideoPlayer.scrubRegression.test.ts` documents its history — a native init that flipped to HTML5 mid-lifecycle and left seeks going down one path while ExoPlayer played on another. Those tests pin the **flag-off** interim override (native response overridden to HTML5, backend stopped once), which the default no longer selects, so they now mock the flag off rather than inherit it: they still guard that path, but they no longer describe what ships. The native scrub/seek path is consequently not covered by the suite and needs device verification | UI | UR-041, UR-003 | Needs device verification |
| DR-159 | The background-audio handoff stops leaking its relative timeline. The handoff plays the episode as a *relative* stream — the audio-only URL is built with `StartTimeTicks` = the position the screen was locked at, so ExoPlayer's zero is the handoff point — and `background_audio_base` holds the offset that turns one back into a real position. The base was a **display-only** correction, applied in exactly two places (the lockscreen scrubber and the internal truncation maths) while every other consumer worked in the relative timeline treating the number as absolute. Each crossing threw away exactly `base` seconds, which is why the jump-back distance varied with where the screen was locked and read as random. Three crossings were live: progress reporting to Jellyfin sent the relative position every 30s, so the server was told `real base` — and since DR-155 now mirrors the server's position back and refreshes on a cache hit, that regressed value returned as the resume point (lock at 40 min, listen to 90, reopen at 50); lockscreen seeks went out absolute and came back relative, against a chunked length-less transcode that cannot honour a seek at all, so a clamped seek landed at stream zero; and media3's own `seekToDefaultPosition`/`seekBack`/`seekForward` bypassed the `ForwardingPlayer` wrapper entirely, reaching the real ExoPlayer — `Util.handlePlayButtonAction` seeking an ended player to the relative zero being the same mechanism as DR-129's truncation bug through a different door. The fix converts **once, at the boundary**: `JellyTauPlayer`'s position tick adds the base (and shifts the duration with it, since the stream's own length is only what remains) before either `nativeOnPositionUpdate` or the lockscreen sees it, so position updates, progress reports, the frontend and the truncation check all speak the episode's timeline and none needs to know a handoff happened. The base is consequently *removed* from `claim_stream_resume`, `truncated_stream_resume_position` and `player_exit_background_audio`, where adding it now double-counts, and the lockscreen's `positionOffsetMs` addition goes with it (the field remains, read-only, as the tick's input). Inbound seeks go the other way: `seek_absolute` is the new boundary for every outside seek, re-opening the stream at the requested position via `resume_stream_at` when a handoff is active — which is what `onSeekTo` had claimed for months in a comment describing code that did not exist — and an ordinary seek otherwise. `seekToDefaultPosition` is swallowed rather than forwarded, since Rust already owns what "play after the stream ended" means and the `play()` that follows reaches it. Exit reads the position *before* clearing either base, or a tick landing in between hands back a relative one | Player | UR-040, UR-005, UR-025 | Done (pending device verification) |
| DR-158 | A watched toggle, on the episode row, the season header, the series and movie hero, and the Episode Focus View. Both halves of the backend already existed and neither had a caller: `mark_played` (`POST /PlayedItems`) was reachable only from the sync drain replaying rows the *reporter* had queued, and `clear_watch_history` (`DELETE /PlayedItems`) only from the destructive "erase this series' history" button — so the sole way to mark something watched was to play it. Jellyfin applies both recursively over a season or series, so the container case needs no client-side fan-out *online*. Offline it does: `storage_set_watched` writes the item **and its descendants** (drawn from `items` by `parent_id`/`album_id`/`season_id`/`series_id`, so an uncached id selects nothing and the statement no-ops instead of raising a foreign-key error), because otherwise marking a season watched with no server would tick the season and leave every episode inside it unwatched. It is deliberately separate from `storage_mark_played`, which stays the single-item "this finished playing" path that increments `play_count`. Un-marking clears the resume position as well as the flag, matching the server. `QueuedOp::MarkUnplayed` gives the queue the missing direction — pushing as `clear_watch_history` — so the toggle works offline both ways rather than only one; without it un-marking would have been the half that needed a connection. The button is an everyday toggle, so unlike `ClearHistoryButton` it does not confirm, and it holds an optimistic state because the caller's `watched` prop only catches up after a reload (a season means a round trip, during which the button would otherwise appear to ignore the tap) | UI | UR-073 | Done |
@@ -356,7 +357,7 @@ Internal architecture, components, and application logic.
| UR-001 | IR-001, IR-002 | - |
| UR-002 | IR-013 | DR-003, DR-012, DR-013, DR-014 |
| UR-003 | IR-003, IR-004, IR-011 | DR-002, DR-004, DR-010 |
| UR-004 | IR-003, IR-004, IR-008, IR-011 | DR-002, DR-004, DR-006, DR-129, DR-171 |
| UR-004 | IR-003, IR-004, IR-008, IR-011 | DR-002, DR-004, DR-006, DR-129, DR-171, DR-176 |
| UR-005 | - | DR-001, DR-005, DR-009 |
| UR-006 | IR-005, IR-006, IR-007, IR-008 | - |
| UR-007 | IR-010 | DR-007, DR-008, DR-016 |
@@ -372,7 +373,7 @@ Internal architecture, components, and application logic.
| UR-017 | - | DR-014, DR-021 |
| UR-018 | IR-013 | DR-015, DR-018, DR-173 |
| UR-019 | IR-015 | DR-022 |
| UR-020 | IR-016, IR-018 | DR-023 |
| UR-020 | IR-016, IR-018 | DR-023, DR-176 |
| UR-021 | IR-016, IR-019 | DR-024 |
| UR-022 | IR-017 | DR-025 |
| UR-023 | IR-010 | DR-026, DR-047, DR-048, DR-049 |
@@ -594,6 +595,7 @@ Internal architecture, components, and application logic.
| UT-160 | The default row height suits its container: it grows with the width, stays inside its bounds, and at phone width still fits two 16:9 tiles side by side | DR-174 | Done |
| UT-161 | A collection type maps to its favourites scope (`movies`/`tvshows`/`music`), every other kind — Live TV, channels, box sets, books, unknown — maps to none rather than to `All`, and a constructed library carries the scope across the wire as `favoritesScope`, omitted entirely when it has none | DR-175 | Done |
| UT-167 | The mosaic's composition: the cross-library favourites entry leads, each library is followed by its own category tile pointing at that category's tab, a category shared by two libraries still yields one tile, a library kind favourites do not carve up yields none, a scope the page offers no tab for is ignored, and every tile is uniquely keyed | DR-174, DR-175 | Done |
| UT-168 | Subtitles are negotiated as sidecars, never burned in: the requested `SubtitleStreamIndex` is the explicit "none" sentinel (`-1`) rather than omitted, every text format we can render (`srt`/`subrip`/`ass`/`ssa`/`vtt`) is advertised as `External`, and the burn-in verdict is by format — text never forces it, image formats (PGSSUB, dvdsub) always do, case-insensitively | DR-176 | Done |
| UT-153 | Scroll handling per navigation kind: a forward move always lands at the top even when the previous page was scrolled and even when the target was visited before, Back restores that route's own saved offset (and the top when it has none), offsets are kept per route rather than shared, a repeated Back still restores, and the initial load leaves the container alone | DR-156 | Done |
| UT-142 | The audio codecs offered for video direct play: a Dolby device's real `MediaCodecList` output drops `ac3`/`eac3`, AMR and raw PCM are dropped too, a fully-supported list is passed through untouched, a list with nothing decodable still claims `aac`, and stray spacing or casing does not decide whether the user gets sound | DR-148 | Done |
| UT-143 | Subtitle URLs resolve to plain strings before they reach the markup (never a Promise), unresolvable tracks are dropped, a stale selection collapses to "Off", and a server-default track is never auto-selected | UR-020, DR-023 | Done |