dtourolle
11d9d760d8
mpv now decodes video on Linux, drawn into a framebuffer we own and blitted
into the default vbox's draw handler. Tauri's widget tree is untouched, so an
upgrade that assumes its own layout cannot invalidate this. Direct play means
the original file, hardware decoding, and no server transcode at all — where
previously every desktop video was re-encoded to h264 for the browser engine,
whatever the file actually was. Off by default: JELLYTAU_NATIVE_VIDEO=1.
That settles finding 2 of playback-backend-unification.md — "native video
cannot be composited with a Tauri webview" — by demonstration rather than
argument, on X11 and Wayland both.
Turning it on exposed nine defects, none of them mpv's. Each was the same
mistake in a different place: a capability written down as a compile-time fact
about the platform, or a state asserted instead of confirmed.
DR-238/246 a seek routed by the stream's container rather than by what the
engine could do with it - correct only while one player handled
those streams, silent the moment another did
DR-239 a property handled but never observed, so the play/pause button
waited for an event that could not arrive
DR-240 fullscreen expanding the document while the window stayed put
DR-241 a seek issued before the engine had a file, failed, and discarded
- which is why resume began at zero
DR-247 a Linux-only gate outliving the caller that made it Linux-only,
breaking the Android build outright
DR-250 a stop aimed at whichever renderer bookkeeping believed was in
charge, missing the one actually making sound
DR-251 a duration of zero believed, leaving the seek bar no scale
DR-252 a junk float converted to a Duration, panicking the backend the
instant a length-less stream appeared
So the MediaPlayer contract (DR-242 … DR-247): `open` carries a start position,
so no caller sequences load-then-seek and none can race an engine's load;
`seek` states a destination and leaves in-place-versus-re-open to the engine;
`snapshot` is one coherent read; and `Phase::Opening` names the window where
intent used to be lost. One conformance suite runs against every engine —
FakePlayer and mpv under cargo test, ExoPlayer instrumented on a device — so an
engine is either correct or visibly failing.
Two of the nine were introduced during this work and caught on hardware, not by
any suite: an over-broad capability that grouped ExoPlayer with mpv, and the
Duration panic. The suites test engines that behave. That is recorded in
docs/native-player-verification.md, which asks for the exact action sequences
that found them.
Verified: all automated gates, conformance (mpv 9/9, legacy 8/9 by design,
ExoPlayer 7/7 on device), and manual desktop and Android passes on real
hardware.
Known open and deliberately shipped: resume reads local progress and never the
server's; the background-audio handoff still declares a state swap it does not
confirm (the symptom is now impossible, the race is not); and `bun run
android:dev` builds an APK carrying the release application id, whose failure
message advises an uninstall that would destroy app data. Fix that last one
before anyone else builds for Android.
Squashed from worktree-linux-native-video, which keeps the per-defect history.
228 lines
12 KiB
Markdown
228 lines
12 KiB
Markdown
# Spec: Two-path media — selectable playback bitrate, independent whole-file download
|
|
|
|
**Status:** Partially implemented. Landed: the cache/download unification
|
|
(DR-126, DR-127 — a cache entry *is* a `downloads` row with a shorter life, and
|
|
eviction only reclaims the temporary tier), local playback of downloaded media
|
|
(DR-128), and the one-path/one-row invariants that followed (DR-133 … DR-138).
|
|
DR-123 is in progress. Still open: the read-through capture itself — DR-122,
|
|
DR-124, DR-125.
|
|
|
|
**DR-121 has shipped and left this spec.** The player quality selector, the
|
|
per-playback bitrate ceiling, and the backend-owned stream decision it needed
|
|
were built as *backend-owned stream selection* (DR-225 … DR-228) and are
|
|
described in
|
|
[01-rust-backend.md](../architecture/01-rust-backend.md#stream-selection) and
|
|
[03-data-flow.md](../architecture/03-data-flow.md#video-stream-selection-flow).
|
|
The settings-level ceiling (DR-162) is the same section. What remains here is the
|
|
*capture* half only — this spec no longer specifies anything about choosing a
|
|
bitrate.
|
|
|
|
**Requirements:** UR-070, UR-071 → DR-122, DR-123, DR-124, DR-125; IR-032
|
|
**Related:** the locally-indexed search and downloaded-browse work, both
|
|
shipped — see
|
|
[03-data-flow.md](../architecture/03-data-flow.md) and
|
|
[06-downloads-and-offline.md](../architecture/06-downloads-and-offline.md)
|
|
|
|
## Summary
|
|
|
|
Two things that are today tangled become explicitly separate:
|
|
|
|
- **The playback path** streams at a bitrate the viewer can change from the
|
|
player. It is ephemeral and its rendition is volatile.
|
|
- **The download path** fetches the whole file at one canonical quality, in the
|
|
background, independently of whatever playback is doing.
|
|
|
|
Bytes fetched for playback are kept **only** when the playback rendition happens
|
|
to be the same artifact the download path would produce — i.e. direct play.
|
|
Otherwise playback bytes are discarded and the download path does its own fetch.
|
|
|
|
## Motivation
|
|
|
|
The appealing version of this — "stream and download at once, switch when enough
|
|
has arrived" — breaks the moment the viewer can change bitrate. A capture taken
|
|
while the rendition changes underneath it is a splice of two encodings: not a
|
|
playable file, and not something that can be honestly recorded as a download.
|
|
Once bitrate is selectable, one stream cannot serve both jobs.
|
|
|
|
Separating the paths also removes the thing that made the original idea
|
|
expensive: there is no mid-playback source swap to engineer, because the download
|
|
never has to take over the live session. It lands on disk and is used at the next
|
|
natural boundary — next episode, or next time the item is played.
|
|
|
|
What exists already and is *not* this: `SmartCache` predictively downloads *other*
|
|
items, `player_preload_upcoming` warms the next one, and
|
|
`refresh_queue_local_sources` swaps queue entries to local at boundaries. All of
|
|
it concerns items you are not currently playing.
|
|
|
|
## Layer assignment
|
|
|
|
| Logic / responsibility | Layer | Why it belongs there |
|
|
|------------------------|-------|----------------------|
|
|
| Available bitrate options for an item | **Rust** | Derived from Jellyfin's media sources and playback-info negotiation; changes with the API. |
|
|
| Mapping a chosen bitrate to transcode parameters | **Rust** | Domain vocabulary. `get_video_download_url` already owns the quality→params mapping; playback must reuse it, not restate it. |
|
|
| Deciding whether playback bytes are keepable (direct play vs transcode) | **Rust** | Depends on the negotiated session. |
|
|
| Canonical download quality | **Rust** | Policy over domain data. |
|
|
| Cache eviction, storage budget, sparse-range bookkeeping | **Rust** | Storage policy. |
|
|
| Promotion to a `downloads` row, and what invalidates a cache entry | **Rust** | Domain state. |
|
|
| Rendering the quality selector; remembering the last choice | **Frontend** | Presentation and a view preference. The *list* comes from Rust. |
|
|
| WiFi-only / opt-in toggles | **Frontend collects, Rust enforces** | The control is UI; the gate must hold even if the UI never calls. |
|
|
|
|
Borderline, recorded: the **default** playback bitrate could look like a user
|
|
preference (frontend). It goes to Rust because it must be reconcilable with what
|
|
the server can actually produce for a given media source — a preference the
|
|
backend has to validate is not a preference the frontend can own alone. The
|
|
frontend stores the user's *choice*; Rust decides what that choice resolves to.
|
|
|
|
## Design
|
|
|
|
### DR-121 — moved out (shipped)
|
|
|
|
Bitrate selection in the player shipped as DR-225 … DR-228; see
|
|
[01-rust-backend.md](../architecture/01-rust-backend.md#stream-selection).
|
|
|
|
The one constraint here that the capture work still has to respect: a quality
|
|
change re-negotiates **within HLS**. Returning a progressive `stream.mp4` for a
|
|
transcode means playback never starts, because the server encodes the whole file
|
|
before serving a byte (DR-140). That is why DR-122 below abandons a capture on a
|
|
quality change rather than trying to splice one.
|
|
|
|
### DR-122 — The playback path is ephemeral
|
|
|
|
Playback bytes are not persisted unless DR-124 says they are keepable. No partial
|
|
capture is ever retained across a quality change: on change, any in-flight capture
|
|
for that session is abandoned and its partial file deleted.
|
|
|
|
### DR-123 — The download path is independent
|
|
|
|
Downloading the whole file is a separate operation through the existing download
|
|
manager, at one canonical quality (default `original`, the direct static copy),
|
|
using `/Videos/{id}/stream.mp4` — progressive and Range-capable, which is what
|
|
the resumable download worker relies on. It is unaffected by what playback is
|
|
doing, and playback is unaffected by it.
|
|
|
|
Once complete it becomes an ordinary download row, so everything already built on
|
|
top of downloads — offline browsing, `refresh_queue_local_sources`, the Downloads
|
|
page — picks it up with no further work.
|
|
|
|
**Prerequisite:** downloaded *video* is currently never played locally.
|
|
`repository_get_video_stream_url` goes straight to the online repo and
|
|
[player/[id]/+page.svelte:316](../../src/routes/player/[id]/+page.svelte#L316)
|
|
calls it with no local check — so a completed video download is still streamed.
|
|
This must be fixed or the whole feature is invisible for video.
|
|
|
|
### DR-124 — Keep playback bytes only when they *are* the download
|
|
|
|
Capture is enabled only where the played bytes and the canonical download artifact
|
|
are the same thing — a **direct-play** session. Then:
|
|
|
|
| Path | Mechanism |
|
|
|---|---|
|
|
| Android / ExoPlayer | `SimpleCache` + `CacheDataSource`, keyed by item id **and** media-source id so renditions never collide. LRU evictor sharing the existing smart-cache budget — not a second budget over the same disk. |
|
|
| Linux audio / MPV | `stream-record`, set through the existing `set_property` plumbing. |
|
|
| Linux video (HLS transcode) | **Not captured.** Segments are not a file; assembling one needs ffmpeg, which is not a dependency and which CI is forbidden from installing at job time. The download path (DR-123) covers this case instead. |
|
|
|
|
Two abandonment rules, both of which must delete the partial rather than promote
|
|
it:
|
|
|
|
- **Seek during an mpv capture.** `stream-record` is documented as intended for
|
|
linear streams; seeking breaks the recording. Straight-through listening
|
|
captures, scrubbing does not.
|
|
- **Any quality change** (DR-122).
|
|
|
|
### DR-125 — Promotion, rendition, and invalidation
|
|
|
|
A capture is promoted to a `downloads` row (`status = 'completed'`) only when it
|
|
covers the whole resource. Partial captures stay cache and remain evictable.
|
|
|
|
A new `downloads.source_rendition` column records the negotiated
|
|
quality/container/codec of whatever produced the bytes; `NULL` for rows fetched by
|
|
the existing paths, which are always `original`. This is what makes an "upgrade to
|
|
original" action possible later, and what stops a 720p capture and a 4K download
|
|
being indistinguishable rows.
|
|
|
|
**Invalidation.** A quality change never touches a file that already exists —
|
|
neither a permanent download nor a completed temporary one. Both remain valid
|
|
copies of the rendition they hold, and deleting either would throw away bytes
|
|
already paid for.
|
|
|
|
What a quality change *does* invalidate is an **in-flight** capture or background
|
|
download of cached media: it is abandoned and restarted at the newly chosen
|
|
quality, because a capture spanning a rendition change is a splice of two
|
|
encodings rather than a playable file (DR-122).
|
|
|
|
So the rule is about *ongoing* work, not stored files. Nothing in this spec
|
|
deletes user data.
|
|
|
|
### Gating
|
|
|
|
Capture and background download obey the existing WiFi-only gate and storage
|
|
budget, and are off unless opted in. Enforcement is in Rust.
|
|
|
|
## Out of scope
|
|
|
|
- **Mid-playback switch onto a completing download.** Two independent paths make
|
|
it unnecessary; the download is used from the next boundary.
|
|
- **Backfilling the unplayed remainder of a capture.** Watch 40 minutes and you
|
|
have 40 minutes; completing it needs sparse-range bookkeeping and a resumable
|
|
tail fetch. The DR-123 download path already produces a complete file, which is
|
|
the reason this can wait.
|
|
- **Bundling ffmpeg** to make transcoded video capturable. Real option, large
|
|
packaging decision, its own proposal.
|
|
- **Routing Linux video playback through `stream.mp4`.** Regresses a documented,
|
|
hard-won fix.
|
|
|
|
## Acceptance criteria
|
|
|
|
- [ ] The player offers the qualities Rust reports, and changing one resumes at
|
|
the same position with audio/subtitle selection preserved.
|
|
- [ ] A quality change abandons any in-flight capture and leaves no partial file.
|
|
- [ ] A quality change never deletes a `downloads` row.
|
|
- [ ] A completed background download of a video is *played from disk* on the next
|
|
play (the DR-123 prerequisite).
|
|
- [ ] A direct-play session played start-to-finish leaves a complete local file
|
|
with no second fetch; replaying it fetches no media bytes.
|
|
- [ ] Seeking during an mpv capture abandons it; no truncated file is promoted.
|
|
- [ ] A transcoded Linux video session is never captured, and never partially
|
|
promoted.
|
|
- [ ] Promoted rows record their rendition; existing paths still record
|
|
`NULL`/`original`.
|
|
- [ ] Gates hold with the setting off *and* with the frontend never sending it.
|
|
- [ ] Eviction cannot delete bytes backing a promoted download row.
|
|
- [ ] `bun run check`, `bun run test`, `cargo fmt`, `cargo clippy`,
|
|
`bun run test:rust`, `bun run check:boundary` pass; `bindings.ts`
|
|
regenerated if Rust types changed.
|
|
|
|
## Testing
|
|
|
|
Rust, table-driven and pure where possible: quality→params resolution shared with
|
|
the download path; keepability (direct play vs transcode vs gate off); promotion
|
|
(complete → promoted, partial → not, seek-abandoned → not, quality-changed → not);
|
|
invalidation (evicts cache, never a download row); rendition round-trip.
|
|
|
|
Android: instrumented — a played direct-play item yields cache entries, and a
|
|
replay issues no media network request.
|
|
|
|
Frontend: the quality list renders from backend data with no item-type or
|
|
codec taxonomy in `src/`; the selector's remembered choice is a view preference.
|
|
|
|
## TRACES
|
|
|
|
| Piece | Tag |
|
|
|---|---|
|
|
| Ephemeral playback / capture abandonment | `// TRACES: UR-070 \| DR-122` |
|
|
| Independent whole-file download + local video playback fix | `// TRACES: UR-071 \| DR-123, IR-032` |
|
|
| ExoPlayer cache / mpv stream-record / keepability | `// TRACES: UR-071 \| DR-124` |
|
|
| Promotion, `source_rendition`, invalidation | `// TRACES: UR-071 \| DR-125` |
|
|
|
|
## Notes for the implementer
|
|
|
|
- **A parallel Claude session is active in this repo.** `git diff` before
|
|
"repairing" anything you did not write.
|
|
- Do not duplicate the quality→transcode-parameter table. Call the existing one.
|
|
- Reuse the smart-cache storage budget; two budgets over one disk is how devices
|
|
fill up.
|
|
- The `downloads` FK to `items` is relaxed (migration 005) — exercise promotion
|
|
for an item that was never cached.
|
|
- Build DR-123's local-playback fix first. Without it nothing in this spec is
|
|
observable for video.
|