fix(playback): ask the renderer what it can decode, in one place
Four bugs, one cause. "What can this device decode" was answered in five
places, four of which assumed the webview was decoding:
- the device profile's direct-play codecs (cfg per platform, inline)
- the transcoding targets (hardcoded "h264,hevc")
- the direct-play audio narrowing (webview list, all platforms)
- the client-side audio override (webview list, all platforms)
- get_video_stream_url's VideoCodec (hardcoded "h264")
On Android the decoder is ExoPlayer, so four of those were simply wrong there,
and the costs were invisible without a device:
- dts is in the tablet's own codec list, gets stripped from the profile, and
is then forced to transcode by a rule about a renderer that is not playing
it.
- An hevc source whose *audio* is eac3 had its **picture fully re-encoded**.
The server's own transcoding URL got this right — VideoCodec=h264,hevc,
TranscodeReasons=AudioCodecNotSupported, video copied — but the moment a
quality change or track switch re-opened the stream through our builder,
the hardcoded h264 turned a cheap audio remux into a full transcode. That
is a quality change silently making playback more expensive, on the exact
path a viewer uses when playback is already struggling.
`renderer_codecs()` and `renderer_can_decode_audio()` are now the single
source, and all five sites read them. On the webview path every value resolves
exactly as before, so desktop behaviour is unchanged by construction; on
Android the profile becomes the device's own.
The list is also what lets the server *copy* rather than re-encode: naming
every codec the renderer can decode is what turns a transcode into a
passthrough when the source is already playable. That is the whole of "use the
best format available".
Also corrects this branch's headline number where it is asserted — the
architecture doc, the desktop-native-video spec and the spike. The measured 85%
Android direct-play rate used a profile containing ac3/eac3; the device it was
later verified on reports neither, so eac3 content correctly transcodes there.
It is a ceiling for an ExoPlayer-appropriate profile, not what the app achieves,
and realising any of it depends on this change. Left in place with the caveat
rather than deleted, because the measurement is real — it just measures
something narrower than it was quoted as measuring.
Unverified: this changes what Android negotiates and has not been exercised on
the tablet yet. Desktop is unchanged by construction but also unre-tested.
This commit is contained in:
@@ -770,11 +770,21 @@ a free passthrough as a server-side re-encode.
|
||||
> | Android / ExoPlayer (`h264,hevc,vp8,vp9,av1,mpeg4` + `ac3,eac3`, 6ch) | 34/40 — **85%** |
|
||||
>
|
||||
> The library is ~80% hevc (`hevc+eac3` alone is a third of it), which is why the
|
||||
> two diverge so hard. **The payoff is overwhelmingly Android**, where 85% of
|
||||
> plays previously burned a transcode nobody needed. Linux stays near 7% until
|
||||
> libmpv decodes the picture — the h264-only profile is a WebKitGTK constraint,
|
||||
> not a JellyTau choice, and is what `linux-native-video-spike.md` exists to
|
||||
> remove. A reviewer should not expect this code to fix Linux on its own.
|
||||
> two diverge so hard.
|
||||
>
|
||||
> **Read that 85% as a ceiling, not a result.** It was measured with a profile
|
||||
> containing `ac3,eac3`. The Android device this was later run on reports neither
|
||||
> in its `MediaCodecList` — no Dolby licence, which is normal for a tablet — so
|
||||
> eac3 content, about a third of the sampled library, correctly transcodes there.
|
||||
> What any given device achieves depends on its own codec list, and on the
|
||||
> profile being derived from the renderer at all (DR-233), which it was not when
|
||||
> the figure was taken.
|
||||
>
|
||||
> **The payoff is still overwhelmingly Android**, because that is where a real
|
||||
> decoder is already doing the work. Linux stays near 7% until libmpv decodes the
|
||||
> picture — the h264-only profile is a WebKitGTK constraint, not a JellyTau
|
||||
> choice, and is what `linux-native-video-spike.md` exists to remove. A reviewer
|
||||
> should not expect this code to fix Linux on its own.
|
||||
|
||||
#### The quality ladder per source
|
||||
|
||||
|
||||
@@ -51,8 +51,18 @@ server:
|
||||
| Android / ExoPlayer — `h264,hevc,vp8,vp9,av1,mpeg4` + `ac3,eac3`, 6ch | **85%** |
|
||||
|
||||
The sampled library is ~80% hevc. **Those rows differ only by which component
|
||||
decodes.** Moving the picture to mpv is what lets the desktop row claim what the
|
||||
machine can actually do, and that — not the compositing — is the product.
|
||||
decodes.**
|
||||
|
||||
Moving the picture to mpv is what lets the desktop row claim what the machine
|
||||
can actually do, and that — not the compositing — is the product.
|
||||
|
||||
> **The 85% is a ceiling, not a shipped result.** It was measured with a profile
|
||||
> containing `ac3,eac3`. The Android device later used for verification reports
|
||||
> neither in its `MediaCodecList` — no Dolby licence, normal for a tablet — so
|
||||
> eac3 content, about a third of the sampled library, correctly transcodes there.
|
||||
> Realising any of this depends on DR-233, deriving the profile from the renderer
|
||||
> rather than from the platform, which is why that requirement is load-bearing
|
||||
> and not tidy-up.
|
||||
|
||||
### One desktop video path, not two
|
||||
|
||||
@@ -210,7 +220,7 @@ applies to the video path — but the multichannel bound still does, since a 5.1
|
||||
track direct-played into a 2-channel sink is silence or inaudible dialogue. Both
|
||||
constraints stay, sourced from the renderer rather than assumed.
|
||||
|
||||
**This converts 7% into ~85%**, and it is also the change most able to break
|
||||
**This is what converts the 7% figure upward** (toward, not necessarily to, the 85% ceiling — see the caveat above), and it is also the change most able to break
|
||||
playback silently — so it lands after compositing is proven, covered by the
|
||||
DR-227 override tests.
|
||||
|
||||
@@ -346,7 +356,7 @@ and shrinks to the surface.
|
||||
does not, the multichannel bound survives both. The DR-233 table as a
|
||||
table-driven test.
|
||||
- **Rust, pure:** `PlaybackInfo` fixtures that transcode under the webview
|
||||
profile and direct-play under the mpv profile — the 7%→85% conversion as a unit
|
||||
profile and direct-play under the mpv profile — the direct-play conversion as a unit
|
||||
test, not only as a measurement.
|
||||
- **Rust:** teardown ordering — callback unregistered before context freed, freed
|
||||
before GL context destroyed. Structure it so the ordering is assertable without
|
||||
|
||||
@@ -315,6 +315,10 @@ anything.
|
||||
| Linux / WebKitGTK — `h264` only, 2ch | **7%** |
|
||||
| Android / ExoPlayer — `h264,hevc,vp8,vp9,av1,mpeg4` + `ac3,eac3`, 6ch | **85%** |
|
||||
|
||||
**The 85% is a ceiling, not a shipped result** — it was measured with a
|
||||
profile containing `ac3,eac3`, which the Android device later used for
|
||||
verification does not support.
|
||||
|
||||
The library sampled is ~80% hevc. Linux sits at 7% **solely because the
|
||||
WebKitGTK profile can only claim h264** — not because of anything about the
|
||||
server or the negotiation. mpv decodes hevc, so widening the Linux device
|
||||
|
||||
Reference in New Issue
Block a user