dtourolle/jellytau
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 40 Wiki Activity

feat(video): render mpv behind the webview, and collapse duplicated helpers

Browse Source 
DR-231 with the design the failed reparent forced. mpv's render API draws into
an FBO we own; the texture is composited by `gdk_cairo_draw_from_gl()` in the
default vbox's own `draw` handler. GTK draws a container before its children, so
the webview lands on top for free — no reparenting, no GtkOverlay, and nothing a
Tauri upgrade can invalidate by assuming its own widget layout.

Split so Windows inherits the useful half: `mpv_render` is the portable side
(render context, framebuffer, GL resolution) and `video_surface` is the GTK side
that consumes it. Nothing in the former is GTK-aware.

Three things the spike paid for, carried over rather than rediscovered:

  - libepoxy exports GL entry points as *data* symbols. `dlsym("epoxy_glFoo")`
    returns the address *of a function pointer*, not of code — returning it
    makes mpv jump into non-executable data and take SIGSEGV on the first GL
    call. The value is read out of that location instead.
  - Frame pacing goes through mpv's update callback plus `report_swap`. Its
    absence looks like a GPU or compositing limit (fine in a window, judders at
    fullscreen) and is neither.
  - The render context is created on `realize` and destroyed on `unrealize`,
    with the update callback unregistered *before* the free, so a callback
    cannot land on a freed pointer. That is DR-232 built in from the start
    rather than retrofitted: the spike had no teardown at all, which remains the
    likeliest explanation for the one SIGSEGV it could not reproduce.

Writing it also caught a bug that would have looked like severe stutter: the
update callback flagged a new frame but never asked GTK to repaint, so decoded
frames would only have reached the screen when something else happened to
invalidate the widget.

Still off by default behind JELLYTAU_NATIVE_VIDEO=1. It compiles and is wired;
no frame has been put on screen yet.

Redundant code, continued. `formatSecondsDuration` had no caller. Three
components had hand-rolled `formatDuration`: Queue's was byte-equivalent to the
shared "mm:ss", while EpisodeFocusView and the library page shared an identical
"1h 23m" shape the util did not offer — so that format joins the other two and
all three components now call one function.

A survey for exported symbols referenced only by tests returns 23 more. They are
deliberately left: spot-checking found `setLogForwarder` is the injection seam
for a lazily-initialised forwarder, and `getCachedImageUrl` is the read path of
a thumbnail cache whose management UI exists in Settings. Neither is dead — one
is test infrastructure and the other is an unwired feature, and deleting either
would remove capability while looking like tidying. The list is worth working
through deliberately, not in a playback branch.
This commit is contained in:
dtourolle
2026-08-22 13:45:04 +02:00
parent 7545de6cc7
commit 45144cb6b0
11 changed files with 1471 additions and 1010 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src-tauri/src/player/mpv_render.rs
+374
View File
@@ -0,0 +1,374 @@
//! mpv's render API, driven into an OpenGL framebuffer we own.
//!
//! This is the half of native video that is not GTK: create a render context
//! over the mpv handle the audio backend already drives, render a frame into a
//! texture, and hand that texture id back for the toolkit to composite.
//!
//! Kept apart from `video_surface` deliberately — everything here is portable
//! across the platforms this app targets, while the surface that consumes it is
//! not. Windows reuses this file unchanged (DR-237).
//!
//! TRACES: UR-080 | DR-231, DR-232, IR-033
use std::ffi::{c_void, CStr, CString};
use std::os::raw::{c_char, c_int};
use std::ptr;
use log::{error, info, warn};
/// GL entry points, resolved once.
///
/// Only the handful needed to own a framebuffer; mpv resolves everything else
/// it needs through [`get_proc_address`].
struct Gl {
gen_framebuffers: unsafe extern "C" fn(c_int, *mut u32),
delete_framebuffers: unsafe extern "C" fn(c_int, *const u32),
bind_framebuffer: unsafe extern "C" fn(u32, u32),
framebuffer_texture_2d: unsafe extern "C" fn(u32, u32, u32, u32, c_int),
gen_textures: unsafe extern "C" fn(c_int, *mut u32),
delete_textures: unsafe extern "C" fn(c_int, *const u32),
bind_texture: unsafe extern "C" fn(u32, u32),
tex_image_2d:
unsafe extern "C" fn(u32, c_int, c_int, c_int, c_int, c_int, u32, u32, *const c_void),
tex_parameteri: unsafe extern "C" fn(u32, u32, c_int),
check_framebuffer_status: unsafe extern "C" fn(u32) -> u32,
}
const GL_TEXTURE_2D: u32 = 0x0DE1;
const GL_FRAMEBUFFER: u32 = 0x8D40;
const GL_COLOR_ATTACHMENT0: u32 = 0x8CE0;
const GL_RGBA: u32 = 0x1908;
const GL_RGBA8: c_int = 0x8058;
const GL_UNSIGNED_BYTE: u32 = 0x1401;
const GL_LINEAR: c_int = 0x2601;
const GL_TEXTURE_MIN_FILTER: u32 = 0x2801;
const GL_TEXTURE_MAG_FILTER: u32 = 0x2800;
const GL_FRAMEBUFFER_COMPLETE: u32 = 0x8CD5;
/// Resolve a GL symbol the way libepoxy actually exports it.
///
/// **This is the trap that cost the spike a debugging cycle.** libepoxy does not
/// export `glFoo` as a function. It exports `epoxy_glFoo` as a *data* symbol
/// holding a lazily-resolving function pointer. So the address `dlsym` returns
/// is the address *of the pointer*, not of any code: returning it makes mpv jump
/// into non-executable data and take SIGSEGV/SEGV_ACCERR on the very first GL
/// call. The value must be read *out of* that location.
///
/// The `epoxy` crate does this correctly and is unusable here — its
/// `gl_generator` dependency pulls a yanked `xml-rs`.
///
/// TRACES: UR-080 | IR-033
unsafe fn resolve(name: &str) -> *mut c_void {
let epoxy_name = match CString::new(format!("epoxy_{name}")) {
Ok(n) => n,
Err(_) => return ptr::null_mut(),
};
let slot = libc::dlsym(libc::RTLD_DEFAULT, epoxy_name.as_ptr());
if !slot.is_null() {
// The symbol holds the function pointer; return what is stored there.
return *(slot as *mut *mut c_void);
}
// Fall back to a plain symbol, for a GL stack that is not behind epoxy.
match CString::new(name) {
Ok(n) => libc::dlsym(libc::RTLD_DEFAULT, n.as_ptr()),
Err(_) => ptr::null_mut(),
}
}
/// What mpv calls to find GL entry points. Same rule as [`resolve`].
unsafe extern "C" fn get_proc_address(_ctx: *mut c_void, name: *const c_char) -> *mut c_void {
if name.is_null() {
return ptr::null_mut();
}
match CStr::from_ptr(name).to_str() {
Ok(n) => resolve(n),
Err(_) => ptr::null_mut(),
}
}
macro_rules! load {
($name:literal) => {{
let p = resolve($name);
if p.is_null() {
error!("[MpvRender] GL symbol not found: {}", $name);
return None;
}
std::mem::transmute(p)
}};
}
impl Gl {
/// Resolve every entry point, or none — a partially-loaded table would fail
/// later at a call site with no context.
///
/// The transmutes are unannotated on purpose: each target type is declared
/// once on the struct field above, and repeating it at the call site would
/// be two places to get the same signature wrong.
#[allow(clippy::missing_transmute_annotations)]
unsafe fn load() -> Option<Self> {
Some(Gl {
gen_framebuffers: load!("glGenFramebuffers"),
delete_framebuffers: load!("glDeleteFramebuffers"),
bind_framebuffer: load!("glBindFramebuffer"),
framebuffer_texture_2d: load!("glFramebufferTexture2D"),
gen_textures: load!("glGenTextures"),
delete_textures: load!("glDeleteTextures"),
bind_texture: load!("glBindTexture"),
tex_image_2d: load!("glTexImage2D"),
tex_parameteri: load!("glTexParameteri"),
check_framebuffer_status: load!("glCheckFramebufferStatus"),
})
}
}
/// A colour-renderable framebuffer mpv draws into, sized to the widget.
struct Target {
fbo: u32,
texture: u32,
width: i32,
height: i32,
}
/// mpv's render context plus the framebuffer it draws into.
///
/// # Lifetime (DR-232)
///
/// The render context must not outlive the GL context it was created against.
/// `Drop` unregisters mpv's update callback *before* freeing the context, so a
/// callback cannot land on a freed pointer, and frees the GL objects while the
/// caller still has the context current. The caller is responsible for making
/// the GL context current around both creation and drop — see `video_surface`.
///
/// This is DR-184 on Android restated: a surface outliving its player. The spike
/// had no defence at all and saw one unexplained SIGSEGV in a decoder thread.
pub struct MpvRenderContext {
ctx: *mut libmpv_sys::mpv_render_context,
gl: Gl,
target: Option<Target>,
}
// The render context is driven only from the GTK main thread; the update
// callback merely schedules a redraw and touches nothing here.
unsafe impl Send for MpvRenderContext {}
impl MpvRenderContext {
/// Create a render context over an existing mpv handle.
///
/// The GL context must already be current on this thread.
///
/// TRACES: UR-080 | DR-231, IR-033
pub unsafe fn new(mpv: *mut libmpv_sys::mpv_handle) -> Option<Self> {
let gl = Gl::load()?;
let mut init = libmpv_sys::mpv_opengl_init_params {
get_proc_address: Some(get_proc_address),
get_proc_address_ctx: ptr::null_mut(),
};
let mut api_type = CString::new("opengl").ok()?;
// `advanced_control` lets mpv tell us when a frame is ready instead of
// us guessing; it is what makes the update callback meaningful (DR-233).
let mut advanced: c_int = 1;
let mut params = [
libmpv_sys::mpv_render_param {
type_: libmpv_sys::mpv_render_param_type_MPV_RENDER_PARAM_API_TYPE,
data: api_type.as_ptr() as *mut c_void,
},
libmpv_sys::mpv_render_param {
type_: libmpv_sys::mpv_render_param_type_MPV_RENDER_PARAM_OPENGL_INIT_PARAMS,
data: &mut init as *mut _ as *mut c_void,
},
libmpv_sys::mpv_render_param {
type_: libmpv_sys::mpv_render_param_type_MPV_RENDER_PARAM_ADVANCED_CONTROL,
data: &mut advanced as *mut _ as *mut c_void,
},
libmpv_sys::mpv_render_param {
type_: 0,
data: ptr::null_mut(),
},
];
let mut ctx: *mut libmpv_sys::mpv_render_context = ptr::null_mut();
let rc = libmpv_sys::mpv_render_context_create(&mut ctx, mpv, params.as_mut_ptr());
// Keep the CString alive until after the call.
let _ = &mut api_type;
if rc < 0 || ctx.is_null() {
error!("[MpvRender] mpv_render_context_create failed: {rc}");
return None;
}
info!("[MpvRender] render context created");
Some(MpvRenderContext {
ctx,
gl,
target: None,
})
}
/// Ask to be told when a new frame is ready.
///
/// Paired with [`report_swap`](Self::report_swap): without both, mpv has
/// nothing to time against. The symptom is misleading — playback looks fine
/// in a window and judders at fullscreen, which reads as a compositing or
/// GPU limit and is neither (DR-233).
///
/// TRACES: UR-080 | DR-233
pub unsafe fn set_update_callback(
&mut self,
callback: libmpv_sys::mpv_render_update_fn,
ctx: *mut c_void,
) {
libmpv_sys::mpv_render_context_set_update_callback(self.ctx, callback, ctx);
}
/// Render the current frame at `width` x `height`, returning the texture id
/// holding it. The GL context must be current.
///
/// TRACES: UR-080 | DR-231
pub unsafe fn render(&mut self, width: i32, height: i32) -> Option<u32> {
if width <= 0 || height <= 0 {
return None;
}
self.ensure_target(width, height)?;
let target = self.target.as_ref()?;
let mut fbo = libmpv_sys::mpv_opengl_fbo {
fbo: target.fbo as c_int,
w: width as c_int,
h: height as c_int,
internal_format: 0,
};
// GTK's cairo surface has its origin at the top left; mpv defaults to
// OpenGL's bottom-left. Without this the picture is drawn upside down —
// which looks like a broken decode rather than a coordinate convention.
let mut flip: c_int = 1;
let mut params = [
libmpv_sys::mpv_render_param {
type_: libmpv_sys::mpv_render_param_type_MPV_RENDER_PARAM_OPENGL_FBO,
data: &mut fbo as *mut _ as *mut c_void,
},
libmpv_sys::mpv_render_param {
type_: libmpv_sys::mpv_render_param_type_MPV_RENDER_PARAM_FLIP_Y,
data: &mut flip as *mut _ as *mut c_void,
},
libmpv_sys::mpv_render_param {
type_: 0,
data: ptr::null_mut(),
},
];
let rc = libmpv_sys::mpv_render_context_render(self.ctx, params.as_mut_ptr());
if rc < 0 {
warn!("[MpvRender] render failed: {rc}");
return None;
}
Some(target.texture)
}
/// Tell mpv the frame reached the screen. See [`set_update_callback`].
///
/// TRACES: UR-080 | DR-233
pub unsafe fn report_swap(&self) {
libmpv_sys::mpv_render_context_report_swap(self.ctx);
}
/// Create or resize the framebuffer. Reused across frames — reallocating per
/// frame would churn GPU memory at the display rate.
unsafe fn ensure_target(&mut self, width: i32, height: i32) -> Option<()> {
if let Some(t) = &self.target {
if t.width == width && t.height == height {
return Some(());
}
}
self.drop_target();
let gl = &self.gl;
let mut texture: u32 = 0;
(gl.gen_textures)(1, &mut texture);
(gl.bind_texture)(GL_TEXTURE_2D, texture);
(gl.tex_image_2d)(
GL_TEXTURE_2D,
0,
GL_RGBA8,
width,
height,
0,
GL_RGBA,
GL_UNSIGNED_BYTE,
ptr::null(),
);
(gl.tex_parameteri)(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
(gl.tex_parameteri)(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
(gl.bind_texture)(GL_TEXTURE_2D, 0);
let mut fbo: u32 = 0;
(gl.gen_framebuffers)(1, &mut fbo);
(gl.bind_framebuffer)(GL_FRAMEBUFFER, fbo);
(gl.framebuffer_texture_2d)(
GL_FRAMEBUFFER,
GL_COLOR_ATTACHMENT0,
GL_TEXTURE_2D,
texture,
0,
);
let status = (gl.check_framebuffer_status)(GL_FRAMEBUFFER);
(gl.bind_framebuffer)(GL_FRAMEBUFFER, 0);
if status != GL_FRAMEBUFFER_COMPLETE {
error!("[MpvRender] framebuffer incomplete: 0x{status:x}");
(gl.delete_framebuffers)(1, &fbo);
(gl.delete_textures)(1, &texture);
return None;
}
self.target = Some(Target {
fbo,
texture,
width,
height,
});
Some(())
}
unsafe fn drop_target(&mut self) {
if let Some(t) = self.target.take() {
(self.gl.delete_framebuffers)(1, &t.fbo);
(self.gl.delete_textures)(1, &t.texture);
}
}
/// Free everything, with the GL context current.
///
/// Explicit rather than left to `Drop` because the ordering matters and the
/// caller is the only one that can guarantee the GL context is current. See
/// DR-232.
pub unsafe fn destroy(mut self) {
// Unregister first: a callback arriving after the free would be a use
// after free, and it is scheduled from mpv's own threads.
libmpv_sys::mpv_render_context_set_update_callback(self.ctx, None, ptr::null_mut());
self.drop_target();
libmpv_sys::mpv_render_context_free(self.ctx);
self.ctx = ptr::null_mut();
info!("[MpvRender] render context freed");
std::mem::forget(self);
}
}
impl Drop for MpvRenderContext {
fn drop(&mut self) {
if !self.ctx.is_null() {
// Reached only if `destroy` was not called — the GL context may not
// be current, so the GL objects are deliberately leaked rather than
// deleted against whatever context happens to be bound. Freeing the
// render context is still safe and is the part that matters.
warn!("[MpvRender] dropped without destroy(); GL objects leaked deliberately");
unsafe {
libmpv_sys::mpv_render_context_set_update_callback(self.ctx, None, ptr::null_mut());
libmpv_sys::mpv_render_context_free(self.ctx);
}
}
}
}