# Spec: Android native video — transparent-webview spike

**Status:** In progress — implemented behind `experimentalNativeVideo`, pending
on-device confirmation. Branch `feat/android-native-video`.
**Requirements:** IR-004, UR-003, UR-004 → DR-001, DR-023, DR-024
**UX spec:** n/a — no intended visual change; the video surface must land exactly where the `<video>` element is today
**Supersedes / revises:** acts on finding 2 of [playback-backend-unification.md](playback-backend-unification.md)

## Summary

Test whether ExoPlayer's existing `SurfaceView` video path can be composited
behind a transparent Tauri WebView on Android. If it works, Android regains
hardware video decoding (MediaCodec) and libass-quality ASS/SSA subtitles, both
of which the current webview path lacks. If it does not, we document why and
delete the dead code.

This is a **spike**, not a feature commitment. The deliverable is a yes/no answer
with evidence, plus either a working path behind a flag or a removal.

## Motivation

`createAdapter()` hardcodes `const effectiveKind = "html5"` and does
`void backendKind`, discarding the `use_html5_element` value Rust computes in
`get_player_status`. As a result:

- `NativePlayerAdapter` is dead code.
- `JellyTauPlayer.kt`'s `getOrCreateSurfaceView()` — which already calls
  `setZOrderMediaOverlay(false)` and wires `setVideoSurfaceHolder` — is
  unreachable.
- Android video decodes in the WebView instead of via MediaCodec, despite
  `CodecDetector.kt` going to the trouble of reporting hardware codec
  capabilities back to Rust for DeviceProfile generation.

The code comment in `nativeAdapter.ts:11-14` justifies this by citing
tauri#10152 as an upstream blocker. **That justification is stale.**

### Why the blocker no longer holds

- tauri#10152 is open but **dead since 2024-07-01**, and it is a *feature
  request* ("Support transparent webviews on mobile"), not a bug report about
  compositing.
- The capability shipped in tauri commit `27d01834` (2024-09-02) — a clippy
  cleanup that moved `transparent()` out of the desktop-gated impl block, fencing
  only the tao call behind `#[cfg(desktop)]`. Because it landed as unrelated
  cleanup, nobody closed the issue.
- The black/white-screen reports (tauri#8381, tauri#9408) were a real but
  *different* bug: a broken JNI signature for `setBackgroundColor`, fixed in
  **wry 0.39.4** (PR #1237). We ship wry 0.55.x.
- Current wry calls `setBackgroundColor(0)` unconditionally on Android when
  transparency is requested.

### The honest caveat

**Nobody has demonstrated SurfaceView-behind-WebView on Tauri Android.** A search
of both `tauri-apps/tauri` and `tauri-apps/wry` issues for `surfaceview` returns
zero results, and the one native-video Tauri plugin
(`YeonV/tauri-plugin-videoplayer`) sidesteps compositing by launching a separate
fullscreen Activity. Nothing upstream blocks this; nothing upstream proves it.
Hence: spike, not feature.

Note this is the *Android* question only. The equivalent Linux compositing
problem is maintainer-declared unfixable and is **not** in scope — see the
unification spec.

## Layer assignment

| Logic / responsibility | Layer | Why it belongs there |
|------------------------|-------|----------------------|
| Which video backend this platform uses | Rust (existing) | `get_player_status` already computes `use_html5_element`. The frontend must *consume* it, not decide it. Restoring that is the point of the spike. |
| Surface creation, z-ordering, `setVideoSurfaceHolder` lifecycle | Kotlin | Android platform mechanics; already written in `JellyTauPlayer.kt`. |
| Seek/audio-track *strategy* | Rust (existing) | Already returned by `player_seek_video` / `player_switch_audio_track`; `NativePlayerAdapter` executes the chosen primitive. Unchanged — this is exactly what the `PlayerAdapter` contract was built for. |
| Positioning the surface under the video viewport | Frontend | Pure presentation/layout. **This is the risk area** — see Design. |

## Design

### Phase 1 — prove compositing (no app changes)

Before touching the adapter factory, verify the primitive works at all:

1. Set `"transparent": true` in `tauri.conf.json` for the Android build, plus
   `html, body { background: transparent; }`.
2. Confirm the WebView is genuinely transparent (a native view behind it is
   visible) and that the app does not regress to a black/white screen.

If this fails, stop — everything downstream is moot, and the finding is that
Tauri Android transparency is still broken in practice despite the shipped fix.

### Phase 2 — un-hardcode the factory

```ts
// src/lib/player/adapters/index.ts
export function createAdapter({ backendKind, host, bridge }: CreateAdapterArgs): PlayerAdapter {
  return backendKind === "native"
    ? new NativePlayerAdapter(host)
    : new Html5PlayerAdapter(host, bridge);
}
```

`backendKind` comes from `get_player_status` (`VideoBackend::Native` on Android).
Gate behind a setting — `experimentalNativeVideo`, default **off** — so a broken
spike cannot ship as a regression. Rust already owns this decision; the flag only
suppresses it.

**Also in scope: remove the user-agent sniffing in
`src/lib/services/webviewAudio.ts:30-41`.** It re-derives which audio backend the
platform has from `navigator.userAgent` ("matching the Rust cfg gate", per its own
comment) — the frontend deciding a backend fact it should be told. Same root cause
as the hardcode above, same fix: consume the value Rust already computes. Fold it
in here rather than leaving a second, subtler copy of the bug behind. If
`get_player_status` does not currently expose enough to cover the audio case, add
the field — that is backend work, and correct.

### Implementation findings (2026-08-11)

Two blockers existed that this spec did not anticipate. Both were in code the
spec assumed was merely *unreachable*; it was also *broken*.

**1. The Kotlin attach chain was severed.** `JellyTauPlayer.setActivity()` had
**zero callers** anywhere in the tree. `currentActivity` was therefore always
null, so `autoAttachSurface()` logged "Cannot attach surface - no Activity
reference" and returned. The `SurfaceView` was created and wired to ExoPlayer but
never added to the view hierarchy — video would have decoded to a surface that
was never on screen, *regardless* of webview transparency. Fixed by calling
`JellyTauPlayer.setActivity(this)` from `MainActivity.onCreate`.

Note the knock-on: `PictureInPictureManager.canEnterPip()` gates on
`VideoOverlayManager.isVideoSurfaceAttached()`, which was permanently false. PiP
on the video path was dead for the same reason.

**2. `createAdapter()` was not the real gate.** It is never called by production
code — `VideoPlayer.svelte` constructs `Html5PlayerAdapter` directly. The actual
override was `VideoPlayer.svelte`'s INTERIM block, which read Rust's
`useHtml5Element`, forced it to `true`, and called `playerStop()` to kill the
native backend `player_play_item` had just started. Both sites are now fixed;
`VideoPlayer.svelte` routes through `createAdapter()` so there is one gate.

**Transparency needs two independent layers cleared,** not one. The spec's
Phase 1 named only `html, body`. Clearing just the page leaves the WebView
widget's own background opaque, which is a black screen with audio — the exact
symptom the INTERIM comment described as "native surface not visible". Both are
now toggled together by `$lib/utils/videoSurface.ts`:

| Layer | Cleared by | Reachable from |
|-------|-----------|----------------|
| WebView widget background + window drawable | `AndroidVideoSurface.setTransparent()` (MainActivity) | Kotlin only |
| `html`/`body` + app-shell `--color-background` | `data-native-video` attribute → app.css | CSS only |

Transparency is scoped to `tauri.android.conf.json` rather than the base config:
a transparent window on Linux is a regression, since nothing renders behind it.
It is also toggled per-session rather than set once — a permanently transparent
window shows the launcher through the rest of the app.

### Phase 3 — surface positioning

The hard part, and where this most likely fails. The webview's `<video>` element
occupies a laid-out box; the `SurfaceView` must be positioned to match it, and
kept matched through scroll, rotation, and mini-player transitions.

Approach: the video view reports its `getBoundingClientRect()` to Rust, which
forwards the rect to Kotlin to position the `SurfaceView`. This is the same
"faking it" technique the ecosystem uses on desktop — acceptable here *only if*
the video is effectively fullscreen on Android, which it is in the player route.

**Explicit failure criterion**: if the surface cannot be kept aligned during
rotation or the mini-player transition without visible artefacts, the spike fails
and we keep HTML5. Do not ship a janky native path for a codec win.

**Update: no rect plumbing was needed.** The premise — that the surface must be
positioned to match a laid-out `<video>` box — does not hold on the player route,
where video is fullscreen. `VideoOverlayManager` adds the SurfaceView at index 0
of `android.R.id.content` with `MATCH_PARENT`, and `fitSurfaceToScreen()`
(`JellyTauPlayer.kt`) already letterboxes/pillarboxes to the real video aspect
ratio and re-centres via a `Gravity.CENTER` `FrameLayout.LayoutParams`. Rotation
is handled by an `OnLayoutChangeListener` that re-fits on any bounds change. The
frontend's native branch is a bare `flex-1` box, so there is no rect to report
and nothing to keep in sync.

This does mean **the mini-player transition is untested** for the native path —
it is the one case the fullscreen assumption does not cover, and it remains an
on-device check.

### A trap for the next implementer

There is a **stale duplicate player** at
`src-tauri/android/app/src/main/java/com/dtourolle/jellytau/player/JellyTauPlayer.kt`
(only commit: `cfddc1e` "First working POC"). No `sourceSets` entry points at it,
so it is not compiled — but edits made there silently do nothing. The canonical
tree is `src-tauri/android/src`, synced into `gen/` by
`scripts/sync-android-sources.sh`.

### What we gain if it works

- **Hardware decode via MediaCodec** — `CodecDetector.kt` already reports
  capabilities; the DeviceProfile would finally match what actually plays.
- **ASS/SSA subtitles** are *not* automatic. ExoPlayer cannot render them; that
  would require libmpv, which is a separate and much larger decision (see the
  unification spec's engine comparison). Scope this spike to hardware decode
  only, and do not claim subtitle improvements from it.

## Out of scope

- Linux native video. Maintainer-declared unfixable on WebKitGTK/Wayland.
- Replacing ExoPlayer with libmpv on Android.
- Windows native video.
- Removing the HTML5 path. It stays as the default and the fallback.

## Acceptance criteria

The spike is **complete** when one of these is true:

**Success path**
- [x] Transparent WebView confirmed working on a physical device (reported by the maintainer; the config that enables it is now committed in `tauri.android.conf.json`).
- [x] `experimentalNativeVideo` off → behaviour byte-identical to today. Guarded by `adapterSelection.test.ts`, which asserts the flag-off case forces HTML5 even when Rust reports native.
- [x] `webviewAudio.ts` no longer inspects `navigator.userAgent`; the platform's audio backend is read from Rust (`player_get_capabilities` → `usesWebviewAudio`).
- [ ] `experimentalNativeVideo` on → video plays via ExoPlayer/MediaCodec, correctly positioned, with working seek, audio-track switch, and subtitle selection through the existing `PlayerAdapter` contract.
- [ ] No artefacts on rotation, background/foreground, or mini-player transition.
- [ ] `adb shell dumpsys media.metrics` (or logcat) confirms a hardware decoder is in use.
- [ ] Measured battery/thermal or CPU improvement over the HTML5 path on the same clip.

**Failure path**
- [ ] The blocking behaviour is documented in this spec with evidence.
- [ ] `NativePlayerAdapter` and the unreachable `SurfaceView` code are deleted, or explicitly retained with a *correct* comment.
- [ ] `nativeAdapter.ts:11-14` no longer cites tauri#10152.

Either way:
- [x] `bun run check` (0 errors), `bun run test` (892 passed), `bun run check:boundary` pass.
- [x] `cargo fmt` / `cargo clippy` clean (no new warnings); `cargo test` passes (603 lib + 7 doc).

> Note: this environment has no host WebKitGTK dev packages, no Android SDK and
> no `bun`, so all of the above were run inside the CI builder image
> (`gitea.tourolle.paris/dtourolle/jellytau-builder:latest`). On Fedora the bind
> mount needs `:z` for SELinux, and `scripts/build-android.sh` hardcodes
> `ANDROID_HOME="$HOME/Android/Sdk"`, so the image's SDK at `/opt/android-sdk`
> must be symlinked there rather than passed by env var.

## Testing

Adapter-selection logic is pure and testable without a device: assert
`createAdapter` returns `NativePlayerAdapter` for `backendKind: "native"` with
the flag on, and `Html5PlayerAdapter` in every other combination — including that
the flag off forces HTML5 even when Rust says native. That last case is the
regression guard.

Everything else is manual on-device; there is no meaningful way to unit-test
surface compositing. Test on at least two devices — compositing behaviour varies
by OEM and Android version.

Per CLAUDE.md, if the spike turns into a bug fix (e.g. seek breaks under the
native adapter), write the failing test first.

## TRACES

- `createAdapter` → `// TRACES: UR-003, UR-004 | DR-023, DR-024`
- Adapter-selection tests → `UT-xxx`
- No new requirement IDs; this spike either satisfies existing IR-004 expectations or documents why it cannot.

## Notes for the implementer

- **Do not skip Phase 1.** If transparency does not work, phases 2 and 3 are
  wasted effort.
- `VideoPlayer.svelte` has a documented hazard: no lifecycle calls after an
  `await` in `onMount` — it flips to HTML5 mode and breaks Android seek. The
  adapter swap touches exactly this code path.
- tauri-specta tagged responses keep Rust field names (`new_url`, not `newUrl`).
- Android source edits go in `src-tauri/android/src`, then run
  `scripts/sync-android-sources.sh`.
- A parallel Claude session may be active — `git diff` first.