first commit

SPEC.md

@@ -0,0 +1,108 @@
+# JRay truth file format
+
+JRay reads "truth" files produced offline by the
+[scene-actor-extraction](https://github.com/dtourolle/scene-actor-extraction)
+pipeline (`result_sink_node`, `Verbosity::minimal`, `schema_version: 1`).
+
+## File location
+
+For a media file `Movie.mkv`, the pipeline writes a sibling file
+`Movie.jray.json` (suffix configurable in the plugin settings, default
+`.jray.json`). The plugin resolves this path from the Jellyfin item's media
+source path by stripping the extension and appending the suffix.
+
+## JSON schema (schema_version 1, minimal verbosity)
+
+```json
+{
+  "schema_version": 1,
+  "movie": "/path/to/Movie.mkv",
+  "sample_fps": 1,
+  "anneal_sec": 2,
+  "actors": [
+    {
+      "name": "Tom Hanks",
+      "imdb_id": "nm0000158",
+      "tmdb_id": "31",
+      "jellyfin_id": "abc123-guid",
+      "scenes": [[12.0, 45.0], [102.5, 150.0]]
+    }
+  ]
+}
+```
+
+- `schema_version`: integer, bump on breaking changes. JRay should refuse (or
+  warn) on a version it doesn't understand.
+- `movie`: absolute path to the source media file at extraction time (informational only).
+- `sample_fps`: frames-per-second the pipeline sampled at.
+- `anneal_sec`: gap (in seconds) below which consecutive detections of the
+  same actor were merged into a single scene window.
+- `actors[]`: one entry per actor detected anywhere in the film.
+  - `name`: display name from the gallery.
+  - `imdb_id` / `tmdb_id` / `jellyfin_id`: identity keys, each `""` if not
+    resolved. JRay should prefer `jellyfin_id` (a Jellyfin Person item GUID)
+    when non-empty, and otherwise resolve `imdb_id`/`tmdb_id` against the
+    item's People `ProviderIds`.
+  - `scenes`: list of `[start_sec, end_sec]` windows (inclusive) during which
+    the actor is on screen.
+
+## Querying "who's on screen at time t"
+
+For a given timestamp `t` (seconds), an actor is visible if any of their
+`scenes` windows satisfies `start <= t <= end`.
+
+## API
+
+### `GET /Plugins/JRay/Items/{itemId}/Timeline`
+
+Returns the full truth file (schema above) for an item, or `404` if no truth
+data exists (neither a managed upload nor a sidecar file).
+
+### `GET /Plugins/JRay/Items/{itemId}/jray?t={seconds}`
+
+Returns an extensible "context at time t" envelope, or `404` if no truth
+data exists for the item:
+
+```json
+{
+  "actors": [
+    { "name": "Tom Hanks", "imdb_id": "nm0000158", "tmdb_id": "31", "jellyfin_id": "abc123-guid" }
+  ]
+}
+```
+
+Future fields (e.g. `locations`, `trivia`) will be added to this object
+without changing the route, so clients should ignore unknown keys.
+
+### `PUT /Plugins/JRay/Items/{itemId}/Truth`
+
+For servers that cannot run the extraction pipeline locally, a remote worker
+may push truth data directly. Requires an administrator API key. Body is a
+truth file (schema above). Returns `204` on success, or `400` if
+`schema_version` is not `1`.
+
+This "managed" truth data takes precedence over any sidecar
+`Movie.jray.json` file for the same item, and is stored independently of the
+media library filesystem.
+
+### `DELETE /Plugins/JRay/Items/{itemId}/Truth`
+
+Removes managed truth data for an item (idempotent, always returns `204`).
+The item falls back to its sidecar truth file, if any, on subsequent reads.
+Requires an administrator API key.
+
+### `GET /Plugins/JRay/ClientScript`
+
+Serves the pause-overlay script that JRay injects into the web client's
+`index.html` (see below). Anonymous access.
+
+## Web client pause overlay
+
+Since Jellyfin has no plugin hook for player UI, JRay injects
+`<script defer src="/Plugins/JRay/ClientScript"></script>` into the web
+client's `index.html` on startup (idempotent, marked with
+`<!-- jray-overlay -->`). The injected script listens for the video player's
+pause event, calls `jray?t=` for the current item and timestamp, and renders
+a small overlay listing on-screen actors. This can be disabled via the
+plugin's "Enable pause overlay" setting, which also removes the injected
+script.