yorickr/plezy
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(linux): HDR video on a native Wayland plane

Browse Source 
Video on Linux went through a Flutter texture: 8-bit sRGB, which cannot carry
HDR at all, and which forced a whole-window Flutter recomposite for every video
frame. This moves it onto a wl_subsurface stacked below the Flutter surface, with
mpv rendering into an EGL window surface on it through the libmpv render API. The
subsurface is desynchronized, so video and UI now present independently.

With the plane in place HDR follows: the surface is described to the compositor
through wp_color_manager_v1 as the source's own curve and gamut - PQ or HLG,
BT.2020 - carrying whatever HDR10 static metadata the stream actually declares.
The description and the buffer it describes land on the same commit, staged and
validated before mpv is switched, so a PQ frame is never presented labelled sRGB.
A five-second watchdog bounds the one wait a compositor could otherwise leave
hanging. A session that cannot host the plane - X11, or a compositor without
wl_subcompositor - fails initialize with VIDEO_PLANE_UNSUPPORTED naming the
reason: the texture path is gone, and refusing by name beats degrading to
something the user cannot see. An SDR output, a missing capability or an 8-bit
config keep the plane and simply leave it undescribed.

The output's colour state is trusted only when it has been earned. Every landed
property step records itself as it lands; a reset or sequence that cannot
finish downgrades its result to unknown and marks the applied-output cache
untrusted until a clean apply earns it back. A plane whose output state cannot
be named is quarantined - hidden, its description withdrawn - and the
quarantine is recorded state: an unrelated visibility change cannot put a
mislabelled plane back on screen, and only a commit that resolves to a nameable
outcome lifts it. A rect collapsing to zero detaches the buffer exactly as
hiding does, a refused setVideoRect drops the Dart-side sent-rect cache so the
next layout pass retries for free, and a refused tone-mapping pick tells the
user instead of dying in a log.

NVIDIA's Wayland EGL (through at least 610.xx) offers no 10-bit unorm window
configs, so the plane takes half-float as the tier between 10-bit unorm and
8-bit, declares the whole surface opaque so the compositor never reads the
alpha those configs carry, and states GL_RGBA16F rather than a 10-bit lie.
Whether the output is in HDR is read from luminance headroom above its own
reference white rather than from the preferred transfer function, which current
KWin no longer answers PQ for; the margin is half a stop, because KWin reports
an undimmed maximum over a software-dimmed SDR white. Validated on an RTX 4090
(driver 610.57.04) under KWin 6.7.4 with locked-exposure photographs.

Who tone-maps is a user choice. The default is the compositor: photographed on a
400-nit HDR output against a PQ chart it keeps 400 -> 1000 nits monotonic and
separated where the player leg flattens them, because the player path drives
mpv's legacy vo_gpu, whose own standalone output scores the same. The gap is the
renderer, not the wiring.

The decision itself - what the source carries, what the output supports, what to
tell mpv and what to tell the compositor - lives in hdr_metadata.h, free of
Wayland and GTK so its luminance validation can be tested without a display
server. Sending an incoherent luminance set is a protocol error that disconnects
the client, so the rules are worth a unit test.

The deb, rpm and pacman packages now declare wayland-client, wayland-egl and EGL:
the plane links them directly and bundle-libs.sh deliberately never bundles them,
since they are coupled to the running compositor and GPU driver.

lib/dev/harness_main.dart is a second entrypoint for measuring this on hardware -
it drives one clip with scripted mpv properties and reports the colour state mpv
actually settled on. Nothing imports it, so it is tree-shaken out of the app.

Verified on a Steam Deck against an external 400-nit HDR display: the compositor
reports PQ / BT.2020, the connector carries HDR_OUTPUT_METADATA, and against mpv
vo=gpu-next on the same frame the shipped build sits 4.90 counts away overall -
closer to the reference HDR player than to its own SDR fallback.
This commit is contained in:
edde746
2026-08-10 08:48:13 +02:00
parent c27dc0a1a7
commit bcd6fe9906
76 changed files with 8025 additions and 1833 deletions
Download Patch File Download Diff File Expand all files Collapse all files
linux/runner/mpv/mpv_player.cc
+587 -161
View File
@@ -4,15 +4,25 @@
#include <epoxy/gl.h>
#include <flutter_linux/flutter_linux.h>
#include <gdk/gdk.h>
#ifdef GDK_WINDOWING_X11
#include <gdk/gdkx.h>
#endif
#ifdef GDK_WINDOWING_WAYLAND
#include <gdk/gdkwayland.h>
#endif
#include <locale.h>
// EGL 1.5 names; EGL_KHR_create_context introduced the same values earlier.
// Declared here so the build does not depend on which EGL headers the distro
// ships - the runtime check is eglCreateContext refusing the attribute, which
// the caller already falls back from.
#ifndef EGL_CONTEXT_MAJOR_VERSION
#define EGL_CONTEXT_MAJOR_VERSION EGL_CONTEXT_CLIENT_VERSION
#endif
#ifndef EGL_CONTEXT_MINOR_VERSION
#define EGL_CONTEXT_MINOR_VERSION 0x30FB
#endif
#include <chrono>
#include <cstdint>
#include <cstring>
#include "sanitize_utf8.h"
@@ -27,6 +37,17 @@ bool EnsureProcessNumericLocale() {
return configured;
}
// Reply userdata for the runner's own `video-params` observation.
//
// Every Dart-facing observation takes its userdata from
// PropertyObservationRegistry, which hands out 1, 2, 3, … one per distinct
// property name and never resets the counter; the two audio observations below
// pass 0, which the registry also never hands out. UINT64_MAX is the one value
// the counter cannot reach without first wrapping — and a wrap would collide
// with those two just as surely, so the scheme already depends on it not
// happening.
constexpr uint64_t kVideoParamsUserdata = UINT64_MAX;
} // namespace
// Flutter on Linux uses EGL (OpenGL ES) for both X11 and Wayland.
@@ -193,15 +214,6 @@ bool TryReleaseNativeRenderTeardown(
return true;
}
bool TryReleaseRetainedNativeRenderContexts(
std::vector<NativeRenderTeardownResource>& resources, const NativeRenderTeardownOperations& operations) {
NativeRenderTeardownBatch batch;
batch.resources = std::move(resources);
const bool complete = TryReleaseNativeRenderTeardown(batch, operations);
resources = std::move(batch.resources);
return complete;
}
MpvPlayer::CallbackContext::Lease::Lease(CallbackContext* context, MpvPlayer* player)
: context_(context), player_(player) {}
@@ -272,21 +284,6 @@ MpvPlayer::MpvPlayer(bool audio_only)
MpvPlayer::~MpvPlayer() { Dispose(); }
bool MpvPlayer::HasRenderContext() const {
std::lock_guard<std::mutex> lock(native_mutex_);
return mpv_gl_ != nullptr;
}
EGLDisplay MpvPlayer::GetEglDisplay() const {
std::lock_guard<std::mutex> lock(native_mutex_);
return egl_display_;
}
EGLContext MpvPlayer::GetEglContext() const {
std::lock_guard<std::mutex> lock(native_mutex_);
return egl_context_;
}
bool MpvPlayer::IsInitialized() const {
std::lock_guard<std::mutex> lock(native_mutex_);
return mpv_ != nullptr && (audio_only_ || mpv_gl_ != nullptr);
@@ -336,10 +333,19 @@ bool MpvPlayer::Initialize() {
mpv_set_option_string(mpv_, "audio-fallback-to-null", "yes");
if (!audio_only_) {
// HDR tone mapping
mpv_set_option_string(mpv_, "tone-mapping", "auto");
mpv_set_option_string(mpv_, "target-colorspace-hint", plezy::mpv_common::TargetColorspaceHint(hdr_enabled_));
// hdr-compute-peak is nested under the same predicate as the tone-map pass -
// it runs exactly when the source's declared peak exceeds target-peak - so it
// costs nothing while the compositor owns tone mapping and gives
// content-adaptive peak detection once we own it.
//
// `tone-mapping` is deliberately *not* set here: it travels with the output
// description and is applied and withdrawn in RunPendingHdrOutput instead.
// See applied_tone_mapping_ in mpv_player.h for why it cannot be global.
mpv_set_option_string(mpv_, "hdr-compute-peak", "auto");
// Declared by vo_gpu_next only, so inert for the render API this player
// runs. Set anyway so the startup value agrees with what a later
// `hdr-enabled` write puts here through SetHDREnabled.
mpv_set_option_string(mpv_, "target-colorspace-hint", plezy::mpv_common::TargetColorspaceHint(hdr_enabled_));
}
mpv_set_option_string(mpv_, "idle", "yes");
mpv_set_option_string(mpv_, "input-default-bindings", "no");
@@ -370,6 +376,17 @@ bool MpvPlayer::Initialize() {
mpv_set_wakeup_callback(mpv_, OnMpvWakeup, callback_context_.get());
mpv_observe_property(mpv_, 0, "current-ao", MPV_FORMAT_STRING);
mpv_observe_property(mpv_, 0, "audio-device-list", MPV_FORMAT_NONE);
if (!audio_only_) {
// One node observation stands in for six blocking sub-property reads. The
// HDR decision runs on the GTK main thread and one of its callers fires on
// every seek, while libmpv's synchronous read hands the request to the core
// and waits for the playloop; every other property access here is async for
// exactly that reason.
//
// An audio-only core has no video-params to report, so it is not asked.
source_hdr_metadata_ = SourceHdrMetadata();
mpv_observe_property(mpv_, kVideoParamsUserdata, "video-params", MPV_FORMAT_NODE);
}
g_message("MPV: Initialization successful (%s)", audio_only_ ? "audio-only" : "render context deferred");
return true;
@@ -377,12 +394,12 @@ bool MpvPlayer::Initialize() {
void MpvPlayer::RetryPendingNativeTeardown() { NativeRenderTeardownQueue::Instance().Retry(); }
bool MpvPlayer::InitRenderContext() {
bool MpvPlayer::InitRenderContextForSurface(EGLDisplay display, EGLConfig config, EGLSurface surface, int depth_bits) {
RetryPendingNativeTeardown();
std::lock_guard<std::mutex> lock(native_mutex_);
if (audio_only_ || disposed_) {
g_warning("MPV: Render context requested for an unavailable player");
g_warning("MPV: Video-plane render context requested for an unavailable player");
return false;
}
if (mpv_gl_) return true;
@@ -390,47 +407,8 @@ bool MpvPlayer::InitRenderContext() {
g_warning("MPV: Cannot create render context - mpv not initialized");
return false;
}
const EGLDisplay flutter_display = eglGetCurrentDisplay();
const EGLContext flutter_context = eglGetCurrentContext();
const EGLSurface flutter_draw = eglGetCurrentSurface(EGL_DRAW);
const EGLSurface flutter_read = eglGetCurrentSurface(EGL_READ);
const EGLenum previous_api = eglQueryAPI();
if (flutter_display == EGL_NO_DISPLAY || flutter_context == EGL_NO_CONTEXT || previous_api == EGL_NONE) {
g_warning("MPV: No EGL context available");
return false;
}
auto restore_flutter = [&]() {
const EGLBoolean api_restored = previous_api == EGL_NONE ? EGL_TRUE : eglBindAPI(previous_api);
const EGLBoolean restored = api_restored == EGL_TRUE
? eglMakeCurrent(flutter_display, flutter_draw, flutter_read, flutter_context)
: EGL_FALSE;
return restored == EGL_TRUE && api_restored == EGL_TRUE;
};
if (!retained_render_contexts_.empty()) {
const bool released =
TryReleaseRetainedNativeRenderContexts(retained_render_contexts_, ProductionTeardownOperations());
const bool flutter_restored = restore_flutter();
if (!released) {
g_warning("MPV: Retained render context still requires a later EGL teardown retry");
}
if (!flutter_restored) {
g_warning("MPV: Failed to restore Flutter EGL state after retained teardown: 0x%x", eglGetError());
}
if (!released || !flutter_restored) return false;
}
EGLint config_id = 0;
if (!eglQueryContext(flutter_display, flutter_context, EGL_CONFIG_ID, &config_id)) {
g_warning("MPV: Failed to query Flutter EGL config: 0x%x", eglGetError());
return false;
}
EGLConfig config = nullptr;
EGLint num_configs = 0;
const EGLint config_attribs[] = {EGL_CONFIG_ID, config_id, EGL_NONE};
if (!eglChooseConfig(flutter_display, config_attribs, &config, 1, &num_configs) || num_configs != 1) {
g_warning("MPV: Failed to select Flutter EGL config: 0x%x", eglGetError());
if (display == EGL_NO_DISPLAY || surface == EGL_NO_SURFACE) {
g_warning("MPV: Video plane provided no usable EGL display or surface");
return false;
}
if (!eglBindAPI(EGL_OPENGL_ES_API)) {
@@ -438,42 +416,104 @@ bool MpvPlayer::InitRenderContext() {
return false;
}
const EGLint context_attribs[] = {EGL_CONTEXT_CLIENT_VERSION, 2, EGL_NONE};
EGLContext candidate_context = eglCreateContext(flutter_display, config, EGL_NO_CONTEXT, context_attribs);
if (candidate_context == EGL_NO_CONTEXT) {
g_warning("MPV: Failed to create isolated EGL context: 0x%x", eglGetError());
if (previous_api != EGL_NONE && !eglBindAPI(previous_api)) {
g_warning("MPV: Failed to restore EGL client API: 0x%x", eglGetError());
// Nothing is shared with Flutter here, so take the highest ES version the
// driver will give. EGL_CONTEXT_CLIENT_VERSION=3 asks for exactly 3.0, which
// reads as "ES 3" while being the oldest of them; 3.2 brings float render
// targets (mpv picks an rgba16f FBO on this path) and ES 3.1 semantics for
// the rest.
//
// It does **not** buy compute shaders. mpv refuses them on any GLES context at
// any version, by construction (`video/out/opengl/ra_gl.c:136-139` in v0.40.0):
//
// // While we can handle compute shaders on GLES the spec (intentionally)
// // does not support binding textures for writing, which all uses inside
// // mpv would require. So disable it unconditionally anyway.
// if (ra->glsl_es) ra->caps &= ~RA_CAP_COMPUTE;
//
// So `hdr-compute-peak` is unreachable through the render API here however
// new the context is - confirmed on hardware with an ES 3.2 context whose
// glDispatchCompute and glBindImageTexture both resolve, where mpv still
// logs "Disabling HDR peak computation (compute shaders=0)". The player-side
// tone map therefore aims at the peak the source declares rather than one
// measured from the frames. Reaching it needs a desktop-GL context on the
// plane, which is the same architectural door as libplacebo and belongs with
// it rather than in a version bump.
//
// EGL_CONTEXT_MINOR_VERSION needs EGL 1.5 or EGL_KHR_create_context. Where
// neither is present eglCreateContext rejects the attribute outright, so the
// legacy CLIENT_VERSION-only request stays as the floor rather than letting
// a missing extension fail context creation altogether.
struct EsVersion {
EGLint major;
EGLint minor;
};
static constexpr EsVersion kPreferredEsVersions[] = {{3, 2}, {3, 1}, {3, 0}, {2, 0}};
EGLContext candidate_context = EGL_NO_CONTEXT;
EGLint chosen_major = 0;
EGLint chosen_minor = 0;
for (const EsVersion& version : kPreferredEsVersions) {
const EGLint context_attribs[] = {
EGL_CONTEXT_MAJOR_VERSION, version.major, EGL_CONTEXT_MINOR_VERSION, version.minor, EGL_NONE};
candidate_context = eglCreateContext(display, config, EGL_NO_CONTEXT, context_attribs);
if (candidate_context != EGL_NO_CONTEXT) {
chosen_major = version.major;
chosen_minor = version.minor;
break;
}
return false;
}
auto destroy_candidate_context = [&]() {
const EGLenum api_before_cleanup = eglQueryAPI();
if (eglGetCurrentContext() == candidate_context) {
if (!eglBindAPI(EGL_OPENGL_ES_API) ||
!eglMakeCurrent(flutter_display, EGL_NO_SURFACE, EGL_NO_SURFACE, EGL_NO_CONTEXT)) {
g_warning("MPV: Failed to release rejected EGL context: 0x%x", eglGetError());
return;
if (candidate_context == EGL_NO_CONTEXT) {
for (const EGLint client_version : {3, 2}) {
const EGLint context_attribs[] = {EGL_CONTEXT_CLIENT_VERSION, client_version, EGL_NONE};
candidate_context = eglCreateContext(display, config, EGL_NO_CONTEXT, context_attribs);
if (candidate_context != EGL_NO_CONTEXT) {
chosen_major = client_version;
chosen_minor = 0;
break;
}
}
if (!eglDestroyContext(flutter_display, candidate_context)) {
g_warning("MPV: Failed to destroy rejected EGL context: 0x%x", eglGetError());
}
if (candidate_context == EGL_NO_CONTEXT) {
g_warning("MPV: Failed to create the video-plane EGL context: 0x%x", eglGetError());
return false;
}
// Report the requested version, not the delivered one - those are different
// claims, and the delivered one is logged below once a context is current.
g_message("MPV video plane: requested OpenGL ES %d.%d", chosen_major, chosen_minor);
auto destroy_candidate_context = [&]() {
if (eglGetCurrentContext() == candidate_context) {
eglMakeCurrent(display, EGL_NO_SURFACE, EGL_NO_SURFACE, EGL_NO_CONTEXT);
}
if (api_before_cleanup != EGL_NONE && !eglBindAPI(api_before_cleanup)) {
g_warning("MPV: Failed to restore EGL API after context cleanup: 0x%x", eglGetError());
if (!eglDestroyContext(display, candidate_context)) {
g_warning("MPV: Failed to destroy rejected video-plane EGL context: 0x%x", eglGetError());
}
};
if (!eglMakeCurrent(flutter_display, EGL_NO_SURFACE, EGL_NO_SURFACE, candidate_context)) {
g_warning("MPV: Failed to activate isolated EGL context: 0x%x", eglGetError());
if (!eglMakeCurrent(display, surface, surface, candidate_context)) {
g_warning("MPV: Failed to activate the video-plane EGL context: 0x%x", eglGetError());
destroy_candidate_context();
if (previous_api != EGL_NONE && !eglBindAPI(previous_api)) {
g_warning("MPV: Failed to restore EGL client API: 0x%x", eglGetError());
}
return false;
}
// What the driver actually gave, and whether mpv will find the entry points
// its compute path needs. Asking for a version is not the same as getting
// it, and mpv's own report of "compute shaders=0" says nothing about which
// half is missing. Both are cheap and both were needed to diagnose this.
const GLubyte* gl_version = glGetString(GL_VERSION);
g_message(
"MPV video plane: GL_VERSION='%s' dispatch_compute=%s image_load_store=%s",
gl_version ? reinterpret_cast<const char*>(gl_version) : "(null)",
eglGetProcAddress("glDispatchCompute") ? "yes" : "no", eglGetProcAddress("glBindImageTexture") ? "yes" : "no");
// Now that a context is current, the surface's swap interval can be set.
// eglSwapBuffers runs on the GTK main thread and must never block: at the
// default interval Mesa throttles it on the compositor's frame callback,
// which an occluded surface never receives. The plane paces itself with its
// own frame callback instead.
if (!eglSwapInterval(display, 0)) {
g_warning("MPV: could not disable EGL swap throttling on the video plane: 0x%x", eglGetError());
}
mpv_opengl_init_params gl_init_params{};
gl_init_params.get_proc_address = get_opengl_proc_address;
gl_init_params.get_proc_address_ctx = nullptr;
@@ -484,53 +524,71 @@ bool MpvPlayer::InitRenderContext() {
{MPV_RENDER_PARAM_INVALID, nullptr},
};
GdkDisplay* gdk_display = gdk_display_get_default();
// The plane only exists on Wayland, and hwdec interop wants the display handle:
// without it VAAPI has to find a device by other means and can quietly end up
// on software decoding, on the path that exists for performance.
#ifdef GDK_WINDOWING_WAYLAND
GdkDisplay* gdk_display = gdk_display_get_default();
if (GDK_IS_WAYLAND_DISPLAY(gdk_display)) {
params[2].type = MPV_RENDER_PARAM_WL_DISPLAY;
params[2].data = gdk_wayland_display_get_wl_display(gdk_display);
}
#endif
#ifdef GDK_WINDOWING_X11
if (GDK_IS_X11_DISPLAY(gdk_display)) {
params[2].type = MPV_RENDER_PARAM_X11_DISPLAY;
params[2].data = gdk_x11_display_get_xdisplay(gdk_display);
}
#endif
mpv_render_context* candidate_gl = nullptr;
const int error = mpv_render_context_create(&candidate_gl, mpv_, params);
const bool restored = restore_flutter();
if (error < 0 || candidate_gl == nullptr || !restored) {
if (error < 0) {
g_warning("MPV: mpv_render_context_create() failed: %s", mpv_error_string(error));
} else if (!restored) {
g_warning("MPV: Failed to restore Flutter EGL state: 0x%x", eglGetError());
} else {
g_warning("MPV: mpv returned a null render context");
}
bool retained_candidate = false;
if (candidate_gl) {
if (eglMakeCurrent(flutter_display, EGL_NO_SURFACE, EGL_NO_SURFACE, candidate_context)) {
mpv_render_context_free(candidate_gl);
} else {
g_warning("MPV: Failed to reactivate rejected EGL context: 0x%x; retaining it for teardown", eglGetError());
retained_render_contexts_.push_back({candidate_gl, flutter_display, candidate_context});
retained_candidate = true;
}
if (!restore_flutter()) {
g_warning("MPV: Failed final Flutter EGL restoration: 0x%x", eglGetError());
}
}
if (!retained_candidate) destroy_candidate_context();
if (error < 0 || candidate_gl == nullptr) {
g_warning("MPV: mpv_render_context_create() failed for the video plane: %s", mpv_error_string(error));
if (candidate_gl) mpv_render_context_free(candidate_gl);
destroy_candidate_context();
return false;
}
egl_display_ = flutter_display;
egl_display_ = display;
egl_context_ = candidate_context;
surface_depth_bits_ = depth_bits > 0 ? depth_bits : 8;
mpv_gl_ = candidate_gl;
mpv_render_context_set_update_callback(mpv_gl_, OnMpvRenderUpdate, callback_context_.get());
g_message("MPV: Render context created with isolated EGL context");
g_message("MPV: Render context created on the Wayland video plane");
return true;
}
bool MpvPlayer::RenderToSurface(EGLSurface surface, int width, int height) {
std::lock_guard<std::mutex> lock(native_mutex_);
if (disposed_ || !mpv_gl_ || egl_context_ == EGL_NO_CONTEXT || surface == EGL_NO_SURFACE) return false;
if (width < 1 || height < 1) return false;
if (!eglBindAPI(EGL_OPENGL_ES_API) || !eglMakeCurrent(egl_display_, surface, surface, egl_context_)) {
g_warning("MPV: Failed to activate the video-plane EGL context for render: 0x%x", eglGetError());
return false;
}
// Consume the redraw latch before rendering: OnMpvRenderUpdate drops further
// notifications until it is cleared.
needs_redraw_.store(false);
mpv_opengl_fbo mpv_fbo{};
mpv_fbo.fbo = 0; // the window surface's default framebuffer
mpv_fbo.w = width;
mpv_fbo.h = height;
// Ignored by the render API's OpenGL backend, which reads the depth param
// instead, but it is what mpv#16818's gpu-next backend will read, so state
// it truthfully rather than leave a lie in place for that day.
mpv_fbo.internal_format = surface_depth_bits_ >= 16 ? GL_RGBA16F : surface_depth_bits_ >= 10 ? GL_RGB10_A2 : GL_RGBA8;
// The default framebuffer is bottom-up relative to mpv's image orientation,
// so this flips.
int flip_y = 1;
// Without this mpv assumes 8 bits and dithers a 10-bit PQ plane down to 8,
// which bands precisely in the dark ramp PQ spends most of its code space on.
int depth = surface_depth_bits_;
mpv_render_param params[] = {
{MPV_RENDER_PARAM_OPENGL_FBO, &mpv_fbo},
{MPV_RENDER_PARAM_FLIP_Y, &flip_y},
{MPV_RENDER_PARAM_DEPTH, &depth},
{MPV_RENDER_PARAM_INVALID, nullptr},
};
mpv_render_context_render(mpv_gl_, params);
return true;
}
@@ -563,6 +621,7 @@ void MpvPlayer::Dispose() {
std::lock_guard<std::mutex> lock(callback_mutex_);
redraw_callback_ = nullptr;
event_callback_ = nullptr;
source_metadata_callback_ = nullptr;
}
auto cancelled = pending_requests_.CancelAll();
@@ -575,13 +634,27 @@ void MpvPlayer::Dispose() {
RemoveTrackedSources();
// Transfer every render/context pair and the shared mpv handle to the
// managed teardown thread. A failed EGL bind leaves the complete pair in
// The plane's context is left current on this thread by RenderToSurface and
// nothing else releases it before the video surface is destroyed - which the
// plugin does *after* this call. An EGLContext can be current to at most one
// thread, so handing it to the teardown worker while it is still bound here
// makes the worker's eglMakeCurrent fail with EGL_BAD_ACCESS; the pair is then
// retained, and by the note below the mpv handle cannot be terminated until
// every pair drains. Repeated open/close would carry a whole stale mpv core
// across each gap. Only our own context is released: Flutter's must be left
// exactly where it is.
if (egl_context_ != EGL_NO_CONTEXT && eglGetCurrentContext() == egl_context_) {
if (!eglMakeCurrent(egl_display_, EGL_NO_SURFACE, EGL_NO_SURFACE, EGL_NO_CONTEXT)) {
g_warning("MPV: Failed to release the video-plane EGL context before teardown: 0x%x", eglGetError());
}
}
// Transfer the render context, the EGL context and the shared mpv handle to
// the managed teardown thread. A failed EGL bind leaves the complete pair in
// the queue; the handle cannot be terminated until every pair is gone.
NativeRenderTeardownBatch teardown;
{
std::lock_guard<std::mutex> lock(native_mutex_);
teardown.resources = std::move(retained_render_contexts_);
if (mpv_gl_ || egl_context_ != EGL_NO_CONTEXT) {
teardown.resources.push_back({mpv_gl_, egl_display_, egl_context_});
}
@@ -591,31 +664,14 @@ void MpvPlayer::Dispose() {
mpv_ = nullptr;
egl_display_ = EGL_NO_DISPLAY;
egl_context_ = EGL_NO_CONTEXT;
// The next player must not decide against this one's colour space.
source_hdr_metadata_ = SourceHdrMetadata();
}
NativeRenderTeardownQueue::Instance().Enqueue(std::move(teardown));
observed_properties_.Clear();
}
void MpvPlayer::Render(int width, int height, int fbo) {
std::lock_guard<std::mutex> lock(native_mutex_);
if (disposed_ || !mpv_gl_) return;
mpv_opengl_fbo mpv_fbo{};
mpv_fbo.fbo = fbo;
mpv_fbo.w = width;
mpv_fbo.h = height;
mpv_fbo.internal_format = 0;
int flip_y = 0;
mpv_render_param params[] = {
{MPV_RENDER_PARAM_OPENGL_FBO, &mpv_fbo},
{MPV_RENDER_PARAM_FLIP_Y, &flip_y},
{MPV_RENDER_PARAM_INVALID, nullptr},
};
mpv_render_context_render(mpv_gl_, params);
}
void MpvPlayer::Command(const std::vector<std::string>& args) { CommandAsync(args, nullptr); }
void MpvPlayer::CommandAsync(const std::vector<std::string>& args, CommandCallback callback) {
@@ -632,6 +688,14 @@ void MpvPlayer::SetProperty(const std::string& name, const std::string& value) {
}
void MpvPlayer::SetPropertyAsync(const std::string& name, const std::string& value, StatusCallback callback) {
#ifdef PLEZY_MPV_PLAYER_LIFECYCLE_TEST
// Ahead of the handle check: a substituted writer stands in for the core, so
// the absence of a real one is not a reason to refuse the write.
if (test_property_write_) {
test_property_write_(name, value, std::move(callback));
return;
}
#endif
if (disposed_ || !mpv_) {
if (callback) callback(MPV_ERROR_UNINITIALIZED);
return;
@@ -644,6 +708,35 @@ void MpvPlayer::SetPropertyAsync(const std::string& name, const std::string& val
plezy::mpv_common::SubmitSetPropertyAsync(mpv_, pending_requests_, name, value, std::move(callback));
}
bool MpvPlayer::ReadSourceHdrMetadata(SourceHdrMetadata* out) {
if (out == nullptr) return false;
std::lock_guard<std::mutex> lock(native_mutex_);
if (disposed_ || !mpv_) return false;
*out = source_hdr_metadata_;
return true;
}
void MpvPlayer::UpdateSourceHdrMetadata(const mpv_node* params) {
{
std::lock_guard<std::mutex> lock(native_mutex_);
source_hdr_metadata_ = ParseSourceHdrMetadata(params);
}
// Outside the lock on purpose: the callback's whole job is to re-run the HDR
// decision, which reads the cache straight back through ReadSourceHdrMetadata.
SourceMetadataCallback callback;
{
std::lock_guard<std::mutex> lock(callback_mutex_);
callback = source_metadata_callback_;
}
if (callback) callback();
}
void MpvPlayer::SetSourceMetadataCallback(SourceMetadataCallback callback) {
std::lock_guard<std::mutex> lock(callback_mutex_);
source_metadata_callback_ = std::move(callback);
}
void MpvPlayer::GetPropertyAsync(const std::string& name, GetPropertyCallback callback) {
if (disposed_ || !mpv_) {
if (callback) callback(MPV_ERROR_UNINITIALIZED, "");
@@ -661,14 +754,6 @@ void MpvPlayer::ObserveProperty(const std::string& name, const std::string& form
mpv_observe_property(mpv_, request.userdata, name.c_str(), request.format);
}
void MpvPlayer::ReportMouseMove(int x, int y) {
if (disposed_ || !mpv_) return;
std::string x_str = std::to_string(x);
std::string y_str = std::to_string(y);
const char* args[] = {"mouse", x_str.c_str(), y_str.c_str(), nullptr};
mpv_command_async(mpv_, 0, args);
}
void MpvPlayer::SetEventCallback(EventCallback callback) {
std::lock_guard<std::mutex> lock(callback_mutex_);
event_callback_ = std::move(callback);
@@ -708,8 +793,8 @@ void MpvPlayer::OnMpvRenderUpdate(void* ctx) {
return;
}
// Flutter texture notification must run on the player's owning GLib
// context, never on mpv's render/VO thread.
// The redraw must run on the player's owning GLib context, never on mpv's
// render/VO thread.
player->ScheduleRedrawSource();
}
@@ -914,6 +999,15 @@ void MpvPlayer::HandleMpvEvent(mpv_event* event) {
if (!prop || !prop->name) break;
mpv_node node = plezy::mpv_common::ExtractPropertyNode(prop);
// This runner's own observation, which nothing on the Dart side asked
// for and nothing there is waiting on. mpv delivers one event per
// observation, so a Dart-side observer of the same property still gets
// its own under its own userdata.
if (event->reply_userdata == kVideoParamsUserdata) {
UpdateSourceHdrMetadata(&node);
break;
}
const auto notice = plezy::mpv_common::ObserveAudioRecoveryProperty(audio_recovery_, event, prop);
if (notice.message) LogRecovery(notice.message);
// Recovery runs off a GLib timer here, so newly queued work has to arm it.
@@ -924,6 +1018,15 @@ void MpvPlayer::HandleMpvEvent(mpv_event* event) {
}
case MPV_EVENT_END_FILE: {
audio_recovery_.SetFileLoaded(false);
// Whatever comes next is a different source until video-params says
// otherwise, and describing it against this one's colour space is the
// one failure worth a transient wrong answer to avoid. No re-apply is
// requested: the plane keeps the description it has until the next
// playback-restart or video-params change, exactly as before.
{
std::lock_guard<std::mutex> lock(native_mutex_);
source_hdr_metadata_ = SourceHdrMetadata();
}
auto* end = static_cast<mpv_event_end_file*>(event->data);
if (!end) break;
FlValue* data = fl_value_new_map();
@@ -1032,6 +1135,329 @@ void MpvPlayer::SendEvent(const std::string& name, FlValue* data) {
fl_value_unref(event_map);
}
void MpvPlayer::ApplyPropertySequence(
std::shared_ptr<std::vector<PropertyChange>> changes, size_t index, StatusCallback callback) {
if (changes == nullptr || index >= changes->size()) {
if (callback) callback(MPV_ERROR_SUCCESS);
return;
}
const PropertyChange& change = (*changes)[index];
SetPropertyAsync(change.name, change.value, [this, changes, index, cb = std::move(callback)](int error) mutable {
if (plezy::mpv_common::SetPropertyStatusSucceeded(error)) {
ApplyPropertySequence(changes, index + 1, std::move(cb));
return;
}
// `index` entries already landed and must come back, newest
// first, so a refused change leaves the previous state intact
// rather than a half-applied mixture of the two.
RollbackPropertySequence(changes, index, error, std::move(cb));
});
}
void MpvPlayer::RollbackPropertySequence(
std::shared_ptr<std::vector<PropertyChange>> changes, size_t undo_count, int failure, StatusCallback callback) {
if (changes == nullptr || undo_count == 0) {
// Only now is mpv genuinely back where it started, so only now may the caller
// hear about it. The original failure is what it needs, not the outcome of the
// unwinding.
if (callback) callback(failure);
return;
}
const size_t index = undo_count - 1;
const PropertyChange& change = (*changes)[index];
SetPropertyAsync(
change.name, change.rollback, [this, changes, index, failure, cb = std::move(callback)](int error) mutable {
if (plezy::mpv_common::SetPropertyStatusSucceeded(error)) {
RollbackPropertySequence(changes, index, failure, std::move(cb));
return;
}
// Carrying on would leave mpv in a state that is neither the
// old one nor the new, and *no* surface description is correct
// for a signal nobody can name. Escalate to the one state that
// is always describable and always accepts its value: SDR.
g_warning(
"MPV: could not restore %s while unwinding a refused output colour space; "
"forcing SDR",
(*changes)[index].name.c_str());
ForceSdrOutput(0, failure, std::move(cb));
});
}
// The one place the applied-output cache is written, so every path that moves a
// property records it the same way. Matched by name, not position: the order the
// sequences use is load-bearing and has to stay free to change without silently
// reassigning the wrong field.
void MpvPlayer::RecordAppliedOutputProperty(const std::string& name, const std::string& value) {
if (name == "target-peak") {
applied_target_peak_ = value;
} else if (name == "target-prim") {
applied_target_prim_ = value;
} else if (name == "target-trc") {
applied_target_trc_ = value;
} else if (name == "tone-mapping") {
applied_tone_mapping_ = value;
}
}
void MpvPlayer::ForceSdrOutput(size_t index, int failure, StatusCallback callback) {
// Same order the apply path uses: the transfer function stops asking for HDR
// before the primaries, operator and peak follow it back.
static const char* const kResetOrder[] = {"target-trc", "target-prim", "tone-mapping", "target-peak"};
constexpr size_t kResetCount = sizeof(kResetOrder) / sizeof(kResetOrder[0]);
if (index >= kResetCount) {
// mpv is SDR now, not back where it started, so any HDR description the
// caller has already committed is a lie about these pixels.
hdr_unwind_result_ = HdrOutputResult::kForcedSdr;
output_state_known_ = true;
if (callback) callback(failure);
return;
}
SetPropertyAsync(kResetOrder[index], "auto", [this, index, failure, cb = std::move(callback)](int error) mutable {
if (plezy::mpv_common::SetPropertyStatusSucceeded(error)) {
// Recorded as it lands, not once the whole reset is through. Recording
// only at the end would leave the cache naming the pre-reset curve for
// every property that did move if a later one is refused, and the no-op
// short-circuit would then answer a repeat request from it - committing an
// HDR description over pixels mpv had already reset to SDR.
RecordAppliedOutputProperty(kResetOrder[index], "auto");
ForceSdrOutput(index + 1, failure, std::move(cb));
return;
}
// `auto` is valid for every one of them, so this failing means mpv is
// no longer taking orders at all - usually because it is being
// disposed. Either way what it emits is now unknowable, and the
// caller must stop presenting the plane rather than guess.
hdr_unwind_result_ = HdrOutputResult::kUnknown;
// And the cache is now a record of what we *asked* for, not what mpv holds:
// some of the reset landed and some did not. Marking it untrusted is what
// stops the short-circuit skipping a later write on the strength of it. The
// strings are left alone deliberately - they are still the best rollback
// targets available if a later sequence gets that far.
output_state_known_ = false;
g_warning(
"MPV: output colour space is no longer commandable; what the plane emits "
"is unknown");
if (cb) cb(failure);
});
}
bool MpvPlayer::CanCommandOutputProperties() const {
#ifdef PLEZY_MPV_PLAYER_LIFECYCLE_TEST
// The substituted writer is the core here; see SetPropertyAsync, which routes
// to it ahead of the same handle check.
if (test_property_write_) return true;
#endif
return !disposed_ && mpv_ != nullptr;
}
#ifdef PLEZY_MPV_PLAYER_LIFECYCLE_TEST
void MpvPlayer::ConfigurePropertyWritesForTesting(PropertyWriteForTesting writer) {
test_property_write_ = std::move(writer);
}
MpvPlayer::AppliedOutputColourSpace MpvPlayer::AppliedOutputColourSpaceForTesting() const {
return {applied_target_trc_, applied_target_prim_, applied_tone_mapping_, applied_target_peak_};
}
#endif
void MpvPlayer::SetHdrOutput(SourceTransfer transfer, uint32_t target_peak_nits, HdrOutputCallback callback) {
if (!CanCommandOutputProperties()) {
// The third place a result is named, and it owes the same honesty as the
// other two: nothing was touched, so the previous state stands - which is
// only worth saying when that state is nameable. Otherwise a request that
// was already queued when the core went away is answered kUnknown by the
// drain while an identical one arriving a moment later hears kRestored.
if (callback) {
callback(output_state_known_ ? HdrOutputResult::kRestored : HdrOutputResult::kUnknown, MPV_ERROR_UNINITIALIZED);
}
return;
}
// Requests are serialized, and queued rather than coalesced.
//
// Playback restarts, preferred-description changes and the two settings can
// each ask for a new output colour space, and every step of a sequence
// completes asynchronously. Two overlapping sequences would interleave: a
// failure in the older one would issue rollbacks that overwrite properties the
// newer one had already set, while the newer one still reported success and
// recorded values mpv no longer holds. That is the divergence the sequencing
// exists to prevent.
//
// Each request keeps its own callback instead of being folded into the newest
// one, because callers commit their own state on success: telling a caller its
// change landed when a *different* request is what actually landed reintroduces
// the same divergence one level up. Strict ordering then makes the bookkeeping
// trivial — the last request to succeed is exactly what mpv holds, so nobody
// needs an epoch to work out whether their commit is still current.
hdr_queue_.push_back(HdrOutputRequest{transfer, target_peak_nits, std::move(callback)});
if (hdr_sequence_in_flight_) return;
RunPendingHdrOutput();
}
void MpvPlayer::RunPendingHdrOutput() {
if (!CanCommandOutputProperties()) {
hdr_sequence_in_flight_ = false;
auto orphaned = std::move(hdr_queue_);
hdr_queue_.clear();
for (auto& request : orphaned) {
// Nothing was touched, so the previous state - whatever it was - still
// stands as far as this request is concerned. Which is only worth telling
// the caller when that state is nameable; if the last unwind gave up
// halfway, "unchanged" describes a colour space nobody knows.
if (request.callback) {
request.callback(
output_state_known_ ? HdrOutputResult::kRestored : HdrOutputResult::kUnknown, MPV_ERROR_UNINITIALIZED);
}
}
return;
}
if (hdr_queue_.empty()) {
hdr_sequence_in_flight_ = false;
return;
}
HdrOutputRequest request = std::move(hdr_queue_.front());
hdr_queue_.pop_front();
hdr_sequence_in_flight_ = true;
// Assume a clean unwind; the escalation path in RollbackPropertySequence and
// ForceSdrOutput moves this on if it cannot manage one.
hdr_unwind_result_ = HdrOutputResult::kRestored;
// Four properties describe one output colour space, so they are applied as a
// unit. A plane whose primaries moved to BT.2020 while its transfer function
// stayed on gamma is neither SDR nor HDR, and the caller describes the surface
// to the compositor on success — a silently half-applied set would have the
// compositor told one thing and shown another.
//
// target-peak is what mpv maps to. Under PQ, left on auto it resolves to the
// format's nominal 10000 nits, so the renderer never tone-maps and the
// compositor owns the decision. Set to the display's real peak, mpv tone-maps
// to it and the caller declares that same peak, leaving the compositor nothing
// to do.
//
// On the SDR fallback the peak and curve are named for accuracy, not to make
// tone mapping happen: mpv 0.40 already resolves target-peak=auto to 203 nits
// and target-trc=auto to gamma 2.2 for an SDR curve, and measurement confirmed
// naming them changed the shadows but not the highlights. What they buy is the
// surface's real terms instead of assumed ones - the compositor's own reference
// white, and sRGB, which is what an undescribed Wayland surface is and what
// this compositor's preferred description for the output says. Hence no
// `enabled` in the peak condition below: an SDR peak is a real instruction, not
// a leftover from an HDR request.
const bool enabled = request.transfer != SourceTransfer::kSdr;
const char* primaries = enabled ? "bt.2020" : "auto";
// The option is an integer in [10, 10000]; anything outside means "auto".
const bool tone_map_here = request.peak_nits >= 10 && request.peak_nits <= kPqMaxLuminanceNits;
const std::string peak = tone_map_here ? std::to_string(request.peak_nits) : std::string("auto");
const char* curve = request.transfer == SourceTransfer::kHlg
? "hlg"
: (request.transfer == SourceTransfer::kPq ? "pq" : (tone_map_here ? "srgb" : "auto"));
// The operator only matters while a tone-map pass runs, and it must go back to
// auto when one does not; see applied_tone_mapping_ in mpv_player.h.
//
// Restricted to the undescribed SDR target, which is where it was measured.
// Player-side mapping onto an HDR output aims at a PQ target instead, and
// nothing has been measured there yet - that needs the external display - so
// it keeps mpv's own choice until it can be judged the same way.
//
// mobius's shape is governed by tone-mapping-param, its transition point: below
// it the curve is 1:1, above it rolls off. Left at mpv's default 0.3 because
// that measured best, not by omission. Raising it trades highlight shoulder for
// in-range luminance, and against libplacebo's rendering of the same chart
// (400/700/1000 -> 238.5/253.8/254.8, 100 nits -> 134.0) the default is closest
// on both counts, with higher values moving away on each:
//
// param 100 nits 400->1000 span
// 0.30 179.0 17.1
// 0.45 184.4 12.1
// 0.60 185.0 7.2
//
// It also does not touch the cost this operator carries. On real 1000-nit
// footage mobius sits 0.027 dxy and ~12% darker than BT.2390 whatever the
// transition point is (0.30/0.38/0.45 measured identical), because that
// difference is gamut handling rather than the tone curve, and a dark scene's
// pixels fall below the transition point in every case.
//
// Not pinned explicitly: the option has no accepted "unset" token - `default`
// is rejected - so writing it would leave a mobius-specific value applied to
// whatever operator runs next, including BT.2390 on the unmeasured HDR-output
// path. Recorded here instead so an upstream default change is diagnosable.
const char* operator_name = (tone_map_here && !enabled) ? "mobius" : "auto";
// playback-restart drives a re-apply and fires on every seek, so most calls
// here ask for the state mpv already holds. The plane's own half already
// short-circuits an identical request; this is the other half. Without it a
// seek costs four property round-trips and a log line saying nothing changed,
// and holds the sequence long enough to defer a real request behind it.
//
// kApplied, because that is the truth the caller acts on: these four values
// *are* in force, so a surface description committed against them stays
// honest. The queue has to keep draining from here exactly as it does on the
// applied path, or a coalesced request behind this one never runs. Unlike that
// path the call is a real recursion rather than a fresh stack, which is fine
// because the plugin coalesces reapplies into a single pending flag: the queue
// holds the one in flight plus at most one waiting.
// output_state_known_ first: the comparison is only meaningful while the cache
// is a record of what mpv holds. A forced-SDR reset that was itself refused
// partway leaves it a record of what was *asked* for, and skipping on that
// would report kApplied for a colour space mpv is not in - which the caller
// then commits an image description against.
if (output_state_known_ && applied_target_trc_ == curve && applied_target_prim_ == primaries &&
applied_tone_mapping_ == operator_name && applied_target_peak_ == peak) {
if (request.callback) request.callback(HdrOutputResult::kApplied, 0);
RunPendingHdrOutput();
return;
}
// The values that decide who tone-maps and against what, none of which is
// visible on screen: two very different curves both look like working video.
// Logged next to the plane's own decisions so a capture can be matched to the
// state that produced it.
g_message(
"MPV: output colour target peak=%s prim=%s trc=%s tone-mapping=%s", peak.c_str(), primaries, curve,
operator_name);
// Dependencies first, peak last, matching the order kResetOrder uses. The peak
// is what decides whether a tone-map pass runs at all, so everything that pass
// depends on is in place before it is named.
auto changes = std::make_shared<std::vector<PropertyChange>>();
changes->push_back({"target-trc", curve, applied_target_trc_});
changes->push_back({"target-prim", primaries, applied_target_prim_});
changes->push_back({"tone-mapping", operator_name, applied_tone_mapping_});
changes->push_back({"target-peak", peak, applied_target_peak_});
ApplyPropertySequence(changes, 0, [this, changes, callback = std::move(request.callback)](int error) {
const bool ok = plezy::mpv_common::SetPropertyStatusSucceeded(error);
// Only a fully applied set becomes the new rollback target; a failed one was
// already unwound, and the unwinding updated these itself if it had to force
// SDR.
if (ok && !disposed_) {
for (const PropertyChange& change : *changes) {
RecordAppliedOutputProperty(change.name, change.value);
}
// A clean apply is the one outcome that leaves mpv exactly where the cache
// says, so it is what re-earns the short-circuit's trust after an unwind
// gave up halfway.
output_state_known_ = true;
}
// This request's own outcome, to this request's own caller. The result names
// what mpv is actually in now, which is what decides whether the caller's
// committed surface description is still true.
//
// kRestored says "mpv is where it was", which only reassures the caller while
// where it was is known. After an unwind that gave up halfway it is not: a
// sequence refused on its very first write unwinds nothing, so it reports the
// untouched kRestored while mpv sits in the half-reset state nobody can name.
// The caller would read that as "your description still holds" and put an
// undescribed plane back on screen. Downgrading to kUnknown is the honest
// answer, and a clean apply - the one thing that re-earns the trust - is
// reported as kApplied above regardless.
const HdrOutputResult result =
ok ? HdrOutputResult::kApplied : (output_state_known_ ? hdr_unwind_result_ : HdrOutputResult::kUnknown);
if (callback) callback(result, error);
// Whatever arrived while this ran runs now — never alongside.
RunPendingHdrOutput();
});
}
void MpvPlayer::SetHDREnabled(bool enabled, StatusCallback callback) {
SetPropertyAsync(
"target-colorspace-hint", plezy::mpv_common::TargetColorspaceHint(enabled),