feat(linux): HDR video on a native Wayland plane
Video on Linux went through a Flutter texture: 8-bit sRGB, which cannot carry HDR at all, and which forced a whole-window Flutter recomposite for every video frame. This moves it onto a wl_subsurface stacked below the Flutter surface, with mpv rendering into an EGL window surface on it through the libmpv render API. The subsurface is desynchronized, so video and UI now present independently. With the plane in place HDR follows: the surface is described to the compositor through wp_color_manager_v1 as the source's own curve and gamut - PQ or HLG, BT.2020 - carrying whatever HDR10 static metadata the stream actually declares. The description and the buffer it describes land on the same commit, staged and validated before mpv is switched, so a PQ frame is never presented labelled sRGB. A five-second watchdog bounds the one wait a compositor could otherwise leave hanging. A session that cannot host the plane - X11, or a compositor without wl_subcompositor - fails initialize with VIDEO_PLANE_UNSUPPORTED naming the reason: the texture path is gone, and refusing by name beats degrading to something the user cannot see. An SDR output, a missing capability or an 8-bit config keep the plane and simply leave it undescribed. The output's colour state is trusted only when it has been earned. Every landed property step records itself as it lands; a reset or sequence that cannot finish downgrades its result to unknown and marks the applied-output cache untrusted until a clean apply earns it back. A plane whose output state cannot be named is quarantined - hidden, its description withdrawn - and the quarantine is recorded state: an unrelated visibility change cannot put a mislabelled plane back on screen, and only a commit that resolves to a nameable outcome lifts it. A rect collapsing to zero detaches the buffer exactly as hiding does, a refused setVideoRect drops the Dart-side sent-rect cache so the next layout pass retries for free, and a refused tone-mapping pick tells the user instead of dying in a log. NVIDIA's Wayland EGL (through at least 610.xx) offers no 10-bit unorm window configs, so the plane takes half-float as the tier between 10-bit unorm and 8-bit, declares the whole surface opaque so the compositor never reads the alpha those configs carry, and states GL_RGBA16F rather than a 10-bit lie. Whether the output is in HDR is read from luminance headroom above its own reference white rather than from the preferred transfer function, which current KWin no longer answers PQ for; the margin is half a stop, because KWin reports an undimmed maximum over a software-dimmed SDR white. Validated on an RTX 4090 (driver 610.57.04) under KWin 6.7.4 with locked-exposure photographs. Who tone-maps is a user choice. The default is the compositor: photographed on a 400-nit HDR output against a PQ chart it keeps 400 -> 1000 nits monotonic and separated where the player leg flattens them, because the player path drives mpv's legacy vo_gpu, whose own standalone output scores the same. The gap is the renderer, not the wiring. The decision itself - what the source carries, what the output supports, what to tell mpv and what to tell the compositor - lives in hdr_metadata.h, free of Wayland and GTK so its luminance validation can be tested without a display server. Sending an incoherent luminance set is a protocol error that disconnects the client, so the rules are worth a unit test. The deb, rpm and pacman packages now declare wayland-client, wayland-egl and EGL: the plane links them directly and bundle-libs.sh deliberately never bundles them, since they are coupled to the running compositor and GPU driver. lib/dev/harness_main.dart is a second entrypoint for measuring this on hardware - it drives one clip with scripted mpv properties and reports the colour state mpv actually settled on. Nothing imports it, so it is tree-shaken out of the app. Verified on a Steam Deck against an external 400-nit HDR display: the compositor reports PQ / BT.2020, the connector carries HDR_OUTPUT_METADATA, and against mpv vo=gpu-next on the same frame the shipped build sits 4.90 counts away overall - closer to the reference HDR player than to its own SDR fallback.
This commit is contained in:
edde746
@@ -0,0 +1,308 @@
|
||||
// Measurement harness entrypoint. NOT part of the app.
|
||||
//
|
||||
// Drives the real PlayerLinux/mpv/Video rendering path with a local file so
|
||||
// the compositing cost can be measured without a Plex server. Build with:
|
||||
// flutter build linux --target=lib/dev/harness_main.dart
|
||||
// Run with:
|
||||
// PLEZY_HARNESS_MEDIA=/path/to/file.mp4 PLEZY_HARNESS_SECONDS=40 ./plezy
|
||||
//
|
||||
// Optional knobs:
|
||||
// PLEZY_HARNESS_MPV_LOG=v|debug mpv's own log stream
|
||||
// PLEZY_HARNESS_MPV_PROPS=name=value,... arbitrary mpv properties, so an
|
||||
// option can be swept without a
|
||||
// rebuild for each value
|
||||
// PLEZY_HARNESS_HDR=1 request HDR passthrough
|
||||
// PLEZY_HARNESS_TONEMAP=compositor|player which side tone-maps; the A/B leg
|
||||
// PLEZY_HARNESS_INSET=<px> toggles a padding every 6s, so the
|
||||
// plane has to move, not just resize
|
||||
import 'dart:async';
|
||||
import 'dart:io';
|
||||
|
||||
import 'package:flutter/material.dart';
|
||||
import 'package:flutter/scheduler.dart';
|
||||
|
||||
import '../mpv/models.dart';
|
||||
import '../mpv/player/player.dart';
|
||||
import '../mpv/video.dart';
|
||||
|
||||
void main() {
|
||||
WidgetsFlutterBinding.ensureInitialized();
|
||||
runApp(const _HarnessApp());
|
||||
}
|
||||
|
||||
class _HarnessApp extends StatefulWidget {
|
||||
const _HarnessApp();
|
||||
|
||||
@override
|
||||
State<_HarnessApp> createState() => _HarnessAppState();
|
||||
}
|
||||
|
||||
class _HarnessAppState extends State<_HarnessApp> {
|
||||
Player? _player;
|
||||
String _status = 'starting';
|
||||
|
||||
int _frames = 0;
|
||||
int _buildUs = 0;
|
||||
int _rasterUs = 0;
|
||||
final Stopwatch _clock = Stopwatch()..start();
|
||||
Timer? _reportTimer;
|
||||
Timer? _probeTimer;
|
||||
Timer? _quitTimer;
|
||||
Timer? _insetTimer;
|
||||
double _inset = 0;
|
||||
|
||||
@override
|
||||
void initState() {
|
||||
super.initState();
|
||||
SchedulerBinding.instance.addTimingsCallback(_onFrames);
|
||||
_reportTimer = Timer.periodic(const Duration(seconds: 2), (_) => _report());
|
||||
_probeTimer = Timer.periodic(const Duration(seconds: 2), (_) => _probe());
|
||||
|
||||
final seconds = int.tryParse(Platform.environment['PLEZY_HARNESS_SECONDS'] ?? '');
|
||||
if (seconds != null && seconds > 0) {
|
||||
_quitTimer = Timer(Duration(seconds: seconds), () {
|
||||
_report();
|
||||
stdout.writeln('HARNESS_DONE');
|
||||
exit(0);
|
||||
});
|
||||
}
|
||||
final inset = double.tryParse(Platform.environment['PLEZY_HARNESS_INSET'] ?? '');
|
||||
if (inset != null && inset > 0) {
|
||||
_insetTimer = Timer.periodic(const Duration(seconds: 6), (_) {
|
||||
setState(() => _inset = _inset == 0 ? inset : 0);
|
||||
stdout.writeln('HARNESS_INSET $_inset');
|
||||
});
|
||||
}
|
||||
_start();
|
||||
}
|
||||
|
||||
void _onFrames(List<FrameTiming> timings) {
|
||||
for (final t in timings) {
|
||||
_frames++;
|
||||
_buildUs += t.buildDuration.inMicroseconds;
|
||||
_rasterUs += t.rasterDuration.inMicroseconds;
|
||||
}
|
||||
}
|
||||
|
||||
void _report() {
|
||||
final n = _frames;
|
||||
final secs = _clock.elapsedMilliseconds / 1000.0;
|
||||
final fps = secs > 0 ? n / secs : 0.0;
|
||||
final build = n > 0 ? (_buildUs / n / 1000.0) : 0.0;
|
||||
final raster = n > 0 ? (_rasterUs / n / 1000.0) : 0.0;
|
||||
stdout.writeln(
|
||||
'FRAMESTAT t=${secs.toStringAsFixed(1)} frames=$n '
|
||||
'fps=${fps.toStringAsFixed(2)} build_ms=${build.toStringAsFixed(2)} '
|
||||
'raster_ms=${raster.toStringAsFixed(2)}',
|
||||
);
|
||||
_frames = 0;
|
||||
_buildUs = 0;
|
||||
_rasterUs = 0;
|
||||
_clock.reset();
|
||||
}
|
||||
|
||||
// Independent of Flutter's frame loop: tells us whether mpv is actually
|
||||
// advancing even when nothing is being composited.
|
||||
Future<void> _probe() async {
|
||||
final player = _player;
|
||||
if (player == null) return;
|
||||
try {
|
||||
final pos = await player.getProperty('time-pos');
|
||||
final paused = await player.getProperty('pause');
|
||||
final dropped = await player.getProperty('frame-drop-count');
|
||||
final decoded = await player.getProperty('decoder-frame-drop-count');
|
||||
stdout.writeln('PROBE time-pos=$pos pause=$paused drops=$dropped dec_drops=$decoded');
|
||||
} catch (e) {
|
||||
stdout.writeln('PROBE_ERROR $e');
|
||||
}
|
||||
}
|
||||
|
||||
// Which side tone-maps, and against what peak, is decided by mpv options that
|
||||
// leave no trace on screen: two very different curves look like "the video
|
||||
// plane works". Reading the effective values back is the only way to tell a
|
||||
// deliberate target from a default nobody chose - and the render API cannot
|
||||
// discover the display for itself the way a windowed mpv does, so the answer
|
||||
// here is not the answer `mpv` alone would give.
|
||||
//
|
||||
// Read after open() because the source-dependent ones are unset until a
|
||||
// format is known.
|
||||
Future<void> _reportColourState(Player player, String when) async {
|
||||
const names = <String>[
|
||||
'target-peak',
|
||||
'target-trc',
|
||||
'target-prim',
|
||||
'tone-mapping',
|
||||
'hdr-compute-peak',
|
||||
'video-params/gamma',
|
||||
'video-params/primaries',
|
||||
'video-params/sig-peak',
|
||||
'video-params/max-luma',
|
||||
];
|
||||
final parts = <String>[];
|
||||
for (final name in names) {
|
||||
try {
|
||||
parts.add('$name=${await player.getProperty(name) ?? "-"}');
|
||||
} catch (_) {
|
||||
// The message is dropped rather than interpolated: this line is parsed
|
||||
// as space-separated name=value pairs, and an exception string carries
|
||||
// spaces of its own.
|
||||
parts.add('$name=<error>');
|
||||
}
|
||||
}
|
||||
stdout.writeln('HARNESS_COLOUR[$when] ${parts.join(' ')}');
|
||||
}
|
||||
|
||||
// build() only shows _status while there is no player, and by this point one
|
||||
// has usually been assigned. Without clearing it a deliberate abort renders as
|
||||
// an empty transparent window, i.e. indistinguishable from a hang.
|
||||
Future<void> _abort(Player? player, String status) async {
|
||||
await player?.dispose();
|
||||
if (!mounted) return;
|
||||
setState(() {
|
||||
_player = null;
|
||||
_status = status;
|
||||
});
|
||||
}
|
||||
|
||||
Future<void> _start() async {
|
||||
final media = Platform.environment['PLEZY_HARNESS_MEDIA'];
|
||||
if (media == null || media.isEmpty) {
|
||||
setState(() => _status = 'set PLEZY_HARNESS_MEDIA');
|
||||
return;
|
||||
}
|
||||
final uri = media.startsWith('/') ? 'file://$media' : media;
|
||||
try {
|
||||
final player = Player();
|
||||
setState(() => _player = player);
|
||||
final level = Platform.environment['PLEZY_HARNESS_MPV_LOG'];
|
||||
if (level != null && level.isNotEmpty) {
|
||||
await player.setLogLevel(level);
|
||||
stdout.writeln('HARNESS_MPV_LOG $level');
|
||||
}
|
||||
// Example: PLEZY_HARNESS_MPV_PROPS='tone-mapping=mobius,target-peak=200'.
|
||||
// Names the plugin intercepts (hdr-enabled, hdr-tone-mapping) also have
|
||||
// dedicated knobs, and those are applied *after* this block, so the
|
||||
// dedicated one wins if both name the same thing. Both writes are logged
|
||||
// and HARNESS_COLOUR[settled] reads the effective state back, so a capture
|
||||
// cannot be silently mislabelled either way.
|
||||
//
|
||||
// Applied before open(), so the first configured frame already has them.
|
||||
//
|
||||
// That timing is also the catch, and it has already produced a wrong
|
||||
// answer: the native side re-applies the whole output description on
|
||||
// playback-restart and on every seek, so anything it manages -
|
||||
// target-peak, target-prim, target-trc, tone-mapping - is overwritten
|
||||
// moments later. A sweep of those reads back as the shipped value while
|
||||
// looking perfectly plausible. Use this for properties the runner does not
|
||||
// set itself; for the ones it does, change the code.
|
||||
//
|
||||
// A malformed or rejected override aborts the leg, for the same reason
|
||||
// PLEZY_HARNESS_TONEMAP does: these captures get labelled with the value
|
||||
// that was asked for, and carrying on would file the default curve under
|
||||
// whatever was requested. A silently wrong label is worse than no capture.
|
||||
final props = Platform.environment['PLEZY_HARNESS_MPV_PROPS'];
|
||||
if (props != null && props.isNotEmpty) {
|
||||
for (final pair in props.split(',')) {
|
||||
final split = pair.indexOf('=');
|
||||
final name = split > 0 ? pair.substring(0, split).trim() : '';
|
||||
final value = split > 0 ? pair.substring(split + 1).trim() : '';
|
||||
String? failure;
|
||||
if (name.isEmpty || value.isEmpty) {
|
||||
failure = 'not name=value';
|
||||
} else {
|
||||
try {
|
||||
await player.setProperty(name, value);
|
||||
stdout.writeln('HARNESS_MPV_PROP $name=$value');
|
||||
} catch (e) {
|
||||
failure = '$e';
|
||||
}
|
||||
}
|
||||
if (failure != null) {
|
||||
stdout.writeln('HARNESS_MPV_PROP_ERROR $pair: $failure');
|
||||
await _abort(player, 'bad PLEZY_HARNESS_MPV_PROPS: $pair');
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
// Selected before hdr-enabled so the first description built already uses
|
||||
// the requested mode; the native side re-applies either way.
|
||||
//
|
||||
// A rejected value aborts instead of carrying on. This harness exists to
|
||||
// produce A/B photographs, and continuing in whatever mode happened to be
|
||||
// active would label the result with a leg that was never shown.
|
||||
final toneMapping = Platform.environment['PLEZY_HARNESS_TONEMAP'];
|
||||
if (toneMapping != null && toneMapping.isNotEmpty) {
|
||||
try {
|
||||
await player.setProperty('hdr-tone-mapping', toneMapping);
|
||||
stdout.writeln('HARNESS_TONEMAP $toneMapping');
|
||||
} catch (e) {
|
||||
stdout.writeln('HARNESS_TONEMAP_ERROR $e');
|
||||
await _abort(player, 'bad PLEZY_HARNESS_TONEMAP: $toneMapping');
|
||||
return;
|
||||
}
|
||||
}
|
||||
if (Platform.environment['PLEZY_HARNESS_HDR'] == '1') {
|
||||
try {
|
||||
await player.setProperty('hdr-enabled', 'yes');
|
||||
stdout.writeln('HARNESS_HDR requested');
|
||||
} catch (e) {
|
||||
stdout.writeln('HARNESS_HDR_ERROR $e');
|
||||
}
|
||||
}
|
||||
await player.open(Media(uri));
|
||||
stdout.writeln('HARNESS_OPENED $uri');
|
||||
// Twice, and the second one is the one that means anything. The native
|
||||
// side applies the output description from playback-restart, which has
|
||||
// not fired yet: reading only here reports the pre-HDR defaults and makes
|
||||
// a working transaction look like it never ran. The delayed read is the
|
||||
// independent check that mpv actually holds what the transaction logged
|
||||
// asking for - our own log says what was requested, not what landed.
|
||||
await _reportColourState(player, 'open');
|
||||
Timer(const Duration(seconds: 6), () async {
|
||||
if (!mounted || _player != player) return;
|
||||
await _reportColourState(player, 'settled');
|
||||
});
|
||||
} catch (e, st) {
|
||||
stdout.writeln('HARNESS_ERROR $e\n$st');
|
||||
await _abort(_player, 'error: $e');
|
||||
}
|
||||
}
|
||||
|
||||
@override
|
||||
void dispose() {
|
||||
_reportTimer?.cancel();
|
||||
_probeTimer?.cancel();
|
||||
_quitTimer?.cancel();
|
||||
_insetTimer?.cancel();
|
||||
SchedulerBinding.instance.removeTimingsCallback(_onFrames);
|
||||
// Reported, not ignored: this harness exists to say what mpv did, and a
|
||||
// teardown that failed is part of that.
|
||||
if (_player case final player?) {
|
||||
unawaited(player.dispose().catchError((Object e) => stdout.writeln('HARNESS_DISPOSE_FAILED $e')));
|
||||
}
|
||||
super.dispose();
|
||||
}
|
||||
|
||||
@override
|
||||
Widget build(BuildContext context) {
|
||||
final player = _player;
|
||||
return MaterialApp(
|
||||
debugShowCheckedModeBanner: false,
|
||||
// Transparent all the way down: in plane mode the video is a Wayland
|
||||
// subsurface *below* this surface, so anything opaque here hides it.
|
||||
home: Scaffold(
|
||||
backgroundColor: Colors.transparent,
|
||||
body: player == null
|
||||
? Center(
|
||||
child: Text(_status, style: const TextStyle(color: Colors.white)),
|
||||
)
|
||||
: Padding(
|
||||
// The inset moves the plane to a non-zero origin, which a plain
|
||||
// window resize would never exercise.
|
||||
padding: EdgeInsets.all(_inset),
|
||||
child: Video(player: player, backgroundColor: Colors.transparent),
|
||||
),
|
||||
),
|
||||
);
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user