edde746
41c6389cd4
Four regression flows, each run on a Pixel 7 and, where relevant, a real Android TV box. Discriminating — the baseline fails, HEAD passes: `07_sheet_back_dismiss` pins both halves of the hosted-sheet fix on touch: the barrier removes the rows behind it from the semantics tree, and one Back closes only the sheet while Settings stays the current route, including the nested per-library options page. It fails onc48cbf70, the commit before8e1904dd. `03_tv_library_focus` gains the same occlusion assertion for the TV sort sheet and fails there too. `08_track_choice_survives_pending_pass` picks a non-default audio and subtitle track, lets playback outlive the automatic pass's 5s attempt and 25s deadline, and asserts the choice is still selected. Ona56b9a3dthe audio reverts to the container's default. Both new flows onboard from a cleared install: the TV regressions in the same group leave "Force TV mode" enabled, and a remembered track selection would pre-select the rows under test. Coverage without a comparable baseline: `09_language_picker_locales` asserts the four new endonyms, switches to Turkish, reads root navigation labels from the generated locale, and restores English. The locales do not exist before 7677d159/100d7729, so there is nothing to fail against — this is forward coverage, not a reproduction. `10_tv_settings_navigation` runs on real Android TV hardware, which no existing flow covers: the rail layout a device reports on its own, the TV dialog path for Manage Libraries, the Apple-only Atmos gate staying closed on Android, and the D-pad-only route to the number spinner's accessibility labels, which a single tap would hide. It passes on either side of the range and is verified against both an empty server and one with a resume position, since rail order shifts with that. It does not assert the15b54e2erow density: that change is invisible to a semantics-tree driver, and a pixel `height` assertion would only hold for one DPR. It registers under a new `android-tv-device` group no workflow dispatches. No TV playback flow is included. Entering content on the TV Recommended view has no stable anchor: the accessibility `focused` flag sits on the hero backdrop rather than the rail card, and a resume position anywhere in the library pushes "Recently Added" below the fold and out of the semantics tree entirely. Search is not a way around it either — inputText does not reach the TV search field. Covering TV playback needs a testID on the rail card, not a cleverer selector. Also repairs three assertions that could never fail: `03_tv_library_focus` gated the sort sheet closing on `notVisible: "Sort by"`, but the header renders "Sort By" and Maestro selectors are case-sensitive regexes, so the wait returned immediately and the next D-pad press landed in the sheet's close animation. `open_codec_sample` matched a card's watch state as `watched|unwatched` only. A codec sample keeps a resume position once any earlier flow has played it, so the row announces "N percent watched" and the subflow stopped finding it on a fixture container that outlives one suite. `06_playback_recovery` tapped "Zulu Zone" out of the Recently Added rail, but every seeded alphabet title shares one dateadded, so which of them the rail returns is a tie-break. The flow only needs some playable movie.
250 lines
8.1 KiB
Python
Executable File
250 lines
8.1 KiB
Python
Executable File
#!/usr/bin/env python3
|
|
"""Run automatic Maestro groups and guarded destructive manual targets."""
|
|
|
|
from __future__ import annotations
|
|
|
|
import argparse
|
|
import sys
|
|
from collections.abc import Sequence
|
|
|
|
import run_maestro
|
|
|
|
|
|
ANDROID_15_INSTRUMENTATION_CLASSES = (
|
|
"androidx.media3.decoder.ffmpeg.PlezyFfmpegPlaybackTest,"
|
|
"com.edde746.plezy.exoplayer.PlezyAudioModePlaybackTest"
|
|
)
|
|
ANDROID_15_INSTRUMENTATION_TARGET = "android-15-instrumentation"
|
|
|
|
|
|
GROUPS: dict[str, tuple[tuple[str, ...], ...]] = {
|
|
"android-15": (
|
|
("basic",),
|
|
("catalog",),
|
|
("media",),
|
|
(
|
|
"basic",
|
|
"--flow",
|
|
".maestro/regression_flows/03_tv_library_focus.yaml",
|
|
"--jellyfin-log",
|
|
"build/maestro-tv/library-focus.log",
|
|
"--diagnostics-dir",
|
|
"build/maestro-tv/library-focus-diagnostics",
|
|
),
|
|
(
|
|
"basic",
|
|
"--flow",
|
|
".maestro/regression_flows/04_tv_player_keys.yaml",
|
|
"--jellyfin-log",
|
|
"build/maestro-tv/player-keys.log",
|
|
"--diagnostics-dir",
|
|
"build/maestro-tv/player-keys-diagnostics",
|
|
),
|
|
(
|
|
"basic",
|
|
"--flow",
|
|
".maestro/regression_flows/05_tv_next_episode_back.yaml",
|
|
"--jellyfin-log",
|
|
"build/maestro-tv/next-episode.log",
|
|
"--diagnostics-dir",
|
|
"build/maestro-tv/next-episode-diagnostics",
|
|
),
|
|
(
|
|
"basic",
|
|
"--flow",
|
|
".maestro/regression_flows/07_sheet_back_dismiss.yaml",
|
|
"--jellyfin-log",
|
|
"build/maestro-sheets/back-dismiss.log",
|
|
"--diagnostics-dir",
|
|
"build/maestro-sheets/back-dismiss-diagnostics",
|
|
),
|
|
(
|
|
"basic",
|
|
"--flow",
|
|
".maestro/regression_flows/08_track_choice_survives_pending_pass.yaml",
|
|
"--jellyfin-log",
|
|
"build/maestro-tracks/pending-pass.log",
|
|
"--diagnostics-dir",
|
|
"build/maestro-tracks/pending-pass-diagnostics",
|
|
),
|
|
(
|
|
"basic",
|
|
"--flow",
|
|
".maestro/regression_flows/09_language_picker_locales.yaml",
|
|
"--jellyfin-log",
|
|
"build/maestro-i18n/language-picker.log",
|
|
"--diagnostics-dir",
|
|
"build/maestro-i18n/language-picker-diagnostics",
|
|
),
|
|
(
|
|
"basic",
|
|
"--fault",
|
|
"music-failure",
|
|
"--flow",
|
|
".maestro/real_flows/02_music_browse.yaml",
|
|
"--jellyfin-log",
|
|
"build/maestro-recovery/music-jellyfin.log",
|
|
"--proxy-journal",
|
|
"build/maestro-recovery/music-proxy-journal.jsonl",
|
|
"--diagnostics-dir",
|
|
"build/maestro-recovery/music-diagnostics",
|
|
),
|
|
(
|
|
"basic",
|
|
"--fault",
|
|
"offline",
|
|
"--flow",
|
|
".maestro/flows/09_download_offline_playback.yaml",
|
|
"--jellyfin-log",
|
|
"build/maestro-offline/jellyfin.log",
|
|
"--proxy-log",
|
|
"build/maestro-offline/jellyfin-proxy.log",
|
|
"--proxy-journal",
|
|
"build/maestro-offline/proxy-journal.jsonl",
|
|
"--diagnostics-dir",
|
|
"build/maestro-offline/diagnostics",
|
|
),
|
|
(
|
|
"basic",
|
|
"--fault",
|
|
"recovery",
|
|
"--flow",
|
|
".maestro/regression_flows/06_playback_recovery.yaml",
|
|
"--jellyfin-log",
|
|
"build/maestro-recovery/jellyfin.log",
|
|
"--proxy-journal",
|
|
"build/maestro-recovery/proxy-journal.jsonl",
|
|
"--diagnostics-dir",
|
|
"build/maestro-recovery/diagnostics",
|
|
),
|
|
),
|
|
"android-9": (
|
|
(
|
|
"basic",
|
|
# API 28's emulator routing to the 10.0.2.2 host alias is unreliable
|
|
# on this image, so reach Jellyfin over an adb reverse mapping the
|
|
# way the media suite already does.
|
|
"--adb-reverse",
|
|
"--flow",
|
|
".maestro/flows/05_playback.yaml",
|
|
"--jellyfin-log",
|
|
"build/maestro-legacy/jellyfin.log",
|
|
"--diagnostics-dir",
|
|
"build/maestro-legacy/diagnostics",
|
|
),
|
|
),
|
|
# Real Android TV hardware only, so no workflow dispatches it. The three
|
|
# `tv` regressions above run on a phone emulator that `onboard_jellyfin_tv`
|
|
# forces into TV mode; this drives the rail layout a device reports on its
|
|
# own. The D-pad-only path is also the only way to reach the TV number
|
|
# spinner, which InputModeTracker hides as soon as a tap arrives. Run as
|
|
# `python3 scripts/run_maestro_ci.py android-tv-device` with
|
|
# MAESTRO_DEVICE_ID set to the box.
|
|
"android-tv-device": (
|
|
(
|
|
"basic",
|
|
"--adb-reverse",
|
|
"--flow",
|
|
".maestro/regression_flows/10_tv_settings_navigation.yaml",
|
|
"--jellyfin-log",
|
|
"build/maestro-tv-device/settings-navigation.log",
|
|
"--diagnostics-dir",
|
|
"build/maestro-tv-device/settings-navigation-diagnostics",
|
|
),
|
|
),
|
|
}
|
|
|
|
|
|
DESTRUCTIVE_MANUAL_TARGETS: dict[str, tuple[tuple[str, ...], ...]] = {
|
|
"profile-regressions": (
|
|
(
|
|
"basic",
|
|
"--flow",
|
|
".maestro/regression_flows/01_profile_switch_isolation.yaml",
|
|
"--jellyfin-log",
|
|
"build/maestro-profile-regressions/profile-switch-isolation-jellyfin.log",
|
|
"--diagnostics-dir",
|
|
"build/maestro-profile-regressions/profile-switch-isolation-diagnostics",
|
|
),
|
|
(
|
|
"basic",
|
|
"--flow",
|
|
".maestro/regression_flows/02_profile_teardown.yaml",
|
|
"--jellyfin-log",
|
|
"build/maestro-profile-regressions/profile-teardown-jellyfin.log",
|
|
"--diagnostics-dir",
|
|
"build/maestro-profile-regressions/profile-teardown-diagnostics",
|
|
),
|
|
),
|
|
}
|
|
|
|
|
|
def run_android_15_instrumentation() -> None:
|
|
print("==> Android 15 filtered instrumentation", flush=True)
|
|
run_maestro._run_checked(
|
|
(
|
|
"android/gradlew",
|
|
"-p",
|
|
"android",
|
|
":app:connectedDebugAndroidTest",
|
|
"-x",
|
|
":app:compileFlutterBuildDebug",
|
|
f"-Pandroid.testInstrumentationRunnerArguments.class={ANDROID_15_INSTRUMENTATION_CLASSES}",
|
|
)
|
|
)
|
|
|
|
|
|
def run_recipes(recipes: tuple[tuple[str, ...], ...]) -> int:
|
|
failed = False
|
|
for arguments in recipes:
|
|
print(f"==> Maestro {' '.join(arguments)}", flush=True)
|
|
exit_status = run_maestro.main(arguments)
|
|
if exit_status >= 128:
|
|
return exit_status
|
|
failed = failed or exit_status != 0
|
|
return 1 if failed else 0
|
|
|
|
|
|
def run_group(name: str) -> int:
|
|
return run_recipes(GROUPS[name])
|
|
|
|
|
|
def run_target(name: str, *, disposable_emulator: bool = False) -> int:
|
|
if name in DESTRUCTIVE_MANUAL_TARGETS:
|
|
if not disposable_emulator:
|
|
print(
|
|
f"Refusing destructive manual target {name!r}: "
|
|
"re-run with --disposable-emulator only on a disposable emulator.",
|
|
file=sys.stderr,
|
|
)
|
|
return 2
|
|
print(f"==> DESTRUCTIVE manual Maestro target: {name}", flush=True)
|
|
return run_recipes(DESTRUCTIVE_MANUAL_TARGETS[name])
|
|
if name == ANDROID_15_INSTRUMENTATION_TARGET:
|
|
run_android_15_instrumentation()
|
|
return 0
|
|
return run_group(name)
|
|
|
|
|
|
def main(argv: Sequence[str] | None = None) -> int:
|
|
parser = argparse.ArgumentParser(description=__doc__)
|
|
parser.add_argument(
|
|
"target",
|
|
choices=(
|
|
*GROUPS,
|
|
ANDROID_15_INSTRUMENTATION_TARGET,
|
|
*DESTRUCTIVE_MANUAL_TARGETS,
|
|
),
|
|
)
|
|
parser.add_argument(
|
|
"--disposable-emulator",
|
|
action="store_true",
|
|
help="opt in to a destructive manual target on a disposable emulator",
|
|
)
|
|
args = parser.parse_args(argv)
|
|
return run_target(args.target, disposable_emulator=args.disposable_emulator)
|
|
|
|
|
|
if __name__ == "__main__":
|
|
raise SystemExit(main())
|