edde746
41a2e996e1
The Dart suite spent 77% of its cost compiling one isolate per test file while `flutter test` used half the cores, and every Maestro flow replayed a full Jellyfin onboarding before its first real assertion. - Add scripts/run_tests.sh, which runs `flutter test` with -j set to the cores the process may actually use instead of the ncpu/2 default. Measured on 8 cores: 190s -> 136s; -j 12 regresses to 165s, so it scales to the core count rather than hard-coding one. CI and CONTRIBUTING use it. A cgroup v2 quota, a cgroup v1 quota, and the cpuset/affinity nproc reports can each be the binding limit independently, so the detector takes the smallest; trusting whichever it found first would oversubscribe 4x on a container holding an 8-CPU quota while pinned to 2. Covered by scripts/test_run_tests.py, which the ci_guard_checks.sh glob picks up. - Add .maestro/subflows/ensure_onboarded.yaml: cold-start the app and only onboard when no session is stored. Flows that just need a signed-in Home use it; 02_onboarding_home, 08_logout, 09_download_offline_playback and the profile regressions keep clearing state. 59s -> 16s per flow. - Guard onboarding's two optional taps behind visibility checks. A missed `optional: true` tap still runs the full element search, costing 3.0s and 7.8s per onboarding to find nothing. - Disable device animation scales in run_maestro.py, restored by the existing cleanup path. CI's emulator got this from the runner flag; physical devices never did. - Shorten the watch_together setup-timeout replacement from 500ms to the 10ms the same file already proves sufficient, and shorten the retry backoff at the one site that missed it: 8.04s -> 1.59s of execution. - Make the LAN discovery waits deadline-based and resend the beacon while polling. Loopback UDP drops datagrams under load, which timed out a wait that could never be satisfied; this was the suite's one flaky test. - Fix 08_logout, which searched for "Logout" and "Are you sure you want to logout?" after both strings became "Log out". The flow had been failing and aborting the suite before 09 ever ran. flutter test 190s -> 131s. Maestro's Android suite 621s -> 385s across the eight flows the baseline reached, and now runs all nine green.
118 lines
3.5 KiB
Bash
Executable File
118 lines
3.5 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
set -uo pipefail
|
|
|
|
# Run the Flutter test suite with a concurrency that matches the host.
|
|
#
|
|
# `flutter test` defaults to ceil(numCPUs / 2), which leaves half the machine
|
|
# idle. That default is a poor fit here because roughly three quarters of this
|
|
# suite's cost is per-file Dart kernel compilation rather than test execution
|
|
# (436 test files, each its own isolate), and compilation scales with cores.
|
|
#
|
|
# Measured on an 8-core host, full suite:
|
|
# -j 4 (the default) 190s
|
|
# -j 6 168s
|
|
# -j 8 136s
|
|
# -j 12 165s
|
|
#
|
|
# One job per core wins; oversubscribing regresses. So scale to the core count
|
|
# instead of hard-coding a number that would oversubscribe smaller CI runners.
|
|
#
|
|
# Any arguments are forwarded to `flutter test`, and an explicit -j/--concurrency
|
|
# still overrides the computed value.
|
|
|
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
|
|
|
|
# Overridable so scripts/test_run_tests.py can point the detector at fixtures.
|
|
: "${PLEZY_CGROUP_ROOT:=/sys/fs/cgroup}"
|
|
|
|
# Cores this process may actually use.
|
|
#
|
|
# Three limits can each be the binding one, and a container can hit any subset
|
|
# of them: a generous CPU quota paired with a narrow cpuset is as common as the
|
|
# reverse. Taking whichever is discovered first would oversubscribe whenever a
|
|
# different one binds, so collect them all and use the smallest.
|
|
#
|
|
# cgroup v2 quota cpu.max ("<quota> <period>", or "max" when unlimited)
|
|
# cgroup v1 quota cpu.cfs_quota_us / cpu.cfs_period_us (-1 when unlimited)
|
|
# affinity/cpuset reported by nproc, which honours sched_getaffinity
|
|
online_cpus() {
|
|
if command -v nproc >/dev/null 2>&1; then
|
|
nproc 2>/dev/null && return
|
|
fi
|
|
if command -v sysctl >/dev/null 2>&1; then
|
|
sysctl -n hw.ncpu 2>/dev/null && return
|
|
fi
|
|
getconf _NPROCESSORS_ONLN 2>/dev/null
|
|
}
|
|
|
|
# ceil(quota / period), skipped unless both are positive integers.
|
|
quota_cpus() {
|
|
local quota="$1" period="$2"
|
|
case "$quota$period" in
|
|
'' | *[!0-9]*) return 1 ;;
|
|
esac
|
|
[ "$period" -gt 0 ] || return 1
|
|
[ "$quota" -gt 0 ] || return 1
|
|
echo $(((quota + period - 1) / period))
|
|
}
|
|
|
|
detect_cpus() {
|
|
local limits=() value quota period
|
|
|
|
value="$(online_cpus)"
|
|
case "$value" in
|
|
'' | *[!0-9]*) ;;
|
|
*) limits+=("$value") ;;
|
|
esac
|
|
|
|
if [ -r "$PLEZY_CGROUP_ROOT/cpu.max" ]; then
|
|
read -r quota period <"$PLEZY_CGROUP_ROOT/cpu.max" || true
|
|
if value="$(quota_cpus "${quota:-}" "${period:-}")"; then
|
|
limits+=("$value")
|
|
fi
|
|
fi
|
|
|
|
if [ -r "$PLEZY_CGROUP_ROOT/cpu/cpu.cfs_quota_us" ] &&
|
|
[ -r "$PLEZY_CGROUP_ROOT/cpu/cpu.cfs_period_us" ]; then
|
|
read -r quota <"$PLEZY_CGROUP_ROOT/cpu/cpu.cfs_quota_us" || true
|
|
read -r period <"$PLEZY_CGROUP_ROOT/cpu/cpu.cfs_period_us" || true
|
|
if value="$(quota_cpus "${quota:-}" "${period:-}")"; then
|
|
limits+=("$value")
|
|
fi
|
|
fi
|
|
|
|
# Nothing readable anywhere: prefer a conservative guess over the host count.
|
|
if [ "${#limits[@]}" -eq 0 ]; then
|
|
echo 4
|
|
return
|
|
fi
|
|
|
|
local smallest="${limits[0]}"
|
|
for value in "${limits[@]}"; do
|
|
[ "$value" -lt "$smallest" ] && smallest="$value"
|
|
done
|
|
[ "$smallest" -lt 1 ] && smallest=1
|
|
echo "$smallest"
|
|
}
|
|
|
|
# Sourced by the tests to exercise the detector; only a direct run continues.
|
|
if [ "${BASH_SOURCE[0]}" != "$0" ]; then
|
|
return 0
|
|
fi
|
|
|
|
cd "$ROOT"
|
|
|
|
for arg in "$@"; do
|
|
case "$arg" in
|
|
-j | --concurrency | -j=* | --concurrency=*)
|
|
exec flutter test "$@"
|
|
;;
|
|
esac
|
|
done
|
|
|
|
CPUS="$(detect_cpus)"
|
|
|
|
echo "==> flutter test -j $CPUS ${*:-}"
|
|
exec flutter test -j "$CPUS" "$@"
|