ci(linux): check the runner's libraries reach the package metadata
The plane added three runtime libraries that bundle-libs.sh deliberately does not bundle, so they have to be declared per distro by hand - and two hand-maintained lists drifting apart is the failure this guard exists to prevent. check_linux_package_deps.py parses the runner's CMake for every pkg-config module it links, follows target_link_libraries to prove each one actually reaches the binary, and requires a package name for it in every distro's depends list. It fails closed on the shapes a naive parser gets wrong: a pkg_check_modules call naming several modules, options preceding the module name, and version constraints like mpv>=0.40 that would otherwise be read as a package nobody ships. The smoke job builds the three packages and reads the dependencies back out of the artifacts, deriving what to expect from build-packages.py rather than restating it - so a library is declared once and verified everywhere. That job is off by default, which is exactly why it must not carry its own copy of the list. The Linux native job names libwayland-dev and libegl-dev instead of riding GTK's and epoxy's transitive dev dependencies, matching the CMake comment's own rationale. In CI the host-dependency guard runs once: the named step covers the staged bundle, and build-packages.py's internal run - which exists for by-hand packaging - is skipped. The smoke job also drops patchelf, which nothing invokes.
This commit is contained in:
edde746
+47
-18
@@ -765,14 +765,17 @@ jobs:
|
||||
fi
|
||||
echo "BUNDLE_DIR=$bundle_dir" >> "$GITHUB_ENV"
|
||||
|
||||
- name: Build Linux Packages
|
||||
shell: bash
|
||||
run: |
|
||||
BUILD_DIR="$BUNDLE_DIR" \
|
||||
ARCH_SUFFIX=${{ matrix.arch }} \
|
||||
OUTPUT_DIR="$GITHUB_WORKSPACE" \
|
||||
python3 linux/packaging/build-packages.py
|
||||
|
||||
# Everything below resolves the bundle *before* packaging, so the packages
|
||||
# and the tarball are cut from one identical tree.
|
||||
#
|
||||
# The plane needs the pinned Wayland-enabled libmpv: a distro libmpv still
|
||||
# plays and still does HDR, but silently drops hwdec to vaapi-copy - which
|
||||
# was measured, not assumed. So libmpv travels with us and no artifact
|
||||
# depends on a host one. That removed the host `mpv` dependency, and with it
|
||||
# the transitive pull of everything libmpv itself needs - libass, pulse,
|
||||
# pipewire, fontconfig and the rest. bundle-libs.sh is what supplies those,
|
||||
# so it has to run before packaging too, or the packages would carry libmpv
|
||||
# and nothing it links.
|
||||
- name: Copy libmpv into bundle
|
||||
shell: bash
|
||||
run: |
|
||||
@@ -781,7 +784,7 @@ jobs:
|
||||
cp -a "$LIBMPV_DIR"/libmpv.so* "$BUNDLE_LIB/"
|
||||
cp -a libmpv-prefix/lib/libshaderc_shared.so* "$BUNDLE_LIB/"
|
||||
|
||||
- name: Bundle shared libraries for portable tarball
|
||||
- name: Bundle shared libraries
|
||||
shell: bash
|
||||
run: bash linux/packaging/bundle-libs.sh "$BUNDLE_DIR"
|
||||
|
||||
@@ -789,17 +792,43 @@ jobs:
|
||||
shell: bash
|
||||
run: cp linux/packaging/plezy.sh "$BUNDLE_DIR/plezy.sh"
|
||||
|
||||
- name: Verify no missing dependencies
|
||||
# Same check the package smoke build runs, against the artifact that
|
||||
# actually ships: a library nobody declares is a broken install, and it is
|
||||
# invisible until a user on a clean machine tries to launch.
|
||||
#
|
||||
# This also stands in for the `ldd ./plezy | grep "not found"` step that
|
||||
# used to run after packaging. That one folded ldd's stderr into grep's
|
||||
# input and dropped its exit status, so ldd failing outright - a missing
|
||||
# loader, an exec-format mismatch, no ldd at all - left the match empty and
|
||||
# printed "All dependencies resolved." This guard runs ldd over every
|
||||
# object under lib/ as well as the executable, fails on an unresolved
|
||||
# soname, fails when ldd cannot read an object, and refuses to pass when
|
||||
# the walk found no host libraries at all. Packaging below only reads the
|
||||
# bundle, so a second shell ldd afterwards could only restate a weaker
|
||||
# subset of what this already proved about the very same tree.
|
||||
- name: Verify every unbundled library the bundle needs is declared
|
||||
shell: bash
|
||||
run: python3 linux/packaging/check-bundle-host-deps.py "$BUNDLE_DIR"
|
||||
|
||||
# Last, from the fully resolved tree above. The host-dependency guard is
|
||||
# skipped because the named step above just ran it against this same
|
||||
# bundle; the internal run exists for by-hand packaging outside CI.
|
||||
- name: Build Linux Packages
|
||||
shell: bash
|
||||
run: |
|
||||
cd "$BUNDLE_DIR"
|
||||
MISSING=$(LD_LIBRARY_PATH=lib ldd ./plezy 2>&1 | grep "not found" || true)
|
||||
if [[ -n "$MISSING" ]]; then
|
||||
echo "ERROR: Unresolved dependencies found:" >&2
|
||||
echo "$MISSING" >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "All dependencies resolved."
|
||||
BUILD_DIR="$BUNDLE_DIR" \
|
||||
ARCH_SUFFIX=${{ matrix.arch }} \
|
||||
OUTPUT_DIR="$GITHUB_WORKSPACE" \
|
||||
PLEZY_SKIP_HOST_DEP_CHECK=1 \
|
||||
python3 linux/packaging/build-packages.py
|
||||
|
||||
# The depends lists reached fpm above; only the packages it wrote can show
|
||||
# they arrived. Same script the smoke build runs, so the two jobs cannot
|
||||
# drift on what counts as declared - and unlike the smoke build, this one
|
||||
# covers arm64 and the artifacts users actually install.
|
||||
- name: Verify the declared dependencies reached the package metadata
|
||||
shell: bash
|
||||
run: python3 linux/packaging/check-package-deps.py "$GITHUB_WORKSPACE" --arch ${{ matrix.arch }}
|
||||
|
||||
- name: Create tarball
|
||||
shell: bash
|
||||
|
||||
+170
-3
@@ -9,6 +9,14 @@ on:
|
||||
branches:
|
||||
- main
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
build_linux_packages:
|
||||
description: >
|
||||
Smoke-build the Linux deb/rpm/pacman packages. Off by default: it needs
|
||||
a release build plus fpm, and build.yml only packages from main, so this
|
||||
is the only way to exercise linux/packaging from a branch.
|
||||
default: false
|
||||
type: boolean
|
||||
|
||||
env:
|
||||
# Only place this workflow names the SDK; .github/actions/setup-flutter-git pins the same release.
|
||||
@@ -244,7 +252,7 @@ jobs:
|
||||
- name: Verify native formatting
|
||||
run: scripts/format_native.sh --check
|
||||
|
||||
- name: Verify Linux native acquisition integrity
|
||||
- name: Verify Linux native acquisition and build plan
|
||||
run: bash linux/packaging/build-libmpv_test.sh
|
||||
|
||||
linux-native-test:
|
||||
@@ -279,7 +287,8 @@ jobs:
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y --no-install-recommends \
|
||||
clang cmake ninja-build pkg-config libgtk-3-dev liblzma-dev \
|
||||
libstdc++-12-dev libmpv-dev libepoxy-dev libcurl4-openssl-dev libevdev-dev
|
||||
libstdc++-12-dev libmpv-dev libepoxy-dev libcurl4-openssl-dev libevdev-dev \
|
||||
libwayland-dev libegl-dev
|
||||
|
||||
- name: Prepare Flutter Linux configuration
|
||||
run: |
|
||||
@@ -295,13 +304,20 @@ jobs:
|
||||
-DPLEZY_BUILD_MPV_RELIABILITY_TESTS=ON \
|
||||
-DPLEZY_MPV_RELIABILITY_SANITIZER=${{ matrix.sanitizer }}
|
||||
|
||||
# `plezy` is the runner itself. Without it nothing in CI ever compiles
|
||||
# my_application.cc, mpv_plugin.cc or the Wayland video plane — the
|
||||
# reliability test targets each pull in only a couple of translation units,
|
||||
# so a break in the rest of linux/runner reached a release build unseen.
|
||||
- name: Build Linux native reliability tests
|
||||
run: |
|
||||
cmake --build build/linux-native-${{ matrix.sanitizer }} --parallel 2 --target \
|
||||
plezy \
|
||||
mpv_player_lifecycle_test \
|
||||
mpv_player_hdr_output_test \
|
||||
mpv_property_result_contract_test \
|
||||
hdr_metadata_test \
|
||||
plane_geometry_test
|
||||
plane_geometry_test \
|
||||
video_params_test
|
||||
|
||||
- name: Run Linux native reliability tests
|
||||
run: |
|
||||
@@ -593,3 +609,154 @@ jobs:
|
||||
|
||||
- name: Run website checks
|
||||
run: scripts/ci_website_checks.sh
|
||||
|
||||
|
||||
# The only job that checks packaging against a real artifact. build.yml also
|
||||
# packages Linux, but refuses any ref but refs/heads/main, and
|
||||
# check_linux_package_deps.py can only compare a hand-written list against
|
||||
# CMake - it cannot prove the list reaches the artifact, nor see the
|
||||
# transitive libraries the bundled libmpv drags in. This can, by reading the
|
||||
# built bundle back with ldd.
|
||||
#
|
||||
# Runs on every push to main and on request, but not on pull requests: it
|
||||
# needs a release build plus fpm, which is minutes of runner time that most
|
||||
# changes here have no reason to pay. That buys post-merge detection rather
|
||||
# than pre-merge, which is the deliberate trade. Dispatch it from a branch
|
||||
# with build_linux_packages when touching linux/packaging - which is the only
|
||||
# way to exercise it before merging.
|
||||
linux-packages:
|
||||
name: Linux package smoke build
|
||||
if: ${{ inputs.build_linux_packages || github.event_name == 'push' }}
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: read
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Setup Flutter
|
||||
uses: subosito/flutter-action@1a449444c387b1966244ae4d4f8c696479add0b2 # v2
|
||||
with:
|
||||
channel: "stable"
|
||||
flutter-version: ${{ env.FLUTTER_VERSION }}
|
||||
cache: true
|
||||
pub-cache: false
|
||||
|
||||
# The packaging deps, minus libmpv: this job builds it from source below,
|
||||
# because the distro's is a different version with different windowing
|
||||
# backends, and a package smoke-built against it cannot show that
|
||||
# build-libmpv.sh still works or that the bundle it produces is coherent.
|
||||
- name: Install packaging dependencies
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y --no-install-recommends \
|
||||
clang cmake meson ninja-build pkg-config nasm libgtk-3-dev liblzma-dev \
|
||||
libstdc++-12-dev libepoxy-dev libcurl4-openssl-dev libevdev-dev \
|
||||
libasound2-dev libass-dev libfreetype-dev libfontconfig-dev libfribidi-dev \
|
||||
libharfbuzz-dev libegl-dev libgl-dev libgnutls28-dev libpipewire-0.3-dev \
|
||||
libva-dev libxkbcommon-dev libpulse-dev libdbus-1-dev libdrm-dev \
|
||||
libgbm-dev libwayland-dev wayland-protocols liblcms2-dev libmujs-dev \
|
||||
liblua5.2-dev rpm libarchive-tools imagemagick ruby-dev build-essential
|
||||
sudo gem install fpm --version 1.17.0 --no-document
|
||||
|
||||
# Keyed the same way build.yml keys it, so editing the script or its pinned
|
||||
# inputs is what invalidates the cache - and this branch's whole point is
|
||||
# that those edits get exercised somewhere.
|
||||
- name: Cache libmpv build
|
||||
id: libmpv-cache
|
||||
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6
|
||||
with:
|
||||
path: libmpv-prefix
|
||||
key: ci-libmpv-${{ runner.arch }}-${{ hashFiles('linux/packaging/build-libmpv.sh', 'linux/packaging/native-inputs.json') }}
|
||||
|
||||
- name: Build libmpv
|
||||
if: steps.libmpv-cache.outputs.cache-hit != 'true'
|
||||
run: bash linux/packaging/build-libmpv.sh
|
||||
|
||||
# The windowing backends the runner depends on, read off the library the
|
||||
# script just produced. The plane hands mpv MPV_RENDER_PARAM_WL_DISPLAY, so
|
||||
# a libmpv without Wayland cannot find the VAAPI device and quietly decodes
|
||||
# in software; X11 and VDPAU are gone with the texture path.
|
||||
- name: Check the built libmpv's backends
|
||||
run: |
|
||||
LIB=$(find libmpv-prefix -name 'libmpv.so.2' | head -1)
|
||||
echo "== $LIB =="
|
||||
ldd "$LIB" | grep -iE 'wayland|libX11|vdpau' || echo '(none)'
|
||||
ldd "$LIB" | grep -q libwayland-client || {
|
||||
echo "::error::the built libmpv does not link libwayland-client, so the video plane cannot work"
|
||||
exit 1
|
||||
}
|
||||
|
||||
- name: Build the release bundle
|
||||
run: |
|
||||
flutter pub get --enforce-lockfile --no-example
|
||||
flutter build linux --release
|
||||
env:
|
||||
PKG_CONFIG_PATH: ${{ github.workspace }}/libmpv-prefix/lib/pkgconfig:${{ github.workspace }}/libmpv-prefix/lib/x86_64-linux-gnu/pkgconfig
|
||||
|
||||
# Mirrors build.yml: resolve the bundle completely, then package from it.
|
||||
# libmpv travels with us, so nothing depends on a host one - which also
|
||||
# removes the transitive pull of everything libmpv links, hence bundle-libs
|
||||
# before packaging rather than after.
|
||||
- name: Copy libmpv into the bundle
|
||||
run: |
|
||||
BUNDLE_LIB=build/linux/x64/release/bundle/lib
|
||||
LIBMPV_DIR=$(dirname "$(find libmpv-prefix -name 'libmpv.so' | head -1)")
|
||||
cp -a "$LIBMPV_DIR"/libmpv.so* "$BUNDLE_LIB/"
|
||||
cp -a libmpv-prefix/lib/libshaderc_shared.so* "$BUNDLE_LIB/"
|
||||
|
||||
- name: Bundle shared libraries
|
||||
run: bash linux/packaging/bundle-libs.sh build/linux/x64/release/bundle
|
||||
|
||||
- name: Copy wrapper script into the bundle
|
||||
run: cp linux/packaging/plezy.sh build/linux/x64/release/bundle/plezy.sh
|
||||
|
||||
# Derives what the resolved bundle still needs from the host and proves
|
||||
# every one of those libraries is declared. This is the check that would
|
||||
# have caught bundling libmpv without also declaring what libmpv links.
|
||||
- name: Verify every unbundled library the bundle needs is declared
|
||||
run: python3 linux/packaging/check-bundle-host-deps.py build/linux/x64/release/bundle
|
||||
|
||||
# OUTPUT_DIR defaults to the repo root; name it so the paths below are not
|
||||
# a guess about where fpm dropped things. The host-dependency guard is
|
||||
# skipped because the named step above just ran it against this same
|
||||
# bundle; the internal run exists for by-hand packaging outside CI.
|
||||
- name: Build the packages
|
||||
run: |
|
||||
mkdir -p "$OUTPUT_DIR"
|
||||
python3 linux/packaging/build-packages.py
|
||||
env:
|
||||
OUTPUT_DIR: ${{ github.workspace }}/packages
|
||||
PLEZY_SKIP_HOST_DEP_CHECK: "1"
|
||||
|
||||
# The guard above reconciles the depends lists with the staged bundle; only
|
||||
# the packages themselves can show that list survived fpm. Every name comes
|
||||
# from build-packages.py, so adding a library there is verified here without
|
||||
# a second edit - and this job is off by default, so a hand-copied list
|
||||
# would rot unseen. The release job in build.yml runs the same script
|
||||
# against the artifacts users install, so the assertions cannot drift.
|
||||
- name: Verify the declared dependencies reached the package metadata
|
||||
run: python3 linux/packaging/check-package-deps.py "${{ github.workspace }}/packages"
|
||||
|
||||
# Same resolved bundle the packages were cut from, so the tarball is not a
|
||||
# second, differently-assembled artifact. It is the one to put on a USB for
|
||||
# a foreign machine, because it needs nothing installed.
|
||||
- name: Create the tarball
|
||||
run: |
|
||||
BUNDLE_DIR=build/linux/x64/release/bundle
|
||||
mkdir -p "$OUTPUT_DIR"
|
||||
tar -czf "$OUTPUT_DIR/plezy-linux-x64.tar.gz" -C "$BUNDLE_DIR" .
|
||||
echo "=== libmpv travelling in every artifact ==="
|
||||
ldd "$BUNDLE_DIR/lib/libmpv.so" | grep -iE 'wayland|libX11|vdpau' || echo '(none)'
|
||||
env:
|
||||
OUTPUT_DIR: ${{ github.workspace }}/packages
|
||||
|
||||
- name: Upload the packages
|
||||
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
||||
with:
|
||||
name: linux-packages-smoke
|
||||
path: ${{ github.workspace }}/packages/plezy-linux-x64.*
|
||||
if-no-files-found: error
|
||||
retention-days: 7
|
||||
|
||||
@@ -18,6 +18,23 @@ print(value)
|
||||
PY
|
||||
}
|
||||
|
||||
# For keys that are genuinely optional, where absence means "not offered"
|
||||
# rather than a broken manifest. Pinned values never come through here: a
|
||||
# missing checksum or commit has to stay fatal.
|
||||
manifest_optional() {
|
||||
python3 - "$NATIVE_INPUTS_MANIFEST" "$1" "$2" <<'PY'
|
||||
import json
|
||||
import sys
|
||||
|
||||
with open(sys.argv[1], encoding="utf-8") as source:
|
||||
manifest = json.load(source)
|
||||
value = manifest["inputs"][sys.argv[2]].get(sys.argv[3], "")
|
||||
if not isinstance(value, str):
|
||||
raise SystemExit(f"invalid manifest value: {sys.argv[2]}.{sys.argv[3]}")
|
||||
print(value)
|
||||
PY
|
||||
}
|
||||
|
||||
FFMPEG_VERSION="$(manifest_value ffmpeg version)"
|
||||
FFMPEG_URL="$(manifest_value ffmpeg url)"
|
||||
FFMPEG_SHA256="$(manifest_value ffmpeg sha256)"
|
||||
@@ -27,6 +44,7 @@ SHADERC_REF="$(manifest_value shaderc ref)"
|
||||
SHADERC_COMMIT="$(manifest_value shaderc commit)"
|
||||
LIBPLACEBO_VERSION="$(manifest_value libplacebo version)"
|
||||
LIBPLACEBO_URL="$(manifest_value libplacebo url)"
|
||||
LIBPLACEBO_MIRROR="$(manifest_optional libplacebo mirror)"
|
||||
LIBPLACEBO_REF="$(manifest_value libplacebo ref)"
|
||||
LIBPLACEBO_COMMIT="$(manifest_value libplacebo commit)"
|
||||
MPV_VERSION="$(manifest_value mpv version)"
|
||||
@@ -55,11 +73,18 @@ download_verified() {
|
||||
|
||||
mkdir -p "$(dirname "$destination")"
|
||||
temporary="$(mktemp "${destination}.tmp.XXXXXX")"
|
||||
# Retries cover the transfer only. A checksum mismatch below is never retried:
|
||||
# that is a tampered or moved artefact, not a flaky connection, and trying
|
||||
# again would only turn a loud failure into an intermittent one.
|
||||
if ! curl \
|
||||
--fail \
|
||||
--location \
|
||||
--silent \
|
||||
--show-error \
|
||||
--retry 3 \
|
||||
--retry-connrefused \
|
||||
--retry-delay 5 \
|
||||
--connect-timeout 30 \
|
||||
--proto '=https,file' \
|
||||
--tlsv1.2 \
|
||||
--output "$temporary" \
|
||||
@@ -85,23 +110,44 @@ checkout_verified_ref() {
|
||||
local ref="$2"
|
||||
local expected_commit="$3"
|
||||
local destination="$4"
|
||||
local mirror="${5:-}"
|
||||
local actual_commit
|
||||
local source
|
||||
local attempt
|
||||
|
||||
if [[ ! "$expected_commit" =~ ^[0-9a-f]{40}$ ]]; then
|
||||
echo "Invalid Git commit pin for $url at $ref" >&2
|
||||
return 1
|
||||
fi
|
||||
|
||||
# Retries and the mirror cover the transfer, never the verification. The commit
|
||||
# pin below is checked identically whichever source answered, so a mirror can
|
||||
# only supply the same tree or fail - it cannot substitute another one.
|
||||
#
|
||||
# This exists because code.videolan.org, the only source fetched over git,
|
||||
# refused connections for well over two minutes at a time across several CI
|
||||
# runs and took every build with it.
|
||||
rm -rf "$destination"
|
||||
if ! git clone --quiet --depth 1 --branch "$ref" --no-checkout \
|
||||
"$url" "$destination"; then
|
||||
rm -rf "$destination"
|
||||
for source in "$url" ${mirror:+"$mirror"}; do
|
||||
for attempt in 1 2 3; do
|
||||
if git clone --quiet --depth 1 --branch "$ref" --no-checkout \
|
||||
"$source" "$destination"; then
|
||||
break 2
|
||||
fi
|
||||
rm -rf "$destination"
|
||||
# No point pausing before giving up on this source.
|
||||
if [ "$attempt" -lt 3 ]; then sleep $((attempt * 5)); fi
|
||||
done
|
||||
echo "Could not clone $source at $ref after 3 attempts" >&2
|
||||
done
|
||||
if [ ! -d "$destination" ]; then
|
||||
echo "No source produced $ref for $url" >&2
|
||||
return 1
|
||||
fi
|
||||
|
||||
actual_commit="$(git -C "$destination" rev-parse 'HEAD^{commit}')"
|
||||
if [ "$actual_commit" != "$expected_commit" ]; then
|
||||
echo "Git ref mismatch for $url at $ref" >&2
|
||||
echo "Git ref mismatch for $ref" >&2
|
||||
echo "Expected: $expected_commit" >&2
|
||||
echo "Actual: $actual_commit" >&2
|
||||
rm -rf "$destination"
|
||||
@@ -201,7 +247,7 @@ main() {
|
||||
echo "==> Building libplacebo $LIBPLACEBO_VERSION (static)..."
|
||||
checkout_verified_ref \
|
||||
"$LIBPLACEBO_URL" "$LIBPLACEBO_REF" "$LIBPLACEBO_COMMIT" \
|
||||
"$srcdir/libplacebo-v${LIBPLACEBO_VERSION}"
|
||||
"$srcdir/libplacebo-v${LIBPLACEBO_VERSION}" "$LIBPLACEBO_MIRROR"
|
||||
cd "libplacebo-v${LIBPLACEBO_VERSION}"
|
||||
git submodule update --init --recursive
|
||||
|
||||
|
||||
@@ -14,6 +14,13 @@ assert_absent() {
|
||||
[ ! -e "$1" ] || fail "unexpected path remains: $1"
|
||||
}
|
||||
|
||||
init_repository() {
|
||||
mkdir -p "$1"
|
||||
git -C "$1" init --quiet
|
||||
git -C "$1" config user.name "Plezy provenance test"
|
||||
git -C "$1" config user.email "provenance-test@invalid.example"
|
||||
}
|
||||
|
||||
temporary="$(mktemp -d)"
|
||||
trap 'rm -rf "$temporary"' EXIT
|
||||
|
||||
@@ -36,10 +43,7 @@ fi
|
||||
|
||||
repository="$temporary/repository"
|
||||
checkout="$temporary/checkout"
|
||||
mkdir -p "$repository"
|
||||
git -C "$repository" init --quiet
|
||||
git -C "$repository" config user.name "Plezy provenance test"
|
||||
git -C "$repository" config user.email "provenance-test@invalid.example"
|
||||
init_repository "$repository"
|
||||
printf 'first\n' >"$repository/input.txt"
|
||||
git -C "$repository" add input.txt
|
||||
git -C "$repository" commit --quiet -m first
|
||||
@@ -49,6 +53,27 @@ checkout_verified_ref "file://$repository" release "$approved_commit" "$checkout
|
||||
[ "$(git -C "$checkout" rev-parse HEAD)" = "$approved_commit" ] ||
|
||||
fail "verified checkout selected the wrong commit"
|
||||
|
||||
# A dead primary must fall through to the mirror, because the whole point is
|
||||
# that one unreachable host cannot stop the build. This runs while the tag still
|
||||
# points at the approved commit; the moved-tag case is below.
|
||||
unreachable="file://$temporary/definitely-not-a-repository"
|
||||
checkout_verified_ref "$unreachable" release "$approved_commit" "$checkout" "file://$repository"
|
||||
[ "$(git -C "$checkout" rev-parse HEAD)" = "$approved_commit" ] ||
|
||||
fail "mirror fallback selected the wrong commit"
|
||||
|
||||
# And the mirror answers to the same pin. A mirror serving a different tree is
|
||||
# the one thing a fallback must never quietly accept.
|
||||
mirror_repository="$temporary/mirror"
|
||||
init_repository "$mirror_repository"
|
||||
printf 'substituted\n' >"$mirror_repository/input.txt"
|
||||
git -C "$mirror_repository" add input.txt
|
||||
git -C "$mirror_repository" commit --quiet -m substituted
|
||||
git -C "$mirror_repository" tag release
|
||||
if checkout_verified_ref "$unreachable" release "$approved_commit" "$checkout" "file://$mirror_repository"; then
|
||||
fail "mirror serving another commit was accepted"
|
||||
fi
|
||||
assert_absent "$checkout"
|
||||
|
||||
printf 'second\n' >"$repository/input.txt"
|
||||
git -C "$repository" commit --quiet -am second
|
||||
git -C "$repository" tag --force release >/dev/null
|
||||
@@ -57,4 +82,188 @@ if checkout_verified_ref "file://$repository" release "$approved_commit" "$check
|
||||
fi
|
||||
assert_absent "$checkout"
|
||||
|
||||
echo "Linux native acquisition verification passed"
|
||||
# ─── The build plan ─────────────────────────────────────────────────────────
|
||||
# Everything above checks how sources arrive. What they are then configured
|
||||
# with decides whether the feature works at all, and it fails quietly: a libmpv
|
||||
# built without -Dwayland=enabled still compiles, still links and still plays,
|
||||
# it just has no Wayland backend, so the render context cannot accept
|
||||
# MPV_RENDER_PARAM_WL_DISPLAY, vaapi finds no device and hardware decoding
|
||||
# drops to a copy-back path. Nothing crashes, so nothing else notices. Run
|
||||
# main() for real against stub build tools and assert the argument vectors they
|
||||
# were handed.
|
||||
|
||||
stub_bin="$temporary/stub-bin"
|
||||
stub_extra="$temporary/stub-extra"
|
||||
records="$temporary/records"
|
||||
mkdir -p "$stub_bin" "$stub_extra" "$records" "$temporary/tmp"
|
||||
|
||||
# Deliberately unlike the pinned versions: every path below is derived from the
|
||||
# manifest, so a stub that matched by accident would prove nothing.
|
||||
ffmpeg_version="9.9.9"
|
||||
shaderc_version="6.6.6"
|
||||
libplacebo_version="7.7.7"
|
||||
mpv_version="8.8.8"
|
||||
|
||||
# Each stub records the vector it was called with, one argument per line, plus
|
||||
# the directory it ran in - which is what tells mpv's meson call apart from
|
||||
# libplacebo's. An argument containing a newline would corrupt the record, and
|
||||
# none of these ever does: they are literal flags and mktemp -d paths.
|
||||
make_stub() {
|
||||
local directory="$1" name="$2" extra="${3:-}"
|
||||
cat >"$directory/$name" <<STUB
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
record="\$(mktemp "$records/record.XXXXXX")"
|
||||
{ printf '%s\n' "$name" "\$(pwd)"; [ \$# -eq 0 ] || printf '%s\n' "\$@"; } >"\$record"
|
||||
$extra
|
||||
STUB
|
||||
chmod +x "$directory/$name"
|
||||
}
|
||||
|
||||
payload="$temporary/payload.bin"
|
||||
printf 'stub archive payload\n' >"$payload"
|
||||
payload_sha256="$(sha256_file "$payload")"
|
||||
|
||||
curl_extra='
|
||||
output=""
|
||||
previous=""
|
||||
for argument in "$@"; do
|
||||
if [ "$previous" = "--output" ]; then output="$argument"; fi
|
||||
previous="$argument"
|
||||
done
|
||||
[ -n "$output" ] || { echo "stub curl: no --output argument" >&2; exit 1; }
|
||||
cp -- "'"$payload"'" "$output"
|
||||
'
|
||||
|
||||
# The unmapped case is fatal on purpose: a new archive in the build plan has to
|
||||
# come here and be described rather than silently extract to nothing.
|
||||
tar_extra='
|
||||
archive="$(basename -- "${!#}")"
|
||||
case "$archive" in
|
||||
ffmpeg.tar.xz) directory="ffmpeg-'"$ffmpeg_version"'" ;;
|
||||
mpv.tar.gz) directory="mpv-'"$mpv_version"'" ;;
|
||||
*) echo "stub tar: unexpected archive $archive" >&2; exit 1 ;;
|
||||
esac
|
||||
mkdir -p "$directory"
|
||||
cp -- "'"$stub_extra"'/configure" "$directory/configure"
|
||||
chmod +x "$directory/configure"
|
||||
'
|
||||
|
||||
# ffmpeg runs ./configure out of its own tarball, so that one is planted by the
|
||||
# tar stub rather than found on PATH.
|
||||
make_stub "$stub_extra" configure
|
||||
make_stub "$stub_bin" curl "$curl_extra"
|
||||
make_stub "$stub_bin" tar "$tar_extra"
|
||||
for tool in make cmake meson ninja; do
|
||||
make_stub "$stub_bin" "$tool"
|
||||
done
|
||||
|
||||
# git stays real, pointed at local repositories: the checkout is pinned by
|
||||
# commit, and a stub that answered rev-parse would be asserting its own input.
|
||||
shaderc_repository="$temporary/shaderc-source"
|
||||
init_repository "$shaderc_repository"
|
||||
mkdir -p "$shaderc_repository/utils"
|
||||
printf '#!/bin/sh\nexit 0\n' >"$shaderc_repository/utils/git-sync-deps"
|
||||
chmod +x "$shaderc_repository/utils/git-sync-deps"
|
||||
git -C "$shaderc_repository" add utils/git-sync-deps
|
||||
# Windows checkouts do not track the mode bit, and the build plan executes it.
|
||||
git -C "$shaderc_repository" update-index --chmod=+x utils/git-sync-deps
|
||||
git -C "$shaderc_repository" commit --quiet -m shaderc
|
||||
git -C "$shaderc_repository" tag release
|
||||
shaderc_commit="$(git -C "$shaderc_repository" rev-parse HEAD)"
|
||||
|
||||
libplacebo_repository="$temporary/libplacebo-source"
|
||||
init_repository "$libplacebo_repository"
|
||||
printf 'stub libplacebo\n' >"$libplacebo_repository/meson.build"
|
||||
git -C "$libplacebo_repository" add meson.build
|
||||
git -C "$libplacebo_repository" commit --quiet -m libplacebo
|
||||
git -C "$libplacebo_repository" tag release
|
||||
libplacebo_commit="$(git -C "$libplacebo_repository" rev-parse HEAD)"
|
||||
|
||||
manifest="$temporary/native-inputs.json"
|
||||
cat >"$manifest" <<JSON
|
||||
{
|
||||
"inputs": {
|
||||
"ffmpeg": {
|
||||
"version": "$ffmpeg_version",
|
||||
"url": "https://stub.invalid/ffmpeg.tar.xz",
|
||||
"sha256": "$payload_sha256"
|
||||
},
|
||||
"shaderc": {
|
||||
"version": "$shaderc_version",
|
||||
"url": "file://$shaderc_repository",
|
||||
"ref": "release",
|
||||
"commit": "$shaderc_commit"
|
||||
},
|
||||
"libplacebo": {
|
||||
"version": "$libplacebo_version",
|
||||
"url": "file://$libplacebo_repository",
|
||||
"ref": "release",
|
||||
"commit": "$libplacebo_commit"
|
||||
},
|
||||
"mpv": {
|
||||
"version": "$mpv_version",
|
||||
"url": "https://stub.invalid/mpv.tar.gz",
|
||||
"sha256": "$payload_sha256"
|
||||
}
|
||||
}
|
||||
}
|
||||
JSON
|
||||
|
||||
# JOBS keeps nproc out of it; TMPDIR keeps main()'s own mktemp -d inside the
|
||||
# directory this test already cleans up.
|
||||
build_log="$temporary/build.log"
|
||||
if ! (
|
||||
export NATIVE_INPUTS_MANIFEST="$manifest"
|
||||
export PATH="$stub_bin:$PATH"
|
||||
export TMPDIR="$temporary/tmp"
|
||||
export PREFIX="$temporary/prefix"
|
||||
export JOBS=1
|
||||
bash "$SCRIPT_DIR/build-libmpv.sh"
|
||||
) >"$build_log" 2>&1; then
|
||||
cat "$build_log" >&2
|
||||
fail "the stubbed build plan did not run to completion"
|
||||
fi
|
||||
|
||||
recorded_call() {
|
||||
local program="$1" directory="$2" candidate
|
||||
for candidate in "$records"/record.*; do
|
||||
[ -f "$candidate" ] || continue
|
||||
if [ "$(sed -n 1p "$candidate")" = "$program" ] &&
|
||||
[ "$(basename -- "$(sed -n 2p "$candidate")")" = "$directory" ]; then
|
||||
printf '%s\n' "$candidate"
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
return 1
|
||||
}
|
||||
|
||||
# -Fxq, so this matches a whole recorded argument. A substring search would
|
||||
# accept -Dwayland=enabled inside a comment, which is exactly the hole here.
|
||||
assert_argument() {
|
||||
local record="$1" argument="$2" description="$3"
|
||||
tail -n +3 "$record" | grep -Fxq -- "$argument" ||
|
||||
fail "$description does not pass $argument"
|
||||
}
|
||||
|
||||
mpv_meson="$(recorded_call meson "mpv-$mpv_version")" ||
|
||||
fail "the build plan never ran meson in the mpv source tree"
|
||||
[ "$(sed -n 3p "$mpv_meson")" = "setup" ] ||
|
||||
fail "mpv's first meson call is no longer 'setup'"
|
||||
|
||||
assert_argument "$mpv_meson" "-Dwayland=enabled" "mpv's meson setup"
|
||||
assert_argument "$mpv_meson" "-Dx11=disabled" "mpv's meson setup"
|
||||
assert_argument "$mpv_meson" "-Dvaapi=enabled" "mpv's meson setup"
|
||||
assert_argument "$mpv_meson" "-Dgl=enabled" "mpv's meson setup"
|
||||
|
||||
# vaapi has to reach ffmpeg too, or mpv's hwdec has no decoder behind it.
|
||||
ffmpeg_configure="$(recorded_call configure "ffmpeg-$ffmpeg_version")" ||
|
||||
fail "the build plan never configured ffmpeg"
|
||||
assert_argument "$ffmpeg_configure" "--enable-vaapi" "ffmpeg's configure"
|
||||
|
||||
# libplacebo runs meson as well, and its call carries no Wayland flag: finding
|
||||
# it separately is what proves the directory above really discriminated.
|
||||
recorded_call meson "libplacebo-v$libplacebo_version" >/dev/null ||
|
||||
fail "the build plan never ran meson in the libplacebo source tree"
|
||||
|
||||
echo "Linux native acquisition and build plan verification passed"
|
||||
|
||||
@@ -3,6 +3,7 @@ import os
|
||||
import re
|
||||
import subprocess
|
||||
import shutil
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# Paths
|
||||
@@ -41,16 +42,50 @@ DISTROS = {
|
||||
# The native video plane links wayland-client, wayland-egl and EGL, and
|
||||
# bundle-libs.sh deliberately never bundles those: they are coupled to
|
||||
# the running compositor and GPU driver. So they have to be declared.
|
||||
#
|
||||
# libmpv itself travels *in* the package rather than being depended on,
|
||||
# because the plane needs the pinned Wayland-enabled build - a distro
|
||||
# libmpv without Wayland silently drops hwdec to vaapi-copy. That means
|
||||
# the libraries the host libmpv used to drag in transitively are now ours
|
||||
# to declare: libva and its drm/wayland backends for hwdec, and drm/gbm
|
||||
# underneath them.
|
||||
#
|
||||
# The X11 and xcb entries are not a mistake on a Wayland-only video
|
||||
# path: GTK links them whatever backend it runs, and ffmpeg's VA-API
|
||||
# brings libva-x11 with it. They used to arrive via the host gtk3 and
|
||||
# mpv packages.
|
||||
#
|
||||
# Miss any one of these and the loader fails before main() runs, so
|
||||
# check-bundle-host-deps.py re-derives the whole list from the built
|
||||
# bundle rather than trusting this comment - main() below runs it before
|
||||
# anything is packaged, and the release workflow runs it again.
|
||||
"depends": [
|
||||
"libgtk-3-0",
|
||||
"libmpv2 | libmpv1",
|
||||
"libepoxy0",
|
||||
"libasound2",
|
||||
"libevdev2",
|
||||
"libglib2.0-0",
|
||||
"libwayland-client0",
|
||||
"libwayland-cursor0",
|
||||
"libwayland-egl1",
|
||||
"libegl1",
|
||||
# libEGL.so.1 defers to libGLdispatch.so.0, which is a separate
|
||||
# package Debian only pulls in behind libegl1. The runner links it
|
||||
# directly, so it is declared directly.
|
||||
"libglvnd0",
|
||||
"libva2",
|
||||
"libva-drm2",
|
||||
"libva-wayland2",
|
||||
"libva-x11-2",
|
||||
"libdrm2",
|
||||
"libgbm1",
|
||||
"libx11-6",
|
||||
"libx11-xcb1",
|
||||
"libxext6",
|
||||
"libxcb1",
|
||||
"libxcb-dri3-0",
|
||||
"libxcb-render0",
|
||||
"libxcb-shm0",
|
||||
],
|
||||
},
|
||||
"rpm": {
|
||||
@@ -58,16 +93,26 @@ DISTROS = {
|
||||
"category": "Multimedia",
|
||||
"ext": "rpm",
|
||||
"compression": ["--rpm-compression", "xzmt"],
|
||||
# Fedora splits glvnd: libglvnd-egl carries libEGL.so.1 but the
|
||||
# libGLdispatch.so.0 it defers to lives in the base libglvnd package.
|
||||
"depends": [
|
||||
"gtk3",
|
||||
"mpv-libs",
|
||||
"libepoxy",
|
||||
"alsa-lib",
|
||||
"libevdev",
|
||||
"glib2",
|
||||
"libwayland-client",
|
||||
"libwayland-cursor",
|
||||
"libwayland-egl",
|
||||
"libglvnd",
|
||||
"libglvnd-egl",
|
||||
"libva",
|
||||
"libdrm",
|
||||
"mesa-libgbm",
|
||||
"libX11",
|
||||
"libX11-xcb",
|
||||
"libXext",
|
||||
"libxcb",
|
||||
],
|
||||
},
|
||||
"pacman": {
|
||||
@@ -75,17 +120,24 @@ DISTROS = {
|
||||
"category": None,
|
||||
"ext": "pkg.tar.zst",
|
||||
"compression": ["--pacman-compression", "zstd"],
|
||||
# Arch ships every libwayland-* in the one `wayland` package, and
|
||||
# libglvnd is what provides libEGL.so.1.
|
||||
# Arch ships every libwayland-* in the one `wayland` package, libglvnd is
|
||||
# what provides both libEGL.so.1 and libGLdispatch.so.0, mesa is what
|
||||
# provides libgbm.so.1, and libx11 carries libX11-xcb.so.1 alongside
|
||||
# libX11.so.6.
|
||||
"depends": [
|
||||
"gtk3",
|
||||
"mpv",
|
||||
"libepoxy",
|
||||
"alsa-lib",
|
||||
"libevdev",
|
||||
"glib2",
|
||||
"wayland",
|
||||
"libglvnd",
|
||||
"libva",
|
||||
"libdrm",
|
||||
"mesa",
|
||||
"libx11",
|
||||
"libxext",
|
||||
"libxcb",
|
||||
],
|
||||
},
|
||||
}
|
||||
@@ -184,9 +236,76 @@ def main():
|
||||
# Verify build exists
|
||||
if not BUILD_DIR.exists():
|
||||
print(f"Error: Build directory not found at {BUILD_DIR}")
|
||||
print("Please run 'flutter build linux --release' first or set BUILD_DIR")
|
||||
print("Please run 'flutter build linux --release' first or set BUILD_DIR.")
|
||||
# The runner requires pkg-config to find mpv, and build-libmpv.sh exports
|
||||
# PKG_CONFIG_PATH only inside its own process. Without carrying it across,
|
||||
# the build either cannot configure at all or - worse, because it looks
|
||||
# like success - links whatever distro libmpv happens to be installed
|
||||
# instead of the pinned Wayland-enabled one these packages assume. CI sets
|
||||
# this explicitly for the same reason and installs no libmpv-dev.
|
||||
print("The build needs the pinned libmpv on its pkg-config path, which build-libmpv.sh")
|
||||
print("exports only for itself:")
|
||||
print(' PKG_CONFIG_PATH="$(pwd)/libmpv-prefix/lib/pkgconfig:$(pwd)/libmpv-prefix/lib/x86_64-linux-gnu/pkgconfig" \\')
|
||||
print(" flutter build linux --release")
|
||||
exit(1)
|
||||
|
||||
# None of the three package manifests declares a host libmpv any more,
|
||||
# because the plane needs the pinned Wayland-enabled build and so ships it.
|
||||
# That makes a bundle without one unpackageable rather than merely thinner:
|
||||
# the runner's DT_NEEDED libmpv.so.2 would be satisfied by nothing and the
|
||||
# loader would fail before main(), on a package that installed cleanly.
|
||||
# Both workflows stage it first; this refuses the standalone path that the
|
||||
# message above otherwise invites.
|
||||
# The versioned soname, not `libmpv.so`. That bare name is the development
|
||||
# linker name; the runner records DT_NEEDED libmpv.so.2 and the loader will
|
||||
# not accept the unversioned file in its place. Matching `libmpv.so*` would
|
||||
# pass a staging tree carrying only the linker name - the exact failure this
|
||||
# check is named for, and the only thing standing between a hand-staged
|
||||
# bundle and a broken package when the host-dep guard is opted out of below.
|
||||
if not sorted((BUILD_DIR / "lib").glob("libmpv.so.[0-9]*")):
|
||||
print(f"Error: no libmpv found in {BUILD_DIR / 'lib'}")
|
||||
print("The packages declare no host libmpv, so one has to travel inside them.")
|
||||
print("Stage the bundle first, exactly as both workflows do:")
|
||||
# meson installs libmpv under an architecture triple on Debian and Ubuntu
|
||||
# while CMake installed shaderc straight into lib/, which is why the two
|
||||
# lines below are not the same shape. The workflows locate libmpv the same
|
||||
# way; a hardcoded libmpv-prefix/lib/libmpv.so* finds nothing there.
|
||||
print(' cp -a "$(dirname "$(find libmpv-prefix -name libmpv.so | head -1)")"/libmpv.so* <bundle>/lib/')
|
||||
# The pinned libmpv links libshaderc_shared, which build-libmpv.sh leaves
|
||||
# in libmpv-prefix - not a directory the loader searches. bundle-libs.sh
|
||||
# cannot recover it either: it resolves what ldd reports, and ldd cannot
|
||||
# find a soname that is only in the build prefix. So it is copied by hand
|
||||
# before the walk, or the bundle gets an unpinned host shaderc at best.
|
||||
print(" cp -a libmpv-prefix/lib/libshaderc_shared.so* <bundle>/lib/")
|
||||
print(" bash linux/packaging/bundle-libs.sh <bundle>")
|
||||
exit(1)
|
||||
|
||||
# Nothing above reconciles the hand-maintained depends lists with what the
|
||||
# staged bundle actually loads, and a package that is short one host library
|
||||
# installs cleanly and then dies in the loader. Run the guard that does
|
||||
# reconcile them before any package exists. As a subprocess under this
|
||||
# interpreter, so its exit code and ::error:: annotations arrive unaltered.
|
||||
guard = SCRIPT_DIR / "check-bundle-host-deps.py"
|
||||
# Truthy tokens only. `PLEZY_SKIP_HOST_DEP_CHECK=0` reads to almost everyone
|
||||
# as "do not skip", and a bare non-empty test would have done the opposite -
|
||||
# which is the single outcome this whole block exists to prevent.
|
||||
skip = os.environ.get("PLEZY_SKIP_HOST_DEP_CHECK", "").strip().lower()
|
||||
if skip in {"1", "true", "yes", "on"}:
|
||||
# The guard reads deb ownership out of dpkg-query, which Fedora and Arch
|
||||
# do not have, so it cannot run there at all. The opt-out is for that
|
||||
# case and is an environment variable precisely so it cannot happen by
|
||||
# accident: a skip the caller did not ask for would hand back exactly the
|
||||
# unverified package this check exists to prevent, so it is never silent.
|
||||
print("WARNING: PLEZY_SKIP_HOST_DEP_CHECK is set, so the host dependency guard did not run.")
|
||||
print(f"WARNING: the depends lists in {Path(__file__).name} are unverified against {BUILD_DIR}.")
|
||||
else:
|
||||
guard_result = subprocess.run([sys.executable, str(guard), str(BUILD_DIR)])
|
||||
if guard_result.returncode != 0:
|
||||
print(f"Error: {guard.name} failed, so no packages were built.")
|
||||
print("Declare the libraries it named, or - only on a host without dpkg-query, such as")
|
||||
print("Fedora or Arch - set PLEZY_SKIP_HOST_DEP_CHECK=1 to package without the check.")
|
||||
exit(guard_result.returncode)
|
||||
|
||||
version = get_version()
|
||||
print(f"Building {ARCH_SUFFIX} packages for {METADATA['name']} version {version}")
|
||||
|
||||
|
||||
@@ -0,0 +1,282 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Check that every library the built bundle needs from the host is declared.
|
||||
|
||||
bundle-libs.sh ships most of what the app links, but deliberately leaves the
|
||||
graphics stack, the audio stack and the C runtime to the host: those are coupled
|
||||
to the running compositor, the GPU driver and the kernel, so a bundled copy is
|
||||
worse than useless. Everything it leaves behind has to be named as a package
|
||||
dependency instead. Miss one and the loader fails before main() runs - which is
|
||||
invisible to any amount of reading package metadata, and invisible to CI unless
|
||||
something derives the requirement from the artifact.
|
||||
|
||||
That matters more than it used to. libmpv now travels inside the package instead
|
||||
of being depended on, because the plane needs the pinned Wayland-enabled build.
|
||||
The host `mpv` dependency that went away had been quietly providing libva, libdrm
|
||||
and their friends transitively.
|
||||
|
||||
The deb answer is derived, not written down: dpkg owns the authoritative mapping
|
||||
from a file to the package providing it, so a library that appears in the bundle
|
||||
cannot slip past by being absent from a list someone forgot to update. rpm and
|
||||
pacman names cannot be resolved on a Debian-family runner, so those stay
|
||||
recorded below; their job is to fail loudly when a new library lands and nobody
|
||||
thought about Fedora and Arch.
|
||||
|
||||
Coverage is checked in one direction only. A declared dependency that nothing
|
||||
links is harmless - it installs a package the user probably has. A linked
|
||||
library that nothing declares is a broken install, so that is the direction
|
||||
worth failing on.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import importlib.util
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# The checkout this script lives in. Derived from its own location rather than
|
||||
# the working directory, so build-packages.py is still found when CI invokes
|
||||
# the check from the build tree; --root stays as the override.
|
||||
PROJECT_ROOT = Path(__file__).resolve().parents[2]
|
||||
|
||||
# Sonames provided by the base system on every distro. No package declares a
|
||||
# dependency on the C runtime or the dynamic loader; they are the floor that
|
||||
# has to exist for dpkg itself to run.
|
||||
BASELINE_PREFIXES = (
|
||||
"linux-vdso.so",
|
||||
"ld-linux",
|
||||
"libc.so",
|
||||
"libm.so",
|
||||
"libpthread.so",
|
||||
"libdl.so",
|
||||
"librt.so",
|
||||
"libmvec.so",
|
||||
"libresolv.so",
|
||||
"libnss_",
|
||||
)
|
||||
|
||||
# Where the loader actually looks for a soname, relative to the bundle root:
|
||||
# linux/CMakeLists.txt gives plezy RPATH $ORIGIN/lib and plezy.sh exports
|
||||
# $INSTALL_DIR/lib. Nothing in the bundle carries a RUNPATH of its own, so this
|
||||
# one directory is the whole search path for every object in it.
|
||||
LOADER_DIRECTORIES = ("lib",)
|
||||
|
||||
# The non-deb package that provides each host soname. Fedora and Arch cannot be
|
||||
# queried from here, so this is the one place where those names are asserted
|
||||
# rather than derived - and an unlisted soname is an error, so adding a library
|
||||
# forces the decision instead of silently shipping a package that cannot start.
|
||||
OTHER_DISTROS = {
|
||||
"libEGL.so.1": {"rpm": "libglvnd-egl", "pacman": "libglvnd"},
|
||||
"libGL.so.1": {"rpm": "libglvnd-glx", "pacman": "libglvnd"},
|
||||
# Fedora splits glvnd by API: -egl, -gles, -glx and -opengl sit over the
|
||||
# base libglvnd that carries libGLdispatch, so GLES is not the EGL package.
|
||||
"libGLESv2.so.2": {"rpm": "libglvnd-gles", "pacman": "libglvnd"},
|
||||
"libGLX.so.0": {"rpm": "libglvnd-glx", "pacman": "libglvnd"},
|
||||
"libGLdispatch.so.0": {"rpm": "libglvnd", "pacman": "libglvnd"},
|
||||
"libOpenGL.so.0": {"rpm": "libglvnd-opengl", "pacman": "libglvnd"},
|
||||
"libX11-xcb.so.1": {"rpm": "libX11-xcb", "pacman": "libx11"},
|
||||
"libX11.so.6": {"rpm": "libX11", "pacman": "libx11"},
|
||||
"libXext.so.6": {"rpm": "libXext", "pacman": "libxext"},
|
||||
"libasound.so.2": {"rpm": "alsa-lib", "pacman": "alsa-lib"},
|
||||
"libdrm.so.2": {"rpm": "libdrm", "pacman": "libdrm"},
|
||||
"libepoxy.so.0": {"rpm": "libepoxy", "pacman": "libepoxy"},
|
||||
"libgbm.so.1": {"rpm": "mesa-libgbm", "pacman": "mesa"},
|
||||
"libva-drm.so.2": {"rpm": "libva", "pacman": "libva"},
|
||||
"libva-wayland.so.2": {"rpm": "libva", "pacman": "libva"},
|
||||
"libva-x11.so.2": {"rpm": "libva", "pacman": "libva"},
|
||||
"libva.so.2": {"rpm": "libva", "pacman": "libva"},
|
||||
"libvdpau.so.1": {"rpm": "libvdpau", "pacman": "libvdpau"},
|
||||
"libvulkan.so.1": {"rpm": "vulkan-loader", "pacman": "vulkan-icd-loader"},
|
||||
"libwayland-client.so.0": {"rpm": "libwayland-client", "pacman": "wayland"},
|
||||
"libwayland-cursor.so.0": {"rpm": "libwayland-cursor", "pacman": "wayland"},
|
||||
"libwayland-egl.so.1": {"rpm": "libwayland-egl", "pacman": "wayland"},
|
||||
"libwayland-server.so.0": {"rpm": "libwayland-server", "pacman": "wayland"},
|
||||
"libxcb-dri3.so.0": {"rpm": "libxcb", "pacman": "libxcb"},
|
||||
"libxcb-present.so.0": {"rpm": "libxcb", "pacman": "libxcb"},
|
||||
"libxcb-randr.so.0": {"rpm": "libxcb", "pacman": "libxcb"},
|
||||
"libxcb-render.so.0": {"rpm": "libxcb", "pacman": "libxcb"},
|
||||
"libxcb-shm.so.0": {"rpm": "libxcb", "pacman": "libxcb"},
|
||||
"libxcb-sync.so.1": {"rpm": "libxcb", "pacman": "libxcb"},
|
||||
"libxcb-xfixes.so.0": {"rpm": "libxcb", "pacman": "libxcb"},
|
||||
"libxcb.so.1": {"rpm": "libxcb", "pacman": "libxcb"},
|
||||
}
|
||||
|
||||
|
||||
# ldd and dpkg-query are the entire evidence base for this check, so they have
|
||||
# to be substitutable: PLEZY_HOST_TOOLS names a directory searched ahead of
|
||||
# PATH, which is what lets scripts/test_check_bundle_host_deps.py drive real
|
||||
# runs of this script against a synthetic bundle on any machine.
|
||||
HOST_TOOLS = os.environ.get("PLEZY_HOST_TOOLS")
|
||||
|
||||
|
||||
def locate(program: str) -> str | None:
|
||||
"""Where `program` resolves, PLEZY_HOST_TOOLS ahead of PATH."""
|
||||
override = shutil.which(program, path=HOST_TOOLS) if HOST_TOOLS else None
|
||||
return override or shutil.which(program)
|
||||
|
||||
|
||||
def run(*command: str) -> subprocess.CompletedProcess:
|
||||
program, *arguments = command
|
||||
return subprocess.run([locate(program) or program, *arguments], capture_output=True, text=True)
|
||||
|
||||
|
||||
def load_distros(root: Path) -> dict:
|
||||
"""Read DISTROS out of build-packages.py rather than duplicating it."""
|
||||
spec = importlib.util.spec_from_file_location("build_packages", root / "linux/packaging/build-packages.py")
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
return module.DISTROS
|
||||
|
||||
|
||||
def host_libraries(bundle: Path) -> tuple[dict[str, str], list[str]]:
|
||||
"""soname -> host path for what the bundle resolves outside itself.
|
||||
|
||||
Also returns problems worth failing on. An unresolved entry is the loudest
|
||||
possible version of the failure this whole check exists to prevent - the
|
||||
loader finding nothing at all - so it must never be quietly skipped for not
|
||||
looking like a path.
|
||||
"""
|
||||
# Only what the loader will search counts as shipped. The dlopen'd
|
||||
# gdk-pixbuf loaders and GIO modules below lib/ are opened by explicit path
|
||||
# and never resolve a soname for anyone, so a nested module whose filename
|
||||
# happens to equal one the executable really takes from the host would
|
||||
# otherwise make that host library look bundled and pass the check
|
||||
# vacuously.
|
||||
shipped = {path.name for directory in LOADER_DIRECTORIES for path in (bundle / directory).glob("*.so*")}
|
||||
# rglob for the objects to inspect, though: those same modules pull in host
|
||||
# libraries nothing else in the bundle links, and scanning only lib/*.so*
|
||||
# left their dependencies out of ldd entirely.
|
||||
binaries = [bundle / "plezy", *sorted(bundle.rglob("*.so*"))]
|
||||
needed: dict[str, str] = {}
|
||||
problems: list[str] = []
|
||||
for binary in binaries:
|
||||
result = run("ldd", str(binary))
|
||||
# ldd exits non-zero for a file it cannot read as an object. Statically
|
||||
# linked objects are the benign case and say so on stdout.
|
||||
if result.returncode != 0 and "not a dynamic executable" not in (result.stdout + result.stderr):
|
||||
problems.append(f"ldd could not read {binary}: {result.stderr.strip() or 'no diagnostic'}")
|
||||
continue
|
||||
for line in result.stdout.splitlines():
|
||||
soname, separator, remainder = line.partition("=>")
|
||||
soname = soname.strip()
|
||||
remainder = remainder.strip()
|
||||
if not separator or not soname:
|
||||
continue
|
||||
if remainder.startswith("not found"):
|
||||
# Unless the bundle ships it. Bundled objects carry no RUNPATH of
|
||||
# their own - bundle-libs.sh copies and strips, it does not
|
||||
# patchelf - so running ldd on one in isolation resolves its
|
||||
# DT_NEEDED entries against the system cache only, and a library
|
||||
# that exists nowhere but the bundle reads as missing. At runtime
|
||||
# the executable's own $ORIGIN/lib finds it, which is what makes
|
||||
# shipping it sufficient. libshaderc_shared is exactly this case:
|
||||
# no distro package installs it, which is why both workflows copy
|
||||
# it in by hand.
|
||||
if soname in shipped:
|
||||
continue
|
||||
problems.append(f"{binary.name} needs {soname}, which resolves to nothing on this machine")
|
||||
continue
|
||||
path = remainder.split(" (")[0]
|
||||
if not path.startswith("/"):
|
||||
continue
|
||||
# A library that ships beside the binary needs nothing declared,
|
||||
# even when the loader happened to resolve this copy from the host.
|
||||
if soname in shipped or soname.startswith(BASELINE_PREFIXES):
|
||||
continue
|
||||
needed[soname] = path
|
||||
return needed, problems
|
||||
|
||||
|
||||
def deb_owner(path: str) -> str:
|
||||
"""The deb package owning a file, following symlinks when dpkg needs it."""
|
||||
for candidate in (path, os.path.realpath(path)):
|
||||
owner = run("dpkg-query", "-S", candidate).stdout.partition(":")[0].strip()
|
||||
if owner:
|
||||
# A diverted or multi-arch answer can carry an architecture suffix.
|
||||
return owner.split(",")[0].split(":")[0]
|
||||
return ""
|
||||
|
||||
|
||||
def deb_names(package: str) -> set[str]:
|
||||
"""The package's own name plus everything it Provides."""
|
||||
provides = run("dpkg-query", "-W", "-f", "${Provides}", package).stdout
|
||||
names = {package}
|
||||
for entry in provides.split(","):
|
||||
name = entry.split("(")[0].strip()
|
||||
if name:
|
||||
names.add(name)
|
||||
return names
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("bundle", type=Path, help="the built Flutter bundle directory")
|
||||
parser.add_argument("--root", type=Path, default=PROJECT_ROOT, help="repository root")
|
||||
arguments = parser.parse_args()
|
||||
|
||||
if not (arguments.bundle / "plezy").exists():
|
||||
print(f"::error::{arguments.bundle}/plezy does not exist - nothing to check", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
# ldd and dpkg-query are the entire evidence base. Without them this walk
|
||||
# finds nothing and would otherwise report a bundle with no host libraries
|
||||
# at all, which is the one failure mode a guard must never have. dpkg-query
|
||||
# also pins the check to a Debian-family host: Fedora and Arch cannot answer
|
||||
# which package owns a file, so the check genuinely cannot run there and says
|
||||
# so instead of passing. build-packages.py carries the deliberate opt-out for
|
||||
# anyone who has to package from such a host anyway.
|
||||
absent = [tool for tool in ("ldd", "dpkg-query") if locate(tool) is None]
|
||||
if absent:
|
||||
print(
|
||||
f"::error::{' and '.join(absent)} not found - this check derives deb package ownership "
|
||||
"from dpkg-query, so it only runs on a Debian or Ubuntu host",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return 1
|
||||
|
||||
distros = load_distros(arguments.root)
|
||||
# "a | b" is satisfied by either name, so compare against the flattened set.
|
||||
declared = {
|
||||
distro: {name.strip() for dependency in config["depends"] for name in dependency.split("|")}
|
||||
for distro, config in distros.items()
|
||||
}
|
||||
|
||||
needed, errors = host_libraries(arguments.bundle)
|
||||
# A real bundle always needs *something* from the host: bundle-libs.sh
|
||||
# deliberately leaves the graphics stack behind, so an empty result means the
|
||||
# walk found nothing rather than that nothing is needed - a silently
|
||||
# successful no-op is the worst outcome for a guard.
|
||||
if not needed and not errors:
|
||||
errors.append(
|
||||
f"no host libraries were found for {arguments.bundle} at all, which cannot be right - "
|
||||
"ldd produced nothing usable, so this proved nothing"
|
||||
)
|
||||
for soname, path in sorted(needed.items()):
|
||||
owner = deb_owner(path)
|
||||
if not owner:
|
||||
errors.append(f"{soname} ({path}) belongs to no deb package, so it cannot be checked or declared")
|
||||
elif not (deb_names(owner) & declared["deb"]):
|
||||
errors.append(f"{soname} comes from deb package '{owner}', which linux/packaging/build-packages.py does not declare")
|
||||
|
||||
others = OTHER_DISTROS.get(soname)
|
||||
if others is None:
|
||||
errors.append(f"{soname} has no rpm/pacman package recorded in {Path(__file__).name}")
|
||||
continue
|
||||
for distro, package in sorted(others.items()):
|
||||
if distro in declared and package not in declared[distro]:
|
||||
errors.append(f"{soname} needs '{package}' on {distro}, which build-packages.py does not declare")
|
||||
|
||||
for error in errors:
|
||||
print(f"::error::{error}", file=sys.stderr)
|
||||
if errors:
|
||||
return 1
|
||||
|
||||
print(f"every one of the {len(needed)} host libraries the bundle needs is declared on all {len(distros)} distros:")
|
||||
for soname in sorted(needed):
|
||||
print(f" {soname:<28} {deb_owner(needed[soname])}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,148 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Read the declared dependencies back out of the packages fpm just produced.
|
||||
|
||||
check-bundle-host-deps.py proves the depends lists in build-packages.py cover
|
||||
every library the staged bundle loads from the host. Nothing proved those lists
|
||||
survived fpm. A renamed flag or a dropped `--depends` produces a package that
|
||||
installs cleanly and then dies in the loader before main(), with every earlier
|
||||
check green - and only the artifact itself can show it. So this reads the
|
||||
dependency metadata out of the finished .deb, .rpm and .pkg.tar.zst and fails
|
||||
when a dependency build-packages.py declares is missing from any of them.
|
||||
|
||||
Every name comes from build-packages.py, so adding a library there is verified
|
||||
here without a second edit. The release job and the package smoke build both run
|
||||
this against the packages they built, which is why the assertions live here
|
||||
rather than inline in two workflows that drift apart.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import importlib.util
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# The checkout this script lives in, derived from its own location so the
|
||||
# packages may be anywhere; --root stays as the override.
|
||||
PROJECT_ROOT = Path(__file__).resolve().parents[2]
|
||||
|
||||
|
||||
class Unreadable(Exception):
|
||||
"""A package whose dependency metadata could not be read at all.
|
||||
|
||||
Kept distinct from "the metadata says nothing depends on X": a missing tool
|
||||
or an unreadable archive proves nothing, and must never be reported as a
|
||||
package that simply declared everything.
|
||||
"""
|
||||
|
||||
|
||||
def run(*command: str) -> str:
|
||||
program, *arguments = command
|
||||
# Resolved once and forked by full path, so what was probed for existence is
|
||||
# exactly what ran.
|
||||
resolved = shutil.which(program)
|
||||
if resolved is None:
|
||||
raise Unreadable(f"{program} is not installed, so this package's metadata cannot be read")
|
||||
result = subprocess.run([resolved, *arguments], capture_output=True, text=True)
|
||||
if result.returncode != 0:
|
||||
raise Unreadable(f"{program} failed: {result.stderr.strip() or result.stdout.strip() or 'no diagnostic'}")
|
||||
return result.stdout
|
||||
|
||||
|
||||
def load_packaging(root: Path):
|
||||
"""build-packages.py itself, so the expected names are never re-typed here."""
|
||||
spec = importlib.util.spec_from_file_location("build_packages", root / "linux/packaging/build-packages.py")
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
def read_metadata(distro: str, package: Path) -> str:
|
||||
"""The dependency text the built package carries, in its own format."""
|
||||
if distro == "deb":
|
||||
return run("dpkg-deb", "-f", str(package), "Depends")
|
||||
if distro == "rpm":
|
||||
return run("rpm", "-qpR", str(package))
|
||||
if distro == "pacman":
|
||||
# A pattern, not a literal name: fpm's pacman writer may store the entry
|
||||
# as `.PKGINFO` or `./.PKGINFO`. Guard the extraction rather than the
|
||||
# filtered list, so "the archive member was not found" stays separable
|
||||
# from "fpm dropped every dependency" - the second is the regression this
|
||||
# script exists to name, and it has to reach the comparison below.
|
||||
pkginfo = run("bsdtar", "-xOf", str(package), "--include", "*.PKGINFO")
|
||||
if not pkginfo.strip():
|
||||
raise Unreadable("could not read .PKGINFO out of the pacman package")
|
||||
return "\n".join(line.removeprefix("depend = ") for line in pkginfo.splitlines() if line.startswith("depend = "))
|
||||
# An unrecognised format is an error, not a skip: a distro added to DISTROS
|
||||
# without a reader here would otherwise ship entirely unverified.
|
||||
raise Unreadable(f"no reader for the {distro} package format is recorded in {Path(__file__).name}")
|
||||
|
||||
|
||||
def declared_names(blob: str) -> set[str]:
|
||||
"""Every package name the metadata requires, without version constraints.
|
||||
|
||||
Names, not a substring search over the whole blob: `libegl1` is a substring
|
||||
of `libegl1-mesa`, so a package that declared neither used to pass on the
|
||||
strength of some unrelated longer dependency. Split on the separators all
|
||||
three formats use, then drop version constraints, rpm's soname decorations
|
||||
and deb's architecture qualifier.
|
||||
"""
|
||||
found = set()
|
||||
for token in re.split(r"[,|\s]+", blob):
|
||||
name = re.sub(r"[<>=].*$", "", token).split("(")[0].split(":")[0]
|
||||
if name:
|
||||
found.add(name)
|
||||
return found
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
parser.add_argument("directory", type=Path, help="where build-packages.py wrote the packages")
|
||||
parser.add_argument("--arch", default="x64", help="the ARCH_SUFFIX the packages were built with")
|
||||
parser.add_argument("--root", type=Path, default=PROJECT_ROOT, help="repository root")
|
||||
arguments = parser.parse_args()
|
||||
|
||||
packaging = load_packaging(arguments.root)
|
||||
errors: list[str] = []
|
||||
# An empty roster would walk no packages and still print success, which is
|
||||
# the one verdict a guard must never reach without evidence.
|
||||
if not packaging.DISTROS:
|
||||
errors.append("build-packages.py defines no package formats, so there was nothing to read back")
|
||||
|
||||
for distro, config in packaging.DISTROS.items():
|
||||
package = arguments.directory / f"{packaging.METADATA['name']}-linux-{arguments.arch}.{config['ext']}"
|
||||
if not package.is_file():
|
||||
errors.append(f"{package} was not produced")
|
||||
continue
|
||||
|
||||
try:
|
||||
metadata = read_metadata(distro, package)
|
||||
except Unreadable as failure:
|
||||
errors.append(f"{package.name}: {failure}")
|
||||
continue
|
||||
|
||||
print(f"{distro}: {' '.join(metadata.split()) or '(nothing)'}")
|
||||
# A format that declares nothing cannot be checked against the package,
|
||||
# so emptying the list would otherwise turn this guard into a no-op.
|
||||
if not config["depends"]:
|
||||
errors.append(f"build-packages.py declares no dependencies for {distro}, so this proved nothing")
|
||||
continue
|
||||
|
||||
declared = declared_names(metadata)
|
||||
for dependency in config["depends"]:
|
||||
# `libmpv2 | libmpv1` is one dependency with two acceptable names.
|
||||
if not any(name.strip() in declared for name in dependency.split("|")):
|
||||
errors.append(f"the {distro} package does not require {dependency}")
|
||||
|
||||
for error in errors:
|
||||
print(f"::error::{error}", file=sys.stderr)
|
||||
if errors:
|
||||
return 1
|
||||
|
||||
print(f"every dependency build-packages.py declares survived fpm into all {len(packaging.DISTROS)} packages")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -29,9 +29,10 @@
|
||||
"kind": "git",
|
||||
"version": "7.351.0",
|
||||
"url": "https://code.videolan.org/videolan/libplacebo.git",
|
||||
"mirror": "https://github.com/haasn/libplacebo.git",
|
||||
"ref": "v7.351.0",
|
||||
"commit": "3188549fba13bbdf3a5a98de2a38c2e71f04e21e",
|
||||
"provenance": "Official VideoLAN GitLab tag record v7.351.0 contains a PGP-signed release message and dereferences to this root commit; its gitlinks pin recursive submodules."
|
||||
"provenance": "Official VideoLAN GitLab tag record v7.351.0 contains a PGP-signed release message and dereferences to this root commit; its gitlinks pin recursive submodules. The mirror is upstream's own author repository (haasn/libplacebo, not a fork), whose annotated v7.351.0 tag object 98c1416e95b21cd767b84ac9ee430ccadf263ff6 dereferences to that same commit. It is consulted only after the primary fails, and the commit pin is verified identically either way, so it cannot introduce a different tree."
|
||||
},
|
||||
"mpv": {
|
||||
"kind": "archive",
|
||||
|
||||
@@ -0,0 +1,291 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Guard the Linux package dependency lists against what the runner links.
|
||||
|
||||
linux/packaging/bundle-libs.sh deliberately refuses to bundle the display- and
|
||||
driver-coupled libraries (libEGL, libwayland-*, libGL, libdrm ...): they must
|
||||
come from the host or the app will not talk to the compositor it is running
|
||||
under. That makes them the package manager's problem, and the depends lists in
|
||||
linux/packaging/build-packages.py are maintained by hand.
|
||||
|
||||
Nothing connected the two. Adding a pkg-config link to the runner produced a
|
||||
binary with an undeclared shared-library dependency, and the failure surfaces
|
||||
only on a user's machine at exec time - a class of bug no compile or unit test
|
||||
can reach. This walks the runner's own link line instead:
|
||||
|
||||
target_link_libraries(${BINARY_NAME} PRIVATE PkgConfig::WAYLAND_EGL)
|
||||
-> pkg_check_modules(WAYLAND_EGL REQUIRED IMPORTED_TARGET wayland-egl)
|
||||
-> RUNTIME_PACKAGES["wayland-egl"] -> libwayland-egl1 / libwayland-egl / wayland
|
||||
|
||||
and requires every distro to declare it. A new pkg-config module fails here
|
||||
until its runtime package names are named for all three.
|
||||
|
||||
What it walks is exactly CMAKE_FILES, the three CMakeLists.txt this checkout
|
||||
owns - and nothing else. The Flutter plugins link into the same binary from
|
||||
linux/flutter/generated_plugins.cmake, whose add_subdirectory() targets live
|
||||
under flutter/ephemeral/.plugin_symlinks/, a directory that only exists after
|
||||
`flutter pub get`. A plugin's own pkg_check_modules is therefore unreadable at
|
||||
pull-request time, and a plugin that starts linking a new host library passes
|
||||
here. linux/packaging/check-bundle-host-deps.py is what covers that: it runs
|
||||
ldd over the built bundle, where every link edge is finally real.
|
||||
"""
|
||||
|
||||
from pathlib import Path
|
||||
import ast
|
||||
import re
|
||||
import sys
|
||||
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
if len(sys.argv) > 2:
|
||||
raise SystemExit(f"Usage: {Path(sys.argv[0]).name} [linux-dir]")
|
||||
LINUX = Path(sys.argv[1]).resolve() if len(sys.argv) == 2 else ROOT / "linux"
|
||||
|
||||
RUNNER_CMAKE = LINUX / "runner/CMakeLists.txt"
|
||||
PACKAGES_PY = LINUX / "packaging/build-packages.py"
|
||||
BUNDLE_SH = LINUX / "packaging/bundle-libs.sh"
|
||||
# pkg_check_modules for targets the runner links may live in any of these.
|
||||
CMAKE_FILES = (RUNNER_CMAKE, LINUX / "CMakeLists.txt", LINUX / "flutter/CMakeLists.txt")
|
||||
|
||||
# pkg-config modules whose library ships *inside* the package instead of being
|
||||
# depended on. libmpv is pinned and Wayland-enabled because the video plane needs
|
||||
# it to be; a distro libmpv silently drops hwdec to vaapi-copy. Bundling it means
|
||||
# there is deliberately no runtime dependency to find, so the walk must not
|
||||
# demand one - but the libraries it links that bundle-libs.sh excludes still have
|
||||
# to be declared, which the packaging job re-derives from the built bundle.
|
||||
BUNDLED_MODULES = {"mpv"}
|
||||
|
||||
# pkg-config module -> the package that ships its runtime library, per distro.
|
||||
# Only modules the runner actually links are consulted, so an unused entry here
|
||||
# is harmless; a missing one is an error.
|
||||
RUNTIME_PACKAGES = {
|
||||
"gtk+-3.0": {"deb": "libgtk-3-0", "rpm": "gtk3", "pacman": "gtk3"},
|
||||
"epoxy": {"deb": "libepoxy0", "rpm": "libepoxy", "pacman": "libepoxy"},
|
||||
# Reached through the `flutter` INTERFACE target rather than named by the
|
||||
# runner, which is why the graph has to cross file boundaries to see them.
|
||||
"glib-2.0": {"deb": "libglib2.0-0", "rpm": "glib2", "pacman": "glib2"},
|
||||
"gio-2.0": {"deb": "libglib2.0-0", "rpm": "glib2", "pacman": "glib2"},
|
||||
"wayland-client": {
|
||||
"deb": "libwayland-client0",
|
||||
"rpm": "libwayland-client",
|
||||
# Arch ships every libwayland-* in the one `wayland` package.
|
||||
"pacman": "wayland",
|
||||
},
|
||||
"wayland-egl": {
|
||||
"deb": "libwayland-egl1",
|
||||
"rpm": "libwayland-egl",
|
||||
"pacman": "wayland",
|
||||
},
|
||||
# libglvnd is the vendor-neutral dispatch that provides libEGL.so.1.
|
||||
"egl": {"deb": "libegl1", "rpm": "libglvnd-egl", "pacman": "libglvnd"},
|
||||
}
|
||||
|
||||
errors: list[str] = []
|
||||
|
||||
|
||||
def require(condition: bool, message: str) -> None:
|
||||
if not condition:
|
||||
errors.append(message)
|
||||
|
||||
|
||||
def read(path: Path) -> str:
|
||||
try:
|
||||
return path.read_text(encoding="utf-8")
|
||||
except OSError as error:
|
||||
errors.append(f"{path}: cannot read: {error}")
|
||||
return ""
|
||||
|
||||
|
||||
# Keywords that carry no target name.
|
||||
LINK_KEYWORDS = {"PRIVATE", "PUBLIC", "INTERFACE", "optimized", "debug", "general"}
|
||||
# Options CMake accepts between IMPORTED_TARGET and the module names, in any order.
|
||||
PKG_OPTIONS = ("REQUIRED", "QUIET", "GLOBAL", "NO_CMAKE_PATH", "NO_CMAKE_ENVIRONMENT_PATH")
|
||||
|
||||
|
||||
def strip_comments(text: str) -> str:
|
||||
"""A `#` comment containing `)` would otherwise truncate a call body.
|
||||
|
||||
That is the fail-open direction: every target after the comment vanishes and
|
||||
the guard still exits 0, which is the whole bug class it exists to catch.
|
||||
"""
|
||||
return re.sub(r"#[^\n]*", "", text)
|
||||
|
||||
|
||||
def link_token(raw: str) -> str:
|
||||
"""`$<LINK_ONLY:PkgConfig::X>` and `"PkgConfig::X"` both name PkgConfig::X."""
|
||||
return re.sub(r"^\$<[^:]*:", "", raw.strip('"')).rstrip(">")
|
||||
|
||||
|
||||
def link_graph(text: str) -> dict[str, list[str]]:
|
||||
"""target -> everything target_link_libraries() gives it, in order."""
|
||||
graph: dict[str, list[str]] = {}
|
||||
for match in re.finditer(r"target_link_libraries\(\s*([^\s)]+)\s*([^)]*)\)", strip_comments(text)):
|
||||
name = match.group(1).replace("${BINARY_NAME}", "BINARY")
|
||||
tokens = [link_token(t) for t in match.group(2).split()]
|
||||
graph.setdefault(name, []).extend(t for t in tokens if t not in LINK_KEYWORDS)
|
||||
return graph
|
||||
|
||||
|
||||
def linked_pkgconfig_targets(text: str) -> set[str]:
|
||||
"""Every PkgConfig:: target that reaches the runner's link line.
|
||||
|
||||
A library hands its dependencies to whatever links it - CMake puts even
|
||||
PRIVATE ones of a static library on the consumer's link line, and an
|
||||
INTERFACE target exists only to propagate them - so an internal target has to
|
||||
be followed rather than treated as a leaf. `wayland_protocols PUBLIC
|
||||
PkgConfig::WAYLAND_CLIENT` and `flutter INTERFACE PkgConfig::GTK` are both
|
||||
invisible otherwise, the latter across a file boundary.
|
||||
"""
|
||||
graph = link_graph(text)
|
||||
targets: set[str] = set()
|
||||
seen: set[str] = set()
|
||||
queue = ["BINARY"]
|
||||
while queue:
|
||||
current = queue.pop()
|
||||
if current in seen:
|
||||
continue
|
||||
seen.add(current)
|
||||
for token in graph.get(current, []):
|
||||
if token.startswith("PkgConfig::"):
|
||||
targets.add(token[len("PkgConfig::") :])
|
||||
elif token in graph:
|
||||
queue.append(token)
|
||||
return targets
|
||||
|
||||
|
||||
def pkgconfig_modules() -> dict[str, list[str]]:
|
||||
"""CMake variable prefix -> every pkg-config module the call names.
|
||||
|
||||
A single call may name several - `pkg_check_modules(X REQUIRED
|
||||
IMPORTED_TARGET a b c)` makes one PkgConfig::X that links all three - and
|
||||
taking only the first is the fail-open direction: the extra libraries reach
|
||||
the binary while the guard reports a clean run. wayland-cursor and
|
||||
xkbcommon are the natural companions of a subsurface and grouping them into
|
||||
the existing call is the natural way to add them, so this is the next edit
|
||||
to this file rather than a hypothetical.
|
||||
"""
|
||||
modules: dict[str, list[str]] = {}
|
||||
options = "|".join(PKG_OPTIONS)
|
||||
for path in CMAKE_FILES:
|
||||
for match in re.finditer(
|
||||
# The options may precede the module names, so skip any run of them
|
||||
# rather than taking the first token and reporting `REQUIRED` as a
|
||||
# package nobody ships. The tail is then every remaining token up to
|
||||
# the closing paren.
|
||||
r"pkg_check_modules\(\s*(\w+)\b[^)]*?IMPORTED_TARGET\s+((?:(?:" + options + r")\s+)*[^)]*)\)",
|
||||
strip_comments(read(path)),
|
||||
):
|
||||
# A moduleSpec is `<name>` or `<name><op><version>`, so the version
|
||||
# constraint has to come off before the name is looked up - otherwise
|
||||
# a perfectly legal `mpv>=0.40` is reported as a package nobody
|
||||
# ships, and the message sends whoever hits it off to invent a
|
||||
# RUNTIME_PACKAGES entry for it. Pinning that minimum is a plausible
|
||||
# next edit here: target-colorspace-hint=auto needs mpv 0.40.
|
||||
names = [
|
||||
re.split(r"[<>=!]", t, maxsplit=1)[0] for t in match.group(2).split() if t not in PKG_OPTIONS
|
||||
]
|
||||
names = [n for n in names if n]
|
||||
if names:
|
||||
modules.setdefault(match.group(1), names)
|
||||
return modules
|
||||
|
||||
|
||||
def declared_depends() -> dict[str, list[str]]:
|
||||
"""distro -> depends list, read from the DISTROS literal by AST."""
|
||||
tree = ast.parse(read(PACKAGES_PY), filename=str(PACKAGES_PY))
|
||||
for node in tree.body:
|
||||
if not isinstance(node, ast.Assign):
|
||||
continue
|
||||
if not any(isinstance(t, ast.Name) and t.id == "DISTROS" for t in node.targets):
|
||||
continue
|
||||
table = ast.literal_eval(node.value)
|
||||
return {name: list(config.get("depends", [])) for name, config in table.items()}
|
||||
errors.append(f"{PACKAGES_PY}: no DISTROS assignment to read the depends lists from")
|
||||
return {}
|
||||
|
||||
|
||||
# Every file, not just the runner's: `flutter` is defined in flutter/CMakeLists.txt
|
||||
# and propagates GTK, GLIB and GIO to whatever links it, so a graph built from one
|
||||
# file treats it as a leaf and never sees them. A member that moved or was renamed
|
||||
# is fatal rather than a smaller walk: its modules drop out of the graph, a
|
||||
# declaration deleted alongside it goes unreported, and every check below then
|
||||
# passes over a tree nobody actually looked at.
|
||||
absent = [path for path in CMAKE_FILES if not path.is_file()]
|
||||
if absent:
|
||||
for path in absent:
|
||||
print(
|
||||
f"ERROR: {path}: expected CMake input is missing, so the dependency walk "
|
||||
"would silently cover less than it claims",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
cmake_text = "\n".join(read(path) for path in CMAKE_FILES)
|
||||
modules = pkgconfig_modules()
|
||||
depends = declared_depends()
|
||||
|
||||
require(bool(depends), "no distro depends lists were found, so nothing was checked")
|
||||
|
||||
# The exclusion list is what makes declaring these mandatory rather than
|
||||
# optional. If bundling ever starts covering them, this guard is the wrong shape.
|
||||
bundle = read(BUNDLE_SH)
|
||||
for pattern in (r"libEGL\.so", r"libwayland.*\.so"):
|
||||
require(
|
||||
pattern in bundle,
|
||||
f"bundle-libs.sh no longer excludes {pattern}: if those are bundled now, "
|
||||
"the depends entries this guard demands may be wrong",
|
||||
)
|
||||
|
||||
linked = linked_pkgconfig_targets(cmake_text)
|
||||
require(
|
||||
bool(linked),
|
||||
"found no PkgConfig:: link reaching ${BINARY_NAME}; the link-line parse is broken, not the build",
|
||||
)
|
||||
|
||||
checked_modules = 0
|
||||
for target in sorted(linked):
|
||||
target_modules = modules.get(target)
|
||||
if not target_modules:
|
||||
errors.append(
|
||||
f"PkgConfig::{target} is linked into the runner but no pkg_check_modules "
|
||||
f"declares it in {', '.join(p.name for p in CMAKE_FILES)}"
|
||||
)
|
||||
continue
|
||||
for module in target_modules:
|
||||
checked_modules += 1
|
||||
if module in BUNDLED_MODULES:
|
||||
# Shipped inside the package, so there is no dependency to find. Still
|
||||
# counted, so the summary keeps naming everything the walk reached and
|
||||
# a module going missing is a drop rather than a silent skip.
|
||||
continue
|
||||
packages = RUNTIME_PACKAGES.get(module)
|
||||
if packages is None:
|
||||
errors.append(
|
||||
f"pkg-config module '{module}' (PkgConfig::{target}) is linked into the runner "
|
||||
f"but has no entry in RUNTIME_PACKAGES: name the package that ships its "
|
||||
f"runtime library on each distro, then declare it in {PACKAGES_PY.name}"
|
||||
)
|
||||
continue
|
||||
for distro, declared in sorted(depends.items()):
|
||||
package = packages.get(distro)
|
||||
if package is None:
|
||||
errors.append(
|
||||
f"RUNTIME_PACKAGES['{module}'] has no '{distro}' package name, so the "
|
||||
f"{distro} package cannot declare a library the runner links"
|
||||
)
|
||||
continue
|
||||
require(
|
||||
package in declared,
|
||||
f"the runner links {module} but the {distro} package does not depend on "
|
||||
f"'{package}'; bundle-libs.sh will not bundle it, so an installed package "
|
||||
f"can fail to start",
|
||||
)
|
||||
|
||||
if errors:
|
||||
for error in errors:
|
||||
print(f"ERROR: {error}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
print(
|
||||
f"linux/runner CMake dependency checks passed ({len(linked)} pkg-config links, {checked_modules} modules); "
|
||||
"Flutter plugin links are out of reach here - check-bundle-host-deps.py covers those from the built bundle"
|
||||
)
|
||||
@@ -5,9 +5,11 @@
|
||||
# and script guards" step in .github/workflows/ci.yml and section 4 of
|
||||
# scripts/ci_checks.sh. The checkers are named explicitly because a few of them
|
||||
# belong to other jobs (check_bun_audit.py needs Bun, check_codegen.py runs via
|
||||
# codegen.sh), but their regression tests are discovered by glob so a newly
|
||||
# added scripts/test_*.py is picked up automatically instead of having to be
|
||||
# remembered in two places.
|
||||
# codegen.sh, and linux/packaging/check-bundle-host-deps.py and
|
||||
# check-package-deps.py need a built bundle and built packages, so both run in
|
||||
# the linux-packages job and again in the release build), but their regression
|
||||
# tests are discovered by glob so a newly added scripts/test_*.py is picked up
|
||||
# automatically instead of having to be remembered in two places.
|
||||
set -euo pipefail
|
||||
shopt -s nullglob
|
||||
|
||||
@@ -24,6 +26,7 @@ for checker in \
|
||||
scripts/check_workflow_action_pins.py \
|
||||
scripts/check_container_image_pins.py \
|
||||
scripts/check_update_packages_workflow.py \
|
||||
scripts/check_linux_package_deps.py \
|
||||
scripts/check_windows_installer.py \
|
||||
scripts/check_windows_msix.py; do
|
||||
python3 "$checker"
|
||||
|
||||
@@ -0,0 +1,430 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Behavior tests for the bundle host dependency guard.
|
||||
|
||||
The nested-module case reproduces the bug that motivated them: bundle-libs.sh
|
||||
installs the dlopen'd gdk-pixbuf loaders and GIO modules into lib/
|
||||
subdirectories, and the guard scanned lib/*.so* only - so those modules counted
|
||||
as bundled while their own host dependencies never reached ldd, and the check
|
||||
reported success over exactly the undeclared library it exists to catch.
|
||||
|
||||
ldd and dpkg-query only answer on a Debian-family machine, so the checker
|
||||
resolves both through PLEZY_HOST_TOOLS and the fixtures here install stubs
|
||||
there. That keeps "a bundle that needs libpng16" a fixture rather than a
|
||||
machine, and keeps the tests exercising the real script end to end.
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
CHECKER = ROOT / "linux/packaging/check-bundle-host-deps.py"
|
||||
BUILD_PACKAGES = ROOT / "linux/packaging/build-packages.py"
|
||||
|
||||
# A host library whose deb owner and rpm/pacman names build-packages.py all
|
||||
# declare, so its presence alone never fails a fixture.
|
||||
DECLARED_SONAME = "libepoxy.so.0"
|
||||
DECLARED_PATH = "/usr/lib/x86_64-linux-gnu/libepoxy.so.0"
|
||||
DECLARED_OWNER = "libepoxy0"
|
||||
|
||||
# One nothing declares. Chosen from the gdk-pixbuf loaders' real dependencies,
|
||||
# which is how this class of miss actually reaches a user.
|
||||
UNDECLARED_SONAME = "libpng16.so.16"
|
||||
UNDECLARED_PATH = "/usr/lib/x86_64-linux-gnu/libpng16.so.16"
|
||||
UNDECLARED_OWNER = "libpng16-16"
|
||||
|
||||
PIXBUF_LOADER = "lib/gdk-pixbuf-2.0/2.10.0/loaders/libpixbufloader-png.so"
|
||||
|
||||
# Both stubs answer from the JSON the fixture writes. Keeping them as scripts
|
||||
# rather than mocks means the checker really forks, parses real ldd-shaped
|
||||
# output and really fails on the exit status it would see in CI.
|
||||
LDD_STUB = r"""import json, os, sys
|
||||
|
||||
answers = json.loads(open(os.environ["PLEZY_TEST_ANSWERS"], encoding="utf-8").read())
|
||||
for line in answers["ldd"].get(os.path.basename(sys.argv[1]), []):
|
||||
print("\t" + line)
|
||||
"""
|
||||
|
||||
DPKG_QUERY_STUB = r"""import json, os, sys
|
||||
|
||||
answers = json.loads(open(os.environ["PLEZY_TEST_ANSWERS"], encoding="utf-8").read())
|
||||
if sys.argv[1] == "-S":
|
||||
owner = answers["owners"].get(sys.argv[2], "")
|
||||
if not owner:
|
||||
sys.stderr.write("dpkg-query: no path found matching pattern %s\n" % sys.argv[2])
|
||||
sys.exit(1)
|
||||
print("%s: %s" % (owner, sys.argv[2]))
|
||||
else:
|
||||
# dpkg-query -W -f ${Provides} <package>
|
||||
sys.stdout.write(answers["provides"].get(sys.argv[-1], ""))
|
||||
"""
|
||||
|
||||
|
||||
def install_stub(scripts: Path, tools: Path, name: str, source: str) -> None:
|
||||
"""Put a fake `name` where the checker's PLEZY_HOST_TOOLS lookup will find it.
|
||||
|
||||
The stub itself is Python; the file the checker spawns has to be something
|
||||
the platform can execute directly, so it gets a wrapper. The two live in
|
||||
different directories because a Windows PATHEXT search would otherwise be
|
||||
free to pick the .py over the .bat.
|
||||
"""
|
||||
stub = scripts / f"{name}.py"
|
||||
stub.write_text(source, encoding="utf-8")
|
||||
if os.name == "nt":
|
||||
(tools / f"{name}.bat").write_text(f'@echo off\r\n"{sys.executable}" "{stub}" %*\r\n', encoding="utf-8")
|
||||
return
|
||||
wrapper = tools / name
|
||||
wrapper.write_text(f'#!/bin/sh\nexec "{sys.executable}" "{stub}" "$@"\n', encoding="utf-8")
|
||||
wrapper.chmod(0o755)
|
||||
|
||||
|
||||
def stage_bundle(
|
||||
staging: Path,
|
||||
ldd: dict[str, list[str]],
|
||||
owners: dict[str, str],
|
||||
shipped: tuple[str, ...] = (),
|
||||
install: tuple[str, ...] = ("ldd", "dpkg-query"),
|
||||
) -> tuple[Path, dict[str, str]]:
|
||||
"""A synthetic bundle plus host-tool stubs, and the environment that finds them."""
|
||||
bundle = staging / "bundle"
|
||||
bundle.mkdir()
|
||||
(bundle / "plezy").write_bytes(b"")
|
||||
for name in shipped:
|
||||
library = bundle / name
|
||||
library.parent.mkdir(parents=True, exist_ok=True)
|
||||
library.write_bytes(b"")
|
||||
|
||||
answers = staging / "answers.json"
|
||||
answers.write_text(json.dumps({"ldd": ldd, "owners": owners, "provides": {}}), encoding="utf-8")
|
||||
scripts, tools = staging / "stubs", staging / "tools"
|
||||
scripts.mkdir()
|
||||
tools.mkdir()
|
||||
for name, source in (("ldd", LDD_STUB), ("dpkg-query", DPKG_QUERY_STUB)):
|
||||
if name in install:
|
||||
install_stub(scripts, tools, name, source)
|
||||
|
||||
return bundle, {
|
||||
**os.environ,
|
||||
"PLEZY_HOST_TOOLS": str(tools),
|
||||
"PLEZY_TEST_ANSWERS": str(answers),
|
||||
}
|
||||
|
||||
|
||||
class BundleHostDepsGuardTest(unittest.TestCase):
|
||||
def _check(
|
||||
self,
|
||||
ldd: dict[str, list[str]],
|
||||
owners: dict[str, str],
|
||||
shipped: tuple[str, ...] = (),
|
||||
cwd: Path | None = None,
|
||||
install: tuple[str, ...] = ("ldd", "dpkg-query"),
|
||||
path: str | None = None,
|
||||
) -> subprocess.CompletedProcess[str]:
|
||||
"""Stage a synthetic bundle plus host-tool stubs and run the real checker."""
|
||||
with tempfile.TemporaryDirectory(prefix="plezy-bundle-deps-test-") as directory:
|
||||
bundle, env = stage_bundle(Path(directory), ldd, owners, shipped, install)
|
||||
if path is not None:
|
||||
env["PATH"] = path
|
||||
|
||||
return subprocess.run(
|
||||
[sys.executable, str(CHECKER), str(bundle)],
|
||||
cwd=cwd or ROOT,
|
||||
check=False,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
env=env,
|
||||
)
|
||||
|
||||
def test_a_bundle_whose_host_libraries_are_all_declared_passes(self) -> None:
|
||||
result = self._check(
|
||||
ldd={
|
||||
"plezy": [
|
||||
# No "=>" on this one: it must not be read as a soname.
|
||||
"linux-vdso.so.1 (0x00007ffd1b3fe000)",
|
||||
f"{DECLARED_SONAME} => {DECLARED_PATH} (0x00007f9c2c000000)",
|
||||
"libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f9c2b800000)",
|
||||
# Resolved from the host copy, but the bundle ships its own.
|
||||
"libmpv.so.2 => /usr/lib/x86_64-linux-gnu/libmpv.so.2 (0x00007f9c2b400000)",
|
||||
],
|
||||
"libgiognutls.so": [f"{DECLARED_SONAME} => {DECLARED_PATH} (0x00007f9c2c000000)"],
|
||||
},
|
||||
owners={DECLARED_PATH: DECLARED_OWNER},
|
||||
shipped=("lib/libmpv.so.2", "lib/gio/modules/libgiognutls.so"),
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertIn("every one of the 1 host libraries", result.stdout)
|
||||
|
||||
def test_an_undeclared_host_library_is_named(self) -> None:
|
||||
result = self._check(
|
||||
ldd={"plezy": [f"{UNDECLARED_SONAME} => {UNDECLARED_PATH} (0x00007f9c2c000000)"]},
|
||||
owners={UNDECLARED_PATH: UNDECLARED_OWNER},
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stdout)
|
||||
self.assertIn(UNDECLARED_SONAME, result.stderr)
|
||||
self.assertIn(f"deb package '{UNDECLARED_OWNER}'", result.stderr)
|
||||
|
||||
def test_a_library_declared_for_deb_but_not_for_rpm_is_still_rejected(self) -> None:
|
||||
"""Every other negative fixture fails on the Debian arm first.
|
||||
|
||||
Fedora and Arch names cannot be resolved on this runner, so they are the
|
||||
half most likely to be forgotten - and a package that installs on Fedora
|
||||
and then cannot start is exactly as broken as one that fails on Debian.
|
||||
libGLESv2 is the live example: it is mapped to libglvnd-gles, which is
|
||||
deliberately not declared because nothing links GLES today, so the moment
|
||||
something does the guard has to say so rather than wave it through on the
|
||||
strength of a satisfied deb dependency.
|
||||
"""
|
||||
soname = "libGLESv2.so.2"
|
||||
path = f"/usr/lib/x86_64-linux-gnu/{soname}"
|
||||
result = self._check(
|
||||
ldd={"plezy": [f"{soname} => {path} (0x00007f9c2c000000)"]},
|
||||
# Owned by a package the deb list does declare, so only the non-deb
|
||||
# half can be what rejects this.
|
||||
owners={path: "libegl1"},
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stdout)
|
||||
self.assertIn("libglvnd-gles", result.stderr)
|
||||
self.assertIn("rpm", result.stderr)
|
||||
# Proving the deb arm was satisfied, so the rejection came from the other.
|
||||
self.assertNotIn("comes from deb package", result.stderr)
|
||||
|
||||
def test_a_walk_that_finds_nothing_is_not_a_pass(self) -> None:
|
||||
"""An empty result means the walk failed, not that nothing is needed.
|
||||
|
||||
bundle-libs.sh always leaves the graphics stack to the host, so a real
|
||||
bundle cannot need zero host libraries. Reporting success here would make
|
||||
every later breakage invisible, which is the worst thing a guard can do.
|
||||
"""
|
||||
result = self._check(ldd={"plezy": []}, owners={})
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stdout)
|
||||
self.assertIn("proved nothing", result.stderr)
|
||||
|
||||
def test_a_library_reached_only_through_a_nested_module_is_still_checked(self) -> None:
|
||||
"""The gdk-pixbuf loaders sit two directories below lib/, and dlopen finds them.
|
||||
|
||||
Nothing else in the bundle links libpng, so a scan that stops at
|
||||
lib/*.so* sees a fully declared bundle and a user sees the loader fail.
|
||||
"""
|
||||
result = self._check(
|
||||
ldd={
|
||||
"plezy": [f"{DECLARED_SONAME} => {DECLARED_PATH} (0x00007f9c2c000000)"],
|
||||
"libpixbufloader-png.so": [f"{UNDECLARED_SONAME} => {UNDECLARED_PATH} (0x00007f9c2b000000)"],
|
||||
},
|
||||
owners={DECLARED_PATH: DECLARED_OWNER, UNDECLARED_PATH: UNDECLARED_OWNER},
|
||||
shipped=(PIXBUF_LOADER,),
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stdout)
|
||||
self.assertIn(UNDECLARED_SONAME, result.stderr)
|
||||
|
||||
def test_a_nested_module_cannot_satisfy_a_top_level_dependency(self) -> None:
|
||||
"""Only lib/ is on the loader path, so only lib/ can make a soname bundled.
|
||||
|
||||
plezy carries RPATH $ORIGIN/lib and plezy.sh exports $INSTALL_DIR/lib.
|
||||
The gdk-pixbuf loaders two directories below are opened by explicit
|
||||
path and resolve nothing for the executable, so a module whose basename
|
||||
happens to equal a host soname must not suppress it - otherwise the
|
||||
package under-declares and the check still passes.
|
||||
"""
|
||||
result = self._check(
|
||||
ldd={
|
||||
"plezy": [
|
||||
f"{DECLARED_SONAME} => {DECLARED_PATH} (0x00007f9c2c000000)",
|
||||
f"{UNDECLARED_SONAME} => {UNDECLARED_PATH} (0x00007f9c2b000000)",
|
||||
]
|
||||
},
|
||||
owners={DECLARED_PATH: DECLARED_OWNER, UNDECLARED_PATH: UNDECLARED_OWNER},
|
||||
shipped=(f"lib/gdk-pixbuf-2.0/2.10.0/loaders/{UNDECLARED_SONAME}",),
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stdout)
|
||||
self.assertIn(UNDECLARED_SONAME, result.stderr)
|
||||
self.assertIn(f"deb package '{UNDECLARED_OWNER}'", result.stderr)
|
||||
|
||||
def test_a_soname_the_bundle_ships_is_not_a_missing_dependency(self) -> None:
|
||||
"""ldd on a bundled library in isolation cannot see its siblings.
|
||||
|
||||
Bundled objects carry no RUNPATH - bundle-libs.sh copies and strips - so
|
||||
a library that exists nowhere but the bundle reads as `not found` when
|
||||
ldd is pointed at one of them directly. At runtime the executable's own
|
||||
$ORIGIN/lib resolves it. libshaderc_shared is the real instance: no
|
||||
distro package ships it, which is why both workflows copy it by hand, so
|
||||
faulting it here would fail the release for a library that is present.
|
||||
"""
|
||||
result = self._check(
|
||||
ldd={
|
||||
"plezy": [f"{DECLARED_SONAME} => {DECLARED_PATH} (0x00007f9c2c000000)"],
|
||||
# The bundled libmpv needs a bundled shaderc, and sees nothing.
|
||||
"libmpv.so.2": ["libshaderc_shared.so.1 => not found"],
|
||||
},
|
||||
owners={DECLARED_PATH: DECLARED_OWNER},
|
||||
shipped=("lib/libmpv.so.2", "lib/libshaderc_shared.so.1"),
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
|
||||
self.assertNotIn("resolves to nothing", result.stderr)
|
||||
|
||||
def test_a_library_that_resolves_to_nothing_fails(self) -> None:
|
||||
"""`=> not found` is the failure this check exists to prevent, already happened."""
|
||||
result = self._check(
|
||||
ldd={"libplezyextra.so.1": ["libfoo.so.1 => not found"]},
|
||||
owners={},
|
||||
shipped=("lib/libplezyextra.so.1",),
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stdout)
|
||||
self.assertIn("libfoo.so.1", result.stderr)
|
||||
self.assertIn("resolves to nothing", result.stderr)
|
||||
|
||||
def test_the_checker_reads_build_packages_from_any_working_directory(self) -> None:
|
||||
"""--root defaulted to cwd, so running the check from the build tree crashed."""
|
||||
result = self._check(
|
||||
ldd={"plezy": [f"{DECLARED_SONAME} => {DECLARED_PATH} (0x00007f9c2c000000)"]},
|
||||
owners={DECLARED_PATH: DECLARED_OWNER},
|
||||
cwd=Path(tempfile.gettempdir()),
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
|
||||
def test_a_host_without_dpkg_query_fails_instead_of_passing(self) -> None:
|
||||
"""Fedora and Arch have no dpkg-query, so the check cannot run there.
|
||||
|
||||
Every answer it gives comes from ldd and dpkg-query. Missing one, the
|
||||
walk would find nothing and report a bundle needing nothing from the
|
||||
host - so the absence has to be an error the reader can act on, never a
|
||||
skip and never a traceback.
|
||||
"""
|
||||
result = self._check(
|
||||
ldd={"plezy": [f"{DECLARED_SONAME} => {DECLARED_PATH} (0x00007f9c2c000000)"]},
|
||||
owners={DECLARED_PATH: DECLARED_OWNER},
|
||||
install=("ldd",),
|
||||
path="",
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stdout)
|
||||
self.assertIn("dpkg-query not found", result.stderr)
|
||||
self.assertIn("Debian or Ubuntu host", result.stderr)
|
||||
self.assertNotIn("Traceback", result.stderr)
|
||||
|
||||
|
||||
class BuildPackagesGuardWiringTest(unittest.TestCase):
|
||||
"""build-packages.py is the only path a maintainer packaging by hand takes.
|
||||
|
||||
CI ran the guard as its own step, so the script itself never did, and its own
|
||||
error message invited the standalone path that skipped it.
|
||||
"""
|
||||
|
||||
def test_an_undeclared_host_library_stops_packaging(self) -> None:
|
||||
with tempfile.TemporaryDirectory(prefix="plezy-packaging-test-") as directory:
|
||||
staging = Path(directory)
|
||||
bundle, env = stage_bundle(
|
||||
staging,
|
||||
ldd={"plezy": [f"{UNDECLARED_SONAME} => {UNDECLARED_PATH} (0x00007f9c2c000000)"]},
|
||||
owners={UNDECLARED_PATH: UNDECLARED_OWNER},
|
||||
shipped=("lib/libmpv.so.2",),
|
||||
)
|
||||
output = staging / "packages"
|
||||
output.mkdir()
|
||||
env |= {"BUILD_DIR": str(bundle), "OUTPUT_DIR": str(output)}
|
||||
|
||||
result = subprocess.run(
|
||||
[sys.executable, str(BUILD_PACKAGES)],
|
||||
cwd=ROOT,
|
||||
check=False,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
env=env,
|
||||
)
|
||||
produced = sorted(path.name for path in output.iterdir())
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stdout)
|
||||
self.assertIn(UNDECLARED_SONAME, result.stderr)
|
||||
self.assertIn("no packages were built", result.stdout)
|
||||
self.assertEqual(produced, [])
|
||||
|
||||
def test_the_opt_out_says_out_loud_what_it_costs(self) -> None:
|
||||
"""The escape hatch for dpkg-query-less hosts must announce itself.
|
||||
|
||||
A skip nobody can see in the log is indistinguishable from the guard
|
||||
having passed, which is the whole failure this wiring removes.
|
||||
"""
|
||||
with tempfile.TemporaryDirectory(prefix="plezy-packaging-optout-test-") as directory:
|
||||
staging = Path(directory)
|
||||
# A copy, not the checkout's script: past this point main() writes
|
||||
# generated icons next to itself, and a test has no business dirtying
|
||||
# the working tree. Everything under test happens before that, and the
|
||||
# copy's missing sibling guard would fail loudly if it did run.
|
||||
packaging = staging / "linux/packaging"
|
||||
packaging.mkdir(parents=True)
|
||||
script = packaging / BUILD_PACKAGES.name
|
||||
script.write_text(BUILD_PACKAGES.read_text(encoding="utf-8"), encoding="utf-8")
|
||||
bundle = staging / "bundle"
|
||||
(bundle / "lib").mkdir(parents=True)
|
||||
(bundle / "lib/libmpv.so.2").write_bytes(b"")
|
||||
|
||||
result = subprocess.run(
|
||||
[sys.executable, str(script)],
|
||||
cwd=ROOT,
|
||||
check=False,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
env={
|
||||
**os.environ,
|
||||
"BUILD_DIR": str(bundle),
|
||||
"OUTPUT_DIR": str(staging / "packages"),
|
||||
"PLEZY_SKIP_HOST_DEP_CHECK": "1",
|
||||
},
|
||||
)
|
||||
|
||||
self.assertIn("PLEZY_SKIP_HOST_DEP_CHECK is set", result.stdout)
|
||||
self.assertIn("unverified", result.stdout)
|
||||
|
||||
def test_a_falsey_opt_out_does_not_skip(self) -> None:
|
||||
"""`=0` means "do not skip" to almost everyone, and must behave that way.
|
||||
|
||||
A bare non-empty test would read `0`, `false` and `no` as consent, which
|
||||
is the exact opposite of what the person typing them meant - and hands
|
||||
back the unverified package this wiring exists to withhold.
|
||||
"""
|
||||
for value in ("0", "false", "no", "off", ""):
|
||||
with self.subTest(value=value), tempfile.TemporaryDirectory(prefix="plezy-packaging-falsey-") as directory:
|
||||
staging = Path(directory)
|
||||
packaging = staging / "linux/packaging"
|
||||
packaging.mkdir(parents=True)
|
||||
script = packaging / BUILD_PACKAGES.name
|
||||
script.write_text(BUILD_PACKAGES.read_text(encoding="utf-8"), encoding="utf-8")
|
||||
bundle = staging / "bundle"
|
||||
(bundle / "lib").mkdir(parents=True)
|
||||
(bundle / "lib/libmpv.so.2").write_bytes(b"")
|
||||
|
||||
result = subprocess.run(
|
||||
[sys.executable, str(script)],
|
||||
cwd=ROOT,
|
||||
check=False,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
env={
|
||||
**os.environ,
|
||||
"BUILD_DIR": str(bundle),
|
||||
"OUTPUT_DIR": str(staging / "packages"),
|
||||
"PLEZY_SKIP_HOST_DEP_CHECK": value,
|
||||
},
|
||||
)
|
||||
|
||||
self.assertNotIn("PLEZY_SKIP_HOST_DEP_CHECK is set", result.stdout)
|
||||
# The guard ran, so packaging stopped on its verdict rather than
|
||||
# continuing past an unverified bundle.
|
||||
self.assertNotEqual(result.returncode, 0)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,334 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Behavior tests for the Linux package dependency guard.
|
||||
|
||||
The first case reproduces the bug that motivated the guard: the native video
|
||||
plane added wayland-client, wayland-egl and egl to the runner's link line while
|
||||
the hand-maintained depends lists went untouched.
|
||||
"""
|
||||
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
CHECKER = ROOT / "scripts/check_linux_package_deps.py"
|
||||
LINUX = ROOT / "linux"
|
||||
# Every file the checker reads, relative to the linux/ directory it is given.
|
||||
FIXTURE_FILES = (
|
||||
"runner/CMakeLists.txt",
|
||||
"CMakeLists.txt",
|
||||
"flutter/CMakeLists.txt",
|
||||
"packaging/build-packages.py",
|
||||
"packaging/bundle-libs.sh",
|
||||
)
|
||||
# The subset the checker parses for pkg_check_modules and the link graph. Each
|
||||
# one carries targets the others cannot see, so none of them is optional.
|
||||
CMAKE_INPUTS = ("runner/CMakeLists.txt", "CMakeLists.txt", "flutter/CMakeLists.txt")
|
||||
|
||||
|
||||
# Pins the walk's reach: gtk+-3.0, mpv, epoxy, glib-2.0, gio-2.0 and the three the
|
||||
# video plane added. A drop here means the parser stopped seeing something rather
|
||||
# than that a link was removed.
|
||||
FULL_WALK_SUMMARY = "(8 pkg-config links, 8 modules)"
|
||||
|
||||
# The success line has to name its own scope. A reader who takes it for a
|
||||
# whole-binary check trusts it past the plugin link edges it never walked, so
|
||||
# the wording is part of what this guard promises.
|
||||
SUCCESS_LINE = "linux/runner CMake dependency checks passed"
|
||||
|
||||
|
||||
class LinuxPackageDepsGuardTest(unittest.TestCase):
|
||||
def _run(self, omit: tuple[str, ...] = (), **edits: str) -> subprocess.CompletedProcess[str]:
|
||||
"""Copy the real linux/ inputs, apply edits, and check the copy."""
|
||||
with tempfile.TemporaryDirectory(prefix="plezy-linux-deps-test-") as directory:
|
||||
linux = Path(directory) / "linux"
|
||||
for name in FIXTURE_FILES:
|
||||
if name in omit:
|
||||
continue
|
||||
target = linux / name
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
target.write_text(edits.get(name, self._source(name)), encoding="utf-8")
|
||||
return subprocess.run(
|
||||
[sys.executable, str(CHECKER), str(linux)],
|
||||
cwd=ROOT,
|
||||
check=False,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
|
||||
def _source(self, name: str) -> str:
|
||||
return (LINUX / name).read_text(encoding="utf-8")
|
||||
|
||||
def _mutate(self, name: str, old: str, new: str) -> str:
|
||||
text = self._source(name).replace(old, new, 1)
|
||||
self.assertNotEqual(text, self._source(name), f"fixture mutation no longer matches: {old!r}")
|
||||
return text
|
||||
|
||||
def test_current_tree_passes(self) -> None:
|
||||
result = self._run()
|
||||
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertIn(SUCCESS_LINE, result.stdout)
|
||||
# Naming the artifact-level check is how the blind spot stays findable.
|
||||
self.assertIn("check-bundle-host-deps.py", result.stdout)
|
||||
self.assertIn(FULL_WALK_SUMMARY, result.stdout)
|
||||
|
||||
def test_a_missing_cmake_input_is_named_rather_than_skipped(self) -> None:
|
||||
"""Losing one of these used to shrink the walk instead of failing it.
|
||||
|
||||
`flutter` is defined in flutter/CMakeLists.txt and propagates GTK, GLIB
|
||||
and GIO; drop the file and those modules leave the graph, so a depends
|
||||
entry deleted with them goes unreported and the guard exits 0 having
|
||||
checked less than it says it did.
|
||||
"""
|
||||
for name in CMAKE_INPUTS:
|
||||
with self.subTest(missing=name):
|
||||
result = self._run(omit=(name,))
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stdout)
|
||||
self.assertIn(str(Path("linux") / name), result.stderr)
|
||||
self.assertIn("is missing", result.stderr)
|
||||
|
||||
def test_every_module_of_a_multi_module_call_is_checked(self) -> None:
|
||||
"""`pkg_check_modules(X REQUIRED IMPORTED_TARGET a b c)` is legal CMake naming three modules.
|
||||
|
||||
Every one of them must be checked: the guard has to reject the call once
|
||||
wayland-cursor is added to it, not just read wayland-client and stop.
|
||||
"""
|
||||
cmake = self._mutate(
|
||||
"runner/CMakeLists.txt",
|
||||
"pkg_check_modules(WAYLAND_CLIENT REQUIRED IMPORTED_TARGET wayland-client)",
|
||||
"pkg_check_modules(WAYLAND_CLIENT REQUIRED IMPORTED_TARGET wayland-client wayland-cursor)",
|
||||
)
|
||||
|
||||
result = self._run(**{"runner/CMakeLists.txt": cmake})
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stdout)
|
||||
self.assertIn("wayland-cursor", result.stderr)
|
||||
|
||||
def test_a_version_constrained_module_spec_still_names_its_module(self) -> None:
|
||||
"""`mpv>=0.40` is a legal moduleSpec whose module name is `mpv`.
|
||||
|
||||
The guard must strip the version constraint and still walk the full tree,
|
||||
rather than reporting `mpv>=0.40` as an undeclared package.
|
||||
"""
|
||||
cmake = self._mutate(
|
||||
"runner/CMakeLists.txt",
|
||||
"pkg_check_modules(MPV REQUIRED IMPORTED_TARGET mpv)",
|
||||
"pkg_check_modules(MPV REQUIRED IMPORTED_TARGET mpv>=0.40)",
|
||||
)
|
||||
|
||||
result = self._run(**{"runner/CMakeLists.txt": cmake})
|
||||
|
||||
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
|
||||
self.assertIn(FULL_WALK_SUMMARY, result.stdout)
|
||||
|
||||
# The three below are fail-open cases: each is legal CMake that a naive regex
|
||||
# silently drops, leaving the guard to exit 0 while a library goes undeclared.
|
||||
def test_a_comment_containing_a_paren_does_not_hide_the_rest_of_the_call(self) -> None:
|
||||
cmake = self._mutate(
|
||||
"runner/CMakeLists.txt",
|
||||
"target_link_libraries(${BINARY_NAME} PRIVATE PkgConfig::EGL)",
|
||||
"target_link_libraries(${BINARY_NAME} PRIVATE\n"
|
||||
" # host EGL (never bundled)\n"
|
||||
" PkgConfig::EGL\n"
|
||||
")",
|
||||
)
|
||||
packages = self._mutate("packaging/build-packages.py", '"libegl1",\n', "")
|
||||
|
||||
result = self._run(**{"runner/CMakeLists.txt": cmake, "packaging/build-packages.py": packages})
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stdout)
|
||||
self.assertIn("links egl but the deb package", result.stderr)
|
||||
|
||||
def test_a_generator_expression_still_names_its_target(self) -> None:
|
||||
cmake = self._mutate(
|
||||
"runner/CMakeLists.txt",
|
||||
"PRIVATE PkgConfig::EGL)",
|
||||
"PRIVATE $<LINK_ONLY:PkgConfig::EGL>)",
|
||||
)
|
||||
packages = self._mutate("packaging/build-packages.py", '"libegl1",\n', "")
|
||||
|
||||
result = self._run(**{"runner/CMakeLists.txt": cmake, "packaging/build-packages.py": packages})
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stdout)
|
||||
self.assertIn("links egl but the deb package", result.stderr)
|
||||
|
||||
def test_a_quoted_target_still_names_its_target(self) -> None:
|
||||
cmake = self._mutate(
|
||||
"runner/CMakeLists.txt",
|
||||
"PRIVATE PkgConfig::EGL)",
|
||||
'PRIVATE "PkgConfig::EGL")',
|
||||
)
|
||||
packages = self._mutate("packaging/build-packages.py", '"libegl1",\n', "")
|
||||
|
||||
result = self._run(**{"runner/CMakeLists.txt": cmake, "packaging/build-packages.py": packages})
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stdout)
|
||||
self.assertIn("links egl but the deb package", result.stderr)
|
||||
|
||||
def test_pkg_check_modules_options_may_precede_the_module_name(self) -> None:
|
||||
# CMake takes its options in any order; reading REQUIRED as the module
|
||||
# name fails closed but sends the reader hunting a package nobody ships.
|
||||
cmake = self._mutate(
|
||||
"runner/CMakeLists.txt",
|
||||
"pkg_check_modules(EGL REQUIRED IMPORTED_TARGET egl)",
|
||||
"pkg_check_modules(EGL IMPORTED_TARGET REQUIRED egl)",
|
||||
)
|
||||
|
||||
result = self._run(**{"runner/CMakeLists.txt": cmake})
|
||||
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertIn(FULL_WALK_SUMMARY, result.stdout)
|
||||
|
||||
def test_the_dependency_lists_before_the_video_plane_are_rejected(self) -> None:
|
||||
# Verbatim the deb list as it stood while the runner already linked
|
||||
# wayland-client, wayland-egl and egl: the shipped bug.
|
||||
packages = self._mutate(
|
||||
"packaging/build-packages.py",
|
||||
'"libwayland-client0",\n "libwayland-cursor0",\n'
|
||||
' "libwayland-egl1",\n "libegl1",\n',
|
||||
"",
|
||||
)
|
||||
|
||||
result = self._run(**{"packaging/build-packages.py": packages})
|
||||
|
||||
self.assertEqual(result.returncode, 1)
|
||||
self.assertIn("wayland-client", result.stderr)
|
||||
self.assertIn("wayland-egl", result.stderr)
|
||||
self.assertIn("libegl1", result.stderr)
|
||||
self.assertIn("can fail to start", result.stderr)
|
||||
|
||||
def test_a_bundled_module_needs_no_declared_dependency(self) -> None:
|
||||
"""libmpv ships inside the package, so the walk must not demand a dep.
|
||||
|
||||
The runner links pkg-config `mpv` and always will; what changed is that
|
||||
the library travels with us, because the plane needs the pinned
|
||||
Wayland-enabled build. Declaring a host mpv would be the bug now.
|
||||
"""
|
||||
result = self._run()
|
||||
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertNotIn("mpv", result.stderr)
|
||||
# Still counted: a bundled module dropping out of the walk is a parser
|
||||
# regression, and the summary is what would show it.
|
||||
self.assertIn(FULL_WALK_SUMMARY, result.stdout)
|
||||
|
||||
def test_one_missing_distro_is_rejected(self) -> None:
|
||||
# A dependency declared for deb but forgotten for rpm still ships broken
|
||||
# on Fedora, so per-distro coverage is the unit, not per-library.
|
||||
packages = self._mutate("packaging/build-packages.py", '"libglvnd-egl",\n', "")
|
||||
|
||||
result = self._run(**{"packaging/build-packages.py": packages})
|
||||
|
||||
self.assertEqual(result.returncode, 1)
|
||||
self.assertIn("rpm package does not depend on 'libglvnd-egl'", result.stderr)
|
||||
self.assertNotIn("deb package does not depend", result.stderr)
|
||||
|
||||
def test_pacman_shared_wayland_package_counts_for_both_modules(self) -> None:
|
||||
# Arch has no separate libwayland-egl, so one entry has to satisfy two
|
||||
# modules. Dropping it must fail for both rather than neither.
|
||||
packages = self._mutate("packaging/build-packages.py", '"wayland",\n', "")
|
||||
|
||||
result = self._run(**{"packaging/build-packages.py": packages})
|
||||
|
||||
self.assertEqual(result.returncode, 1)
|
||||
self.assertIn("links wayland-client but the pacman package", result.stderr)
|
||||
self.assertIn("links wayland-egl but the pacman package", result.stderr)
|
||||
|
||||
def test_new_pkgconfig_link_without_a_package_mapping_is_rejected(self) -> None:
|
||||
# The forward-looking half: the next library added to the runner has to
|
||||
# name its runtime package before it can ship.
|
||||
cmake = self._mutate(
|
||||
"runner/CMakeLists.txt",
|
||||
"target_link_libraries(${BINARY_NAME} PRIVATE PkgConfig::EGL)",
|
||||
"pkg_check_modules(PIPEWIRE REQUIRED IMPORTED_TARGET libpipewire-0.3)\n"
|
||||
"target_link_libraries(${BINARY_NAME} PRIVATE PkgConfig::EGL)\n"
|
||||
"target_link_libraries(${BINARY_NAME} PRIVATE PkgConfig::PIPEWIRE)",
|
||||
)
|
||||
|
||||
result = self._run(**{"runner/CMakeLists.txt": cmake})
|
||||
|
||||
self.assertEqual(result.returncode, 1)
|
||||
self.assertIn("libpipewire-0.3", result.stderr)
|
||||
self.assertIn("RUNTIME_PACKAGES", result.stderr)
|
||||
|
||||
def test_a_grouped_link_is_read_past_the_first_target(self) -> None:
|
||||
# CMake happily takes several targets in one call. A parse that stopped at
|
||||
# the first would wave the rest through while they sat on the link line.
|
||||
cmake = self._mutate(
|
||||
"runner/CMakeLists.txt",
|
||||
"target_link_libraries(${BINARY_NAME} PRIVATE PkgConfig::EGL)",
|
||||
"pkg_check_modules(PIPEWIRE REQUIRED IMPORTED_TARGET libpipewire-0.3)\n"
|
||||
"target_link_libraries(${BINARY_NAME} PRIVATE PkgConfig::EGL PkgConfig::PIPEWIRE)",
|
||||
)
|
||||
|
||||
result = self._run(**{"runner/CMakeLists.txt": cmake})
|
||||
|
||||
self.assertEqual(result.returncode, 1)
|
||||
self.assertIn("libpipewire-0.3", result.stderr)
|
||||
|
||||
def test_a_transitive_link_through_an_internal_target_is_followed(self) -> None:
|
||||
# wayland_protocols already hands PkgConfig::WAYLAND_CLIENT to whatever
|
||||
# links it. A static library's dependencies land on the consumer's link
|
||||
# line, so stopping at the internal target would miss a real dependency.
|
||||
cmake = self._mutate(
|
||||
"runner/CMakeLists.txt",
|
||||
"target_link_libraries(wayland_protocols PUBLIC PkgConfig::WAYLAND_CLIENT)",
|
||||
"pkg_check_modules(PIPEWIRE REQUIRED IMPORTED_TARGET libpipewire-0.3)\n"
|
||||
"target_link_libraries(wayland_protocols PUBLIC PkgConfig::WAYLAND_CLIENT "
|
||||
"PkgConfig::PIPEWIRE)",
|
||||
)
|
||||
|
||||
result = self._run(**{"runner/CMakeLists.txt": cmake})
|
||||
|
||||
self.assertEqual(result.returncode, 1)
|
||||
self.assertIn("libpipewire-0.3", result.stderr)
|
||||
|
||||
def test_dropping_the_direct_wayland_client_link_still_finds_it(self) -> None:
|
||||
# The runner names wayland-client directly *and* gets it through
|
||||
# wayland_protocols. Removing the direct link must not silence the guard,
|
||||
# because the binary still links the library either way.
|
||||
cmake = self._mutate(
|
||||
"runner/CMakeLists.txt",
|
||||
"target_link_libraries(${BINARY_NAME} PRIVATE PkgConfig::WAYLAND_CLIENT)\n",
|
||||
"",
|
||||
)
|
||||
packages = self._mutate("packaging/build-packages.py", '"libwayland-client0",\n', "")
|
||||
|
||||
result = self._run(
|
||||
**{"runner/CMakeLists.txt": cmake, "packaging/build-packages.py": packages}
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 1)
|
||||
self.assertIn("links wayland-client but the deb package", result.stderr)
|
||||
|
||||
def test_a_link_with_no_pkg_check_modules_is_rejected(self) -> None:
|
||||
cmake = self._mutate(
|
||||
"runner/CMakeLists.txt",
|
||||
"pkg_check_modules(EGL REQUIRED IMPORTED_TARGET egl)",
|
||||
"# EGL declaration removed",
|
||||
)
|
||||
|
||||
result = self._run(**{"runner/CMakeLists.txt": cmake})
|
||||
|
||||
self.assertEqual(result.returncode, 1)
|
||||
self.assertIn("PkgConfig::EGL is linked into the runner", result.stderr)
|
||||
|
||||
def test_bundling_the_excluded_libraries_is_rejected(self) -> None:
|
||||
# The guard's premise is that these come from the host. If bundle-libs.sh
|
||||
# starts shipping them, the demand for a depends entry needs rethinking
|
||||
# rather than silently continuing to hold.
|
||||
bundle = self._mutate("packaging/bundle-libs.sh", r"libwayland.*\.so|", "")
|
||||
|
||||
result = self._run(**{"packaging/bundle-libs.sh": bundle})
|
||||
|
||||
self.assertEqual(result.returncode, 1)
|
||||
self.assertIn("no longer excludes", result.stderr)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,229 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Behavior tests for the package metadata read-back guard.
|
||||
|
||||
The guard exists because nothing between build-packages.py and the upload read
|
||||
anything back out of what fpm wrote: a dropped or renamed `--depends` shipped a
|
||||
package that installs cleanly and dies in the loader, with every earlier check
|
||||
green. These tests hold that line, and the substring case below is the specific
|
||||
regression the shell version it replaced once had - `libegl1` is a substring of
|
||||
`libegl1-mesa`, so a package declaring neither used to pass.
|
||||
|
||||
dpkg-deb, rpm and bsdtar only answer on a machine that has them, so the fixtures
|
||||
put stubs first on PATH and let the real script fork them. That keeps "an rpm
|
||||
missing libdrm" a fixture rather than a machine, and keeps every assertion
|
||||
running against the script CI runs.
|
||||
"""
|
||||
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
# The Windows-safe stub installer, rather than a second copy of its PATHEXT
|
||||
# reasoning. scripts/ is sys.path[0] however this file is invoked.
|
||||
from test_check_bundle_host_deps import install_stub
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
CHECKER = ROOT / "linux/packaging/check-package-deps.py"
|
||||
BUILD_PACKAGES = ROOT / "linux/packaging/build-packages.py"
|
||||
|
||||
# The expected names come from build-packages.py the same way the guard reads
|
||||
# them, so a library added there is exercised here without editing a fixture.
|
||||
_spec = importlib.util.spec_from_file_location("build_packages", BUILD_PACKAGES)
|
||||
PACKAGING = importlib.util.module_from_spec(_spec)
|
||||
_spec.loader.exec_module(PACKAGING)
|
||||
DISTROS = PACKAGING.DISTROS
|
||||
NAME = PACKAGING.METADATA["name"]
|
||||
|
||||
# The tool each package format is read with, which is also what the fixtures stub.
|
||||
TOOLS = {"deb": "dpkg-deb", "rpm": "rpm", "pacman": "bsdtar"}
|
||||
|
||||
# One stub per tool: it prints the fixture's answer for its own name, or fails
|
||||
# the way an unreadable archive does. Baking the name in keeps the stub blind to
|
||||
# the arguments, so it cannot accidentally pass by echoing its input.
|
||||
STUB = """import json, os, sys
|
||||
|
||||
answers = json.loads(open(os.environ["PLEZY_TEST_ANSWERS"], encoding="utf-8").read())
|
||||
answer = answers.get({name!r})
|
||||
if answer is None:
|
||||
sys.stderr.write({name!r} + ": cannot read this archive\\n")
|
||||
sys.exit(1)
|
||||
sys.stdout.write(answer)
|
||||
"""
|
||||
|
||||
|
||||
def declared_first_names(distro: str) -> list[str]:
|
||||
"""What a correct package declares: one acceptable name per dependency."""
|
||||
return [dependency.split("|")[0].strip() for dependency in DISTROS[distro]["depends"]]
|
||||
|
||||
|
||||
def decorate(names: list[str], version: str, qualifier: str) -> list[str]:
|
||||
"""Dress the first two names the way the real tools report them.
|
||||
|
||||
A constraint, an architecture qualifier and rpm's soname decoration are noise
|
||||
around a package name. Applying them to fixture names rather than asserting
|
||||
on a hand-written blob means the stripping is tested against the list the
|
||||
packages really declare.
|
||||
"""
|
||||
if len(names) < 2:
|
||||
return names
|
||||
return [f"{names[0]} {version}", f"{names[1]}{qualifier}", *names[2:]]
|
||||
|
||||
|
||||
def deb_metadata(names: list[str] | None = None) -> str:
|
||||
"""`dpkg-deb -f ... Depends` output."""
|
||||
names = declared_first_names("deb") if names is None else names
|
||||
return ", ".join(decorate(names, "(>= 3.24.0)", ":amd64")) + "\n"
|
||||
|
||||
|
||||
def rpm_metadata(names: list[str] | None = None) -> str:
|
||||
"""`rpm -qpR` output, including the requires rpm adds by itself."""
|
||||
names = declared_first_names("rpm") if names is None else names
|
||||
automatic = ["/bin/sh", "libc.so.6(GLIBC_2.34)(64bit)", "rpmlib(PayloadIsXz) <= 5.2-1"]
|
||||
return "\n".join(automatic + decorate(names, ">= 3.24", "(x86-64)")) + "\n"
|
||||
|
||||
|
||||
def pkginfo_metadata(names: list[str] | None = None) -> str:
|
||||
"""A whole .PKGINFO, so the `depend = ` filter is what isolates the names."""
|
||||
names = declared_first_names("pacman") if names is None else names
|
||||
header = [f"pkgname = {NAME}", "pkgver = 1.2.3-1", "arch = x86_64"]
|
||||
depends = [f"depend = {name}" for name in decorate(names, ">=3.24", "")]
|
||||
return "\n".join(header + depends) + "\n"
|
||||
|
||||
|
||||
def correct_metadata() -> dict[str, str]:
|
||||
return {"dpkg-deb": deb_metadata(), "rpm": rpm_metadata(), "bsdtar": pkginfo_metadata()}
|
||||
|
||||
|
||||
class PackageDepsReadBackTest(unittest.TestCase):
|
||||
def _check(
|
||||
self,
|
||||
metadata: dict[str, str | None],
|
||||
arch: str = "x64",
|
||||
produce: tuple[str, ...] = ("deb", "rpm", "pacman"),
|
||||
install: tuple[str, ...] = ("dpkg-deb", "rpm", "bsdtar"),
|
||||
path: str | None = None,
|
||||
) -> subprocess.CompletedProcess[str]:
|
||||
"""Stage packages plus tool stubs and run the real guard over them."""
|
||||
with tempfile.TemporaryDirectory(prefix="plezy-package-deps-test-") as directory:
|
||||
staging = Path(directory)
|
||||
packages = staging / "packages"
|
||||
packages.mkdir()
|
||||
for distro in produce:
|
||||
(packages / f"{NAME}-linux-{arch}.{DISTROS[distro]['ext']}").write_bytes(b"")
|
||||
|
||||
answers = staging / "answers.json"
|
||||
answers.write_text(json.dumps(metadata), encoding="utf-8")
|
||||
scripts, tools = staging / "stubs", staging / "tools"
|
||||
scripts.mkdir()
|
||||
tools.mkdir()
|
||||
for tool in install:
|
||||
install_stub(scripts, tools, tool, STUB.format(name=tool))
|
||||
|
||||
return subprocess.run(
|
||||
[sys.executable, str(CHECKER), str(packages), "--arch", arch],
|
||||
cwd=ROOT,
|
||||
check=False,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
env={
|
||||
**os.environ,
|
||||
# The stubs first, so a machine that really has dpkg-deb
|
||||
# answers from the fixture. `path=""` leaves only the stubs,
|
||||
# which is how a tool is made genuinely absent.
|
||||
"PATH": str(tools) + os.pathsep + (os.environ.get("PATH", "") if path is None else path),
|
||||
"PLEZY_TEST_ANSWERS": str(answers),
|
||||
},
|
||||
)
|
||||
|
||||
def test_packages_carrying_every_declared_dependency_pass(self) -> None:
|
||||
result = self._check(correct_metadata())
|
||||
|
||||
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
|
||||
self.assertIn("survived fpm", result.stdout)
|
||||
# Version constraints, deb's arch qualifier and rpm's soname decorations
|
||||
# are noise around a name, not a different package.
|
||||
self.assertNotIn("::error::", result.stderr)
|
||||
|
||||
def test_a_dependency_fpm_dropped_is_named(self) -> None:
|
||||
"""The regression the guard exists for: a name that reached fpm and not the package."""
|
||||
for distro in DISTROS:
|
||||
with self.subTest(distro=distro):
|
||||
dropped = declared_first_names(distro)[0]
|
||||
kept = declared_first_names(distro)[1:]
|
||||
metadata = correct_metadata()
|
||||
metadata[TOOLS[distro]] = {
|
||||
"deb": deb_metadata,
|
||||
"rpm": rpm_metadata,
|
||||
"pacman": pkginfo_metadata,
|
||||
}[distro](kept)
|
||||
|
||||
result = self._check(metadata)
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stdout)
|
||||
self.assertIn(f"the {distro} package does not require {dropped}", result.stderr)
|
||||
self.assertNotIn("survived fpm", result.stdout)
|
||||
|
||||
def test_a_longer_package_name_does_not_satisfy_a_shorter_one(self) -> None:
|
||||
"""`libegl1-mesa` is not `libegl1`, however much of one it contains."""
|
||||
names = [f"{name}-mesa" if name == "libegl1" else name for name in declared_first_names("deb")]
|
||||
self.assertIn("libegl1-mesa", names, "the deb list no longer contains libegl1")
|
||||
metadata = correct_metadata() | {"dpkg-deb": deb_metadata(names)}
|
||||
|
||||
result = self._check(metadata)
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stdout)
|
||||
self.assertIn("the deb package does not require libegl1", result.stderr)
|
||||
|
||||
def test_a_package_fpm_never_wrote_fails(self) -> None:
|
||||
result = self._check(correct_metadata(), produce=("deb", "pacman"))
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stdout)
|
||||
self.assertIn(f"{NAME}-linux-x64.rpm", result.stderr)
|
||||
self.assertIn("was not produced", result.stderr)
|
||||
self.assertNotIn("survived fpm", result.stdout)
|
||||
|
||||
def test_an_unreadable_pkginfo_is_not_a_missing_dependency(self) -> None:
|
||||
""""the archive member was not found" must stay separable from "fpm dropped everything"."""
|
||||
result = self._check(correct_metadata() | {"bsdtar": ""})
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stdout)
|
||||
self.assertIn("could not read .PKGINFO", result.stderr)
|
||||
self.assertNotIn("does not require", result.stderr)
|
||||
|
||||
def test_a_tool_that_cannot_run_is_a_failure_not_a_pass(self) -> None:
|
||||
"""A guard that proved nothing must never report that it proved something.
|
||||
|
||||
Both halves matter: a reader absent from the machine, and one present but
|
||||
refusing the archive. Either way nothing was read, so nothing is declared.
|
||||
"""
|
||||
for distro, tool in TOOLS.items():
|
||||
with self.subTest(missing=tool):
|
||||
installed = tuple(name for name in TOOLS.values() if name != tool)
|
||||
absent = self._check(correct_metadata(), install=installed, path="")
|
||||
self.assertEqual(absent.returncode, 1, absent.stdout)
|
||||
self.assertIn(f"{tool} is not installed", absent.stderr)
|
||||
self.assertNotIn("survived fpm", absent.stdout)
|
||||
|
||||
with self.subTest(failing=tool):
|
||||
broken = self._check(correct_metadata() | {tool: None})
|
||||
self.assertEqual(broken.returncode, 1, broken.stdout)
|
||||
self.assertIn(f"{tool} failed", broken.stderr)
|
||||
self.assertNotIn(f"the {distro} package does not require", broken.stderr)
|
||||
|
||||
def test_the_release_architecture_is_read_from_its_own_files(self) -> None:
|
||||
"""The release job ships arm64 too, and x64 filenames must not stand in for it."""
|
||||
result = self._check(correct_metadata(), arch="arm64")
|
||||
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
|
||||
|
||||
missing = self._check(correct_metadata(), arch="x64", produce=())
|
||||
self.assertEqual(missing.returncode, 1, missing.stdout)
|
||||
self.assertIn(f"{NAME}-linux-x64.deb", missing.stderr)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -88,6 +88,12 @@ def _validate_native(root: Path, errors: list[str]) -> None:
|
||||
_require_text(value.get("provenance"), f"{label}.provenance", errors)
|
||||
if url and not url.startswith("https://"):
|
||||
errors.append(f"{label}.url: production source must use HTTPS")
|
||||
# A fallback source is optional, but it is a production source when it is
|
||||
# used, so it answers to the same rule as the primary.
|
||||
mirror = value.get("mirror")
|
||||
if mirror is not None:
|
||||
if not isinstance(mirror, str) or not mirror.startswith("https://"):
|
||||
errors.append(f"{label}.mirror: production source must use HTTPS")
|
||||
if version and url and name in {"ffmpeg", "mpv", "simdutf"} and version not in url:
|
||||
errors.append(f"{label}.url: must identify declared version {version}")
|
||||
if kind == "archive":
|
||||
|
||||
Reference in New Issue
Block a user