ci(linux): check the runner's libraries reach the package metadata
The plane added three runtime libraries that bundle-libs.sh deliberately does not bundle, so they have to be declared per distro by hand - and two hand-maintained lists drifting apart is the failure this guard exists to prevent. check_linux_package_deps.py parses the runner's CMake for every pkg-config module it links, follows target_link_libraries to prove each one actually reaches the binary, and requires a package name for it in every distro's depends list. It fails closed on the shapes a naive parser gets wrong: a pkg_check_modules call naming several modules, options preceding the module name, and version constraints like mpv>=0.40 that would otherwise be read as a package nobody ships. The smoke job builds the three packages and reads the dependencies back out of the artifacts, deriving what to expect from build-packages.py rather than restating it - so a library is declared once and verified everywhere. That job is off by default, which is exactly why it must not carry its own copy of the list. The Linux native job names libwayland-dev and libegl-dev instead of riding GTK's and epoxy's transitive dev dependencies, matching the CMake comment's own rationale. In CI the host-dependency guard runs once: the named step covers the staged bundle, and build-packages.py's internal run - which exists for by-hand packaging - is skipped. The smoke job also drops patchelf, which nothing invokes.
This commit is contained in:
edde746
@@ -0,0 +1,291 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Guard the Linux package dependency lists against what the runner links.
|
||||
|
||||
linux/packaging/bundle-libs.sh deliberately refuses to bundle the display- and
|
||||
driver-coupled libraries (libEGL, libwayland-*, libGL, libdrm ...): they must
|
||||
come from the host or the app will not talk to the compositor it is running
|
||||
under. That makes them the package manager's problem, and the depends lists in
|
||||
linux/packaging/build-packages.py are maintained by hand.
|
||||
|
||||
Nothing connected the two. Adding a pkg-config link to the runner produced a
|
||||
binary with an undeclared shared-library dependency, and the failure surfaces
|
||||
only on a user's machine at exec time - a class of bug no compile or unit test
|
||||
can reach. This walks the runner's own link line instead:
|
||||
|
||||
target_link_libraries(${BINARY_NAME} PRIVATE PkgConfig::WAYLAND_EGL)
|
||||
-> pkg_check_modules(WAYLAND_EGL REQUIRED IMPORTED_TARGET wayland-egl)
|
||||
-> RUNTIME_PACKAGES["wayland-egl"] -> libwayland-egl1 / libwayland-egl / wayland
|
||||
|
||||
and requires every distro to declare it. A new pkg-config module fails here
|
||||
until its runtime package names are named for all three.
|
||||
|
||||
What it walks is exactly CMAKE_FILES, the three CMakeLists.txt this checkout
|
||||
owns - and nothing else. The Flutter plugins link into the same binary from
|
||||
linux/flutter/generated_plugins.cmake, whose add_subdirectory() targets live
|
||||
under flutter/ephemeral/.plugin_symlinks/, a directory that only exists after
|
||||
`flutter pub get`. A plugin's own pkg_check_modules is therefore unreadable at
|
||||
pull-request time, and a plugin that starts linking a new host library passes
|
||||
here. linux/packaging/check-bundle-host-deps.py is what covers that: it runs
|
||||
ldd over the built bundle, where every link edge is finally real.
|
||||
"""
|
||||
|
||||
from pathlib import Path
|
||||
import ast
|
||||
import re
|
||||
import sys
|
||||
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
if len(sys.argv) > 2:
|
||||
raise SystemExit(f"Usage: {Path(sys.argv[0]).name} [linux-dir]")
|
||||
LINUX = Path(sys.argv[1]).resolve() if len(sys.argv) == 2 else ROOT / "linux"
|
||||
|
||||
RUNNER_CMAKE = LINUX / "runner/CMakeLists.txt"
|
||||
PACKAGES_PY = LINUX / "packaging/build-packages.py"
|
||||
BUNDLE_SH = LINUX / "packaging/bundle-libs.sh"
|
||||
# pkg_check_modules for targets the runner links may live in any of these.
|
||||
CMAKE_FILES = (RUNNER_CMAKE, LINUX / "CMakeLists.txt", LINUX / "flutter/CMakeLists.txt")
|
||||
|
||||
# pkg-config modules whose library ships *inside* the package instead of being
|
||||
# depended on. libmpv is pinned and Wayland-enabled because the video plane needs
|
||||
# it to be; a distro libmpv silently drops hwdec to vaapi-copy. Bundling it means
|
||||
# there is deliberately no runtime dependency to find, so the walk must not
|
||||
# demand one - but the libraries it links that bundle-libs.sh excludes still have
|
||||
# to be declared, which the packaging job re-derives from the built bundle.
|
||||
BUNDLED_MODULES = {"mpv"}
|
||||
|
||||
# pkg-config module -> the package that ships its runtime library, per distro.
|
||||
# Only modules the runner actually links are consulted, so an unused entry here
|
||||
# is harmless; a missing one is an error.
|
||||
RUNTIME_PACKAGES = {
|
||||
"gtk+-3.0": {"deb": "libgtk-3-0", "rpm": "gtk3", "pacman": "gtk3"},
|
||||
"epoxy": {"deb": "libepoxy0", "rpm": "libepoxy", "pacman": "libepoxy"},
|
||||
# Reached through the `flutter` INTERFACE target rather than named by the
|
||||
# runner, which is why the graph has to cross file boundaries to see them.
|
||||
"glib-2.0": {"deb": "libglib2.0-0", "rpm": "glib2", "pacman": "glib2"},
|
||||
"gio-2.0": {"deb": "libglib2.0-0", "rpm": "glib2", "pacman": "glib2"},
|
||||
"wayland-client": {
|
||||
"deb": "libwayland-client0",
|
||||
"rpm": "libwayland-client",
|
||||
# Arch ships every libwayland-* in the one `wayland` package.
|
||||
"pacman": "wayland",
|
||||
},
|
||||
"wayland-egl": {
|
||||
"deb": "libwayland-egl1",
|
||||
"rpm": "libwayland-egl",
|
||||
"pacman": "wayland",
|
||||
},
|
||||
# libglvnd is the vendor-neutral dispatch that provides libEGL.so.1.
|
||||
"egl": {"deb": "libegl1", "rpm": "libglvnd-egl", "pacman": "libglvnd"},
|
||||
}
|
||||
|
||||
errors: list[str] = []
|
||||
|
||||
|
||||
def require(condition: bool, message: str) -> None:
|
||||
if not condition:
|
||||
errors.append(message)
|
||||
|
||||
|
||||
def read(path: Path) -> str:
|
||||
try:
|
||||
return path.read_text(encoding="utf-8")
|
||||
except OSError as error:
|
||||
errors.append(f"{path}: cannot read: {error}")
|
||||
return ""
|
||||
|
||||
|
||||
# Keywords that carry no target name.
|
||||
LINK_KEYWORDS = {"PRIVATE", "PUBLIC", "INTERFACE", "optimized", "debug", "general"}
|
||||
# Options CMake accepts between IMPORTED_TARGET and the module names, in any order.
|
||||
PKG_OPTIONS = ("REQUIRED", "QUIET", "GLOBAL", "NO_CMAKE_PATH", "NO_CMAKE_ENVIRONMENT_PATH")
|
||||
|
||||
|
||||
def strip_comments(text: str) -> str:
|
||||
"""A `#` comment containing `)` would otherwise truncate a call body.
|
||||
|
||||
That is the fail-open direction: every target after the comment vanishes and
|
||||
the guard still exits 0, which is the whole bug class it exists to catch.
|
||||
"""
|
||||
return re.sub(r"#[^\n]*", "", text)
|
||||
|
||||
|
||||
def link_token(raw: str) -> str:
|
||||
"""`$<LINK_ONLY:PkgConfig::X>` and `"PkgConfig::X"` both name PkgConfig::X."""
|
||||
return re.sub(r"^\$<[^:]*:", "", raw.strip('"')).rstrip(">")
|
||||
|
||||
|
||||
def link_graph(text: str) -> dict[str, list[str]]:
|
||||
"""target -> everything target_link_libraries() gives it, in order."""
|
||||
graph: dict[str, list[str]] = {}
|
||||
for match in re.finditer(r"target_link_libraries\(\s*([^\s)]+)\s*([^)]*)\)", strip_comments(text)):
|
||||
name = match.group(1).replace("${BINARY_NAME}", "BINARY")
|
||||
tokens = [link_token(t) for t in match.group(2).split()]
|
||||
graph.setdefault(name, []).extend(t for t in tokens if t not in LINK_KEYWORDS)
|
||||
return graph
|
||||
|
||||
|
||||
def linked_pkgconfig_targets(text: str) -> set[str]:
|
||||
"""Every PkgConfig:: target that reaches the runner's link line.
|
||||
|
||||
A library hands its dependencies to whatever links it - CMake puts even
|
||||
PRIVATE ones of a static library on the consumer's link line, and an
|
||||
INTERFACE target exists only to propagate them - so an internal target has to
|
||||
be followed rather than treated as a leaf. `wayland_protocols PUBLIC
|
||||
PkgConfig::WAYLAND_CLIENT` and `flutter INTERFACE PkgConfig::GTK` are both
|
||||
invisible otherwise, the latter across a file boundary.
|
||||
"""
|
||||
graph = link_graph(text)
|
||||
targets: set[str] = set()
|
||||
seen: set[str] = set()
|
||||
queue = ["BINARY"]
|
||||
while queue:
|
||||
current = queue.pop()
|
||||
if current in seen:
|
||||
continue
|
||||
seen.add(current)
|
||||
for token in graph.get(current, []):
|
||||
if token.startswith("PkgConfig::"):
|
||||
targets.add(token[len("PkgConfig::") :])
|
||||
elif token in graph:
|
||||
queue.append(token)
|
||||
return targets
|
||||
|
||||
|
||||
def pkgconfig_modules() -> dict[str, list[str]]:
|
||||
"""CMake variable prefix -> every pkg-config module the call names.
|
||||
|
||||
A single call may name several - `pkg_check_modules(X REQUIRED
|
||||
IMPORTED_TARGET a b c)` makes one PkgConfig::X that links all three - and
|
||||
taking only the first is the fail-open direction: the extra libraries reach
|
||||
the binary while the guard reports a clean run. wayland-cursor and
|
||||
xkbcommon are the natural companions of a subsurface and grouping them into
|
||||
the existing call is the natural way to add them, so this is the next edit
|
||||
to this file rather than a hypothetical.
|
||||
"""
|
||||
modules: dict[str, list[str]] = {}
|
||||
options = "|".join(PKG_OPTIONS)
|
||||
for path in CMAKE_FILES:
|
||||
for match in re.finditer(
|
||||
# The options may precede the module names, so skip any run of them
|
||||
# rather than taking the first token and reporting `REQUIRED` as a
|
||||
# package nobody ships. The tail is then every remaining token up to
|
||||
# the closing paren.
|
||||
r"pkg_check_modules\(\s*(\w+)\b[^)]*?IMPORTED_TARGET\s+((?:(?:" + options + r")\s+)*[^)]*)\)",
|
||||
strip_comments(read(path)),
|
||||
):
|
||||
# A moduleSpec is `<name>` or `<name><op><version>`, so the version
|
||||
# constraint has to come off before the name is looked up - otherwise
|
||||
# a perfectly legal `mpv>=0.40` is reported as a package nobody
|
||||
# ships, and the message sends whoever hits it off to invent a
|
||||
# RUNTIME_PACKAGES entry for it. Pinning that minimum is a plausible
|
||||
# next edit here: target-colorspace-hint=auto needs mpv 0.40.
|
||||
names = [
|
||||
re.split(r"[<>=!]", t, maxsplit=1)[0] for t in match.group(2).split() if t not in PKG_OPTIONS
|
||||
]
|
||||
names = [n for n in names if n]
|
||||
if names:
|
||||
modules.setdefault(match.group(1), names)
|
||||
return modules
|
||||
|
||||
|
||||
def declared_depends() -> dict[str, list[str]]:
|
||||
"""distro -> depends list, read from the DISTROS literal by AST."""
|
||||
tree = ast.parse(read(PACKAGES_PY), filename=str(PACKAGES_PY))
|
||||
for node in tree.body:
|
||||
if not isinstance(node, ast.Assign):
|
||||
continue
|
||||
if not any(isinstance(t, ast.Name) and t.id == "DISTROS" for t in node.targets):
|
||||
continue
|
||||
table = ast.literal_eval(node.value)
|
||||
return {name: list(config.get("depends", [])) for name, config in table.items()}
|
||||
errors.append(f"{PACKAGES_PY}: no DISTROS assignment to read the depends lists from")
|
||||
return {}
|
||||
|
||||
|
||||
# Every file, not just the runner's: `flutter` is defined in flutter/CMakeLists.txt
|
||||
# and propagates GTK, GLIB and GIO to whatever links it, so a graph built from one
|
||||
# file treats it as a leaf and never sees them. A member that moved or was renamed
|
||||
# is fatal rather than a smaller walk: its modules drop out of the graph, a
|
||||
# declaration deleted alongside it goes unreported, and every check below then
|
||||
# passes over a tree nobody actually looked at.
|
||||
absent = [path for path in CMAKE_FILES if not path.is_file()]
|
||||
if absent:
|
||||
for path in absent:
|
||||
print(
|
||||
f"ERROR: {path}: expected CMake input is missing, so the dependency walk "
|
||||
"would silently cover less than it claims",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
cmake_text = "\n".join(read(path) for path in CMAKE_FILES)
|
||||
modules = pkgconfig_modules()
|
||||
depends = declared_depends()
|
||||
|
||||
require(bool(depends), "no distro depends lists were found, so nothing was checked")
|
||||
|
||||
# The exclusion list is what makes declaring these mandatory rather than
|
||||
# optional. If bundling ever starts covering them, this guard is the wrong shape.
|
||||
bundle = read(BUNDLE_SH)
|
||||
for pattern in (r"libEGL\.so", r"libwayland.*\.so"):
|
||||
require(
|
||||
pattern in bundle,
|
||||
f"bundle-libs.sh no longer excludes {pattern}: if those are bundled now, "
|
||||
"the depends entries this guard demands may be wrong",
|
||||
)
|
||||
|
||||
linked = linked_pkgconfig_targets(cmake_text)
|
||||
require(
|
||||
bool(linked),
|
||||
"found no PkgConfig:: link reaching ${BINARY_NAME}; the link-line parse is broken, not the build",
|
||||
)
|
||||
|
||||
checked_modules = 0
|
||||
for target in sorted(linked):
|
||||
target_modules = modules.get(target)
|
||||
if not target_modules:
|
||||
errors.append(
|
||||
f"PkgConfig::{target} is linked into the runner but no pkg_check_modules "
|
||||
f"declares it in {', '.join(p.name for p in CMAKE_FILES)}"
|
||||
)
|
||||
continue
|
||||
for module in target_modules:
|
||||
checked_modules += 1
|
||||
if module in BUNDLED_MODULES:
|
||||
# Shipped inside the package, so there is no dependency to find. Still
|
||||
# counted, so the summary keeps naming everything the walk reached and
|
||||
# a module going missing is a drop rather than a silent skip.
|
||||
continue
|
||||
packages = RUNTIME_PACKAGES.get(module)
|
||||
if packages is None:
|
||||
errors.append(
|
||||
f"pkg-config module '{module}' (PkgConfig::{target}) is linked into the runner "
|
||||
f"but has no entry in RUNTIME_PACKAGES: name the package that ships its "
|
||||
f"runtime library on each distro, then declare it in {PACKAGES_PY.name}"
|
||||
)
|
||||
continue
|
||||
for distro, declared in sorted(depends.items()):
|
||||
package = packages.get(distro)
|
||||
if package is None:
|
||||
errors.append(
|
||||
f"RUNTIME_PACKAGES['{module}'] has no '{distro}' package name, so the "
|
||||
f"{distro} package cannot declare a library the runner links"
|
||||
)
|
||||
continue
|
||||
require(
|
||||
package in declared,
|
||||
f"the runner links {module} but the {distro} package does not depend on "
|
||||
f"'{package}'; bundle-libs.sh will not bundle it, so an installed package "
|
||||
f"can fail to start",
|
||||
)
|
||||
|
||||
if errors:
|
||||
for error in errors:
|
||||
print(f"ERROR: {error}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
print(
|
||||
f"linux/runner CMake dependency checks passed ({len(linked)} pkg-config links, {checked_modules} modules); "
|
||||
"Flutter plugin links are out of reach here - check-bundle-host-deps.py covers those from the built bundle"
|
||||
)
|
||||
@@ -5,9 +5,11 @@
|
||||
# and script guards" step in .github/workflows/ci.yml and section 4 of
|
||||
# scripts/ci_checks.sh. The checkers are named explicitly because a few of them
|
||||
# belong to other jobs (check_bun_audit.py needs Bun, check_codegen.py runs via
|
||||
# codegen.sh), but their regression tests are discovered by glob so a newly
|
||||
# added scripts/test_*.py is picked up automatically instead of having to be
|
||||
# remembered in two places.
|
||||
# codegen.sh, and linux/packaging/check-bundle-host-deps.py and
|
||||
# check-package-deps.py need a built bundle and built packages, so both run in
|
||||
# the linux-packages job and again in the release build), but their regression
|
||||
# tests are discovered by glob so a newly added scripts/test_*.py is picked up
|
||||
# automatically instead of having to be remembered in two places.
|
||||
set -euo pipefail
|
||||
shopt -s nullglob
|
||||
|
||||
@@ -24,6 +26,7 @@ for checker in \
|
||||
scripts/check_workflow_action_pins.py \
|
||||
scripts/check_container_image_pins.py \
|
||||
scripts/check_update_packages_workflow.py \
|
||||
scripts/check_linux_package_deps.py \
|
||||
scripts/check_windows_installer.py \
|
||||
scripts/check_windows_msix.py; do
|
||||
python3 "$checker"
|
||||
|
||||
@@ -0,0 +1,430 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Behavior tests for the bundle host dependency guard.
|
||||
|
||||
The nested-module case reproduces the bug that motivated them: bundle-libs.sh
|
||||
installs the dlopen'd gdk-pixbuf loaders and GIO modules into lib/
|
||||
subdirectories, and the guard scanned lib/*.so* only - so those modules counted
|
||||
as bundled while their own host dependencies never reached ldd, and the check
|
||||
reported success over exactly the undeclared library it exists to catch.
|
||||
|
||||
ldd and dpkg-query only answer on a Debian-family machine, so the checker
|
||||
resolves both through PLEZY_HOST_TOOLS and the fixtures here install stubs
|
||||
there. That keeps "a bundle that needs libpng16" a fixture rather than a
|
||||
machine, and keeps the tests exercising the real script end to end.
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
CHECKER = ROOT / "linux/packaging/check-bundle-host-deps.py"
|
||||
BUILD_PACKAGES = ROOT / "linux/packaging/build-packages.py"
|
||||
|
||||
# A host library whose deb owner and rpm/pacman names build-packages.py all
|
||||
# declare, so its presence alone never fails a fixture.
|
||||
DECLARED_SONAME = "libepoxy.so.0"
|
||||
DECLARED_PATH = "/usr/lib/x86_64-linux-gnu/libepoxy.so.0"
|
||||
DECLARED_OWNER = "libepoxy0"
|
||||
|
||||
# One nothing declares. Chosen from the gdk-pixbuf loaders' real dependencies,
|
||||
# which is how this class of miss actually reaches a user.
|
||||
UNDECLARED_SONAME = "libpng16.so.16"
|
||||
UNDECLARED_PATH = "/usr/lib/x86_64-linux-gnu/libpng16.so.16"
|
||||
UNDECLARED_OWNER = "libpng16-16"
|
||||
|
||||
PIXBUF_LOADER = "lib/gdk-pixbuf-2.0/2.10.0/loaders/libpixbufloader-png.so"
|
||||
|
||||
# Both stubs answer from the JSON the fixture writes. Keeping them as scripts
|
||||
# rather than mocks means the checker really forks, parses real ldd-shaped
|
||||
# output and really fails on the exit status it would see in CI.
|
||||
LDD_STUB = r"""import json, os, sys
|
||||
|
||||
answers = json.loads(open(os.environ["PLEZY_TEST_ANSWERS"], encoding="utf-8").read())
|
||||
for line in answers["ldd"].get(os.path.basename(sys.argv[1]), []):
|
||||
print("\t" + line)
|
||||
"""
|
||||
|
||||
DPKG_QUERY_STUB = r"""import json, os, sys
|
||||
|
||||
answers = json.loads(open(os.environ["PLEZY_TEST_ANSWERS"], encoding="utf-8").read())
|
||||
if sys.argv[1] == "-S":
|
||||
owner = answers["owners"].get(sys.argv[2], "")
|
||||
if not owner:
|
||||
sys.stderr.write("dpkg-query: no path found matching pattern %s\n" % sys.argv[2])
|
||||
sys.exit(1)
|
||||
print("%s: %s" % (owner, sys.argv[2]))
|
||||
else:
|
||||
# dpkg-query -W -f ${Provides} <package>
|
||||
sys.stdout.write(answers["provides"].get(sys.argv[-1], ""))
|
||||
"""
|
||||
|
||||
|
||||
def install_stub(scripts: Path, tools: Path, name: str, source: str) -> None:
|
||||
"""Put a fake `name` where the checker's PLEZY_HOST_TOOLS lookup will find it.
|
||||
|
||||
The stub itself is Python; the file the checker spawns has to be something
|
||||
the platform can execute directly, so it gets a wrapper. The two live in
|
||||
different directories because a Windows PATHEXT search would otherwise be
|
||||
free to pick the .py over the .bat.
|
||||
"""
|
||||
stub = scripts / f"{name}.py"
|
||||
stub.write_text(source, encoding="utf-8")
|
||||
if os.name == "nt":
|
||||
(tools / f"{name}.bat").write_text(f'@echo off\r\n"{sys.executable}" "{stub}" %*\r\n', encoding="utf-8")
|
||||
return
|
||||
wrapper = tools / name
|
||||
wrapper.write_text(f'#!/bin/sh\nexec "{sys.executable}" "{stub}" "$@"\n', encoding="utf-8")
|
||||
wrapper.chmod(0o755)
|
||||
|
||||
|
||||
def stage_bundle(
|
||||
staging: Path,
|
||||
ldd: dict[str, list[str]],
|
||||
owners: dict[str, str],
|
||||
shipped: tuple[str, ...] = (),
|
||||
install: tuple[str, ...] = ("ldd", "dpkg-query"),
|
||||
) -> tuple[Path, dict[str, str]]:
|
||||
"""A synthetic bundle plus host-tool stubs, and the environment that finds them."""
|
||||
bundle = staging / "bundle"
|
||||
bundle.mkdir()
|
||||
(bundle / "plezy").write_bytes(b"")
|
||||
for name in shipped:
|
||||
library = bundle / name
|
||||
library.parent.mkdir(parents=True, exist_ok=True)
|
||||
library.write_bytes(b"")
|
||||
|
||||
answers = staging / "answers.json"
|
||||
answers.write_text(json.dumps({"ldd": ldd, "owners": owners, "provides": {}}), encoding="utf-8")
|
||||
scripts, tools = staging / "stubs", staging / "tools"
|
||||
scripts.mkdir()
|
||||
tools.mkdir()
|
||||
for name, source in (("ldd", LDD_STUB), ("dpkg-query", DPKG_QUERY_STUB)):
|
||||
if name in install:
|
||||
install_stub(scripts, tools, name, source)
|
||||
|
||||
return bundle, {
|
||||
**os.environ,
|
||||
"PLEZY_HOST_TOOLS": str(tools),
|
||||
"PLEZY_TEST_ANSWERS": str(answers),
|
||||
}
|
||||
|
||||
|
||||
class BundleHostDepsGuardTest(unittest.TestCase):
|
||||
def _check(
|
||||
self,
|
||||
ldd: dict[str, list[str]],
|
||||
owners: dict[str, str],
|
||||
shipped: tuple[str, ...] = (),
|
||||
cwd: Path | None = None,
|
||||
install: tuple[str, ...] = ("ldd", "dpkg-query"),
|
||||
path: str | None = None,
|
||||
) -> subprocess.CompletedProcess[str]:
|
||||
"""Stage a synthetic bundle plus host-tool stubs and run the real checker."""
|
||||
with tempfile.TemporaryDirectory(prefix="plezy-bundle-deps-test-") as directory:
|
||||
bundle, env = stage_bundle(Path(directory), ldd, owners, shipped, install)
|
||||
if path is not None:
|
||||
env["PATH"] = path
|
||||
|
||||
return subprocess.run(
|
||||
[sys.executable, str(CHECKER), str(bundle)],
|
||||
cwd=cwd or ROOT,
|
||||
check=False,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
env=env,
|
||||
)
|
||||
|
||||
def test_a_bundle_whose_host_libraries_are_all_declared_passes(self) -> None:
|
||||
result = self._check(
|
||||
ldd={
|
||||
"plezy": [
|
||||
# No "=>" on this one: it must not be read as a soname.
|
||||
"linux-vdso.so.1 (0x00007ffd1b3fe000)",
|
||||
f"{DECLARED_SONAME} => {DECLARED_PATH} (0x00007f9c2c000000)",
|
||||
"libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f9c2b800000)",
|
||||
# Resolved from the host copy, but the bundle ships its own.
|
||||
"libmpv.so.2 => /usr/lib/x86_64-linux-gnu/libmpv.so.2 (0x00007f9c2b400000)",
|
||||
],
|
||||
"libgiognutls.so": [f"{DECLARED_SONAME} => {DECLARED_PATH} (0x00007f9c2c000000)"],
|
||||
},
|
||||
owners={DECLARED_PATH: DECLARED_OWNER},
|
||||
shipped=("lib/libmpv.so.2", "lib/gio/modules/libgiognutls.so"),
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertIn("every one of the 1 host libraries", result.stdout)
|
||||
|
||||
def test_an_undeclared_host_library_is_named(self) -> None:
|
||||
result = self._check(
|
||||
ldd={"plezy": [f"{UNDECLARED_SONAME} => {UNDECLARED_PATH} (0x00007f9c2c000000)"]},
|
||||
owners={UNDECLARED_PATH: UNDECLARED_OWNER},
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stdout)
|
||||
self.assertIn(UNDECLARED_SONAME, result.stderr)
|
||||
self.assertIn(f"deb package '{UNDECLARED_OWNER}'", result.stderr)
|
||||
|
||||
def test_a_library_declared_for_deb_but_not_for_rpm_is_still_rejected(self) -> None:
|
||||
"""Every other negative fixture fails on the Debian arm first.
|
||||
|
||||
Fedora and Arch names cannot be resolved on this runner, so they are the
|
||||
half most likely to be forgotten - and a package that installs on Fedora
|
||||
and then cannot start is exactly as broken as one that fails on Debian.
|
||||
libGLESv2 is the live example: it is mapped to libglvnd-gles, which is
|
||||
deliberately not declared because nothing links GLES today, so the moment
|
||||
something does the guard has to say so rather than wave it through on the
|
||||
strength of a satisfied deb dependency.
|
||||
"""
|
||||
soname = "libGLESv2.so.2"
|
||||
path = f"/usr/lib/x86_64-linux-gnu/{soname}"
|
||||
result = self._check(
|
||||
ldd={"plezy": [f"{soname} => {path} (0x00007f9c2c000000)"]},
|
||||
# Owned by a package the deb list does declare, so only the non-deb
|
||||
# half can be what rejects this.
|
||||
owners={path: "libegl1"},
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stdout)
|
||||
self.assertIn("libglvnd-gles", result.stderr)
|
||||
self.assertIn("rpm", result.stderr)
|
||||
# Proving the deb arm was satisfied, so the rejection came from the other.
|
||||
self.assertNotIn("comes from deb package", result.stderr)
|
||||
|
||||
def test_a_walk_that_finds_nothing_is_not_a_pass(self) -> None:
|
||||
"""An empty result means the walk failed, not that nothing is needed.
|
||||
|
||||
bundle-libs.sh always leaves the graphics stack to the host, so a real
|
||||
bundle cannot need zero host libraries. Reporting success here would make
|
||||
every later breakage invisible, which is the worst thing a guard can do.
|
||||
"""
|
||||
result = self._check(ldd={"plezy": []}, owners={})
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stdout)
|
||||
self.assertIn("proved nothing", result.stderr)
|
||||
|
||||
def test_a_library_reached_only_through_a_nested_module_is_still_checked(self) -> None:
|
||||
"""The gdk-pixbuf loaders sit two directories below lib/, and dlopen finds them.
|
||||
|
||||
Nothing else in the bundle links libpng, so a scan that stops at
|
||||
lib/*.so* sees a fully declared bundle and a user sees the loader fail.
|
||||
"""
|
||||
result = self._check(
|
||||
ldd={
|
||||
"plezy": [f"{DECLARED_SONAME} => {DECLARED_PATH} (0x00007f9c2c000000)"],
|
||||
"libpixbufloader-png.so": [f"{UNDECLARED_SONAME} => {UNDECLARED_PATH} (0x00007f9c2b000000)"],
|
||||
},
|
||||
owners={DECLARED_PATH: DECLARED_OWNER, UNDECLARED_PATH: UNDECLARED_OWNER},
|
||||
shipped=(PIXBUF_LOADER,),
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stdout)
|
||||
self.assertIn(UNDECLARED_SONAME, result.stderr)
|
||||
|
||||
def test_a_nested_module_cannot_satisfy_a_top_level_dependency(self) -> None:
|
||||
"""Only lib/ is on the loader path, so only lib/ can make a soname bundled.
|
||||
|
||||
plezy carries RPATH $ORIGIN/lib and plezy.sh exports $INSTALL_DIR/lib.
|
||||
The gdk-pixbuf loaders two directories below are opened by explicit
|
||||
path and resolve nothing for the executable, so a module whose basename
|
||||
happens to equal a host soname must not suppress it - otherwise the
|
||||
package under-declares and the check still passes.
|
||||
"""
|
||||
result = self._check(
|
||||
ldd={
|
||||
"plezy": [
|
||||
f"{DECLARED_SONAME} => {DECLARED_PATH} (0x00007f9c2c000000)",
|
||||
f"{UNDECLARED_SONAME} => {UNDECLARED_PATH} (0x00007f9c2b000000)",
|
||||
]
|
||||
},
|
||||
owners={DECLARED_PATH: DECLARED_OWNER, UNDECLARED_PATH: UNDECLARED_OWNER},
|
||||
shipped=(f"lib/gdk-pixbuf-2.0/2.10.0/loaders/{UNDECLARED_SONAME}",),
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stdout)
|
||||
self.assertIn(UNDECLARED_SONAME, result.stderr)
|
||||
self.assertIn(f"deb package '{UNDECLARED_OWNER}'", result.stderr)
|
||||
|
||||
def test_a_soname_the_bundle_ships_is_not_a_missing_dependency(self) -> None:
|
||||
"""ldd on a bundled library in isolation cannot see its siblings.
|
||||
|
||||
Bundled objects carry no RUNPATH - bundle-libs.sh copies and strips - so
|
||||
a library that exists nowhere but the bundle reads as `not found` when
|
||||
ldd is pointed at one of them directly. At runtime the executable's own
|
||||
$ORIGIN/lib resolves it. libshaderc_shared is the real instance: no
|
||||
distro package ships it, which is why both workflows copy it by hand, so
|
||||
faulting it here would fail the release for a library that is present.
|
||||
"""
|
||||
result = self._check(
|
||||
ldd={
|
||||
"plezy": [f"{DECLARED_SONAME} => {DECLARED_PATH} (0x00007f9c2c000000)"],
|
||||
# The bundled libmpv needs a bundled shaderc, and sees nothing.
|
||||
"libmpv.so.2": ["libshaderc_shared.so.1 => not found"],
|
||||
},
|
||||
owners={DECLARED_PATH: DECLARED_OWNER},
|
||||
shipped=("lib/libmpv.so.2", "lib/libshaderc_shared.so.1"),
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
|
||||
self.assertNotIn("resolves to nothing", result.stderr)
|
||||
|
||||
def test_a_library_that_resolves_to_nothing_fails(self) -> None:
|
||||
"""`=> not found` is the failure this check exists to prevent, already happened."""
|
||||
result = self._check(
|
||||
ldd={"libplezyextra.so.1": ["libfoo.so.1 => not found"]},
|
||||
owners={},
|
||||
shipped=("lib/libplezyextra.so.1",),
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stdout)
|
||||
self.assertIn("libfoo.so.1", result.stderr)
|
||||
self.assertIn("resolves to nothing", result.stderr)
|
||||
|
||||
def test_the_checker_reads_build_packages_from_any_working_directory(self) -> None:
|
||||
"""--root defaulted to cwd, so running the check from the build tree crashed."""
|
||||
result = self._check(
|
||||
ldd={"plezy": [f"{DECLARED_SONAME} => {DECLARED_PATH} (0x00007f9c2c000000)"]},
|
||||
owners={DECLARED_PATH: DECLARED_OWNER},
|
||||
cwd=Path(tempfile.gettempdir()),
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
|
||||
def test_a_host_without_dpkg_query_fails_instead_of_passing(self) -> None:
|
||||
"""Fedora and Arch have no dpkg-query, so the check cannot run there.
|
||||
|
||||
Every answer it gives comes from ldd and dpkg-query. Missing one, the
|
||||
walk would find nothing and report a bundle needing nothing from the
|
||||
host - so the absence has to be an error the reader can act on, never a
|
||||
skip and never a traceback.
|
||||
"""
|
||||
result = self._check(
|
||||
ldd={"plezy": [f"{DECLARED_SONAME} => {DECLARED_PATH} (0x00007f9c2c000000)"]},
|
||||
owners={DECLARED_PATH: DECLARED_OWNER},
|
||||
install=("ldd",),
|
||||
path="",
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stdout)
|
||||
self.assertIn("dpkg-query not found", result.stderr)
|
||||
self.assertIn("Debian or Ubuntu host", result.stderr)
|
||||
self.assertNotIn("Traceback", result.stderr)
|
||||
|
||||
|
||||
class BuildPackagesGuardWiringTest(unittest.TestCase):
|
||||
"""build-packages.py is the only path a maintainer packaging by hand takes.
|
||||
|
||||
CI ran the guard as its own step, so the script itself never did, and its own
|
||||
error message invited the standalone path that skipped it.
|
||||
"""
|
||||
|
||||
def test_an_undeclared_host_library_stops_packaging(self) -> None:
|
||||
with tempfile.TemporaryDirectory(prefix="plezy-packaging-test-") as directory:
|
||||
staging = Path(directory)
|
||||
bundle, env = stage_bundle(
|
||||
staging,
|
||||
ldd={"plezy": [f"{UNDECLARED_SONAME} => {UNDECLARED_PATH} (0x00007f9c2c000000)"]},
|
||||
owners={UNDECLARED_PATH: UNDECLARED_OWNER},
|
||||
shipped=("lib/libmpv.so.2",),
|
||||
)
|
||||
output = staging / "packages"
|
||||
output.mkdir()
|
||||
env |= {"BUILD_DIR": str(bundle), "OUTPUT_DIR": str(output)}
|
||||
|
||||
result = subprocess.run(
|
||||
[sys.executable, str(BUILD_PACKAGES)],
|
||||
cwd=ROOT,
|
||||
check=False,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
env=env,
|
||||
)
|
||||
produced = sorted(path.name for path in output.iterdir())
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stdout)
|
||||
self.assertIn(UNDECLARED_SONAME, result.stderr)
|
||||
self.assertIn("no packages were built", result.stdout)
|
||||
self.assertEqual(produced, [])
|
||||
|
||||
def test_the_opt_out_says_out_loud_what_it_costs(self) -> None:
|
||||
"""The escape hatch for dpkg-query-less hosts must announce itself.
|
||||
|
||||
A skip nobody can see in the log is indistinguishable from the guard
|
||||
having passed, which is the whole failure this wiring removes.
|
||||
"""
|
||||
with tempfile.TemporaryDirectory(prefix="plezy-packaging-optout-test-") as directory:
|
||||
staging = Path(directory)
|
||||
# A copy, not the checkout's script: past this point main() writes
|
||||
# generated icons next to itself, and a test has no business dirtying
|
||||
# the working tree. Everything under test happens before that, and the
|
||||
# copy's missing sibling guard would fail loudly if it did run.
|
||||
packaging = staging / "linux/packaging"
|
||||
packaging.mkdir(parents=True)
|
||||
script = packaging / BUILD_PACKAGES.name
|
||||
script.write_text(BUILD_PACKAGES.read_text(encoding="utf-8"), encoding="utf-8")
|
||||
bundle = staging / "bundle"
|
||||
(bundle / "lib").mkdir(parents=True)
|
||||
(bundle / "lib/libmpv.so.2").write_bytes(b"")
|
||||
|
||||
result = subprocess.run(
|
||||
[sys.executable, str(script)],
|
||||
cwd=ROOT,
|
||||
check=False,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
env={
|
||||
**os.environ,
|
||||
"BUILD_DIR": str(bundle),
|
||||
"OUTPUT_DIR": str(staging / "packages"),
|
||||
"PLEZY_SKIP_HOST_DEP_CHECK": "1",
|
||||
},
|
||||
)
|
||||
|
||||
self.assertIn("PLEZY_SKIP_HOST_DEP_CHECK is set", result.stdout)
|
||||
self.assertIn("unverified", result.stdout)
|
||||
|
||||
def test_a_falsey_opt_out_does_not_skip(self) -> None:
|
||||
"""`=0` means "do not skip" to almost everyone, and must behave that way.
|
||||
|
||||
A bare non-empty test would read `0`, `false` and `no` as consent, which
|
||||
is the exact opposite of what the person typing them meant - and hands
|
||||
back the unverified package this wiring exists to withhold.
|
||||
"""
|
||||
for value in ("0", "false", "no", "off", ""):
|
||||
with self.subTest(value=value), tempfile.TemporaryDirectory(prefix="plezy-packaging-falsey-") as directory:
|
||||
staging = Path(directory)
|
||||
packaging = staging / "linux/packaging"
|
||||
packaging.mkdir(parents=True)
|
||||
script = packaging / BUILD_PACKAGES.name
|
||||
script.write_text(BUILD_PACKAGES.read_text(encoding="utf-8"), encoding="utf-8")
|
||||
bundle = staging / "bundle"
|
||||
(bundle / "lib").mkdir(parents=True)
|
||||
(bundle / "lib/libmpv.so.2").write_bytes(b"")
|
||||
|
||||
result = subprocess.run(
|
||||
[sys.executable, str(script)],
|
||||
cwd=ROOT,
|
||||
check=False,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
env={
|
||||
**os.environ,
|
||||
"BUILD_DIR": str(bundle),
|
||||
"OUTPUT_DIR": str(staging / "packages"),
|
||||
"PLEZY_SKIP_HOST_DEP_CHECK": value,
|
||||
},
|
||||
)
|
||||
|
||||
self.assertNotIn("PLEZY_SKIP_HOST_DEP_CHECK is set", result.stdout)
|
||||
# The guard ran, so packaging stopped on its verdict rather than
|
||||
# continuing past an unverified bundle.
|
||||
self.assertNotEqual(result.returncode, 0)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,334 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Behavior tests for the Linux package dependency guard.
|
||||
|
||||
The first case reproduces the bug that motivated the guard: the native video
|
||||
plane added wayland-client, wayland-egl and egl to the runner's link line while
|
||||
the hand-maintained depends lists went untouched.
|
||||
"""
|
||||
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
CHECKER = ROOT / "scripts/check_linux_package_deps.py"
|
||||
LINUX = ROOT / "linux"
|
||||
# Every file the checker reads, relative to the linux/ directory it is given.
|
||||
FIXTURE_FILES = (
|
||||
"runner/CMakeLists.txt",
|
||||
"CMakeLists.txt",
|
||||
"flutter/CMakeLists.txt",
|
||||
"packaging/build-packages.py",
|
||||
"packaging/bundle-libs.sh",
|
||||
)
|
||||
# The subset the checker parses for pkg_check_modules and the link graph. Each
|
||||
# one carries targets the others cannot see, so none of them is optional.
|
||||
CMAKE_INPUTS = ("runner/CMakeLists.txt", "CMakeLists.txt", "flutter/CMakeLists.txt")
|
||||
|
||||
|
||||
# Pins the walk's reach: gtk+-3.0, mpv, epoxy, glib-2.0, gio-2.0 and the three the
|
||||
# video plane added. A drop here means the parser stopped seeing something rather
|
||||
# than that a link was removed.
|
||||
FULL_WALK_SUMMARY = "(8 pkg-config links, 8 modules)"
|
||||
|
||||
# The success line has to name its own scope. A reader who takes it for a
|
||||
# whole-binary check trusts it past the plugin link edges it never walked, so
|
||||
# the wording is part of what this guard promises.
|
||||
SUCCESS_LINE = "linux/runner CMake dependency checks passed"
|
||||
|
||||
|
||||
class LinuxPackageDepsGuardTest(unittest.TestCase):
|
||||
def _run(self, omit: tuple[str, ...] = (), **edits: str) -> subprocess.CompletedProcess[str]:
|
||||
"""Copy the real linux/ inputs, apply edits, and check the copy."""
|
||||
with tempfile.TemporaryDirectory(prefix="plezy-linux-deps-test-") as directory:
|
||||
linux = Path(directory) / "linux"
|
||||
for name in FIXTURE_FILES:
|
||||
if name in omit:
|
||||
continue
|
||||
target = linux / name
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
target.write_text(edits.get(name, self._source(name)), encoding="utf-8")
|
||||
return subprocess.run(
|
||||
[sys.executable, str(CHECKER), str(linux)],
|
||||
cwd=ROOT,
|
||||
check=False,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
|
||||
def _source(self, name: str) -> str:
|
||||
return (LINUX / name).read_text(encoding="utf-8")
|
||||
|
||||
def _mutate(self, name: str, old: str, new: str) -> str:
|
||||
text = self._source(name).replace(old, new, 1)
|
||||
self.assertNotEqual(text, self._source(name), f"fixture mutation no longer matches: {old!r}")
|
||||
return text
|
||||
|
||||
def test_current_tree_passes(self) -> None:
|
||||
result = self._run()
|
||||
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertIn(SUCCESS_LINE, result.stdout)
|
||||
# Naming the artifact-level check is how the blind spot stays findable.
|
||||
self.assertIn("check-bundle-host-deps.py", result.stdout)
|
||||
self.assertIn(FULL_WALK_SUMMARY, result.stdout)
|
||||
|
||||
def test_a_missing_cmake_input_is_named_rather_than_skipped(self) -> None:
|
||||
"""Losing one of these used to shrink the walk instead of failing it.
|
||||
|
||||
`flutter` is defined in flutter/CMakeLists.txt and propagates GTK, GLIB
|
||||
and GIO; drop the file and those modules leave the graph, so a depends
|
||||
entry deleted with them goes unreported and the guard exits 0 having
|
||||
checked less than it says it did.
|
||||
"""
|
||||
for name in CMAKE_INPUTS:
|
||||
with self.subTest(missing=name):
|
||||
result = self._run(omit=(name,))
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stdout)
|
||||
self.assertIn(str(Path("linux") / name), result.stderr)
|
||||
self.assertIn("is missing", result.stderr)
|
||||
|
||||
def test_every_module_of_a_multi_module_call_is_checked(self) -> None:
|
||||
"""`pkg_check_modules(X REQUIRED IMPORTED_TARGET a b c)` is legal CMake naming three modules.
|
||||
|
||||
Every one of them must be checked: the guard has to reject the call once
|
||||
wayland-cursor is added to it, not just read wayland-client and stop.
|
||||
"""
|
||||
cmake = self._mutate(
|
||||
"runner/CMakeLists.txt",
|
||||
"pkg_check_modules(WAYLAND_CLIENT REQUIRED IMPORTED_TARGET wayland-client)",
|
||||
"pkg_check_modules(WAYLAND_CLIENT REQUIRED IMPORTED_TARGET wayland-client wayland-cursor)",
|
||||
)
|
||||
|
||||
result = self._run(**{"runner/CMakeLists.txt": cmake})
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stdout)
|
||||
self.assertIn("wayland-cursor", result.stderr)
|
||||
|
||||
def test_a_version_constrained_module_spec_still_names_its_module(self) -> None:
|
||||
"""`mpv>=0.40` is a legal moduleSpec whose module name is `mpv`.
|
||||
|
||||
The guard must strip the version constraint and still walk the full tree,
|
||||
rather than reporting `mpv>=0.40` as an undeclared package.
|
||||
"""
|
||||
cmake = self._mutate(
|
||||
"runner/CMakeLists.txt",
|
||||
"pkg_check_modules(MPV REQUIRED IMPORTED_TARGET mpv)",
|
||||
"pkg_check_modules(MPV REQUIRED IMPORTED_TARGET mpv>=0.40)",
|
||||
)
|
||||
|
||||
result = self._run(**{"runner/CMakeLists.txt": cmake})
|
||||
|
||||
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
|
||||
self.assertIn(FULL_WALK_SUMMARY, result.stdout)
|
||||
|
||||
# The three below are fail-open cases: each is legal CMake that a naive regex
|
||||
# silently drops, leaving the guard to exit 0 while a library goes undeclared.
|
||||
def test_a_comment_containing_a_paren_does_not_hide_the_rest_of_the_call(self) -> None:
|
||||
cmake = self._mutate(
|
||||
"runner/CMakeLists.txt",
|
||||
"target_link_libraries(${BINARY_NAME} PRIVATE PkgConfig::EGL)",
|
||||
"target_link_libraries(${BINARY_NAME} PRIVATE\n"
|
||||
" # host EGL (never bundled)\n"
|
||||
" PkgConfig::EGL\n"
|
||||
")",
|
||||
)
|
||||
packages = self._mutate("packaging/build-packages.py", '"libegl1",\n', "")
|
||||
|
||||
result = self._run(**{"runner/CMakeLists.txt": cmake, "packaging/build-packages.py": packages})
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stdout)
|
||||
self.assertIn("links egl but the deb package", result.stderr)
|
||||
|
||||
def test_a_generator_expression_still_names_its_target(self) -> None:
|
||||
cmake = self._mutate(
|
||||
"runner/CMakeLists.txt",
|
||||
"PRIVATE PkgConfig::EGL)",
|
||||
"PRIVATE $<LINK_ONLY:PkgConfig::EGL>)",
|
||||
)
|
||||
packages = self._mutate("packaging/build-packages.py", '"libegl1",\n', "")
|
||||
|
||||
result = self._run(**{"runner/CMakeLists.txt": cmake, "packaging/build-packages.py": packages})
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stdout)
|
||||
self.assertIn("links egl but the deb package", result.stderr)
|
||||
|
||||
def test_a_quoted_target_still_names_its_target(self) -> None:
|
||||
cmake = self._mutate(
|
||||
"runner/CMakeLists.txt",
|
||||
"PRIVATE PkgConfig::EGL)",
|
||||
'PRIVATE "PkgConfig::EGL")',
|
||||
)
|
||||
packages = self._mutate("packaging/build-packages.py", '"libegl1",\n', "")
|
||||
|
||||
result = self._run(**{"runner/CMakeLists.txt": cmake, "packaging/build-packages.py": packages})
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stdout)
|
||||
self.assertIn("links egl but the deb package", result.stderr)
|
||||
|
||||
def test_pkg_check_modules_options_may_precede_the_module_name(self) -> None:
|
||||
# CMake takes its options in any order; reading REQUIRED as the module
|
||||
# name fails closed but sends the reader hunting a package nobody ships.
|
||||
cmake = self._mutate(
|
||||
"runner/CMakeLists.txt",
|
||||
"pkg_check_modules(EGL REQUIRED IMPORTED_TARGET egl)",
|
||||
"pkg_check_modules(EGL IMPORTED_TARGET REQUIRED egl)",
|
||||
)
|
||||
|
||||
result = self._run(**{"runner/CMakeLists.txt": cmake})
|
||||
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertIn(FULL_WALK_SUMMARY, result.stdout)
|
||||
|
||||
def test_the_dependency_lists_before_the_video_plane_are_rejected(self) -> None:
|
||||
# Verbatim the deb list as it stood while the runner already linked
|
||||
# wayland-client, wayland-egl and egl: the shipped bug.
|
||||
packages = self._mutate(
|
||||
"packaging/build-packages.py",
|
||||
'"libwayland-client0",\n "libwayland-cursor0",\n'
|
||||
' "libwayland-egl1",\n "libegl1",\n',
|
||||
"",
|
||||
)
|
||||
|
||||
result = self._run(**{"packaging/build-packages.py": packages})
|
||||
|
||||
self.assertEqual(result.returncode, 1)
|
||||
self.assertIn("wayland-client", result.stderr)
|
||||
self.assertIn("wayland-egl", result.stderr)
|
||||
self.assertIn("libegl1", result.stderr)
|
||||
self.assertIn("can fail to start", result.stderr)
|
||||
|
||||
def test_a_bundled_module_needs_no_declared_dependency(self) -> None:
|
||||
"""libmpv ships inside the package, so the walk must not demand a dep.
|
||||
|
||||
The runner links pkg-config `mpv` and always will; what changed is that
|
||||
the library travels with us, because the plane needs the pinned
|
||||
Wayland-enabled build. Declaring a host mpv would be the bug now.
|
||||
"""
|
||||
result = self._run()
|
||||
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertNotIn("mpv", result.stderr)
|
||||
# Still counted: a bundled module dropping out of the walk is a parser
|
||||
# regression, and the summary is what would show it.
|
||||
self.assertIn(FULL_WALK_SUMMARY, result.stdout)
|
||||
|
||||
def test_one_missing_distro_is_rejected(self) -> None:
|
||||
# A dependency declared for deb but forgotten for rpm still ships broken
|
||||
# on Fedora, so per-distro coverage is the unit, not per-library.
|
||||
packages = self._mutate("packaging/build-packages.py", '"libglvnd-egl",\n', "")
|
||||
|
||||
result = self._run(**{"packaging/build-packages.py": packages})
|
||||
|
||||
self.assertEqual(result.returncode, 1)
|
||||
self.assertIn("rpm package does not depend on 'libglvnd-egl'", result.stderr)
|
||||
self.assertNotIn("deb package does not depend", result.stderr)
|
||||
|
||||
def test_pacman_shared_wayland_package_counts_for_both_modules(self) -> None:
|
||||
# Arch has no separate libwayland-egl, so one entry has to satisfy two
|
||||
# modules. Dropping it must fail for both rather than neither.
|
||||
packages = self._mutate("packaging/build-packages.py", '"wayland",\n', "")
|
||||
|
||||
result = self._run(**{"packaging/build-packages.py": packages})
|
||||
|
||||
self.assertEqual(result.returncode, 1)
|
||||
self.assertIn("links wayland-client but the pacman package", result.stderr)
|
||||
self.assertIn("links wayland-egl but the pacman package", result.stderr)
|
||||
|
||||
def test_new_pkgconfig_link_without_a_package_mapping_is_rejected(self) -> None:
|
||||
# The forward-looking half: the next library added to the runner has to
|
||||
# name its runtime package before it can ship.
|
||||
cmake = self._mutate(
|
||||
"runner/CMakeLists.txt",
|
||||
"target_link_libraries(${BINARY_NAME} PRIVATE PkgConfig::EGL)",
|
||||
"pkg_check_modules(PIPEWIRE REQUIRED IMPORTED_TARGET libpipewire-0.3)\n"
|
||||
"target_link_libraries(${BINARY_NAME} PRIVATE PkgConfig::EGL)\n"
|
||||
"target_link_libraries(${BINARY_NAME} PRIVATE PkgConfig::PIPEWIRE)",
|
||||
)
|
||||
|
||||
result = self._run(**{"runner/CMakeLists.txt": cmake})
|
||||
|
||||
self.assertEqual(result.returncode, 1)
|
||||
self.assertIn("libpipewire-0.3", result.stderr)
|
||||
self.assertIn("RUNTIME_PACKAGES", result.stderr)
|
||||
|
||||
def test_a_grouped_link_is_read_past_the_first_target(self) -> None:
|
||||
# CMake happily takes several targets in one call. A parse that stopped at
|
||||
# the first would wave the rest through while they sat on the link line.
|
||||
cmake = self._mutate(
|
||||
"runner/CMakeLists.txt",
|
||||
"target_link_libraries(${BINARY_NAME} PRIVATE PkgConfig::EGL)",
|
||||
"pkg_check_modules(PIPEWIRE REQUIRED IMPORTED_TARGET libpipewire-0.3)\n"
|
||||
"target_link_libraries(${BINARY_NAME} PRIVATE PkgConfig::EGL PkgConfig::PIPEWIRE)",
|
||||
)
|
||||
|
||||
result = self._run(**{"runner/CMakeLists.txt": cmake})
|
||||
|
||||
self.assertEqual(result.returncode, 1)
|
||||
self.assertIn("libpipewire-0.3", result.stderr)
|
||||
|
||||
def test_a_transitive_link_through_an_internal_target_is_followed(self) -> None:
|
||||
# wayland_protocols already hands PkgConfig::WAYLAND_CLIENT to whatever
|
||||
# links it. A static library's dependencies land on the consumer's link
|
||||
# line, so stopping at the internal target would miss a real dependency.
|
||||
cmake = self._mutate(
|
||||
"runner/CMakeLists.txt",
|
||||
"target_link_libraries(wayland_protocols PUBLIC PkgConfig::WAYLAND_CLIENT)",
|
||||
"pkg_check_modules(PIPEWIRE REQUIRED IMPORTED_TARGET libpipewire-0.3)\n"
|
||||
"target_link_libraries(wayland_protocols PUBLIC PkgConfig::WAYLAND_CLIENT "
|
||||
"PkgConfig::PIPEWIRE)",
|
||||
)
|
||||
|
||||
result = self._run(**{"runner/CMakeLists.txt": cmake})
|
||||
|
||||
self.assertEqual(result.returncode, 1)
|
||||
self.assertIn("libpipewire-0.3", result.stderr)
|
||||
|
||||
def test_dropping_the_direct_wayland_client_link_still_finds_it(self) -> None:
|
||||
# The runner names wayland-client directly *and* gets it through
|
||||
# wayland_protocols. Removing the direct link must not silence the guard,
|
||||
# because the binary still links the library either way.
|
||||
cmake = self._mutate(
|
||||
"runner/CMakeLists.txt",
|
||||
"target_link_libraries(${BINARY_NAME} PRIVATE PkgConfig::WAYLAND_CLIENT)\n",
|
||||
"",
|
||||
)
|
||||
packages = self._mutate("packaging/build-packages.py", '"libwayland-client0",\n', "")
|
||||
|
||||
result = self._run(
|
||||
**{"runner/CMakeLists.txt": cmake, "packaging/build-packages.py": packages}
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 1)
|
||||
self.assertIn("links wayland-client but the deb package", result.stderr)
|
||||
|
||||
def test_a_link_with_no_pkg_check_modules_is_rejected(self) -> None:
|
||||
cmake = self._mutate(
|
||||
"runner/CMakeLists.txt",
|
||||
"pkg_check_modules(EGL REQUIRED IMPORTED_TARGET egl)",
|
||||
"# EGL declaration removed",
|
||||
)
|
||||
|
||||
result = self._run(**{"runner/CMakeLists.txt": cmake})
|
||||
|
||||
self.assertEqual(result.returncode, 1)
|
||||
self.assertIn("PkgConfig::EGL is linked into the runner", result.stderr)
|
||||
|
||||
def test_bundling_the_excluded_libraries_is_rejected(self) -> None:
|
||||
# The guard's premise is that these come from the host. If bundle-libs.sh
|
||||
# starts shipping them, the demand for a depends entry needs rethinking
|
||||
# rather than silently continuing to hold.
|
||||
bundle = self._mutate("packaging/bundle-libs.sh", r"libwayland.*\.so|", "")
|
||||
|
||||
result = self._run(**{"packaging/bundle-libs.sh": bundle})
|
||||
|
||||
self.assertEqual(result.returncode, 1)
|
||||
self.assertIn("no longer excludes", result.stderr)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,229 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Behavior tests for the package metadata read-back guard.
|
||||
|
||||
The guard exists because nothing between build-packages.py and the upload read
|
||||
anything back out of what fpm wrote: a dropped or renamed `--depends` shipped a
|
||||
package that installs cleanly and dies in the loader, with every earlier check
|
||||
green. These tests hold that line, and the substring case below is the specific
|
||||
regression the shell version it replaced once had - `libegl1` is a substring of
|
||||
`libegl1-mesa`, so a package declaring neither used to pass.
|
||||
|
||||
dpkg-deb, rpm and bsdtar only answer on a machine that has them, so the fixtures
|
||||
put stubs first on PATH and let the real script fork them. That keeps "an rpm
|
||||
missing libdrm" a fixture rather than a machine, and keeps every assertion
|
||||
running against the script CI runs.
|
||||
"""
|
||||
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
# The Windows-safe stub installer, rather than a second copy of its PATHEXT
|
||||
# reasoning. scripts/ is sys.path[0] however this file is invoked.
|
||||
from test_check_bundle_host_deps import install_stub
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
CHECKER = ROOT / "linux/packaging/check-package-deps.py"
|
||||
BUILD_PACKAGES = ROOT / "linux/packaging/build-packages.py"
|
||||
|
||||
# The expected names come from build-packages.py the same way the guard reads
|
||||
# them, so a library added there is exercised here without editing a fixture.
|
||||
_spec = importlib.util.spec_from_file_location("build_packages", BUILD_PACKAGES)
|
||||
PACKAGING = importlib.util.module_from_spec(_spec)
|
||||
_spec.loader.exec_module(PACKAGING)
|
||||
DISTROS = PACKAGING.DISTROS
|
||||
NAME = PACKAGING.METADATA["name"]
|
||||
|
||||
# The tool each package format is read with, which is also what the fixtures stub.
|
||||
TOOLS = {"deb": "dpkg-deb", "rpm": "rpm", "pacman": "bsdtar"}
|
||||
|
||||
# One stub per tool: it prints the fixture's answer for its own name, or fails
|
||||
# the way an unreadable archive does. Baking the name in keeps the stub blind to
|
||||
# the arguments, so it cannot accidentally pass by echoing its input.
|
||||
STUB = """import json, os, sys
|
||||
|
||||
answers = json.loads(open(os.environ["PLEZY_TEST_ANSWERS"], encoding="utf-8").read())
|
||||
answer = answers.get({name!r})
|
||||
if answer is None:
|
||||
sys.stderr.write({name!r} + ": cannot read this archive\\n")
|
||||
sys.exit(1)
|
||||
sys.stdout.write(answer)
|
||||
"""
|
||||
|
||||
|
||||
def declared_first_names(distro: str) -> list[str]:
|
||||
"""What a correct package declares: one acceptable name per dependency."""
|
||||
return [dependency.split("|")[0].strip() for dependency in DISTROS[distro]["depends"]]
|
||||
|
||||
|
||||
def decorate(names: list[str], version: str, qualifier: str) -> list[str]:
|
||||
"""Dress the first two names the way the real tools report them.
|
||||
|
||||
A constraint, an architecture qualifier and rpm's soname decoration are noise
|
||||
around a package name. Applying them to fixture names rather than asserting
|
||||
on a hand-written blob means the stripping is tested against the list the
|
||||
packages really declare.
|
||||
"""
|
||||
if len(names) < 2:
|
||||
return names
|
||||
return [f"{names[0]} {version}", f"{names[1]}{qualifier}", *names[2:]]
|
||||
|
||||
|
||||
def deb_metadata(names: list[str] | None = None) -> str:
|
||||
"""`dpkg-deb -f ... Depends` output."""
|
||||
names = declared_first_names("deb") if names is None else names
|
||||
return ", ".join(decorate(names, "(>= 3.24.0)", ":amd64")) + "\n"
|
||||
|
||||
|
||||
def rpm_metadata(names: list[str] | None = None) -> str:
|
||||
"""`rpm -qpR` output, including the requires rpm adds by itself."""
|
||||
names = declared_first_names("rpm") if names is None else names
|
||||
automatic = ["/bin/sh", "libc.so.6(GLIBC_2.34)(64bit)", "rpmlib(PayloadIsXz) <= 5.2-1"]
|
||||
return "\n".join(automatic + decorate(names, ">= 3.24", "(x86-64)")) + "\n"
|
||||
|
||||
|
||||
def pkginfo_metadata(names: list[str] | None = None) -> str:
|
||||
"""A whole .PKGINFO, so the `depend = ` filter is what isolates the names."""
|
||||
names = declared_first_names("pacman") if names is None else names
|
||||
header = [f"pkgname = {NAME}", "pkgver = 1.2.3-1", "arch = x86_64"]
|
||||
depends = [f"depend = {name}" for name in decorate(names, ">=3.24", "")]
|
||||
return "\n".join(header + depends) + "\n"
|
||||
|
||||
|
||||
def correct_metadata() -> dict[str, str]:
|
||||
return {"dpkg-deb": deb_metadata(), "rpm": rpm_metadata(), "bsdtar": pkginfo_metadata()}
|
||||
|
||||
|
||||
class PackageDepsReadBackTest(unittest.TestCase):
|
||||
def _check(
|
||||
self,
|
||||
metadata: dict[str, str | None],
|
||||
arch: str = "x64",
|
||||
produce: tuple[str, ...] = ("deb", "rpm", "pacman"),
|
||||
install: tuple[str, ...] = ("dpkg-deb", "rpm", "bsdtar"),
|
||||
path: str | None = None,
|
||||
) -> subprocess.CompletedProcess[str]:
|
||||
"""Stage packages plus tool stubs and run the real guard over them."""
|
||||
with tempfile.TemporaryDirectory(prefix="plezy-package-deps-test-") as directory:
|
||||
staging = Path(directory)
|
||||
packages = staging / "packages"
|
||||
packages.mkdir()
|
||||
for distro in produce:
|
||||
(packages / f"{NAME}-linux-{arch}.{DISTROS[distro]['ext']}").write_bytes(b"")
|
||||
|
||||
answers = staging / "answers.json"
|
||||
answers.write_text(json.dumps(metadata), encoding="utf-8")
|
||||
scripts, tools = staging / "stubs", staging / "tools"
|
||||
scripts.mkdir()
|
||||
tools.mkdir()
|
||||
for tool in install:
|
||||
install_stub(scripts, tools, tool, STUB.format(name=tool))
|
||||
|
||||
return subprocess.run(
|
||||
[sys.executable, str(CHECKER), str(packages), "--arch", arch],
|
||||
cwd=ROOT,
|
||||
check=False,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
env={
|
||||
**os.environ,
|
||||
# The stubs first, so a machine that really has dpkg-deb
|
||||
# answers from the fixture. `path=""` leaves only the stubs,
|
||||
# which is how a tool is made genuinely absent.
|
||||
"PATH": str(tools) + os.pathsep + (os.environ.get("PATH", "") if path is None else path),
|
||||
"PLEZY_TEST_ANSWERS": str(answers),
|
||||
},
|
||||
)
|
||||
|
||||
def test_packages_carrying_every_declared_dependency_pass(self) -> None:
|
||||
result = self._check(correct_metadata())
|
||||
|
||||
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
|
||||
self.assertIn("survived fpm", result.stdout)
|
||||
# Version constraints, deb's arch qualifier and rpm's soname decorations
|
||||
# are noise around a name, not a different package.
|
||||
self.assertNotIn("::error::", result.stderr)
|
||||
|
||||
def test_a_dependency_fpm_dropped_is_named(self) -> None:
|
||||
"""The regression the guard exists for: a name that reached fpm and not the package."""
|
||||
for distro in DISTROS:
|
||||
with self.subTest(distro=distro):
|
||||
dropped = declared_first_names(distro)[0]
|
||||
kept = declared_first_names(distro)[1:]
|
||||
metadata = correct_metadata()
|
||||
metadata[TOOLS[distro]] = {
|
||||
"deb": deb_metadata,
|
||||
"rpm": rpm_metadata,
|
||||
"pacman": pkginfo_metadata,
|
||||
}[distro](kept)
|
||||
|
||||
result = self._check(metadata)
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stdout)
|
||||
self.assertIn(f"the {distro} package does not require {dropped}", result.stderr)
|
||||
self.assertNotIn("survived fpm", result.stdout)
|
||||
|
||||
def test_a_longer_package_name_does_not_satisfy_a_shorter_one(self) -> None:
|
||||
"""`libegl1-mesa` is not `libegl1`, however much of one it contains."""
|
||||
names = [f"{name}-mesa" if name == "libegl1" else name for name in declared_first_names("deb")]
|
||||
self.assertIn("libegl1-mesa", names, "the deb list no longer contains libegl1")
|
||||
metadata = correct_metadata() | {"dpkg-deb": deb_metadata(names)}
|
||||
|
||||
result = self._check(metadata)
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stdout)
|
||||
self.assertIn("the deb package does not require libegl1", result.stderr)
|
||||
|
||||
def test_a_package_fpm_never_wrote_fails(self) -> None:
|
||||
result = self._check(correct_metadata(), produce=("deb", "pacman"))
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stdout)
|
||||
self.assertIn(f"{NAME}-linux-x64.rpm", result.stderr)
|
||||
self.assertIn("was not produced", result.stderr)
|
||||
self.assertNotIn("survived fpm", result.stdout)
|
||||
|
||||
def test_an_unreadable_pkginfo_is_not_a_missing_dependency(self) -> None:
|
||||
""""the archive member was not found" must stay separable from "fpm dropped everything"."""
|
||||
result = self._check(correct_metadata() | {"bsdtar": ""})
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stdout)
|
||||
self.assertIn("could not read .PKGINFO", result.stderr)
|
||||
self.assertNotIn("does not require", result.stderr)
|
||||
|
||||
def test_a_tool_that_cannot_run_is_a_failure_not_a_pass(self) -> None:
|
||||
"""A guard that proved nothing must never report that it proved something.
|
||||
|
||||
Both halves matter: a reader absent from the machine, and one present but
|
||||
refusing the archive. Either way nothing was read, so nothing is declared.
|
||||
"""
|
||||
for distro, tool in TOOLS.items():
|
||||
with self.subTest(missing=tool):
|
||||
installed = tuple(name for name in TOOLS.values() if name != tool)
|
||||
absent = self._check(correct_metadata(), install=installed, path="")
|
||||
self.assertEqual(absent.returncode, 1, absent.stdout)
|
||||
self.assertIn(f"{tool} is not installed", absent.stderr)
|
||||
self.assertNotIn("survived fpm", absent.stdout)
|
||||
|
||||
with self.subTest(failing=tool):
|
||||
broken = self._check(correct_metadata() | {tool: None})
|
||||
self.assertEqual(broken.returncode, 1, broken.stdout)
|
||||
self.assertIn(f"{tool} failed", broken.stderr)
|
||||
self.assertNotIn(f"the {distro} package does not require", broken.stderr)
|
||||
|
||||
def test_the_release_architecture_is_read_from_its_own_files(self) -> None:
|
||||
"""The release job ships arm64 too, and x64 filenames must not stand in for it."""
|
||||
result = self._check(correct_metadata(), arch="arm64")
|
||||
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
|
||||
|
||||
missing = self._check(correct_metadata(), arch="x64", produce=())
|
||||
self.assertEqual(missing.returncode, 1, missing.stdout)
|
||||
self.assertIn(f"{NAME}-linux-x64.deb", missing.stderr)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -88,6 +88,12 @@ def _validate_native(root: Path, errors: list[str]) -> None:
|
||||
_require_text(value.get("provenance"), f"{label}.provenance", errors)
|
||||
if url and not url.startswith("https://"):
|
||||
errors.append(f"{label}.url: production source must use HTTPS")
|
||||
# A fallback source is optional, but it is a production source when it is
|
||||
# used, so it answers to the same rule as the primary.
|
||||
mirror = value.get("mirror")
|
||||
if mirror is not None:
|
||||
if not isinstance(mirror, str) or not mirror.startswith("https://"):
|
||||
errors.append(f"{label}.mirror: production source must use HTTPS")
|
||||
if version and url and name in {"ffmpeg", "mpv", "simdutf"} and version not in url:
|
||||
errors.append(f"{label}.url: must identify declared version {version}")
|
||||
if kind == "archive":
|
||||
|
||||
Reference in New Issue
Block a user