From 4292633820a364d45a824bb519a9c8f980ac4cff Mon Sep 17 00:00:00 2001 From: Daniel Hilst <392820+dhilst@users.noreply.github.com> Date: Thu, 6 Aug 2026 11:20:27 -0300 Subject: [PATCH] feat(xcat-dep): add the Ubuntu/Debian sbuild dependency-build matrix The Ubuntu/Debian .deb dependency build lived only in the xCAT CI (xcat-core-ci-cd), so xcat-dep could not build its apt dependency packages standalone. Bring the two CI-only drivers into the repo, alongside the existing build-apt-repo.sh + per-package make_deb.sh: - mk-dep-chroots.sh : create the per-codename sbuild chroots (root, per host). - build-dep-debs.sh : build every dep .deb per codename INSIDE the matching sbuild chroot (correct libc/toolchain) and stage them for build-apt-repo.sh; xCAT-genesis-base (Arch:all) built once and staged into every codename (cross-arch netboot, #7610). Document the Ubuntu/Debian sbuild flow in BUILD.md. The scripts still carry CI-specific path assumptions (a shared-tree bind-mount in mk-dep-chroots.sh); genericizing them is tracked separately, and the CI keeps its own copies until this lands upstream. Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com> --- BUILD.md | 43 +++++++++++++ build-dep-debs.sh | 154 ++++++++++++++++++++++++++++++++++++++++++++++ mk-dep-chroots.sh | 51 +++++++++++++++ 3 files changed, 248 insertions(+) create mode 100755 build-dep-debs.sh create mode 100755 mk-dep-chroots.sh diff --git a/BUILD.md b/BUILD.md index 43fd88f..2c7c5a5 100644 --- a/BUILD.md +++ b/BUILD.md @@ -260,6 +260,49 @@ find /build-output/mockbuild-all//build-logs -type f | sort - `mock target not found` - Validate with `mock -r --print-root-path` and install the required mock config packages. +# Ubuntu / Debian dependency build (apt, sbuild) + +The EL/SUSE path above uses `mockbuild-all.pl` (rpm). The Ubuntu/Debian dependency packages are +built as **.deb** and assembled into a signed **apt** repository, with the compiled deps built +**per codename inside matching `sbuild` chroots** so each binary links against that release's +libc/toolchain (a noble/glibc-2.39 binary won't run on focal/glibc-2.31). + +Codename ↔ version: `focal`=20.04, `jammy`=22.04, `noble`=24.04, `resolute`=26.04. + +## Scripts + +- **`mk-dep-chroots.sh`** — create the per-codename `sbuild` chroots the deb build needs. Run as + **root** on the Ubuntu build host (the amd64 host for `amd64`, the ppc host for `ppc64el`); one + chroot per codename (`--sbuild`). Idempotent. Gotchas baked in: `archive.ubuntu.com` + times out from some hosts (use a fast mirror, override with `MIRROR=`); the chroot `sources.list` + must carry **main + universe** (build-deps like `quilt` live in universe); a missing debootstrap + script is symlinked to the generic one. Env: `MIRROR`, `DEB_ARCH` (default `amd64`), `CODENAMES` + (default `focal jammy noble resolute`). + +- **`build-dep-debs.sh "" [] []`** — build + every xcat-dep `.deb` for this host's arch and stage them under `/repos/apt//` + for each requested codename. Run once per arch (amd64 on the x86 host, ppc64el on the ppc host). + The compiled deps (ipmitool, syslinux, conserver, goconserver, grub2-xcat, elilo, xnba) are built + **inside** the matching `schroot -c --sbuild` — never built once and re-labeled. + `xCAT-genesis-base-{amd64,ppc64el}` are `Architecture:all`, so the amd64 host builds them once and + stages them into every codename (cross-arch netboot, issue #7610). A missing chroot fails that + codename loudly (no silent re-label). + +- **`build-apt-repo.sh`** (already in this repo) — assemble the staged per-codename debs into ONE apt + tree signed with the xCAT key (the same key as xcat-core apt). + +## Flow (per arch) + +``` +mk-dep-chroots.sh # once, as root: create the sbuild chroots +build-dep-debs.sh "focal jammy noble resolute" # build + stage the .debs per codename +build-apt-repo.sh ... # assemble + sign the apt tree +``` + +> NOTE: `mk-dep-chroots.sh` / `build-dep-debs.sh` originated in the xCAT CI (xcat-core-ci-cd) and +> still carry CI-specific path assumptions (e.g. a bind-mount of the shared build tree in +> `mk-dep-chroots.sh`); genericizing them for standalone use is tracked in the xCAT CI issue queue. + # References - [mock project repository](https://github.com/rpm-software-management/mock) diff --git a/build-dep-debs.sh b/build-dep-debs.sh new file mode 100755 index 0000000..5822cd0 --- /dev/null +++ b/build-dep-debs.sh @@ -0,0 +1,154 @@ +#!/usr/bin/env bash +# +# build-dep-debs.sh "" [] [] +# +# Build every xcat-dep .deb for this host's arch (DEB_ARCH env, default from dpkg) and STAGE the +# resulting debs into /repos/apt// for each requested codename so build-apt-repo.sh +# can assemble them. Run once per arch (amd64 on the x86 ubuntu host, ppc64el on the ppc ubuntu host). +# +# PER-CODENAME BUILDS (correctness): the compiled deps (ipmitool, syslinux, conserver, goconserver, +# grub2-xcat, elilo, xnba) are C/Go binaries that link against the target codename's libc/toolchain, so +# each is built INSIDE the matching sbuild chroot (`schroot -c --sbuild`) -- NOT built +# once on the host and re-labeled into every codename dir (a noble/glibc-2.39 binary won't run on a +# focal/glibc-2.31 MN). Build-deps are installed in the session from each package's debian/control. +# The chroots must exist (create them with ci/mk-dep-chroots.sh); a missing chroot FAILS that codename +# LOUDLY (no silent re-label). The chroots need main + universe (quilt et al. live in universe). +# +# Cross-arch genesis (issue #7610): in 2.17 the apt repo shipped BOTH xcat-genesis-base-amd64 and +# xcat-genesis-base-ppc64el so an amd64 MN could netboot ppc nodes (and vice versa). Both are +# Architecture:all (codename-agnostic), so the amd64 host builds them ONCE and stages into every codename. +# +# Codename <-> version: focal=20.04 jammy=22.04 noble=24.04 resolute=26.04. +# +set -uo pipefail +PREFIX="${1:?PREFIX required}"; DISTS="${2:?DISTS required}"; GENESIS_RPM="${3:-}" +GENESIS_RPM_PPC="${4:-${GENESIS_BASE_RPM_PPC:-}}" +DEB_ARCH="${DEB_ARCH:-$(dpkg --print-architecture)}" +DEP="$PREFIX/source/xcat-dep" +APT="$PREFIX/repos/apt" +[ -d "$DEP" ] || { echo "FATAL: no xcat-dep checkout at $DEP" >&2; exit 1; } + +declare -A C2V=( [focal]=ubuntu20.04 [jammy]=ubuntu22.04 [noble]=ubuntu24.04 [resolute]=ubuntu26.04 ) +PKGS="ipmitool syslinux conserver goconserver grub2-xcat elilo xnba" + +echo "[build-dep-debs] DEB_ARCH=$DEB_ARCH DISTS=$DISTS (per-codename chroot builds)" + +# build one codename's compiled deps inside its sbuild chroot; stage into repos/apt//. +# The whole per-codename build runs in ONE schroot session (ephemeral overlay); the produced debs are +# copied OUT to $out on the shared tree (bind-mounted rw into the chroot). Returns non-zero if any +# package failed to build for this codename. +build_codename() { + local cn="$1" ver="${C2V[$1]:-}" chroot="${1}-${DEB_ARCH}-sbuild" + [ -n "$ver" ] || { echo "FATAL: unknown codename '$cn'" >&2; return 1; } + if ! schroot -l 2>/dev/null | grep -qx "chroot:${chroot}"; then + echo "FATAL: sbuild chroot '$chroot' missing for codename '$cn' -- create it with ci/mk-dep-chroots.sh (build-once-per-codename requires it; refusing to re-label a host build)" >&2 + return 1 + fi + local out="$PREFIX/debs-$cn-$DEB_ARCH"; rm -rf "$out"; mkdir -p "$out" + echo "== [$cn] building [$PKGS] in chroot $chroot -> $out ==" + # schroot SANITIZES the environment, so pass DEP/OUT/PKGS as POSITIONAL ARGS to the inner bash + # (env vars would arrive empty inside the chroot). `-s ` => args become $1/$2/$3 for the + # stdin (heredoc) script; the 'INNER' heredoc is quoted so inner $pkg/$W do not expand out here. + schroot -c "$chroot" -u root -d / -- bash -uo pipefail -s "$DEP" "$out" "$PKGS" <<'INNER' + DEP="$1"; OUT="$2"; PKGS="$3" + export DEBIAN_FRONTEND=noninteractive DEB_BUILD_OPTIONS=nocheck + for t in 1 2 3; do apt-get update -q && break; sleep 5; done + # common tools the make_deb.sh scripts use beyond debian/control Build-Depends (e.g. goconserver's + # make_deb.sh git-clones + go-builds; others wget/curl their upstream tarball). + apt-get install -y --no-install-recommends git wget curl ca-certificates golang-go devscripts >/dev/null 2>&1 || true + fail="" + for pkg in $PKGS; do + [ -f "$DEP/$pkg/make_deb.sh" ] || { echo " skip $pkg (no make_deb.sh)"; continue; } + W=$(mktemp -d); cp -a "$DEP/$pkg" "$W/$pkg"; cd "$W/$pkg" + if [ -f debian/control ]; then + # install Build-Depends parsed from debian/control (strip version/arch qualifiers) + BD=$(sed -n '/^Build-Depends:/,/^\S/p' debian/control | tr ',' '\n' \ + | sed -E 's/^Build-Depends://; s/\(.*\)//; s/\[.*\]//; s/[[:space:]]//g' \ + | grep -E '^[a-z0-9]' | grep -v '^debhelper-compat' | sort -u | tr '\n' ' ') + [ -n "$BD" ] && { apt-get install -y $BD >/dev/null 2>&1 || echo " [warn] $pkg: some build-deps failed to install"; } + fi + chmod +x make_deb.sh + if ./make_deb.sh > "$OUT/$pkg.buildlog" 2>&1; then + # make_deb.sh drops the .deb(s) beside the package dir (the build tree is removed by it) + found=$(find "$W" -maxdepth 2 -name '*.deb' ! -name '*-dbgsym_*' -print) + if [ -n "$found" ]; then echo "$found" | while read -r d; do cp -v "$d" "$OUT/"; done; echo " OK $pkg" + else echo " FAIL $pkg (built but produced no .deb)"; fail="$fail $pkg"; fi + else + echo " FAIL $pkg (build error -- see $pkg.buildlog)"; fail="$fail $pkg" + fi + cd /; rm -rf "$W" + done + [ -z "$fail" ] || { echo "FAILED_PACKAGES:$fail"; exit 1; } +INNER + local rc=$? + mkdir -p "$APT/$ver" + cp -v "$out"/*.deb "$APT/$ver/" 2>/dev/null || true + return $rc +} + +overall=0 +for cn in $DISTS; do + mkdir -p "$APT/${C2V[$cn]:?unknown codename $cn}" + build_codename "$cn" || overall=1 +done + +# Convert a noarch xCAT-genesis-base rpm (path or URL) into an Architecture:all deb. +# convert_genesis_rpm_to_deb +convert_genesis_rpm_to_deb() { + local rpm="$1" pkgname="$2" outdir="$3" work ver pkgd + work="$(mktemp -d)" + ( cd "$work" && (rpm2cpio "$rpm" 2>/dev/null || curl -fsSL "$rpm" | rpm2cpio) | cpio -idm --quiet ) + ver="$(rpm -qp --qf '%{VERSION}-%{RELEASE}' "$rpm" 2>/dev/null || echo 2.18.0-snap)" + pkgd="$work/pkg/$pkgname"; mkdir -p "$pkgd/DEBIAN" "$pkgd/opt/xcat" + cp -a "$work"/opt/xcat/* "$pkgd/opt/xcat/" 2>/dev/null || cp -a "$work"/* "$pkgd/opt/xcat/" 2>/dev/null || true + cat > "$pkgd/DEBIAN/control" < +Description: xCAT genesis base (diskless boot image), converted from the rpm +CTRL + dpkg-deb --build "$pkgd" "$outdir/${pkgname}_${ver}_all.deb" + rm -rf "$work" +} + +# xCAT-genesis-base debs: Architecture:all, codename-agnostic -> built ONCE (amd64 host) into a genesis +# staging dir, then staged into EVERY requested codename dir. Produces BOTH the amd64 genesis (native) +# AND, for cross-arch (#7610), the ppc64el genesis from the ppc64 rpm -- so the amd64 apt repo can +# netboot ppc nodes. If no rpm given, reuse an already-staged/known deb. +if [ "$DEB_ARCH" = amd64 ]; then + GEN="$PREFIX/genesis-debs"; rm -rf "$GEN"; mkdir -p "$GEN" + # --- amd64 genesis (native) --- + gdeb="$(ls "$APT"/*/xcat-genesis-base-amd64_*_all.deb 2>/dev/null | head -1 || true)" + if [ -n "$GENESIS_RPM" ]; then + echo "== converting amd64 genesis-base rpm -> deb: $GENESIS_RPM ==" + convert_genesis_rpm_to_deb "$GENESIS_RPM" xcat-genesis-base-amd64 "$GEN" + elif [ -n "$gdeb" ]; then + echo "== reusing already-staged amd64 genesis-base deb: $gdeb =="; cp "$gdeb" "$GEN/" + else + reuse="$(ls /opt/xcat-ci-shared/builds/*/debs/xcat-genesis-base-amd64_2.18*_all.deb 2>/dev/null | tail -1 || true)" + [ -n "$reuse" ] || { echo "FATAL: no GENESIS_BASE_RPM given and no genesis-base deb to reuse" >&2; exit 1; } + echo "== reusing amd64 genesis-base deb: $reuse =="; cp "$reuse" "$GEN/" + fi + # --- ppc64el genesis (cross-arch, issue #7610) --- + gdeb_ppc="$(ls "$APT"/*/xcat-genesis-base-ppc64el_*_all.deb 2>/dev/null | head -1 || true)" + if [ -n "$GENESIS_RPM_PPC" ]; then + echo "== converting ppc64el genesis-base rpm -> deb: $GENESIS_RPM_PPC ==" + convert_genesis_rpm_to_deb "$GENESIS_RPM_PPC" xcat-genesis-base-ppc64el "$GEN" + elif [ -n "$gdeb_ppc" ]; then + echo "== reusing already-staged ppc64el genesis-base deb: $gdeb_ppc =="; cp "$gdeb_ppc" "$GEN/" + else + echo "WARN: no ppc64 genesis rpm (arg 4 / GENESIS_BASE_RPM_PPC) and none staged --" >&2 + echo " apt repo will NOT ship xcat-genesis-base-ppc64el; an amd64 MN cannot netboot" >&2 + echo " ppc nodes (issue #7610). Pass the ppc64 xCAT-genesis-base rpm to fix." >&2 + fi + # stage the arch:all genesis debs into every requested codename + for cn in $DISTS; do cp -v "$GEN"/*.deb "$APT/${C2V[$cn]}/" 2>/dev/null || true; done +fi + +if [ "$overall" -ne 0 ]; then + echo "[build-dep-debs] FATAL: one or more codenames had package build failures (see repos/apt//*.buildlog)" >&2 +else + echo "[build-dep-debs] staged $DEB_ARCH debs into repos/apt/{$(echo $DISTS | tr ' ' ',')}" +fi +exit $overall diff --git a/mk-dep-chroots.sh b/mk-dep-chroots.sh new file mode 100755 index 0000000..31a280c --- /dev/null +++ b/mk-dep-chroots.sh @@ -0,0 +1,51 @@ +#!/bin/bash +# +# mk-dep-chroots.sh -- create the per-codename sbuild chroots the xcat-dep per-codename deb build +# (build-dep-debs.sh) needs. Run as ROOT on the ubuntu build host (xcat-master-ub for amd64; the ppc +# ubuntu host for ppc64el). Idempotent: recreates each chroot from scratch. +# +# Why not just build on the host: the compiled deps link against the target codename's libc/toolchain, +# so a focal deb MUST be built in a focal chroot (a noble/glibc-2.39 binary won't run on focal). This +# creates one chroot per codename that build-dep-debs.sh drives via `schroot -c --sbuild`. +# +# GOTCHAS baked in here (learned the hard way): +# * archive.ubuntu.com TIMES OUT from the build host -> use a fast BR mirror (override with MIRROR=). +# * build-deps like `quilt` live in UNIVERSE -> the chroot sources.list must carry main + universe +# (sbuild-createchroot's default is main only, which makes every build fail on `quilt`). +# * debootstrap may lack a script for a codename -> symlink it to the generic `gutsy` script. +# * build-dep-debs.sh reads the source + writes debs under /opt/xcat-ci-shared -> bind-mount it in. +# +# Env: MIRROR (default BR archive), DEB_ARCH (default amd64), CODENAMES (default all four). +set -u +MIRROR="${MIRROR:-http://br.archive.ubuntu.com/ubuntu}" +ARCH="${DEB_ARCH:-amd64}" +CODENAMES="${CODENAMES:-focal jammy noble resolute}" + +command -v sbuild-createchroot >/dev/null 2>&1 \ + || { echo "installing schroot/sbuild/debootstrap"; DEBIAN_FRONTEND=noninteractive apt-get install -y -q schroot sbuild debootstrap; } +for cn in $CODENAMES; do + [ -e "/usr/share/debootstrap/scripts/$cn" ] || ln -sf gutsy "/usr/share/debootstrap/scripts/$cn" +done + +for cn in $CODENAMES; do + echo "=== creating ${cn}-${ARCH} $(date +%H:%M:%S) via ${MIRROR} ===" + rm -rf "/srv/chroot/${cn}-${ARCH}"; rm -f /etc/schroot/chroot.d/${cn}-${ARCH}* 2>/dev/null + if sbuild-createchroot --arch="$ARCH" "$cn" "/srv/chroot/${cn}-${ARCH}" "$MIRROR"; then + # enable main + universe (+ updates/security) so build-deps in universe (quilt, ...) resolve + cat > "/srv/chroot/${cn}-${ARCH}/etc/apt/sources.list" </dev/null \ + || echo '/opt/xcat-ci-shared /opt/xcat-ci-shared none rw,bind 0 0' >> /etc/schroot/sbuild/fstab + +echo "ALL_CHROOTS_DONE (registered: $(schroot -l 2>/dev/null | grep -c '^chroot:'))"