Addresses the three blocking issues from the PR #63 review.
1. THE TWO ARCH JOBS COULD BOTH REWRITE THE PUBLISHED REPOSITORY.
The amd64 and ppc64el runs execute concurrently on their two hosts against the
same --apt-dir. The run lock does not protect the repository -- it is per-arch by
design, precisely so the two arches CAN build in parallel -- so both were free to
wipe and repopulate the same pool/, dists/, Release, InRelease and tarball at the
same time and interleave into a corrupt but green repository.
Split by role, so architecture jobs produce staging artifacts and one locked
finalization step publishes atomically:
* A run that builds no longer publishes. It fills staging/<codename>/<arch>/
and stops, saying so and printing the finalize command. Publishing happens
only with --publish, or implicitly on a run that builds nothing
(--skip-build), which IS the finalization step; --skip-createrepo still
forces it off.
* publish_repo() takes ONE GLOBAL publish lock (.sbuild-all.publish.lock), not
the per-arch build lock, so even a cron run racing a manual one on the same
host serializes instead of interleaving.
* It publishes atomically. assemble_into() builds and signs the complete tree
in a side directory (<apt-dir>.publish-<run-id>.<pid>), the gate runs against
THAT tree, and swap_into_place() then renames it onto --apt-dir. A reader --
the deploy rsync, an apt client on a served tree -- sees either the previous
complete repo or the new complete repo, never a half-wiped pool or an index
that disagrees with its Release. A failure anywhere before the swap leaves
the published tree untouched and removes the side tree. The side tree is
seeded from the current published one, so codenames outside --dists survive.
* The tarball moved inside publish, under the same lock: previously every
per-arch build tarred the shared apt tree while the other arch rewrote it.
2. build_deb_in_chroot() WAS NOT FAIL-HARD, AND THE ENVIRONMENT WAS NOT CLEAN.
The common build-tooling install ended in `|| true`; a failed Build-Depends
installation only warned; and the Build-Depends came from a sed pipeline over
debian/control that stripped version constraints `(>= 12)`, could not express
alternatives `a | b`, and mangled arch qualifiers. Worse, the per-codename
chroots are long-lived and shared by all seven packages, and nothing asserted
that a schroot session is throwaway -- so on a chroot without a union mount,
package N's build-dependencies stayed installed for package N+1 and a package
whose debian/control forgets a Build-Depends would build green on a sibling's
leftovers.
Making dependency setup fatal is only half the fix; it means nothing if a stale
environment can satisfy an undeclared dependency. Both halves are addressed:
* The in-chroot program is now generated by the pure, unit-tested
BuildUtils::chroot_build_script() and runs under `set -euo pipefail`.
apt-get update and the common tooling go through an apt_retry helper that
retries a transient mirror hiccup and then FAILS the build. Build-Depends are
installed with mk-build-deps (devscripts + equivs), which hands
debian/control's relationships to apt verbatim, and a failure is fatal. The
mk-build-deps dummy package is excluded from deb collection alongside dbgsym.
* ensure_disposable_chroot() repairs a chroot.d entry that lacks
union-type=overlay, and build_deb_in_chroot() re-reads `schroot --config` and
hard-fails on a chroot that is still not disposable, naming the fix.
BuildUtils::chroot_is_disposable() is the pure predicate.
3. THE REPOSITORY GATE STILL HAD FALSE PASSES.
* Standalone --verify-repo skipped signature verification whenever --gpg-home
was not passed, so the common `--verify-repo <dir>` invocation silently
checked completeness only. Signatures are now verified BY DEFAULT there;
--no-verify-signature is the explicit opt-out. (The automatic pre-swap gate
still requires a signature iff --gpg-sign was used, so an intentionally
unsigned tree does not false-fail -- and publishing unsigned now warns.)
* The expected architecture set was inferred from what happened to be present,
so an entirely missing secondary architecture read as "this run did not build
it" and passed. It is now always a CLAIM: --expect-arch (repeatable, accepts
a space/comma list) if given, else the staged arch set when publishing, else
each codename's own Release "Architectures:" line when verifying standalone
(BuildUtils::parse_release_architectures). The pure
BuildUtils::verify_repo_arches then reports an expected arch with no NATIVE
package as MISSING-ARCH and natives for an unexpected arch as
UNEXPECTED-ARCH (a stale architecture). Native detection matters because the
Architecture:all packages ride into every arch's index, so a non-empty
binary-<arch>/Packages is not evidence that arch was built.
* Release now advertises exactly the expected arch set and only those
binary-<arch> indices are written, so an amd64-only build no longer
advertises a ppc64el it cannot serve.
* An expected cell with no manifest section used to be skipped with a note --
a third free pass. It is now a hard NO-MANIFEST error.
TESTS
t/sbuild-all.t gains verify_repo_arches (both directions), the
parse_release_architectures parser, chroot_is_disposable, regression guards on
the generated in-chroot script (no `|| true`, no warn-only build-deps, no sed
extraction, mk-build-deps present, every apt-get behind the fatal helper), and
the build_deb_in_chroot disposability guard driven through a stub schroot.
t/verify-repo.t is new: it drives the real `sbuild-all.pl --verify-repo` against
fixture apt trees and asserts each former false pass now fails -- missing
secondary architecture (with and without --expect-arch), arch:all-only index,
unsigned repo with no --gpg-home, missing manifest section -- and that an honest
single-arch repo still passes.
Validated on the build hosts: a full focal+noble amd64 build of all seven
packages through the new fatal dependency path (mk-build-deps installs the
declared Build-Depends inside the oldest, focal, chroot), and the publish path
end to end -- a staging-only run leaves the repo untouched; publish assembles
aside, gates, and swaps; an unrelated codename survives the swap; stale debs are
dropped; a failed gate leaves the published repo byte-identical and cleans up the
side tree.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Landing-review nits: the Packages note said 'conserver' where the shipped package
is 'conserver-xcat'; and add a test asserting control_binary_arch returns the full
multi-token Architecture value (documents the commit's intent; behaviorally inert).
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Move the pyodbc note under a dedicated 'Packages notes' section and add a note
that conserver is shipped for completeness/backward-compat only: core packages
depend on goconserver, so conserver is not pulled in as a dependency -- to use it
you must install it explicitly, then disable the goconserver service and enable
the conserver service.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The previous commit added the arch:all boot tools (syslinux-xcat/grub2-xcat/
elilo-xcat/xnba-undi) to the ppc64el manifest and taught build_one_codename to
skip BUILDING them on non-amd64 -- but validate_manifest, which runs per-arch on
every non-dry-run invocation and is NOT gated by --skip-createrepo, still
demanded them. On the ppc64el build stage (--arch ppc64el, no --skip-build) the
builder skipped the four (correct -- their source is x86-only) so they were never
staged, then validate_manifest reported them MISSING and aborted the whole ppc
build before assembly: every BUILD_PPC=true run would have failed.
Fix the drift structurally: extract the skip rule into one pure, tested decider
BuildUtils::skip_arch_all_on() behind a shared sbuild-all.pl helper
pkg_skip_on_arch(), consulted by BOTH build_one_codename and validate_manifest,
so a package the build skips is never demanded by the per-arch validation. The
arch:all debs' presence on ppc is still verified later against the published
index by verify_assembled_repo. Also make control_binary_arch return the full
Architecture value (not just the first token of a multi-arch list) and add a
regression test for the shared skip rule.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Reflect the manifest change (arch:all boot tools syslinux-xcat/grub2-xcat/
elilo-xcat/xnba-undi are required-present on ppc64el, single-producer on amd64
via build_one_codename's arch:all skip) and note that pyodbc is not built
because Ubuntu (apt) and EL (appstream/EPEL) provide python3-pyodbc.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The ppc64el manifest sections listed only the natively-built deps
(ipmitool-xcat, conserver-xcat, goconserver) + genesis, omitting the four
noarch boot components syslinux-xcat, grub2-xcat, elilo-xcat and xnba-undi. A
ppc management node needs these for netboot -- the EL manifest and the
historical 2.16 ppc dep repo both ship them (grub2-xcat + syslinux-xcat
especially; yaboot-xcat was dropped in 2.18 as obsolete). They ARE published
to the ppc index today (Architecture:all rides into every binary-<arch>
index), but the completeness gate never asserted their presence for ppc.
List them as required-present on every ppc64el target so the gate verifies the
ppc repo carries them. They remain SINGLE-PRODUCER: their source is x86-only
(syslinux compiles with nasm/gcc-multilib), so build_one_codename now skips an
Architecture:all package on any non-amd64 arch -- detected via a new pure,
unit-tested control_binary_arch() helper -- so listing them for ppc64el drives
verification, not a second (failing) build.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The prior commit scoped the apt verify gate to '--arch floor UNION present
arches', but detected presence with a non-empty binary-<arch>/Packages. That
is wrong: every binary-<arch> index carries the Architecture:all debs
(grub2-xcat, genesis), so a BUILD_PPC=false run has a non-empty binary-ppc64el
index built purely from arch:all debs -- and the gate would still demand the
native ppc compiled deps (ipmitool-xcat, conserver-xcat, goconserver) it never
built, the exact false-fail the change was meant to remove.
Detect a genuine per-arch build via a new pure helper index_has_native_arch(),
which is true only when the index has a stanza with Architecture == that arch
(not merely Architecture:all). Unit-tested happy + the arch:all-only sad case.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Two review follow-ups to the Ubuntu matrix build:
1. verify_assembled_repo hardcoded {amd64, ppc64el}, so a single-arch run
(BUILD_PPC=false, or --verify-repo on an amd64-only tree) false-failed
demanding a ppc index it never built. The gate now verifies the --arch
set as a required FLOOR unioned with any arch that actually published a
non-empty index: the multi-arch assemble (invoked --arch amd64) still
verifies the ppc64el debs it carries, while a genuine single-arch run no
longer demands the absent arch. No Jenkinsfile/invocation change, so
master's shared inline job stays compatible.
2. standard_options advertised finalize-xcat-dep!/force-unlock!, which
sbuild-all.pl never wires (--force-unlock would even error as unknown).
Drop them.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Re-review polish:
- resolve_expected_key returns the primary-key fingerprint only when EXACTLY ONE key
matches --gpg-key-id (undef if absent/ambiguous, never the raw id); expected_is_fpr keys
off defined() so a hex key-id absent from the keyring hard-fails SIGKEY (clear) instead of
a misleading WRONGKEY.
- refresh the stale header comments on sig_observed_key (now 2-arg, strict, no presence-only
fallback) and repo_present_from_index (exact <name>-<arch>, not a ^<name>- prefix).
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Align the apt gate with the EL gate so both AGREE on success/failure:
- Duplicate: parse_packages_index now DIES loudly on a package with two DISTINCT versions
(stale .deb not cleaned), mirroring EL's rpm_version -- no more silent keep-highest.
Removed the dpkg keep-highest oracle; added a happy/sad test.
- Signature: sig_observed_key returns the signer fingerprint or undef (no presence-only
fallback); the gate hard-fails (SIGKEY) if --gpg-key-id doesn't resolve to a fingerprint,
so it always confirms the repo was signed by EXACTLY the CLI key. Signature is required
only when --gpg-sign was used.
- Align the MISSING message with EL (undef pin -> '*'). Document the gate + idiosyncrasies
in BUILD.md.
prove t/sbuild-all.t: 98/98.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Review follow-up on the apt-repo gate:
- FALSE-PASS (concern #7610): genesis name-resolution picked xcat-genesis-base-amd64
alphabetically for BOTH cells (both genesis debs are Architecture:all and appear in every
arch index), so the ppc64el cell never verified its NATIVE xcat-genesis-base-ppc64el -- a
dropped ppc genesis passed. Extract a PURE, unit-tested resolve_present_names that resolves
the arch-suffixed genesis to THIS cell's arch only (never a different arch), with a test that
reproduces the masked-genesis case.
- signature: reject EXPKEYSIG/REVKEYSIG/EXPSIG (expired/revoked keys emit VALIDSIG too); drop
the short-GOODSIG-keyid fallback (could never equal the 40-hex expected fpr -> spurious WRONGKEY).
- FALSE-FAIL: the post-assembly auto-run now requires a signature only when --gpg-sign was
actually used (an unsigned-by-choice repo no longer dies UNSIGNED); standalone --verify-repo
keeps checking whenever a gpg key/home is configured.
- cosmetic: de-duplicate the MISSING-INDEX message.
prove t/sbuild-all.t: pure gate tests incl. the new genesis-resolution cases.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Adds a real gate on the ASSEMBLED apt repo, per codename x arch, using
debs-manifest.conf as the single source of truth, layered pure/testable:
- BuildUtils: verify_repo_packages(\%expected,\%present) (MISSING/VERSION),
verify_repo_signature(\%expected,\%observed) (UNSIGNED/WRONGKEY), and
parse_packages_index($text) -- all PURE and unit-tested (happy+sad, no dpkg-deb).
- sbuild-all.pl does the IO via one sub verify_assembled_repo: parses each published
binary-<arch>/Packages (resolving arch-suffixed names like xcat-genesis-base-<arch>,
reducing to upstream via deb_upstream_version to compare against the manifest pin),
runs gpg --verify on each dists/<cn>/InRelease and extracts the signer fingerprint,
then delegates to the two pure deciders and dies listing every [<cn>/<arch>] problem.
- Runs AUTOMATICALLY at the end of assemble_apt (once Packages + signed Release exist);
suppressible with --no-verify-repo; skipped under --dry-run. Also a standalone,
lock-free, build-free '--verify-repo=<apt_dir>' mode using the script's --manifest/
--dists/--gpg-key-id/--gpg-home. Replaces the coarse pool-global hard-coded check.
prove t/sbuild-all.t: 90/90 (was 71). Smoke-tested: complete tree passes; dropped pkg
-> MISSING; wrong version -> VERSION; missing index -> MISSING-INDEX; all die nonzero.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Closes the reproducibility/supply-chain gap left as a TODO: the Ubuntu goconserver
build resolved all transitive modules live from the network via 'go mod tidy', so the
compiled binary was not reproducible and the graph was unverified.
Commit goconserver/gomod/go.{mod,sum}, generated with the build's own pinned toolchain
(go 1.25.12) at REF=6166fe5, with the etcd backend removed (its coreos/bbolt dep, now
go.etcd.io/bbolt, breaks 'go mod tidy') and kr/pty replaced by creack/pty -- exactly the
steps the build performs. sbuild.pl now overlays the committed pair into the cloned tree
and builds with GOFLAGS=-mod=mod, so modules are downloaded but PINNED + integrity-checked
by go.sum; no 'go mod tidy'. Verified on xcat-master-ub: both entrypoints (goconserver.go,
cmd/congo.go) compile CGO-free against the committed go.sum. gomod/README.md documents
regeneration. Mirrors the EL branch's pinning model (its go.sum is generated with a
different Go, so a fresh Ubuntu-toolchain pair is used rather than copying it).
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Follow-up to the review-fix commit, which broke the two-arch build:
- The run lock was on <output_root>/.sbuild-all.lock, shared by BOTH arch stages.
But within ONE pipeline run the amd64 and ppc64el stages run concurrently on their
own hosts against the same --output-root -- a shared lock serializes/deadlocks them.
Lock per-arch (<output_root>/.sbuild-all.<arch>.lock): only same-arch stages (same
host) contend, so a local flock is authoritative and cross-run same-arch races are
still blocked, without touching the intended amd64||ppc64el parallelism.
- wipe_tree died on .nfsXXXX silly-rename artifacts (an already-unlinked file a
still-open handle -- e.g. an aborted peer build -- keeps alive). Those are not stale
debs and self-heal. Build #26 hit exactly this: the ppc stage's wipe FATAL'd on
leftover .nfs* from an aborted run, and failFast then SIGTERM'd the amd64 conserver
builds mid-apt (the 'conserver rc=1' was collateral). wipe_tree now retries once,
then dies ONLY if a real (non-.nfs*) file survives; .nfs*-only leftovers are
tolerated (assemble globs *.deb, never .nfs*, so they can't reach the repo).
perl -c clean; prove t/sbuild-all.t: 71/71.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Review follow-up for the Ubuntu sbuild matrix:
- Add a fail-fast exclusive flock over the whole run (<output-root>/.sbuild-all.lock,
file-scoped handle) so two overlapping runs can't corrupt the shared staging/apt
tree -- this is the root of the observed 'remove_tree .../staging/<cn>/<arch>:
Directory not empty' (an NFS silly-rename from a concurrent run).
- wipe_tree(): remove_tree that captures {error} and dies loud, so an ENOTEMPTY no
longer carps-and-continues leaving stale debs; used for all staging/pool/dists wipes.
- Wire the tested, hash-based cross_copy_genesis_deb into build_genesis (was a naive
glob+copy, so the unit-tested stale-dropping copier was dead code); remove the
genuinely-unused deb_snap_version/rewrite_changelog_top helpers + their subtests
(compiled deps intentionally ship their tracked changelog version).
- Dedupe assemble_apt on binary Package+Architecture (keep highest via
dpkg --compare-versions) so a double-produced genesis can't land two versions in
the pool, independent of the --skip-genesis contract.
- Derive Release Architectures from the arches actually staged (non-empty
binary-<arch>/Packages), not a hard-coded 'amd64 ppc64el'.
- goconserver: guard 'go mod init' when a go.mod exists (+ TODO to commit go.sum for
the pinned SHA). Accept-and-ignore the unused per-package --log-dir/--build-number/
--skip-install flags (documented). Remove orphaned make_deb.sh dispatchers
(build-debs-all, build.sh, ipmitool/build.sh) + update the READMEs.
perl -c clean; prove t/sbuild-all.t: 71/71.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Remove the '(how the review's correctness concerns are met)' aside from the Design
heading and use xcat@example.com in the usage example instead of a real signing
address.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The build looped over codenames serially per host, so a 4-codename matrix was 2
parallel streams (one per arch/host) rather than 8. Fork one child per codename
(core Perl fork(), no extra module), each building in its own <codename>-<arch>-sbuild
chroot; default is all codenames in parallel (--parallel-targets N caps it, 1 =
serial, --dry-run stays serial). With the two arches on their two hosts that yields
8 concurrent build streams, 4 per host. Any codename's failure fails the run.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
sbuild-all.pl defaulted the chroot bootstrap mirror to the amd64 archive
(br.archive.ubuntu.com/ubuntu) for both arches, so auto-initializing a ppc64el
sbuild chroot would fail -- ppc64el is not served by archive.ubuntu.com, it lives
on ports.ubuntu.com/ubuntu-ports. Default the mirror per --arch (ppc64el ->
ubuntu-ports) unless --mirror is given explicitly, matching the existing
noble-ppc64el chroot.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The per-package <dep>/sbuild.pl were generic wrappers that shelled out to each
package's make_deb.sh, so a package's build brain was split across two files and
package-specific fixes (e.g. goconserver's Go toolchain) landed in make_deb.sh
instead of the builder -- inconsistent with the EL side, where <dep>/mockbuild.pl
IS the per-package builder.
Make each <dep>/sbuild.pl own its build (mirroring <dep>/mockbuild.pl), absorbing
its make_deb.sh (source prep, patches, toolchain, dpkg-buildpackage), and remove
all seven make_deb.sh. The common chroot orchestration -- ephemeral schroot session,
apt update, build-dep install, out-of-tree copy, SOURCE_DATE_EPOCH, deb collection +
host-side verification -- moves into BuildUtils::build_deb_in_chroot; each builder
supplies only its package-specific recipe (passed base64-encoded to avoid quoting
interplay through schroot).
Also make the older Ubuntu codenames buildable:
- goconserver: install a pinned modern Go (1.25.12) in the build -- focal/jammy ship
a Go too old to even auto-switch toolchains, and goconserver's pinned deps need
Go >= 1.25 (it is a static CGO-free binary, so the pinned toolchain is portable and
reproducible across codenames).
- ipmitool + goconserver: lower debian/compat 13 -> 12 and Build-Depends debhelper
(>= 12), since Ubuntu 20.04 (focal) ships debhelper 12; compat 12 also builds cleanly
on newer codenames.
Validated: all recipe shapes build green through the new path -- tarball
(ipmitool@focal), git-clone+pinned-Go (goconserver@focal), tarball+patches
(syslinux@noble), in-place (grub2-xcat), in-place+custom-rules (xnba); 80/80 unit
tests still pass.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Acceptance requirements for the Ubuntu dep build (VersatusHPC/xcat-core#49):
- Add full POD to sbuild-all.pl (NAME/SYNOPSIS/DESCRIPTION/PHASES/OPTIONS) with
--man and --help via Pod::Usage, so `perldoc sbuild-all.pl` / `--man` document
the tool (parity with the EL mockbuild-all.pl). Drops the drift-prone usage() sub.
- BUILD.md: explicit Ubuntu-scoped instructions for building ONE specific version
(--dists <codename> / --target <codename>-<arch>) and ALL supported versions.
Also: make_tarball creates its run dir before taring (an assemble-only run has no
genesis phase to create it), so the tarball phase never errors.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The pipeline invokes it as ./sbuild-all.pl; without the exec bit that returns
126 (Permission denied).
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Each <dep>/sbuild.pl copies the built .deb(s) to --result-dir from INSIDE the
schroot session. That only reaches the host when --result-dir is on a path
bind-mounted into the chroot (the shared /opt/xcat-ci-shared tree, as the CI
uses). A mis-configured result-dir (e.g. a chroot-local /tmp) would let the
build "succeed" yet leave nothing on the host -- a silent no-output. Verify
host-side after the session that the .deb(s) are actually present and fail
loud with a pointed message otherwise (consistent with the fail-hard design).
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The Ubuntu/Debian dependency build shipped as three bash scripts
(build-dep-debs.sh, build-apt-repo.sh, mk-dep-chroots.sh) whose review
(PR #63) surfaced correctness problems: partial/stale output could be
published, the rpm->deb genesis conversion dropped the maintained package
semantics (Depends/Breaks/Replaces + maintainer scripts), the arch matrix
was invalid (x86-only syslinux/elilo/xnba treated as ppc64el packages, and
Architecture:all packages with no single producer), several required
failures exited zero, and the build/repo scripts disagreed on their staging
path and codename set (focal missing from the assembler).
Rewrite it as proper, unit-tested Perl mirroring the EL side
(mockbuild-all.pl / MockBuildUtils.pm / <dep>/mockbuild.pl / t/*.t /
packages-manifest.conf), sharing one CLI vocabulary:
- BuildUtils.pm: shared, testable helpers + the canonical CLI spec, plus the
Debian-specific helpers (out-of-tree changelog stamping, genesis control
preservation, deb inspection, cross-arch genesis provisioning).
- sbuild-all.pl: the orchestrator, absorbing all three shell scripts. Builds
+ validates into a fresh per-arch staging tree and only (re)assembles the
published apt repo from validated staging -- so partial/failed output never
ships and stale debs never accumulate. Auto-initializes the per-codename
sbuild chroots on first run. Fails the whole run non-zero on any missing
chroot/package/artifact or version-pin mismatch.
- <dep>/sbuild.pl x7: per-package builders that drive each package's
MAINTAINED debian/ in the matching chroot (never re-implemented), so the
converted/built packages keep their control metadata and maintainer scripts.
- debs-manifest.conf: per-[<codename>-<arch>] required set + version pins,
encoding the per-arch package sets (x86 boot components built once on amd64
as the single producer; ppc64el builds only the arch-specific compiled deps).
- t/sbuild-all.t: fixture tests for every pure helper.
- goconserver/make_deb.sh: pin the upstream SHA instead of cloning a moving
branch, so every matrix cell builds the same source (reproducible).
Codename set unified across build, assembly, chroots and docs (focal IS
supported). BUILD.md documents the new flow.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The Ubuntu/Debian .deb dependency build lived only in the xCAT CI
(xcat-core-ci-cd), so xcat-dep could not build its apt dependency packages
standalone. Bring the two CI-only drivers into the repo, alongside the existing
build-apt-repo.sh + per-package make_deb.sh:
- mk-dep-chroots.sh : create the per-codename sbuild chroots (root, per host).
- build-dep-debs.sh : build every dep .deb per codename INSIDE the matching
sbuild chroot (correct libc/toolchain) and stage them for
build-apt-repo.sh; xCAT-genesis-base (Arch:all) built once
and staged into every codename (cross-arch netboot, #7610).
Document the Ubuntu/Debian sbuild flow in BUILD.md. The scripts still carry
CI-specific path assumptions (a shared-tree bind-mount in mk-dep-chroots.sh);
genericizing them is tracked separately, and the CI keeps its own copies until
this lands upstream.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Commit the upstream source tarballs the mock builders consume so a build never
has to fetch them from the network -- offline/reproducible builds, and
resilience to upstream URL rot for these older releases:
- ipmitool/ipmitool-1.8.18.tar.gz (re-normalized to the release tarball)
- syslinux/syslinux-6.03.tar.xz
- perl-Crypt-SSLeay/Crypt-SSLeay-0.72.tar.gz
The per-package builders use the local tarball when present and only fall back
to the upstream URL if it is missing (ipmitool/syslinux mockbuild.pl; and
mockbuild-perl-packages.pl 'spec' mode for perl-Crypt-SSLeay).
Also folded in:
- mockbuild-all.pl: switch createrepo -> createrepo_c (--database,
--set-timestamp-to-revision) for deterministic, upstream-matching repo
metadata.
- perl-HTTP-Async / perl-Net-HTTPS-NB specs: reword the brp-compress comment.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Register conserver-xcat in mockbuild-all.pl's dep-builder set so the
traditional C conserver (8.2.1) is built per-EL/arch alongside goconserver.
xCAT itself requires goconserver, so conserver stays a build-on-demand
artifact, but wiring it into the full dep build keeps it produced and
signed with the rest of xcat-dep for sites that want it.
conserver/mockbuild.pl gains the --build-timestamp option that
mockbuild-all passes to every builder (SOURCE_DATE_EPOCH for deterministic
builds); without it the child invocation would abort on an unknown option.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
conserver.spec had not been built on a modern EL toolchain and no longer
compiled on EL9/EL10:
- %prep used the bare %patch / %patch1 macros, which rpm 4.18+ (EL9/EL10)
rejects with "Patch number not specified". Switched to explicit
Patch0:/Patch1: with the numbered %patch0/%patch1 macros, which apply
cleanly on EL8 through EL10.
- BuildRequires listed only openssl-devel, so on EL9/EL10's minimal mock
buildroot the toolchain was absent and %configure failed with
"C compiler cannot create executables". Added gcc, make and glibc-devel.
Also add conserver/mockbuild.pl, a standalone per-EL builder matching the
other xcat-dep builders (goconserver/ipmitool): it stages the sources and
spec, builds the SRPM, mock-rebuilds it in the target chroot, copies the
RPMs to --result-dir, and smoke-tests console/conserver in the chroot.
conserver is not in the default mockbuild-all.pl set (xCAT uses goconserver),
so this builder is run on demand. Built + smoke-tested conserver-xcat-8.2.1
for alma+epel-{8,9,10}-x86_64.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
mockbuild-all.pl treated xnba-undi/grub2-xcat as x86-only @SHARED_NOARCH imported
onto ppc, so an x86 build failure rippled into a fatal "grub2-xcat missing" on
ppc; both actually build on any arch (noarch repackaging of committed artifacts)
and xCAT requires them on ppc too. Un-gate xnba-undi and drop the import so each
host builds a complete, self-sufficient dep repo. Add a single --output that
re-roots all NFS-shared output plus a fail-fast lock at <output>/.lock (owner-pid
guarded so forked build children do not delete it). Default gpg-key-name to the
real "xCAT Signing Key" and --xcat-source to ../xcat-core.
Make concurrent builds safe: give the genesis buildrpms.pl its own HOME/rpmbuild
tree (a shared /root/rpmbuild raced across parallel targets), and nest the
xnba/goconserver rpmbuild dirs under the run-scoped --work-dir (they hard-coded
/var/tmp/xcat-rpmbuild-* and wiped each other). Add --parallel-targets/--max-parallel,
defaulting to serial since the per-package scripts still share repo tarballs under
full parallelism.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
grub2-xcat/mockbuild.pl fetched a grub2 source RPM before building, and that
fetch was the only failing step (the pinned CentOS Stream URL now 404s, and the
dynamic dnf fallback is fragile on EL10). The download is dead weight: the
package is a pure noarch repackaging of the committed grub2-res.tar.gz and the
mock build never consumed the src.rpm. Remove the resolve+download machinery.
Epoch: 1 stays so 1:1.0 satisfies xCAT-server's legacy >= 2.02 pin and upgrades
over old 0:2.02 remain clean.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
xCAT Requires xnba-undi and grub2-xcat on every arch, but they only build on x86_64
(xnba-undi is an x86 UNDI netboot ROM; grub2-xcat wraps the distro grub2). Since both
are noarch, a ppc build now imports them from a built x86_64 dep repo via the new
--import-noarch-repo <dir>. Both are added to the required-deps assertion (with a hint
to pass --import-noarch-repo on non-x86_64 builds), so an incomplete dep repo fails at
build time instead of at xCAT install time.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
xCAT-server Requires: grub2-xcat >= 2.02-0.76.el7.1.snap201905160255, but the el10
rewrite reset grub2-xcat to Version 1.0 -- which can never satisfy that. Add Epoch: 1
so 1:1.0-2 outranks 0:2.02 and the (unchanged) xCAT-server dependency resolves.
Also stop pinning the upstream grub2 source rpm URL (the distro rolls grub2 forward
and prunes the old src.rpm from the mirror -> 404). grub2-xcat/mockbuild.pl now
resolves the current grub2 source rpm URL via dnf (download --source --url, then
repoquery --location) unless --upstream-url is given.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
mockbuild-all.pl now assembles a signed, deployable xcat-dep repo under
--repo-dep/rh{8,9,10}/<arch>, each with xcat-dep.repo, mklocalrepo.sh and
buildinfo.txt, ready to push to xcat.org (obsoletes cluster-test.pl's dep
collection). Default build loops rh8/rh9/rh10 for the host arch only; --target
still selects a single target. xCAT-genesis-base is collected into each per-EL
repo. New options: --repo-dep, --gpg-sign, --gpg-key-name, --gpg-home. No dhcp-
packages are built (DHCP is a rich dep in xCAT.spec), so nothing to exclude for el10.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Commit f6f8640 added the /var/lib/goconserver datadir so goconserver can create
its nodes.json store and makegocons can register consoles, but kept Release: 2.
The fixed build then shared an identical NVR (goconserver-0.3.3-2) with the
pre-fix build, so a stale pre-fix -2 could be (and was) served to management
nodes where makegocons still failed ("open /var/lib/goconserver/nodes.json: no
such file or directory"). Bump Release to 3 so the datadir fix is a distinct,
upgradeable version that cannot collide with the pre-fix -2.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
xCAT-genesis-base's dracut initramfs bundles the build chroot kernel +
glibc/busybox/perl, so it is OS-dependent and cannot ship in the single flat
xcat-core. Build it here, per target, via buildrpms.pl --package
xCAT-genesis-base (which derives the same snap Release from xcat-core Gitepoch,
matching xCAT-genesis-scripts). Add --skip-genesis to opt out.
In the split pipeline (--skip-xcat) the orchestrator (cluster-test.pl) routes
the resulting genesis-base from the xCAT dist tree into the per-EL xcat-dep
repo -- robust to this script exiting non-zero on tolerated dep-builder
failures -- so this script no longer collects the xCAT dist tree in that mode.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
perl-HTTP-Async and perl-Net-HTTPS-NB fail to build on openSUSE Leap 15
for two reasons. First, the patch is applied with "%patch 0 -p1"; rpm
4.14+ no longer accepts the space-separated number form and aborts %prep
with "%patch without corresponding Patch: tag" -- the supported spelling
is "%patch0 -p1". Second, %files is driven by a version-filelist built
during %install that records the man pages as *.3pm, but openSUSE's
brp-compress then gzips them to *.3pm.gz, so %files fails with "File not
found" for every man page. Disable the install-post hooks with
%define __os_install_post %{nil} (harmless for these noarch pure-perl
modules) so the packaged names match the file list.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
On openSUSE Leap 15 (OpenSSL 3) the build fails compiling
src/plugins/lan/auth.c with "unknown type name MD2_CTX". OpenSSL 3
removed MD2 from the default provider but still installs a stub
openssl/md2.h, so configure's AC_CHECK_HEADER probe succeeds and defines
HAVE_CRYPTO_MD2 -- yet MD2_CTX and the MD2_* functions no longer exist,
so auth.c's MD2 code path will not compile. Export
ac_cv_header_openssl_md2_h=no in %build to force the probe negative,
selecting the existing no-MD2 branch (which just warns that MD2 IPMI
authcodes are unsupported -- an obsolete, insecure scheme). Systems with
a real md2.h are unaffected.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
perl-HTTP-Async and perl-Net-HTTPS-NB used the bare-number "%patch 0" form,
which only rpm >= 4.18 (EL10) understands. On EL8 (rpm 4.14) and EL9 (rpm 4.16)
the SRPM build fails with "%patch without corresponding Patch: tag", so neither
perl-HTTP-Async 0.30-3 nor perl-Net-HTTPS-NB 0.14-3 could be rebuilt for rh8/rh9
and the 2.17 -> 2.18 upgrade reported "nothing provides" for both.
Use the portable "%patch -P 0" form so the patch applies on EL8/EL9/EL10.
The syslinux-xcat subpackage is BuildArch: noarch but deliberately bundles the
arch-dependent PXELINUX/SYSLINUX bootloader blobs under
/opt/xcat/share/xcat/netboot/syslinux. EL10 rpm does not flag these real-mode
binaries, but EL8/EL9 rpm aborts the build with "Arch dependent binaries in
noarch package", so syslinux-xcat 6.03 could not be rebuilt for rh8/rh9 and the
2.17 -> 2.18 upgrade had no syslinux-xcat >= 6.03-1 to satisfy xCAT 2.18.
Set %_binaries_in_noarch_packages_terminate_build 0 so the self-contained xcat
netboot payload builds across EL8/EL9/EL10.
The repository generator always iterated every supported distribution and
aborted if any one had no packages, so it could not assemble a repo for a
single release. It also only published a pre-exported key file and warned when
that file was absent. Accept optional distribution arguments to build a subset,
and export the signing public key from the keyring when no pre-exported key
file exists.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Building the genesis-base deb failed at dh_installdocs because LICENSE.html
could not be found: only the debian/ directory and the source RPM were copied
into the build root, while debian/docs references LICENSE.html. Copy
LICENSE.html into the build root alongside debian/. Also drop the dead
'svn update' call left over from the pre-git build.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
goconserver imports the abandoned kr/pty, whose pty.Start sets the controlling
tty in a way Go >= 1.15 rejects ("fork/exec ...: Setctty set but Ctty not valid
in child"). On current toolchains this breaks the console backend: rcons
connects but the session produces no output. Replace kr/pty with the
maintained, API-compatible creack/pty fork at build time.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The goconserver deb shipped no systemd unit, so enabling or starting the
goconserver service after installation had nothing to start (makegocons failed
with "Failed to start goconserver service"). Install the provided
goconserver.service unit during packaging.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Daniel Hilst
Markus Hilger