buildrpms_source_only.t's CLI half failed in CI: the runner has no
Parallel::ForkManager, so buildrpms.pl aborted at compile time and never
reached the option check the test is about. The module is needed only by
the test suite -- buildrpms.pl is not a runtime dependency of any package
-- so it goes in the workflow apt list.
The same half also escaped its scratch tree. Before buildrpms.pl looks at
@ARGV it rewrites the tracked Gitinfo in its working directory and creates
$HOME/rpmbuild, so running it from the checkout left the tree dirty and
reached into the developer's home to exercise argument parsing. It now
runs from a staged copy with HOME pointed at the sandbox.
Exit 2 is pinned rather than "non-zero", though perl also exits 2 on a
compile abort -- which is exactly how this assertion stayed green in CI
while the program could not load. The message assertion is what separates
the two, and the comment now says so.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Four fixes from @viniciusferrao's review plus the CI break his review predates.
orig tarball version. dpkg looks for <source>_<upstream>.orig.tar.gz with no
Debian revision, and the call site passed the full Version-Release. The rule now
lives in BuildUtils::upstream_version and orig_tarball_name applies it, so the
call site cannot get it wrong whichever string it is handed. Currently dormant --
every package is Format: 1.0, so the quilt branch does not run, which is why the
differential build did not catch it.
--dest could write to the filesystem root. Cwd::abs_path returns undef when a
PARENT component is missing (a missing leaf is fine), and the caller interpolated
that, so `--dest /no/such/parent/out` became `/debs` and `/xcat-core` at /.
Replaced with BuildUtils::resolve_dest, which is rel2abs and purely lexical --
correct for an output directory that does not exist yet.
Generated debian/control left behind. xCAT-genesis-scripts has no debian/control
of its own; it is generated from control-<arch>. The cleanup restored only files
that already existed, so the generated one stayed. Worse than dirty: ppc64el ran
last, so the restore put back the amd64 BACKUP and the leftover was the wrong
architecture's control, which a later single-arch build would have started from.
with_prepared_tree now records created files and removes them. Verified by a real
build: the checkout is byte-clean afterwards, matching the oracle.
CI install step. build-ubunturepo wrote its repo to $curdir/../../xcat-core,
which under GitHub's work/<repo>/<repo> layout IS $RUNNER_WORKSPACE, so
install_xcat's `./mklocalrepo.sh` happened to be in the directory it chdir'd to.
builddebs.pl writes inside the checkout instead -- that outside-the-checkout path
is what used to rm -rf the tree -- so install_xcat now names the script by its
real location and fails with a clear message if the build produced no repository.
This is what reddened xcat_pr_test at 2m13s; the builder itself was fine (the
exact CI invocation, `./builddebs.pl --force` with no --dest, returns 0 with all
14 packages).
The executable bit was already fixed before the review landed.
Both new helpers are tested and mutation-verified: not stripping the revision
reddens 3 assertions, swapping rel2abs back to abs_path reddens 2. Equivalence
re-measured after these changes -- all 14 packages identical to build-ubunturepo
in control and in every non-changelog file by md5.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
build-ubunturepo was 710 lines of shell doing the Debian half of what
buildrpms.pl does for rpms, with no code in common and a different CLI. It also
carried paths that are dead: GSA uploads, the PROMOTE/PREGA release flows, and a
-d mode that built an xcat-dep repository from a different project's packages.
builddebs.pl replaces it and mirrors buildrpms.pl -- Getopt::Long options, one
package list, build then index then sign -- so the two builders read the same way
and share BuildUtils.pm.
The design rests on one fact: xcat-core debs are Perl. They are byte-identical
for every Ubuntu release, so they are built ONCE and the same files are published
into every codename. Only xCAT, xCATsn and xCAT-genesis-scripts carry an
architecture, and there the difference is packaging metadata, not compiled
output. That is why this needs no sbuild and no per-codename chroot -- unlike
xcat-dep, whose packages are compiled and genuinely differ per release.
BuildUtils.pm holds what both builders need and what was worth making testable:
the Version-Release derivation from the commit time, the xCAT-probe helper
staging, the deb arch and dist tables, the debian/control version pinning, the
changelog rewrite, the reprepro conf generation, and the build lock. Every
function is pure or takes its side effect as an argument, so build_utils.t (45
assertions) drives each one rather than grepping a builder for evidence that it
is called. Verified by mutation: shrinking the arch table reddens 1, dropping
the /g from the control pin reddens 2.
The env-var CLI maps to options: BUILDALL=1 -> --force, GPGSIGN=1 -> --gpg-sign,
GPG_HOME -> --gpg-home, DEST -> --dest, DISTS -> --dist (repeatable). UP=0 has no
equivalent because uploading is gone -- the CD pipeline's deploy step publishes.
Callers updated: github_action_xcat_test.pl and travis.pl. The comment in
github_action_xcat_test.pl explaining why CI copies the tree before building is
corrected -- build-ubunturepo rm -rf'd $curdir/../../xcat-core, which under
GitHub's work/<repo>/<repo> layout is the checkout's own parent; builddebs.pl
writes under dist/debs inside the checkout and restores every file it edits, so
the copy is now only isolating the tests from build residue.
Two tests moved with it. build_ubunturepo_lock.t extracted the lock out of the
shell with a regex and ran that; the lock is now a function, so builddebs_lock.t
calls it -- and asserts what actually matters, that two builds of one checkout
fail fast while two builds of different checkouts run concurrently.
ubuntu_2604_pkglist.t asserted that resolute appeared in a shell fragment of
build-ubunturepo's source; it now asks BuildUtils for the release list and checks
a resolute stanza reaches conf/distributions. That assertion would have passed on
any file containing the fragment and broken on a reflow that changed nothing.
Verified: prove -r xCAT-test/unit fails on 6 files here against 7 on
upstream/master, the difference being apache_config_sources.t, fixed by the
preceding commit. The remaining 6 are missing DB modules on the machine that ran
it and are identical on both.
NOT done here, and required before this can merge: the Ubuntu core CD pipelines
still invoke ./build-ubunturepo (ci/ubuntu/Jenkinsfile.core-ubuntu-{devel,stable}
in VersatusHPC/xcat-core-ci-cd, and the inline script in each live Jenkins job).
Those must be switched to builddebs.pl in the same change window.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
buildrpms.pl is what actually builds xcat-core now: every core CD pipeline calls
it (ci/{el,suse,ubuntu}/Jenkinsfile.core-*), and it reaches none of these
scripts. What is left of them is a second, diverging build path that nothing
runs -- buildcore.sh has no caller at all (its one mention in
Jenkinsfile.core-el-stable is a comment), makerpm's only caller is
buildlocal.sh, and buildlocal.sh has no caller.
That divergence broke master. #7774 added a source-only mode to these scripts
rather than to buildrpms.pl, giving makerpm an unguarded
. "$(dirname "$0")/build-utils/source-only.sh"
at line 13, used on every build path (xcat_configure_rpm_build_mode,
xcat_announce_build), not only under SRCONLY. apache_config_sources.t stages
makerpm into a fixture that creates build-utils/ but copies only
sync-xcat-apache-configs into it, so makerpm died at line 13 and nothing reached
SOURCES/. Because prove -r xCAT-test/unit runs the whole suite, that reddened
xcat_pr_test on every open PR. It merged on a green run from 2026-08-27;
apache_config_sources.t landed 2026-08-30, and nothing rebuilt it in between.
Removed: makerpm, buildcore.sh, buildlocal.sh, build-utils/source-only.sh,
build-utils/buildcore-source-only.sh, and the tests that covered only them
(makerpm_source_only.t, buildcore_source_only.t). The makerpm half of
apache_config_sources.t goes with makerpm; its sync-xcat-apache-configs half
stays, because buildrpms.pl invokes that helper directly. The source-only
capability is preserved as buildrpms.pl --source-only, in the preceding commit.
build-ubunturepo is deliberately NOT touched. It is not a legacy script: both
Ubuntu core CD pipelines invoke it, github_action_xcat_test.pl:325 invokes it on
every PR, and it was last fixed this week. buildrpms.pl is RPM-only -- mock and
createrepo_c, no dpkg path -- so there is nothing to migrate it to. Giving
Debian builds an equivalent modern builder is separate work.
Verified: prove -r xCAT-test/unit before and after differs by exactly one file
-- apache_config_sources.t fails on upstream/master (4a0d9e0bb, tests 64-65) and
passes here. The other six failures are identical on both and are missing DB
modules on the machine that ran it. No remaining reference to any removed file
except the one comment in buildrpms.pl naming where its templates came from.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
buildrpms.pl already produces a source rpm for every package on every run --
buildall() is createmockconfig -> buildsources -> buildspkgs (mock --buildsrpm)
-> buildpkgs (mock --rebuild). Source-only is that sequence without the last
step, so it belongs here rather than in a parallel implementation: the spec, the
staged sources and the mock root are identical either way, and anything built
beside them can drift from what a real build does.
--source-only stops after buildspkgs. Two things downstream had to learn about
it, and both are about not publishing something untrue:
- index_repo no longer re-indexes the binary directory. Running createrepo_c
over a directory with no binaries in it would replace working metadata with
metadata for an empty repository -- a repo that resolves nothing. The srpm
index is still regenerated.
- write_repo_metadata_dir emits nothing. The .repo file and buildinfo describe
an installable binary repository, which this mode does not produce.
--source-only with --merge-core-repos is refused: one builds packages, the other
assembles per-arch trees that are already built.
buildrpms.pl cannot be loaded by a test -- it runs mkdir, git and read_text at
file scope -- so the two routines whose behaviour changed are lifted out with a
regex and eval'd into a scratch package with their collaborators stubbed, per
the code standard, with BAIL_OUT if the extraction stops matching. The CLI
contract is exercised by running the real program. Verified by mutation:
removing the index_repo guard reddens 2 of the 7 assertions.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Scan the z/VM, bmcconfig, energy, PPC configuration, CIM and z/VM
utility sources. Fail when a printSyslog line or a checkSSH_Rc command
string carries a password variable, when a directory query output logs
without the redactor, when the masked log lines lose their mask or
their position built operand, when the executed commands lose their
real values, when the CIM verbose dump returns to the raw request, or
when the attribute and access messages hold a password value again.
Drive the real redact_directory_entry with sentinel USER, IDENTITY,
IDENT, MDISK range, DEVNO and V-DISK, APPCPASS and keyword records in
both spellings, with and without one or more comment stars, cover the
COMMAND statement whole, keep the statements beside the passwords, and
prove that a passwordless record never masks across the line into the
next record. Assert that both clone loops redact the query output at the
source, that the directory helpers hand a redacted copy to the failure
checker, and that every error branch behind the output check, the
directory fetch, the mini disk fetch and the four disk list callers,
echoes the redacted text.
On a netplan-rendered node the reconfigure path takes the interface down
unconditionally, while the `netplan apply` that brings it back is gated on
reboot_nic_bool. In the diskful provision postscripts stage reboot_nic_bool is
0, so configeth downs the NIC and nothing brings it back until the node
reboots -- over the very interface the postscripts are talking on.
The other two arms pair their halves: the ifupdown arm answers `ifdown` with an
unconditional `ifup`, and the redhat arm gates both the down and the up. Only
the netplan arm takes one half.
Nothing in the suite covers it. This drives the two blocks configeth actually
executes -- the down-selection inside the modify branch and the restart block
underneath it -- with `ip`, `ifdown`, `ifup` and `netplan` shadowed by shell
functions, which bash resolves ahead of $PATH, so every call is recorded rather
than run and the host's network is never touched.
Six of the eight assertions pass already and are there to pin the arms that are
correct: redhat gating both halves, ifupdown pairing its own, and the
reboot_nic_bool=1 path doing the down before the apply. The two that fail are
the bug.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Cover the order that names the install device: installnic, then primarynic,
then mac.mac. Either attribute may name an interface or carry an address, and
the keyword mac returns to mac.mac.
Cover the device the kickstart names for each of those inputs. A node that
sets neither attribute keeps the address it has today. Cover the defect the
change closes, where a mac.mac entry that holds several untagged addresses
resolves to the last of them.
Cover that Ubuntu keeps its own pair of a name and an address over the same
resolution, and that the unique local address still comes from the hardware
address.
Pin the riscv64 %ifarch blocks in the xCAT, xCATsn and genesis specs
(genesis token, ipmitool-xcat, no x86 PXE loaders) and the riscv64
entries in buildcore.sh, buildlocal.sh and the buildrpms.pl notes.
Resolve the rocky10/rhels10 point releases through
imgutils::get_profile_def_filename with the OS search list emulated, and
assert the riscv64 files exist, win the lookup, match the x86_64 package
lists and point service images at the riscv64 dependency repository.
Pin the riscv64 installer kernel lookup in anaconda.pm and
geninitrd.pm, and evaluate the rh/genimage default driver and resolver
library blocks for riscv64 and the existing architectures.
Add an autotest nodeset case with bogus riscv64 grub2 nodes and a
rhels10.99 riscv64 osimage, checking the per-node grub2 files, the
grub2.riscv64 loader link and the hex-ip/mac links.
Drive mknb riscv64 --configfileonly through the existing stubs and
assert the grub2 network configuration: name, cpu guard, kernel, initrd,
xcatd endpoint, serial console, lzma preference, no PXELINUX/xNBA/
petitboot output, removal without artifacts and for :noboot interfaces.
Also install a riscv64 Genesis export and reject it for another
architecture.
Exercise _default_netboot() with the database-backed modules stubbed:
riscv64 defaults to grub2 and keeps any grub2 variant, the x86, PowerNV,
ppc, onie and aarch64 outcomes are unchanged, and undefined inputs do
not warn.
Assert the Kea xcat-riscv64 class (architecture 0x001b only, grub2
boot file, present with and without xNBA loaders) and pin the ISC
subnet block so the riscv64 branch stays ahead of the /yaboot
fallback that would otherwise shadow it.
Pin lookupNetboot, the profiled-node netboot rule table and the schema
descriptions for riscv64 without loading the database-backed modules:
the shipped subroutines are extracted from the source and evaluated
directly, so the test runs without DBI.
Daniel Hilst
Vinícius Ferrão