Twelve conflicts, all from xcat-core/#7970 landing.
Eleven are the one sed line in the netboot postinstall profiles, where #7970 and
this branch replaced the same line with different patterns. Master's wins: it
anchors on optional leading blanks and preserves a trailing comment, which this
branch's `s/^SELINUX=.*$/SELINUX=disabled/` would have eaten. This branch's own
postinstall_selinux_config.bats asserts "the comment that names permissive keeps
its text", so the better pattern is the one its test already wanted -- 6 of 6
pass on master's version. What master's pattern does not catch is a value
outside enforcing|permissive, which nothing here tests and no stock config
ships.
The twelfth is .github/workflows/xcat_test.yml: master added a step to the
existing job and this branch adds a whole selinux_policy_test job after it.
Resolved as the union, with master's step where it put it; the file still parses
and names three jobs.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Source-only signing modified retained binary RPMs and repository signatures while leaving the binary index unchanged. It also removed and recreated the bootstrap alias.
Limit signing to the source repository in this mode and leave the existing binary alias unchanged.
Two conflicts, both from xcat-core/#7978 landing on master.
build-utils/lib/XCAT/BuildUtils.pm: both branches added a package_skip_reason,
#7978 with a general @PACKAGE_SKIPS table and this branch with its own
%EL_ONLY_PACKAGES and a second sub. git merges them as separate hunks because
they sit apart in the file, and Perl takes the LAST definition, so keeping both
silently deletes whichever rule came first. There is now one table and one sub.
A rule names either the target prefixes that do NOT build the package, or, with
`only`, the pattern of the targets that DO. The second form is what an EL-only
package needs: naming the families that are not EL means a family added later
tries to build it, which is how openSUSE reached xCAT-selinux and failed
xcat-core-build-opensuse #42 on a missing selinux-policy-devel.
buildrpms.pl: an empty build plan takes upstream's answer, a clean return rather
than this branch's die. That is the decision #7978 exists for -- the dep
pipeline asks for xCAT-genesis-base alone and no openSUSE target builds it -- so
reinstating the die would fail a build that has nothing to do.
selinux_el_only_scope.t is 37 of 37 and opensuse_no_legacy_genesis.t 22 of 22 on
the merged tree, and the whole unit suite passes: 301 files, 9801 tests.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
xcat-dep-build-opensuse15-x86_64 fails its xCAT-genesis-base step. BuildRequires
resolve, and verify-genesis-payload then reports 566 missing zoneinfo/posix paths,
etc/redhat-release, and 45 EL paths and tool names -- net-tools-deprecated,
kmod-compat, iputils, systemd-sysvinit, timezone, nc, mkdosfs and the lvm symlinks.
The manifest is written for EL, so the package cannot be built on openSUSE.
openSUSE ships no legacy Genesis, so it does not need to be. package_skip_reason names
the pairs a target does not build and why, buildrpms.pl prints each skipped pair, and a
plan that skips everything returns 0 rather than failing: the dep pipeline asks for
xCAT-genesis-base alone, so on openSUSE its whole plan is empty and an empty plan is
the plan.
Dropping the package alone would make xCAT unresolvable there.
xCAT-genesis-scripts.spec requires xCAT-genesis-base-%{tarch} unconditionally and xCAT
requires xCAT-genesis-scripts, so zypper install xCAT would fail on a missing package
-- later than the build and looking like something else. That Requires now stands
behind %if ! 0%{?suse_version}; the Leap chroot's /usr/lib/rpm/suse/macros sets
%suse_version 1500, and no EL or openEuler target defines it.
xCAT-test/unit/opensuse_no_legacy_genesis.t covers the skip, the empty plan and the
conditional Requires.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
xCAT-selinux is in the default package set of buildrpms.pl, so every target
builds it. openEuler 20.03sp4 and 22.03sp4 carry selinux-policy 3.14.2, which
defines none of unconfined_server_stream_connectto(), files_map_etc_files() or
chronyd_manage_pid_files(). m4 leaves an undefined interface name in its
output and checkmodule stops with a syntax error, so rpmbuild fails and the
core build for that target publishes no repository at all.
SELinux support is EL only. XCAT::BuildUtils::package_skip_reason names the
packages an openEuler target does not build, and buildrpms.pl prints each pair
it skips. openEuler 24.03sp4 carries selinux-policy 40.7 and does compile the
module. It is excluded as well, so no openEuler release ships a package the
other releases cannot build.
xCAT-server and xCATsn required xCAT-selinux when selinux-policy-targeted was
installed, on openEuler too. openEuler defines %openEuler and no %rhel, so the
condition now names EL 8 and later and Fedora only. Without this an openEuler
management node cannot resolve xCAT-server.
xCAT-test/unit/selinux_el_only_scope.t fails on the parent commit.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
RPM signing can change bytes without changing the file size or whole-second
modification time used by the metadata cache. Rebuild binary and source
indexes after signing so their checksums describe the final RPMs.
Keep incremental indexing for unsigned builds.
xCAT-test/unit/fips_policy_sources.t ran build-utils/sync-fips-policy as a
subprocess, so a file in the unit directory measured a program rather than a
routine, and it asserted on the diagnostics the script printed instead of on an
answer. buildrpms.pl and builddebs.pl started the same script with system() to
check the generated libraries before packaging xCAT or xCAT-genesis-scripts.
XCAT::BuildUtils gains fips_policy_artifacts, fips_policy_plan and
sync_fips_policy. fips_policy_plan reads the canonical policy and both runtime
libraries and reports what generation would write, without writing.
sync_fips_policy applies that plan in write mode and returns the stale paths,
the paths it rewrote and the diagnostics for its caller to print. The two
builders now call sync_fips_policy in their own process and print the
diagnostics before they stop, which keeps the output xCAT-test/native/
fips_build_guard.t asserts on. build-utils/sync-fips-policy keeps its
--check and --write contract and its exit status, and is now a thin caller.
The unit test calls the sub in-process and asserts on the returned hash. It
also covers a library holding two policy blocks and a library replaced by a
symbolic link, which the subprocess form did not reach. Ten mutants of the
module, one per guard, each fail the assertion that names that guard.
xCAT-test/bats/fips_policy_artifacts.bats copies XCAT::BuildUtils into its
fixture, because the script it runs now loads the module.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Generate the Genesis and postscript policy blocks from one POSIX source.
Keep the runtime libraries self-contained and reject stale generated blocks
before RPM or Debian builds.
Preserve the existing state predicate, key algorithms, key sizes and
public-key export fallbacks. Keep caller-specific helpers separate.
xCAT shipped no SELinux policy. On an enforcing management node httpd
could not read /install, which had no file context, or the /tftpboot
files that xcat.conf serves, and xcatws.cgi could not connect to xcatd.
The new noarch xCAT-selinux package builds the xcat module with the
selinux-policy-devel Makefile. The module labels /install
public_content_t and leaves each rootimg tree out of the relabel. It lets
httpd read tftpdir_t and lets httpd_sys_script_t connect to ntop_port_t.
The scriptlets use the selinux-policy macros, and %posttrans relabels
/install. They do not change the SELinux mode.
checkmodule writes module version 19, so the package built in the EL10
mock root installs on EL8, EL9 and EL10. On a Red Hat family build,
xCAT-server and xCATsn require xCAT-selinux when selinux-policy-targeted
is installed. SUSE and Debian builds do not change.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Generate self-contained RPM and Debian upgrade scripts from the shared
source. Package transactions no longer load functions from the node-facing
postscript payload. Keep the existing command, logging and failure policies.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
buildrpms.pl staged each build's sources under $HOME/rpmbuild/sources.
mock now builds as the chrootuid of the target, uid 1000 for openEuler
24.03. On the Jenkins agents HOME is /root, mode 0550, so mock --sources
failed with "Permission denied" on the staging directory.
The staging directories now go under dist/sources in the checkout, next
to the mock result directories. The per-process names, the delete on
reap and the sweep of dead pids stay the same. buildrpms.pl makes the
staging tree readable by every user after it stages the sources, so the
umask of the build does not decide what mock can read.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
mock_build_owner parsed chrootuid and chrootgid from the text of the
rendered configuration. A value set in an included file, an assignment
with an inline comment or a double-quoted key, and an include after an
assignment all gave a uid that mock does not build as. Also, a root
build that stopped part way left root-owned 0644 logs in the result
directories, and the next build, as uid 1000, could not append to them.
mock_build_owner now asks mock's own loader, mockbuild.config.load_config,
for chrootuid and chrootgid of the chroot that buildrpms.pl passes to
mock -r. It fails with a clear message when mock is not installed.
prepare_mock_resultdirs moves to XCAT::BuildUtils. It also gives the
files already in both result directories to that owner, so a retry can
append to the old logs and replace a package of the same version. When
mock builds as the caller, nothing changes owner.
native/mock_resultdir_owner.t gave the wrong uid for four of its
configuration cases, and failed the 11 retry checks, before this change.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
PR #7897 made mock create its result directory as root by appending
config_opts['chrootuid'] = 0 to every per-package configuration. chrootuid
also selects the uid rpmbuild runs as, so every target now builds as root,
and the 1000 that openeuler-24.03-ppc64le declares no longer applies inside
the chroot.
Remove the assignment and give the result directory to the uid the target
declares. mock_build_owner reads chrootuid and chrootgid from the rendered
configuration, and prepare_mock_resultdirs creates dist/<target>/rpms and its
SRPMS subdirectory and chowns both before the build starts. openeuler-24.03-ppc64le
is the only target that declares a uid; elsewhere mock builds as the caller and
the directories are left alone.
buildrpms_source_only.t records the stages buildall runs, so it now expects the
new one between createmockconfig and buildsources.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Every package of xcat-core-build-openeuler24.03-ppc64le #1 died before rpmbuild
started:
ERROR: Could not create dir dist/openeuler-24.03-ppc64le/rpms/SRPMS/.
Error: [Errno 13] Permission denied: 'dist'
30 such lines, 0 srpms. buildrpms.pl passes a relative --resultdir inside a source
tree owned by root, and mock 6.8 creates that directory as chrootuid.
openeuler-24.03-ppc64le.cfg declares chrootuid 1000 for the native dependency
prerequisites, and createmockconfig copied the target configuration verbatim, so
the core build inherited a uid that cannot write there. No openEuler x86_64
configuration declares one, which is why only the ppc64le target fails.
createmockconfig had no unit test because it built its text inline and wrote it
straight to /etc/mock. The rendering moves to XCAT::BuildUtils::mock_config_text,
which takes the base text and returns the rendered text; the script keeps the two
file operations.
buildrpms_mock_resultdir_uid.t drives it directly and fails 2 of its 11 checks, both
on the uid the rendered configuration carries.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The staging directory was keyed by package, target and uniqueext. Nothing
deleted it, so every pair left a SOURCES tree under $HOME/rpmbuild, and an
archive from an earlier run stayed beside the next run's sources.
Each forked builder now stages into $HOME/rpmbuild/sources/SOURCES.<host>.<pid>.
The pid is unique among live processes on one host, and the host name keeps two
hosts apart when $HOME is on NFS. prepare_build_sources_dir creates the directory
empty. The parent deletes it when it reaps the builder, whatever the exit, and
abort_builds deletes it for each builder that exits after SIGTERM. At start, the
run deletes the directories of this host whose pid is not alive.
buildrpms_sources_per_package.t covers the name, the empty directory, the
deletion after success and failure, and the sweep of a dead pid next to a live
one and another host.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
xCATsn.spec extracts templates.tar.gz in %{prefix}/share/xcat. buildrpms.pl
creates that archive from the checkout root, so its members start with
xCAT/templates and the templates install under share/xcat/xCAT/templates.
With one shared SOURCES directory the xCAT build often wrote the archive
first, rooted at templates/, and the existence check kept it.
The tar step moves unchanged into stage_xcatsn_templates in
XCAT::BuildUtils. xcatsn_templates_staged.t stages the archive and reads its
members. It fails on this commit: the members start with xCAT/, and a second
staging keeps the old archive.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Every RPM target of every family failed in buildrpms.pl before it built a package:
Can't rename('.xCAT-probe-2.20.0.UNf6IG', '/xCAT-probe-2.20.0.tar.gz'):
Invalid cross-device link at ./buildrpms.pl line 386
The staging directory is set per package in the child that buildall forks. main()
staged the xCAT-probe tarball before that fork, where $SOURCES is still the empty
string, so the destination was composed as /xCAT-probe-<version>.tar.gz.
stage_xcat_probe_sources in XCAT::BuildUtils replaces prepare_xcat_probe_source_tar.
It takes the staging directory, refuses an unset or absent one, and composes the file
name itself. buildsources calls it for the xCAT-probe package, in the child, where the
directory exists.
xcat_probe_sources_staged.t runs the staging and reads the archive: the helpers are at
xCAT-probe/lib/perl/xCAT, no temporary file is left behind, and each refusal names the
guard that fired. Removing either guard turns it red. On the unfixed tree
"./buildrpms.pl --package xCAT-probe --source-only" dies at line 386; with this change
it reaches the build lock.
Seven assertions in xcat_probe_package_payload.t matched the text of buildrpms.pl
rather than running it, and four could not fail: three matched the xCAT-release
routine, which stages the same way, and one spanned the rest of the file. All seven
stayed green while every RPM target died. They are gone.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
A core build failed in %prep with "File
/builddir/build/SOURCES/xCAT-test-2.20.0.tar.gz is smaller than 13 bytes". Its
src.rpm carried 0 bytes where two sibling builds of the same commit carried
1671225.
buildrpms.pl staged every tarball into one $HOME/rpmbuild/SOURCES while forking
a child per package and target, so one child's tar overwrote an archive another
child was copying with mock --sources.
build_sources_dir keys the staging directory per package and target, like the
mock chroot. buildrpms_sources_per_package.t covers it.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
builddeb-genesis-base built its package list inline, between apt-get
update and apt-get install. A test could reach the list only by cutting
the REQUIRED_PACKAGES assignment out of the script and evaluating it.
It could not reach the choice between renamed packages (bind9-dnsutils
or dnsutils, util-linux-extra or util-linux) at all. The payload check
in verify-genesis-payload was bash, so a test could only run the script
and read its stderr.
XCAT::GenesisBuildRoot::required_packages() now returns the list for a
dpkg architecture. A code ref says which packages the release carries;
the default asks apt-cache. builddeb-genesis-base calls it at the same
point in the build and installs the same packages.
XCAT::GenesisPayload holds the mandatory-command list and the payload
check. verify-genesis-payload runs its main() and keeps the same
arguments, exit codes and messages. buildrpms.pl stages the module
beside the script. Both modules use core Perl only, because the build
root has perl-base and nothing more.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Master fixed debuild-xcat-genesis-base and added a bats test for it. The
branch deleted that script as unused. The merge keeps the script and
moves it to xCAT-genesis-base with the other Genesis base files.
Master also stages verify-genesis-payload for the spec %install check.
stage_genesis_base_sources() now stages it too, and
genesis_base_sources_staged.t expects it in the tarball.
The workflow, builds.rst and xcat_probe_package_payload.t take the
master version.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
buildsources_genesis_base() lived in buildrpms.pl, which runs mkdir, git
and file writes at file scope and cannot be loaded. Its test therefore
extracted the sub with a regex and evaluated a copy of the source.
Move the staging into XCAT::BuildUtils as stage_genesis_base_sources(),
which takes the checkout, the tarball path and SOURCE_DATE_EPOCH.
buildsources_genesis_base() now calls it. The tarball content and the
failure on a missing xCAT-genesis-base directory do not change.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Brings in #7866, which removes --database from the createrepo call. Without it the
branch cannot index a repository on the shared build tree: createrepo_c tries to
write primary.sqlite there and the NFS re-export answers
Cannot open .repodata/primary.sqlite: Can not create db_info table: disk I/O error
which failed every EL build in xcat-ci #44 while the Ubuntu builds, which do not use
createrepo, passed.
A build tree can live on an NFS re-export. The kernel refuses locks on one --
"Clients are not allowed to get file locks or delegations from a reexport
server" -- so every flock() there answers errno 524, and a build that takes one
dies before it starts.
buildrpms.pl's per-target lock and BuildUtils.pm's take_build_lock, which
builddebs.pl calls for the Ubuntu core build, are both atomic mkdir claims now.
Each records its owner and names it when it refuses.
A directory is not released by a filehandle closing, which is how both locks
were freed before. buildrpms.pl releases from END, and again in abort_builds
because that handler re-raises the signal with DEFAULT and END blocks do not run
then -- a killed build would otherwise strand the lock for every later one.
BuildUtils returns a small object whose DESTROY releases it, preserving the
caller's "hold the returned value" contract.
Both releases are guarded by owning pid: both scripts fork, and the flock they
replace could not be released by a child.
builddebs_lock.t closed the returned value to prove the lock is released, which
is "Not a GLOB reference" against the new contract. It now lets the value go out
of scope. What it asserts is unchanged: a second build of the same checkout is
refused, and the next one succeeds once the first releases.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
createrepo_dir passed --database, which writes *.sqlite.bz2. Building those
needs SQLite, and SQLite needs POSIX locks. A build tree can live on an NFS
re-export, where the kernel refuses locks outright: every attempt answers
errno 524.
So every target died with "Cannot open .repodata/primary.sqlite: Can not create
db_info table: disk I/O error", and the build staged nothing.
Measured on such a share, with a local control: a bare sqlite3 connect fails
there and succeeds on local disk; createrepo_c fails with --database and
succeeds without it, emitting primary/filelists/other as *.xml.zst.
Nothing this project ships reads the sqlite metadata. dnf on el8+ and zypper
both read the XML.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Merge current master into the openEuler support branch. Keep native
kernel, DHCP, locale and timezone requirements with the upstream payload
verifier and EL10 package paths. Preserve signed openEuler installation.
Update the native fixtures to check the merged payload contract.
The directory holds the Genesis image sources and, beside them, a builder that
no pipeline runs. buildrpm, debuild-xcat-genesis-base, install, install.ubuntu,
check, cmdlist_check, findmcprpms, xCAT-genesis-builder.spec and the root
makerpm-genesisbuilder have no caller. So has build-ubunturepo, which
builddebs.pl replaced, and its -r flag has set an unread variable since 2013.
docs/source/references/coral/known_issues/genesis_base.rst tells the reader to
install an rpm no build produces, and go-xcat offers to remove that package.
Delete all of them, and rename the directory xCAT-genesis-base for what it
holds: the dracut modules the EL spec and the Ubuntu builder read, the
OpenEmbedded layers the GitHub job builds, the deb packaging, the image scripts
and 80-net-name-slot.rules.
Every reader of the old path is updated: buildrpms.pl, .gitattributes, the OE
workflow and the OpenEmbedded tests. buildrpms.pl mapped the package
xCAT-genesis-base to the directory xCAT-genesis-builder; the two names are the
same now, so the mapping goes.
xcat_probe_package_payload.t read build-ubunturepo for the probe helper staging.
builddebs.pl stages them with XCAT::BuildUtils::stage_probe_helpers, which
build_utils.t drives directly.
genesis_base_sources_staged.t passes with this change.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
A compute node fetches the legacy Genesis kernel and initramfs, the kernel starts, and then
nothing else happens: doxcat never runs, the node acquires no address, sshd refuses every
connection and the node stays at status=powering-on. The five genesis test cases in
xCAT-test/autotest/testcase/genesis have never passed. VersatusHPC/xcat-internal#78.
Three holes in the image, each fatal on its own. dracut_105/el/xcat-cmdline.sh ends in
`while :; do tmux attach-session -t doxcat || tmux new-session -s doxcat doxcat; done`, and the
image carries no locale data, so tmux exits with "need UTF-8 locale" and the loop spins
without ever reaching doxcat. module-setup.sh does not install
/usr/libexec/openssh/sshd-session, which OpenSSH 9.8 and later exec for every connection and
which EL9 now ships. xCAT-genesis-base.spec does not BuildRequire dhcp-client, so dhclient is
absent from the build chroot; dracut_install reports the missing binary and returns, and the
module install function keeps going, so the image ships without it.
xcat-cmdline.sh now resolves xcat_console_mode() once and runs doxcat directly when the
terminal multiplexer cannot start a session; the same shape replaces the screen loop on
Ubuntu. module-setup.sh installs the OpenSSH session helpers and the C.utf8 locale where they
exist. The spec BuildRequires dhcp-client on the releases that package it, and runs the new
xCAT-genesis-builder/verify-genesis-payload over the extracted payload, which fails the build
when sshd needs a helper the image lacks, when tmux has no UTF-8 locale, or when a binary the
caller named is missing.
The same runs exposed four defects in the test cases themselves. test.sh defined its synthetic
node as ppc64le whatever the management node was, so nodeset could not find a genesis kernel on
x86_64. genesistest.pl get_os() matched neither AlmaLinux nor Rocky and reported the OS as
unsupported. The -g check read $? instead of check_genesis_file()'s return value, so it could
never fail. And testxdsh() met "REMOTE HOST IDENTIFICATION HAS CHANGED" from the second boot
on, because Genesis makes new host keys every boot and nothing dropped the stale known_hosts
entry. test.sh now derives the node arch from uname and takes the tftp root from TFTPDIR,
get_os() recognises the redhat family, report_genesis_files() carries the result to an exit
status, and forget_host_keys() runs makeknownhosts -r before each probe.
Tests: genesis_console_mode.t drives xcat_console_mode() with the multiplexer shadowed;
genesis_payload_verification.t drives the verifier over payload trees carrying each hole;
genesis_testcase_helpers.t drives get_os(), check_genesis_file(), report_genesis_files() and
testxdsh(); genesis_incorrectmasterip_check.t runs test.sh against a scratch tftp root. Each
fails on the parent commit. The verifier also reports all three holes against the released
xCAT-genesis-base-x86_64-2.19.0-snap202609021858 payload.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
(cherry picked from commit cb6021eb3cdb3abc75e4dd6704cb42b28074e140)
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The rebase onto master left the repository with two modules named
BuildUtils.pm: the shared build helpers at the root, package BuildUtils, and
the target architecture parser at build-utils/lib/XCAT/BuildUtils.pm, package
XCAT::BuildUtils. buildrpms.pl loaded both, one through `@INC` and one through a
path require. A reader cannot tell which module a BuildUtils reference names,
and the test sandbox staged the wrong one.
Move the shared helpers into build-utils/lib/XCAT/BuildUtils.pm as
XCAT::BuildUtils, and export targetarch_from_target beside them. Both builders
and the four tests now put build-utils/lib on `@INC` and import from the one
module. targetarch_from_target keeps its behaviour: it returns the same
architecture as before for suffixed targets, empty and undefined input, mixed
case and every architecture token.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The two builders wrote the same run-a-command-or-stop step in opposite
polarities. builddebs.pl used sh(...) == 0 or die; buildrpms.pl used
sh(...) and die, and also used the == 0 or die form for its sh_retry calls, so
both directions appeared in one file. The 'and die' spelling reads as though
the die is what happens next rather than what happens on failure, which is a
poor thing to have to re-read at every call site.
Add BuildUtils::sh_or_die and convert the fourteen plain sh() call sites to it.
The failure message now also carries the exit code, which every one of the old
spellings discarded -- a build that failed said only that a command failed, not
what it returned. The sh_retry sites keep their own form: retry is a different
operation and sh_retry is local to buildrpms.pl.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The two builders read and wrote files three different ways. buildrpms.pl used
File::Slurper, builddebs.pl opened and closed handles inline, and BuildUtils.pm
carried its own read_file/write_file pair to avoid the dependency. The result
was that the same operation was spelled differently in each place, and the
Version stamp was read by hand in both builders: buildrpms.pl chomped it ten
lines after reading it, which is how a stamp keeps a trailing newline nobody
notices until it reaches a package name.
Drop the hand-rolled pair and use File::Slurper throughout, and add
BuildUtils::read_line for the one-line Version and Release stamps -- it strips
the newline at the point of reading and reports an absent file as undef, which
is what builddebs.pl's fallback to snap_release needs.
File::Slurper is already required by buildrpms.pl and is already in the CI
package list, so this adds no new build-host prerequisite. The lock file still
uses a raw open: flock needs a real handle.
Behaviour note: Version is now read as its first line rather than the whole
file with the final newline removed. For the one-line file it is, these are the
same; a malformed multi-line Version now yields its first line instead of an
embedded newline.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Both builders ship a mklocalrepo.sh beside the packages they publish, and each
wrote it its own way: builddebs.pl opened, printed and closed the file by hand
and then chmod'ed it, while buildrpms.pl used File::Slurper and a separate
chmod. Writing the text and setting the executable bit are one operation -- a
copy published without 0775 is published broken -- but nothing tied them
together, so each caller had to remember the second step.
Add BuildUtils::write_script, which writes the file and sets the mode, and use
it from both builders. The mode is now asserted in build_utils.t instead of
being left to the callers to repeat.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Both builders wrote the same six fields beside a published repository --
VERSION, RELEASE, BUILD_TIME, BUILD_MACHINE, COMMIT_ID, COMMIT_ID_LONG -- each
assembling them by hand, and each shelling out to hostname to do it.
builddebs.pl also ran git rev-parse a second time to get the commit, although
it had already derived the same value into $GITINFO earlier in the file. It now
uses that.
The filenames and the time formats stay as they were. deploy.sh copies
builddebs.pl's buildinfo verbatim and cluster-test.pl parses buildrpms.pl's
buildinfo.txt, so both are a contract with consumers outside this repository;
the shared helper takes the format from its caller rather than picking one.
Confirmed byte for byte that each builder's stamp is unchanged.
Covered by tests: the field names and their order, the seven-character short
commit, and that the two callers' formats still differ.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Both builders defined their own sh() and usage(). The usage() copies differed
only in spelling, but the sh() copies disagreed about what they return:
buildrpms.pl shifted system()'s result into an exit code, builddebs.pl returned
the raw wait status. The same command therefore reported 1 from one builder and
256 from the other, and a caller comparing sh() against a particular code was
correct in only one of them. Both current call sites only test for zero, so
nothing was broken yet.
BuildUtils.pm now provides both, shifting the status as buildrpms.pl did.
pod2usage reads the POD of the running program, so each builder keeps its own
help text while sharing how it is printed and the status it exits with.
sh() echoed the command under the builder's own --verbose flag, which a shared
function cannot see. $BuildUtils::VERBOSE is set once after option parsing
instead of threading the flag through every call site.
Covered by tests: that sh() returns the exit code rather than the wait status
it is packed into, and that a verbose run still echoes the command.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
buildrpms.pl derived the git revision and SOURCE_DATE_EPOCH with its own inline
backticks while BuildUtils.pm already provided git_revision() and
source_date_epoch() for builddebs.pl. Both did the same thing, and the copies
in buildrpms.pl were the weaker ones: neither fell back to the tracked Gitinfo
and Gitepoch files, so a build from a source tarball with no git history
recorded the revision as "unknown" and stamped the current time as the build
epoch, losing reproducibility exactly where it matters most.
Verified that the helper returns the same revision and epoch as the code it
replaces on a checkout with history.
buildrpms.pl now loads BuildUtils.pm from its own directory, so the source-only
test stages the module into its sandbox alongside Version. That test copies the
builder rather than running it in the checkout, because it rewrites Gitinfo and
creates $HOME/rpmbuild before it looks at its arguments.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
buildrpms.pl already produces a source rpm for every package on every run --
buildall() is createmockconfig -> buildsources -> buildspkgs (mock --buildsrpm)
-> buildpkgs (mock --rebuild). Source-only is that sequence without the last
step, so it belongs here rather than in a parallel implementation: the spec, the
staged sources and the mock root are identical either way, and anything built
beside them can drift from what a real build does.
--source-only stops after buildspkgs. Two things downstream had to learn about
it, and both are about not publishing something untrue:
- index_repo no longer re-indexes the binary directory. Running createrepo_c
over a directory with no binaries in it would replace working metadata with
metadata for an empty repository -- a repo that resolves nothing. The srpm
index is still regenerated.
- write_repo_metadata_dir emits nothing. The .repo file and buildinfo describe
an installable binary repository, which this mode does not produce.
--source-only with --merge-core-repos is refused: one builds packages, the other
assembles per-arch trees that are already built.
buildrpms.pl cannot be loaded by a test -- it runs mkdir, git and read_text at
file scope -- so the two routines whose behaviour changed are lifted out with a
regex and eval'd into a scratch package with their collaborators stubbed, per
the code standard, with BAIL_OUT if the extraction stops matching. The CLI
contract is exercised by running the real program. Verified by mutation:
removing the index_repo guard reddens 2 of the 7 assertions.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
targetarch_from_target took the last dash-separated part of the mock
target name as the architecture. The riscv64 forcearch configuration is
named rocky-10-riscv64-xcat, so riscv64 builds looked for
xCAT-genesis-scripts-xcat srpms and failed. Take the last part that
names an architecture and only fall back to the last part when none
does; stock targets are unaffected.
Add riscv64 to the xCAT and xCATsn build paths, keep the unavailable legacy Genesis dependency conditional, collect the architecture RPMs, and document the forcearch mock target.