xCAT/xcat-dep
2
0
Fork 0
mirror of https://github.com/xcat2/xcat-dep.git synced 2026-09-12 12:36:23 +00:00
Code Issues Projects Releases Wiki Activity

fix(xcat-dep): drop host-install smoke; fix ppc mock-root collision

Browse Source 
PR #62 review raised two build-side problems:

1. The child builders installed each freshly built RPM onto the build host
   ("dnf -y install" + a run smoke). Because mockbuild-all builds el8/el9/el10
   on one host, this installs a foreign-EL RPM into the host RPM database and
   corrupts it. The real install-and-run verification already happens in the CI
   Test phase (cluster-test.pl boots a matching MN and installs xCAT + the
   freshly built xcat-dep there), so the host-side smoke was redundant. Remove
   the install/smoke step from every builder and the perl aggregate builder, and
   drop the now-dead --skip-install flag (builders, mockbuild-all.pl, and the
   pipeline invocations). The perl builder's --jobs 1 throttle existed only to
   avoid host dnf-lock contention during that install, so it goes too (perl
   packages build in parallel again). Also drop goconserver's now-unused run_rc.

2. build_mock_uniqueext truncated the run id by keeping the LAST 24 chars, which
   for the 7-char "ppc64le" arch dropped the leading EL digit -- so
   alma+epel-{8,9,10}-ppc64le collapsed to the same run part. goconserver
   compiles every EL in the el10 chroot (build_cfg rewritten to -10-), so the
   chroot name is identical across the three ELs and the uniqueext was the only
   thing keeping their mock roots apart: with parallel targets the three ppc
   goconserver builds raced in one root. Keep a readable leading token AND append
   a short digest of the full id so distinct ids always yield distinct uniqueext.
   Moved the helper into MockBuildUtils.pm and added fixtures (distinct per EL on
   a long run id, both arches, determinism).

Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
This commit is contained in:
Daniel Hilst
2026-08-21 18:42:28 -03:00
parent e898911431
commit 5addcaa976
12 changed files with 90 additions and 311 deletions
Download Patch File Download Diff File Expand all files Collapse all files
BUILD.md
+14 -24
View File
@@ -75,12 +75,16 @@ the whole run.
1. Optional chroot cleanup (`--scrub-all-chroots`)
2. Parallel build execution — only the target's manifest packages; any failure fails the run
3. Post-build chroot scrub — reclaims each build step's mock chroot (unless `--keep-buildroots`)
4. Optional install/smoke checks inside child builders (disabled with `--skip-install`)
5. Binary RPM collection into `repo/<ARCH>/`
6. Source RPM collection into `repo-src/`
7. `createrepo --update` on both repo trees
8. Tarball creation for both repo trees
9. Summary generation (`summary.txt`)
4. Binary RPM collection into `repo/<ARCH>/`
5. Source RPM collection into `repo-src/`
6. `createrepo --update` on both repo trees
7. Tarball creation for both repo trees
8. Summary generation (`summary.txt`)
The build process does **not** install any built RPM onto the build host. Installing an EL8/EL9
package on the (single, possibly EL10) build host corrupts the host RPM database; the real
install-and-run verification happens in the CI's separate Test phase (`cluster-test.pl` boots a
matching MN and installs xCAT + the freshly built xcat-dep there).
# Packages notes
@@ -96,8 +100,6 @@ the whole run.
Use these flags to skip specific operations:
- `--skip-install`
- Skips install/smoke checks performed by child builder scripts after RPM build.
- `--skip-genesis`
- Skips the `xCAT-genesis-base` build (`<XCAT_SOURCE>/buildrpms.pl --package xCAT-genesis-base`).
- `--skip-xcat-dep`
@@ -194,15 +196,15 @@ perl ./mockbuild-all.pl \
Notes:
- Install/smoke checks run by default inside child builders.
- Add `--skip-install` to skip those checks.
- The build never installs a built RPM onto the build host (see above); install-and-run
verification is the CI Test phase's job.
- Add `--skip-genesis` to skip the `xCAT-genesis-base` build (the only step that invokes
`<XCAT_SOURCE>/buildrpms.pl`).
- `<RUN_ID>` is optional; when omitted it is timestamp-based.
# Common Build Modes
xcat-dep repo (with install/smoke checks):
xcat-dep repo:
```bash
cd <REPO_ROOT>
@@ -212,17 +214,6 @@ perl ./mockbuild-all.pl \
--scrub-all-chroots
```
xcat-dep repo (skip install/smoke checks):
```bash
cd <REPO_ROOT>
perl ./mockbuild-all.pl \
--repo-root <REPO_ROOT> \
--xcat-source <XCAT_SOURCE> \
--scrub-all-chroots \
--skip-install
```
Dependency repo without the `xCAT-genesis-base` build:
```bash
@@ -231,8 +222,7 @@ perl ./mockbuild-all.pl \
--repo-root <REPO_ROOT> \
--xcat-source <XCAT_SOURCE> \
--scrub-all-chroots \
--skip-genesis \
--skip-install
--skip-genesis
```
Collection-only pass from existing build artifacts:
MockBuildUtils.pm
+39
View File
@@ -10,6 +10,7 @@ use File::Basename qw(basename);
use File::Copy qw(copy);
use File::Find;
use Sys::Hostname;
use Digest::MD5 qw(md5_hex);
our @EXPORT_OK = qw(
sh_quote print_step
@@ -17,6 +18,7 @@ our @EXPORT_OK = qw(
verify_repo_packages verify_repo_signature
rpm_version rpm_release rpm_sigmd5 rpm_is_signed restamp_release_line
cross_copy_genesis finalize_xcat_dep bump_dep_release_suffix
build_mock_uniqueext
);
# sh_quote: single-quote a string for safe use in a shell command.
@@ -377,4 +379,41 @@ sub bump_dep_release_suffix {
return $bumped;
}
# build_mock_uniqueext: a mock --uniqueext UNIQUE per (run, build-step) so concurrent mock builds
# never share a chroot root (/var/lib/mock/<cfg>-<uniqueext>). $run is the per-target run id
# (e.g. "alma+epel-8-ppc64le-<n>"), $seq orders the step, $label names the package.
#
# The run id must NOT be blindly tail-truncated. The per-target id leads with the EL/arch token, and
# for the 7-char "ppc64le" arch the EL digit is exactly what falls off the front of a keep-the-last-24
# truncation -- so alma+epel-{8,9,10}-ppc64le all collapse to the same run part. That is catastrophic
# for goconserver, which compiles EVERY EL in the el10 chroot (build_cfg rewritten to -10-): the
# chroot NAME is then identical across the three ELs, and the uniqueext is the ONLY thing keeping
# their roots apart, so three concurrent el8/el9/el10 ppc64le goconserver builds race in one root.
# When the id is too long, keep a readable leading token AND append a short digest of the FULL id, so
# distinct ids always yield distinct uniqueext regardless of where in the string they differ.
sub build_mock_uniqueext {
my ($run, $seq, $label) = @_;
my $run_part = defined($run) ? $run : 'run';
$run_part =~ s/[^A-Za-z0-9_.-]+/-/g;
$run_part =~ s/^-+|-+$//g;
$run_part = 'run' if $run_part eq '';
if (length($run_part) > 24) {
my $digest = substr(md5_hex($run_part), 0, 8);
(my $head = substr($run_part, 0, 15)) =~ s/-+$//;
$run_part = "$head-$digest";
}
my $label_part = defined($label) ? $label : 'step';
$label_part =~ s/[^A-Za-z0-9_.-]+/-/g;
$label_part =~ s/^-+|-+$//g;
$label_part = 'step' if $label_part eq '';
$label_part = substr($label_part, 0, 20) if length($label_part) > 20;
my $idx = defined($seq) ? int($seq) : 0;
$idx = 0 if $idx < 0;
return sprintf("mba-%02d-%s-%s", $idx, $run_part, $label_part);
}
1;
conserver/mockbuild.pl
+4 -20
View File
@@ -1,11 +1,10 @@
#!/usr/bin/perl
#
# mockbuild.pl - build conserver-xcat (the traditional C conserver, 8.2.1) for one
# mock target and smoke-test the resulting binaries. Mirrors the other xcat-dep
# mock target. Mirrors the other xcat-dep
# builders (goconserver/mockbuild.pl, ipmitool/mockbuild.pl): stage sources + spec,
# build a SRPM, `mock --rebuild` it in the target chroot, copy the RPMs to
# --result-dir, then (unless --skip-install) install into the chroot and run
# `console -V` / `conserver -V` to confirm the binaries work.
# build a SRPM, `mock --rebuild` it in the target chroot, and copy the RPMs to
# --result-dir.
#
# conserver is NOT part of the default mockbuild-all.pl dep set (xCAT uses goconserver),
# so this builder is standalone. Usage:
@@ -28,7 +27,6 @@ my $mock_cfg = '';
my $mock_uniqueext = '';
my $result_dir = "$script_dir/../build-output/list-conserver/conserver";
my $log_dir = "$script_dir/../build-logs/list-conserver/conserver";
my $skip_install = 0;
my $build_timestamp;
GetOptions(
@@ -37,7 +35,6 @@ GetOptions(
'mock-uniqueext=s' => \$mock_uniqueext,
'result-dir=s' => \$result_dir,
'log-dir=s' => \$log_dir,
'skip-install!' => \$skip_install,
'build-timestamp=i' => \$build_timestamp,
) or die usage();
@@ -112,19 +109,6 @@ for my $r (@rpms) {
}
print "built: " . join(', ', map { basename($_) } @rpms) . "\n";
# ---- smoke test: install into the chroot and run the binaries ---------------
unless ($skip_install) {
print "== Smoke test (install + run) ==\n";
run("mock -r " . sh_quote($mock_cfg) . $uniq . " --install " . sh_quote($main)
. " > " . sh_quote("$log_dir/install.log") . " 2>&1");
# console (client) and conserver (daemon) both print their version to stderr/stdout.
my $smoke = capture("mock -r " . sh_quote($mock_cfg) . $uniq
. " --chroot -- " . sh_quote('/usr/bin/console -V 2>&1; /usr/sbin/conserver -V 2>&1')
. " 2>&1");
print " output: $smoke\n";
die "FATAL: smoke test did not report version $version\n" unless $smoke =~ /\Q$version\E/;
print "smoke test PASSED (console/conserver report $version)\n";
}
print "DONE: conserver-xcat $version for $mock_cfg -> $result_dir\n";
# ---- helpers ----------------------------------------------------------------
@@ -143,5 +127,5 @@ sub capture {
sub sh_quote { my ($s) = @_; $s =~ s/'/'\\''/g; return "'$s'"; }
sub usage {
return "usage: mockbuild.pl --mock-cfg <cfg> [--result-dir DIR] [--work-dir DIR]\n"
. " [--log-dir DIR] [--mock-uniqueext EXT] [--skip-install]\n";
. " [--log-dir DIR] [--mock-uniqueext EXT]\n";
}
elilo/mockbuild.pl
-35
View File
@@ -19,7 +19,6 @@ my $mock_cfg = '';
my $mock_uniqueext = '';
my $result_dir = "$repo_root/build-output/list3/elilo-xcat";
my $log_dir = "$repo_root/build-logs/list3/elilo-xcat";
my $skip_install = 0;
my $build_timestamp;
GetOptions(
@@ -29,7 +28,6 @@ GetOptions(
'mock-uniqueext=s' => \$mock_uniqueext,
'result-dir=s' => \$result_dir,
'log-dir=s' => \$log_dir,
'skip-install!' => \$skip_install,
'build-timestamp=i' => \$build_timestamp,
) or die usage();
@@ -79,7 +77,6 @@ print "log_dir: $log_dir\n";
print "mock_cfg: $mock_cfg\n";
print "mock_uniqueext: " . ($mock_uniqueext ne '' ? $mock_uniqueext : '(none)') . "\n";
print "source_file:$source_file\n";
print "skip_install: $skip_install\n";
make_path($result_dir);
make_path($log_dir);
@@ -207,37 +204,6 @@ for my $log (qw(build.log root.log state.log hw_info.log installed_pkgs.log)) {
or die "Failed to copy $src to $log_dir: $!\n";
}
if (!$skip_install) {
print_step("Install RPM and run smoke tests");
run("dnf -y install " . sh_quote($main_rpm));
my $efi_file = '/tftpboot/xcat/elilo-x64.efi';
die "Missing installed EFI binary: $efi_file\n" if !-f $efi_file;
my $file_log = "$log_dir/smoke-file.log";
my $qf_log = "$log_dir/smoke-rpm-qf.log";
my $rc_file = run_capture_rc("file $efi_file", $file_log);
my $rc_qf = run_capture_rc("rpm -qf $efi_file", $qf_log);
die "Smoke check failed: file returned $rc_file\n" if $rc_file != 0;
die "Smoke check failed: rpm -qf returned $rc_qf\n" if $rc_qf != 0;
my $file_out = slurp($file_log);
my $qf_out = slurp($qf_log);
die "EFI file signature check failed:\n$file_out\n"
if $file_out !~ /(EFI application|PE32\+ executable)/i;
die "Installed file is not owned by elilo-xcat:\n$qf_out\n"
if $qf_out !~ /^elilo-xcat-/m;
my $summary = "$log_dir/smoke-summary.txt";
open my $sfh, '>', $summary or die "Cannot write $summary: $!\n";
print {$sfh} "efi_file=$efi_file\n";
print {$sfh} "rc_file=$rc_file\n";
print {$sfh} "rc_qf=$rc_qf\n";
close $sfh;
}
print_step("Completed");
print "Main RPM: $main_rpm\n";
print "Artifacts: $result_dir\n";
@@ -253,7 +219,6 @@ Usage: $0 [options]
--mock-uniqueext TXT Optional mock --uniqueext suffix to isolate concurrent builds
--result-dir PATH Output RPM/SRPM directory (default: $result_dir)
--log-dir PATH Log directory (default: $log_dir)
--skip-install Skip dnf install + smoke tests
--build-timestamp EPOCH Unix timestamp for SOURCE_DATE_EPOCH (deterministic builds)
USAGE
}
goconserver/mockbuild.pl
-24
View File
@@ -17,7 +17,6 @@ my $mock_cfg = '';
my $mock_uniqueext = '';
my $result_dir = "$repo_root/build-output/list5/goconserver";
my $log_dir = "$repo_root/build-logs/list5/goconserver";
my $skip_install = 0;
my $version = '0.3.3';
my $go_repo = 'https://github.com/xcat2/goconserver.git';
# Immutable pin: goconserver 0.3.3 is unreleased (newest tag v0.3.2) so it lives only on master.
@@ -33,7 +32,6 @@ GetOptions(
'mock-uniqueext=s' => \$mock_uniqueext,
'result-dir=s' => \$result_dir,
'log-dir=s' => \$log_dir,
'skip-install!' => \$skip_install,
'version=s' => \$version,
'go-repo=s' => \$go_repo,
'go-ref=s' => \$go_ref,
@@ -95,7 +93,6 @@ print "arch: $arch\n";
print "version: $version\n";
print "go_ref: $go_ref\n";
print "release_suffix: " . ($release_suffix ne '' ? $release_suffix : '(none)') . "\n";
print "skip_install: $skip_install\n";
make_path($result_dir);
make_path($log_dir);
@@ -288,19 +285,6 @@ for my $log (qw(build.log root.log state.log)) {
system("mock -r " . sh_quote($build_cfg) . $mock_uniqueext_opt .
" --scrub=chroot --scrub=bootstrap >" . sh_quote("$log_dir/mock-scrub.log") . " 2>&1");
if (!$skip_install) {
print_step("Install and smoke test");
my $main_rpm = $arch_rpms[0];
run("dnf -y install " . sh_quote($main_rpm) . " >" . sh_quote("$log_dir/dnf-install.log") . " 2>&1");
die "Missing /usr/bin/goconserver\n" if !-x '/usr/bin/goconserver';
die "Missing /usr/bin/congo\n" if !-x '/usr/bin/congo';
my $rc_help = run_rc("goconserver -h >" . sh_quote("$log_dir/smoke-help.log") . " 2>&1");
die "goconserver -h failed (rc=$rc_help)\n" if $rc_help > 1;
my $rc_congo = run_rc("congo -h >" . sh_quote("$log_dir/smoke-congo.log") . " 2>&1");
die "congo -h failed (rc=$rc_congo)\n" if $rc_congo > 1;
print "Smoke tests passed.\n";
}
print_step("Completed");
print "Results in: $result_dir\n";
exit 0;
@@ -321,7 +305,6 @@ Options:
--mock-uniqueext STR Mock uniqueext (for concurrency isolation under mockbuild-all.pl)
--result-dir PATH Output directory for RPMs
--log-dir PATH Output directory for logs
--skip-install Skip dnf install + smoke tests
--version VER Version string (default: 0.3.3)
--go-repo URL Git repo URL (default: github.com/xcat2/goconserver)
--go-ref REF Git ref/SHA to build (default: the pinned commit)
@@ -352,13 +335,6 @@ sub run {
}
}
sub run_rc {
my ($cmd) = @_;
print "+ $cmd\n";
my $rc = system($cmd);
return $rc == -1 ? 255 : ($rc >> 8);
}
sub capture {
my ($cmd) = @_;
my $out = `$cmd`;
grub2-xcat/mockbuild.pl
-46
View File
@@ -20,7 +20,6 @@ my $mock_cfg = '';
my $mock_uniqueext = '';
my $result_dir = "$repo_root/build-output/list3/grub2-xcat";
my $log_dir = "$repo_root/build-logs/list3/grub2-xcat";
my $skip_install = 0;
my $build_timestamp;
GetOptions(
@@ -30,7 +29,6 @@ GetOptions(
'mock-uniqueext=s' => \$mock_uniqueext,
'result-dir=s' => \$result_dir,
'log-dir=s' => \$log_dir,
'skip-install!' => \$skip_install,
'build-timestamp=i' => \$build_timestamp,
) or die usage();
@@ -88,7 +86,6 @@ print "result_dir: $result_dir\n";
print "log_dir: $log_dir\n";
print "mock_cfg: $mock_cfg\n";
print "mock_uniqueext: " . ($mock_uniqueext ne '' ? $mock_uniqueext : '(none)') . "\n";
print "skip_install: $skip_install\n";
make_path($result_dir);
make_path($log_dir);
@@ -218,48 +215,6 @@ for my $log (qw(build.log root.log state.log hw_info.log installed_pkgs.log)) {
or die "Failed to copy $src to $log_dir: $!\n";
}
if (!$skip_install) {
print_step("Install RPM and run smoke tests");
run("dnf -y install " . sh_quote($main_rpm));
my $core = '/tftpboot/boot/grub2/powerpc-ieee1275/core.elf';
my $grub2ppc = '/tftpboot/boot/grub2/grub2.ppc';
die "Missing installed core image: $core\n" if !-f $core;
die "Missing installed post script output: $grub2ppc\n" if !-f $grub2ppc;
my $file_core_log = "$log_dir/smoke-file-core.log";
my $file_ppc_log = "$log_dir/smoke-file-grub2ppc.log";
my $qf_log = "$log_dir/smoke-rpm-qf.log";
my $rc_file_core = run_capture_rc("file $core", $file_core_log);
my $rc_file_ppc = run_capture_rc("file $grub2ppc", $file_ppc_log);
my $rc_qf = run_capture_rc("rpm -qf $core", $qf_log);
my $rc_cmp = run_capture_rc("cmp -s $core $grub2ppc", "$log_dir/smoke-cmp.log");
die "Smoke check failed: file core returned $rc_file_core\n" if $rc_file_core != 0;
die "Smoke check failed: file grub2.ppc returned $rc_file_ppc\n" if $rc_file_ppc != 0;
die "Smoke check failed: rpm -qf returned $rc_qf\n" if $rc_qf != 0;
die "Smoke check failed: core.elf and grub2.ppc differ (cmp rc=$rc_cmp)\n" if $rc_cmp != 0;
my $core_out = slurp($file_core_log);
my $qf_out = slurp($qf_log);
die "Core image signature check failed:\n$core_out\n"
if $core_out !~ /ELF|data/i;
die "Installed core image is not owned by grub2-xcat:\n$qf_out\n"
if $qf_out !~ /^grub2-xcat-/m;
my $summary = "$log_dir/smoke-summary.txt";
open my $sfh, '>', $summary or die "Cannot write $summary: $!\n";
print {$sfh} "core=$core\n";
print {$sfh} "grub2ppc=$grub2ppc\n";
print {$sfh} "rc_file_core=$rc_file_core\n";
print {$sfh} "rc_file_ppc=$rc_file_ppc\n";
print {$sfh} "rc_qf=$rc_qf\n";
print {$sfh} "rc_cmp=$rc_cmp\n";
close $sfh;
}
print_step("Completed");
print "Main RPM: $main_rpm\n";
print "Artifacts: $result_dir\n";
@@ -275,7 +230,6 @@ Usage: $0 [options]
--mock-uniqueext TXT Optional mock --uniqueext suffix to isolate concurrent builds
--result-dir PATH Output RPM/SRPM directory (default: $result_dir)
--log-dir PATH Log directory (default: $log_dir)
--skip-install Skip dnf install + smoke tests
--build-timestamp EPOCH Unix timestamp for deterministic builds (SOURCE_DATE_EPOCH)
USAGE
}
ipmitool/mockbuild.pl
-58
View File
@@ -19,7 +19,6 @@ my $mock_cfg = '';
my $mock_uniqueext = '';
my $result_dir = "$repo_root/build-output/list3/ipmitool-xcat";
my $log_dir = "$repo_root/build-logs/list3/ipmitool-xcat";
my $skip_install = 0;
my $build_timestamp;
GetOptions(
@@ -29,7 +28,6 @@ GetOptions(
'mock-uniqueext=s' => \$mock_uniqueext,
'result-dir=s' => \$result_dir,
'log-dir=s' => \$log_dir,
'skip-install!' => \$skip_install,
'build-timestamp=i' => \$build_timestamp,
) or die usage();
@@ -79,7 +77,6 @@ print "log_dir: $log_dir\n";
print "mock_cfg: $mock_cfg\n";
print "mock_uniqueext: " . ($mock_uniqueext ne '' ? $mock_uniqueext : '(none)') . "\n";
print "source_file:$source_file\n";
print "skip_install: $skip_install\n";
print "SOURCE_DATE_EPOCH: $SOURCE_DATE_EPOCH\n";
make_path($result_dir);
@@ -206,60 +203,6 @@ for my $log (qw(build.log root.log state.log hw_info.log installed_pkgs.log)) {
or die "Failed to copy $src to $log_dir: $!\n";
}
if (!$skip_install) {
print_step("Install RPM and run smoke tests");
run("dnf -y install " . sh_quote($main_rpm));
my $bin = '/opt/xcat/bin/ipmitool-xcat';
die "Missing installed binary: $bin\n" if !-x $bin;
my $help_short_log = "$log_dir/smoke-help-short.log";
my $help_long_log = "$log_dir/smoke-help-long.log";
my $version_log = "$log_dir/smoke-version.log";
my $open_log = "$log_dir/smoke-open-mc-info.log";
my $ldd_log = "$log_dir/smoke-ldd.log";
my $rc_help_short = run_capture_rc("$bin -h", $help_short_log);
my $rc_help_long = run_capture_rc("$bin --help", $help_long_log);
my $rc_version = run_capture_rc("$bin -V", $version_log);
my $rc_open = run_capture_rc("$bin -I open mc info", $open_log);
my $rc_ldd = run_capture_rc("ldd $bin", $ldd_log);
die "Smoke check failed: -h returned $rc_help_short\n" if $rc_help_short != 0;
die "Smoke check failed: -V returned $rc_version\n" if $rc_version != 0;
die "Smoke check failed: ldd returned $rc_ldd\n" if $rc_ldd != 0;
my $help_short_out = slurp($help_short_log);
my $help_long_out = slurp($help_long_log);
my $version_out = slurp($version_log);
my $open_out = slurp($open_log);
my $ldd_out = slurp($ldd_log);
die "Short help output does not contain usage text\n"
if $help_short_out !~ /usage:/i;
die "Long help output does not contain usage text\n"
if $help_long_out !~ /usage:/i;
die "Long help returned unexpected rc=$rc_help_long (expected 0 or 1)\n"
if $rc_help_long != 0 && $rc_help_long != 1;
die "Version output missing expected version string\n"
if $version_out !~ /ipmitool-xcat version \Q$version\E/i;
die "ldd output missing libcrypto dependency\n"
if $ldd_out !~ /libcrypto/;
if ($rc_open != 0 && $open_out !~ m{Could not open device|/dev/ipmi}) {
die "IPMI probe failed with unexpected output:\n$open_out\n";
}
my $summary = "$log_dir/smoke-summary.txt";
open my $sfh, '>', $summary or die "Cannot write $summary: $!\n";
print {$sfh} "binary=$bin\n";
print {$sfh} "rc_help_short=$rc_help_short\n";
print {$sfh} "rc_help_long=$rc_help_long\n";
print {$sfh} "rc_version=$rc_version\n";
print {$sfh} "rc_open=$rc_open\n";
print {$sfh} "rc_ldd=$rc_ldd\n";
close $sfh;
}
print_step("Completed");
print "Main RPM: $main_rpm\n";
print "Artifacts: $result_dir\n";
@@ -275,7 +218,6 @@ Usage: $0 [options]
--mock-uniqueext TXT Optional mock --uniqueext suffix to isolate concurrent builds
--result-dir PATH Output RPM/SRPM directory (default: $result_dir)
--log-dir PATH Log directory (default: $log_dir)
--skip-install Skip dnf install + smoke tests
--build-timestamp N Unix epoch for SOURCE_DATE_EPOCH (deterministic builds)
USAGE
}
mockbuild-all.pl
+2 -28
View File
@@ -16,7 +16,8 @@ use lib $RealBin;
use MockBuildUtils qw(sh_quote print_step version_matches required_pkgs
read_manifest verify_repo_packages verify_repo_signature
rpm_version rpm_release rpm_sigmd5 restamp_release_line
cross_copy_genesis finalize_xcat_dep bump_dep_release_suffix);
cross_copy_genesis finalize_xcat_dep bump_dep_release_suffix
build_mock_uniqueext);
# --- Mount-namespace isolation: guard the host cgroup against mock teardown propagation ----------
# mock mounts /sys/fs/cgroup into every build chroot. On these systemd build hosts every mount is
@@ -80,7 +81,6 @@ my $build_number;
# newest tag is v0.3.2), so it exists only on master -- pin an immutable SHA instead of the moving
# branch so the build is reproducible. Bump this deliberately when uptaking a new goconserver.
my $GOCONSERVER_REF = '6166fe5ec1c5b3c20475e322a9f0e8e93c87e45f';
my $skip_install = 0;
my $skip_build = 0;
my $skip_xcat_dep = 0;
my $skip_perl = 0;
@@ -134,7 +134,6 @@ GetOptions(
'run-id=s' => \$run_id,
'build-timestamp=i' => \$build_timestamp,
'build-number=i' => \$build_number,
'skip-install!' => \$skip_install,
'skip-build!' => \$skip_build,
'skip-xcat-dep!' => \$skip_xcat_dep,
'skip-perl!' => \$skip_perl,
@@ -443,7 +442,6 @@ print "skip_build: $skip_build\n";
print "skip_xcat_dep: $skip_xcat_dep\n";
print "skip_perl: $skip_perl\n";
print "skip_genesis: $skip_genesis\n";
print "skip_install: $skip_install\n";
print "skip_createrepo: $skip_createrepo\n";
print "skip_tarball: $skip_tarball\n";
print "scrub_all_chroots:$scrub_all_chroots\n";
@@ -485,7 +483,6 @@ if (!$skip_build) {
# host-local, run-scoped work dir so /tmp doesn't collide between runs
'--work-dir', sh_quote("/tmp/mockbuild-all-$run_id/$name"),
'--build-timestamp', $SOURCE_DATE_EPOCH,
($skip_install ? '--skip-install' : ()),
# goconserver generates its spec at build time (from an upstream clone), so the
# in-tree spec Release bump above cannot reach it. Hand the CD suffix down so its
# NVR advances per run too, and pin the clone to an immutable commit (not the moving
@@ -529,7 +526,6 @@ if (!$skip_build) {
# packages; the srpm-mode ones (HTML-Form, IO-Stty, Net-Telnet) build from a
# committed .src.rpm, so hand the suffix down for the builder to re-stamp them.
($RELEASE_BUMP ne '' ? ('--release-suffix', sh_quote($RELEASE_BUMP)) : ()),
($skip_install ? '--skip-install' : ()),
($keep_buildroots ? '--keep-buildroots' : ()),
);
push @build_steps, {
@@ -972,7 +968,6 @@ Options:
0/auto = host nproc (default: auto)
--run-id ID Run identifier suffix (default: derived from build timestamp)
--build-timestamp EPOCH Unix epoch for deterministic builds (default: Gitepoch or git log)
--skip-install Skip install/smoke tests in child builder scripts
--skip-build Skip all build steps and only collect/create repo/tarballs
--skip-xcat-dep Skip xcat-dep mockbuild.pl package steps
--skip-perl Skip perl package build step
@@ -1408,27 +1403,6 @@ sub resolve_mock_cfg {
. "(tried /etc/mock/${os_id}+epel-${rel}-${arch}.cfg and /etc/mock/${short}+epel-${rel}-${arch}.cfg)\n";
}
sub build_mock_uniqueext {
my ($run, $seq, $label) = @_;
my $run_part = defined($run) ? $run : 'run';
$run_part =~ s/[^A-Za-z0-9_.-]+/-/g;
$run_part =~ s/^-+|-+$//g;
$run_part = 'run' if $run_part eq '';
$run_part = substr($run_part, -24) if length($run_part) > 24;
my $label_part = defined($label) ? $label : 'step';
$label_part =~ s/[^A-Za-z0-9_.-]+/-/g;
$label_part =~ s/^-+|-+$//g;
$label_part = 'step' if $label_part eq '';
$label_part = substr($label_part, 0, 20) if length($label_part) > 20;
my $idx = defined($seq) ? int($seq) : 0;
$idx = 0 if $idx < 0;
return sprintf("mba-%02d-%s-%s", $idx, $run_part, $label_part);
}
sub resolve_xcat_source {
my ($requested, $root) = @_;
# Prefer the sibling ../xcat-core (the real layout: source/xcat-core beside source/xcat-dep)
mockbuild-perl-packages.pl
-25
View File
@@ -18,8 +18,6 @@ my $result_dir = '';
my $log_dir = '';
my $packages_csv = '';
my $jobs = 0;
my $skip_install = 0;
my $allow_erasing = 0;
my $build_timestamp;
# CD version bump: appended to the Release of the srpm-mode packages (HTML-Form, IO-Stty,
# Net-Telnet), which build from a committed .src.rpm and so are NOT covered by mockbuild-all's
@@ -35,8 +33,6 @@ GetOptions(
'log-dir=s' => \$log_dir,
'packages=s' => \$packages_csv,
'jobs=i' => \$jobs,
'skip-install!' => \$skip_install,
'allow-erasing!' => \$allow_erasing,
'build-timestamp=i' => \$build_timestamp,
'release-suffix=s' => \$release_suffix,
) or die usage();
@@ -170,10 +166,6 @@ $jobs = 1 if $jobs < 1;
if (@packages && $jobs > scalar(@packages)) {
$jobs = scalar(@packages);
}
if (!$skip_install && $jobs > 1) {
print "INFO: --skip-install is disabled; forcing --jobs 1 to avoid host dnf lock contention\n";
$jobs = 1;
}
make_path($result_dir);
make_path($log_dir);
@@ -189,8 +181,6 @@ print "mock_cfg: $mock_cfg\n";
print "mock_uniqueext: " . ($mock_uniqueext ne '' ? $mock_uniqueext : '(none)') . "\n";
print "packages: " . join(', ', @packages) . "\n";
print "jobs: $jobs\n";
print "skip_install:$skip_install\n";
print "allow_erasing:$allow_erasing\n";
print "release_suffix:" . ($release_suffix ne '' ? $release_suffix : '(none)') . "\n";
print_step("Mock config check");
@@ -230,8 +220,6 @@ for my $idx (0 .. $#packages) {
mock_cfg => $mock_cfg,
mock_uniqueext => $pkg_uniqueext,
arch => $arch,
skip_install => $skip_install,
allow_erasing => $allow_erasing,
release_suffix => $release_suffix,
);
unless ($keep_buildroots) {
@@ -328,8 +316,6 @@ sub build_package {
my $mock_cfg = $args{mock_cfg};
my $mock_uniqueext = $args{mock_uniqueext};
my $arch = $args{arch};
my $skip_install = $args{skip_install};
my $allow_erasing = $args{allow_erasing};
my $release_suffix = $args{release_suffix};
my $pkg_run_dir = "$work_dir/$pkg";
@@ -506,15 +492,6 @@ sub build_package {
}
}
if (!$skip_install) {
my $install_cmd = "dnf -y install ";
$install_cmd .= "--allowerasing " if $allow_erasing;
run($install_cmd . sh_quote($main_rpm));
my $module = $cfg->{module};
my $rc_mod = run_capture_rc("perl -M$module -e 1", "$pkg_log/smoke-perl-module.log");
die "Perl module import failed for $pkg ($module), rc=$rc_mod\n" if $rc_mod != 0;
}
$summary = "$pkg PASS main_rpm=" . basename($main_rpm);
$ok = 1;
};
@@ -561,8 +538,6 @@ Usage: $0 [options]
--build-timestamp EPOCH Unix epoch for SOURCE_DATE_EPOCH (deterministic builds)
--release-suffix STR CD bump appended to the Release of the srpm-mode packages that build
from a committed .src.rpm (HTML-Form, IO-Stty, Net-Telnet)
--skip-install Skip dnf install + perl module import checks
--allow-erasing Allow dnf to erase conflicting packages during install smoke tests
USAGE
}
syslinux/mockbuild.pl
-46
View File
@@ -19,7 +19,6 @@ my $mock_cfg = '';
my $mock_uniqueext = '';
my $result_dir = "$repo_root/build-output/list3/syslinux-xcat";
my $log_dir = "$repo_root/build-logs/list3/syslinux-xcat";
my $skip_install = 0;
my $build_timestamp;
GetOptions(
@@ -29,7 +28,6 @@ GetOptions(
'mock-uniqueext=s' => \$mock_uniqueext,
'result-dir=s' => \$result_dir,
'log-dir=s' => \$log_dir,
'skip-install!' => \$skip_install,
'build-timestamp=i' => \$build_timestamp,
) or die usage();
@@ -87,7 +85,6 @@ print "version: $version\n";
print "mock_cfg: $mock_cfg\n";
print "mock_uniqueext: " . ($mock_uniqueext ne '' ? $mock_uniqueext : '(none)') . "\n";
print "source_file:$source_file\n";
print "skip_install: $skip_install\n";
make_path($result_dir);
make_path($log_dir);
@@ -226,48 +223,6 @@ for my $log (qw(build.log root.log state.log hw_info.log installed_pkgs.log)) {
or die "Failed to copy $src to $log_dir: $!\n";
}
if (!$skip_install) {
print_step("Install RPM(s) and run smoke tests");
run("dnf -y install " . sh_quote($xcat_rpm));
my $pxe_file = '/opt/xcat/share/xcat/netboot/syslinux/pxelinux.0';
die "Missing installed PXE file: $pxe_file\n" if !-f $pxe_file;
my $file_log = "$log_dir/smoke-file.log";
my $qf_log = "$log_dir/smoke-rpm-qf.log";
my $rc_file = run_capture_rc("file $pxe_file", $file_log);
my $rc_qf = run_capture_rc("rpm -qf $pxe_file", $qf_log);
die "Smoke check failed: file returned $rc_file\n" if $rc_file != 0;
die "Smoke check failed: rpm -qf returned $rc_qf\n" if $rc_qf != 0;
my $qf_out = slurp($qf_log);
die "Installed file is not owned by syslinux-xcat:\n$qf_out\n"
if $qf_out !~ /^syslinux-xcat-/m;
# EL10 hosts may already carry syslinux-nonlinux, which conflicts with
# directly installing the rebuilt syslinux payload. The xcat subpackage
# is the artifact we need to smoke-test on the host; repository-level
# dependency validation happens later in the full install phase.
my $syslinux_help_log = "$log_dir/smoke-syslinux-help.log";
if (-x '/usr/bin/syslinux') {
my $rc_help = run_capture_rc("/usr/bin/syslinux --help", $syslinux_help_log);
my $help_out = slurp($syslinux_help_log);
die "syslinux --help returned unexpected rc=$rc_help\n"
if $rc_help != 0 && $rc_help != 1;
die "syslinux --help output missing expected usage text\n"
if $help_out !~ /usage|syslinux/i;
}
my $summary = "$log_dir/smoke-summary.txt";
open my $sfh, '>', $summary or die "Cannot write $summary: $!\n";
print {$sfh} "pxe_file=$pxe_file\n";
print {$sfh} "rc_file=$rc_file\n";
print {$sfh} "rc_qf=$rc_qf\n";
print {$sfh} "main_rpm_available=" . ($main_rpm ? 1 : 0) . "\n";
close $sfh;
}
print_step("Completed");
print "syslinux-xcat RPM: $xcat_rpm\n";
print "Artifacts: $result_dir\n";
@@ -284,7 +239,6 @@ Usage: $0 [options]
--result-dir PATH Output RPM/SRPM directory (default: $result_dir)
--log-dir PATH Log directory (default: $log_dir)
--build-timestamp EPOCH SOURCE_DATE_EPOCH for deterministic builds
--skip-install Skip dnf install + smoke tests
USAGE
}
t/mockbuild-all.t
+31 -1
View File
@@ -12,7 +12,8 @@ use File::Path qw(make_path);
use File::Basename qw(basename);
use MockBuildUtils qw(required_pkgs version_matches rpm_sigmd5 rpm_version rpm_release rpm_is_signed
restamp_release_line cross_copy_genesis finalize_xcat_dep read_manifest
verify_repo_packages verify_repo_signature bump_dep_release_suffix);
verify_repo_packages verify_repo_signature bump_dep_release_suffix
build_mock_uniqueext);
# Run a printing sub with STDOUT muted so its progress lines do not pollute TAP.
sub quiet(&) {
@@ -325,4 +326,33 @@ is(rpm_release(tempdir(CLEANUP => 1), 'nonexistent-pkg'), undef, 'rpm_release is
'verify_repo_signature: mismatch reported as WRONGKEY <unit>: signed by <obs>, expected <exp>');
}
# ---- build_mock_uniqueext: distinct per target so concurrent mock roots never collide ---------
# (PR #62 review) A long (timestamp) run id must not tail-truncate away the leading EL/arch token:
# for the 7-char "ppc64le" arch that dropped the EL digit, so alma+epel-{8,9,10}-ppc64le collapsed to
# one uniqueext -- and goconserver builds all three ELs in the SAME el10 chroot, so the roots raced.
{
my $seq = 6; my $label = 'goconserver';
# The reproducing case: the default timestamp run id (long), folded with the per-target prefix.
my @ppc = map { build_mock_uniqueext("alma+epel-$_-ppc64le-20260821-210716", $seq, $label) } (8, 9, 10);
my %seen; $seen{$_}++ for @ppc;
is(scalar(keys %seen), 3,
'build_mock_uniqueext: el8/el9/el10 ppc64le get DISTINCT uniqueext on a long run id (no collision)');
like($ppc[0], qr/^mba-06-alma-epel-8-/, 'uniqueext keeps a readable leading EL/arch token');
# x86_64 (6-char arch) was never broken -- assert it stays distinct too.
my @x86 = map { build_mock_uniqueext("alma+epel-$_-x86_64-20260821-210716", $seq, $label) } (8, 9, 10);
my %sx; $sx{$_}++ for @x86;
is(scalar(keys %sx), 3, 'build_mock_uniqueext: el8/el9/el10 x86_64 also distinct');
# Short run ids (e.g. the CD "$BUILD_NUMBER") are unchanged and already distinct per target.
isnt(build_mock_uniqueext('alma+epel-8-ppc64le-104', $seq, $label),
build_mock_uniqueext('alma+epel-9-ppc64le-104', $seq, $label),
'build_mock_uniqueext: short (build-number) run ids distinct per target');
# Same run id + same step -> stable (deterministic; a re-run reuses/scrubs the same root).
is(build_mock_uniqueext('alma+epel-8-ppc64le-20260821-210716', $seq, $label),
build_mock_uniqueext('alma+epel-8-ppc64le-20260821-210716', $seq, $label),
'build_mock_uniqueext: deterministic for a given (run, seq, label)');
}
done_testing;
xnba/mockbuild.pl
-4
View File
@@ -19,7 +19,6 @@ my $mock_cfg = '';
my $mock_uniqueext = '';
my $result_dir = "$repo_root/build-output/list3/xnba-undi";
my $log_dir = "$repo_root/build-logs/list3/xnba-undi";
my $skip_install = 0;
my $build_timestamp;
GetOptions(
@@ -28,7 +27,6 @@ GetOptions(
'mock-uniqueext=s' => \$mock_uniqueext,
'result-dir=s' => \$result_dir,
'log-dir=s' => \$log_dir,
'skip-install!' => \$skip_install,
'build-timestamp=i' => \$build_timestamp,
) or die usage();
@@ -73,7 +71,6 @@ print "work_dir: $work_dir\n";
print "result_dir: $result_dir\n";
print "log_dir: $log_dir\n";
print "mock_cfg: $mock_cfg\n";
print "skip_install: $skip_install\n";
make_path($result_dir);
make_path($log_dir);
@@ -176,7 +173,6 @@ Options:
--mock-uniqueext STR Mock uniqueext value
--result-dir PATH Output directory for RPMs
--log-dir PATH Output directory for logs
--skip-install Skip install verification
--build-timestamp EPOCH Unix timestamp for reproducible builds
USAGE
}