xCAT/xcat-dep
2
0
Fork 0
mirror of https://github.com/xcat2/xcat-dep.git synced 2026-09-12 04:26:25 +00:00
Code Issues Projects Releases Wiki Activity

feat(xcat-dep): merge master and share one Genesis pool across every apt suite

Browse Source 
PR #65 changed where the OpenEmbedded Genesis packages live. On the RPM side
they are published once under xcat-dep/common; on the APT side, once under
pool/main/xcat-genesis-openembedded, with every suite's Packages index pointing
at that one copy instead of each suite carrying its own. It made that change in
build-apt-repo.sh -- the script this branch deletes, having absorbed apt assembly
and signing into sbuild-all.pl -- so the layout is ported here rather than lost.

sbuild-all.pl:

- --genesis-release now rebuilds a single shared pool inside the side tree and
  indexes it into every suite, so a release is stored once rather than once per
  codename. The debs are Architecture:all and identical everywhere; the previous
  per-suite copy multiplied hundreds of megabytes by the number of suites.
- Because every suite's index points into that pool, publishing a release must
  cover every suite: a run whose --dists omits one is refused, instead of leaving
  that suite indexing files the new release retired.
- OpenEmbedded Genesis debs are dropped from suite pools unconditionally now,
  not only when a release is being published -- they belong to the shared pool.
- Published files get an explicit mode 0644: they are served by a web server
  running as another user, and inheriting the builder's umask is how that breaks.

master's transaction machinery (per-file backups, --force-unlock recovery of an
interrupted publisher) is deliberately NOT ported: it exists because the shell
publisher writes into the live repository. sbuild-all.pl assembles a complete
side tree, gates it, and renames it into place under one global publish lock, so
a failed or killed run leaves the published repository untouched and there is no
half-written state to recover. The tests assert that guarantee directly.

The APT consumer tests now drive sbuild-all.pl's publish path with master's new
expectations: the shared pool holds one complete release, suite pools hold none
of it, every suite/arch index carries the shared Filename, a later single-suite
rebuild keeps using the pool, a partial-suite release is refused, and a
publication that cannot be signed leaves the packages, indexes and key exactly
as they were.

Full suite green: 416 tests on xcat-master (rome01, EL10) and 414 on
xcat-master-ub, where the APT cases actually run.

Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
This commit is contained in:
Daniel Hilst
2026-08-25 18:10:35 -03:00
parent de98bb24f1 8ca6032927
commit 2a4eb248b4
9 changed files with 977 additions and 185 deletions
Download Patch File Download Diff File Expand all files Collapse all files
BUILD.md
+31 -13
View File
@@ -87,6 +87,8 @@ Use these flags to skip specific operations:
- Adds extra artifact roots to the collection phase (repeatable).
- `--dry-run`
- Prints planned actions without executing them.
- `--force-unlock`
- Removes a stale lock after the previous publisher has been checked.
# Prerequisites
@@ -168,14 +170,21 @@ perl ./mockbuild-all.pl \
--genesis-release /path/to/xcat-genesis-release
```
The release is checked before any package is collected. Its RPMs and SRPMs are
copied into each generated EL repository. Stale OpenEmbedded Genesis packages
are removed from the output first, while the existing per-EL Genesis package
remains available.
The release is checked before any package is collected. Its binary RPMs are
published once under `xcat-dep/common`. Source RPMs stay in the verified
release directory. Existing per-EL repositories keep the old Genesis packages
and contain no OpenEmbedded copies.
The build holds separate locks for its work area and the published repository.
It prepares the complete common repository in a temporary directory, then
replaces the previous repository only after package verification, metadata
generation, and signing have succeeded. If a stopped publisher leaves staging
or backup directories behind, rerun it with ``--force-unlock`` to recover the
previous repository before starting a new publication.
Repository publication requires a release containing every supported Genesis
architecture. The packages are `noarch`, and every management-node repository
receives the full set of target images.
architecture. The packages are `noarch`, and the common repository contains
the full set of target images.
The release checksums cover the unsigned input packages. If repository signing
is enabled, `rpmsign` changes the deployed RPM bytes after collection.
@@ -187,9 +196,7 @@ a separate xcat-core change.
Omit `--genesis-release` to keep using the existing Genesis builder.
The APT side takes the same option, on the run that **publishes** (the release is
copied into every selected suite while the publish lock is held, so what is indexed
and signed is what was verified):
The APT side takes the same option, on the run that **publishes**:
```bash
./sbuild-all.pl --skip-build --skip-genesis \
@@ -198,10 +205,21 @@ and signed is what was verified):
--gpg-sign --gpg-key-id <id> --gpg-home <gpg-home>
```
Anything staged under the OpenEmbedded Genesis package name is dropped when
`--genesis-release` is given: the verified release is the only source of those
packages. `sbuild-all.pl` loads the release reader only when the option is used, so
an apt build without it does not need `perl-File-Slurper` on the build host.
The debs are published **once**, under `pool/main/xcat-genesis-openembedded`, and
every suite's `Packages` index points at that one copy — they are
`Architecture: all` and identical for every codename. Because every suite indexes
it, a release must be published for all of them: a run whose `--dists` omits a
suite is refused rather than leaving that suite indexing retired files. Each
package is copied and re-checked against the release checksums while the publish
lock is held, so what is indexed and signed is what was verified, and anything
staged under the OpenEmbedded Genesis package name is dropped: the release is the
only source of those packages. `sbuild-all.pl` loads the release reader only when
the option is used, so an apt build without it does not need `perl-File-Slurper`
on the build host.
The per-target repository tarballs do not contain `xcat-dep/common`. For an
offline installation, mirror the common repository with its metadata before
disconnecting the installation network.
# Build Unified Repository Without xCAT (`--skip-xcat`)
genesis-openembedded/README.md
+34 -12
View File
@@ -56,8 +56,8 @@ the release, so only accept a directory produced by a trusted build host.
`verify-release` also checks the identity of each package, including a fixed
build host and a build time taken from the source epoch. Those are reproduced
by rpm 4.14.3, 4.16.1.3, 4.19.1.1 and 6.0.2, so an EL8 or later builder -- and
a current Fedora one -- produces a release the verifier accepts.
by rpm 4.14.3, 4.16.1.3, 4.19.1.1, and 6.0.2. An EL8 or later builder, or a
current Fedora builder, produces a release the verifier accepts.
Pass that same directory to the repository builders:
@@ -72,18 +72,40 @@ perl ./sbuild-all.pl \
[other build options]
```
The RPM builder keeps its old per-EL Genesis build and adds the new packages.
The APT builder does the same for each selected suite, during its locked,
atomic publish step. Both consumers require
all seven architectures and verify package identities and checksums before
collecting packages. Every management-node repository receives every target
image so it can provision nodes of another architecture.
The RPM builder publishes the binary packages once under `xcat-dep/common`.
The per-EL repositories keep the old Genesis packages and contain no copies of
the OpenEmbedded packages. Source RPMs stay in the verified release directory.
The publisher locks the shared repository and replaces `common` only after the
new package set, metadata, signatures, and local setup files are ready.
The APT builder publishes the DEBs once under
`pool/main/xcat-genesis-openembedded`. Every suite indexes those same files.
Genesis publication updates all suites together, so a run whose `--dists` omits
one is refused. Later suite rebuilds keep using the shared pool. Pass a new
release only when replacing the Genesis packages.
APT metadata is assembled in a side tree and swapped onto the published
repository with a single rename, under one global publish lock, so a failed or
interrupted publication leaves the previous repository exactly as it was.
Both consumers require all seven architectures and verify package identities
and checksums before publication. A management node can install an image for a
different target architecture.
Without `--genesis-release`, both builders keep their existing behavior. The
new packages do not provide, replace, or obsolete the old package names. A
separate xcat-core change will select the OpenEmbedded package and install
namespace after the repositories carry it. Generated images and packages
belong in release storage, not in Git.
new packages do not provide, replace, or obsolete the old package names.
xcat-core selects the OpenEmbedded install namespace when an image package is
present and falls back to the old Genesis image otherwise. Package installation
or update runs `mknb` for that package's architecture on nodes with local TFTP
storage. With ``site.sharedtftp=0``, update service nodes before the management
node so they can serve the exact architecture name immediately. Removing an
image package retires its published files and rebuilds any legacy fallback that
is still installed. Generated images and packages belong in release storage,
not in Git.
The per-target RPM tarballs do not include `xcat-dep/common`. Mirror that
repository separately, including its metadata, when preparing an offline
installation.
Run the package tests on a Linux builder with RPM, DEB, and repository tools:
genesis-openembedded/activate
Executable
+60
View File
@@ -0,0 +1,60 @@
#!/bin/sh
genesis_is_host_root() {
[ -f /proc/cmdline ] || return 1
host_root=$(stat -c '%i %d' / 2>/dev/null) || return 1
init_root=$(stat -c '%i %d' /proc/1/root/. 2>/dev/null) || return 1
[ "$host_root" = "$init_root" ]
}
genesis_mknb_exists() {
[ -x /opt/xcat/sbin/mknb ]
}
genesis_is_service_node() {
if command -v rpm >/dev/null 2>&1 \
&& rpm -q xCATsn >/dev/null 2>&1; then
return 0
fi
command -v dpkg-query >/dev/null 2>&1 || return 1
dpkg-query -W -f='${Status}' xcatsn 2>/dev/null \
| grep -q '^install ok installed$'
}
genesis_uses_shared_tftp() {
[ -x /opt/xcat/sbin/tabdump ] || return 1
sharedtftp=$(
/opt/xcat/sbin/tabdump site 2>/dev/null \
| awk -F '"' '$2 == "sharedtftp" { print $4; exit }'
)
[ "$sharedtftp" = 1 ]
}
genesis_run_mknb() {
# shellcheck disable=SC1091
[ ! -r /etc/profile.d/xcat.sh ] || . /etc/profile.d/xcat.sh
/opt/xcat/sbin/mknb "$1"
}
genesis_activation_main() {
architecture=$1
case "$architecture" in
x86|x86_64|ppc64|ppc64le|armv7hf|aarch64|riscv64) ;;
*)
echo "Invalid Genesis architecture: $architecture" >&2
return 0
;;
esac
genesis_is_host_root || return 0
genesis_mknb_exists || return 0
if genesis_is_service_node && genesis_uses_shared_tftp; then
return 0
fi
if ! genesis_run_mknb "$architecture"; then
echo "WARNING: mknb $architecture failed; rerun it after xcatd is available." >&2
fi
return 0
}
genesis_activation_main "$@"
genesis-openembedded/package
+34 -1
View File
@@ -91,6 +91,10 @@ for my $entry (_flat_files($export_dir)) {
copy("$export_dir/$entry", "$source_root/image/$entry")
or die "Cannot copy export file $entry: $!\n";
}
copy("$FindBin::Bin/activate", "$source_root/activate")
or die "Cannot copy the Genesis activation helper: $!\n";
chmod(0755, "$source_root/activate")
or die "Cannot set the Genesis activation helper mode: $!\n";
write_binary("$source_root/xcat-core-revision", "$revision\n");
local $ENV{SOURCE_DATE_EPOCH} = $source_date_epoch;
@@ -158,7 +162,8 @@ sub _build_deb {
my $root = "$work/deb-root";
my $image = "$root/opt/xcat/share/xcat/netboot/genesis-openembedded/$architecture";
my $doc = "$root/usr/share/doc/$package_name";
make_path("$root/DEBIAN", $image, $doc, "$staging/deb");
my $helper = "$root/usr/libexec/xcat/genesis-openembedded-activate-$architecture";
make_path("$root/DEBIAN", $image, $doc, dirname($helper), "$staging/deb");
for my $entry (_flat_files("$source_root/image")) {
copy("$source_root/image/$entry", "$image/$entry")
@@ -168,6 +173,9 @@ sub _build_deb {
copy("$source_root/xcat-core-revision", "$doc/xcat-core-revision")
or die "Cannot stage DEB revision: $!\n";
chmod(0644, "$doc/xcat-core-revision") or die "Cannot set revision mode: $!\n";
copy("$source_root/activate", $helper)
or die "Cannot stage DEB activation helper: $!\n";
chmod(0755, $helper) or die "Cannot set activation helper mode: $!\n";
my $installed_kib = _tree_bytes($root);
$installed_kib = int(($installed_kib + 1023) / 1024);
@@ -182,6 +190,31 @@ sub _build_deb {
. " The OpenEmbedded Genesis image used for node discovery, inventory,\n"
. " and service actions.\n",
);
write_binary(
"$root/DEBIAN/postinst",
"#!/bin/sh\n"
. "/usr/libexec/xcat/genesis-openembedded-activate-$architecture $architecture\n"
. "exit 0\n",
);
chmod(0755, "$root/DEBIAN/postinst")
or die "Cannot set post-installation script mode: $!\n";
write_binary(
"$root/DEBIAN/postrm",
"#!/bin/sh\n"
. "case \"\$1\" in\n"
. " remove|purge)\n"
. " host_root=\$(stat -c '%i %d' / 2>/dev/null) || host_root=\n"
. " init_root=\$(stat -c '%i %d' /proc/1/root/. 2>/dev/null) || init_root=\n"
. " if [ -f /proc/cmdline ] && [ -n \"\$host_root\" ] && [ \"\$host_root\" = \"\$init_root\" ] && [ -x /opt/xcat/sbin/mknb ]; then\n"
. " /opt/xcat/sbin/mknb $architecture --remove-openembedded || :\n"
. " /opt/xcat/sbin/mknb $architecture || :\n"
. " fi\n"
. " ;;\n"
. "esac\n"
. "exit 0\n",
);
chmod(0755, "$root/DEBIAN/postrm")
or die "Cannot set post-removal script mode: $!\n";
_write_deb_md5sums($root);
_normalize_mtime($root, $source_date_epoch);
genesis-openembedded/rpm/xCAT-genesis-openembedded.spec
+18
View File
@@ -30,6 +30,22 @@ install -m 0644 image/* "$RPM_BUILD_ROOT/opt/xcat/share/xcat/netboot/genesis-ope
# Keep this path stable across RPM build hosts. %%{_docdir} differs on SUSE.
install -d -m 0755 "$RPM_BUILD_ROOT/usr/share/doc/%{name}"
install -m 0644 xcat-core-revision "$RPM_BUILD_ROOT/usr/share/doc/%{name}/"
install -d -m 0755 "$RPM_BUILD_ROOT/usr/libexec/xcat"
install -m 0755 activate "$RPM_BUILD_ROOT/usr/libexec/xcat/genesis-openembedded-activate-%{genesis_arch}"
%posttrans
/usr/libexec/xcat/genesis-openembedded-activate-%{genesis_arch} %{genesis_arch}
%postun
if [ "$1" -eq 0 ] && [ -f /proc/cmdline ]; then
host_root=$(stat -c '%i %d' / 2>/dev/null) || host_root=
init_root=$(stat -c '%i %d' /proc/1/root/. 2>/dev/null) || init_root=
if [ -n "$host_root" ] && [ "$host_root" = "$init_root" ] \
&& [ -x /opt/xcat/sbin/mknb ]; then
/opt/xcat/sbin/mknb %{genesis_arch} --remove-openembedded || :
/opt/xcat/sbin/mknb %{genesis_arch} || :
fi
fi
%files
%defattr(-,root,root,-)
@@ -41,3 +57,5 @@ install -m 0644 xcat-core-revision "$RPM_BUILD_ROOT/usr/share/doc/%{name}/"
/opt/xcat/share/xcat/netboot/genesis-openembedded/%{genesis_arch}
%dir /usr/share/doc/%{name}
%doc /usr/share/doc/%{name}/xcat-core-revision
%dir /usr/libexec/xcat
/usr/libexec/xcat/genesis-openembedded-activate-%{genesis_arch}
mockbuild-all.pl
+176 -72
View File
@@ -9,7 +9,7 @@ use File::Copy qw(copy);
use File::Find qw(find);
use File::Glob qw(bsd_glob);
use File::Path qw(make_path remove_tree);
use File::Temp qw(tempfile);
use File::Temp qw(tempdir tempfile);
use FindBin;
use Getopt::Long qw(GetOptions);
use Parallel::ForkManager;
@@ -63,9 +63,14 @@ my $repo_dep = '';
my $gpg_sign = 0;
my $gpg_key_name = 'xCAT Signing Key';
my $gpg_home = '';
my $gpg_program = '';
my $force_unlock = 0;
my $HELD_LOCK; # path of the output lock this process owns (for cleanup on exit)
my $LOCK_OWNER_PID; # pid that created the lock; forked children must NOT remove it
my @HELD_LOCKS;
my $LOCK_OWNER_PID;
my ($COMMON_STAGE, $COMMON_DESTINATION, $COMMON_BACKUP);
for my $sig (qw(INT TERM HUP)) {
$SIG{$sig} = sub { exit 1; };
}
GetOptions(
'repo-root=s' => \$repo_root,
@@ -133,10 +138,15 @@ $output_base = abs_path($output_base)
$output_root = "$output_base/mockbuild-all" if $output_root eq '';
# Deployable per-EL xcat-dep repo root (rh8/rh9/rh10/<arch> assembled here).
$repo_dep = "$output_base/xcat-dep" if $repo_dep eq '';
make_path($repo_dep) if !-d $repo_dep;
$repo_dep = abs_path($repo_dep)
or die "Cannot resolve --repo-dep directory\n";
# Fail-fast lock on the output base so a second run against the same --output aborts instead of
# racing on the shared NFS tree. Held for the whole invocation; released by the exit handlers.
acquire_output_lock($output_base, $force_unlock);
acquire_repository_lock($repo_dep, $force_unlock)
if $repo_dep ne $output_base;
$xcat_src = resolve_xcat_source($xcat_src, $repo_root);
@@ -155,7 +165,7 @@ for my $bin (qw(perl uname createrepo_c tar find rpm)) {
}
require_command('mock') if $scrub_all_chroots;
require_command('rpmsign') if $gpg_sign;
require_command('gpg') if $gpg_sign;
$gpg_program = require_command('gpg') if $gpg_sign;
if ($genesis_release ne '') {
$genesis_release = abs_path($genesis_release)
@@ -191,7 +201,8 @@ print_step('Targets to build');
print " $_\n" for @build_targets;
print "output_base: $output_base\n";
print "deploy repo-dep: $repo_dep\n";
print "lock: $output_base/.lock (held)\n";
print "output lock: $output_base/.lock (held)\n";
print "repository lock: $repo_dep/.lock (held)\n";
print "gpg_sign: $gpg_sign\n";
print "gpg_key_name: $gpg_key_name\n" if $gpg_sign;
print "gpg_home: " . ($gpg_home ne '' ? $gpg_home : '(default keyring)') . "\n" if $gpg_sign;
@@ -230,6 +241,8 @@ for my $tgt (@build_targets) {
$tgt_pm->wait_all_children;
die "FATAL: $tgt_fail target(s) failed\n" if $tgt_fail;
publish_genesis_common_repo() if $genesis_release;
print_step('All targets completed');
exit 0;
@@ -519,12 +532,6 @@ if (!$dry_run && $copied == 0) {
die "No binary RPMs were collected. Check build logs and collection roots.\n";
}
if ($genesis_release) {
$copied += $dry_run
? preview_genesis_release_packages('rpm', $repo_dir)
: install_genesis_release_packages('rpm', $repo_dir);
}
# Ensure the OS-dependent xCAT-genesis-base rpm (built by the genesis step above)
# lands in the dep repo even when the full xCAT core is built elsewhere (--skip-xcat).
if (!$skip_genesis && !$dry_run) {
@@ -550,15 +557,6 @@ if (!$dry_run && $copied_srpms == 0) {
print "WARN: No source RPMs were collected. SRPM repo and tarball may be empty.\n";
}
if ($genesis_release) {
$copied_srpms += $dry_run
? preview_genesis_release_packages('srpm', $srpm_repo_dir)
: install_genesis_release_packages('srpm', $srpm_repo_dir);
}
assert_genesis_release_copied($repo_dir, $srpm_repo_dir)
if $genesis_release && !$dry_run;
if (!$skip_createrepo) {
run_step(
step => 'Run createrepo',
@@ -652,11 +650,7 @@ sub deploy_target {
my $destination = "$dest/" . basename($rpm);
publish_file($rpm, $destination);
}
if ($genesis_release) {
my @keep = map { basename($_) } genesis_release_files('rpm');
remove_genesis_packages($dest, 0, \@keep);
verify_genesis_release_packages('rpm', $dest);
}
remove_genesis_packages($dest, 0) if $genesis_release;
assert_required_deps($dest);
sign_and_index_repo($dest);
write_dep_repo_metadata($dest, $rel);
@@ -664,6 +658,48 @@ sub deploy_target {
print "Deployed rh$rel/$arch: $n rpms\n";
}
sub publish_genesis_common_repo {
my $dest = "$repo_dep/common";
print_step("Publish OpenEmbedded Genesis -> $dest");
if ($dry_run) {
preview_genesis_release_packages('rpm', $dest);
return;
}
$COMMON_STAGE = tempdir('.common.XXXXXXXX', DIR => $repo_dep, CLEANUP => 0);
my $published = publish_genesis_release_packages('rpm', $COMMON_STAGE);
verify_genesis_release_packages('rpm', $COMMON_STAGE);
sign_and_index_repo($COMMON_STAGE);
write_common_repo_metadata($COMMON_STAGE);
chmod(0755, $COMMON_STAGE)
or die "Cannot make $COMMON_STAGE traversable: $!\n";
replace_common_repository($COMMON_STAGE, $dest);
print "Published common Genesis repository: $published rpms\n";
}
sub replace_common_repository {
my ($staged, $destination) = @_;
my $backup = "$repo_dep/.common.previous.$$";
remove_tree($backup) if -e $backup || -l $backup;
$COMMON_DESTINATION = $destination;
if (-e $destination || -l $destination) {
$COMMON_BACKUP = $backup;
rename($destination, $backup)
or die "Cannot preserve $destination before publication: $!\n";
}
unless (rename($staged, $destination)) {
my $error = $!;
rename($backup, $destination) if $COMMON_BACKUP && -d $backup;
die "Cannot publish $destination: $error\n";
}
undef($COMMON_STAGE);
undef($COMMON_DESTINATION);
remove_tree($backup) if $COMMON_BACKUP && -d $backup;
undef($COMMON_BACKUP);
}
sub publish_file {
my ($source, $destination) = @_;
my ($temporary_fh, $temporary) = tempfile(
@@ -706,7 +742,8 @@ sub sign_and_index_repo {
my @rpms = grep { !/\.src\.rpm$/ } bsd_glob("$dir/*.rpm");
if ($gpg_sign && @rpms) {
local $ENV{GNUPGHOME} = $gpg_home if $gpg_home;
run_simple(qq(rpmsign --define "%_gpg_name $gpg_key_name" --addsign )
run_simple('rpmsign --define ' . shell_quote("%_gpg_name $gpg_key_name")
. ' --define ' . shell_quote("%__gpg $gpg_program") . ' --addsign '
. join(' ', map { shell_quote($_) } @rpms));
}
run_simple(createrepo_c_cmd($dir));
@@ -735,25 +772,62 @@ $gpgkey_line
EOF
close $r;
write_local_repo_helper($dir);
write_buildinfo($dir, "rh$rel/$arch");
}
sub write_common_repo_metadata {
my ($dir) = @_;
my $baseurl = "https://xcat.org/files/xcat/repos/yum/devel/xcat-dep/common";
my $gpgcheck = $gpg_sign ? 1 : 0;
my $gpgkey_line = $gpg_sign ? "gpgkey=$baseurl/repodata/repomd.xml.key" : "# gpgkey=";
open my $r, '>', "$dir/xcat-dep-common.repo"
or die "Cannot write $dir/xcat-dep-common.repo: $!\n";
print {$r} <<"EOF";
[xcat-dep-common]
name=xCAT 2 common dependencies
baseurl=$baseurl
enabled=1
gpgcheck=$gpgcheck
repo_gpgcheck=$gpgcheck
skip_if_unavailable=1
$gpgkey_line
EOF
close $r;
write_local_repo_helper($dir);
write_buildinfo($dir, 'common');
}
sub write_local_repo_helper {
my ($dir) = @_;
open my $m, '>', "$dir/mklocalrepo.sh" or die "Cannot write $dir/mklocalrepo.sh: $!\n";
print {$m} <<'EOS';
#!/bin/sh
cd `dirname $0`
REPOFILE=`basename xcat-*.repo`
if [[ $REPOFILE == "xcat-*.repo" ]]; then
echo "ERROR: For xcat-dep, please execute $0 in the correct <os>/<arch> subdirectory"
SCRIPT_DIRECTORY=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
cd "$SCRIPT_DIRECTORY" || exit 1
set -- xcat-*.repo
if [ "$#" -ne 1 ] || [ "$1" = "xcat-*.repo" ]; then
echo "ERROR: Execute $0 in an xcat-dep repository directory"
exit 1
fi
REPOFILE=$1
DIRECTORY="/etc/yum.repos.d"
if [ ! -d "$DIRECTORY" ]; then
DIRECTORY="/etc/zypp/repos.d"
fi
sed -e 's|baseurl=.*|baseurl=file://'"`pwd`"'|' $REPOFILE | sed -e 's|gpgkey=.*|gpgkey=file://'"`pwd`"'/repodata/repomd.xml.key|' > "$DIRECTORY/$REPOFILE"
cd -
CURRENT_DIRECTORY=$(pwd)
sed -e 's|baseurl=.*|baseurl=file://'"$CURRENT_DIRECTORY"'|' "$REPOFILE" \
| sed -e 's|gpgkey=.*|gpgkey=file://'"$CURRENT_DIRECTORY"'/repodata/repomd.xml.key|' \
> "$DIRECTORY/$REPOFILE"
EOS
close $m;
chmod 0775, "$dir/mklocalrepo.sh";
}
sub write_buildinfo {
my ($dir, $target) = @_;
my $build_time = strftime("%a %b %e %H:%M:%S %Z %Y", gmtime($SOURCE_DATE_EPOCH));
my $build_machine = `hostname`; chomp $build_machine;
my $commit = `git -C "$repo_root" rev-parse HEAD 2>/dev/null`; chomp $commit;
@@ -762,7 +836,7 @@ EOS
my $release = strftime('snap%Y%m%d%H%M', gmtime($SOURCE_DATE_EPOCH));
open my $b, '>', "$dir/buildinfo.txt" or die "Cannot write $dir/buildinfo.txt: $!\n";
print {$b} <<"EOF";
TARGET=rh$rel/$arch
TARGET=$target
RELEASE=$release
BUILD_TIME=$build_time
BUILD_MACHINE=$build_machine
@@ -786,10 +860,10 @@ Options:
it. A fail-fast lock is held at <PATH>/.lock, so pass distinct paths
to run on two hosts in parallel on one NFS (default: <repo-root>/build-output)
--output-root PATH Override the derived build tree root (default: <output>/mockbuild-all)
--repo-dep PATH Override the derived deployable per-EL output root; rh8/rh9/rh10/<arch>
are assembled + signed here (default: <output>/xcat-dep)
--repo-dep PATH Override the deployable output root; rh8/rh9/rh10/<arch> and common
are assembled and signed here (default: <output>/xcat-dep)
--force-unlock Remove a stale <output>/.lock before acquiring it
--gpg-sign Sign rpms + repomd.xml of each per-EL repo
--gpg-sign Sign RPMs and repomd.xml in every published repository
--gpg-key-name NAME GPG key name (default: "xCAT Signing Key")
--gpg-home PATH GNUPGHOME for signing (default: system keyring)
--target NAME Build only this target (<ID>+epel-<REL>-<ARCH>); default is
@@ -811,8 +885,7 @@ Options:
--skip-genesis Skip the existing per-EL Genesis image build
--skip-createrepo Skip createrepo
--skip-tarball Skip binary/SRPM tarball creation
--genesis-release PATH Add a verified OpenEmbedded Genesis RPM release alongside the
existing per-EL Genesis packages
--genesis-release PATH Publish a verified OpenEmbedded Genesis RPM release in common
--scrub-all-chroots Run mock -r <target> --scrub=all before build/collect
--collect-dir PATH Additional directory to scan recursively for RPMs (repeatable)
--dry-run Print planned commands without executing
@@ -1019,16 +1092,16 @@ sub remove_genesis_packages {
}
}
sub install_genesis_release_packages {
sub publish_genesis_release_packages {
my ($prefix, $destination_root) = @_;
remove_genesis_packages($destination_root, $prefix eq 'srpm');
remove_genesis_packages($destination_root, 1) if $prefix eq 'rpm';
my $copied = 0;
for my $relative (genesis_release_files($prefix)) {
my $source = "$genesis_release/$relative";
my $destination = "$destination_root/" . basename($relative);
copy($source, $destination)
or die "Cannot install Genesis release package $source: $!\n";
publish_file($source, $destination);
verify_release_file($genesis_release_checksums, $relative, $destination);
$copied++;
}
@@ -1036,16 +1109,13 @@ sub install_genesis_release_packages {
return $copied;
}
# Dry runs copy nothing, but they must still report what a real run would publish:
# collect_rpms and collect_srpms drop every xCAT-genesis-openembedded package whenever
# --genesis-release is given, so without this preview a dry run describes a repository
# with no Genesis packages at all while the real run installs the whole set.
# Dry runs copy nothing, but they must still report the shared packages a real run publishes.
sub preview_genesis_release_packages {
my ($prefix, $destination_root) = @_;
my @files = genesis_release_files($prefix);
die "Genesis release has no $prefix packages\n" unless @files;
for my $relative (@files) {
print "DRY-RUN install Genesis release package: $genesis_release/$relative"
print "DRY-RUN publish Genesis release package: $genesis_release/$relative"
. " -> $destination_root/" . basename($relative) . "\n";
}
return scalar(@files);
@@ -1070,17 +1140,6 @@ sub verify_genesis_release_packages {
return scalar(@files);
}
sub assert_genesis_release_copied {
my ($binary_directory, $source_directory) = @_;
for my $entry (
[ 'rpm', $binary_directory ],
[ 'srpm', $source_directory ],
) {
my ($prefix, $destination_root) = @{$entry};
verify_genesis_release_packages($prefix, $destination_root);
}
}
sub collect_rpms {
my (%args) = @_;
my $roots = $args{roots} // [];
@@ -1249,16 +1308,16 @@ sub resolve_xcat_source {
# Fail-fast advisory lock on the output base. Uses an atomic mkdir (portable and reliable over
# NFS, unlike flock) of "<base>/.lock". A second run against the same --output dies immediately
# rather than racing on the shared tree. Only the process that created the lock removes it.
sub acquire_output_lock {
my ($base, $force) = @_;
sub acquire_named_lock {
my ($base, $label, $force) = @_;
my $lock = "$base/.lock";
if ($force && -d $lock) {
print "force-unlock: removing stale lock $lock\n";
_rmdir_lock($lock);
}
if (mkdir $lock) {
$HELD_LOCK = $lock;
$LOCK_OWNER_PID = $$;
push(@HELD_LOCKS, $lock);
$LOCK_OWNER_PID //= $$;
my $host = capture_command('uname', '-n') || 'unknown';
if (open my $fh, '>', "$lock/owner") {
print {$fh} "host=$host\npid=$$\nepoch=" . time() . "\n";
@@ -1271,28 +1330,73 @@ sub acquire_output_lock {
my $info = '';
if (open my $fh, '<', "$lock/owner") { local $/; $info = <$fh>; close $fh; }
$info =~ s/\s+/ /g;
die "output $base is locked ($lock): $info\n"
. "another mockbuild-all run owns it; use a different --output or --force-unlock if stale.\n";
die "$label $base is locked ($lock): $info\n"
. "another mockbuild-all run owns it; use a different destination or --force-unlock if stale.\n";
}
die "Cannot create lock $lock: $!\n";
}
sub acquire_output_lock {
my ($base, $force) = @_;
acquire_named_lock($base, 'output', $force);
}
sub acquire_repository_lock {
my ($base, $force) = @_;
_recover_common_repository($base) if $force;
acquire_named_lock($base, 'repository', $force);
}
sub _recover_common_repository {
my ($base) = @_;
my $destination = "$base/common";
my @backups = sort {
((stat($a))[9] // 0) <=> ((stat($b))[9] // 0)
} grep { -d $_ && !-l $_ } bsd_glob("$base/.common.previous.*");
if (!-e $destination && !-l $destination && @backups) {
my $backup = pop(@backups);
rename($backup, $destination)
or die "Cannot restore interrupted common repository $backup: $!\n";
}
remove_tree($_) for grep { -d $_ && !-l $_ } @backups;
for my $staging (bsd_glob("$base/.common.*")) {
next if $staging =~ m{/\.common\.previous\.};
remove_tree($staging) if -d $staging && !-l $staging;
}
}
sub _rmdir_lock {
my ($lock) = @_;
unlink "$lock/owner";
rmdir $lock;
}
# Release the lock on any exit path (normal, die, or signal) -- but ONLY in the process that
# created it. Forked children (per-builder and per-target ForkManager workers) inherit
# $HELD_LOCK; without the pid guard their exit would delete the parent's lock mid-run.
sub _release_lock_if_owner {
return unless $HELD_LOCK && defined $LOCK_OWNER_PID && $$ == $LOCK_OWNER_PID;
_rmdir_lock($HELD_LOCK) if -d $HELD_LOCK;
# Release locks on every exit path, but only from the process that acquired them.
# Forked build workers inherit the lock list and must leave the parent's locks alone.
sub _restore_common_repository {
return unless defined($LOCK_OWNER_PID) && $$ == $LOCK_OWNER_PID;
remove_tree($COMMON_STAGE)
if $COMMON_STAGE && -d $COMMON_STAGE && !-l $COMMON_STAGE;
if ($COMMON_DESTINATION && $COMMON_BACKUP
&& !-e $COMMON_DESTINATION && -d $COMMON_BACKUP) {
rename($COMMON_BACKUP, $COMMON_DESTINATION);
} elsif ($COMMON_DESTINATION && $COMMON_BACKUP
&& -d $COMMON_DESTINATION && -d $COMMON_BACKUP) {
remove_tree($COMMON_BACKUP);
}
}
END { _release_lock_if_owner(); }
for my $sig (qw(INT TERM HUP)) {
$SIG{$sig} = sub { _release_lock_if_owner(); exit 1; };
sub _release_locks_if_owner {
return unless defined($LOCK_OWNER_PID) && $$ == $LOCK_OWNER_PID;
for my $lock (reverse(@HELD_LOCKS)) {
_rmdir_lock($lock) if -d $lock;
}
}
END {
_restore_common_repository();
_release_locks_if_owner();
}
sub read_os_release {
sbuild-all.pl
+45 -19
View File
@@ -98,6 +98,10 @@ my $gpg_key_id = 'xcat@megware.com';
my $gpg_home = '';
my $genesis_release = ''; # OpenEmbedded Genesis package release to publish alongside
my $genesis_release_checksums; # its verified SHA256SUMS, read once at startup
# The OpenEmbedded Genesis debs are published ONCE, in a pool of their own that every suite indexes.
# They are Architecture:all and identical for all suites, so a per-suite copy would multiply hundreds
# of megabytes by the number of codenames for no gain.
my $GENESIS_POOL_RELATIVE = 'pool/main/xcat-genesis-openembedded';
my @genesis_debs; # native xcat-genesis-base-<arch> deb(s): path or URL (preferred)
my $genesis_rpm = ''; # fallback: native-arch genesis rpm to convert
my $genesis_rpm_ppc = ''; # fallback: cross-arch ppc genesis rpm to convert (amd64 host)
@@ -290,6 +294,12 @@ if ($genesis_release ne '') {
die "FATAL: Genesis release changed during verification\n"
unless XCAT::BuildUtils::hashes_equal($before, $after);
$genesis_release_checksums = $before;
# Every suite's Packages index points into the shared Genesis pool, and publishing a release
# replaces that pool -- so a run that rebuilt only some suites would leave the others indexing
# files that no longer exist. Publish a release for all of them or for none.
my @missing = grep { my $c = $_; !grep { $_ eq $c } @dist_list } known_codenames();
die "FATAL: --genesis-release updates the shared Genesis pool every suite indexes, so it must\n"
. " cover all of them; --dists omits: @missing\n" if @missing;
print_step('Genesis release');
print " $genesis_release (" . scalar(genesis_release_debs()) . " deb packages)\n";
}
@@ -980,31 +990,38 @@ sub genesis_release_debs {
keys %{ $genesis_release_checksums // {} };
}
# install_genesis_release_debs($pool, $flat): copy every release deb into this codename's pool (and
# the flat per-version dir), verifying each copy against the release checksums. Called with the
# publish lock held, between the pool wipe and apt-ftparchive, so the bytes indexed and signed are
# exactly the bytes verified here.
# install_genesis_release_debs($dir): (re)build the SHARED Genesis pool inside the side tree from
# the verified release, verifying each copy against the release checksums. Called once per publish,
# with the publish lock held and before any apt-ftparchive run, so the bytes indexed and signed are
# exactly the bytes verified here. Wiping first is what retires packages an earlier release left.
#
# A plain copy, never link(): the pool file must be its own inode. A hard link would leave the
# published package and the verified release sharing one, where a write through either path silently
# changes what the other holds.
sub install_genesis_release_debs {
my ($pool, $flat) = @_;
my ($dir) = @_;
my @files = genesis_release_debs();
die "FATAL: Genesis release has no deb packages\n" unless @files;
my $pool = "$dir/$GENESIS_POOL_RELATIVE";
wipe_tree($pool);
make_path($pool);
for my $relative (@files) {
my $base = basename($relative);
copy("$genesis_release/$relative", "$pool/$base")
or die "FATAL: cannot install Genesis release package $relative -> $pool: $!\n";
chmod(0644, "$pool/$base")
or die "FATAL: cannot set mode on $pool/$base: $!\n";
XCAT::GenesisRelease::verify_release_file($genesis_release_checksums, $relative, "$pool/$base");
copy("$pool/$base", "$flat/$base")
or die "FATAL: cannot install Genesis release package $relative -> $flat: $!\n";
}
return scalar(@files);
}
sub assemble_into {
my ($dir, $expect) = @_;
if ($genesis_release ne '') {
my $n = install_genesis_release_debs($dir);
print " published + verified $n Genesis release package(s) into $GENESIS_POOL_RELATIVE\n";
}
for my $cn (@dist_list) {
my $ver = codename_to_version($cn);
my $pool = "$dir/pool/main/$cn";
@@ -1020,10 +1037,10 @@ sub assemble_into {
# holds regardless of the --skip-genesis single-producer contract.
my %best; # "name|arch" => { file => path, ver => version }
for my $deb (glob("$staging/$cn/*/*.deb")) {
# With --genesis-release the release is the ONLY source of OpenEmbedded Genesis packages:
# drop anything staged under that name so a leftover from an earlier run cannot be
# published as if it had been verified.
next if $genesis_release ne '' && basename($deb) =~ /^xcat-genesis-openembedded-/;
# OpenEmbedded Genesis debs never belong to a suite pool -- they are published once
# into the shared pool. Drop anything staged under that name, so a leftover from an
# earlier run cannot be published as if it had come from a verified release.
next if basename($deb) =~ /^xcat-genesis-openembedded-/;
my $name = deb_field($deb, 'Package');
my $darch = deb_field($deb, 'Architecture');
my $dver = deb_field($deb, 'Version');
@@ -1043,16 +1060,18 @@ sub assemble_into {
my $b = basename($deb);
link($deb, "$pool/$b") or copy($deb, "$pool/$b");
copy($deb, "$dir/$ver/$b");
}
if ($genesis_release ne '') {
my $n = install_genesis_release_debs($pool, "$dir/$ver");
print " installed + verified $n Genesis release package(s) into $cn\n";
# Served by a web server running as somebody else -- do not inherit the builder umask.
chmod(0644, "$pool/$b", "$dir/$ver/$b");
}
# Packages index per EXPECTED binary-<arch>: an arch's index carries that arch's debs + all
# Architecture:all. Only the expected arches get an index -- writing a binary-ppc64el index
# containing nothing but the Architecture:all debs would advertise a ppc64el repo that cannot
# actually satisfy a ppc64el client (and would then be mistaken for "ppc was published").
# This suite's own pool PLUS the shared Genesis pool: one Packages index, two sources, each
# deb's Filename staying relative to the repository root so clients fetch it where it is.
my $all = `cd ${\ sh_quote($dir)} && apt-ftparchive packages pool/main/$cn`;
$all .= `cd ${\ sh_quote($dir)} && apt-ftparchive packages $GENESIS_POOL_RELATIVE`
if -d "$dir/$GENESIS_POOL_RELATIVE";
for my $a (@$expect) {
my $bindir = "$dir/dists/$cn/main/binary-$a";
make_path($bindir);
@@ -1096,6 +1115,7 @@ sub assemble_into {
my $keysrc = "$repo_root/repomd.xml.key";
if (-f $keysrc) { copy($keysrc, "$dir/xcat-dep.asc"); }
else { run("${g}gpg --armor --export " . sh_quote($gpg_key_id) . " > " . sh_quote("$dir/xcat-dep.asc"), nofail => 1); }
chmod(0644, "$dir/xcat-dep.asc") if -f "$dir/xcat-dep.asc";
}
}
@@ -1402,10 +1422,16 @@ and copies the verified bytes into every selected suite.
The release must be B<complete> (every supported Genesis architecture) and must carry C<deb>
packages. It is validated before any build or publish: its C<SHA256SUMS> is read, the shared
verifier runs, and the checksums are read again -- a release rewritten together with its checksums
while the verifier runs is rejected. During publication each package is copied into the codename's
pool and re-checked against those checksums, with the publish lock held, so what is indexed and
signed is exactly what was verified. Anything staged under the OpenEmbedded Genesis package name is
dropped: the release is the only source of those packages.
while the verifier runs is rejected.
The packages are published B<once>, into F<pool/main/xcat-genesis-openembedded>, and every suite's
C<Packages> index points at that one copy: they are C<Architecture: all> and identical everywhere,
so a per-suite copy would multiply hundreds of megabytes by the number of codenames. Because every
suite indexes it, a release must be published for B<all> of them -- a run whose C<--dists> omits a
suite is refused rather than leaving that suite indexing files the new release retired. Each package
is copied and re-checked against the release checksums with the publish lock held, so what is
indexed and signed is exactly what was verified, and anything staged under the OpenEmbedded Genesis
package name is dropped: the release is the only source of those packages.
The OpenEmbedded packages carry their own names and install under
F</opt/xcat/share/xcat/netboot/genesis-openembedded/>, so publishing them does not replace the
t/genesis_openembedded_consumer.t
+557 -65
View File
@@ -8,7 +8,9 @@ use File::Path qw(make_path);
use Fcntl qw(:flock);
use File::Temp qw(tempdir);
use FindBin;
use POSIX ();
use Test::More;
use Time::HiRes qw(sleep);
use lib "$FindBin::Bin/../lib";
use lib "$FindBin::Bin/lib";
@@ -41,6 +43,10 @@ my $version = '2.19.0';
my $release = 'snap202608210726';
my $epoch = 1787293573;
my $tmp = tempdir(CLEANUP => 1);
# every suite sbuild-all.pl knows; publishing a Genesis release must cover all of them
my @APT_SUITES = qw(focal jammy noble resolute);
test_activation_helper();
if ($ENV{XCAT_GENESIS_CI}) {
BAIL_OUT('CI requires Linux root') unless $^O eq 'linux' && $> == 0;
@@ -50,21 +56,25 @@ if ($ENV{XCAT_GENESIS_CI}) {
}
SKIP: {
skip 'RPM repository tools require a root Linux builder', 28
skip 'RPM repository tools require a root Linux builder', 69
unless $^O eq 'linux'
&& $> == 0
&& command_exists('rpmbuild')
&& command_exists('rpmsign')
&& command_exists('rpm')
&& command_exists('createrepo_c');
test_rpm_consumer();
test_signed_common_rpm_repository();
test_legacy_rpm_consumer();
test_partial_rpm_release();
test_failed_build_release();
test_dry_run_release();
test_rpm_repository_lock();
test_rpm_signal_cleanup();
}
SKIP: {
skip 'APT repository tools are not installed', 24
skip 'APT repository tools are not installed', 58
unless $^O eq 'linux'
&& command_exists('dpkg-deb')
&& command_exists('apt-ftparchive');
@@ -79,17 +89,39 @@ done_testing();
sub test_rpm_consumer {
my $release_root = make_package_release("$tmp/rpm", 'rpm');
my $package = "xCAT-genesis-openembedded-x86_64-$version-$release.noarch.rpm";
my $source_package = "xCAT-genesis-openembedded-x86_64-$version-$release.src.rpm";
my $output = "$tmp/rpm output";
my $target = 'test+epel-10-' . capture_command('uname', '-m');
my $run = "$target-consumer";
my $run_repo = "$output/mockbuild-all/$run/repo/" . capture_command('uname', '-m');
my $source_repo = "$output/mockbuild-all/$run/repo-src";
my $deploy_repo = "$output/xcat-dep/rh10/" . capture_command('uname', '-m');
make_path($run_repo, $source_repo, $deploy_repo);
my $common_repo = "$output/xcat-dep/common";
make_path($run_repo, $source_repo, $deploy_repo, $common_repo);
write_binary("$run_repo/xCAT-genesis-openembedded-stale.noarch.rpm", 'stale');
write_binary("$source_repo/xCAT-genesis-openembedded-stale.src.rpm", 'stale');
write_binary("$deploy_repo/xCAT-genesis-openembedded-stale.noarch.rpm", 'stale');
write_binary("$common_repo/xCAT-genesis-openembedded-stale.noarch.rpm", 'stale');
write_binary("$common_repo/xCAT-genesis-openembedded-stale.src.rpm", 'stale');
my $rpm_scripts = capture_command(
'rpm', '-qp', '--scripts', "$release_root/rpm/$package"
);
like($rpm_scripts, qr{genesis-openembedded-activate-x86_64 x86_64},
'the RPM refreshes its architecture after a package transaction');
like($rpm_scripts, qr{mknb x86_64 --remove-openembedded},
'the RPM removes published artifacts when the image is erased');
like($rpm_scripts, qr{/proc/1/root},
'the RPM removal path refuses to operate from a chroot');
like(
capture_command('rpm', '-qpl', "$release_root/rpm/$package"),
qr{/usr/libexec/xcat/genesis-openembedded-activate-x86_64$}m,
'the RPM contains its architecture-specific activation helper',
);
like(
capture_command('rpm', '-qpl', "$release_root/rpm/$package"),
qr{^/usr/libexec/xcat/?$}m,
'the RPM owns its private helper directory',
);
my $dependencies = "$tmp/rpm-dependencies";
my $scratch_repo_root = "$tmp/rpm-repo-root";
@@ -126,14 +158,19 @@ sub test_rpm_consumer {
);
is($status, 0, 'RPM repository accepts a verified Genesis release');
is(digest_file("$deploy_repo/$package"), digest_file("$release_root/rpm/$package"),
is(
sprintf('%04o', (stat($common_repo))[2] & 0x0fff),
'0755',
'the common repository is traversable by an unprivileged server',
);
is(digest_file("$common_repo/$package"), digest_file("$release_root/rpm/$package"),
'deployed RPM matches the release');
is(
sprintf('%04o', (stat("$deploy_repo/$package"))[2] & 0x0fff),
sprintf('%04o', (stat("$common_repo/$package"))[2] & 0x0fff),
sprintf('%04o', (stat("$release_root/rpm/$package"))[2] & 0x0fff),
'deployed RPM keeps the release file mode',
);
opendir(my $deploy_dh, $deploy_repo) or die $!;
opendir(my $deploy_dh, $common_repo) or die $!;
my @staging_files = grep { /^\.xcat-deploy\./ } readdir($deploy_dh);
closedir($deploy_dh) or die $!;
is_deeply(\@staging_files, [], 'RPM deployment leaves no staging files');
@@ -143,30 +180,108 @@ sub test_rpm_consumer {
'stale source RPM is removed');
ok(!-e "$deploy_repo/xCAT-genesis-openembedded-stale.noarch.rpm",
'stale deployed RPM is removed');
is(digest_file("$source_repo/$source_package"),
digest_file("$release_root/srpm/$source_package"),
'source RPM matches the release');
ok(-f "$deploy_repo/repodata/repomd.xml", 'RPM repository metadata is generated');
ok(-f "$common_repo/repodata/repomd.xml", 'RPM repository metadata is generated');
ok(-f "$deploy_repo/xCAT-genesis-base-x86_64-1.noarch.rpm",
'legacy Genesis package remains available');
ok(!-e "$run_repo/xCAT-genesis-openembedded-x86_64-$version-old.noarch.rpm",
'stale OpenEmbedded RPM is not collected');
ok(!-e "$source_repo/xCAT-genesis-openembedded-x86_64-$version-old.src.rpm",
'stale OpenEmbedded SRPM is not collected');
like(read_binary("$output/mockbuild-all/$run/summary.txt"), qr/^copied_rpms=15$/m,
'release RPM is counted alongside required dependencies');
my @expected_packages = sort map {
rpm_package_name($_) . "-$version-$release.noarch.rpm"
} architectures();
my @common_packages = genesis_rpm_names($common_repo);
is_deeply(\@common_packages, \@expected_packages,
'common repository contains one complete Genesis release');
is_deeply(
[ genesis_rpm_names($deploy_repo) ],
[],
'per-EL repository contains no OpenEmbedded Genesis packages',
);
is_deeply(
[ genesis_rpm_names($run_repo) ],
[],
'target staging repository contains no OpenEmbedded Genesis packages',
);
is_deeply(
[ genesis_rpm_names($source_repo) ],
[],
'target source repository contains no OpenEmbedded Genesis packages',
);
my $common_config = read_binary("$common_repo/xcat-dep-common.repo");
like($common_config, qr/^\[xcat-dep-common\]$/m,
'common repository has its own repository ID');
like($common_config, qr{/xcat-dep/common$}m,
'common repository configuration uses the shared URL');
like($common_config, qr/^skip_if_unavailable=1$/m,
'the published common repository stays optional during outages');
like($common_config, qr/^repo_gpgcheck=0$/m,
'unsigned test metadata is declared explicitly');
ok(-x "$common_repo/mklocalrepo.sh", 'common repository supports offline setup');
my $local_repo_helper = read_binary("$common_repo/mklocalrepo.sh");
unlike($local_repo_helper, qr/\[\[/,
'the offline helper does not require Bash conditionals');
unlike($local_repo_helper, qr/`/,
'the offline helper uses POSIX command substitution');
like(read_binary("$common_repo/buildinfo.txt"), qr/^TARGET=common$/m,
'common repository records its target');
like(read_binary("$output/mockbuild-all/$run/summary.txt"), qr/^copied_rpms=8$/m,
'repository summary counts the collected dependencies');
my $retained = "$common_repo/xCAT-genesis-openembedded-retained.noarch.rpm";
my $repomd = "$common_repo/repodata/repomd.xml";
my $repomd_before = digest_file($repomd);
write_binary($retained, 'previous complete release');
my $fail_bin = "$tmp/rpm-fail-bin";
make_path($fail_bin);
write_binary(
"$fail_bin/createrepo_c",
<<'SH',
#!/bin/sh
for argument in "$@"; do last=$argument; done
case "$last" in
*/common|*/.common.*) exit 1 ;;
esac
exec "$XCAT_TEST_CREATEREPO" "$@"
SH
);
chmod(0755, "$fail_bin/createrepo_c") or die $!;
local $ENV{XCAT_TEST_CREATEREPO} = capture_command(
'sh', '-c', 'command -v createrepo_c'
);
local $ENV{PATH} = "$fail_bin:$ENV{PATH}";
my $failed_log = "$tmp/rpm-publication-failure.log";
my $failed_status = run_capture(
$failed_log,
$^X, $rpm_consumer,
'--repo-root', $scratch_repo_root,
'--output', $output,
'--target', $target,
'--run-id', 'publication-failure',
'--build-timestamp', $epoch,
'--skip-build', '--skip-xcat', '--skip-xcat-dep', '--skip-perl',
'--skip-createrepo', '--skip-tarball',
'--genesis-release', $release_root,
'--collect-dir', $dependencies,
);
isnt($failed_status, 0, 'a failed RPM metadata build aborts publication');
ok(-f $retained, 'a failed RPM publication keeps the previous package set');
is(digest_file($repomd), $repomd_before,
'a failed RPM publication keeps the previous metadata');
}
# The apt consumer is sbuild-all.pl's PUBLISH phase: it stages nothing here, it assembles the
# already-staged debs plus the Genesis release into a side tree, gates it, and swaps it in. Every run
# below is therefore a publish-only run (--skip-build) over a hand-made staging tree, with the
# manifest reduced to the one package --skip-genesis drops so the completeness check has nothing to
# demand of a fixture.
# The apt consumer is sbuild-all.pl's PUBLISH phase: it stages nothing here -- it assembles the
# already-staged debs plus the Genesis release into a side tree, gates it, and swaps that tree onto
# the published repository with a single rename. Every run below is therefore a publish-only run
# (--skip-build) over a hand-made staging tree, with the manifest reduced to the one package
# --skip-genesis drops, so the completeness gate has nothing to demand of a fixture.
sub run_apt_consumer {
my (%args) = @_;
my $manifest = "$args{output}/manifest.conf";
make_path($args{output});
write_binary($manifest, "[noble-amd64]\nxcat-genesis-base=*\n");
my @dists = @{ $args{dists} // \@APT_SUITES };
my $manifest = $args{manifest};
unless ($manifest) {
$manifest = "$args{output}/manifest.conf";
make_path($args{output});
write_binary($manifest,
join('', map { "[$_-amd64]\nxcat-genesis-base=*\n" } @APT_SUITES));
}
return run_capture(
$args{log},
$^X, $deb_consumer,
@@ -174,15 +289,17 @@ sub run_apt_consumer {
'--output-root', $args{output},
'--apt-dir', $args{apt_dir},
'--manifest', $manifest,
'--dists', 'noble',
'--dists', join(' ', @dists),
'--arch', 'amd64',
'--skip-build', '--skip-genesis', '--skip-tarball',
'--publish', '--expect-arch', 'amd64 ppc64el',
'--no-verify-repo',
($args{verify} ? () : ('--no-verify-repo')),
@{ $args{extra} // [] },
);
}
# Stage one legacy genesis-base deb for the noble suite only -- enough for the runs that publish
# no Genesis release.
sub stage_legacy_deb {
my ($root, $output) = @_;
my $staging = "$output/staging/noble/amd64";
@@ -191,15 +308,43 @@ sub stage_legacy_deb {
return $staging;
}
# Stage one legacy genesis-base deb for each suite, standing in for a completed build.
sub stage_apt_suites {
my ($output, $package_root, $content) = @_;
my $package = "$package_root.deb";
make_legacy_deb($package_root, $package, $content);
for my $codename (@APT_SUITES) {
my $staged = "$output/staging/$codename/amd64";
make_path($staged);
copy($package, "$staged/xcat-genesis-base-amd64_1_all.deb") or die $!;
}
return "$output/staging/noble/amd64";
}
sub test_deb_consumer {
my $release_root = make_package_release("$tmp/deb", 'deb');
my $package = "xcat-genesis-openembedded-x86-64_${version}-${release}_all.deb";
my $apt_root = "$tmp/apt";
my $output = "$tmp/deb-output";
my $staging = stage_legacy_deb("$tmp/dummy-deb", $output);
write_binary("$staging/xcat-genesis-openembedded-stale.deb", 'stale');
make_path("$apt_root/pool/main/noble");
write_binary("$apt_root/pool/main/noble/xcat-genesis-openembedded-old.deb", 'stale');
my $staged = stage_apt_suites($output, "$tmp/dummy-deb");
my $shared_pool = "$apt_root/pool/main/xcat-genesis-openembedded";
make_path($shared_pool);
write_binary("$staged/xcat-genesis-openembedded-stale.deb", 'stale');
write_binary("$shared_pool/xcat-genesis-openembedded-old.deb", 'stale');
my $postinst = capture_command(
'dpkg-deb', '--info', "$release_root/deb/$package", 'postinst'
);
isnt($postinst, '', 'the DEB includes a post-installation script');
like($postinst, qr{genesis-openembedded-activate-x86_64 x86_64},
'the DEB refreshes its architecture after configuration');
my $postrm = capture_command(
'dpkg-deb', '--info', "$release_root/deb/$package", 'postrm'
);
like($postrm, qr{mknb x86_64 --remove-openembedded},
'the DEB removes published artifacts when the image is erased');
like($postrm, qr{/proc/1/root},
'the DEB removal path refuses to operate from a chroot');
local $ENV{SOURCE_DATE_EPOCH} = $epoch;
my $log = "$tmp/deb-consumer.log";
@@ -207,52 +352,215 @@ sub test_deb_consumer {
log => $log, output => $output, apt_dir => $apt_root,
extra => [ '--genesis-release', $release_root ],
);
my $pool_package = "$apt_root/pool/main/noble/$package";
my $pool_package = "$shared_pool/$package";
my $amd64 = "$apt_root/dists/noble/main/binary-amd64/Packages";
my $ppc64el = "$apt_root/dists/noble/main/binary-ppc64el/Packages";
is($status, 0, 'APT repository accepts a verified Genesis release');
like(read_binary($log), qr/installed \+ verified 7 Genesis release package\(s\) into noble/,
'APT repository verifies every copied package as it pools it');
is(digest_file($pool_package), digest_file("$release_root/deb/$package"),
'pooled DEB matches the release');
is(digest_file("$apt_root/ubuntu24.04/$package"), digest_file("$release_root/deb/$package"),
'the flat per-version directory carries the release too');
ok(!-e "$apt_root/pool/main/noble/xcat-genesis-openembedded-old.deb",
is(
sprintf('%04o', (stat($pool_package))[2] & 0x0fff),
'0644',
'the pooled DEB is readable by an unprivileged server',
);
ok(!-e "$shared_pool/xcat-genesis-openembedded-old.deb",
'stale pooled DEB is removed');
like(read_binary($amd64), qr/^Package: xcat-genesis-openembedded-x86-64$/m,
'all-architecture DEB is indexed for amd64');
like(read_binary($ppc64el), qr/^Package: xcat-genesis-openembedded-x86-64$/m,
'all-architecture DEB is indexed for ppc64el');
ok(-f "$apt_root/dists/noble/Release", 'APT Release metadata is generated');
my @expected_packages = sort map {
deb_package_name($_) . "_${version}-${release}_all.deb"
} architectures();
is_deeply([ genesis_deb_names($shared_pool) ], \@expected_packages,
'shared APT pool contains one complete Genesis release');
my @suite_packages;
for my $codename (@APT_SUITES) {
push(@suite_packages, map { "$codename/$_" }
genesis_deb_names("$apt_root/pool/main/$codename"));
}
is_deeply(\@suite_packages, [], 'suite pools contain no OpenEmbedded Genesis packages');
for my $codename (@APT_SUITES) {
for my $architecture (qw(amd64 ppc64el)) {
my $packages = read_binary(
"$apt_root/dists/$codename/main/binary-$architecture/Packages"
);
like($packages,
qr{^Filename: pool/main/xcat-genesis-openembedded/\Q$package\E$}m,
"$codename $architecture index uses the shared Genesis package",
);
}
}
is_deeply(
[ grep { !-f "$apt_root/dists/$_/Release" } @APT_SUITES ],
[],
'APT Release metadata is generated for every suite',
);
like(read_binary($amd64), qr/^Package: xcat-genesis-base-amd64$/m,
'legacy Genesis DEB remains available');
ok(!-e "$apt_root/pool/main/noble/xcat-genesis-openembedded-stale.deb",
'stale OpenEmbedded DEB is not collected');
'a staged OpenEmbedded DEB is not published into a suite pool');
# The published package must be a file of its own: sharing an inode with the release
# would make a later write through either path change what the other one holds.
my @pooled = stat($pool_package);
my @released = stat("$release_root/deb/$package");
isnt("$pooled[0]:$pooled[1]", "$released[0]:$released[1]",
'pooled DEB is published independently of the release file');
my $publication_log = read_binary($log);
my $verified_at = index($publication_log, 'published + verified');
my $indexed_at = index($publication_log, 'assembled + ');
ok($verified_at >= 0 && $verified_at < $indexed_at,
'staged DEBs are verified before any suite index is generated');
# A staged package under the release's own name is dropped, not published: with
# --genesis-release the verified release is the ONLY source of those packages.
# A release rewrites the shared pool every suite indexes, so it cannot be published for
# some suites only -- the others would be left indexing files the new release retired.
my $before_subset = digest_file($pool_package);
my $subset_log = "$tmp/deb-subset.log";
my $subset_status = run_apt_consumer(
log => $subset_log, output => $output, apt_dir => $apt_root,
dists => ['noble'],
extra => [ '--genesis-release', $release_root ],
);
isnt($subset_status, 0, 'Genesis publication rejects a partial suite update');
like(read_binary($subset_log), qr/must\n?\s*cover all of them/,
'partial suite failure explains the consistency requirement');
is(digest_file($pool_package), $before_subset,
'partial suite failure leaves the shared package unchanged');
# A later suite rebuild carries no release and must keep indexing the shared pool.
my $refresh_log = "$tmp/deb-refresh.log";
my $refresh_status = run_apt_consumer(
log => $refresh_log, output => $output, apt_dir => $apt_root,
dists => ['noble'],
);
is($refresh_status, 0, 'a suite refresh reuses the shared Genesis pool');
is(digest_file($pool_package), $before_subset,
'a suite refresh leaves the shared package unchanged');
like(read_binary($amd64),
qr{^Filename: pool/main/xcat-genesis-openembedded/\Q$package\E$}m,
'the refreshed suite still indexes the shared Genesis package');
# The repository changes with ONE rename, at the very end: a publication that fails while it
# assembles must leave every published byte -- packages, indexes and key -- exactly as it was.
# Signing with a key the keyring does not hold fails inside that phase.
my $package_before = digest_file(
"$apt_root/pool/main/noble/xcat-genesis-base-amd64_1_all.deb");
my $metadata_before = digest_file($amd64);
my $empty_keyring = "$tmp/apt-empty-keyring";
make_path($empty_keyring);
chmod(0700, $empty_keyring) or die $!;
my $failed_log = "$tmp/deb-failed-publish.log";
my $failed_status = run_apt_consumer(
log => $failed_log, output => $output, apt_dir => $apt_root,
dists => ['noble'],
extra => [ '--gpg-sign', '--gpg-key-id', 'absent@example.invalid',
'--gpg-home', $empty_keyring ],
);
isnt($failed_status, 0, 'a publication that cannot be signed is not published');
like(read_binary($failed_log), qr/NOT publishing/,
'the refusal says the published repository was left alone');
is(digest_file("$apt_root/pool/main/noble/xcat-genesis-base-amd64_1_all.deb"),
$package_before, 'a failed publication leaves the published package bytes');
is(digest_file($amd64), $metadata_before,
'a failed publication leaves the published package index');
is(digest_file($pool_package), $before_subset,
'a failed publication leaves the shared Genesis pool');
# A rebuilt deb keeps its filename; the published copy must become the new one.
my $legacy_pool = "$apt_root/pool/main/noble/xcat-genesis-base-amd64_1_all.deb";
my $legacy_before = digest_file($legacy_pool);
make_legacy_deb(
"$tmp/rebuilt-legacy-deb",
"$staged/xcat-genesis-base-amd64_1_all.deb",
'rebuilt repository test package',
);
my $replace_log = "$tmp/deb-replace.log";
my $replace_status = run_apt_consumer(
log => $replace_log, output => $output, apt_dir => $apt_root,
dists => ['noble'],
);
is($replace_status, 0, 'a rebuilt DEB may keep its published filename');
isnt(digest_file($legacy_pool), $legacy_before,
'the rebuilt DEB replaces the earlier package');
# A staged package under a release package's own name is dropped, not published.
my $collision = "$tmp/apt-collision";
my $collision_output = "$tmp/deb-collision-output";
my $collision_staging = stage_legacy_deb("$tmp/collision-deb", $collision_output);
write_binary("$collision_staging/$package", 'different');
my $collision_staged = stage_apt_suites($collision_output, "$tmp/collision-deb");
write_binary("$collision_staged/$package", 'different');
my $collision_log = "$tmp/deb-collision.log";
my $collision_status = run_apt_consumer(
log => $collision_log, output => $collision_output, apt_dir => $collision,
extra => [ '--genesis-release', $release_root ],
);
is($collision_status, 0, 'verified release replaces a stale staged package');
is(digest_file("$collision/pool/main/noble/$package"),
is(digest_file("$collision/pool/main/xcat-genesis-openembedded/$package"),
digest_file("$release_root/deb/$package"),
'pooled package still matches the verified release');
}
sub test_signed_common_rpm_repository {
my $release_root = make_package_release("$tmp/rpm-signed", 'rpm');
my $package = "xCAT-genesis-openembedded-x86_64-$version-$release.noarch.rpm";
my $output = "$tmp/rpm-signed-output";
my $target = 'test+epel-10-' . capture_command('uname', '-m');
my $dependencies = "$tmp/rpm-signed-dependencies";
my $scratch_repo_root = "$tmp/rpm-signed-repo-root";
my $common = "$output/xcat-dep/common";
my $gpg_home = "$tmp/rpm-signing-key";
my $identity = 'xCAT repository test <xcat-repository-test@example.invalid>';
make_rpm_dependencies($dependencies, "$release_root/rpm/$package");
make_path($scratch_repo_root, $gpg_home);
chmod(0700, $gpg_home) or die $!;
system(
'gpg', '--batch', '--homedir', $gpg_home, '--passphrase', '',
'--quick-generate-key', $identity, 'rsa2048', 'sign', '0',
) == 0 or die "could not create the repository test key";
my @perl_lib;
push(@perl_lib, write_forkmanager_stub("$tmp/perl-signed-stub"))
unless eval { require Parallel::ForkManager; 1 };
push(@perl_lib, $ENV{PERL5LIB})
if defined($ENV{PERL5LIB}) && $ENV{PERL5LIB} ne '';
local $ENV{PERL5LIB} = join(':', @perl_lib);
my $log = "$tmp/rpm-signed-consumer.log";
my $status = run_capture(
$log,
$^X, $rpm_consumer,
'--repo-root', $scratch_repo_root,
'--xcat-source', $repo_root,
'--output', $output,
'--target', $target,
'--run-id', 'signed-consumer',
'--build-timestamp', $epoch,
'--skip-build', '--skip-xcat', '--skip-xcat-dep', '--skip-perl',
'--skip-genesis', '--skip-createrepo', '--skip-tarball',
'--collect-dir', $dependencies,
'--genesis-release', $release_root,
'--gpg-sign', '--gpg-key-name', $identity, '--gpg-home', $gpg_home,
);
diag(read_binary($log)) if $status;
is($status, 0, 'the common RPM repository can be signed');
ok(-f "$common/repodata/repomd.xml.key",
'the common repository exports its own public key');
is(
system(
'gpg', '--batch', '--homedir', $gpg_home, '--verify',
"$common/repodata/repomd.xml.asc",
"$common/repodata/repomd.xml",
) >> 8,
0,
'the common repository metadata signature verifies',
);
like(
read_binary("$common/xcat-dep-common.repo"),
qr{^gpgkey=https://xcat\.org/files/xcat/repos/yum/devel/xcat-dep/common/repodata/repomd\.xml\.key$}m,
'the common repository file points to its own exported key',
);
like(read_binary("$common/xcat-dep-common.repo"), qr/^repo_gpgcheck=1$/m,
'the signed common repository requires metadata verification');
like(read_binary("$common/xcat-dep-common.repo"), qr/^skip_if_unavailable=1$/m,
'the signed common repository remains optional during outages');
}
sub test_legacy_rpm_consumer {
my $release_root = make_package_release("$tmp/rpm-legacy", 'rpm', 'x86_64');
my $package = "xCAT-genesis-openembedded-x86_64-$version-$release.noarch.rpm";
@@ -293,6 +601,8 @@ sub test_legacy_rpm_consumer {
[],
'RPM legacy path does not add OpenEmbedded packages',
);
ok(!-d "$output/xcat-dep/common",
'RPM legacy path does not create the common repository');
}
sub test_legacy_deb_consumer {
@@ -316,9 +626,9 @@ sub test_partial_rpm_release {
my $release_root = make_package_release("$tmp/rpm-partial", 'rpm', 'x86_64');
my $output = "$tmp/partial-output";
my $target = 'test+epel-10-' . capture_command('uname', '-m');
my $deployed = "$output/xcat-dep/rh10/" . capture_command('uname', '-m');
my $existing = "$deployed/xCAT-genesis-base-existing.noarch.rpm";
make_path($deployed);
my $common = "$output/xcat-dep/common";
my $existing = "$common/xCAT-genesis-openembedded-existing.noarch.rpm";
make_path($common);
write_binary($existing, 'existing release');
my $log = "$tmp/rpm-partial.log";
@@ -368,14 +678,14 @@ sub test_failed_build_release {
my $package = "xCAT-genesis-openembedded-x86_64-$version-$release.noarch.rpm";
my $output = "$tmp/empty-output";
my $target = 'test+epel-10-' . capture_command('uname', '-m');
my $deployed = "$output/xcat-dep/rh10/" . capture_command('uname', '-m');
my $common = "$output/xcat-dep/common";
my $scratch_repo_root = "$tmp/rpm-empty-root";
my $collected = "$tmp/rpm-empty-collect";
my $run_repo = "$output/mockbuild-all/$target-empty/repo/" . capture_command('uname', '-m');
my $results = "$output/mockbuild-all/$target-empty/build-results/ipmitool-xcat";
my $stale = "$run_repo/ipmitool-xcat-0-stale.noarch.rpm";
my $kept = "$results/ipmitool-xcat-0-earlier.noarch.rpm";
make_path($deployed, $scratch_repo_root, $collected, $run_repo, $results);
make_path($common, $scratch_repo_root, $collected, $run_repo, $results);
write_binary($stale, 'package left by an earlier run');
write_binary($kept, 'build output an earlier run produced');
@@ -397,7 +707,7 @@ sub test_failed_build_release {
isnt($status, 0, 'a run that built nothing fails even with a Genesis release');
like(read_binary($log), qr/No binary RPMs were collected/,
'the failure names the empty collection, not the missing dependencies');
ok(!-e "$deployed/$package",
ok(!-e "$common/$package",
'a run that built nothing publishes no release package');
ok(!-e $stale,
'a package left by an earlier run is cleared from the staging repository');
@@ -434,16 +744,113 @@ sub test_dry_run_release {
my $printed = read_binary($log);
is($status, 0, 'a dry run accepts a verified Genesis release');
like($printed, qr/^DRY-RUN install Genesis release package: .*\Q$package\E$/m,
'the dry run reports the release packages it would install');
like($printed, qr/^Collected binary RPMs: 15$/m,
'the dry run counts the release packages a real run installs');
like($printed,
qr{^DRY-RUN publish Genesis release package: .*\Q$package\E -> .*/xcat-dep/common/\Q$package\E$}m,
'the dry run reports the shared package destination');
like($printed, qr/^Collected binary RPMs: 8$/m,
'the dry run counts the packages collected for the target repository');
ok(!-e "$run_repo/$package", 'the dry run installs nothing');
}
# The apt tree is published under sbuild-all.pl's own publish lock (flock, one global holder), not
# the mkdir lock the shell publisher used. A run that cannot take it must publish NOTHING rather than
# assemble into a tree another publisher owns.
sub test_rpm_repository_lock {
my $output = "$tmp/rpm-lock-output";
my $repository = "$tmp/rpm-shared-repository";
make_path("$repository/.lock");
write_binary("$repository/.lock/owner", "host=other\npid=1\nepoch=1\n");
my @perl_lib;
push(@perl_lib, write_forkmanager_stub("$tmp/perl-lock-stub"))
unless eval { require Parallel::ForkManager; 1 };
push(@perl_lib, $ENV{PERL5LIB})
if defined($ENV{PERL5LIB}) && $ENV{PERL5LIB} ne '';
local $ENV{PERL5LIB} = join(':', @perl_lib);
my $log = "$tmp/rpm-repository-lock.log";
my $status = run_capture(
$log,
$^X, $rpm_consumer,
'--repo-root', $repo_root,
'--output', $output,
'--repo-dep', $repository,
'--target', 'test+epel-10-' . capture_command('uname', '-m'),
'--skip-build', '--skip-xcat', '--skip-xcat-dep', '--skip-perl',
'--skip-createrepo', '--skip-tarball', '--dry-run',
);
isnt($status, 0, 'a shared RPM repository cannot have two publishers');
like(read_binary($log), qr/repository \Q$repository\E is locked/,
'the lock failure names the shared repository');
my $backup = "$repository/.common.previous.999";
my $staging = "$repository/.common.abandoned";
make_path($backup, $staging);
write_binary("$backup/marker", "previous repository\n");
my $forced_log = "$tmp/rpm-repository-force.log";
my $forced_status = run_capture(
$forced_log,
$^X, $rpm_consumer,
'--repo-root', $repo_root,
'--output', "$tmp/rpm-force-output",
'--repo-dep', $repository,
'--target', 'test+epel-10-' . capture_command('uname', '-m'),
'--skip-build', '--skip-xcat', '--skip-xcat-dep', '--skip-perl',
'--skip-createrepo', '--skip-tarball', '--dry-run', '--force-unlock',
);
is($forced_status, 0, '--force-unlock recovers an interrupted RPM publication');
ok(-f "$repository/common/marker",
'the interrupted common repository is restored before publication');
ok(!-d $staging, 'abandoned common repository staging is removed');
}
sub test_rpm_signal_cleanup {
my $output = "$tmp/rpm-signal-output";
my $repository = "$tmp/rpm-signal-repository";
my $signal_bin = "$tmp/rpm-signal-bin";
my $log = "$tmp/rpm-signal.log";
make_path($repository, $signal_bin);
write_binary(
"$signal_bin/uname",
"#!/bin/sh\n"
. "if [ \"\$1\" = -m ]; then sleep 60; exit 1; fi\n"
. "exec /usr/bin/uname \"\$@\"\n",
);
chmod(0755, "$signal_bin/uname") or die $!;
my $pid = fork();
die "Cannot fork signal test: $!" unless defined($pid);
if ($pid == 0) {
POSIX::setpgid(0, 0);
local $ENV{PATH} = "$signal_bin:$ENV{PATH}";
open(STDOUT, '>', $log) or die "open $log: $!";
open(STDERR, '>&', fileno(STDOUT)) or die "redirect stderr: $!";
exec(
$^X, $rpm_consumer,
'--repo-root', $repo_root,
'--xcat-source', $repo_root,
'--output', $output,
'--repo-dep', $repository,
'--target', 'test+epel-10-x86_64',
'--skip-build', '--skip-xcat', '--skip-xcat-dep', '--skip-perl',
'--skip-createrepo', '--skip-tarball', '--dry-run',
);
exit 127;
}
my $locked = 0;
for (1 .. 200) {
if (-d "$output/.lock" && -d "$repository/.lock") {
$locked = 1;
last;
}
sleep(0.05);
}
ok($locked, 'the signal test reaches the locked publication phase');
kill('TERM', -$pid);
waitpid($pid, 0);
is($? >> 8, 1, 'SIGTERM follows the publisher cleanup exit path');
ok(!-d "$output/.lock", 'SIGTERM releases the output lock');
ok(!-d "$repository/.lock", 'SIGTERM releases the repository lock');
}
sub test_publish_lock {
my $apt_root = "$tmp/apt-lock";
my $output = "$tmp/deb-lock-output";
@@ -466,11 +873,65 @@ sub test_publish_lock {
close($held);
# sbuild-all.pl never publishes in place: it assembles a COMPLETE side tree and renames it onto
# the repository, so there is no half-written state to recover and no per-file backup to restore.
# A side tree left by a run that died is simply inert -- the next run builds and publishes its own.
my $abandoned = "$apt_root.publish-abandoned.999";
make_path("$abandoned/dists/noble");
write_binary("$abandoned/dists/noble/Release", "abandoned\n");
my $freed_log = "$tmp/deb-freed.log";
my $freed_status = run_apt_consumer(
log => $freed_log, output => $output, apt_dir => $apt_root);
log => $freed_log, output => $output, apt_dir => $apt_root, dists => ['noble']);
is($freed_status, 0, 'the publish runs once the lock is released');
ok(-f "$apt_root/dists/noble/Release", 'the released lock lets the tree be published');
isnt(read_binary("$apt_root/dists/noble/Release"), "abandoned\n",
'a side tree abandoned by a dead run is never published');
}
sub test_activation_helper {
my $activation = read_binary("$repo_root/genesis-openembedded/activate");
unlike($activation, qr/XCAT_GENESIS_ROOT/,
'the root package helper has no environment-controlled execution root');
$activation =~ s/\ngenesis_activation_main "\$\@"\s*\z/\n/
or BAIL_OUT('the activation helper has no reusable main boundary');
my $driver = "$tmp/activation-driver";
my $log = "$tmp/activation.log";
my $output = "$tmp/activation-output.log";
write_binary(
$driver,
$activation
. <<'SH',
genesis_is_host_root() { return 0; }
genesis_mknb_exists() { return 0; }
genesis_is_service_node() { [ "${XCAT_TEST_SERVICE_NODE:-0}" = 1 ]; }
genesis_uses_shared_tftp() { [ "${XCAT_TEST_SHAREDTFTP:-0}" = 1 ]; }
genesis_run_mknb() {
printf '%s\n' "$1" >>"$XCAT_TEST_LOG"
return "${XCAT_TEST_MKNB_STATUS:-0}"
}
genesis_activation_main "$@"
SH
);
chmod(0755, $driver) or die "Cannot make activation driver executable: $!";
local $ENV{XCAT_TEST_LOG} = $log;
my $status = run_capture($output, $driver, 'x86_64');
is($status, 0, 'the activation helper accepts a local-TFTP node');
is(read_binary($log), "x86_64\n", 'the activation helper runs mknb for one architecture');
write_binary($log, '');
local $ENV{XCAT_TEST_SERVICE_NODE} = 1;
local $ENV{XCAT_TEST_SHAREDTFTP} = 1;
$status = run_capture($output, $driver, 'ppc64le');
is($status, 0, 'a shared-TFTP service node is accepted');
is(read_binary($log), '', 'a shared-TFTP service node does not rebuild Genesis');
local $ENV{XCAT_TEST_SERVICE_NODE} = 0;
local $ENV{XCAT_TEST_MKNB_STATUS} = 1;
$status = run_capture($output, $driver, 'ppc64le');
is($status, 0, 'an mknb failure does not fail the package transaction');
}
sub make_package_release {
@@ -522,6 +983,24 @@ sub make_package_release {
return $release_root;
}
sub genesis_rpm_names {
my ($directory) = @_;
return () unless -d $directory;
opendir(my $dh, $directory) or die $!;
my @names = sort grep { /^xCAT-genesis-openembedded-.*\.rpm$/ } readdir($dh);
closedir($dh) or die $!;
return @names;
}
sub genesis_deb_names {
my ($directory) = @_;
return () unless -d $directory;
opendir(my $dh, $directory) or die $!;
my @names = sort grep { /^xcat-genesis-openembedded-.*\.deb$/ } readdir($dh);
closedir($dh) or die $!;
return @names;
}
sub make_rpm_dependencies {
my ($directory, $package) = @_;
make_path($directory);
@@ -535,16 +1014,29 @@ sub make_rpm_dependencies {
}
sub make_legacy_deb {
my ($root, $output) = @_;
my ($root, $output, $description) = @_;
$description //= 'repository test package';
make_path("$root/DEBIAN");
write_binary(
"$root/DEBIAN/control",
"Package: xcat-genesis-base-amd64\nVersion: 1\nArchitecture: all\n"
. "Maintainer: xCAT <xcat-user\@lists.sourceforge.net>\n"
. "Description: repository test package\n",
. "Description: $description\n",
);
die "Cannot build legacy test DEB\n"
if run_capture(
"$root.log", 'dpkg-deb', '--root-owner-group', '--build', $root, $output,
);
}
sub make_apt_inputs {
my ($apt_root, $package_root) = @_;
my $package = "$package_root.deb";
make_legacy_deb($package_root, $package);
for my $version (qw(ubuntu22.04 ubuntu24.04 ubuntu26.04)) {
my $input = "$apt_root/$version";
make_path($input);
copy($package, "$input/xcat-genesis-base-amd64_1_all.deb") or die $!;
}
return "$apt_root/ubuntu24.04";
}
t/genesis_openembedded_release.t
+22 -3
View File
@@ -381,7 +381,16 @@ sub exercise_packager {
);
is(run_capture($contents_log, 'rpm', '-qp', '--scripts', "$first/$relative"), 0,
'RPM script metadata can be read');
is(read_binary($contents_log), '', 'RPM has no package scripts');
like(
read_binary($contents_log),
qr{posttrans scriptlet.*\n/usr/libexec/xcat/genesis-openembedded-activate-x86_64 x86_64\n}s,
'RPM refreshes the installed architecture after the transaction',
);
like(
read_binary($contents_log),
qr{postuninstall scriptlet.*mknb x86_64 --remove-openembedded}s,
'RPM retires published artifacts after erasing the image',
);
} else {
is(run_capture($contents_log, 'dpkg-deb', '-c', "$first/$relative"), 0,
'DEB payload can be listed');
@@ -395,8 +404,18 @@ sub exercise_packager {
opendir(my $control_dh, $control) or die $!;
my @control_files = sort grep { $_ ne '.' && $_ ne '..' } readdir($control_dh);
closedir($control_dh) or die $!;
is_deeply(\@control_files, [ qw(control md5sums) ],
'DEB has no maintainer scripts or triggers');
is_deeply(\@control_files, [ qw(control md5sums postinst postrm) ],
'DEB contains the expected installation and removal scripts');
like(
read_binary("$control/postinst"),
qr{^#!/bin/sh\n/usr/libexec/xcat/genesis-openembedded-activate-x86_64 x86_64\nexit 0\n$},
'DEB refreshes the installed architecture after configuration',
);
like(
read_binary("$control/postrm"),
qr{mknb x86_64 --remove-openembedded},
'DEB retires published artifacts after erasing the image',
);
}
}