diff --git a/BUILD.md b/BUILD.md index f581119..1c87ae4 100644 --- a/BUILD.md +++ b/BUILD.md @@ -87,6 +87,8 @@ Use these flags to skip specific operations: - Adds extra artifact roots to the collection phase (repeatable). - `--dry-run` - Prints planned actions without executing them. +- `--force-unlock` + - Removes a stale lock after the previous publisher has been checked. # Prerequisites @@ -168,14 +170,21 @@ perl ./mockbuild-all.pl \ --genesis-release /path/to/xcat-genesis-release ``` -The release is checked before any package is collected. Its RPMs and SRPMs are -copied into each generated EL repository. Stale OpenEmbedded Genesis packages -are removed from the output first, while the existing per-EL Genesis package -remains available. +The release is checked before any package is collected. Its binary RPMs are +published once under `xcat-dep/common`. Source RPMs stay in the verified +release directory. Existing per-EL repositories keep the old Genesis packages +and contain no OpenEmbedded copies. + +The build holds separate locks for its work area and the published repository. +It prepares the complete common repository in a temporary directory, then +replaces the previous repository only after package verification, metadata +generation, and signing have succeeded. If a stopped publisher leaves staging +or backup directories behind, rerun it with ``--force-unlock`` to recover the +previous repository before starting a new publication. Repository publication requires a release containing every supported Genesis -architecture. The packages are `noarch`, and every management-node repository -receives the full set of target images. +architecture. The packages are `noarch`, and the common repository contains +the full set of target images. The release checksums cover the unsigned input packages. If repository signing is enabled, `rpmsign` changes the deployed RPM bytes after collection. @@ -187,9 +196,7 @@ a separate xcat-core change. Omit `--genesis-release` to keep using the existing Genesis builder. -The APT side takes the same option, on the run that **publishes** (the release is -copied into every selected suite while the publish lock is held, so what is indexed -and signed is what was verified): +The APT side takes the same option, on the run that **publishes**: ```bash ./sbuild-all.pl --skip-build --skip-genesis \ @@ -198,10 +205,21 @@ and signed is what was verified): --gpg-sign --gpg-key-id --gpg-home ``` -Anything staged under the OpenEmbedded Genesis package name is dropped when -`--genesis-release` is given: the verified release is the only source of those -packages. `sbuild-all.pl` loads the release reader only when the option is used, so -an apt build without it does not need `perl-File-Slurper` on the build host. +The debs are published **once**, under `pool/main/xcat-genesis-openembedded`, and +every suite's `Packages` index points at that one copy — they are +`Architecture: all` and identical for every codename. Because every suite indexes +it, a release must be published for all of them: a run whose `--dists` omits a +suite is refused rather than leaving that suite indexing retired files. Each +package is copied and re-checked against the release checksums while the publish +lock is held, so what is indexed and signed is what was verified, and anything +staged under the OpenEmbedded Genesis package name is dropped: the release is the +only source of those packages. `sbuild-all.pl` loads the release reader only when +the option is used, so an apt build without it does not need `perl-File-Slurper` +on the build host. + +The per-target repository tarballs do not contain `xcat-dep/common`. For an +offline installation, mirror the common repository with its metadata before +disconnecting the installation network. # Build Unified Repository Without xCAT (`--skip-xcat`) diff --git a/genesis-openembedded/README.md b/genesis-openembedded/README.md index fd647a8..33075a0 100644 --- a/genesis-openembedded/README.md +++ b/genesis-openembedded/README.md @@ -56,8 +56,8 @@ the release, so only accept a directory produced by a trusted build host. `verify-release` also checks the identity of each package, including a fixed build host and a build time taken from the source epoch. Those are reproduced -by rpm 4.14.3, 4.16.1.3, 4.19.1.1 and 6.0.2, so an EL8 or later builder -- and -a current Fedora one -- produces a release the verifier accepts. +by rpm 4.14.3, 4.16.1.3, 4.19.1.1, and 6.0.2. An EL8 or later builder, or a +current Fedora builder, produces a release the verifier accepts. Pass that same directory to the repository builders: @@ -72,18 +72,40 @@ perl ./sbuild-all.pl \ [other build options] ``` -The RPM builder keeps its old per-EL Genesis build and adds the new packages. -The APT builder does the same for each selected suite, during its locked, -atomic publish step. Both consumers require -all seven architectures and verify package identities and checksums before -collecting packages. Every management-node repository receives every target -image so it can provision nodes of another architecture. +The RPM builder publishes the binary packages once under `xcat-dep/common`. +The per-EL repositories keep the old Genesis packages and contain no copies of +the OpenEmbedded packages. Source RPMs stay in the verified release directory. +The publisher locks the shared repository and replaces `common` only after the +new package set, metadata, signatures, and local setup files are ready. + +The APT builder publishes the DEBs once under +`pool/main/xcat-genesis-openembedded`. Every suite indexes those same files. +Genesis publication updates all suites together, so a run whose `--dists` omits +one is refused. Later suite rebuilds keep using the shared pool. Pass a new +release only when replacing the Genesis packages. + +APT metadata is assembled in a side tree and swapped onto the published +repository with a single rename, under one global publish lock, so a failed or +interrupted publication leaves the previous repository exactly as it was. + +Both consumers require all seven architectures and verify package identities +and checksums before publication. A management node can install an image for a +different target architecture. Without `--genesis-release`, both builders keep their existing behavior. The -new packages do not provide, replace, or obsolete the old package names. A -separate xcat-core change will select the OpenEmbedded package and install -namespace after the repositories carry it. Generated images and packages -belong in release storage, not in Git. +new packages do not provide, replace, or obsolete the old package names. +xcat-core selects the OpenEmbedded install namespace when an image package is +present and falls back to the old Genesis image otherwise. Package installation +or update runs `mknb` for that package's architecture on nodes with local TFTP +storage. With ``site.sharedtftp=0``, update service nodes before the management +node so they can serve the exact architecture name immediately. Removing an +image package retires its published files and rebuilds any legacy fallback that +is still installed. Generated images and packages belong in release storage, +not in Git. + +The per-target RPM tarballs do not include `xcat-dep/common`. Mirror that +repository separately, including its metadata, when preparing an offline +installation. Run the package tests on a Linux builder with RPM, DEB, and repository tools: diff --git a/genesis-openembedded/activate b/genesis-openembedded/activate new file mode 100755 index 0000000..9a027a4 --- /dev/null +++ b/genesis-openembedded/activate @@ -0,0 +1,60 @@ +#!/bin/sh + +genesis_is_host_root() { + [ -f /proc/cmdline ] || return 1 + host_root=$(stat -c '%i %d' / 2>/dev/null) || return 1 + init_root=$(stat -c '%i %d' /proc/1/root/. 2>/dev/null) || return 1 + [ "$host_root" = "$init_root" ] +} + +genesis_mknb_exists() { + [ -x /opt/xcat/sbin/mknb ] +} + +genesis_is_service_node() { + if command -v rpm >/dev/null 2>&1 \ + && rpm -q xCATsn >/dev/null 2>&1; then + return 0 + fi + command -v dpkg-query >/dev/null 2>&1 || return 1 + dpkg-query -W -f='${Status}' xcatsn 2>/dev/null \ + | grep -q '^install ok installed$' +} + +genesis_uses_shared_tftp() { + [ -x /opt/xcat/sbin/tabdump ] || return 1 + sharedtftp=$( + /opt/xcat/sbin/tabdump site 2>/dev/null \ + | awk -F '"' '$2 == "sharedtftp" { print $4; exit }' + ) + [ "$sharedtftp" = 1 ] +} + +genesis_run_mknb() { + # shellcheck disable=SC1091 + [ ! -r /etc/profile.d/xcat.sh ] || . /etc/profile.d/xcat.sh + /opt/xcat/sbin/mknb "$1" +} + +genesis_activation_main() { + architecture=$1 + case "$architecture" in + x86|x86_64|ppc64|ppc64le|armv7hf|aarch64|riscv64) ;; + *) + echo "Invalid Genesis architecture: $architecture" >&2 + return 0 + ;; + esac + + genesis_is_host_root || return 0 + genesis_mknb_exists || return 0 + if genesis_is_service_node && genesis_uses_shared_tftp; then + return 0 + fi + if ! genesis_run_mknb "$architecture"; then + echo "WARNING: mknb $architecture failed; rerun it after xcatd is available." >&2 + fi + return 0 +} + +genesis_activation_main "$@" diff --git a/genesis-openembedded/package b/genesis-openembedded/package index 937e380..516a640 100755 --- a/genesis-openembedded/package +++ b/genesis-openembedded/package @@ -91,6 +91,10 @@ for my $entry (_flat_files($export_dir)) { copy("$export_dir/$entry", "$source_root/image/$entry") or die "Cannot copy export file $entry: $!\n"; } +copy("$FindBin::Bin/activate", "$source_root/activate") + or die "Cannot copy the Genesis activation helper: $!\n"; +chmod(0755, "$source_root/activate") + or die "Cannot set the Genesis activation helper mode: $!\n"; write_binary("$source_root/xcat-core-revision", "$revision\n"); local $ENV{SOURCE_DATE_EPOCH} = $source_date_epoch; @@ -158,7 +162,8 @@ sub _build_deb { my $root = "$work/deb-root"; my $image = "$root/opt/xcat/share/xcat/netboot/genesis-openembedded/$architecture"; my $doc = "$root/usr/share/doc/$package_name"; - make_path("$root/DEBIAN", $image, $doc, "$staging/deb"); + my $helper = "$root/usr/libexec/xcat/genesis-openembedded-activate-$architecture"; + make_path("$root/DEBIAN", $image, $doc, dirname($helper), "$staging/deb"); for my $entry (_flat_files("$source_root/image")) { copy("$source_root/image/$entry", "$image/$entry") @@ -168,6 +173,9 @@ sub _build_deb { copy("$source_root/xcat-core-revision", "$doc/xcat-core-revision") or die "Cannot stage DEB revision: $!\n"; chmod(0644, "$doc/xcat-core-revision") or die "Cannot set revision mode: $!\n"; + copy("$source_root/activate", $helper) + or die "Cannot stage DEB activation helper: $!\n"; + chmod(0755, $helper) or die "Cannot set activation helper mode: $!\n"; my $installed_kib = _tree_bytes($root); $installed_kib = int(($installed_kib + 1023) / 1024); @@ -182,6 +190,31 @@ sub _build_deb { . " The OpenEmbedded Genesis image used for node discovery, inventory,\n" . " and service actions.\n", ); + write_binary( + "$root/DEBIAN/postinst", + "#!/bin/sh\n" + . "/usr/libexec/xcat/genesis-openembedded-activate-$architecture $architecture\n" + . "exit 0\n", + ); + chmod(0755, "$root/DEBIAN/postinst") + or die "Cannot set post-installation script mode: $!\n"; + write_binary( + "$root/DEBIAN/postrm", + "#!/bin/sh\n" + . "case \"\$1\" in\n" + . " remove|purge)\n" + . " host_root=\$(stat -c '%i %d' / 2>/dev/null) || host_root=\n" + . " init_root=\$(stat -c '%i %d' /proc/1/root/. 2>/dev/null) || init_root=\n" + . " if [ -f /proc/cmdline ] && [ -n \"\$host_root\" ] && [ \"\$host_root\" = \"\$init_root\" ] && [ -x /opt/xcat/sbin/mknb ]; then\n" + . " /opt/xcat/sbin/mknb $architecture --remove-openembedded || :\n" + . " /opt/xcat/sbin/mknb $architecture || :\n" + . " fi\n" + . " ;;\n" + . "esac\n" + . "exit 0\n", + ); + chmod(0755, "$root/DEBIAN/postrm") + or die "Cannot set post-removal script mode: $!\n"; _write_deb_md5sums($root); _normalize_mtime($root, $source_date_epoch); diff --git a/genesis-openembedded/rpm/xCAT-genesis-openembedded.spec b/genesis-openembedded/rpm/xCAT-genesis-openembedded.spec index 965203c..b8d8baa 100644 --- a/genesis-openembedded/rpm/xCAT-genesis-openembedded.spec +++ b/genesis-openembedded/rpm/xCAT-genesis-openembedded.spec @@ -30,6 +30,22 @@ install -m 0644 image/* "$RPM_BUILD_ROOT/opt/xcat/share/xcat/netboot/genesis-ope # Keep this path stable across RPM build hosts. %%{_docdir} differs on SUSE. install -d -m 0755 "$RPM_BUILD_ROOT/usr/share/doc/%{name}" install -m 0644 xcat-core-revision "$RPM_BUILD_ROOT/usr/share/doc/%{name}/" +install -d -m 0755 "$RPM_BUILD_ROOT/usr/libexec/xcat" +install -m 0755 activate "$RPM_BUILD_ROOT/usr/libexec/xcat/genesis-openembedded-activate-%{genesis_arch}" + +%posttrans +/usr/libexec/xcat/genesis-openembedded-activate-%{genesis_arch} %{genesis_arch} + +%postun +if [ "$1" -eq 0 ] && [ -f /proc/cmdline ]; then + host_root=$(stat -c '%i %d' / 2>/dev/null) || host_root= + init_root=$(stat -c '%i %d' /proc/1/root/. 2>/dev/null) || init_root= + if [ -n "$host_root" ] && [ "$host_root" = "$init_root" ] \ + && [ -x /opt/xcat/sbin/mknb ]; then + /opt/xcat/sbin/mknb %{genesis_arch} --remove-openembedded || : + /opt/xcat/sbin/mknb %{genesis_arch} || : + fi +fi %files %defattr(-,root,root,-) @@ -41,3 +57,5 @@ install -m 0644 xcat-core-revision "$RPM_BUILD_ROOT/usr/share/doc/%{name}/" /opt/xcat/share/xcat/netboot/genesis-openembedded/%{genesis_arch} %dir /usr/share/doc/%{name} %doc /usr/share/doc/%{name}/xcat-core-revision +%dir /usr/libexec/xcat +/usr/libexec/xcat/genesis-openembedded-activate-%{genesis_arch} diff --git a/mockbuild-all.pl b/mockbuild-all.pl index 608cc47..331577f 100755 --- a/mockbuild-all.pl +++ b/mockbuild-all.pl @@ -9,7 +9,7 @@ use File::Copy qw(copy); use File::Find qw(find); use File::Glob qw(bsd_glob); use File::Path qw(make_path remove_tree); -use File::Temp qw(tempfile); +use File::Temp qw(tempdir tempfile); use FindBin; use Getopt::Long qw(GetOptions); use Parallel::ForkManager; @@ -63,9 +63,14 @@ my $repo_dep = ''; my $gpg_sign = 0; my $gpg_key_name = 'xCAT Signing Key'; my $gpg_home = ''; +my $gpg_program = ''; my $force_unlock = 0; -my $HELD_LOCK; # path of the output lock this process owns (for cleanup on exit) -my $LOCK_OWNER_PID; # pid that created the lock; forked children must NOT remove it +my @HELD_LOCKS; +my $LOCK_OWNER_PID; +my ($COMMON_STAGE, $COMMON_DESTINATION, $COMMON_BACKUP); +for my $sig (qw(INT TERM HUP)) { + $SIG{$sig} = sub { exit 1; }; +} GetOptions( 'repo-root=s' => \$repo_root, @@ -133,10 +138,15 @@ $output_base = abs_path($output_base) $output_root = "$output_base/mockbuild-all" if $output_root eq ''; # Deployable per-EL xcat-dep repo root (rh8/rh9/rh10/ assembled here). $repo_dep = "$output_base/xcat-dep" if $repo_dep eq ''; +make_path($repo_dep) if !-d $repo_dep; +$repo_dep = abs_path($repo_dep) + or die "Cannot resolve --repo-dep directory\n"; # Fail-fast lock on the output base so a second run against the same --output aborts instead of # racing on the shared NFS tree. Held for the whole invocation; released by the exit handlers. acquire_output_lock($output_base, $force_unlock); +acquire_repository_lock($repo_dep, $force_unlock) + if $repo_dep ne $output_base; $xcat_src = resolve_xcat_source($xcat_src, $repo_root); @@ -155,7 +165,7 @@ for my $bin (qw(perl uname createrepo_c tar find rpm)) { } require_command('mock') if $scrub_all_chroots; require_command('rpmsign') if $gpg_sign; -require_command('gpg') if $gpg_sign; +$gpg_program = require_command('gpg') if $gpg_sign; if ($genesis_release ne '') { $genesis_release = abs_path($genesis_release) @@ -191,7 +201,8 @@ print_step('Targets to build'); print " $_\n" for @build_targets; print "output_base: $output_base\n"; print "deploy repo-dep: $repo_dep\n"; -print "lock: $output_base/.lock (held)\n"; +print "output lock: $output_base/.lock (held)\n"; +print "repository lock: $repo_dep/.lock (held)\n"; print "gpg_sign: $gpg_sign\n"; print "gpg_key_name: $gpg_key_name\n" if $gpg_sign; print "gpg_home: " . ($gpg_home ne '' ? $gpg_home : '(default keyring)') . "\n" if $gpg_sign; @@ -230,6 +241,8 @@ for my $tgt (@build_targets) { $tgt_pm->wait_all_children; die "FATAL: $tgt_fail target(s) failed\n" if $tgt_fail; +publish_genesis_common_repo() if $genesis_release; + print_step('All targets completed'); exit 0; @@ -519,12 +532,6 @@ if (!$dry_run && $copied == 0) { die "No binary RPMs were collected. Check build logs and collection roots.\n"; } -if ($genesis_release) { - $copied += $dry_run - ? preview_genesis_release_packages('rpm', $repo_dir) - : install_genesis_release_packages('rpm', $repo_dir); -} - # Ensure the OS-dependent xCAT-genesis-base rpm (built by the genesis step above) # lands in the dep repo even when the full xCAT core is built elsewhere (--skip-xcat). if (!$skip_genesis && !$dry_run) { @@ -550,15 +557,6 @@ if (!$dry_run && $copied_srpms == 0) { print "WARN: No source RPMs were collected. SRPM repo and tarball may be empty.\n"; } -if ($genesis_release) { - $copied_srpms += $dry_run - ? preview_genesis_release_packages('srpm', $srpm_repo_dir) - : install_genesis_release_packages('srpm', $srpm_repo_dir); -} - -assert_genesis_release_copied($repo_dir, $srpm_repo_dir) - if $genesis_release && !$dry_run; - if (!$skip_createrepo) { run_step( step => 'Run createrepo', @@ -652,11 +650,7 @@ sub deploy_target { my $destination = "$dest/" . basename($rpm); publish_file($rpm, $destination); } - if ($genesis_release) { - my @keep = map { basename($_) } genesis_release_files('rpm'); - remove_genesis_packages($dest, 0, \@keep); - verify_genesis_release_packages('rpm', $dest); - } + remove_genesis_packages($dest, 0) if $genesis_release; assert_required_deps($dest); sign_and_index_repo($dest); write_dep_repo_metadata($dest, $rel); @@ -664,6 +658,48 @@ sub deploy_target { print "Deployed rh$rel/$arch: $n rpms\n"; } +sub publish_genesis_common_repo { + my $dest = "$repo_dep/common"; + print_step("Publish OpenEmbedded Genesis -> $dest"); + + if ($dry_run) { + preview_genesis_release_packages('rpm', $dest); + return; + } + + $COMMON_STAGE = tempdir('.common.XXXXXXXX', DIR => $repo_dep, CLEANUP => 0); + my $published = publish_genesis_release_packages('rpm', $COMMON_STAGE); + verify_genesis_release_packages('rpm', $COMMON_STAGE); + sign_and_index_repo($COMMON_STAGE); + write_common_repo_metadata($COMMON_STAGE); + chmod(0755, $COMMON_STAGE) + or die "Cannot make $COMMON_STAGE traversable: $!\n"; + replace_common_repository($COMMON_STAGE, $dest); + print "Published common Genesis repository: $published rpms\n"; +} + +sub replace_common_repository { + my ($staged, $destination) = @_; + my $backup = "$repo_dep/.common.previous.$$"; + remove_tree($backup) if -e $backup || -l $backup; + + $COMMON_DESTINATION = $destination; + if (-e $destination || -l $destination) { + $COMMON_BACKUP = $backup; + rename($destination, $backup) + or die "Cannot preserve $destination before publication: $!\n"; + } + unless (rename($staged, $destination)) { + my $error = $!; + rename($backup, $destination) if $COMMON_BACKUP && -d $backup; + die "Cannot publish $destination: $error\n"; + } + undef($COMMON_STAGE); + undef($COMMON_DESTINATION); + remove_tree($backup) if $COMMON_BACKUP && -d $backup; + undef($COMMON_BACKUP); +} + sub publish_file { my ($source, $destination) = @_; my ($temporary_fh, $temporary) = tempfile( @@ -706,7 +742,8 @@ sub sign_and_index_repo { my @rpms = grep { !/\.src\.rpm$/ } bsd_glob("$dir/*.rpm"); if ($gpg_sign && @rpms) { local $ENV{GNUPGHOME} = $gpg_home if $gpg_home; - run_simple(qq(rpmsign --define "%_gpg_name $gpg_key_name" --addsign ) + run_simple('rpmsign --define ' . shell_quote("%_gpg_name $gpg_key_name") + . ' --define ' . shell_quote("%__gpg $gpg_program") . ' --addsign ' . join(' ', map { shell_quote($_) } @rpms)); } run_simple(createrepo_c_cmd($dir)); @@ -735,25 +772,62 @@ $gpgkey_line EOF close $r; + write_local_repo_helper($dir); + write_buildinfo($dir, "rh$rel/$arch"); +} + +sub write_common_repo_metadata { + my ($dir) = @_; + my $baseurl = "https://xcat.org/files/xcat/repos/yum/devel/xcat-dep/common"; + my $gpgcheck = $gpg_sign ? 1 : 0; + my $gpgkey_line = $gpg_sign ? "gpgkey=$baseurl/repodata/repomd.xml.key" : "# gpgkey="; + open my $r, '>', "$dir/xcat-dep-common.repo" + or die "Cannot write $dir/xcat-dep-common.repo: $!\n"; + print {$r} <<"EOF"; +[xcat-dep-common] +name=xCAT 2 common dependencies +baseurl=$baseurl +enabled=1 +gpgcheck=$gpgcheck +repo_gpgcheck=$gpgcheck +skip_if_unavailable=1 +$gpgkey_line +EOF + close $r; + + write_local_repo_helper($dir); + write_buildinfo($dir, 'common'); +} + +sub write_local_repo_helper { + my ($dir) = @_; + open my $m, '>', "$dir/mklocalrepo.sh" or die "Cannot write $dir/mklocalrepo.sh: $!\n"; print {$m} <<'EOS'; #!/bin/sh -cd `dirname $0` -REPOFILE=`basename xcat-*.repo` -if [[ $REPOFILE == "xcat-*.repo" ]]; then - echo "ERROR: For xcat-dep, please execute $0 in the correct / subdirectory" +SCRIPT_DIRECTORY=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd) +cd "$SCRIPT_DIRECTORY" || exit 1 +set -- xcat-*.repo +if [ "$#" -ne 1 ] || [ "$1" = "xcat-*.repo" ]; then + echo "ERROR: Execute $0 in an xcat-dep repository directory" exit 1 fi +REPOFILE=$1 DIRECTORY="/etc/yum.repos.d" if [ ! -d "$DIRECTORY" ]; then DIRECTORY="/etc/zypp/repos.d" fi -sed -e 's|baseurl=.*|baseurl=file://'"`pwd`"'|' $REPOFILE | sed -e 's|gpgkey=.*|gpgkey=file://'"`pwd`"'/repodata/repomd.xml.key|' > "$DIRECTORY/$REPOFILE" -cd - +CURRENT_DIRECTORY=$(pwd) +sed -e 's|baseurl=.*|baseurl=file://'"$CURRENT_DIRECTORY"'|' "$REPOFILE" \ + | sed -e 's|gpgkey=.*|gpgkey=file://'"$CURRENT_DIRECTORY"'/repodata/repomd.xml.key|' \ + > "$DIRECTORY/$REPOFILE" EOS close $m; chmod 0775, "$dir/mklocalrepo.sh"; +} +sub write_buildinfo { + my ($dir, $target) = @_; my $build_time = strftime("%a %b %e %H:%M:%S %Z %Y", gmtime($SOURCE_DATE_EPOCH)); my $build_machine = `hostname`; chomp $build_machine; my $commit = `git -C "$repo_root" rev-parse HEAD 2>/dev/null`; chomp $commit; @@ -762,7 +836,7 @@ EOS my $release = strftime('snap%Y%m%d%H%M', gmtime($SOURCE_DATE_EPOCH)); open my $b, '>', "$dir/buildinfo.txt" or die "Cannot write $dir/buildinfo.txt: $!\n"; print {$b} <<"EOF"; -TARGET=rh$rel/$arch +TARGET=$target RELEASE=$release BUILD_TIME=$build_time BUILD_MACHINE=$build_machine @@ -786,10 +860,10 @@ Options: it. A fail-fast lock is held at /.lock, so pass distinct paths to run on two hosts in parallel on one NFS (default: /build-output) --output-root PATH Override the derived build tree root (default: /mockbuild-all) - --repo-dep PATH Override the derived deployable per-EL output root; rh8/rh9/rh10/ - are assembled + signed here (default: /xcat-dep) + --repo-dep PATH Override the deployable output root; rh8/rh9/rh10/ and common + are assembled and signed here (default: /xcat-dep) --force-unlock Remove a stale /.lock before acquiring it - --gpg-sign Sign rpms + repomd.xml of each per-EL repo + --gpg-sign Sign RPMs and repomd.xml in every published repository --gpg-key-name NAME GPG key name (default: "xCAT Signing Key") --gpg-home PATH GNUPGHOME for signing (default: system keyring) --target NAME Build only this target (+epel--); default is @@ -811,8 +885,7 @@ Options: --skip-genesis Skip the existing per-EL Genesis image build --skip-createrepo Skip createrepo --skip-tarball Skip binary/SRPM tarball creation - --genesis-release PATH Add a verified OpenEmbedded Genesis RPM release alongside the - existing per-EL Genesis packages + --genesis-release PATH Publish a verified OpenEmbedded Genesis RPM release in common --scrub-all-chroots Run mock -r --scrub=all before build/collect --collect-dir PATH Additional directory to scan recursively for RPMs (repeatable) --dry-run Print planned commands without executing @@ -1019,16 +1092,16 @@ sub remove_genesis_packages { } } -sub install_genesis_release_packages { +sub publish_genesis_release_packages { my ($prefix, $destination_root) = @_; remove_genesis_packages($destination_root, $prefix eq 'srpm'); + remove_genesis_packages($destination_root, 1) if $prefix eq 'rpm'; my $copied = 0; for my $relative (genesis_release_files($prefix)) { my $source = "$genesis_release/$relative"; my $destination = "$destination_root/" . basename($relative); - copy($source, $destination) - or die "Cannot install Genesis release package $source: $!\n"; + publish_file($source, $destination); verify_release_file($genesis_release_checksums, $relative, $destination); $copied++; } @@ -1036,16 +1109,13 @@ sub install_genesis_release_packages { return $copied; } -# Dry runs copy nothing, but they must still report what a real run would publish: -# collect_rpms and collect_srpms drop every xCAT-genesis-openembedded package whenever -# --genesis-release is given, so without this preview a dry run describes a repository -# with no Genesis packages at all while the real run installs the whole set. +# Dry runs copy nothing, but they must still report the shared packages a real run publishes. sub preview_genesis_release_packages { my ($prefix, $destination_root) = @_; my @files = genesis_release_files($prefix); die "Genesis release has no $prefix packages\n" unless @files; for my $relative (@files) { - print "DRY-RUN install Genesis release package: $genesis_release/$relative" + print "DRY-RUN publish Genesis release package: $genesis_release/$relative" . " -> $destination_root/" . basename($relative) . "\n"; } return scalar(@files); @@ -1070,17 +1140,6 @@ sub verify_genesis_release_packages { return scalar(@files); } -sub assert_genesis_release_copied { - my ($binary_directory, $source_directory) = @_; - for my $entry ( - [ 'rpm', $binary_directory ], - [ 'srpm', $source_directory ], - ) { - my ($prefix, $destination_root) = @{$entry}; - verify_genesis_release_packages($prefix, $destination_root); - } -} - sub collect_rpms { my (%args) = @_; my $roots = $args{roots} // []; @@ -1249,16 +1308,16 @@ sub resolve_xcat_source { # Fail-fast advisory lock on the output base. Uses an atomic mkdir (portable and reliable over # NFS, unlike flock) of "/.lock". A second run against the same --output dies immediately # rather than racing on the shared tree. Only the process that created the lock removes it. -sub acquire_output_lock { - my ($base, $force) = @_; +sub acquire_named_lock { + my ($base, $label, $force) = @_; my $lock = "$base/.lock"; if ($force && -d $lock) { print "force-unlock: removing stale lock $lock\n"; _rmdir_lock($lock); } if (mkdir $lock) { - $HELD_LOCK = $lock; - $LOCK_OWNER_PID = $$; + push(@HELD_LOCKS, $lock); + $LOCK_OWNER_PID //= $$; my $host = capture_command('uname', '-n') || 'unknown'; if (open my $fh, '>', "$lock/owner") { print {$fh} "host=$host\npid=$$\nepoch=" . time() . "\n"; @@ -1271,28 +1330,73 @@ sub acquire_output_lock { my $info = ''; if (open my $fh, '<', "$lock/owner") { local $/; $info = <$fh>; close $fh; } $info =~ s/\s+/ /g; - die "output $base is locked ($lock): $info\n" - . "another mockbuild-all run owns it; use a different --output or --force-unlock if stale.\n"; + die "$label $base is locked ($lock): $info\n" + . "another mockbuild-all run owns it; use a different destination or --force-unlock if stale.\n"; } die "Cannot create lock $lock: $!\n"; } +sub acquire_output_lock { + my ($base, $force) = @_; + acquire_named_lock($base, 'output', $force); +} + +sub acquire_repository_lock { + my ($base, $force) = @_; + _recover_common_repository($base) if $force; + acquire_named_lock($base, 'repository', $force); +} + +sub _recover_common_repository { + my ($base) = @_; + my $destination = "$base/common"; + my @backups = sort { + ((stat($a))[9] // 0) <=> ((stat($b))[9] // 0) + } grep { -d $_ && !-l $_ } bsd_glob("$base/.common.previous.*"); + + if (!-e $destination && !-l $destination && @backups) { + my $backup = pop(@backups); + rename($backup, $destination) + or die "Cannot restore interrupted common repository $backup: $!\n"; + } + remove_tree($_) for grep { -d $_ && !-l $_ } @backups; + + for my $staging (bsd_glob("$base/.common.*")) { + next if $staging =~ m{/\.common\.previous\.}; + remove_tree($staging) if -d $staging && !-l $staging; + } +} + sub _rmdir_lock { my ($lock) = @_; unlink "$lock/owner"; rmdir $lock; } -# Release the lock on any exit path (normal, die, or signal) -- but ONLY in the process that -# created it. Forked children (per-builder and per-target ForkManager workers) inherit -# $HELD_LOCK; without the pid guard their exit would delete the parent's lock mid-run. -sub _release_lock_if_owner { - return unless $HELD_LOCK && defined $LOCK_OWNER_PID && $$ == $LOCK_OWNER_PID; - _rmdir_lock($HELD_LOCK) if -d $HELD_LOCK; +# Release locks on every exit path, but only from the process that acquired them. +# Forked build workers inherit the lock list and must leave the parent's locks alone. +sub _restore_common_repository { + return unless defined($LOCK_OWNER_PID) && $$ == $LOCK_OWNER_PID; + remove_tree($COMMON_STAGE) + if $COMMON_STAGE && -d $COMMON_STAGE && !-l $COMMON_STAGE; + if ($COMMON_DESTINATION && $COMMON_BACKUP + && !-e $COMMON_DESTINATION && -d $COMMON_BACKUP) { + rename($COMMON_BACKUP, $COMMON_DESTINATION); + } elsif ($COMMON_DESTINATION && $COMMON_BACKUP + && -d $COMMON_DESTINATION && -d $COMMON_BACKUP) { + remove_tree($COMMON_BACKUP); + } } -END { _release_lock_if_owner(); } -for my $sig (qw(INT TERM HUP)) { - $SIG{$sig} = sub { _release_lock_if_owner(); exit 1; }; + +sub _release_locks_if_owner { + return unless defined($LOCK_OWNER_PID) && $$ == $LOCK_OWNER_PID; + for my $lock (reverse(@HELD_LOCKS)) { + _rmdir_lock($lock) if -d $lock; + } +} +END { + _restore_common_repository(); + _release_locks_if_owner(); } sub read_os_release { diff --git a/sbuild-all.pl b/sbuild-all.pl index 407cb82..be4cb45 100755 --- a/sbuild-all.pl +++ b/sbuild-all.pl @@ -98,6 +98,10 @@ my $gpg_key_id = 'xcat@megware.com'; my $gpg_home = ''; my $genesis_release = ''; # OpenEmbedded Genesis package release to publish alongside my $genesis_release_checksums; # its verified SHA256SUMS, read once at startup +# The OpenEmbedded Genesis debs are published ONCE, in a pool of their own that every suite indexes. +# They are Architecture:all and identical for all suites, so a per-suite copy would multiply hundreds +# of megabytes by the number of codenames for no gain. +my $GENESIS_POOL_RELATIVE = 'pool/main/xcat-genesis-openembedded'; my @genesis_debs; # native xcat-genesis-base- deb(s): path or URL (preferred) my $genesis_rpm = ''; # fallback: native-arch genesis rpm to convert my $genesis_rpm_ppc = ''; # fallback: cross-arch ppc genesis rpm to convert (amd64 host) @@ -290,6 +294,12 @@ if ($genesis_release ne '') { die "FATAL: Genesis release changed during verification\n" unless XCAT::BuildUtils::hashes_equal($before, $after); $genesis_release_checksums = $before; + # Every suite's Packages index points into the shared Genesis pool, and publishing a release + # replaces that pool -- so a run that rebuilt only some suites would leave the others indexing + # files that no longer exist. Publish a release for all of them or for none. + my @missing = grep { my $c = $_; !grep { $_ eq $c } @dist_list } known_codenames(); + die "FATAL: --genesis-release updates the shared Genesis pool every suite indexes, so it must\n" + . " cover all of them; --dists omits: @missing\n" if @missing; print_step('Genesis release'); print " $genesis_release (" . scalar(genesis_release_debs()) . " deb packages)\n"; } @@ -980,31 +990,38 @@ sub genesis_release_debs { keys %{ $genesis_release_checksums // {} }; } -# install_genesis_release_debs($pool, $flat): copy every release deb into this codename's pool (and -# the flat per-version dir), verifying each copy against the release checksums. Called with the -# publish lock held, between the pool wipe and apt-ftparchive, so the bytes indexed and signed are -# exactly the bytes verified here. +# install_genesis_release_debs($dir): (re)build the SHARED Genesis pool inside the side tree from +# the verified release, verifying each copy against the release checksums. Called once per publish, +# with the publish lock held and before any apt-ftparchive run, so the bytes indexed and signed are +# exactly the bytes verified here. Wiping first is what retires packages an earlier release left. # # A plain copy, never link(): the pool file must be its own inode. A hard link would leave the # published package and the verified release sharing one, where a write through either path silently # changes what the other holds. sub install_genesis_release_debs { - my ($pool, $flat) = @_; + my ($dir) = @_; my @files = genesis_release_debs(); die "FATAL: Genesis release has no deb packages\n" unless @files; + my $pool = "$dir/$GENESIS_POOL_RELATIVE"; + wipe_tree($pool); + make_path($pool); for my $relative (@files) { my $base = basename($relative); copy("$genesis_release/$relative", "$pool/$base") or die "FATAL: cannot install Genesis release package $relative -> $pool: $!\n"; + chmod(0644, "$pool/$base") + or die "FATAL: cannot set mode on $pool/$base: $!\n"; XCAT::GenesisRelease::verify_release_file($genesis_release_checksums, $relative, "$pool/$base"); - copy("$pool/$base", "$flat/$base") - or die "FATAL: cannot install Genesis release package $relative -> $flat: $!\n"; } return scalar(@files); } sub assemble_into { my ($dir, $expect) = @_; + if ($genesis_release ne '') { + my $n = install_genesis_release_debs($dir); + print " published + verified $n Genesis release package(s) into $GENESIS_POOL_RELATIVE\n"; + } for my $cn (@dist_list) { my $ver = codename_to_version($cn); my $pool = "$dir/pool/main/$cn"; @@ -1020,10 +1037,10 @@ sub assemble_into { # holds regardless of the --skip-genesis single-producer contract. my %best; # "name|arch" => { file => path, ver => version } for my $deb (glob("$staging/$cn/*/*.deb")) { - # With --genesis-release the release is the ONLY source of OpenEmbedded Genesis packages: - # drop anything staged under that name so a leftover from an earlier run cannot be - # published as if it had been verified. - next if $genesis_release ne '' && basename($deb) =~ /^xcat-genesis-openembedded-/; + # OpenEmbedded Genesis debs never belong to a suite pool -- they are published once + # into the shared pool. Drop anything staged under that name, so a leftover from an + # earlier run cannot be published as if it had come from a verified release. + next if basename($deb) =~ /^xcat-genesis-openembedded-/; my $name = deb_field($deb, 'Package'); my $darch = deb_field($deb, 'Architecture'); my $dver = deb_field($deb, 'Version'); @@ -1043,16 +1060,18 @@ sub assemble_into { my $b = basename($deb); link($deb, "$pool/$b") or copy($deb, "$pool/$b"); copy($deb, "$dir/$ver/$b"); - } - if ($genesis_release ne '') { - my $n = install_genesis_release_debs($pool, "$dir/$ver"); - print " installed + verified $n Genesis release package(s) into $cn\n"; + # Served by a web server running as somebody else -- do not inherit the builder umask. + chmod(0644, "$pool/$b", "$dir/$ver/$b"); } # Packages index per EXPECTED binary-: an arch's index carries that arch's debs + all # Architecture:all. Only the expected arches get an index -- writing a binary-ppc64el index # containing nothing but the Architecture:all debs would advertise a ppc64el repo that cannot # actually satisfy a ppc64el client (and would then be mistaken for "ppc was published"). + # This suite's own pool PLUS the shared Genesis pool: one Packages index, two sources, each + # deb's Filename staying relative to the repository root so clients fetch it where it is. my $all = `cd ${\ sh_quote($dir)} && apt-ftparchive packages pool/main/$cn`; + $all .= `cd ${\ sh_quote($dir)} && apt-ftparchive packages $GENESIS_POOL_RELATIVE` + if -d "$dir/$GENESIS_POOL_RELATIVE"; for my $a (@$expect) { my $bindir = "$dir/dists/$cn/main/binary-$a"; make_path($bindir); @@ -1096,6 +1115,7 @@ sub assemble_into { my $keysrc = "$repo_root/repomd.xml.key"; if (-f $keysrc) { copy($keysrc, "$dir/xcat-dep.asc"); } else { run("${g}gpg --armor --export " . sh_quote($gpg_key_id) . " > " . sh_quote("$dir/xcat-dep.asc"), nofail => 1); } + chmod(0644, "$dir/xcat-dep.asc") if -f "$dir/xcat-dep.asc"; } } @@ -1402,10 +1422,16 @@ and copies the verified bytes into every selected suite. The release must be B (every supported Genesis architecture) and must carry C packages. It is validated before any build or publish: its C is read, the shared verifier runs, and the checksums are read again -- a release rewritten together with its checksums -while the verifier runs is rejected. During publication each package is copied into the codename's -pool and re-checked against those checksums, with the publish lock held, so what is indexed and -signed is exactly what was verified. Anything staged under the OpenEmbedded Genesis package name is -dropped: the release is the only source of those packages. +while the verifier runs is rejected. + +The packages are published B, into F, and every suite's +C index points at that one copy: they are C and identical everywhere, +so a per-suite copy would multiply hundreds of megabytes by the number of codenames. Because every +suite indexes it, a release must be published for B of them -- a run whose C<--dists> omits a +suite is refused rather than leaving that suite indexing files the new release retired. Each package +is copied and re-checked against the release checksums with the publish lock held, so what is +indexed and signed is exactly what was verified, and anything staged under the OpenEmbedded Genesis +package name is dropped: the release is the only source of those packages. The OpenEmbedded packages carry their own names and install under F, so publishing them does not replace the diff --git a/t/genesis_openembedded_consumer.t b/t/genesis_openembedded_consumer.t index b62850d..6ff5af2 100644 --- a/t/genesis_openembedded_consumer.t +++ b/t/genesis_openembedded_consumer.t @@ -8,7 +8,9 @@ use File::Path qw(make_path); use Fcntl qw(:flock); use File::Temp qw(tempdir); use FindBin; +use POSIX (); use Test::More; +use Time::HiRes qw(sleep); use lib "$FindBin::Bin/../lib"; use lib "$FindBin::Bin/lib"; @@ -41,6 +43,10 @@ my $version = '2.19.0'; my $release = 'snap202608210726'; my $epoch = 1787293573; my $tmp = tempdir(CLEANUP => 1); +# every suite sbuild-all.pl knows; publishing a Genesis release must cover all of them +my @APT_SUITES = qw(focal jammy noble resolute); + +test_activation_helper(); if ($ENV{XCAT_GENESIS_CI}) { BAIL_OUT('CI requires Linux root') unless $^O eq 'linux' && $> == 0; @@ -50,21 +56,25 @@ if ($ENV{XCAT_GENESIS_CI}) { } SKIP: { - skip 'RPM repository tools require a root Linux builder', 28 + skip 'RPM repository tools require a root Linux builder', 69 unless $^O eq 'linux' && $> == 0 && command_exists('rpmbuild') + && command_exists('rpmsign') && command_exists('rpm') && command_exists('createrepo_c'); test_rpm_consumer(); + test_signed_common_rpm_repository(); test_legacy_rpm_consumer(); test_partial_rpm_release(); test_failed_build_release(); test_dry_run_release(); + test_rpm_repository_lock(); + test_rpm_signal_cleanup(); } SKIP: { - skip 'APT repository tools are not installed', 24 + skip 'APT repository tools are not installed', 58 unless $^O eq 'linux' && command_exists('dpkg-deb') && command_exists('apt-ftparchive'); @@ -79,17 +89,39 @@ done_testing(); sub test_rpm_consumer { my $release_root = make_package_release("$tmp/rpm", 'rpm'); my $package = "xCAT-genesis-openembedded-x86_64-$version-$release.noarch.rpm"; - my $source_package = "xCAT-genesis-openembedded-x86_64-$version-$release.src.rpm"; my $output = "$tmp/rpm output"; my $target = 'test+epel-10-' . capture_command('uname', '-m'); my $run = "$target-consumer"; my $run_repo = "$output/mockbuild-all/$run/repo/" . capture_command('uname', '-m'); my $source_repo = "$output/mockbuild-all/$run/repo-src"; my $deploy_repo = "$output/xcat-dep/rh10/" . capture_command('uname', '-m'); - make_path($run_repo, $source_repo, $deploy_repo); + my $common_repo = "$output/xcat-dep/common"; + make_path($run_repo, $source_repo, $deploy_repo, $common_repo); write_binary("$run_repo/xCAT-genesis-openembedded-stale.noarch.rpm", 'stale'); write_binary("$source_repo/xCAT-genesis-openembedded-stale.src.rpm", 'stale'); write_binary("$deploy_repo/xCAT-genesis-openembedded-stale.noarch.rpm", 'stale'); + write_binary("$common_repo/xCAT-genesis-openembedded-stale.noarch.rpm", 'stale'); + write_binary("$common_repo/xCAT-genesis-openembedded-stale.src.rpm", 'stale'); + + my $rpm_scripts = capture_command( + 'rpm', '-qp', '--scripts', "$release_root/rpm/$package" + ); + like($rpm_scripts, qr{genesis-openembedded-activate-x86_64 x86_64}, + 'the RPM refreshes its architecture after a package transaction'); + like($rpm_scripts, qr{mknb x86_64 --remove-openembedded}, + 'the RPM removes published artifacts when the image is erased'); + like($rpm_scripts, qr{/proc/1/root}, + 'the RPM removal path refuses to operate from a chroot'); + like( + capture_command('rpm', '-qpl', "$release_root/rpm/$package"), + qr{/usr/libexec/xcat/genesis-openembedded-activate-x86_64$}m, + 'the RPM contains its architecture-specific activation helper', + ); + like( + capture_command('rpm', '-qpl', "$release_root/rpm/$package"), + qr{^/usr/libexec/xcat/?$}m, + 'the RPM owns its private helper directory', + ); my $dependencies = "$tmp/rpm-dependencies"; my $scratch_repo_root = "$tmp/rpm-repo-root"; @@ -126,14 +158,19 @@ sub test_rpm_consumer { ); is($status, 0, 'RPM repository accepts a verified Genesis release'); - is(digest_file("$deploy_repo/$package"), digest_file("$release_root/rpm/$package"), + is( + sprintf('%04o', (stat($common_repo))[2] & 0x0fff), + '0755', + 'the common repository is traversable by an unprivileged server', + ); + is(digest_file("$common_repo/$package"), digest_file("$release_root/rpm/$package"), 'deployed RPM matches the release'); is( - sprintf('%04o', (stat("$deploy_repo/$package"))[2] & 0x0fff), + sprintf('%04o', (stat("$common_repo/$package"))[2] & 0x0fff), sprintf('%04o', (stat("$release_root/rpm/$package"))[2] & 0x0fff), 'deployed RPM keeps the release file mode', ); - opendir(my $deploy_dh, $deploy_repo) or die $!; + opendir(my $deploy_dh, $common_repo) or die $!; my @staging_files = grep { /^\.xcat-deploy\./ } readdir($deploy_dh); closedir($deploy_dh) or die $!; is_deeply(\@staging_files, [], 'RPM deployment leaves no staging files'); @@ -143,30 +180,108 @@ sub test_rpm_consumer { 'stale source RPM is removed'); ok(!-e "$deploy_repo/xCAT-genesis-openembedded-stale.noarch.rpm", 'stale deployed RPM is removed'); - is(digest_file("$source_repo/$source_package"), - digest_file("$release_root/srpm/$source_package"), - 'source RPM matches the release'); - ok(-f "$deploy_repo/repodata/repomd.xml", 'RPM repository metadata is generated'); + ok(-f "$common_repo/repodata/repomd.xml", 'RPM repository metadata is generated'); ok(-f "$deploy_repo/xCAT-genesis-base-x86_64-1.noarch.rpm", 'legacy Genesis package remains available'); - ok(!-e "$run_repo/xCAT-genesis-openembedded-x86_64-$version-old.noarch.rpm", - 'stale OpenEmbedded RPM is not collected'); - ok(!-e "$source_repo/xCAT-genesis-openembedded-x86_64-$version-old.src.rpm", - 'stale OpenEmbedded SRPM is not collected'); - like(read_binary("$output/mockbuild-all/$run/summary.txt"), qr/^copied_rpms=15$/m, - 'release RPM is counted alongside required dependencies'); + my @expected_packages = sort map { + rpm_package_name($_) . "-$version-$release.noarch.rpm" + } architectures(); + my @common_packages = genesis_rpm_names($common_repo); + is_deeply(\@common_packages, \@expected_packages, + 'common repository contains one complete Genesis release'); + is_deeply( + [ genesis_rpm_names($deploy_repo) ], + [], + 'per-EL repository contains no OpenEmbedded Genesis packages', + ); + is_deeply( + [ genesis_rpm_names($run_repo) ], + [], + 'target staging repository contains no OpenEmbedded Genesis packages', + ); + is_deeply( + [ genesis_rpm_names($source_repo) ], + [], + 'target source repository contains no OpenEmbedded Genesis packages', + ); + my $common_config = read_binary("$common_repo/xcat-dep-common.repo"); + like($common_config, qr/^\[xcat-dep-common\]$/m, + 'common repository has its own repository ID'); + like($common_config, qr{/xcat-dep/common$}m, + 'common repository configuration uses the shared URL'); + like($common_config, qr/^skip_if_unavailable=1$/m, + 'the published common repository stays optional during outages'); + like($common_config, qr/^repo_gpgcheck=0$/m, + 'unsigned test metadata is declared explicitly'); + ok(-x "$common_repo/mklocalrepo.sh", 'common repository supports offline setup'); + my $local_repo_helper = read_binary("$common_repo/mklocalrepo.sh"); + unlike($local_repo_helper, qr/\[\[/, + 'the offline helper does not require Bash conditionals'); + unlike($local_repo_helper, qr/`/, + 'the offline helper uses POSIX command substitution'); + like(read_binary("$common_repo/buildinfo.txt"), qr/^TARGET=common$/m, + 'common repository records its target'); + like(read_binary("$output/mockbuild-all/$run/summary.txt"), qr/^copied_rpms=8$/m, + 'repository summary counts the collected dependencies'); + + my $retained = "$common_repo/xCAT-genesis-openembedded-retained.noarch.rpm"; + my $repomd = "$common_repo/repodata/repomd.xml"; + my $repomd_before = digest_file($repomd); + write_binary($retained, 'previous complete release'); + my $fail_bin = "$tmp/rpm-fail-bin"; + make_path($fail_bin); + write_binary( + "$fail_bin/createrepo_c", + <<'SH', +#!/bin/sh +for argument in "$@"; do last=$argument; done +case "$last" in + */common|*/.common.*) exit 1 ;; +esac +exec "$XCAT_TEST_CREATEREPO" "$@" +SH + ); + chmod(0755, "$fail_bin/createrepo_c") or die $!; + local $ENV{XCAT_TEST_CREATEREPO} = capture_command( + 'sh', '-c', 'command -v createrepo_c' + ); + local $ENV{PATH} = "$fail_bin:$ENV{PATH}"; + my $failed_log = "$tmp/rpm-publication-failure.log"; + my $failed_status = run_capture( + $failed_log, + $^X, $rpm_consumer, + '--repo-root', $scratch_repo_root, + '--output', $output, + '--target', $target, + '--run-id', 'publication-failure', + '--build-timestamp', $epoch, + '--skip-build', '--skip-xcat', '--skip-xcat-dep', '--skip-perl', + '--skip-createrepo', '--skip-tarball', + '--genesis-release', $release_root, + '--collect-dir', $dependencies, + ); + isnt($failed_status, 0, 'a failed RPM metadata build aborts publication'); + ok(-f $retained, 'a failed RPM publication keeps the previous package set'); + is(digest_file($repomd), $repomd_before, + 'a failed RPM publication keeps the previous metadata'); } -# The apt consumer is sbuild-all.pl's PUBLISH phase: it stages nothing here, it assembles the -# already-staged debs plus the Genesis release into a side tree, gates it, and swaps it in. Every run -# below is therefore a publish-only run (--skip-build) over a hand-made staging tree, with the -# manifest reduced to the one package --skip-genesis drops so the completeness check has nothing to -# demand of a fixture. +# The apt consumer is sbuild-all.pl's PUBLISH phase: it stages nothing here -- it assembles the +# already-staged debs plus the Genesis release into a side tree, gates it, and swaps that tree onto +# the published repository with a single rename. Every run below is therefore a publish-only run +# (--skip-build) over a hand-made staging tree, with the manifest reduced to the one package +# --skip-genesis drops, so the completeness gate has nothing to demand of a fixture. + sub run_apt_consumer { my (%args) = @_; - my $manifest = "$args{output}/manifest.conf"; - make_path($args{output}); - write_binary($manifest, "[noble-amd64]\nxcat-genesis-base=*\n"); + my @dists = @{ $args{dists} // \@APT_SUITES }; + my $manifest = $args{manifest}; + unless ($manifest) { + $manifest = "$args{output}/manifest.conf"; + make_path($args{output}); + write_binary($manifest, + join('', map { "[$_-amd64]\nxcat-genesis-base=*\n" } @APT_SUITES)); + } return run_capture( $args{log}, $^X, $deb_consumer, @@ -174,15 +289,17 @@ sub run_apt_consumer { '--output-root', $args{output}, '--apt-dir', $args{apt_dir}, '--manifest', $manifest, - '--dists', 'noble', + '--dists', join(' ', @dists), '--arch', 'amd64', '--skip-build', '--skip-genesis', '--skip-tarball', '--publish', '--expect-arch', 'amd64 ppc64el', - '--no-verify-repo', + ($args{verify} ? () : ('--no-verify-repo')), @{ $args{extra} // [] }, ); } +# Stage one legacy genesis-base deb for the noble suite only -- enough for the runs that publish +# no Genesis release. sub stage_legacy_deb { my ($root, $output) = @_; my $staging = "$output/staging/noble/amd64"; @@ -191,15 +308,43 @@ sub stage_legacy_deb { return $staging; } +# Stage one legacy genesis-base deb for each suite, standing in for a completed build. +sub stage_apt_suites { + my ($output, $package_root, $content) = @_; + my $package = "$package_root.deb"; + make_legacy_deb($package_root, $package, $content); + for my $codename (@APT_SUITES) { + my $staged = "$output/staging/$codename/amd64"; + make_path($staged); + copy($package, "$staged/xcat-genesis-base-amd64_1_all.deb") or die $!; + } + return "$output/staging/noble/amd64"; +} + sub test_deb_consumer { my $release_root = make_package_release("$tmp/deb", 'deb'); my $package = "xcat-genesis-openembedded-x86-64_${version}-${release}_all.deb"; my $apt_root = "$tmp/apt"; my $output = "$tmp/deb-output"; - my $staging = stage_legacy_deb("$tmp/dummy-deb", $output); - write_binary("$staging/xcat-genesis-openembedded-stale.deb", 'stale'); - make_path("$apt_root/pool/main/noble"); - write_binary("$apt_root/pool/main/noble/xcat-genesis-openembedded-old.deb", 'stale'); + my $staged = stage_apt_suites($output, "$tmp/dummy-deb"); + my $shared_pool = "$apt_root/pool/main/xcat-genesis-openembedded"; + make_path($shared_pool); + write_binary("$staged/xcat-genesis-openembedded-stale.deb", 'stale'); + write_binary("$shared_pool/xcat-genesis-openembedded-old.deb", 'stale'); + + my $postinst = capture_command( + 'dpkg-deb', '--info', "$release_root/deb/$package", 'postinst' + ); + isnt($postinst, '', 'the DEB includes a post-installation script'); + like($postinst, qr{genesis-openembedded-activate-x86_64 x86_64}, + 'the DEB refreshes its architecture after configuration'); + my $postrm = capture_command( + 'dpkg-deb', '--info', "$release_root/deb/$package", 'postrm' + ); + like($postrm, qr{mknb x86_64 --remove-openembedded}, + 'the DEB removes published artifacts when the image is erased'); + like($postrm, qr{/proc/1/root}, + 'the DEB removal path refuses to operate from a chroot'); local $ENV{SOURCE_DATE_EPOCH} = $epoch; my $log = "$tmp/deb-consumer.log"; @@ -207,52 +352,215 @@ sub test_deb_consumer { log => $log, output => $output, apt_dir => $apt_root, extra => [ '--genesis-release', $release_root ], ); - my $pool_package = "$apt_root/pool/main/noble/$package"; + my $pool_package = "$shared_pool/$package"; my $amd64 = "$apt_root/dists/noble/main/binary-amd64/Packages"; - my $ppc64el = "$apt_root/dists/noble/main/binary-ppc64el/Packages"; is($status, 0, 'APT repository accepts a verified Genesis release'); - like(read_binary($log), qr/installed \+ verified 7 Genesis release package\(s\) into noble/, - 'APT repository verifies every copied package as it pools it'); is(digest_file($pool_package), digest_file("$release_root/deb/$package"), 'pooled DEB matches the release'); - is(digest_file("$apt_root/ubuntu24.04/$package"), digest_file("$release_root/deb/$package"), - 'the flat per-version directory carries the release too'); - ok(!-e "$apt_root/pool/main/noble/xcat-genesis-openembedded-old.deb", + is( + sprintf('%04o', (stat($pool_package))[2] & 0x0fff), + '0644', + 'the pooled DEB is readable by an unprivileged server', + ); + ok(!-e "$shared_pool/xcat-genesis-openembedded-old.deb", 'stale pooled DEB is removed'); - like(read_binary($amd64), qr/^Package: xcat-genesis-openembedded-x86-64$/m, - 'all-architecture DEB is indexed for amd64'); - like(read_binary($ppc64el), qr/^Package: xcat-genesis-openembedded-x86-64$/m, - 'all-architecture DEB is indexed for ppc64el'); - ok(-f "$apt_root/dists/noble/Release", 'APT Release metadata is generated'); + my @expected_packages = sort map { + deb_package_name($_) . "_${version}-${release}_all.deb" + } architectures(); + is_deeply([ genesis_deb_names($shared_pool) ], \@expected_packages, + 'shared APT pool contains one complete Genesis release'); + my @suite_packages; + for my $codename (@APT_SUITES) { + push(@suite_packages, map { "$codename/$_" } + genesis_deb_names("$apt_root/pool/main/$codename")); + } + is_deeply(\@suite_packages, [], 'suite pools contain no OpenEmbedded Genesis packages'); + for my $codename (@APT_SUITES) { + for my $architecture (qw(amd64 ppc64el)) { + my $packages = read_binary( + "$apt_root/dists/$codename/main/binary-$architecture/Packages" + ); + like($packages, + qr{^Filename: pool/main/xcat-genesis-openembedded/\Q$package\E$}m, + "$codename $architecture index uses the shared Genesis package", + ); + } + } + is_deeply( + [ grep { !-f "$apt_root/dists/$_/Release" } @APT_SUITES ], + [], + 'APT Release metadata is generated for every suite', + ); like(read_binary($amd64), qr/^Package: xcat-genesis-base-amd64$/m, 'legacy Genesis DEB remains available'); ok(!-e "$apt_root/pool/main/noble/xcat-genesis-openembedded-stale.deb", - 'stale OpenEmbedded DEB is not collected'); + 'a staged OpenEmbedded DEB is not published into a suite pool'); # The published package must be a file of its own: sharing an inode with the release # would make a later write through either path change what the other one holds. my @pooled = stat($pool_package); my @released = stat("$release_root/deb/$package"); isnt("$pooled[0]:$pooled[1]", "$released[0]:$released[1]", 'pooled DEB is published independently of the release file'); + my $publication_log = read_binary($log); + my $verified_at = index($publication_log, 'published + verified'); + my $indexed_at = index($publication_log, 'assembled + '); + ok($verified_at >= 0 && $verified_at < $indexed_at, + 'staged DEBs are verified before any suite index is generated'); - # A staged package under the release's own name is dropped, not published: with - # --genesis-release the verified release is the ONLY source of those packages. + # A release rewrites the shared pool every suite indexes, so it cannot be published for + # some suites only -- the others would be left indexing files the new release retired. + my $before_subset = digest_file($pool_package); + my $subset_log = "$tmp/deb-subset.log"; + my $subset_status = run_apt_consumer( + log => $subset_log, output => $output, apt_dir => $apt_root, + dists => ['noble'], + extra => [ '--genesis-release', $release_root ], + ); + isnt($subset_status, 0, 'Genesis publication rejects a partial suite update'); + like(read_binary($subset_log), qr/must\n?\s*cover all of them/, + 'partial suite failure explains the consistency requirement'); + is(digest_file($pool_package), $before_subset, + 'partial suite failure leaves the shared package unchanged'); + + # A later suite rebuild carries no release and must keep indexing the shared pool. + my $refresh_log = "$tmp/deb-refresh.log"; + my $refresh_status = run_apt_consumer( + log => $refresh_log, output => $output, apt_dir => $apt_root, + dists => ['noble'], + ); + is($refresh_status, 0, 'a suite refresh reuses the shared Genesis pool'); + is(digest_file($pool_package), $before_subset, + 'a suite refresh leaves the shared package unchanged'); + like(read_binary($amd64), + qr{^Filename: pool/main/xcat-genesis-openembedded/\Q$package\E$}m, + 'the refreshed suite still indexes the shared Genesis package'); + + # The repository changes with ONE rename, at the very end: a publication that fails while it + # assembles must leave every published byte -- packages, indexes and key -- exactly as it was. + # Signing with a key the keyring does not hold fails inside that phase. + my $package_before = digest_file( + "$apt_root/pool/main/noble/xcat-genesis-base-amd64_1_all.deb"); + my $metadata_before = digest_file($amd64); + my $empty_keyring = "$tmp/apt-empty-keyring"; + make_path($empty_keyring); + chmod(0700, $empty_keyring) or die $!; + my $failed_log = "$tmp/deb-failed-publish.log"; + my $failed_status = run_apt_consumer( + log => $failed_log, output => $output, apt_dir => $apt_root, + dists => ['noble'], + extra => [ '--gpg-sign', '--gpg-key-id', 'absent@example.invalid', + '--gpg-home', $empty_keyring ], + ); + isnt($failed_status, 0, 'a publication that cannot be signed is not published'); + like(read_binary($failed_log), qr/NOT publishing/, + 'the refusal says the published repository was left alone'); + is(digest_file("$apt_root/pool/main/noble/xcat-genesis-base-amd64_1_all.deb"), + $package_before, 'a failed publication leaves the published package bytes'); + is(digest_file($amd64), $metadata_before, + 'a failed publication leaves the published package index'); + is(digest_file($pool_package), $before_subset, + 'a failed publication leaves the shared Genesis pool'); + + # A rebuilt deb keeps its filename; the published copy must become the new one. + my $legacy_pool = "$apt_root/pool/main/noble/xcat-genesis-base-amd64_1_all.deb"; + my $legacy_before = digest_file($legacy_pool); + make_legacy_deb( + "$tmp/rebuilt-legacy-deb", + "$staged/xcat-genesis-base-amd64_1_all.deb", + 'rebuilt repository test package', + ); + my $replace_log = "$tmp/deb-replace.log"; + my $replace_status = run_apt_consumer( + log => $replace_log, output => $output, apt_dir => $apt_root, + dists => ['noble'], + ); + is($replace_status, 0, 'a rebuilt DEB may keep its published filename'); + isnt(digest_file($legacy_pool), $legacy_before, + 'the rebuilt DEB replaces the earlier package'); + + # A staged package under a release package's own name is dropped, not published. my $collision = "$tmp/apt-collision"; my $collision_output = "$tmp/deb-collision-output"; - my $collision_staging = stage_legacy_deb("$tmp/collision-deb", $collision_output); - write_binary("$collision_staging/$package", 'different'); + my $collision_staged = stage_apt_suites($collision_output, "$tmp/collision-deb"); + write_binary("$collision_staged/$package", 'different'); my $collision_log = "$tmp/deb-collision.log"; my $collision_status = run_apt_consumer( log => $collision_log, output => $collision_output, apt_dir => $collision, extra => [ '--genesis-release', $release_root ], ); is($collision_status, 0, 'verified release replaces a stale staged package'); - is(digest_file("$collision/pool/main/noble/$package"), + is(digest_file("$collision/pool/main/xcat-genesis-openembedded/$package"), digest_file("$release_root/deb/$package"), 'pooled package still matches the verified release'); } +sub test_signed_common_rpm_repository { + my $release_root = make_package_release("$tmp/rpm-signed", 'rpm'); + my $package = "xCAT-genesis-openembedded-x86_64-$version-$release.noarch.rpm"; + my $output = "$tmp/rpm-signed-output"; + my $target = 'test+epel-10-' . capture_command('uname', '-m'); + my $dependencies = "$tmp/rpm-signed-dependencies"; + my $scratch_repo_root = "$tmp/rpm-signed-repo-root"; + my $common = "$output/xcat-dep/common"; + my $gpg_home = "$tmp/rpm-signing-key"; + my $identity = 'xCAT repository test '; + make_rpm_dependencies($dependencies, "$release_root/rpm/$package"); + make_path($scratch_repo_root, $gpg_home); + chmod(0700, $gpg_home) or die $!; + system( + 'gpg', '--batch', '--homedir', $gpg_home, '--passphrase', '', + '--quick-generate-key', $identity, 'rsa2048', 'sign', '0', + ) == 0 or die "could not create the repository test key"; + + my @perl_lib; + push(@perl_lib, write_forkmanager_stub("$tmp/perl-signed-stub")) + unless eval { require Parallel::ForkManager; 1 }; + push(@perl_lib, $ENV{PERL5LIB}) + if defined($ENV{PERL5LIB}) && $ENV{PERL5LIB} ne ''; + local $ENV{PERL5LIB} = join(':', @perl_lib); + + my $log = "$tmp/rpm-signed-consumer.log"; + my $status = run_capture( + $log, + $^X, $rpm_consumer, + '--repo-root', $scratch_repo_root, + '--xcat-source', $repo_root, + '--output', $output, + '--target', $target, + '--run-id', 'signed-consumer', + '--build-timestamp', $epoch, + '--skip-build', '--skip-xcat', '--skip-xcat-dep', '--skip-perl', + '--skip-genesis', '--skip-createrepo', '--skip-tarball', + '--collect-dir', $dependencies, + '--genesis-release', $release_root, + '--gpg-sign', '--gpg-key-name', $identity, '--gpg-home', $gpg_home, + ); + + diag(read_binary($log)) if $status; + is($status, 0, 'the common RPM repository can be signed'); + ok(-f "$common/repodata/repomd.xml.key", + 'the common repository exports its own public key'); + is( + system( + 'gpg', '--batch', '--homedir', $gpg_home, '--verify', + "$common/repodata/repomd.xml.asc", + "$common/repodata/repomd.xml", + ) >> 8, + 0, + 'the common repository metadata signature verifies', + ); + like( + read_binary("$common/xcat-dep-common.repo"), + qr{^gpgkey=https://xcat\.org/files/xcat/repos/yum/devel/xcat-dep/common/repodata/repomd\.xml\.key$}m, + 'the common repository file points to its own exported key', + ); + like(read_binary("$common/xcat-dep-common.repo"), qr/^repo_gpgcheck=1$/m, + 'the signed common repository requires metadata verification'); + like(read_binary("$common/xcat-dep-common.repo"), qr/^skip_if_unavailable=1$/m, + 'the signed common repository remains optional during outages'); +} + sub test_legacy_rpm_consumer { my $release_root = make_package_release("$tmp/rpm-legacy", 'rpm', 'x86_64'); my $package = "xCAT-genesis-openembedded-x86_64-$version-$release.noarch.rpm"; @@ -293,6 +601,8 @@ sub test_legacy_rpm_consumer { [], 'RPM legacy path does not add OpenEmbedded packages', ); + ok(!-d "$output/xcat-dep/common", + 'RPM legacy path does not create the common repository'); } sub test_legacy_deb_consumer { @@ -316,9 +626,9 @@ sub test_partial_rpm_release { my $release_root = make_package_release("$tmp/rpm-partial", 'rpm', 'x86_64'); my $output = "$tmp/partial-output"; my $target = 'test+epel-10-' . capture_command('uname', '-m'); - my $deployed = "$output/xcat-dep/rh10/" . capture_command('uname', '-m'); - my $existing = "$deployed/xCAT-genesis-base-existing.noarch.rpm"; - make_path($deployed); + my $common = "$output/xcat-dep/common"; + my $existing = "$common/xCAT-genesis-openembedded-existing.noarch.rpm"; + make_path($common); write_binary($existing, 'existing release'); my $log = "$tmp/rpm-partial.log"; @@ -368,14 +678,14 @@ sub test_failed_build_release { my $package = "xCAT-genesis-openembedded-x86_64-$version-$release.noarch.rpm"; my $output = "$tmp/empty-output"; my $target = 'test+epel-10-' . capture_command('uname', '-m'); - my $deployed = "$output/xcat-dep/rh10/" . capture_command('uname', '-m'); + my $common = "$output/xcat-dep/common"; my $scratch_repo_root = "$tmp/rpm-empty-root"; my $collected = "$tmp/rpm-empty-collect"; my $run_repo = "$output/mockbuild-all/$target-empty/repo/" . capture_command('uname', '-m'); my $results = "$output/mockbuild-all/$target-empty/build-results/ipmitool-xcat"; my $stale = "$run_repo/ipmitool-xcat-0-stale.noarch.rpm"; my $kept = "$results/ipmitool-xcat-0-earlier.noarch.rpm"; - make_path($deployed, $scratch_repo_root, $collected, $run_repo, $results); + make_path($common, $scratch_repo_root, $collected, $run_repo, $results); write_binary($stale, 'package left by an earlier run'); write_binary($kept, 'build output an earlier run produced'); @@ -397,7 +707,7 @@ sub test_failed_build_release { isnt($status, 0, 'a run that built nothing fails even with a Genesis release'); like(read_binary($log), qr/No binary RPMs were collected/, 'the failure names the empty collection, not the missing dependencies'); - ok(!-e "$deployed/$package", + ok(!-e "$common/$package", 'a run that built nothing publishes no release package'); ok(!-e $stale, 'a package left by an earlier run is cleared from the staging repository'); @@ -434,16 +744,113 @@ sub test_dry_run_release { my $printed = read_binary($log); is($status, 0, 'a dry run accepts a verified Genesis release'); - like($printed, qr/^DRY-RUN install Genesis release package: .*\Q$package\E$/m, - 'the dry run reports the release packages it would install'); - like($printed, qr/^Collected binary RPMs: 15$/m, - 'the dry run counts the release packages a real run installs'); + like($printed, + qr{^DRY-RUN publish Genesis release package: .*\Q$package\E -> .*/xcat-dep/common/\Q$package\E$}m, + 'the dry run reports the shared package destination'); + like($printed, qr/^Collected binary RPMs: 8$/m, + 'the dry run counts the packages collected for the target repository'); ok(!-e "$run_repo/$package", 'the dry run installs nothing'); } -# The apt tree is published under sbuild-all.pl's own publish lock (flock, one global holder), not -# the mkdir lock the shell publisher used. A run that cannot take it must publish NOTHING rather than -# assemble into a tree another publisher owns. +sub test_rpm_repository_lock { + my $output = "$tmp/rpm-lock-output"; + my $repository = "$tmp/rpm-shared-repository"; + make_path("$repository/.lock"); + write_binary("$repository/.lock/owner", "host=other\npid=1\nepoch=1\n"); + + my @perl_lib; + push(@perl_lib, write_forkmanager_stub("$tmp/perl-lock-stub")) + unless eval { require Parallel::ForkManager; 1 }; + push(@perl_lib, $ENV{PERL5LIB}) + if defined($ENV{PERL5LIB}) && $ENV{PERL5LIB} ne ''; + local $ENV{PERL5LIB} = join(':', @perl_lib); + + my $log = "$tmp/rpm-repository-lock.log"; + my $status = run_capture( + $log, + $^X, $rpm_consumer, + '--repo-root', $repo_root, + '--output', $output, + '--repo-dep', $repository, + '--target', 'test+epel-10-' . capture_command('uname', '-m'), + '--skip-build', '--skip-xcat', '--skip-xcat-dep', '--skip-perl', + '--skip-createrepo', '--skip-tarball', '--dry-run', + ); + isnt($status, 0, 'a shared RPM repository cannot have two publishers'); + like(read_binary($log), qr/repository \Q$repository\E is locked/, + 'the lock failure names the shared repository'); + + my $backup = "$repository/.common.previous.999"; + my $staging = "$repository/.common.abandoned"; + make_path($backup, $staging); + write_binary("$backup/marker", "previous repository\n"); + my $forced_log = "$tmp/rpm-repository-force.log"; + my $forced_status = run_capture( + $forced_log, + $^X, $rpm_consumer, + '--repo-root', $repo_root, + '--output', "$tmp/rpm-force-output", + '--repo-dep', $repository, + '--target', 'test+epel-10-' . capture_command('uname', '-m'), + '--skip-build', '--skip-xcat', '--skip-xcat-dep', '--skip-perl', + '--skip-createrepo', '--skip-tarball', '--dry-run', '--force-unlock', + ); + is($forced_status, 0, '--force-unlock recovers an interrupted RPM publication'); + ok(-f "$repository/common/marker", + 'the interrupted common repository is restored before publication'); + ok(!-d $staging, 'abandoned common repository staging is removed'); +} + +sub test_rpm_signal_cleanup { + my $output = "$tmp/rpm-signal-output"; + my $repository = "$tmp/rpm-signal-repository"; + my $signal_bin = "$tmp/rpm-signal-bin"; + my $log = "$tmp/rpm-signal.log"; + make_path($repository, $signal_bin); + write_binary( + "$signal_bin/uname", + "#!/bin/sh\n" + . "if [ \"\$1\" = -m ]; then sleep 60; exit 1; fi\n" + . "exec /usr/bin/uname \"\$@\"\n", + ); + chmod(0755, "$signal_bin/uname") or die $!; + + my $pid = fork(); + die "Cannot fork signal test: $!" unless defined($pid); + if ($pid == 0) { + POSIX::setpgid(0, 0); + local $ENV{PATH} = "$signal_bin:$ENV{PATH}"; + open(STDOUT, '>', $log) or die "open $log: $!"; + open(STDERR, '>&', fileno(STDOUT)) or die "redirect stderr: $!"; + exec( + $^X, $rpm_consumer, + '--repo-root', $repo_root, + '--xcat-source', $repo_root, + '--output', $output, + '--repo-dep', $repository, + '--target', 'test+epel-10-x86_64', + '--skip-build', '--skip-xcat', '--skip-xcat-dep', '--skip-perl', + '--skip-createrepo', '--skip-tarball', '--dry-run', + ); + exit 127; + } + + my $locked = 0; + for (1 .. 200) { + if (-d "$output/.lock" && -d "$repository/.lock") { + $locked = 1; + last; + } + sleep(0.05); + } + ok($locked, 'the signal test reaches the locked publication phase'); + kill('TERM', -$pid); + waitpid($pid, 0); + is($? >> 8, 1, 'SIGTERM follows the publisher cleanup exit path'); + ok(!-d "$output/.lock", 'SIGTERM releases the output lock'); + ok(!-d "$repository/.lock", 'SIGTERM releases the repository lock'); +} + sub test_publish_lock { my $apt_root = "$tmp/apt-lock"; my $output = "$tmp/deb-lock-output"; @@ -466,11 +873,65 @@ sub test_publish_lock { close($held); + # sbuild-all.pl never publishes in place: it assembles a COMPLETE side tree and renames it onto + # the repository, so there is no half-written state to recover and no per-file backup to restore. + # A side tree left by a run that died is simply inert -- the next run builds and publishes its own. + my $abandoned = "$apt_root.publish-abandoned.999"; + make_path("$abandoned/dists/noble"); + write_binary("$abandoned/dists/noble/Release", "abandoned\n"); + my $freed_log = "$tmp/deb-freed.log"; my $freed_status = run_apt_consumer( - log => $freed_log, output => $output, apt_dir => $apt_root); + log => $freed_log, output => $output, apt_dir => $apt_root, dists => ['noble']); is($freed_status, 0, 'the publish runs once the lock is released'); ok(-f "$apt_root/dists/noble/Release", 'the released lock lets the tree be published'); + isnt(read_binary("$apt_root/dists/noble/Release"), "abandoned\n", + 'a side tree abandoned by a dead run is never published'); +} + +sub test_activation_helper { + my $activation = read_binary("$repo_root/genesis-openembedded/activate"); + unlike($activation, qr/XCAT_GENESIS_ROOT/, + 'the root package helper has no environment-controlled execution root'); + $activation =~ s/\ngenesis_activation_main "\$\@"\s*\z/\n/ + or BAIL_OUT('the activation helper has no reusable main boundary'); + + my $driver = "$tmp/activation-driver"; + my $log = "$tmp/activation.log"; + my $output = "$tmp/activation-output.log"; + write_binary( + $driver, + $activation + . <<'SH', +genesis_is_host_root() { return 0; } +genesis_mknb_exists() { return 0; } +genesis_is_service_node() { [ "${XCAT_TEST_SERVICE_NODE:-0}" = 1 ]; } +genesis_uses_shared_tftp() { [ "${XCAT_TEST_SHAREDTFTP:-0}" = 1 ]; } +genesis_run_mknb() { + printf '%s\n' "$1" >>"$XCAT_TEST_LOG" + return "${XCAT_TEST_MKNB_STATUS:-0}" +} +genesis_activation_main "$@" +SH + ); + chmod(0755, $driver) or die "Cannot make activation driver executable: $!"; + + local $ENV{XCAT_TEST_LOG} = $log; + my $status = run_capture($output, $driver, 'x86_64'); + is($status, 0, 'the activation helper accepts a local-TFTP node'); + is(read_binary($log), "x86_64\n", 'the activation helper runs mknb for one architecture'); + + write_binary($log, ''); + local $ENV{XCAT_TEST_SERVICE_NODE} = 1; + local $ENV{XCAT_TEST_SHAREDTFTP} = 1; + $status = run_capture($output, $driver, 'ppc64le'); + is($status, 0, 'a shared-TFTP service node is accepted'); + is(read_binary($log), '', 'a shared-TFTP service node does not rebuild Genesis'); + + local $ENV{XCAT_TEST_SERVICE_NODE} = 0; + local $ENV{XCAT_TEST_MKNB_STATUS} = 1; + $status = run_capture($output, $driver, 'ppc64le'); + is($status, 0, 'an mknb failure does not fail the package transaction'); } sub make_package_release { @@ -522,6 +983,24 @@ sub make_package_release { return $release_root; } +sub genesis_rpm_names { + my ($directory) = @_; + return () unless -d $directory; + opendir(my $dh, $directory) or die $!; + my @names = sort grep { /^xCAT-genesis-openembedded-.*\.rpm$/ } readdir($dh); + closedir($dh) or die $!; + return @names; +} + +sub genesis_deb_names { + my ($directory) = @_; + return () unless -d $directory; + opendir(my $dh, $directory) or die $!; + my @names = sort grep { /^xcat-genesis-openembedded-.*\.deb$/ } readdir($dh); + closedir($dh) or die $!; + return @names; +} + sub make_rpm_dependencies { my ($directory, $package) = @_; make_path($directory); @@ -535,16 +1014,29 @@ sub make_rpm_dependencies { } sub make_legacy_deb { - my ($root, $output) = @_; + my ($root, $output, $description) = @_; + $description //= 'repository test package'; make_path("$root/DEBIAN"); write_binary( "$root/DEBIAN/control", "Package: xcat-genesis-base-amd64\nVersion: 1\nArchitecture: all\n" . "Maintainer: xCAT \n" - . "Description: repository test package\n", + . "Description: $description\n", ); die "Cannot build legacy test DEB\n" if run_capture( "$root.log", 'dpkg-deb', '--root-owner-group', '--build', $root, $output, ); } + +sub make_apt_inputs { + my ($apt_root, $package_root) = @_; + my $package = "$package_root.deb"; + make_legacy_deb($package_root, $package); + for my $version (qw(ubuntu22.04 ubuntu24.04 ubuntu26.04)) { + my $input = "$apt_root/$version"; + make_path($input); + copy($package, "$input/xcat-genesis-base-amd64_1_all.deb") or die $!; + } + return "$apt_root/ubuntu24.04"; +} diff --git a/t/genesis_openembedded_release.t b/t/genesis_openembedded_release.t index 276061b..365be5b 100644 --- a/t/genesis_openembedded_release.t +++ b/t/genesis_openembedded_release.t @@ -381,7 +381,16 @@ sub exercise_packager { ); is(run_capture($contents_log, 'rpm', '-qp', '--scripts', "$first/$relative"), 0, 'RPM script metadata can be read'); - is(read_binary($contents_log), '', 'RPM has no package scripts'); + like( + read_binary($contents_log), + qr{posttrans scriptlet.*\n/usr/libexec/xcat/genesis-openembedded-activate-x86_64 x86_64\n}s, + 'RPM refreshes the installed architecture after the transaction', + ); + like( + read_binary($contents_log), + qr{postuninstall scriptlet.*mknb x86_64 --remove-openembedded}s, + 'RPM retires published artifacts after erasing the image', + ); } else { is(run_capture($contents_log, 'dpkg-deb', '-c', "$first/$relative"), 0, 'DEB payload can be listed'); @@ -395,8 +404,18 @@ sub exercise_packager { opendir(my $control_dh, $control) or die $!; my @control_files = sort grep { $_ ne '.' && $_ ne '..' } readdir($control_dh); closedir($control_dh) or die $!; - is_deeply(\@control_files, [ qw(control md5sums) ], - 'DEB has no maintainer scripts or triggers'); + is_deeply(\@control_files, [ qw(control md5sums postinst postrm) ], + 'DEB contains the expected installation and removal scripts'); + like( + read_binary("$control/postinst"), + qr{^#!/bin/sh\n/usr/libexec/xcat/genesis-openembedded-activate-x86_64 x86_64\nexit 0\n$}, + 'DEB refreshes the installed architecture after configuration', + ); + like( + read_binary("$control/postrm"), + qr{mknb x86_64 --remove-openembedded}, + 'DEB retires published artifacts after erasing the image', + ); } }