diff --git a/.github/workflows/xcat_test.yml b/.github/workflows/xcat_test.yml index f5fb41db8..b90553a92 100644 --- a/.github/workflows/xcat_test.yml +++ b/.github/workflows/xcat_test.yml @@ -7,7 +7,7 @@ jobs: steps: - uses: actions/checkout@v6 - name: Install dependencies - run: sudo env DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends --no-install-suggests build-essential fakeroot reprepro devscripts debhelper libcapture-tiny-perl libjson-perl libsoap-lite-perl libdbi-perl libcgi-pm-perl quilt openssh-server dpkg looptools genometools software-properties-common + run: sudo env DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends --no-install-suggests build-essential fakeroot reprepro devscripts debhelper libcapture-tiny-perl libfile-slurper-perl libjson-perl libsoap-lite-perl libdbi-perl libcgi-pm-perl quilt openssh-server dpkg looptools genometools software-properties-common - name: Run tests run: perl github_action_xcat_test.pl diff --git a/docs/source/developers/guides/code/genesis_openembedded_plan.rst b/docs/source/developers/guides/code/genesis_openembedded_plan.rst index b8afa4eaf..92472c44e 100644 --- a/docs/source/developers/guides/code/genesis_openembedded_plan.rst +++ b/docs/source/developers/guides/code/genesis_openembedded_plan.rst @@ -230,8 +230,16 @@ xCAT server boundary -------------------- The OpenEmbedded layer builds and exports a kernel, compressed initramfs, -checksums, reports, and optional signed extensions. Installing those -artifacts and writing network boot configuration belongs to xCAT server code. +checksums, reports, and optional signed extensions. Packages install each +export under ``genesis-openembedded/ARCH``. ``mknb`` verifies and publishes +that export when present, while retaining the old Genesis path as a fallback. + +The management-node and service-node packages recommend the ``x86_64`` and +``ppc64le`` images. These are weak dependencies so an older or partial mirror +does not block an xCAT upgrade. Other target images can be installed from the +same common repository before running ``mknb ARCH``. RPM builds based on RPM +4.11 omit the recommendations because that version cannot parse weak dependency +tags. Install the required image package explicitly on those systems. Server integration should be reviewed separately from the image. Independent bugs found while testing Genesis, such as TFTP path handling or Kea policy, diff --git a/docs/source/guides/admin-guides/references/man8/mknb.8.rst b/docs/source/guides/admin-guides/references/man8/mknb.8.rst index 0aafa64ae..de6f39c38 100644 --- a/docs/source/guides/admin-guides/references/man8/mknb.8.rst +++ b/docs/source/guides/admin-guides/references/man8/mknb.8.rst @@ -19,7 +19,9 @@ SYNOPSIS ******** -\ **mknb**\ \ *arch*\ +\ **mknb**\ \ *arch*\ + +\ **mknb**\ \ *arch*\ \ **-**\ **-remove-openembedded**\ *********** @@ -32,13 +34,17 @@ It creates a network boot root image (used for node discovery, BMC programming, for the same architecture that the management node is. So you normally do not need to run the \ **mknb**\ command yourself. -If you make custom changes to the network boot root image, you will need to run \ **mknb**\ again to regenerate the diskless image to include your changes. If you have an xCAT Hierarchical Cluster with Service Nodes having local /tftpboot directories (site.sharedtftp=0), you will need to copy the generated root image to each Service Node. +If you make custom changes to the network boot root image, run \ **mknb**\ again to regenerate it. In an xCAT hierarchical cluster where service nodes have local \ ``/tftpboot``\ directories (\ ``site.sharedtftp=0``\ ), install and refresh an OpenEmbedded image on the service nodes before the management node starts using its exact architecture name. Legacy images must still be copied to each service node. -An exported Genesis image is identified by ``xcat-genesis.manifest``, which records its format version and architecture. The directory also contains ``kernel``, ``initramfs.cpio.gz``, and ``SHA256SUMS`` in ``/opt/xcat/share/xcat/netboot/genesis/ARCH``. \ **mknb**\ verifies the manifest and both boot files before publishing them under the configured TFTP root. An incomplete or invalid export leaves the current boot files unchanged. The legacy ``fs/`` layout remains supported and is packed locally. +An OpenEmbedded Genesis package installs an exported image under \ ``/opt/xcat/share/xcat/netboot/genesis-openembedded/ARCH``\ . \ **mknb**\ prefers that image when it is installed. Otherwise it uses the old image under \ ``/opt/xcat/share/xcat/netboot/genesis/ARCH``\ . -When multiple IPv4 addresses are configured for the same network, \ **mknb**\ uses a locally assigned ``site.master`` for the xcatd endpoint, or the first address reported by the operating system when ``site.master`` is not local. POWER discovery configurations also use this address for their kernel and initrd URLs. +An export is identified by \ ``xcat-genesis.manifest``\ , which records its format version and architecture. The directory also contains \ ``kernel``\ , \ ``initramfs.cpio.gz``\ , and \ ``SHA256SUMS``\ . \ **mknb**\ verifies the manifest and both boot files before publishing them under the configured TFTP root. An incomplete or invalid export leaves the current boot files unchanged. The legacy \ ``fs/``\ layout remains supported and is packed locally. -Presently, the architectures x86_64 and ppc64 are supported. For ppc64le, use the ppc64 architecture. +When multiple IPv4 addresses are configured for the same network, \ **mknb**\ uses a locally assigned \ ``site.master``\ for the xcatd endpoint, or the first address reported by the operating system when \ ``site.master``\ is not local. POWER discovery configurations also use this address for their kernel and initrd URLs. + +OpenEmbedded images use the exact architecture names \ ``x86``\ , \ ``x86_64``\ , \ ``ppc64``\ , \ ``ppc64le``\ , \ ``armv7hf``\ , \ ``aarch64``\ , and \ ``riscv64``\ . If an OpenEmbedded \ ``ppc64le``\ image is not installed, \ **mknb**\ keeps the old behavior and uses the legacy \ ``ppc64``\ image. + +Canonical \ ``ppc64``\ images are big-endian. xCAT marks them so \ ``ppc64le``\ nodes do not use them as a legacy little-endian fallback. \ **mknb**\ also refuses to replace a marked \ ``ppc64``\ image with that fallback. ******* @@ -47,10 +53,16 @@ OPTIONS -\ *arch*\ - +\ *arch*\ + The hardware architecture for which to build the boot image. - + + + +\ **-**\ **-remove-openembedded**\ + + Remove the published boot files for the selected OpenEmbedded architecture. Image packages use this option when they are removed. + diff --git a/docs/source/guides/install-guides/common_sections.rst b/docs/source/guides/install-guides/common_sections.rst index fb1249172..7c52fdb67 100644 --- a/docs/source/guides/install-guides/common_sections.rst +++ b/docs/source/guides/install-guides/common_sections.rst @@ -145,6 +145,64 @@ Unless you are downloading ``xcat-dep`` to match a specific version of xCAT, it' .. END_configure_xcat_local_repo_xcat-dep_RPM +.. BEGIN_configure_xcat_local_repo_xcat-dep_COMMON_MIRROR + +**[xcat-dep common]** + +The xcat-dep archives do not contain the common repository. Mirror it on a +connected host with ``dnf-plugins-core`` before disconnecting the installation +network: :: + + mkdir -p ~/xcat/xcat-dep/common + dnf reposync \ + --repofrompath=xcat-dep-common,https://xcat.org/files/xcat/repos/yum/latest/xcat-dep/common \ + --repoid=xcat-dep-common --download-metadata \ + --download-path ~/xcat/xcat-dep/common --norepopath + curl -fL -o ~/xcat/xcat-dep/common/repodata/repomd.xml.asc \ + https://xcat.org/files/xcat/repos/yum/latest/xcat-dep/common/repodata/repomd.xml.asc + curl -fL -o ~/xcat/xcat-dep/common/repodata/repomd.xml.key \ + https://xcat.org/files/xcat/repos/yum/latest/xcat-dep/common/repodata/repomd.xml.key + +Copy the complete mirror to the management node. + +.. END_configure_xcat_local_repo_xcat-dep_COMMON_MIRROR + +.. BEGIN_configure_xcat_local_repo_xcat-dep_DNF + +Create the local DNF repository definition: :: + + cat >/etc/yum.repos.d/xcat-dep-common.repo <<'EOF' + [xcat-dep-common] + name=xCAT common dependencies + baseurl=file:///root/xcat/xcat-dep/common + enabled=1 + skip_if_unavailable=0 + gpgcheck=1 + repo_gpgcheck=1 + gpgkey=file:///root/xcat/xcat-dep/common/repodata/repomd.xml.key + EOF + dnf clean metadata + dnf makecache --disablerepo='*' --enablerepo=xcat-dep-common + +.. END_configure_xcat_local_repo_xcat-dep_DNF + +.. BEGIN_configure_xcat_local_repo_xcat-dep_ZYPPER + +Create the local zypper repository definition: :: + + cat >/etc/zypp/repos.d/xcat-dep-common.repo <<'EOF' + [xcat-dep-common] + name=xCAT common dependencies + baseurl=file:///root/xcat/xcat-dep/common + enabled=1 + autorefresh=0 + gpgcheck=1 + gpgkey=file:///root/xcat/xcat-dep/common/repodata/repomd.xml.key + EOF + zypper --gpg-auto-import-keys refresh xcat-dep-common + +.. END_configure_xcat_local_repo_xcat-dep_ZYPPER + .. BEGIN_configure_xcat_local_repo_xcat-dep_DEBIAN **[xcat-dep]** @@ -225,4 +283,3 @@ xCAT is started automatically after the installation, but the following commands .. END_verifying_xcat - diff --git a/docs/source/guides/install-guides/yum/configure_xcat.rst b/docs/source/guides/install-guides/yum/configure_xcat.rst index fba2ec140..617bdc4ee 100644 --- a/docs/source/guides/install-guides/yum/configure_xcat.rst +++ b/docs/source/guides/install-guides/yum/configure_xcat.rst @@ -6,14 +6,16 @@ xCAT software and repo files can be obtained from: ` - creates a network boot root image for node discovery and flashing B I +B I B<--remove-openembedded> + =head1 DESCRIPTION @@ -14,13 +16,17 @@ It creates a network boot root image (used for node discovery, BMC programming, for the same architecture that the management node is. So you normally do not need to run the B command yourself. -If you make custom changes to the network boot root image, you will need to run B again to regenerate the diskless image to include your changes. If you have an xCAT Hierarchical Cluster with Service Nodes having local /tftpboot directories (site.sharedtftp=0), you will need to copy the generated root image to each Service Node. +If you make custom changes to the network boot root image, run B again to regenerate it. In an xCAT hierarchical cluster where service nodes have local C directories (C), install and refresh an OpenEmbedded image on the service nodes before the management node starts using its exact architecture name. Legacy images must still be copied to each service node. -An exported Genesis image is identified by C, which records its format version and architecture. The directory also contains C, C, and C in C. B verifies the manifest and both boot files before publishing them under the configured TFTP root. An incomplete or invalid export leaves the current boot files unchanged. The legacy C layout remains supported and is packed locally. +An OpenEmbedded Genesis package installs an exported image under C. B prefers that image when it is installed. Otherwise it uses the old image under C. + +An export is identified by C, which records its format version and architecture. The directory also contains C, C, and C. B verifies the manifest and both boot files before publishing them under the configured TFTP root. An incomplete or invalid export leaves the current boot files unchanged. The legacy C layout remains supported and is packed locally. When multiple IPv4 addresses are configured for the same network, B uses a locally assigned C for the xcatd endpoint, or the first address reported by the operating system when C is not local. POWER discovery configurations also use this address for their kernel and initrd URLs. -Presently, the architectures x86_64 and ppc64 are supported. For ppc64le, use the ppc64 architecture. +OpenEmbedded images use the exact architecture names C, C, C, C, C, C, and C. If an OpenEmbedded C image is not installed, B keeps the old behavior and uses the legacy C image. + +Canonical C images are big-endian. xCAT marks them so C nodes do not use them as a legacy little-endian fallback. B also refuses to replace a marked C image with that fallback. =head1 OPTIONS @@ -30,6 +36,10 @@ Presently, the architectures x86_64 and ppc64 are supported. For ppc64le, use th The hardware architecture for which to build the boot image. +=item B<--remove-openembedded> + +Remove the published boot files for the selected OpenEmbedded architecture. Image packages use this option when they are removed. + =back =head1 RETURN VALUE diff --git a/xCAT-release/xCAT-release.spec b/xCAT-release/xCAT-release.spec index 27e5792ed..7f6e3d523 100644 --- a/xCAT-release/xCAT-release.spec +++ b/xCAT-release/xCAT-release.spec @@ -23,11 +23,13 @@ mkdir -p %{buildroot}%{_sysconfdir}/yum.repos.d mkdir -p %{buildroot}%{_sysconfdir}/pki/rpm-gpg install -m 0644 xcat-core.repo %{buildroot}%{_sysconfdir}/yum.repos.d/xcat-core.repo install -m 0644 xcat-dep.repo %{buildroot}%{_sysconfdir}/yum.repos.d/xcat-dep.repo +install -m 0644 xcat-dep-common.repo %{buildroot}%{_sysconfdir}/yum.repos.d/xcat-dep-common.repo install -m 0644 RPM-GPG-KEY-xCAT %{buildroot}%{_sysconfdir}/pki/rpm-gpg/RPM-GPG-KEY-xCAT %files %config(noreplace) %{_sysconfdir}/yum.repos.d/xcat-core.repo %config(noreplace) %{_sysconfdir}/yum.repos.d/xcat-dep.repo +%config(noreplace) %{_sysconfdir}/yum.repos.d/xcat-dep-common.repo %{_sysconfdir}/pki/rpm-gpg/RPM-GPG-KEY-xCAT %changelog diff --git a/xCAT-release/xcat-dep-common.repo b/xCAT-release/xcat-dep-common.repo new file mode 100644 index 000000000..47f829d33 --- /dev/null +++ b/xCAT-release/xcat-dep-common.repo @@ -0,0 +1,8 @@ +[xcat-dep-common] +name=xCAT common dependencies +baseurl=https://xcat.org/files/xcat/repos/yum/latest/xcat-dep/common +enabled=1 +skip_if_unavailable=1 +gpgcheck=1 +repo_gpgcheck=1 +gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-xCAT diff --git a/xCAT-server/lib/xcat/plugins/anaconda.pm b/xCAT-server/lib/xcat/plugins/anaconda.pm index 6ad77d3b3..87ec419cc 100644 --- a/xCAT-server/lib/xcat/plugins/anaconda.pm +++ b/xCAT-server/lib/xcat/plugins/anaconda.pm @@ -68,6 +68,31 @@ sub handled_commands }; } +sub _find_genesis_boot_files +{ + my ($tftpdir, $arch) = @_; + return unless defined($arch) && $arch =~ /\A[A-Za-z0-9_]+\z/; + + my $directory = "$tftpdir/xcat"; + my $kernel = "genesis.kernel.$arch"; + return unless -r "$directory/$kernel"; + + my $lzma = "genesis.fs.$arch.lzma"; + my $gzip = "genesis.fs.$arch.gz"; + my $initrd; + if (-r "$directory/$lzma" && -r "$directory/$gzip") { + $initrd = -C "$directory/$lzma" > -C "$directory/$gzip" + ? $gzip + : $lzma; + } elsif (-r "$directory/$lzma") { + $initrd = $lzma; + } elsif (-r "$directory/$gzip") { + $initrd = $gzip; + } + return unless defined($initrd); + return ($kernel, $initrd); +} + sub preprocess_request { my $req = shift; @@ -1854,12 +1879,9 @@ sub mksysclone =cut my $ramdisk_size = 200000; - my $kernpath = `ls -l $tftpdir/xcat/|grep "genesis.kernel.$arch"|awk '{print \$9}'`; - chomp($kernpath); - my $initrdpath = `ls -l $tftpdir/xcat/|grep "genesis.fs.$arch"| awk '{print \$9}'`; - chomp($initrdpath); + my ($kernpath, $initrdpath) = _find_genesis_boot_files($tftpdir, $arch); - if ($kernpath ne '' and $initrdpath ne '') + if (defined($kernpath) && defined($initrdpath)) { #We have a shot... my $ent = $rents{$node}->[0]; diff --git a/xCAT-server/lib/xcat/plugins/destiny.pm b/xCAT-server/lib/xcat/plugins/destiny.pm index 38a88042b..6c33617f0 100644 --- a/xCAT-server/lib/xcat/plugins/destiny.pm +++ b/xCAT-server/lib/xcat/plugins/destiny.pm @@ -57,6 +57,23 @@ sub handled_commands { } } +sub _genesis_boot_arch { + my ($directory, $requested_arch) = @_; + my $arch = $requested_arch eq 'ppc64el' ? 'ppc64le' : $requested_arch; + + if ($arch eq 'ppc64le' + && !-r "$directory/xcat/genesis.kernel.ppc64le" + && !-r "$directory/xcat/genesis.exact-arch.ppc64") { + return 'ppc64'; + } + return $arch; +} + +sub _genesis_uses_power_console { + my ($arch) = @_; + return $arch eq 'ppc64' || $arch eq 'ppc64le'; +} + sub process_request { $request = shift; $callback = shift; @@ -602,10 +619,7 @@ sub setdestiny { xCAT::MsgUtils->report_node_error($callback, $_, "No archictecture defined in nodetype table for the node."); next; } - my $arch = $ent->{arch}; - if ($arch eq "ppc64le" or $arch eq "ppc64el") { - $arch = "ppc64"; - } + my $arch = _genesis_boot_arch($tftpdir, $ent->{arch}); my $ent = $resents->{$_}->[0]; #$restab->getNodeAttribs($_,[qw(xcatmaster)]); my $master; my $kcmdline = "quiet "; @@ -637,7 +651,7 @@ sub setdestiny { $ent = $hments->{$_}->[0]; #$nodehm->getNodeAttribs($_,['serialport','serialspeed','serialflow']); if ($ent and defined($ent->{serialport})) { - if ($arch eq "ppc64") { + if (_genesis_uses_power_console($arch)) { $kcmdline .= "console=tty0 console=hvc" . $ent->{serialport}; } else { $kcmdline .= "console=tty0 console=ttyS" . $ent->{serialport}; diff --git a/xCAT-server/lib/xcat/plugins/mknb.pm b/xCAT-server/lib/xcat/plugins/mknb.pm index b385f0698..a3a2a5b91 100644 --- a/xCAT-server/lib/xcat/plugins/mknb.pm +++ b/xCAT-server/lib/xcat/plugins/mknb.pm @@ -9,6 +9,14 @@ use File::Path; use File::Copy; my $GENESIS_EXPORT_MANIFEST = 'xcat-genesis.manifest'; +my %GENESIS_ARCHITECTURES = map { $_ => 1 } + qw(x86 x86_64 ppc64 ppc64le armv7hf aarch64 riscv64); + +sub _canonical_genesis_arch { + my ($arch) = @_; + return unless defined($arch); + return $arch eq 'ppc64el' ? 'ppc64le' : $arch; +} sub handled_commands { return { @@ -38,6 +46,27 @@ sub _select_network_addresses { return (\%legacy, \%selected); } +sub _select_genesis_source { + my ($xcatroot, $requested_arch) = @_; + my $arch = _canonical_genesis_arch($requested_arch); + return unless defined($arch); + return unless $GENESIS_ARCHITECTURES{$arch}; + + my $netboot = "$xcatroot/share/xcat/netboot"; + my $openembedded = "$netboot/genesis-openembedded/$arch"; + return ($openembedded, $arch, 'openembedded') + if -d $openembedded; + + my $legacy_arch = $arch eq 'ppc64le' ? 'ppc64' : $arch; + my $legacy = "$netboot/genesis/$legacy_arch"; + return ($legacy, $legacy_arch, 'legacy') if -d $legacy; + + my $classic = "$netboot/$legacy_arch"; + return ($classic, $legacy_arch, 'classic') if -d $classic; + + return; +} + sub _genesis_export_manifest_present { my ($directory) = @_; my $manifest = "$directory/$GENESIS_EXPORT_MANIFEST"; @@ -174,6 +203,8 @@ sub _install_prebuilt_genesis { my @artifacts = ( [ 'kernel', "$destination_dir/genesis.kernel.$arch" ], [ 'initramfs.cpio.gz', "$destination_dir/genesis.fs.$arch.gz" ], + [ $GENESIS_EXPORT_MANIFEST, + "$destination_dir/genesis.exact-arch.$arch" ], ); my @staged; @@ -243,9 +274,44 @@ sub _install_prebuilt_genesis { } unlink(values(%backups)); + unlink("$destination_dir/genesis.fs.$arch.lzma"); return ("$destination_dir/genesis.fs.$arch.gz", undef); } +sub _remove_openembedded_genesis { + my ($tftpdir, $requested_arch) = @_; + my $arch = _canonical_genesis_arch($requested_arch); + return (0, 'Missing Genesis architecture') unless defined($arch); + return (0, "Unsupported Genesis architecture: $requested_arch") + unless $GENESIS_ARCHITECTURES{$arch}; + + my $directory = "$tftpdir/xcat"; + my @artifacts = ( + "$directory/genesis.kernel.$arch", + "$directory/genesis.fs.$arch.gz", + "$directory/genesis.fs.$arch.lzma", + "$directory/genesis.exact-arch.$arch", + ); + my $removed = 0; + my @failed; + foreach my $artifact (@artifacts) { + next unless -e $artifact || -l $artifact; + unless (unlink($artifact)) { + push(@failed, $artifact); + next; + } + $removed++; + } + if (@failed == 1) { + return ($removed, "Unable to remove Genesis artifact: $failed[0]"); + } + if (@failed) { + return ($removed, + 'Unable to remove Genesis artifacts: ' . join(', ', @failed)); + } + return ($removed, undef); +} + sub genesis_lzma_command { my ($have_lzma, $have_xz) = @_; return 'lzma -C crc32 -9' if $have_lzma; @@ -341,21 +407,58 @@ sub process_request { } my $tftpdir = xCAT::TableUtils->getTftpDir(); - my $arch = $request->{arg}->[0]; - if (!$arch) { - $callback->({ error => "Need to specify architecture (x86, x86_64 or ppc64)" }, { errorcode => [1] }); + my $requested_arch = $request->{arg}->[0]; + if (!$requested_arch) { + $callback->({ error => "Need to specify architecture (x86, x86_64, ppc64, ppc64le, armv7hf, aarch64 or riscv64)" }, { errorcode => [1] }); return; - } elsif ($arch eq "ppc64le" or $arch eq "ppc64el") { - $callback->({ data => "The arch:$arch is not supported, using \"ppc64\" instead" }); - $arch = 'ppc64'; - $request->{arg}->[0] = $arch; } - my $genesis_dir = "$::XCATROOT/share/xcat/netboot/genesis/$arch"; - unless (-d "$::XCATROOT/share/xcat/netboot/$arch" or -d $genesis_dir) { - $callback->({ error => "Unable to find directory $::XCATROOT/share/xcat/netboot/$arch or $::XCATROOT/share/xcat/netboot/genesis/$arch", errorcode => [1] }); + my $canonical_arch = _canonical_genesis_arch($requested_arch); + if (($request->{arg}->[1] // '') eq '--remove-openembedded') { + unless ($GENESIS_ARCHITECTURES{$canonical_arch}) { + $callback->({ error => "Unsupported Genesis architecture: $requested_arch", errorcode => [1] }); + return; + } + my $source = "$::XCATROOT/share/xcat/netboot/genesis-openembedded/$canonical_arch"; + if (-d $source || -l $source) { + $callback->({ error => "Cannot remove boot artifacts while OpenEmbedded Genesis $canonical_arch is installed", errorcode => [1] }); + return; + } + my ($removed, $remove_error) = + _remove_openembedded_genesis($tftpdir, $canonical_arch); + if ($remove_error) { + $callback->({ error => $remove_error, errorcode => [1] }); + return; + } + $callback->({ data => "Removed $removed OpenEmbedded Genesis artifacts for $canonical_arch" }); return; } + + my ($genesis_dir, $arch, $genesis_type) = + _select_genesis_source($::XCATROOT, $requested_arch); + unless (defined($genesis_dir) && -d $genesis_dir) { + $callback->({ error => "Unable to find a Genesis image for architecture $requested_arch", errorcode => [1] }); + return; + } + if ($canonical_arch eq 'ppc64le' && $arch eq 'ppc64') { + my $marker = "$tftpdir/xcat/genesis.exact-arch.ppc64"; + if (-e $marker || -l $marker) { + $callback->({ + error => 'Cannot use the legacy ppc64le fallback while a canonical ppc64 image is published', + errorcode => [1], + }); + return; + } + } + if ($requested_arch eq 'ppc64el') { + $callback->({ data => 'Using the canonical architecture name ppc64le' }); + } + if (($requested_arch eq 'ppc64le' || $requested_arch eq 'ppc64el') + && $arch eq 'ppc64') { + $callback->({ data => 'OpenEmbedded ppc64le is not installed, using the legacy ppc64 image' }); + } + $request->{arg}->[0] = $arch; + my $configfileonly = $request->{arg}->[1]; if ($configfileonly and $configfileonly ne "-c" and $configfileonly ne "--configfileonly") { $callback->({ error => "The option $configfileonly is not supported", errorcode => [1] }); @@ -364,7 +467,10 @@ sub process_request { goto CREAT_CONF_FILE; } if (_prebuilt_genesis_requested($genesis_dir)) { - $callback->({ data => ["Installing exported Genesis image for $arch"] }); + my $image_name = $genesis_type eq 'openembedded' + ? 'OpenEmbedded Genesis' + : 'exported Genesis'; + $callback->({ data => ["Installing $image_name image for $arch"] }); my ($installed_initrd, $install_error) = _install_prebuilt_genesis($genesis_dir, $tftpdir, $arch); if ($install_error) { @@ -375,7 +481,8 @@ sub process_request { $invisibletouch = 1; goto CREAT_CONF_FILE; } - if (_genesis_export_manifest_present($genesis_dir)) { + if ($genesis_type eq 'openembedded' + || _genesis_export_manifest_present($genesis_dir)) { $callback->({ error => ["Incomplete Genesis export: $genesis_dir"], errorcode => [1], @@ -441,19 +548,19 @@ sub process_request { mkpath("$tftpdir/xcat"); } my $rc; - if (-e "$::XCATROOT/share/xcat/netboot/genesis/$arch") { - $rc = system("shopt -s dotglob; GLOBIGNORE=\".:..\" cp -a $::XCATROOT/share/xcat/netboot/genesis/$arch/fs/* $tempdir"); - $rc = system("cp -a $::XCATROOT/share/xcat/netboot/genesis/$arch/kernel $tftpdir/xcat/genesis.kernel.$arch"); + if ($genesis_type eq 'legacy') { + $rc = system("shopt -s dotglob; GLOBIGNORE=\".:..\" cp -a $genesis_dir/fs/* $tempdir"); + $rc = system("cp -a $genesis_dir/kernel $tftpdir/xcat/genesis.kernel.$arch"); $invisibletouch = 1; } else { - $rc = system("cp -a $::XCATROOT/share/xcat/netboot/$arch/nbroot/* $tempdir"); + $rc = system("cp -a $genesis_dir/nbroot/* $tempdir"); } if ($rc) { system("rm -rf $tempdir"); if ($invisibletouch) { - $callback->({ error => ["Failed to copy $::XCATROOT/share/xcat/netboot/genesis/$arch/fs contents"], errorcode => [1] }); + $callback->({ error => ["Failed to copy $genesis_dir/fs contents"], errorcode => [1] }); } else { - $callback->({ error => ["Failed to copy $::XCATROOT/share/xcat/netboot/$arch/nbroot/ contents"], errorcode => [1] }); + $callback->({ error => ["Failed to copy $genesis_dir/nbroot contents"], errorcode => [1] }); } return; } @@ -522,6 +629,13 @@ sub process_request { $callback->({ data => ["Creating filesystem file in $tftpdir/xcat failed"] }); return; } + my $exact_arch_marker = "$tftpdir/xcat/genesis.exact-arch.$arch"; + if (($genesis_type eq 'legacy' || $genesis_type eq 'classic') + && (-e $exact_arch_marker || -l $exact_arch_marker) + && !unlink($exact_arch_marker)) { + $callback->({ error => ["Unable to remove Genesis architecture marker: $exact_arch_marker"], errorcode => [1] }); + return; + } CREAT_CONF_FILE: if ($configfileonly) { diff --git a/xCAT-server/lib/xcat/plugins/sles.pm b/xCAT-server/lib/xcat/plugins/sles.pm index ec35e0f15..6aaa5cfb9 100644 --- a/xCAT-server/lib/xcat/plugins/sles.pm +++ b/xCAT-server/lib/xcat/plugins/sles.pm @@ -44,6 +44,31 @@ sub handled_commands }; } +sub _find_genesis_boot_files +{ + my ($tftpdir, $arch) = @_; + return unless defined($arch) && $arch =~ /\A[A-Za-z0-9_]+\z/; + + my $directory = "$tftpdir/xcat"; + my $kernel = "genesis.kernel.$arch"; + return unless -r "$directory/$kernel"; + + my $lzma = "genesis.fs.$arch.lzma"; + my $gzip = "genesis.fs.$arch.gz"; + my $initrd; + if (-r "$directory/$lzma" && -r "$directory/$gzip") { + $initrd = -C "$directory/$lzma" > -C "$directory/$gzip" + ? $gzip + : $lzma; + } elsif (-r "$directory/$lzma") { + $initrd = $lzma; + } elsif (-r "$directory/$gzip") { + $initrd = $gzip; + } + return unless defined($initrd); + return ($kernel, $initrd); +} + sub precreate_sles11_mypostscript { my ($node, $osver, $callback) = @_; @@ -1450,10 +1475,10 @@ sub mksysclone # copy kernel and initrd from image dir to /tftpboot my $ramdisk_size = 200000; + my ($genesis_kernel, $genesis_initrd) = + _find_genesis_boot_files($tftpdir, $arch); - if (-r "$tftpdir/xcat/genesis.kernel.$arch" - and (-r "$tftpdir/xcat/genesis.fs.$arch.gz" - or -r "$tftpdir/xcat/genesis.fs.$arch.lzma")) + if (defined($genesis_kernel) && defined($genesis_initrd)) { #We have a shot... my $ent = $rents{$node}->[0]; @@ -1505,12 +1530,8 @@ sub mksysclone } $kcmdline .= " XCAT=$xcatmaster:$xcatdport xcatd=$xcatmaster:$xcatdport SCRIPTNAME=$imagename"; - my $i = "xcat/genesis.fs.$arch.gz"; - if (-r "$tftpdir/xcat/genesis.fs.$arch.lzma") { - $i = "xcat/genesis.fs.$arch.lzma"; - } - $bootparams->{$node}->[0]->{kernel} = "xcat/genesis.kernel.$arch"; - $bootparams->{$node}->[0]->{initrd} = $i; + $bootparams->{$node}->[0]->{kernel} = "xcat/$genesis_kernel"; + $bootparams->{$node}->[0]->{initrd} = "xcat/$genesis_initrd"; $bootparams->{$node}->[0]->{kcmdline} = $kcmdline; } else diff --git a/xCAT-server/sbin/xcatconfig b/xCAT-server/sbin/xcatconfig index 829116d93..f4c1394ab 100755 --- a/xCAT-server/sbin/xcatconfig +++ b/xCAT-server/sbin/xcatconfig @@ -1991,10 +1991,48 @@ sub setupLinuxexports =head3 mknb Creates a network boot root image on Linux - Run mknb for each installed xCAT-genesis-scripts/xCAT-genesis-base architecture and place the result in /tftpboot + Run mknb for each installed Genesis architecture and place the result in /tftpboot =cut #----------------------------------------------------------------------------- +sub _installed_genesis_architectures +{ + my ($xcatroot) = @_; + my @supported = qw(aarch64 armv7hf ppc64 ppc64le riscv64 x86 x86_64); + my %supported = map { $_ => 1 } @supported; + my %installed; + + my $openembedded = "$xcatroot/share/xcat/netboot/genesis-openembedded"; + if (opendir(my $dh, $openembedded)) { + while (my $entry = readdir($dh)) { + next unless $supported{$entry}; + my $path = "$openembedded/$entry"; + $installed{$entry} = 1 if -d $path && !-l $path; + } + closedir($dh); + } + + my $legacy = "$xcatroot/share/xcat/netboot/genesis"; + foreach my $arch (@supported) { + my $path = "$legacy/$arch"; + $installed{$arch} = 1 if -d $path; + } + + return grep { $installed{$_} } @supported; +} + +sub _genesis_architectures_to_build +{ + my ($xcatroot, $include_legacy) = @_; + my @installed = _installed_genesis_architectures($xcatroot); + return @installed if $include_legacy; + + return grep { + -d "$xcatroot/share/xcat/netboot/genesis-openembedded/$_" + && !-l "$xcatroot/share/xcat/netboot/genesis-openembedded/$_" + } @installed; +} + sub mknb { # The xCAT-genesis-scripts.spec file touches /etc/xcat/genesis-scripts-updated or @@ -2014,20 +2052,20 @@ sub mknb xCAT::MsgUtils->message('I', "Running '$cmd', triggered by the installation/update of xCAT-genesis-base ..."); } } - if ($run_mknb) { - foreach (qw(ppc64 x86_64)) { - my $genesis_base_pkg = "xCAT-genesis-base-$_"; - if (system("rpm -q $genesis_base_pkg >/dev/null 2>&1") != 0) { - xCAT::MsgUtils->message('I', "Skipping '$cmd $_' because $genesis_base_pkg is not installed."); - next; - } - system("$cmd $_"); + my @architectures = + _genesis_architectures_to_build($::XCATROOT, $run_mknb); + if (@architectures) { + unless ($run_mknb) { + xCAT::MsgUtils->message('I', "Running '$cmd' for installed OpenEmbedded Genesis images ..."); + } + foreach my $arch (@architectures) { + system("$cmd $arch"); if ($? != 0) { my $rc = $? >> 8; - xCAT::MsgUtils->message('E', "The 'mknb $_' command returned error code: $rc."); + xCAT::MsgUtils->message('E', "The 'mknb $arch' command returned error code: $rc."); } else { - xCAT::MsgUtils->message('I', "The 'mknb $_' command completed successfully."); + xCAT::MsgUtils->message('I', "The 'mknb $arch' command completed successfully."); } } } diff --git a/xCAT-server/share/xcat/tools/go-xcat b/xCAT-server/share/xcat/tools/go-xcat index 897f6ff5f..c8943d303 100755 --- a/xCAT-server/share/xcat/tools/go-xcat +++ b/xCAT-server/share/xcat/tools/go-xcat @@ -189,6 +189,7 @@ GO_XCAT_DEFAULT_INSTALL_PATH="/install/xcat" # The package list of xcat-core GO_XCAT_CORE_PACKAGE_LIST=() GO_XCAT_DEP_PACKAGE_LIST=() +GO_XCAT_DEP_REPOSITORY_IDS=(xcat-dep) # The package list of all the packages should be installed GO_XCAT_INSTALL_LIST=(perl-xCAT xCAT-client xCAT xCAT-buildkit @@ -204,12 +205,20 @@ GO_XCAT_INSTALL_LIST=(perl-xcat xcat-client xcat xcat-buildkit # The package list of all the packages should be uninstalled GO_XCAT_UNINSTALL_LIST=("${GO_XCAT_INSTALL_LIST[@]}" goconserver xCAT-SoftLayer xCAT-confluent xCAT-csm xCAT-genesis-builder + xCAT-genesis-openembedded-x86 xCAT-genesis-openembedded-x86_64 + xCAT-genesis-openembedded-ppc64 xCAT-genesis-openembedded-ppc64le + xCAT-genesis-openembedded-armv7hf xCAT-genesis-openembedded-aarch64 + xCAT-genesis-openembedded-riscv64 xCAT-openbmc-py xCAT-probe xCAT-test xCAT-vlan xCATsn xCAT-UI-deps xCAT-buildkit conserver-xcat yaboot-xcat) # For Debian/Ubuntu, it will need a slightly different package list type dpkg >/dev/null 2>&1 && GO_XCAT_UNINSTALL_LIST=("${GO_XCAT_INSTALL_LIST[@]}" goconserver xcat-confluent xcat-probe xcat-test xcat-vlan xcatsn + xcat-genesis-openembedded-x86 xcat-genesis-openembedded-x86-64 + xcat-genesis-openembedded-ppc64 xcat-genesis-openembedded-ppc64le + xcat-genesis-openembedded-armv7hf xcat-genesis-openembedded-aarch64 + xcat-genesis-openembedded-riscv64 xcat-buildkit conserver-xcat) PATH="/usr/sbin:/usr/bin:/sbin:/bin" @@ -1336,6 +1345,7 @@ function add_repo_by_url_yum_or_zypper() [[ "$1" = "-y" ]] && yes=("-y") && shift local url="$1" local repo_id="$2" + local policy="${3:-}" local tmp="" local install_path="${GO_XCAT_DEFAULT_INSTALL_PATH}" case "${url%%://*}" in @@ -1352,7 +1362,8 @@ function add_repo_by_url_yum_or_zypper() ;; *) # assume it is the base url of the repo tmp="${TMP_DIR}/tmp_repo.repo" - while read -r ; do echo "${REPLY}" ; done >"${tmp}" <<-EOF + { + while read -r ; do echo "${REPLY}" ; done <<-EOF [${repo_id}] name=${repo_id} baseurl=${url%/} @@ -1360,6 +1371,11 @@ function add_repo_by_url_yum_or_zypper() gpgcheck=1 gpgkey=${url%/}/repodata/repomd.xml.key EOF + if [[ "${policy}" = "optional" ]]; then + echo "repo_gpgcheck=1" + echo "skip_if_unavailable=1" + fi + } >"${tmp}" add_repo_by_file "${tmp}" "${repo_id}" return "$?" ;; @@ -1388,15 +1404,22 @@ function add_repo_by_url_yum_or_zypper() [[ "${url:0:1}" = "/" ]] || url="${PWD}/${url}" # directory tmp="${TMP_DIR}/tmp_repo.repo" - while read -r ; do echo "${REPLY}" ; done >"${tmp}" <<-EOF + { + while read -r ; do echo "${REPLY}" ; done <<-EOF [${repo_id}] name=${repo_id} baseurl=file://${url%/} enabled=1 EOF + if [[ "${policy}" = "optional" ]]; then + echo "skip_if_unavailable=1" + fi + } >"${tmp}" if [ -f "${url%/}/repodata/repomd.xml.asc" ] then echo "gpgcheck=1" >>"${tmp}" + [[ "${policy}" != "optional" ]] \ + || echo "repo_gpgcheck=1" >>"${tmp}" else echo "gpgcheck=0" >>"${tmp}" fi @@ -1491,6 +1514,9 @@ function remove_repo_yum() "xcat-dep") mv /etc/yum.repos.d/xCAT-dep.repo{,.nouse} 2>/dev/null ;; + "xcat-dep-common") + mv /etc/yum.repos.d/xCAT-dep-common.repo{,.nouse} 2>/dev/null + ;; esac yum clean metadata : @@ -1509,6 +1535,9 @@ function remove_repo_zypper() "xcat-dep") mv /etc/zypp/repos.d/xCAT-dep.repo{,.nouse} 2>/dev/null ;; + "xcat-dep-common") + mv /etc/zypp/repos.d/xCAT-dep-common.repo{,.nouse} 2>/dev/null + ;; esac : } @@ -1592,6 +1621,61 @@ function add_xcat_core_repo() function_dispatch "${FUNCNAME[0]}" "$@" } +function add_xcat_dep_common_repo_yum_or_zypper() +{ + type yum >/dev/null 2>&1 || type zypper >/dev/null 2>&1 || return 255 + local url="$1" + local ver="$2" + local common_url="" + local metadata="" + [[ -z "${ver}" ]] && ver="latest" + + [[ -z "${url}" ]] && + url="${GO_XCAT_DEFAULT_BASE_URL}/yum/${ver}/xcat-dep" + case "${url##*/}" in + *".repo"|*".tar"|*".tar.Z"|*".tar.bz"|*".tar.bz2"|*".tar.gz"|*".tar.xz"|*".tbz"|*".tbz2"|*".tgz"|*".tz"|*".txz") + return 0 + ;; + esac + + case "${url%%://*}" in + "ftp"|"http"|"https") + common_url="${url%/}/common" + metadata="${TMP_DIR}/xcat-dep-common-repomd.xml" + download_file "${common_url}/repodata/repomd.xml" "${metadata}" \ + >/dev/null 2>&1 || return 0 + ;; + "file") + url="${url#file://}" + [[ "${url:0:1}" = "/" ]] || url="${PWD}/${url}" + common_url="${url%/}/common" + [[ -f "${common_url}/repodata/repomd.xml" ]] || return 0 + ;; + *) + [[ "${url:0:1}" = "/" ]] || url="${PWD}/${url}" + common_url="${url%/}/common" + [[ -f "${common_url}/repodata/repomd.xml" ]] || return 0 + ;; + esac + + add_repo_by_url_yum_or_zypper \ + "${common_url}" "xcat-dep-common" optional || return 0 +} + +function xcat_dep_common_repo_configured() +{ + [[ -f /etc/yum.repos.d/xcat-dep-common.repo \ + || -f /etc/zypp/repos.d/xcat-dep-common.repo ]] +} + +function refresh_xcat_dep_repository_ids() +{ + GO_XCAT_DEP_REPOSITORY_IDS=(xcat-dep) + xcat_dep_common_repo_configured \ + && GO_XCAT_DEP_REPOSITORY_IDS+=(xcat-dep-common) + : +} + function add_xcat_dep_repo_yum_or_zypper() { type yum >/dev/null 2>&1 || type zypper >/dev/null 2>&1 || return 255 @@ -1636,8 +1720,10 @@ function add_xcat_dep_repo_yum_or_zypper() url="${tmp}" ;; *) - url="${url}/${distro}/${GO_XCAT_ARCH}" - add_repo_by_url_yum_or_zypper "${url}" "xcat-dep" + local repository_root="${url}" + url="${repository_root}/${distro}/${GO_XCAT_ARCH}" + add_repo_by_url_yum_or_zypper "${url}" "xcat-dep" || return "$?" + add_xcat_dep_common_repo_yum_or_zypper "${repository_root}" "${ver}" return "$?" ;; esac @@ -1657,8 +1743,10 @@ function add_xcat_dep_repo_yum_or_zypper() then # make sure it is an absolute pathname. [[ "${url:0:1}" = "/" ]] || url="${PWD}/${url}" - url="${url}/${distro}/${GO_XCAT_ARCH}" - add_repo_by_url_yum_or_zypper "${url}" "xcat-dep" + local repository_root="${url}" + url="${repository_root}/${distro}/${GO_XCAT_ARCH}" + add_repo_by_url_yum_or_zypper "${url}" "xcat-dep" || return "$?" + add_xcat_dep_common_repo_yum_or_zypper "${repository_root}" "${ver}" return "$?" fi warn_if_bad "1" "invalid xcat-dep URL" @@ -1969,10 +2057,10 @@ function trash_xcat() rm -rf /etc/xcat rm -rf /etc/xcatdockerca rm -f /etc/apt/sources.list.d/xcat-{core,dep}.list - rm -f /etc/yum.repos.d/xCAT-{core,dep}.repo{,.nouse} - rm -f /etc/yum.repos.d/xcat-{core,dep}.repo - rm -f /etc/zypp/repos.d/xCAT-{core,dep}.repo{,.nouse} - rm -f /etc/zypp/repos.d/xcat-{core,dep}.repo + rm -f /etc/yum.repos.d/xCAT-{core,dep,dep-common}.repo{,.nouse} + rm -f /etc/yum.repos.d/xcat-{core,dep,dep-common}.repo + rm -f /etc/zypp/repos.d/xCAT-{core,dep,dep-common}.repo{,.nouse} + rm -f /etc/zypp/repos.d/xcat-{core,dep,dep-common}.repo rm -rf /install/postscripts rm -rf /opt/xcat rm -rf /root/.xcat @@ -2002,7 +2090,12 @@ function uninstall_xcat_completely() function list_xcat_packages() { GO_XCAT_CORE_PACKAGE_LIST=($(get_package_list xcat-core)) - GO_XCAT_DEP_PACKAGE_LIST=($(get_package_list xcat-dep)) + GO_XCAT_DEP_PACKAGE_LIST=($( + for repo_id in "${GO_XCAT_DEP_REPOSITORY_IDS[@]}" + do + get_package_list "${repo_id}" + done | tr ' ' '\n' | sort -u + )) [ "${#GO_XCAT_CORE_PACKAGE_LIST[@]}" -gt "0" ] warn_if_bad "$?" "Failed to get package list from repository \`xcat-core'." || return 1 @@ -2435,6 +2528,7 @@ ERR_MSG="$( "${GO_XCAT_DEP_URL}" "${GO_XCAT_VERSION}" then debug_trace update_repo && exit 0 + debug_trace remove_repo "xcat-dep-common" debug_trace remove_repo "xcat-dep" fi debug_trace remove_repo "xcat-core" @@ -2450,6 +2544,7 @@ then boo_boo_if_bad "${RET}" fi echo "done" +refresh_xcat_dep_repository_ids case "${GO_XCAT_ACTION}" in "check") diff --git a/xCAT-test/unit/anaconda_genesis_boot_files.t b/xCAT-test/unit/anaconda_genesis_boot_files.t new file mode 100644 index 000000000..6b7c6286f --- /dev/null +++ b/xCAT-test/unit/anaconda_genesis_boot_files.t @@ -0,0 +1,53 @@ +#!/usr/bin/env perl +use strict; +use warnings; + +use File::Path qw(make_path); +use File::Slurper qw(write_text); +use File::Temp qw(tempdir); +use FindBin; +use Test::More; + +BEGIN { + $ENV{XCATROOT} = "$FindBin::Bin/../../xCAT-server"; +} + +use lib "$FindBin::Bin/../../perl-xCAT"; +use lib "$FindBin::Bin/../../xCAT-server/lib/perl"; +use lib "$FindBin::Bin/../../xCAT-server/lib/xcat/plugins"; + +require anaconda; + +my $tftp = tempdir(CLEANUP => 1); +make_path("$tftp/xcat"); +for my $path (qw( + genesis.kernel.ppc64 + genesis.kernel.ppc64le + genesis.fs.ppc64.lzma + genesis.fs.ppc64le.gz +)) { + write_text("$tftp/xcat/$path", "test\n"); +} + +my ($kernel, $initrd) = + xCAT_plugin::anaconda::_find_genesis_boot_files($tftp, 'ppc64'); +is($kernel, 'genesis.kernel.ppc64', + 'the legacy POWER kernel lookup is an exact architecture match'); +is($initrd, 'genesis.fs.ppc64.lzma', + 'the legacy POWER initramfs lookup is an exact architecture match'); +unlike($kernel, qr/\n/, 'the selected kernel path contains one file'); + +($kernel, $initrd) = + xCAT_plugin::anaconda::_find_genesis_boot_files($tftp, 'ppc64le'); +is($kernel, 'genesis.kernel.ppc64le', + 'the OpenEmbedded POWER kernel remains independently selectable'); +is($initrd, 'genesis.fs.ppc64le.gz', + 'the OpenEmbedded POWER initramfs remains independently selectable'); + +unlink("$tftp/xcat/genesis.fs.ppc64le.gz"); +($kernel, $initrd) = + xCAT_plugin::anaconda::_find_genesis_boot_files($tftp, 'ppc64le'); +is($kernel, undef, 'a missing initramfs yields no partial boot selection'); +is($initrd, undef, 'a missing initramfs leaves both paths undefined'); + +done_testing(); diff --git a/xCAT-test/unit/destiny_genesis_architecture.t b/xCAT-test/unit/destiny_genesis_architecture.t new file mode 100644 index 000000000..e98aa678d --- /dev/null +++ b/xCAT-test/unit/destiny_genesis_architecture.t @@ -0,0 +1,71 @@ +#!/usr/bin/env perl +use strict; +use warnings; + +use File::Path qw(make_path); +use File::Temp qw(tempdir); +use FindBin; +use Test::More; + +my $source = "$FindBin::Bin/../../xCAT-server/lib/xcat/plugins/destiny.pm"; +open(my $source_fh, '<', $source) or die "open $source: $!"; +my $content = do { local $/; <$source_fh> }; +close($source_fh) or die "close $source: $!"; + +my @routines; +for my $name (qw(_genesis_boot_arch _genesis_uses_power_console)) { + my ($routine) = $content =~ /^(sub \Q$name\E\s*\{.*?^\})/ms; + BAIL_OUT("could not extract $name from destiny.pm") unless $routine; + push(@routines, $routine); +} +eval join("\n", @routines); ## no critic (BuiltinFunctions::ProhibitStringyEval) +BAIL_OUT("could not load Genesis architecture helpers: $@") if $@; + +my $tftp = tempdir(CLEANUP => 1); +make_path("$tftp/xcat"); + +is( + _genesis_boot_arch($tftp, 'ppc64le'), + 'ppc64', + 'ppc64le keeps the legacy POWER fallback when no exact image exists', +); + +open(my $marker_fh, '>', "$tftp/xcat/genesis.exact-arch.ppc64") + or die "create exact POWER marker: $!"; +close($marker_fh) or die "close exact POWER marker: $!"; + +is( + _genesis_boot_arch($tftp, 'ppc64le'), + 'ppc64le', + 'ppc64le does not fall back to a canonical big-endian ppc64 image', +); +unlink("$tftp/xcat/genesis.exact-arch.ppc64") + or die "remove exact POWER marker: $!"; + +open(my $kernel_fh, '>', "$tftp/xcat/genesis.kernel.ppc64le") + or die "create exact POWER kernel: $!"; +close($kernel_fh) or die "close exact POWER kernel: $!"; + +is( + _genesis_boot_arch($tftp, 'ppc64le'), + 'ppc64le', + 'ppc64le uses the exact OpenEmbedded boot artifact', +); +is( + _genesis_boot_arch($tftp, 'ppc64el'), + 'ppc64le', + 'the Debian spelling resolves to the exact ppc64le artifact', +); +is(_genesis_boot_arch($tftp, 'x86_64'), 'x86_64', 'other architectures are unchanged'); + +ok(_genesis_uses_power_console('ppc64'), 'legacy POWER uses the hypervisor console'); +ok(_genesis_uses_power_console('ppc64le'), 'ppc64le uses the hypervisor console'); +ok(!_genesis_uses_power_console('x86_64'), 'x86_64 keeps the serial console path'); + +like( + $content, + qr/my \$arch = _genesis_boot_arch\(\$tftpdir, \$ent->\{arch\}\);/, + 'destiny applies the tested architecture selection to nodeset', +); + +done_testing(); diff --git a/xCAT-test/unit/genesis_openembedded_activation.t b/xCAT-test/unit/genesis_openembedded_activation.t new file mode 100644 index 000000000..8294c57df --- /dev/null +++ b/xCAT-test/unit/genesis_openembedded_activation.t @@ -0,0 +1,163 @@ +#!/usr/bin/env perl +use strict; +use warnings; + +use FindBin; +use lib "$FindBin::Bin/../lib"; +use Test::More; + +use XCAT::Test::File qw(slurp_repo_file); + +my $rpm_weak_dependencies = join( + "\n", + '%if 0%{?fedora} || 0%{?rhel} >= 8 || 0%{?suse_version} >= 1500', + 'Recommends: xCAT-genesis-openembedded-x86_64', + 'Recommends: xCAT-genesis-openembedded-ppc64le', + '%endif', +); + +my $rpm_spec = slurp_repo_file('xCAT/xCAT.spec'); +like( + $rpm_spec, + qr/^\Q$rpm_weak_dependencies\E$/m, + 'RPM weak dependencies stay inside their compatibility guard', +); +unlike( + $rpm_spec, + qr/^Requires:\s+xCAT-genesis-openembedded-/m, + 'missing OpenEmbedded images do not block an RPM upgrade', +); + +my $deb_control = slurp_repo_file('xCAT/debian/control'); +like( + $deb_control, + qr/^Recommends:.*\bxcat-genesis-openembedded-x86-64\b/m, + 'DEB installations recommend the first-class x86_64 image', +); +like( + $deb_control, + qr/^Recommends:.*\bxcat-genesis-openembedded-ppc64le\b/m, + 'DEB installations recommend the first-class ppc64le image', +); +unlike( + $deb_control, + qr/^Depends:.*\bxcat-genesis-openembedded-/m, + 'missing OpenEmbedded images do not block a DEB upgrade', +); + +my $sn_rpm_spec = slurp_repo_file('xCATsn/xCATsn.spec'); +like( + $sn_rpm_spec, + qr/^\Q$rpm_weak_dependencies\E$/m, + 'service-node weak dependencies stay inside their compatibility guard', +); + +my $sn_deb_control = slurp_repo_file('xCATsn/debian/control'); +like( + $sn_deb_control, + qr/^Recommends:.*\bxcat-genesis-openembedded-x86-64\b/m, + 'DEB service nodes recommend the first-class x86_64 image', +); +like( + $sn_deb_control, + qr/^Recommends:.*\bxcat-genesis-openembedded-ppc64le\b/m, + 'DEB service nodes recommend the first-class ppc64le image', +); + +my $go_xcat = slurp_repo_file('xCAT-server/share/xcat/tools/go-xcat'); +unlike( + $go_xcat, + qr/GO_XCAT_LIBRARY_ONLY/, + 'go-xcat cannot be disabled by an inherited test environment variable', +); +for my $architecture (qw(x86 x86_64 ppc64 ppc64le armv7hf aarch64 riscv64)) { + like( + $go_xcat, + qr/\bxCAT-genesis-openembedded-\Q$architecture\E\b/, + "go-xcat removes the $architecture RPM image", + ); + (my $deb_architecture = $architecture) =~ tr/_/-/; + like( + $go_xcat, + qr/\bxcat-genesis-openembedded-\Q$deb_architecture\E\b/, + "go-xcat removes the $architecture DEB image", + ); +} + +my $mknb_pod = slurp_repo_file('xCAT-client/pods/man8/mknb.8.pod'); +like( + $mknb_pod, + qr{/opt/xcat/share/xcat/netboot/genesis-openembedded/ARCH}, + 'the mknb man page documents the OpenEmbedded install namespace', +); +like( + $mknb_pod, + qr/x86.*x86_64.*ppc64.*ppc64le.*armv7hf.*aarch64.*riscv64/s, + 'the mknb man page lists every exact OpenEmbedded architecture', +); +unlike( + $mknb_pod, + qr/For ppc64le, use the ppc64 architecture/, + 'the mknb man page no longer presents ppc64 as the ppc64le name', +); + +my $offline_guide = + slurp_repo_file('docs/source/guides/install-guides/common_sections.rst'); +like( + $offline_guide, + qr/reposync.*--repofrompath=xcat-dep-common,https:\/\/xcat\.org\/.*\/xcat-dep\/common.*--repoid=xcat-dep-common.*--download-metadata/s, + 'the offline mirror command defines the common repository itself', +); +like( + $offline_guide, + qr{repodata/repomd\.xml\.asc}s, + 'the offline mirror includes the repository metadata signature', +); +like( + $offline_guide, + qr{repodata/repomd\.xml\.key}s, + 'the offline mirror includes its signing key', +); +like( + $offline_guide, + qr{baseurl=file://.*xcat-dep/common}s, + 'the offline guide points a repository file at the mirror', +); +like( + $offline_guide, + qr/dnf makecache.*--enablerepo=xcat-dep-common/s, + 'the offline guide verifies the mirrored common repository', +); +unlike( + $offline_guide, + qr/install .*xCAT-release.*local xcat-core/is, + 'the offline guide does not assume xCAT-release is in the core tarball', +); +unlike( + $offline_guide, + qr/Run .*mklocalrepo\.sh.*mirrored directory/is, + 'the offline guide does not depend on files reposync cannot download', +); + +my $yum_guide = + slurp_repo_file('docs/source/guides/install-guides/yum/configure_xcat.rst'); +like( + $yum_guide, + qr/start-after: BEGIN_configure_xcat_local_repo_xcat-dep_DNF/, + 'the DNF guide includes the DNF common repository steps', +); + +my $zypper_guide = + slurp_repo_file('docs/source/guides/install-guides/zypper/configure_xcat.rst'); +like( + $zypper_guide, + qr/start-after: BEGIN_configure_xcat_local_repo_xcat-dep_ZYPPER/, + 'the zypper guide includes its own common repository steps', +); +unlike( + $zypper_guide, + qr/start-after: BEGIN_configure_xcat_local_repo_xcat-dep_DNF/, + 'the zypper guide does not include DNF-only setup', +); + +done_testing(); diff --git a/xCAT-test/unit/go_xcat_common_repository.t b/xCAT-test/unit/go_xcat_common_repository.t new file mode 100644 index 000000000..21931d248 --- /dev/null +++ b/xCAT-test/unit/go_xcat_common_repository.t @@ -0,0 +1,165 @@ +#!/usr/bin/env perl +use strict; +use warnings; + +use File::Path qw(make_path); +use File::Temp qw(tempdir); +use FindBin; +use Test::More; + +my $go_xcat = "$FindBin::Bin/../../xCAT-server/share/xcat/tools/go-xcat"; +my $tmpdir = tempdir(CLEANUP => 1); +my $driver = "$tmpdir/driver.sh"; + +open(my $driver_fh, '>', $driver) or die "open $driver: $!"; +print {$driver_fh} <<'DRIVER'; +#!/bin/bash +set -euo pipefail + +function_body=$( + for function_name in \ + add_xcat_dep_common_repo_yum_or_zypper \ + xcat_dep_common_repo_configured \ + refresh_xcat_dep_repository_ids + do + awk -v name="$function_name" ' + $0 == "function " name "()" { copy = 1 } + copy { print } + copy && /^}$/ { exit } + ' "$GO_XCAT_SOURCE" + done +) +eval "$function_body" + +TMP_DIR=$TEST_TMP +GO_XCAT_DEFAULT_BASE_URL=https://repo.example.invalid +GO_XCAT_DEP_REPOSITORY_IDS=(xcat-dep) + +yum() { :; } + +download_file() { + printf '%s\n' "$1" >>"$DOWNLOAD_LOG" + [[ ${COMMON_PRESENT:-0} == 1 ]] || return 1 + : >"$2" +} + +add_repo_by_url_yum_or_zypper() { + printf '%s %s\n' "$1" "$2" >>"$ADD_LOG" +} + +xcat_dep_common_repo_configured() { [[ -s "$ADD_LOG" ]]; } +( add_xcat_dep_common_repo_yum_or_zypper "$@" ) +refresh_xcat_dep_repository_ids +printf '%s\n' "${GO_XCAT_DEP_REPOSITORY_IDS[*]}" >"$ID_LOG" +DRIVER +close($driver_fh) or die "close $driver: $!"; +chmod(0755, $driver) or die "chmod $driver: $!"; + +sub run_case { + my ($name, $present, @arguments) = @_; + my $case_dir = "$tmpdir/$name"; + make_path($case_dir); + local %ENV = ( + %ENV, + ADD_LOG => "$case_dir/add.log", + COMMON_PRESENT => $present, + DOWNLOAD_LOG => "$case_dir/download.log", + GO_XCAT_SOURCE => $go_xcat, + ID_LOG => "$case_dir/id.log", + TEST_TMP => $case_dir, + ); + my $status = system('bash', $driver, @arguments); + return ($status >> 8, $case_dir); +} + +sub read_file { + my ($path) = @_; + return '' unless -f $path; + open(my $fh, '<', $path) or die "open $path: $!"; + my $content = do { local $/; <$fh> }; + close($fh) or die "close $path: $!"; + return $content; +} + +my ($status, $case_dir) = run_case('remote-present', 1, '', 'latest'); +is($status, 0, 'an available common repository is accepted'); +is( + read_file("$case_dir/download.log"), + "https://repo.example.invalid/yum/latest/xcat-dep/common/repodata/repomd.xml\n", + 'the default repository is probed before it is enabled', +); +is( + read_file("$case_dir/add.log"), + "https://repo.example.invalid/yum/latest/xcat-dep/common xcat-dep-common\n", + 'the common repository uses its own repository ID', +); +is(read_file("$case_dir/id.log"), "xcat-dep xcat-dep-common\n", + 'common packages are included in repository listings'); + +($status, $case_dir) = run_case('remote-missing', 0, '', '2.18'); +is($status, 0, 'a release without the common repository remains usable'); +is(read_file("$case_dir/add.log"), '', 'a missing common repository is not enabled'); +is(read_file("$case_dir/id.log"), "xcat-dep\n", + 'legacy package listings remain unchanged when common is absent'); + +($status, $case_dir) = run_case( + 'repo-file', 1, 'https://repo.example.invalid/custom/xcat-dep.repo', 'latest' +); +is($status, 0, 'a custom repository file remains supported'); +is(read_file("$case_dir/download.log"), '', + 'the common URL is not guessed from a custom repository file'); +is(read_file("$case_dir/add.log"), '', + 'a custom repository file does not enable an unrelated repository'); + +my $local_root = "$tmpdir/local-repository"; +make_path("$local_root/common/repodata"); +open(my $repomd_fh, '>', "$local_root/common/repodata/repomd.xml") or die $!; +close($repomd_fh) or die $!; +($status, $case_dir) = run_case('local-present', 0, $local_root, 'latest'); +is($status, 0, 'a local common repository is accepted'); +is( + read_file("$case_dir/add.log"), + "$local_root/common xcat-dep-common\n", + 'the local common repository is enabled beside the distribution repository', +); + +my $template_driver = "$tmpdir/template-driver.sh"; +open(my $template_fh, '>', $template_driver) or die "open $template_driver: $!"; +print {$template_fh} <<'DRIVER'; +#!/bin/bash +set -euo pipefail + +function_body=$( + awk ' + /^function add_repo_by_url_yum_or_zypper\(\)/ { copy = 1 } + copy { print } + copy && /^}$/ { exit } + ' "$GO_XCAT_SOURCE" +) +eval "$function_body" + +TMP_DIR=$TEST_TMP +GO_XCAT_DEFAULT_INSTALL_PATH=/install/xcat +yum() { :; } +add_repo_by_file() { cp "$1" "$REPO_LOG"; } +add_repo_by_url_yum_or_zypper \ + https://repo.example.invalid/xcat-dep/common xcat-dep-common optional +DRIVER +close($template_fh) or die "close $template_driver: $!"; +chmod(0755, $template_driver) or die "chmod $template_driver: $!"; + +my $template_log = "$tmpdir/generated-common.repo"; +local %ENV = ( + %ENV, + GO_XCAT_SOURCE => $go_xcat, + REPO_LOG => $template_log, + TEST_TMP => $tmpdir, +); +$status = system('bash', $template_driver); +is($status >> 8, 0, 'go-xcat can generate the optional common repository'); +like(read_file($template_log), qr/^skip_if_unavailable=1$/m, + 'the generated common repository stays optional during outages'); +like(read_file($template_log), qr/^repo_gpgcheck=1$/m, + 'the generated common repository verifies signed metadata'); + +done_testing(); diff --git a/xCAT-test/unit/mknb_exported_genesis.t b/xCAT-test/unit/mknb_exported_genesis.t index c864b4a2e..faaf980a2 100644 --- a/xCAT-test/unit/mknb_exported_genesis.t +++ b/xCAT-test/unit/mknb_exported_genesis.t @@ -4,7 +4,7 @@ use warnings; ## no critic (Modules::RequireFilenameMatchesPackage, TestingAndDebugging::ProhibitNoStrict, TestingAndDebugging::ProhibitNoWarnings) use Digest::SHA qw(sha256_hex); -use File::Path qw(make_path); +use File::Path qw(make_path remove_tree); use File::Temp qw(tempdir); use FindBin; use Test::More; @@ -106,6 +106,8 @@ my $tmpdir = tempdir(CLEANUP => 1); my $export = "$tmpdir/export"; my $tftpdir = "$tmpdir/tftpboot"; prepare_export($export, 'new kernel', 'new initramfs'); +make_path("$tftpdir/xcat"); +write_file("$tftpdir/xcat/genesis.fs.x86_64.lzma", 'old initramfs'); ok( xCAT_plugin::mknb::_prebuilt_genesis_requested($export), @@ -124,10 +126,61 @@ my ($published_kernel, $published_initramfs) = published_files($tftpdir, 'x86_64'); is(read_file($published_kernel), 'new kernel', 'the kernel is published'); is(read_file($published_initramfs), 'new initramfs', 'the initramfs is published'); +ok(!-e "$tftpdir/xcat/genesis.fs.x86_64.lzma", + 'installing an export removes the obsolete legacy initramfs'); +is( + read_file("$tftpdir/xcat/genesis.exact-arch.x86_64"), + export_manifest('x86_64'), + 'the canonical architecture marker is published with the image', +); is(sprintf('%04o', (stat($published_kernel))[2] & oct('7777')), '0644', 'the kernel is readable by TFTP'); is(sprintf('%04o', (stat($published_initramfs))[2] & oct('7777')), '0644', 'the initramfs is readable by TFTP'); is_deeply([temporary_files("$tftpdir/xcat")], [], 'staging files are removed'); +write_file("$tftpdir/xcat/genesis.kernel.ppc64le", 'stale kernel'); +write_file("$tftpdir/xcat/genesis.fs.ppc64le.gz", 'stale initramfs'); +write_file("$tftpdir/xcat/genesis.fs.ppc64le.lzma", 'stale legacy initramfs'); +write_file("$tftpdir/xcat/genesis.exact-arch.ppc64le", export_manifest('ppc64le')); +write_file("$tftpdir/xcat/genesis.kernel.ppc64", 'legacy kernel'); +my ($removed, $remove_error) = + xCAT_plugin::mknb::_remove_openembedded_genesis($tftpdir, 'ppc64le'); +is($remove_error, undef, 'OpenEmbedded boot artifacts can be retired cleanly'); +is($removed, 4, 'all exact-architecture boot artifacts are retired'); +ok(!-e "$tftpdir/xcat/genesis.kernel.ppc64le", + 'the exact OpenEmbedded kernel is removed'); +ok(!-e "$tftpdir/xcat/genesis.fs.ppc64le.gz" + && !-e "$tftpdir/xcat/genesis.fs.ppc64le.lzma", + 'the exact OpenEmbedded initramfs files are removed'); +ok(!-e "$tftpdir/xcat/genesis.exact-arch.ppc64le", + 'the canonical architecture marker is removed'); +ok(-f "$tftpdir/xcat/genesis.kernel.ppc64", + 'retiring ppc64le does not remove the legacy ppc64 fallback'); + +my $failed_kernel = "$tftpdir/xcat/genesis.kernel.aarch64"; +make_path($failed_kernel); +for my $artifact (qw( + genesis.fs.aarch64.gz + genesis.fs.aarch64.lzma + genesis.exact-arch.aarch64 +)) { + write_file("$tftpdir/xcat/$artifact", 'stale artifact'); +} +($removed, $remove_error) = + xCAT_plugin::mknb::_remove_openembedded_genesis($tftpdir, 'aarch64'); +is($removed, 3, 'artifact cleanup continues after an unlink failure'); +like( + $remove_error, + qr/Unable to remove Genesis artifact: \Q$failed_kernel\E/, + 'artifact cleanup reports the failed path', +); +ok(-d $failed_kernel, 'the failed artifact remains in place'); +ok( + !-e "$tftpdir/xcat/genesis.fs.aarch64.gz" + && !-e "$tftpdir/xcat/genesis.fs.aarch64.lzma" + && !-e "$tftpdir/xcat/genesis.exact-arch.aarch64", + 'artifact cleanup removes every remaining path', +); + write_file($published_kernel, 'current kernel'); write_file($published_initramfs, 'current initramfs'); write_file("$export/kernel", 'corrupt kernel'); @@ -332,4 +385,133 @@ is( 'an incomplete marked export keeps the published initramfs', ); +$::XCATROOT = "$tmpdir/openembedded-xcatroot"; +my $openembedded_process_export = + "$::XCATROOT/share/xcat/netboot/genesis-openembedded/ppc64le"; +my $legacy_process_export = + "$::XCATROOT/share/xcat/netboot/genesis/ppc64"; +prepare_export( + $openembedded_process_export, + 'openembedded ppc64le kernel', + 'openembedded ppc64le initramfs', + 'ppc64le', +); +prepare_export( + $legacy_process_export, + 'legacy ppc64 kernel', + 'legacy ppc64 initramfs', + 'ppc64', +); +$xCAT::TableUtils::tftpdir = "$tmpdir/openembedded-tftpboot"; +@responses = (); +xCAT_plugin::mknb::process_request( + { arg => ['ppc64le'] }, + sub { push(@responses, @_); }, +); +ok( + !grep({ ref($_) eq 'HASH' && $_->{error} } @responses), + 'mknb installs an exact ppc64le OpenEmbedded export', +); +my ($openembedded_kernel, $openembedded_initramfs) = + published_files($xCAT::TableUtils::tftpdir, 'ppc64le'); +is(read_file($openembedded_kernel), 'openembedded ppc64le kernel', + 'mknb prefers the OpenEmbedded kernel when legacy Genesis is also installed'); +is(read_file($openembedded_initramfs), 'openembedded ppc64le initramfs', + 'mknb prefers the OpenEmbedded initramfs when legacy Genesis is also installed'); +my $ppc64le_config = read_file( + "$xCAT::TableUtils::tftpdir/pxelinux.cfg/p/192.0.2.0_24" +); +like($ppc64le_config, qr/genesis\.kernel\.ppc64le/, + 'POWER boot configuration keeps the exact ppc64le artifact name'); +like($ppc64le_config, qr/genesis\.fs\.ppc64le\.gz/, + 'POWER boot configuration uses the exact ppc64le initramfs name'); + +remove_tree($openembedded_process_export); +write_file( + "$xCAT::TableUtils::tftpdir/xcat/genesis.kernel.ppc64", + 'canonical big-endian ppc64 kernel', +); +write_file( + "$xCAT::TableUtils::tftpdir/xcat/genesis.exact-arch.ppc64", + export_manifest('ppc64'), +); +@responses = (); +xCAT_plugin::mknb::process_request( + { arg => ['ppc64le'] }, + sub { push(@responses, @_); }, +); +ok( + grep( + { ref($_) eq 'HASH' && $_->{error} + && $_->{error} =~ /canonical ppc64 image/ } + @responses + ), + 'the legacy ppc64le fallback cannot replace a canonical ppc64 image', +); +is( + read_file("$xCAT::TableUtils::tftpdir/xcat/genesis.kernel.ppc64"), + 'canonical big-endian ppc64 kernel', + 'a refused ppc64le fallback leaves the canonical ppc64 kernel intact', +); + +my $selection_root = "$tmpdir/selection-root"; +my $legacy_x86 = "$selection_root/share/xcat/netboot/genesis/x86_64"; +my $openembedded_x86 = + "$selection_root/share/xcat/netboot/genesis-openembedded/x86_64"; +make_path("$legacy_x86/fs", $openembedded_x86); + +my ($selected_dir, $selected_arch, $selected_type) = + xCAT_plugin::mknb::_select_genesis_source($selection_root, 'x86_64'); +is($selected_dir, $openembedded_x86, + 'an installed OpenEmbedded export takes precedence over legacy Genesis'); +is($selected_arch, 'x86_64', 'the OpenEmbedded x86_64 name is unchanged'); +is($selected_type, 'openembedded', 'the selected source is identified'); + +remove_tree($openembedded_x86); +($selected_dir, $selected_arch, $selected_type) = + xCAT_plugin::mknb::_select_genesis_source($selection_root, 'x86_64'); +is($selected_dir, $legacy_x86, 'legacy Genesis remains the fallback'); +is($selected_arch, 'x86_64', 'the legacy x86_64 name is unchanged'); +is($selected_type, 'legacy', 'the fallback source is identified'); + +my $linked_legacy = "$tmpdir/linked-legacy-x86_64"; +make_path("$linked_legacy/fs"); +remove_tree($legacy_x86); +symlink($linked_legacy, $legacy_x86) + or die "Unable to create legacy Genesis directory symlink: $!"; +($selected_dir, $selected_arch, $selected_type) = + xCAT_plugin::mknb::_select_genesis_source($selection_root, 'x86_64'); +is($selected_dir, $legacy_x86, 'legacy Genesis directory symlinks remain usable'); +is($selected_type, 'legacy', 'a linked legacy source keeps its source type'); + +my $openembedded_ppc64le = + "$selection_root/share/xcat/netboot/genesis-openembedded/ppc64le"; +my $legacy_ppc64 = "$selection_root/share/xcat/netboot/genesis/ppc64"; +make_path($openembedded_ppc64le, "$legacy_ppc64/fs"); +($selected_dir, $selected_arch, $selected_type) = + xCAT_plugin::mknb::_select_genesis_source($selection_root, 'ppc64le'); +is($selected_dir, $openembedded_ppc64le, + 'ppc64le selects its exact OpenEmbedded export'); +is($selected_arch, 'ppc64le', 'ppc64le is not rewritten to ppc64'); +is($selected_type, 'openembedded', 'ppc64le uses the OpenEmbedded source'); + +($selected_dir, $selected_arch, $selected_type) = + xCAT_plugin::mknb::_select_genesis_source($selection_root, 'ppc64el'); +is($selected_dir, $openembedded_ppc64le, + 'the Debian spelling resolves to the canonical ppc64le export'); +is($selected_arch, 'ppc64le', 'the canonical architecture name is returned'); + +remove_tree($openembedded_ppc64le); +($selected_dir, $selected_arch, $selected_type) = + xCAT_plugin::mknb::_select_genesis_source($selection_root, 'ppc64le'); +is($selected_dir, $legacy_ppc64, + 'ppc64le falls back to the old combined POWER image when needed'); +is($selected_arch, 'ppc64', 'only the legacy fallback uses the old ppc64 name'); +is($selected_type, 'legacy', 'the POWER fallback is identified as legacy'); + +my $unsafe = xCAT_plugin::mknb::_select_genesis_source( + $selection_root, '../../outside' +); +is($unsafe, undef, 'unsupported architecture names cannot escape the image root'); + done_testing(); diff --git a/xCAT-test/unit/sles_genesis_boot_files.t b/xCAT-test/unit/sles_genesis_boot_files.t new file mode 100644 index 000000000..061097249 --- /dev/null +++ b/xCAT-test/unit/sles_genesis_boot_files.t @@ -0,0 +1,64 @@ +#!/usr/bin/env perl +use strict; +use warnings; + +use File::Path qw(make_path); +use File::Temp qw(tempdir); +use FindBin; +use Test::More; +use Time::HiRes qw(sleep); + +BEGIN { + $ENV{XCATROOT} = "$FindBin::Bin/../../xCAT-server"; +} + +use lib "$FindBin::Bin/../../perl-xCAT"; +use lib "$FindBin::Bin/../../xCAT-server/lib/perl"; +use lib "$FindBin::Bin/../../xCAT-server/lib/xcat/plugins"; + +require sles; + +sub write_file { + my ($path) = @_; + open(my $fh, '>', $path) or die "open $path: $!"; + print {$fh} "test\n"; + close($fh) or die "close $path: $!"; +} + +my $tftp = tempdir(CLEANUP => 1); +make_path("$tftp/xcat"); + +my $kernel = "$tftp/xcat/genesis.kernel.x86_64"; +my $lzma = "$tftp/xcat/genesis.fs.x86_64.lzma"; +my $gzip = "$tftp/xcat/genesis.fs.x86_64.gz"; +write_file($_) for ($kernel, $lzma); +sleep(1.1); +write_file($gzip); + +my ($selected_kernel, $selected_initrd) = + xCAT_plugin::sles::_find_genesis_boot_files($tftp, 'x86_64'); +is($selected_kernel, 'genesis.kernel.x86_64', + 'SLES selects the matching Genesis kernel'); +is($selected_initrd, 'genesis.fs.x86_64.gz', + 'SLES selects the newer gzip initramfs'); + +sleep(1.1); +write_file($lzma); +($selected_kernel, $selected_initrd) = + xCAT_plugin::sles::_find_genesis_boot_files($tftp, 'x86_64'); +is($selected_initrd, 'genesis.fs.x86_64.lzma', + 'SLES keeps a newer legacy initramfs usable'); + +unlink($gzip); +($selected_kernel, $selected_initrd) = + xCAT_plugin::sles::_find_genesis_boot_files($tftp, 'x86_64'); +is($selected_initrd, 'genesis.fs.x86_64.lzma', + 'SLES accepts the legacy initramfs by itself'); + +unlink($lzma); +($selected_kernel, $selected_initrd) = + xCAT_plugin::sles::_find_genesis_boot_files($tftp, 'x86_64'); +is($selected_kernel, undef, 'a missing initramfs yields no partial boot selection'); +is($selected_initrd, undef, 'a missing initramfs leaves both paths undefined'); + +done_testing(); diff --git a/xCAT-test/unit/xcat_release_package.t b/xCAT-test/unit/xcat_release_package.t index 1f4cbcf99..b8e1f55e4 100644 --- a/xCAT-test/unit/xcat_release_package.t +++ b/xCAT-test/unit/xcat_release_package.t @@ -16,6 +16,7 @@ like($spec, qr/^BuildArch:\s+noarch$/m, 'package is architecture independent'); like($spec, qr/^Requires:\s+dnf$/m, 'package is limited to DNF-based systems'); like($spec, qr/^%config\(noreplace\) .*xcat-core\.repo$/m, 'core repo preserves local changes'); like($spec, qr/^%config\(noreplace\) .*xcat-dep\.repo$/m, 'dependency repo preserves local changes'); +like($spec, qr/^%config\(noreplace\) .*xcat-dep-common\.repo$/m, 'common dependency repo preserves local changes'); like($spec, qr{RPM-GPG-KEY-xCAT}, 'package installs the signing key'); my $core = read_file('xCAT-release/xcat-core.repo'); @@ -34,6 +35,19 @@ like( 'dependency repo follows the DNF release and architecture variables' ); +my $common_dep = read_file('xCAT-release/xcat-dep-common.repo'); +assert_repo_security($common_dep, 'common dependency'); +like( + $common_dep, + qr/^skip_if_unavailable=1$/m, + 'an unavailable common repository does not block package operations', +); +like( + $common_dep, + qr{^baseurl=https://xcat\.org/files/xcat/repos/yum/latest/xcat-dep/common$}m, + 'common dependency repo is independent of the management-node distribution' +); + my $key = read_file('xCAT-release/RPM-GPG-KEY-xCAT'); like($key, qr/^-----BEGIN PGP PUBLIC KEY BLOCK-----$/m, 'signing key is ASCII armored'); is( diff --git a/xCAT-test/unit/xcatconfig_genesis_openembedded.t b/xCAT-test/unit/xcatconfig_genesis_openembedded.t new file mode 100644 index 000000000..b59d6eba6 --- /dev/null +++ b/xCAT-test/unit/xcatconfig_genesis_openembedded.t @@ -0,0 +1,74 @@ +#!/usr/bin/env perl +use strict; +use warnings; + +use File::Path qw(make_path remove_tree); +use File::Temp qw(tempdir); +use FindBin; +use Test::More; + +my $source = "$FindBin::Bin/../../xCAT-server/sbin/xcatconfig"; +open(my $source_fh, '<', $source) or die "open $source: $!"; +my $content = do { local $/; <$source_fh> }; +close($source_fh) or die "close $source: $!"; + +my @routines; +for my $name (qw(_installed_genesis_architectures _genesis_architectures_to_build)) { + my ($routine) = $content =~ /^(sub \Q$name\E\s*\{.*?^\})/ms; + BAIL_OUT("could not extract $name from xcatconfig") unless $routine; + push(@routines, $routine); +} +eval join("\n", @routines); ## no critic (BuiltinFunctions::ProhibitStringyEval) +BAIL_OUT("could not load Genesis architecture helpers: $@") if $@; + +my $tmpdir = tempdir(CLEANUP => 1); +make_path( + "$tmpdir/share/xcat/netboot/genesis/ppc64/fs", + "$tmpdir/share/xcat/netboot/genesis/x86_64/fs", + "$tmpdir/share/xcat/netboot/genesis-openembedded/aarch64", + "$tmpdir/share/xcat/netboot/genesis-openembedded/ppc64le", + "$tmpdir/share/xcat/netboot/genesis-openembedded/x86_64", +); +my $symlink_target = "$tmpdir/openembedded-riscv64"; +make_path($symlink_target); +symlink( + $symlink_target, + "$tmpdir/share/xcat/netboot/genesis-openembedded/riscv64", +) or die "create OpenEmbedded directory symlink: $!"; + +is_deeply( + [ _installed_genesis_architectures($tmpdir) ], + [ qw(aarch64 ppc64 ppc64le x86_64) ], + 'xcatconfig ignores linked OpenEmbedded images and finds legacy images', +); + +is_deeply( + [ _genesis_architectures_to_build($tmpdir, 0) ], + [ qw(aarch64 ppc64le x86_64) ], + 'an ordinary xCAT update rebuilds only installed OpenEmbedded images', +); +is_deeply( + [ _genesis_architectures_to_build($tmpdir, 1) ], + [ qw(aarch64 ppc64 ppc64le x86_64) ], + 'a legacy package trigger rebuilds every installed Genesis image', +); + +make_path("$tmpdir/share/xcat/netboot/genesis-openembedded/unsupported"); +is_deeply( + [ _installed_genesis_architectures($tmpdir) ], + [ qw(aarch64 ppc64 ppc64le x86_64) ], + 'unknown directories are not passed to mknb', +); + +remove_tree("$tmpdir/share/xcat/netboot/genesis/ppc64"); +my $legacy_target = "$tmpdir/legacy-ppc64"; +make_path("$legacy_target/fs"); +symlink($legacy_target, "$tmpdir/share/xcat/netboot/genesis/ppc64") + or die "create legacy directory symlink: $!"; +is_deeply( + [ _installed_genesis_architectures($tmpdir) ], + [ qw(aarch64 ppc64 ppc64le x86_64) ], + 'legacy Genesis directory symlinks remain supported', +); + +done_testing(); diff --git a/xCAT/debian/control b/xCAT/debian/control index 742ea46c8..22cf3e165 100644 --- a/xCAT/debian/control +++ b/xCAT/debian/control @@ -10,7 +10,7 @@ Homepage: https://xcat.org/ Package: xcat Architecture: amd64 ppc64el Depends: ${perl:Depends}, goconserver(>= 0.3.3-snap000000000000), xcat-server (>= 2.13-snap000000000000), xcat-client (>= 2.13-snap000000000000), libdbd-sqlite3-perl, isc-dhcp-server | kea, bind9, apache2, nfs-kernel-server, libxml-parser-perl, rsync, tftpd-hpa, libnet-telnet-perl, xcat-genesis-scripts-amd64 (>= 2.13-snap000000000000) -Recommends: net-tools, nmap, kea, tftp-hpa, ipmitool-xcat (>= 1.8.17-1), syslinux[any-amd64], libsys-virt-perl, syslinux-xcat, xnba-undi, elilo-xcat, xcat-buildkit (>= 2.13-snap000000000000), xcat-probe (>= 2.13-snap000000000000) +Recommends: net-tools, nmap, kea, tftp-hpa, ipmitool-xcat (>= 1.8.17-1), syslinux[any-amd64], libsys-virt-perl, syslinux-xcat, xnba-undi, elilo-xcat, xcat-buildkit (>= 2.13-snap000000000000), xcat-probe (>= 2.13-snap000000000000), xcat-genesis-openembedded-x86-64, xcat-genesis-openembedded-ppc64le Suggests: yaboot-xcat Description: Metapackage for a common, default xCAT setup xCAT is Extreme Cluster/Cloud Administration Toolkit. xCAT offers complete diff --git a/xCAT/xCAT.spec b/xCAT/xCAT.spec index 93cc8d32e..998f8d6cd 100644 --- a/xCAT/xCAT.spec +++ b/xCAT/xCAT.spec @@ -55,6 +55,11 @@ Requires: xCAT-server = 4:%{version}-%{release} %if %nots390x Requires: xCAT-probe = 4:%{version}-%{release} Requires: xCAT-genesis-scripts-%{genesistarch} = 1:%{version}-%{release} +# RPM 4.11 does not recognize weak dependency tags. +%if 0%{?fedora} || 0%{?rhel} >= 8 || 0%{?suse_version} >= 1500 +Recommends: xCAT-genesis-openembedded-x86_64 +Recommends: xCAT-genesis-openembedded-ppc64le +%endif %endif Requires: rsync diff --git a/xCATsn/debian/control b/xCATsn/debian/control index 9f841dded..d66ea6c13 100644 --- a/xCATsn/debian/control +++ b/xCATsn/debian/control @@ -9,7 +9,7 @@ Homepage: https://xcat.org/ Package: xcatsn Architecture: amd64 ppc64el Depends: ${perl:Depends}, goconserver (>=0.3.3-snap000000000000), xcat-server (>= 2.13-snap000000000000), xcat-client (>= 2.13-snap000000000000), libdbd-sqlite3-perl, libxml-parser-perl, tftpd-hpa, libnet-telnet-perl, isc-dhcp-server | kea, bind9, apache2, nfs-kernel-server, xcat-genesis-scripts-amd64 (>= 2.13-snap000000000000) -Recommends: net-tools, nmap, kea, tftp-hpa, ipmitool-xcat (>= 1.8.17-1), syslinux[any-amd64], libsys-virt-perl, syslinux-xcat, xnba-undi, elilo-xcat, xcat-buildkit (>= 2.13-snap000000000000), xcat-probe (>= 2.13-snap000000000000) +Recommends: net-tools, nmap, kea, tftp-hpa, ipmitool-xcat (>= 1.8.17-1), syslinux[any-amd64], libsys-virt-perl, syslinux-xcat, xnba-undi, elilo-xcat, xcat-buildkit (>= 2.13-snap000000000000), xcat-probe (>= 2.13-snap000000000000), xcat-genesis-openembedded-x86-64, xcat-genesis-openembedded-ppc64le Suggests: yaboot-xcat Description: Metapackage for a common, default xCAT service node setup xCATsn is a service node management package intended for at-scale diff --git a/xCATsn/xCATsn.spec b/xCATsn/xCATsn.spec index b8c605082..d9585edc0 100644 --- a/xCATsn/xCATsn.spec +++ b/xCATsn/xCATsn.spec @@ -21,6 +21,11 @@ Requires: perl-DBD-SQLite Requires: xCAT-client = 4:%{version}-%{release} Requires: xCAT-server = 4:%{version}-%{release} Requires: xCAT-probe = 4:%{version}-%{release} +# RPM 4.11 does not recognize weak dependency tags. +%if 0%{?fedora} || 0%{?rhel} >= 8 || 0%{?suse_version} >= 1500 +Recommends: xCAT-genesis-openembedded-x86_64 +Recommends: xCAT-genesis-openembedded-ppc64le +%endif # Match xCAT-genesis-scripts package naming by build architecture. %ifarch i386 i586 i686 x86