copycds only defines a <osver>-<arch>-netboot-compute osimage when the netboot
compute pkglist/exlist/postinstall for that distro+arch exist under
share/xcat/netboot/<distro>/. For AlmaLinux the ppc64le set was present only for
alma10, so on an el8/el9 ppc64le management node copycds created just the
install-* osimages and no netboot one.
The stateless provisioning test then fails in a confusing way: the case's own
`chdef -t osimage -o <osver>-ppc64le-netboot-compute synclists=...` auto-creates
a bare object, so lsdef reports the image exists (imagetype=NIM and nothing
else), while genimage rejects it with
Error: Cannot find image '<osver>-ppc64le-netboot-compute' from the osimage table.
and packimage plus the install that follows fail with it.
Two gaps are filled:
- share/xcat/netboot/rh/compute.rhels9.ppc64le.pkglist did not exist at all
(rhels8 and rhels10 both ship one). Add it with the same content as the
rhels9 x86_64 list, matching rhels10 where the two arches are identical.
- alma8/alma9 ppc64le compute exlist/pkglist/postinstall are added as symlinks
into the rh/ equivalents, exactly as alma10 ppc64le and rocky10 ppc64le
already do.
With these present, copycds produces a fully populated netboot-compute osimage
(provmethod=netboot, pkglist/exlist/postinstall/rootimgdir set) as it already
does on alma10 ppc64le.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
(cherry picked from commit 8bb0606120)
Goconserver.pm shells out to `service goconserver start|stop|restart` (and
`service conserver stop`). That command is provided by the initscripts package,
which is NOT installed on a minimal EL9/EL10 management node -- there, every one
of these calls fails with rc!=0.
The damaging case is restart_service(). makegocons calls build_conf(), which
rewrites /etc/goconserver/server.conf with the cert-enabled configuration
(global.ssl_key_file/ssl_cert_file/ssl_ca_cert_file), and then calls
restart_service() so the daemon picks it up. When the restart fails, xCAT only
logs "Could not restart goconserver service." and the daemon keeps running with
the configuration it read at boot -- the packaged default, which has no ssl_*
fields. goconserver's TLS is gated on those fields being set (sslEnable stays
false), so it silently serves plain HTTP on the api port while Goconserver.pm
always talks to it over https. Every subsequent request then dies with
SSL connect attempt failed error:0A0000C6:SSL routines::packet length too long
so makegocons/makegocons -d report "Failed to send delete request." and the
console can never be registered. On EL8 the bug is invisible because initscripts
happens to be installed there.
Use the xCAT::Utils service helpers (startservice/stopservice/restartservice),
which resolve the unit through servicemap() and issue `systemctl <action> <unit>`
on systemd hosts, falling back to `service`/`initctl` only where appropriate.
This is the same mechanism the rest of the tree (AAsn.pm and friends) already
uses for named, dhcpd, nfs and others.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
(cherry picked from commit b42bbe0e02)
envvar() resolves a template variable purely from %ENV, so a template that
references $XCATROOT expands to an empty string whenever XCATROOT is not
present in the environment of the process doing the substitution. $::XCATROOT
is set as a package global at module load and is reliably available, so use it
for that one variable before falling back to %ENV.
Recovered from the unmerged lenovobuild branch (original da72c6b8).
Co-authored-by: Jarrod Johnson <10814490+jjohnson42@users.noreply.github.com>
(cherry picked from commit 34866ebc23)
`ip -6 route` output for ECMP/multipath routes carries `nexthop` continuation
lines, and a `default` route has no network prefix. donets() treated both as
networks and added bogus entries to the networks table. Skip them alongside
the existing link-local/unreachable/via/lo filtering.
Recovered from the unmerged lenovobuild branch (original 7094ba0c).
Co-authored-by: Jarrod Johnson <10814490+jjohnson42@users.noreply.github.com>
(cherry picked from commit c97e973f36)
donets() only skips the exact address 'fe80::/64' when scanning routes, so a
link-local route with any other prefix (e.g. a longer fe80:: subnet) is added
to the networks table as a bogus network. Match any fe80:: prefix instead of
the single literal value.
Recovered from the unmerged lenovobuild branch (original 8b79cf85).
Co-authored-by: Jarrod Johnson <10814490+jjohnson42@users.noreply.github.com>
(cherry picked from commit d2be0c3865)
nodech's ^= operator removed only a single literal value, so
"nodech n1 groups^=a,b" tried to strip the combined string "a,b" and left the
individual groups in place. Split the value on commas and remove each piece.
Recovered from the unmerged lenovobuild branch (original 95c78b33).
Co-authored-by: Jarrod Johnson <10814490+jjohnson42@users.noreply.github.com>
(cherry picked from commit f71ddef6ba)
nodech's ,= operator appended the whole right-hand side as one value, so
"nodech n1 groups,=a,b" added the literal "a,b" and its duplicate check only
compared against that combined string. Split the value on commas and add each
piece individually, skipping any already present.
Recovered from the unmerged lenovobuild branch (original 086b0c0d).
Co-authored-by: Jarrod Johnson <10814490+jjohnson42@users.noreply.github.com>
(cherry picked from commit 6e33661fc0)
getzonesfornet() derives the number of reverse-zone nibbles as
$nibbs = $maskbits / 4, then decrements it once per hex nibble of the network
prefix. For a sub-nibble (not 4-bit-aligned) IPv6 mask $nibbs can go negative
before the padding loop, and `while ($nibbs)` then never terminates: it keeps
decrementing past zero while appending "0." to $rev, spinning forever and
growing the string until the process is killed.
Make the padding loop `while ($nibbs > 0)` so it can never run away, and
return early only when $nibbs is genuinely negative. $nibbs == 0 is the normal
nibble-aligned case (e.g. a /64) and must still emit its reverse zone.
Recovered from the unmerged lenovobuild branch (original 0e070cd2). The
original guarded with `$nibbs < 1`, which also dropped the valid $nibbs == 0
case and left standard /64 subnets with no reverse zone; corrected to
`$nibbs < 0` after lab validation on a real provisioning cluster.
Co-authored-by: Jarrod Johnson <10814490+jjohnson42@users.noreply.github.com>
(cherry picked from commit f0a879491d)
mknb writes the compressed genesis filesystem by streaming cpio output
straight onto the final path in $tftpdir/xcat (genesis.fs.$arch.lzma or .gz).
When two mknb/nodeset runs target a shared $tftpdir at once, one run reads or
overwrites the other's half-written image, producing a corrupt genesis.fs and
failed netboots.
Write each image to a uniquely-suffixed temporary file
(xCAT::Utils::genpassword(24)) and File::Copy::move() it into place. move()
within the same directory is a rename, so the final path only ever appears
complete. Both modules are already imported by mknb.pm.
Recovered from the unmerged lenovobuild branch (originals c651e7b8 and its
fix da0723f2, which switched the suffix to genpassword after the first
attempt only produced a single character). master lacks the branch's xz
path, so only the lzma and gzip paths are adjusted here.
Co-authored-by: Jarrod Johnson <10814490+jjohnson42@users.noreply.github.com>
(cherry picked from commit 6da5efd29a)
replaycons chose the confluent console log directory
(/var/log/confluent/consoles) whenever the confetty binary happened to be
present. On a host that has confluent installed but runs conserver as the
active console service, that directory is empty and replaycons could not find
the log. Gate on the site table's consoleservice attribute instead -- the
same setting rcons uses -- so replaycons reads /var/log/confluent/consoles
only when confluent is actually the configured console service.
Recovered from the unmerged lenovobuild branch (originals 3ce4d81b3, 57f6ddd38).
Co-authored-by: Jarrod Johnson <10814490+jjohnson42@users.noreply.github.com>
(cherry picked from commit 21da014ed7)
Modern x86-64 UEFI firmware (including OVMF and recent iPXE) reports DHCP
client architecture 00:07, but the xNBA network-boot conditionals only matched
00:09. A UEFI node would load xnba.efi, re-request with user-class xNBA and
arch 00:07, fail to match, and be handed xnba.efi again -- an endless boot
loop. Accept 00:07 alongside 00:09 in the xNBA UEFI branches of both the
network-level (addnet) and node-level (addnode) dhcpd statements.
Validated on an OVMF UEFI client: it progresses from xnba.efi to the .uefi
genesis config and boots, instead of looping on xnba.efi.
Recovered from the unmerged lenovobuild branch (5d697b5c, 86b0e12e).
Co-authored-by: Jarrod Johnson <10814490+jjohnson42@users.noreply.github.com>
(cherry picked from commit 60bd6e7cd9)
The DNS-managed-networks check used my_ip_facing($net->{net}), which only
returns local addresses on the interface directly facing that network. A
nameserver that is a local address reachable via a different (routed)
interface was therefore treated as external, and the network was wrongly
dropped from ddns management.
Use thishostisnot(), which recognizes any local address regardless of the
interface, and keep the existing <xcatmaster> and site-nameserver special
cases.
Recovered from the unmerged lenovobuild branch (original 3c343f83), adapted
to preserve those special cases: lab validation showed the original replaced
them wholesale, which would exclude any network whose nameserver is the
literal <xcatmaster> token (thishostisnot('<xcatmaster>') is true).
Co-authored-by: Jarrod Johnson <10814490+jjohnson42@users.noreply.github.com>
(cherry picked from commit a0dfc3343a)
ddns.pm's DNS-update send has a 3-try retry loop, but it only retried on
NOTAUTH and -- on the final attempt -- the `next` fell out of the loop without
reporting failure. makedns then printed "Completed updating DNS records." and
"DNS setup is completed" even when the update was never accepted.
Two problems this causes:
- SERVFAIL was never retried. named routinely replies SERVFAIL transiently
right after a zone (re)load, before the zone is ready to accept dynamic
updates; the update succeeds a second or two later. This reliably breaks real
flat-cluster provisioning on el9/el10, where the test case's makedns -n runs
right after named reloads the zone with the new compute node.
- A persistent NOTAUTH/SERVFAIL (retries exhausted) was silently swallowed and
reported as success, so a genuinely failed DNS update looked like it worked.
Extract the batch send into send_ddns_update(), which retries NOTAUTH and
SERVFAIL (pausing on SERVFAIL), returns 0 only on NOERROR, and otherwise emits
the error and returns 1. Both batch-send sites now `return 1 if
send_ddns_update(...)`, so a real failure aborts add_or_delete_records instead
of falsely reporting success.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
(cherry picked from commit 9dd025bf05)
go-xcat's install list still named yaboot-xcat, which was obsoleted in the
xcat-dep 2.18 rebuild and is no longer published. It also lists the cross-arch
ppc64 genesis packages, which the arch-split 2.18 dep repo no longer places in
the x86_64 view an x86 management node reads. Because EL dnf is strict by
default, any one of these unresolvable names aborts the entire `dnf install`
transaction, so `go-xcat install` fails outright on EL9/EL10 even though every
package the node actually needs is available.
Drop yaboot-xcat from the install list (kept in the uninstall list so existing
installs are still cleaned up) and pass `dnf --setopt=strict=0` so a package
missing from the enabled repository is warned and skipped instead of failing
the whole install. The existing post-install smoke test still catches a
genuinely broken install.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
(cherry picked from commit 1b5938969c)
cons/kvm drives the KVM guest serial console through a shared tmux session on
the hypervisor (tmux, unlike screen, is present on every libvirt host we target
incl. ppc64le hosts that ship no screen). The KVM host package lists were never
updated for that switch, so an xCAT-provisioned KVM host got 'screen' (now
unused) -- or nothing -- where tmux is required, leaving rcons broken.
Ship tmux in the KVM host pkglists:
- install/rh, install/ubuntu: screen -> tmux / add tmux.
- netboot/{alma,centos,rocky}: were symlinks to the RHEL 5.4 list; make them
real lists with tmux (the rhel5/6/7 lists keep screen).
- install/suse: add a KVM host list (SUSE had none) carrying tmux.
screen stays where it is still used: the Xen console, and the EOL rhel5/6/7 and
centos6/7 lists.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
(cherry picked from commit c1db278e1a)
The diskful service-node otherpkgs pkglist exists for every EL version and
both arches EXCEPT rhels10 ppc64le: install/rh/ has
service.rhels10.x86_64.otherpkgs.pkglist (and install/alma/ symlinks it) but
no ppc64le counterpart, while rhels5/6/7/8/9 and the netboot/ tree carry both
arches. xcat-test's SN_setup_case points a ppc64le service node's osimage
otherpkglist at install/<distro>/service.<ver>.ppc64le.otherpkgs.pkglist; with
the file absent the list resolves empty, so the otherpkgs postscript installs
nothing -- xCATsn never lands, xcatd never starts on the service node, and
every hierarchical dispatch fails with "<sn>:3001 No route to host /
Connection refused", cascading the whole *_hierarchy test suite.
Add install/rh/service.rhels10.ppc64le.otherpkgs.pkglist (xCATsn + the rh10
ppc64le goconserver, mirroring rhels9) and the install/alma symlink to it,
matching the existing x86_64 layout.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
(cherry picked from commit 9ea3de343b)
cons/kvm ran `ssh -t <vmhost> screen ... <pty>`, which requires screen on every
hypervisor. Bare libvirt hosts (notably ppc64le power servers) ship tmux but not
screen, so goconserver's console fork failed and no console was captured.
Use tmux (present on all our libvirt hosts) for the shared detach/reattach
session, wrapping `virsh console` keyed on the domain name (== the xCAT node name
for KVM) -- stable across guest reboots, unlike the raw serial pty. The session
status bar is disabled so the captured console log stays clean serial.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
(cherry picked from commit e331290414)
build_xmldesc only recognised cpumodel "ppc64" (big-endian) when deciding to set
<os type arch='ppc64' machine='pseries'>, but ppc64le hosts report cpumodel
"ppc64le" (virNodeGetInfo model). A guest on a ppc64le hypervisor was therefore
emitted as an x86-style domain -- no machine type and with pae/acpi/apic -- and
libvirt rejected it: "machine type 'pseries-*' does not support ACPI".
Recognise "ppc64le" alongside "ppc64" for the arch/machine block, and omit the
x86-only pae/acpi/apic features on both. Little- and big-endian pseries guests
both use libvirt <os> arch "ppc64".
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
(cherry picked from commit 37666b92cd)
The single flat xcat-core repo serves el8/el9/el10 from one build, but two spec
decisions were resolved at BUILD time and so were wrong for the other ELs that
share the same repo.
1. DHCP backend. xCAT.spec and xCATsn.spec selected the dhcp provider with a
build-time %if (rhel >= 10 -> kea, else dhcpd), so an el10 build wrongly
required kea on el8/el9 (and an el8/el9 build wrongly required dhcpd on
el10). Replace it with an install-time RPM rich dependency:
Requires: (kea if (system-release >= 10) else /usr/sbin/dhcpd)
Requires: (kea-hooks if (system-release >= 10))
dnf now resolves it per client: kea on el10+ (which dropped ISC dhcp from the
distro), dhcpd on el8/el9. SLES has no "system-release" provide, so the
condition is false there and it falls back to /usr/sbin/dhcpd, preserving
prior behavior. system-release is versioned per release package
(el10=10.x, el9=9.x, el8=8.x).
2. openssl. Make the xCAT-server openssl Requires uniform across EL (non-SUSE)
instead of el10-only, so the produced package set does not depend on which EL
the build host happened to be.
Also drop xCAT-genesis-base from the default @PACKAGES set in buildrpms.pl: its
initramfs bundles the build-chroot kernel/glibc and is therefore OS- and
arch-dependent, so it cannot ship in the single flat core. It is now built per
target by the xcat-dep pipeline (mockbuild-all.pl, via
`buildrpms.pl --package xCAT-genesis-base`) and ships in xcat-dep/rh<N>. The
explicit `--package xCAT-genesis-base` build path is retained.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
(cherry picked from commit 529c821177)
The Ubuntu netboot image inherited the build host's /etc/hostname --
genimage's chroot shares the build host's UTS namespace, so /etc/hostname
ended up as the MN's name and every diskless node booted as that name
instead of its own ("hostname poisoning"). xCAT keys on IP/MAC so
provisioning still succeeds, but the node's local identity (hostname,
prompt, syslog) is wrong and N nodes share one name. The image also
shipped no netplan, so systemd-networkd-wait-online failed. In
compute.postinstall, clear /etc/hostname so each node takes its name at
boot from DHCP (udhcpc option host-name) / xCAT, and drop in a DHCP
netplan (optional, dhcp-identifier mac) so systemd-networkd configures the
boot NIC. Ubuntu-only.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
(cherry picked from commit 3a5cbef630)
The Ubuntu live-server ISO copycds imports is not a complete apt pool, so
both Ubuntu install paths fail without an external mirror: the netboot
genimage's debootstrap from the ISO-only pkgdir cannot resolve packages,
and the Subiquity diskful install (Template.pm) fell back to the airgapped
file:///cdrom config. Default site.ubuntu_apt_mirror to
http://archive.ubuntu.com/ubuntu when unset (overridable for local/geo/
ports mirrors). In the Ubuntu genimage, when pkgdir has no http mirror,
build the debootstrap source and chroot sources.list from that mirror with
the codename derived from osvers and the release/-updates/-security pockets
(main + universe) -- the compute pkglist needs universe (busybox-static,
dracut*) and the security-updated openssh. Ubuntu-only; EL netboot
(netboot/rh) and copycds/packimage are untouched.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
(cherry picked from commit 03e1559989)
When a node's install NIC is selected by MAC (installnic=mac), the generated
netplan was invalid: it named the ethernet device the literal string 'mac' and
set its MAC match to the full pipe-delimited list of all node MACs. The result
had no valid interface, so neither the installer environment nor the installed
system came up on the network. Match the install NIC by interface-name glob in
the installer, and in the target use only the primary MAC and omit set-name
when the install NIC is selected by MAC.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
(cherry picked from commit c6e38483fb)
The generated Subiquity autoinstall config defined no identity section. Newer
Subiquity treats a config without an identity as incomplete and stops at an
interactive "Continue with autoinstall?" confirmation, so the install never
proceeds unattended (the prompt's advice to add 'autoinstall' to the kernel
command line is a red herring -- it was already present). Provide an identity
with a password sourced from the cluster passwd table so the install runs
non-interactively.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
(cherry picked from commit 86e77bcd7f)
The Ubuntu live-server install media is not a complete apt package mirror.
With the generated apt configuration pinned to the media only, an install that
needs a package absent from the media stalls instead of fetching it. Emit an
online-mirror apt configuration when one is configured (site.ubuntu_apt_mirror)
so missing packages are retrieved from the archive; offline behaviour is
unchanged when it is unset.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
(cherry picked from commit 846e611772)
Daniel Hilst
Vinícius Ferrão
xcat2-backport-automation[bot]
Vinícius Ferrão
Markus Hilger