Five comments in the discovery path explained a consequence chain or
justified the code to a reviewer rather than stating what is not visible
locally: the payload protocol the code below already shows, a rationale
for the missing boot loader message, the two Genesis compression
variants, the HTTP and TFTP entry pair, and the :noboot divergence from
the PXELINUX files. Say the non-obvious part once.
Comments only; no executable line changes.
Firmware configured for UEFI HTTP boot sends client architecture 28
(0x001c) and only accepts an offer whose boot file is a URL and whose
reply carries the HTTPClient vendor class; it ignores the PXE offer that
answers architecture 27. riscv64 nodes therefore could not boot at all
from firmware set up that way.
Answer them with the same grub2 image over HTTP. The URL has to name the
management node address of the network the request came in on, so the
class belongs to the subnet, like the POWER OPAL and xNBA network classes
next to it, and it is only offered while the boot loader is actually
published. The ISC backend renders the matching subnet branch.
This covers every client without a reservation, which is the discovery
case. A node that nodeset has configured keeps its per-node boot loader
over TFTP, as on the other architectures.
The grub2 discovery configuration loaded the Genesis kernel and initramfs
over TFTP, a lockstep protocol that acknowledges every block and runs one
server process per client. Fetching the same 79 MiB Genesis image from a
node on the lab network took 61.5 s over TFTP and 1.2 s over HTTP, and a
whole cluster discovering at once queues on the TFTP server.
Write two entries instead. The default one sets root to the HTTP server
of the management node on that network and loads the same files from
below the TFTP root, the way nodeset does for netboot=grub2-http; the
second keeps the TFTP paths for a management node that does not serve the
TFTP root over HTTP, and "set fallback=1" moves to it when GRUB cannot
fetch the payload over HTTP. site.httpport is honoured.
riscv64 nodes boot through UEFI and grub2 only, and xCAT builds no boot
loader: /tftpboot/boot/grub2/grub2.riscv64 has to come from the xcat-dep
grub2-xcat package or be copied by hand, which is a step an admin only
finds out about when a node times out in firmware.
The EL riscv64 media carry exactly that image as EFI/BOOT/grubriscv64.efi,
so copycd publishes it after a successful media copy, and says so. An
image the management node already has is never replaced, and the media of
every other architecture is untouched.
EL10 defines no default crash kernel reservation for riscv64, so the
installer's kdump add-on falls back to writing the literal
"crashkernel=auto" into the boot loader arguments of the installed
system. EL10 kernels dropped support for that value: nothing is
reserved, but the string is on the command line, so kdump.service passes
its condition and then fails on every installed riscv64 node.
Turn the add-on off in the riscv64 templates. Nodes come up with kdump
inactive instead of failed, and a node that should take crash dumps gets
a real reservation the usual way, through linuximage.addkcmdline or
bootparams.addkcmdline.
The kdump branch of the diskless kernel command line has a default
reservation for ppc64 and x86 only. On any other architecture an image
with linuximage.dump set but no linuximage.crashkernelsize got dump= and
no crashkernel= at all, so the kernel reserved nothing and kdump could
never run. EL has no default reservation for riscv64 either
(kdumpctl get-default-crashkernel is empty there), so nothing else fills
the gap.
Give riscv64 the same treatment as the architectures around it and
default to 256M. An explicit crashkernelsize still wins, and images
without dump are unchanged.
The EL10 anaconda has no RISC-V EFI platform: on riscv64 it asks for the
x86 UEFI boot loader packages (grub2-efi-x64, shim-x64), which do not
exist, and registers the UEFI boot entry as \EFI\<distro>\shimx64.efi,
so a kickstart install stops at the missing packages and, once tolerated,
leaves a system the firmware cannot boot.
Add riscv64 templates for the rocky10/rhels10 compute and service
profiles that are the shared templates plus %packages --ignoremissing and
a %post fix-up, riscv64 package lists that add grub2-efi-riscv64 and
efibootmgr to the shared lists, and the fix-up itself
(post.rhels10.riscv64): it copies \EFI\<distro>\grubriscv64.efi to the
removable-media path \EFI\BOOT\BOOTRISCV64.EFI and re-points the UEFI
boot entry at grubriscv64.efi, so the installed node boots from disk with
or without usable NVRAM. Reinstalling a node replaces that entry instead
of adding another one.
The %post of a kickstart is a single shell script, into which xCAT
splices every #INCLUDE, and post.rhels10/post.rhels8 end it with
"exit 0"; the fix-up is therefore included ahead of them. The shared
templates and other architectures are unchanged.
perl-DB_File is only used by the Confluent client
(lib/xcat/Confluent/Client.pm). EL10 dropped libdb, EPEL re-adds it only
on its own architectures, and riscv64 has no perl-DB_File at all, so a
hard Requires makes xCAT-server uninstallable on a riscv64 management
node. Ask for it weakly: dnf installs it where it exists and skips it
where it does not; the Confluent client stays optional.
The dependency generator also turns Confluent/Client.pm's "use DB_File"
into a hard perl(DB_File) requirement regardless of the Recommends, so
that one generated requirement is excluded as well, appended to whatever
filter the build root already set.
Weak dependencies need rpm 4.12, so both only apply on a build host that
has it (EL8 and later, or SUSE 15 and later); older build hosts keep the
hard requirement they have today.
Provide the compute and service profiles for rocky10 and rhels10 on
riscv64: netboot pkglist/exlist/postinstall files and the service
otherpkgs lists for netboot and install. They mirror the x86_64 profiles
(every package resolves in the Rocky 10 riscv64 BaseOS/AppStream/CRB
repositories and the minimal-environment group exists there), with the
kbd keymap exclude spelled correctly, the duplicate man exclude dropped,
and goconserver pulled from the rh10/riscv64 dependency repository.
EL riscv64 media lay out the installer kernel and initrd under
images/pxeboot exactly like x86 and aarch64 media, but anaconda.pm only
looked there for those two families and geninitrd.pm refused riscv64
outright ("unknow arch"). Treat riscv64 like x86/aarch64 in both
places and recognise riscv64 kernels when a driver disk updates the
installer kernel. There is no riscv64 SUSE media, so geninitrd keeps the
unsupported-architecture error for sles/suse rather than reading the x86
SUSE layout.
Diskless images get a riscv64 default network driver list (virtio,
Intel, Realtek, Broadcom and Mellanox) and take the resolver libraries
from lib64, which is where riscv64 EL puts them.
mknb only knew how to publish a discovery boot configuration for x86
(PXELINUX and xNBA) and POWER (petitboot). Any other architecture got a
Genesis kernel and initramfs under /tftpboot/xcat and nothing that would
make a firmware boot them, so riscv64 discovery could not start.
riscv64 nodes boot through UEFI and grub2. Write one grub2 configuration
per network, /tftpboot/boot/grub2/grub.cfg-<network hex prefix>, using
the same network keys as the PXELINUX files. A net-booted grub2.riscv64
searches grub.cfg-01-<mac>, grub.cfg-<8 hex ip> and then shorter prefixes
of the ip, so the per-node files that nodeset writes keep priority and
the network file is only reached by clients without a node configuration.
The file is regenerated from the published Genesis artifacts (lzma
preferred over gzip), guarded by $grub_cpu so other grub2 architectures
can share it later, carries the xcatd endpoint, the serial console and
BOOTIF=$net_default_mac, and is dropped for networks served by a
:noboot interface. It is written by name rather than into an existing
file, because on a /32 network nodeset's hard link for the node carries
the same name.
Publishing a Genesis image now also drops the other compression variant
of that architecture, so a leftover genesis.fs.<arch>.lzma can no longer
be paired with a freshly published kernel by this configuration or by
--configfileonly. And since these configurations are only reachable
through grub2.<arch>, which xCAT does not build, a missing boot loader is
reported instead of leaving the nodes to time out in firmware.
xcatconfig now also runs mknb riscv64 when xCAT-genesis-base-riscv64 is
installed, and the usage text lists the architecture.
Discovery left noderes.netboot untouched for any architecture outside
x86, ppc and armv7l, so a discovered riscv64 node had no boot method and
nodeset failed to find a plugin for it.
Move the default-netboot ladder into _default_netboot(), which returns
the method to set or undef, and teach it that riscv64 nodes boot through
UEFI and grub2. The existing x86, PowerNV, ppc and onie rules are
unchanged; aarch64 is deliberately left as it was. The platform of the
discovery request is only read when the request carries it, so a node
that reports none does not gain the key, which would end up stored as
discovery data.
RISC-V 64-bit UEFI firmware identifies itself with DHCP option 93
client-system-architecture 27 (0x001b, IANA processor architecture
types). Neither DHCP backend knew the value: Kea handed such clients no
boot file and ISC dhcpd fell through to the /yaboot catch-all.
Add an xcat-riscv64 Kea client class and an ISC subnet branch that send
them boot/grub2/grub2.riscv64, the same shape as the aarch64 entries.
The UEFI HTTP boot id (0x001c) is left alone: it needs a URL boot file
and the HTTPClient vendor class, which is a separate change.
The IMM pending_build_id property is not guaranteed to identify the primary UEFI bank. Keep it out of the active version value and expose it as a separate firmware inventory record.
Recovered from original commit b79c005061 by Jarrod Johnson.
Co-authored-by: Jarrod Johnson <jarrod.b.johnson@gmail.com>
Remove the unreferenced Debian-local copies of setupNFSTree and setupStatemnt. Debian, Anaconda, and SLES callers already use xCAT::SvrUtils, and xCAT plugin dispatch does not expose these private symbols.
Signed-off-by: Vinícius Ferrão <2031761+viniciusferrao@users.noreply.github.com>
xCAT appends dracut driver disks to the installer initrd as /dd.img. EL6 auto-loads that embedded image, while Anaconda 7 and newer require inst.dd=/dd.img on the kernel command line.
Record successful injection beside the generated initrd so nodeset --noupdateinitrd reuses the same decision as a normal nodeset. Clear the marker when rebuilding, validate the temporary archive paths, and do not create it when the disk cannot be copied, archived, or appended.
Own collection, sensitivity, finalization, and reset as one request-scoped state object so xcatd only forwards callbacks and appends the finalized text.
Resolve optional Mellanox network drivers against the target image's selected kernel. Keep real mlx_en modules, fall back to mlx4_en, include available mlx5_core, and omit missing defaults without changing explicitly requested netdrivers.
A confluent attribute update keeps every attribute that the request does not
name. The export only named the attributes that the switch table holds, so a
switch row that is deleted stayed in confluent, and an interface that is
renamed left the attributes of the old name beside the attributes of the new
one. Confluent then holds two ports for one node, which defeats the discovery
that this export exists for.
Name the topology attributes that the switch table no longer holds, with no
value, so that confluent removes them. Confluent removes the attributes that
the request names with no value before it sets the rest of the request, and it
accepts a wildcard for the attributes of every interface. The wildcard does
not match the attributes that carry no interface, so those are named on their
own.
A node that confluent does not hold yet has nothing to remove, so the request
that creates a node is unchanged.
A confluent that accepts a wildcard in the name of an attribute arrived in
3.4.0. An older confluent reads the name as the name of one attribute, finds
no attribute of that name and reports nothing. The topology of an interface
that is gone then stays, as it does today.
makeconfluentcfg exports enclosure.manager and enclosure.bay from the mpa
and id columns. Those columns belong to the mp table. When the command runs
with a node range it reads them from the mp table, but when it runs with no
argument it reads them from the nodepos table, which has only node, rack, u,
chassis, slot, room and height. The two attributes were therefore always
empty for a whole cluster export.
Read them from the mp table in both branches.
makeconfluentcfg gives confluent the console settings, the credentials of the
hardware manager, the location and the enclosure of each node. It does not
give the switch and the port that the node is cabled to, which confluent uses
to find a node by the port it answers on.
Read the switch table and give confluent net.switch and net.switchport.
A node has one row in that table for each of its interfaces, thus keep every
row and not the first one. A row that names an interface gives
net.<interface>.switch and net.<interface>.switchport, so a node with more
than one interface keeps the port of each. A row that names no interface
gives the names without an interface.
Read the table with the node list when the command receives a node range and
read the whole table when it does not, as the command already does for the
other tables.
A cluster whose switch table is empty receives the configuration that it
receives today.
Recovered from the lenovobuild branch, where this arrived as one commit and
two repairs of it: the first keeps one row for each node, which loses every
interface but one, and reads the switch columns from the nodepos table in the
branch that takes no node range, where that table has no such columns and the
feature does nothing.
The mgtifname of a network can name more than one interface, separated by !.
The test for an InfiniBand interface ends at the end of the value, thus it
recognises eth0!ib0 but not ib0!eth0, and a node on such a network receives no
second host entry and no address over IPoIB.
Accept the name in any position.
A node that discovery finds over ethernet is known by its ethernet mac. When
that node boots over IPoIB, the request carries the InfiniBand identity of the
adapter and not the ethernet mac. dhcpd finds no host entry for that identity
and answers nothing, thus the node does not boot and the log gives no reason.
dhcp.pm already gives hardware type 32 to a node whose mac attribute holds an
8 or 9 byte fabric address, but that needs the fabric address before the node
boots, and discovery records the ethernet mac.
A Mellanox adapter makes its port GUID from the ethernet mac, by the insertion
of 03:00 in the middle. Thus the InfiniBand identity of the node is already
known. Write it as a second host entry with the -xcat-ib suffix and hardware type
32, so a request over either fabric finds the node and
receives the same address. Remove that entry with the node.
Write the second entry only for a node whose network an IPoIB interface
serves. A cluster with no InfiniBand keeps the host entries that it has today.
This is for the ISC backend. makedhcp returns into the Kea code before this
routine when Kea is the backend, and a Kea server does not answer an IPoIB
client, thus there is nothing there for a second entry to answer.
Recovered from the lenovobuild branch. Reimplemented against master: the
original writes the second entry for every ethernet node, which makes two
host entries for each node of a cluster that has no InfiniBand.
Vinícius Ferrão
Daniel Hilst
Vinícius Ferrão
Vinícius Ferrão