xCAT/xcat-core
2
0
Fork 0
mirror of https://github.com/xcat2/xcat-core.git synced 2026-09-21 16:39:30 +00:00
Code Issues Releases Wiki Activity

fix(xcat-core): the legacy Genesis image never reaches doxcat, so no node boots a shell

Browse Source 
A compute node fetches the legacy Genesis kernel and initramfs, the kernel starts, and then
nothing else happens: doxcat never runs, the node acquires no address, sshd refuses every
connection and the node stays at status=powering-on. The five genesis test cases in
xCAT-test/autotest/testcase/genesis have never passed. VersatusHPC/xcat-internal#78.

Three holes in the image, each fatal on its own. dracut_105/el/xcat-cmdline.sh ends in
`while :; do tmux attach-session -t doxcat || tmux new-session -s doxcat doxcat; done`, and the
image carries no locale data, so tmux exits with "need UTF-8 locale" and the loop spins
without ever reaching doxcat. module-setup.sh does not install
/usr/libexec/openssh/sshd-session, which OpenSSH 9.8 and later exec for every connection and
which EL9 now ships. xCAT-genesis-base.spec does not BuildRequire dhcp-client, so dhclient is
absent from the build chroot; dracut_install reports the missing binary and returns, and the
module install function keeps going, so the image ships without it.

xcat-cmdline.sh now resolves xcat_console_mode() once and runs doxcat directly when the
terminal multiplexer cannot start a session; the same shape replaces the screen loop on
Ubuntu. module-setup.sh installs the OpenSSH session helpers and the C.utf8 locale where they
exist. The spec BuildRequires dhcp-client on the releases that package it, and runs the new
xCAT-genesis-builder/verify-genesis-payload over the extracted payload, which fails the build
when sshd needs a helper the image lacks, when tmux has no UTF-8 locale, or when a binary the
caller named is missing.

The same runs exposed four defects in the test cases themselves. test.sh defined its synthetic
node as ppc64le whatever the management node was, so nodeset could not find a genesis kernel on
x86_64. genesistest.pl get_os() matched neither AlmaLinux nor Rocky and reported the OS as
unsupported. The -g check read $? instead of check_genesis_file()'s return value, so it could
never fail. And testxdsh() met "REMOTE HOST IDENTIFICATION HAS CHANGED" from the second boot
on, because Genesis makes new host keys every boot and nothing dropped the stale known_hosts
entry. test.sh now derives the node arch from uname and takes the tftp root from TFTPDIR,
get_os() recognises the redhat family, report_genesis_files() carries the result to an exit
status, and forget_host_keys() runs makeknownhosts -r before each probe.

Tests: genesis_console_mode.t drives xcat_console_mode() with the multiplexer shadowed;
genesis_payload_verification.t drives the verifier over payload trees carrying each hole;
genesis_testcase_helpers.t drives get_os(), check_genesis_file(), report_genesis_files() and
testxdsh(); genesis_incorrectmasterip_check.t runs test.sh against a scratch tftp root. Each
fails on the parent commit. The verifier also reports all three holes against the released
xCAT-genesis-base-x86_64-2.19.0-snap202609021858 payload.

Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
(cherry picked from commit cb6021eb3cdb3abc75e4dd6704cb42b28074e140)
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
This commit is contained in:
Daniel Hilst
2026-09-04 13:33:45 -03:00
parent ea1cb82a6d
commit ff20388fed
13 changed files with 598 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files
buildrpms.pl
+3
View File
@@ -346,6 +346,9 @@ sub buildsources_genesis_base($) {
"Error copying dracut_105 sources");
cp "xCAT-genesis-builder/80-net-name-slot.rules",
"$staging_root/80-net-name-slot.rules";
# %install runs this against the extracted payload before it becomes an rpm.
cp "xCAT-genesis-builder/verify-genesis-payload",
"$staging_root/verify-genesis-payload";
unlink $support_tarball if -f $support_tarball;
sh_or_die(qq(tar --sort=name --owner=0 --group=0 --mtime="\@$SOURCE_DATE_EPOCH" -cjf "$support_tarball" -C "$staging_parent" xCAT-genesis-base-build-support),
xCAT-genesis-builder/dracut_105/el/module-setup.sh
+16
View File
@@ -48,6 +48,22 @@ install() {
#dracut_install libvirtd /usr/share/libvirt/cpu_map.xml /usr/bin/qemu-img /usr/libexec/qemu-kvm
dracut_install mkswap df ifenslave ssh-keygen scp clear
dracut_install dhclient lldpad
# OpenSSH 9.8 moved the per-connection work into sshd-session, which sshd execs by
# absolute path. Without it every connection to Genesis is refused.
for _sshd_helper in \
/usr/libexec/openssh/sshd-session \
/usr/libexec/openssh/sshd-auth \
/usr/lib/openssh/sshd-session \
/usr/lib/openssh/sshd-auth
do
_dracut_install_opt "$_sshd_helper"
done
# tmux exits under the C locale, and the image carries no locale data of its own.
for _lc_file in /usr/lib/locale/C.utf8/LC_*; do
_dracut_install_opt "$_lc_file"
done
dracut_install /lib64/libnss_dns.so.2
dracut_install poweroff hwclock date /usr/share/terminfo/x/xterm /usr/share/terminfo/s/screen /etc/nsswitch.conf /etc/services
dracut_install /sbin/rsyslogd /etc/protocols umount /bin/rpm /usr/lib/rpm/rpmrc
xCAT-genesis-builder/dracut_105/el/xcat-cmdline.sh
+24 -3
View File
@@ -2,6 +2,20 @@
root=1
rootok=1
netroot=xcat
# The image ships the C.UTF-8 locale only. tmux refuses to start under the C locale.
export LC_ALL=C.UTF-8
# tmux exits when the image carries no UTF-8 locale. doxcat is the whole of Genesis, so it
# must run whether or not the multiplexer starts. Prints tmux or direct.
xcat_console_mode() {
if tmux -f /dev/null new-session -d -s xcatprobe true >/dev/null 2>&1; then
tmux kill-session -t xcatprobe >/dev/null 2>&1
echo tmux
else
echo direct
fi
}
clear
echo PS1="'"'[xCAT Genesis running on \H \w]\$ '"'" > /.bashrc
echo PS1="'"'[xCAT Genesis running on \H \w]\$ '"'" > /.bash_profile
@@ -39,10 +53,13 @@ mkdir -p /var/lib/dhclient/
mkdir -p /var/log
ip link set lo up
echo '127.0.0.1 localhost' >> /etc/hosts
if grep -q console=ttyS /proc/cmdline; then
XCAT_CONSOLE_MODE="$(xcat_console_mode)"
if [ "$XCAT_CONSOLE_MODE" = "tmux" ]; then
if grep -q console=ttyS /proc/cmdline; then
while :; do sleep 1; tmux attach-session -t doxcat </dev/tty1 &>/dev/tty1; clear &>/dev/tty1 ; done &
fi
while :; do tmux new-session < /dev/tty2 &> /dev/tty2 ; done &
fi
while :; do tmux new-session < /dev/tty2 &> /dev/tty2 ; done &
# The section below is just for System P LE hardware discovery
@@ -87,4 +104,8 @@ elif [[ ${ARCH} =~ x86_64 ]]; then
done
fi
while :; do tmux attach-session -t doxcat || tmux new-session -s doxcat doxcat; done
if [ "$XCAT_CONSOLE_MODE" = "tmux" ]; then
while :; do tmux attach-session -t doxcat || tmux new-session -s doxcat doxcat; done
else
while :; do doxcat; sleep 5; done
fi
xCAT-genesis-builder/dracut_105/ubuntu/module-setup.sh
+16
View File
@@ -53,6 +53,22 @@ install() {
#dracut_install libvirtd /usr/share/libvirt/cpu_map.xml /usr/bin/qemu-img /usr/libexec/qemu-kvm
dracut_install mkswap df ifenslave ssh-keygen scp clear
dracut_install dhclient lldpad
# OpenSSH 9.8 moved the per-connection work into sshd-session, which sshd execs by
# absolute path. Without it every connection to Genesis is refused.
for _sshd_helper in \
/usr/libexec/openssh/sshd-session \
/usr/libexec/openssh/sshd-auth \
/usr/lib/openssh/sshd-session \
/usr/lib/openssh/sshd-auth
do
_dracut_install_opt "$_sshd_helper"
done
# tmux exits under the C locale, and the image carries no locale data of its own.
for _lc_file in /usr/lib/locale/C.utf8/LC_*; do
_dracut_install_opt "$_lc_file"
done
_dracut_install_opt "/lib/$TRIPLET/libnss_dns.so.2"
dracut_install poweroff hwclock date /usr/share/terminfo/x/xterm /usr/share/terminfo/s/screen /etc/nsswitch.conf /etc/services
dracut_install /usr/sbin/rsyslogd /etc/protocols umount /usr/bin/dpkg
xCAT-genesis-builder/dracut_105/ubuntu/xcat-cmdline.sh
+24 -3
View File
@@ -2,6 +2,20 @@
root=1
rootok=1
netroot=xcat
# The image ships the C.UTF-8 locale only. tmux refuses to start under the C locale.
export LC_ALL=C.UTF-8
# screen exits when the image carries no usable terminal. doxcat is the whole of Genesis, so
# it must run whether or not the multiplexer starts. Prints screen or direct.
xcat_console_mode() {
if screen -ln -d -m -S xcatprobe true >/dev/null 2>&1; then
screen -S xcatprobe -X quit >/dev/null 2>&1
echo screen
else
echo direct
fi
}
clear
echo PS1="'"'[xCAT Genesis running on \H \w]\$ '"'" > /.bashrc
echo PS1="'"'[xCAT Genesis running on \H \w]\$ '"'" > /.bash_profile
@@ -39,10 +53,13 @@ mkdir -p /var/lib/dhclient/
mkdir -p /var/log
ip link set lo up
echo '127.0.0.1 localhost' >> /etc/hosts
if grep -q console=ttyS /proc/cmdline; then
XCAT_CONSOLE_MODE="$(xcat_console_mode)"
if [ "$XCAT_CONSOLE_MODE" = "screen" ]; then
if grep -q console=ttyS /proc/cmdline; then
while :; do sleep 1; screen -S console -ln screen -x doxcat </dev/tty1 &>/dev/tty1; clear &>/dev/tty1 ; done &
fi
while :; do screen -ln < /dev/tty2 &> /dev/tty2 ; done &
fi
while :; do screen -ln < /dev/tty2 &> /dev/tty2 ; done &
# The section below is just for System P LE hardware discovery
@@ -87,4 +104,8 @@ elif [[ ${ARCH} =~ x86_64 ]]; then
done
fi
while :; do screen -dr doxcat || screen -S doxcat -L -ln doxcat; done
if [ "$XCAT_CONSOLE_MODE" = "screen" ]; then
while :; do screen -dr doxcat || screen -S doxcat -L -ln doxcat; done
else
while :; do doxcat; sleep 5; done
fi
xCAT-genesis-builder/verify-genesis-payload
Executable
+63
View File
@@ -0,0 +1,63 @@
#!/bin/bash
#
# verify-genesis-payload <payload-root> [required-path ...]
#
# dracut_install() reports a missing binary and returns, so the module install function keeps
# going and the image ships without it. Three such holes reached a release: no dhclient, no
# sshd-session and no UTF-8 locale. Check the extracted payload before it becomes an rpm.
#
# Paths are relative to <payload-root>. The caller adds what only it knows (dhclient is not
# packaged on every release); the rules below come from the payload itself.
set -u
payload=${1:-}
if [ -z "$payload" ] || [ ! -d "$payload" ]; then
echo "verify-genesis-payload: not a payload directory: ${payload:-<empty>}" >&2
exit 2
fi
shift
missing=""
# have PATH: true when the payload carries PATH as a file, following the usr-merge symlinks
# the image ships (/sbin -> usr/sbin).
have() {
[ -e "$payload/$1" ]
}
require() {
local path=$1 why=$2
have "$path" || missing="$missing
$path ($why)"
}
for path in "$@"; do
require "$path" "required by the build"
done
require usr/sbin/sshd "Genesis is reached over ssh"
# OpenSSH 9.8 split the per-connection work into sshd-session, which sshd execs by absolute
# path. EL9 carries OpenSSH 9.9, so an image with sshd alone refuses every connection.
if have usr/sbin/sshd && grep -qa 'sshd-session' "$payload/usr/sbin/sshd" 2>/dev/null; then
if ! have usr/libexec/openssh/sshd-session && ! have usr/lib/openssh/sshd-session; then
missing="$missing
usr/libexec/openssh/sshd-session (this sshd execs it for every connection)"
fi
fi
# tmux exits under the C locale. The hook falls back to running doxcat directly, so this is
# not fatal to booting, but a Genesis shell without tmux loses the console attach.
if have usr/bin/tmux && ! have usr/lib/locale/C.utf8/LC_CTYPE; then
missing="$missing
usr/lib/locale/C.utf8/LC_CTYPE (tmux refuses to start without a UTF-8 locale)"
fi
if [ -n "$missing" ]; then
echo "verify-genesis-payload: $payload is incomplete:$missing" >&2
exit 1
fi
echo "verify-genesis-payload: $payload is complete"
exit 0
xCAT-genesis-builder/xCAT-genesis-base.spec
+14
View File
@@ -53,6 +53,11 @@ BuildRequires: efibootmgr
BuildRequires: dosfstools
BuildRequires: dracut
BuildRequires: dracut-network
# doxcat drives the ISC client with -cf/-pf/-lf. RHEL 10 dropped dhcp-client, so el10
# genesis has no DHCP client yet.
%if 0%{?rhel} && 0%{?rhel} < 10
BuildRequires: dhcp-client
%endif
BuildRequires: ethtool
BuildRequires: gawk
BuildRequires: ipmitool
@@ -224,6 +229,15 @@ test -n "$KERNEL_IMAGE"
test -e "$KERNEL_IMAGE"
cp "$KERNEL_IMAGE" "$GENESIS_ROOT/kernel"
# dracut_install reports a missing binary and returns, so a hole in the image reaches the
# rpm silently. Three of them did.
GENESIS_REQUIRED=""
%if 0%{?rhel} && 0%{?rhel} < 10
GENESIS_REQUIRED="usr/sbin/dhclient"
%endif
bash "%{_builddir}/xCAT-genesis-base-build-support/verify-genesis-payload" \
"$GENESIS_FS" $GENESIS_REQUIRED
find "$GENESIS_TMPDIR" -type c -delete
cp -a "$GENESIS_TMPDIR/%{prefix}/." "$RPM_BUILD_ROOT/%{prefix}/"
xCAT-test/autotest/testcase/genesis/genesistest.pl
+33 -8
View File
@@ -68,13 +68,7 @@ if (!defined($noderange)) {
}
my $os = &get_os;
if ($check_genesis_file) {
send_msg(2, "[$$]:Check if genesis packages are installed on mn...............");
&check_genesis_file(&get_arch);
if ($?) {
send_msg(0, "genesis packages are not installed");
} else {
send_msg(2, "genesis packages are installed");
}
exit 1 if &report_genesis_files(&get_arch);
}
my $master=`lsdef -t site -i master -c 2>&1 | awk -F'=' '{print \$2}'`;
if (!$master) { $master=hostname(); }
@@ -148,6 +142,21 @@ if ($clear_env) {
send_msg(2, "[$$]:Clear genesis test enviroment success...............");
}
##################################
#report_genesis_files
#################################
sub report_genesis_files {
my ($arch) = @_;
send_msg(2, "[$$]:Check if genesis packages are installed on mn...............");
# The caller used to test $?, which holds the exit status of the last child process, not
# this return value. A node with no genesis packages therefore reported success.
if (&check_genesis_file($arch)) {
send_msg(0, "genesis packages are not installed");
return 1;
}
send_msg(2, "genesis packages are installed");
return 0;
}
##################################
#check_genesis_file
#################################
sub check_genesis_file {
@@ -264,6 +273,17 @@ sub rungenesisimg {
########################################
####sleep while for xdsh $$CN could work
#########################################
##########################################
####forget the node ssh host keys
##########################################
sub forget_host_keys {
my ($noderange) = @_;
# Genesis makes new host keys on every boot, and each case boots the node several times.
# The stale known_hosts entry then makes ssh refuse the changed key, and xdsh cannot reach
# the Genesis shell.
system("makeknownhosts $noderange -r >/dev/null 2>&1");
return 0;
}
sub testxdsh {
my $value = shift;
my $checkstring;
@@ -285,6 +305,8 @@ sub testxdsh {
return 1;
}
&forget_host_keys($noderange);
# Check shell prompt on the node to verify it is running Genesis
`xdsh $noderange -t 2 "echo \\\$PS1" | grep "Genesis"`;
if ($?) {
@@ -365,7 +387,10 @@ sub get_os {
my $output = `cat /etc/*release* 2>&1`;
if ($output =~ /suse/i) {
$os = "sles";
} elsif ($output =~ /Red Hat/i) {
} elsif ($output =~ /Red Hat/i
or $output =~ /\b(?:almalinux|rocky|centos|fedora|oracle\s+linux)\b/i
or $output =~ /^ID_LIKE=.*\brhel\b/mi) {
# AlmaLinux and Rocky release files name neither Red Hat nor themselves as one.
$os = "redhat";
} elsif ($output =~ /ubuntu/i) {
$os = "ubuntu";
xCAT-test/autotest/testcase/genesis/test.sh
+10 -4
View File
@@ -16,6 +16,12 @@ function runcmd(){
# We should be using private networks
TESTNODE=testnode
TESTNODE_IP="192.168.3.1"
# nodeset resolves the genesis kernel by the node arch. A hardcoded ppc64le node fails on
# every other management node with "Could not find genesis.kernel.ppc64".
TESTNODE_ARCH="$(uname -m)"
# The boot-loader configuration lives under the tftp root. Overridable so the check can run
# against a scratch tree.
TFTPDIR="${TFTPDIR:-/tftpboot}"
MASTER_PRIVATE_IP="192.168.1.1"
MASTER_PRIVATE_NETMASK="255.255.0.0"
@@ -23,7 +29,7 @@ MASTER_PRIVATE_NETWORK="192_168_0_0-255_255_0_0"
function check_destiny() {
cmd="chdef ${TESTNODE} arch=ppc64le cons=ipmi groups=all ip=${TESTNODE_IP} mac=4e:ee:ee:ee:ee:0e netboot=$NETBOOT tftpserver=$MASTER_PRIVATE_IP xcatmaster=$MASTER_PRIVATE_IP";
cmd="chdef ${TESTNODE} arch=${TESTNODE_ARCH} cons=ipmi groups=all ip=${TESTNODE_IP} mac=4e:ee:ee:ee:ee:0e netboot=$NETBOOT tftpserver=$MASTER_PRIVATE_IP xcatmaster=$MASTER_PRIVATE_IP";
runcmd $cmd;
lsdef ${TESTNODE}
@@ -86,11 +92,11 @@ while [ "$#" -ge "0" ]; do
"--check" )
NETBOOT=$2;
if [[ $NETBOOT =~ petitboot ]];then
SHELLFOLDER="/tftpboot/petitboot";
SHELLFOLDER="$TFTPDIR/petitboot";
elif [[ $NETBOOT =~ xnba ]];then
SHELLFOLDER="/tftpboot/xcat/xnba/nodes"
SHELLFOLDER="$TFTPDIR/xcat/xnba/nodes"
else
SHELLFOLDER="/tftpboot/boot/grub2";
SHELLFOLDER="$TFTPDIR/boot/grub2";
fi
check_destiny ;
if [[ $? -eq 1 ]];then
xCAT-test/unit/genesis_console_mode.t
+86
View File
@@ -0,0 +1,86 @@
#!/usr/bin/env perl
# Drive xcat_console_mode() out of the Genesis dracut cmdline hook.
#
# The hook cannot be sourced: it mounts filesystems, starts udev and ends in an endless
# loop. Extract the one function and run it with the terminal multiplexer shadowed.
use strict;
use warnings;
use File::Path qw(make_path);
use File::Slurper qw(read_text write_text);
use File::Temp qw(tempdir);
use FindBin;
use lib "$FindBin::Bin/../lib";
use Test::More;
use XCAT::Test::File qw(repo_path);
my %HOOK = (
el => { path => 'xCAT-genesis-builder/dracut_105/el/xcat-cmdline.sh', mux => 'tmux' },
ubuntu => { path => 'xCAT-genesis-builder/dracut_105/ubuntu/xcat-cmdline.sh', mux => 'screen' },
);
plan tests => 5 * scalar(keys %HOOK) + 2;
my $tmpdir = tempdir(CLEANUP => 1);
# The failure this captures: with no UTF-8 locale in the image, tmux exits and the old
# unconditional `while :; do tmux ...; done` never reached doxcat.
my $el = read_text(repo_path($HOOK{el}{path}));
ok($el !~ qr/^while :; do tmux attach-session/m,
'el: no unguarded tmux loop is left at column 0');
ok($el =~ qr/^export LC_ALL=C\.UTF-8$/m,
'el: the hook exports a UTF-8 locale so tmux can start');
foreach my $family (sort keys %HOOK) {
my $hook = repo_path($HOOK{$family}{path});
my $mux = $HOOK{$family}{mux};
my $body = extract_function($hook, 'xcat_console_mode', $family);
is(run_mode($body, $mux, 0), 'direct',
"$family: xcat_console_mode reports direct when $mux cannot start a session");
is(run_mode($body, $mux, 1), $mux,
"$family: xcat_console_mode reports $mux when $mux can start a session");
my $text = read_text($hook);
ok($text =~ qr/^XCAT_CONSOLE_MODE="\$\(xcat_console_mode\)"$/m,
"$family: the hook resolves the console mode once");
my $guard = qq{if [ "\$XCAT_CONSOLE_MODE" = "$mux" ]; then};
ok(index($text, $guard) >= 0,
"$family: the doxcat loop is guarded by the console mode");
ok($text =~ qr/\Qelse\E\n\s+while :; do doxcat; sleep 5; done\n\Qfi\E/,
"$family: doxcat runs directly when $mux is not usable");
}
#---
# extract_function: lift one shell function out of a script that cannot be sourced.
# Bails out when the function stops being extractable, so a rename fails loudly instead of
# leaving the test asserting nothing.
#---
sub extract_function {
my ($path, $name, $label) = @_;
my $text = read_text($path);
my ($body) = $text =~ /^($name\(\)\s*\{.*?^\})$/ms;
BAIL_OUT("$label: $name() not found in $path") unless defined $body;
return $body;
}
#---
# run_mode: run the extracted function with the multiplexer shadowed by a stub that either
# starts a session or refuses, the way tmux refuses without a UTF-8 locale.
#---
sub run_mode {
my ($body, $mux, $mux_works) = @_;
my $dir = tempdir(DIR => $tmpdir, CLEANUP => 1);
my $bin = "$dir/bin";
make_path($bin);
write_text("$bin/$mux", $mux_works
? "#!/bin/sh\nexit 0\n"
: "#!/bin/sh\necho '$mux: need UTF-8 locale (LC_CTYPE) but have ANSI_X3.4-1968' >&2\nexit 1\n");
chmod 0755, "$bin/$mux";
write_text("$dir/probe.sh", "$body\nxcat_console_mode\n");
my $out = `PATH="$bin:\$PATH" /bin/bash "$dir/probe.sh" 2>/dev/null`;
chomp $out;
return $out;
}
xCAT-test/unit/genesis_incorrectmasterip_check.t
+84
View File
@@ -0,0 +1,84 @@
#!/usr/bin/env perl
# Run the nodeset_shell_incorrectmasterip check against a scratch tftp root, with the xCAT
# commands and the net tools shadowed.
use strict;
use warnings;
use File::Path qw(make_path);
use File::Slurper qw(read_text write_text);
use File::Temp qw(tempdir);
use FindBin;
use lib "$FindBin::Bin/../lib";
use Test::More;
use XCAT::Test::File qw(repo_path);
my $script = repo_path('xCAT-test/autotest/testcase/genesis/test.sh');
plan skip_all => 'genesis test.sh not found' unless -f $script;
plan tests => 5;
my $host_arch = `uname -m`;
chomp $host_arch;
# The case defined its node as ppc64le whatever the management node was, so nodeset could not
# find a genesis kernel for it on x86_64 and the case could never pass there.
my $run = run_check('xnba', write_boot_file => 1);
is($run->{status}, 0, 'the xnba check passes when nodeset writes the boot file')
or diag($run->{output});
like($run->{chdef}, qr/\barch=\Q$host_arch\E\b/,
'the test node is defined with the management node arch');
ok($host_arch eq 'ppc64le' || $run->{chdef} !~ /\barch=ppc64le\b/,
'the test node arch is not pinned to ppc64le');
# A nodeset that writes nothing must fail the check, not pass it.
my $empty = run_check('xnba', write_boot_file => 0);
isnt($empty->{status}, 0, 'the check fails when nodeset writes no boot file');
# grub2 and petitboot read their configuration from other directories under the tftp root.
my $grub = run_check('grub2', write_boot_file => 1);
is($grub->{status}, 0, 'the grub2 check reads the grub2 directory')
or diag($grub->{output});
#---
# run_check: run `test.sh --check <loader>` against a scratch tftp root. test.sh resets PATH,
# so the xCAT commands are shadowed with shell functions, which bash resolves first. The fake
# nodeset writes the boot file the check greps, so the assertion is on the check, not on xCAT.
#---
sub run_check {
my ($loader, %opt) = @_;
my $root = tempdir(CLEANUP => 1);
my $tftp = "$root/tftpboot";
make_path("$tftp/xcat/xnba/nodes", "$tftp/boot/grub2", "$tftp/petitboot");
my $folder = $loader eq 'xnba' ? "$tftp/xcat/xnba/nodes"
: $loader eq 'petitboot' ? "$tftp/petitboot"
: "$tftp/boot/grub2";
my $write = $opt{write_boot_file}
? "printf 'xcatd=192.168.1.1:3001 destiny=shell\\n' > '$folder/testnode'"
: ":";
my $driver = "$root/driver.sh";
write_text($driver, <<"DRIVER");
chdef() { echo "\$@" >> '$root/chdef.log'; }
lsdef() {
if [ "\$1" = "-t" ] && [ "\$2" = "site" ]; then echo "clustersite: master=192.168.9.9"; return 0; fi
echo "Object name: testnode"
}
ifconfig() { printf 'eth0: flags\\n inet 192.168.9.9\\n\\n'; }
netstat() { printf 'Kernel\\nIface\\neth0\\neth1\\nlo\\n'; }
ip() { return 0; }
makenetworks() { return 0; }
tabdump() { return 0; }
makehosts() { return 0; }
rmdef() { return 0; }
nodeset() { $write; return 0; }
export TFTPDIR='$tftp'
. '$script' --check $loader
DRIVER
my $out = `/bin/bash "$driver" 2>&1`;
my $status = $? >> 8;
my $chdef = -f "$root/chdef.log" ? read_text("$root/chdef.log") : '';
return { status => $status, output => $out, chdef => $chdef };
}
xCAT-test/unit/genesis_payload_verification.t
+87
View File
@@ -0,0 +1,87 @@
#!/usr/bin/env perl
# Drive verify-genesis-payload against payload trees that reproduce the three holes the
# released legacy Genesis image shipped with.
use strict;
use warnings;
use File::Path qw(make_path);
use File::Slurper qw(read_text write_text);
use File::Temp qw(tempdir);
use FindBin;
use lib "$FindBin::Bin/../lib";
use Test::More;
use XCAT::Test::File qw(repo_path);
my $verifier = repo_path('xCAT-genesis-builder/verify-genesis-payload');
plan skip_all => 'verify-genesis-payload not found' unless -f $verifier;
plan tests => 9;
my $tmpdir = tempdir(CLEANUP => 1);
# A complete payload: OpenSSH 9.9 sshd plus its session helper, tmux plus a UTF-8 locale.
my $good = build_payload(sshd_execs_session => 1, session_helper => 1, tmux => 1, locale => 1, dhclient => 1);
my ($rc, $err) = run($good, 'usr/sbin/dhclient');
is($rc, 0, 'a complete payload passes') or diag($err);
# doxcat calls dhclient with ISC flags. The released el9 image carried dhclient.conf and
# dhclient-script but no dhclient, so Genesis never acquired an address.
my $nodhcp = build_payload(sshd_execs_session => 1, session_helper => 1, tmux => 1, locale => 1, dhclient => 0);
($rc, $err) = run($nodhcp, 'usr/sbin/dhclient');
isnt($rc, 0, 'a payload without dhclient fails');
like($err, qr{usr/sbin/dhclient}, 'the missing dhclient is named');
# sshd 9.9 execs /usr/libexec/openssh/sshd-session for every connection.
my $nohelper = build_payload(sshd_execs_session => 1, session_helper => 0, tmux => 1, locale => 1, dhclient => 1);
($rc, $err) = run($nohelper, 'usr/sbin/dhclient');
isnt($rc, 0, 'a payload whose sshd execs sshd-session but does not ship it fails');
like($err, qr{sshd-session}, 'the missing sshd-session is named');
# OpenSSH 8 does not use the helper, so el8 must still pass without it.
my $openssh8 = build_payload(sshd_execs_session => 0, session_helper => 0, tmux => 1, locale => 1, dhclient => 1);
($rc, $err) = run($openssh8, 'usr/sbin/dhclient');
is($rc, 0, 'an OpenSSH 8 payload passes without sshd-session') or diag($err);
# tmux without a UTF-8 locale is what stopped doxcat from ever running.
my $nolocale = build_payload(sshd_execs_session => 1, session_helper => 1, tmux => 1, locale => 0, dhclient => 1);
($rc, $err) = run($nolocale, 'usr/sbin/dhclient');
isnt($rc, 0, 'a payload with tmux and no UTF-8 locale fails');
like($err, qr{C\.utf8}, 'the missing locale is named');
($rc, $err) = run("$tmpdir/does-not-exist");
is($rc >> 0, 2, 'a missing payload directory is a usage error');
#---
# build_payload: make a payload tree with the pieces the verifier reasons about.
#---
sub build_payload {
my (%opt) = @_;
my $root = tempdir(DIR => $tmpdir, CLEANUP => 1);
make_path("$root/usr/sbin", "$root/usr/bin", "$root/usr/libexec/openssh");
write_text("$root/usr/sbin/sshd",
$opt{sshd_execs_session}
? "OpenSSH_9.9p1\n/usr/libexec/openssh/sshd-session\n"
: "OpenSSH_8.0p1\n");
write_text("$root/usr/libexec/openssh/sshd-session", "helper\n") if $opt{session_helper};
write_text("$root/usr/bin/tmux", "tmux\n") if $opt{tmux};
if ($opt{locale}) {
make_path("$root/usr/lib/locale/C.utf8");
write_text("$root/usr/lib/locale/C.utf8/LC_CTYPE", "ctype\n");
}
write_text("$root/usr/sbin/dhclient", "dhclient\n") if $opt{dhclient};
return $root;
}
#---
# run: run the verifier and return its exit status and stderr.
#---
sub run {
my ($root, @required) = @_;
my $errfile = "$tmpdir/err.$$";
my $cmd = join ' ', map { "'$_'" } ($verifier, $root, @required);
system("/bin/bash $cmd >/dev/null 2>$errfile");
my $status = $? >> 8;
my $err = -f $errfile ? read_text($errfile) : '';
unlink $errfile;
return ($status, $err);
}
xCAT-test/unit/genesis_testcase_helpers.t
+138
View File
@@ -0,0 +1,138 @@
#!/usr/bin/env perl
# Drive the genesis test case helpers. genesistest.pl needs a management node, so lift the
# routines out and run them with rpm, dpkg and cat shadowed.
use strict;
use warnings;
use File::Path qw(make_path);
use File::Slurper qw(read_text write_text);
use File::Temp qw(tempdir);
use FindBin;
use lib "$FindBin::Bin/../lib";
use Test::More;
use XCAT::Test::File qw(repo_path);
my $helper = repo_path('xCAT-test/autotest/testcase/genesis/genesistest.pl');
my $shell = repo_path('xCAT-test/autotest/testcase/genesis/test.sh');
plan skip_all => 'genesis testcase helpers not found' unless -f $helper && -f $shell;
plan tests => 11;
my $tmpdir = tempdir(CLEANUP => 1);
my $source = read_text($helper);
eval_subs($source, qw(get_os get_arch check_genesis_file));
# get_os drives every later branch. AlmaLinux and Rocky release files say neither "Red Hat"
# nor "suse" nor "ubuntu", so the management node read as unknown and the check was skipped.
is(os_for("AlmaLinux release 9.8 (Olive Jaguar)\n"), 'redhat', 'AlmaLinux is a redhat family node');
is(os_for("Rocky Linux release 9.5 (Blue Onyx)\n"), 'redhat', 'Rocky is a redhat family node');
is(os_for("Red Hat Enterprise Linux release 9.5\n"), 'redhat', 'RHEL is still a redhat family node');
is(os_for("SUSE Linux Enterprise Server 15 SP6\n"), 'sles', 'SLES is still detected');
is(os_for("NAME=\"Ubuntu\"\nID=ubuntu\n"), 'ubuntu', 'Ubuntu is still detected');
# check_genesis_file answers with a return value. The caller used to read $? instead, so a
# management node with no genesis packages reported success.
{
no warnings 'once';
local $GenesisTest::os = 'redhat';
is(rpm_check("xCAT-genesis-base-x86_64-2.19.0-snap1.noarch\nxCAT-genesis-scripts-x86_64-2.19.0-snap1.noarch\n"),
0, 'both genesis packages installed reports success');
is(rpm_check("xCAT-genesis-scripts-x86_64-2.19.0-snap1.noarch\n"),
1, 'a missing genesis-base reports failure');
eval_subs($source, qw(report_genesis_files));
is(report_files("xCAT-genesis-scripts-x86_64-2.19.0-snap1.noarch\n"),
1, 'report_genesis_files propagates the failure to its caller');
}
# Genesis generates new host keys at every boot and each case boots the node several times, so
# the second boot met "REMOTE HOST IDENTIFICATION HAS CHANGED" and xdsh could not reach it.
{
no warnings 'once';
eval_subs($source, qw(forget_host_keys testxdsh));
local $GenesisTest::noderange = 'xcat71-cn';
my $run = run_testxdsh(3, genesis_prompt => 1, cmdline => 'destiny=shell');
is($run->{status}, 0, 'testxdsh succeeds when the node answers in the Genesis shell');
like($run->{makeknownhosts}, qr/\bxcat71-cn\b/, 'the node host keys are forgotten first');
like($run->{makeknownhosts}, qr/-r/, 'makeknownhosts is asked to remove them');
}
#---
# run_testxdsh: drive testxdsh with makeknownhosts and xdsh shadowed. xdsh is asked twice --
# once for the prompt, once for the file -- and the stub answers both from its arguments.
#---
sub run_testxdsh {
my ($value, %opt) = @_;
my $dir = tempdir(DIR => $tmpdir, CLEANUP => 1);
my $log = "$dir/makeknownhosts.log";
write_text("$dir/makeknownhosts", "#!/bin/sh\necho \"\$@\" >> '$log'\n");
my $prompt = $opt{genesis_prompt} ? '[xCAT Genesis running on node]' : 'sh-5.1';
write_text("$dir/xdsh", "#!/bin/sh\nfor a in \"\$@\"; do\n case \"\$a\" in\n */cmdline|/proc/cmdline) printf '%s\\n' '$opt{cmdline}'; exit 0;;\n esac\ndone\nprintf '%s\\n' '$prompt'\n");
chmod 0755, "$dir/makeknownhosts", "$dir/xdsh";
local $ENV{PATH} = "$dir:$ENV{PATH}";
my $status = GenesisTest::testxdsh($value);
return { status => $status, makeknownhosts => (-f $log ? read_text($log) : '') };
}
#---
# eval_subs: lift named subs out of the script and compile them into a scratch package, so
# they can be run without a management node. Bails out when a sub stops being extractable.
#---
sub eval_subs {
my ($text, @names) = @_;
my $code = "package GenesisTest;\nno strict;\nno warnings;\nour \$os;\nour \$check_genesis_file;\nour \$noderange;\n";
$code .= "sub send_msg { push \@GenesisTest::MSG, \$_[1]; return 0; }\n";
foreach my $name (@names) {
my ($body) = $text =~ /^(sub \Q$name\E \{.*?^\})$/ms;
BAIL_OUT("sub $name() not found in $helper") unless defined $body;
$code .= "$body\n";
}
$code .= "1;\n";
eval $code or BAIL_OUT("cannot compile the extracted helpers: $@");
}
#---
# os_for: run get_os with `cat` shadowed so it reads the release text under test.
#---
sub os_for {
my ($release) = @_;
local $ENV{PATH} = stub_bin(cat => "#!/bin/sh\nprintf '%s' " . shell_quote($release)) . ":$ENV{PATH}";
return GenesisTest::get_os();
}
#---
# rpm_check: run check_genesis_file with `rpm` shadowed so `rpm -qa` lists the given packages.
#---
sub rpm_check {
my ($installed) = @_;
local $ENV{PATH} = stub_bin(rpm => "#!/bin/sh\nprintf '%s' " . shell_quote($installed)) . ":$ENV{PATH}";
return GenesisTest::check_genesis_file('x86_64');
}
sub report_files {
my ($installed) = @_;
local $ENV{PATH} = stub_bin(rpm => "#!/bin/sh\nprintf '%s' " . shell_quote($installed)) . ":$ENV{PATH}";
return GenesisTest::report_genesis_files('x86_64');
}
#---
# shell_quote: single-quote a string for /bin/sh.
#---
sub shell_quote {
my ($v) = @_;
$v =~ s/'/'\\''/g;
return "'$v'";
}
#---
# stub_bin: a directory holding one shadow command, ahead of the real one on PATH.
#---
sub stub_bin {
my (%cmd) = @_;
my $dir = tempdir(DIR => $tmpdir, CLEANUP => 1);
while (my ($name, $body) = each %cmd) {
write_text("$dir/$name", $body);
chmod 0755, "$dir/$name";
}
return $dir;
}