Exercise the complete ospkgs and otherpkgs scripts with recorded package
commands. Check unattended options, pkglist environment, failure status,
and CUDA architecture handling without extracting source fragments.
Use the production repository probes and package names. Check that
missing repositories and failed queries stop both installers before any
package transaction, and that similar package names do not satisfy EPEL.
Run OS detection in a private filesystem with the real release parser.
Remove the rewritten-source Perl harness and require GNU awk for the
legacy Red Hat fallback.
Replace the extracted-function Perl runner with BATS tests that invoke
routeop in isolated Linux namespaces. Check route commands, persistent
entries, matching routes, and replacement failures.
The persist test sourced only xcatlib.sh and never reached the gate in configeth that skipped an
ifcfg profile, so putting that gate back left every test green. Its refusal test also passed on a
127 from a missing function.
The test now calls the repair through xcat_nm_persist_nic_extra_params with nmcli stubbed to
report an ifcfg file and a keyfile. It also captures three faults of xcat_persist_nic_extra_param:
it keeps an old value, it joins the key to a last line with no newline, and it appends a keyfile
key after a section that follows [user].
nic_cfg.bats captures that nm_show prints no profile when NM reports an ifcfg file, so the xcattest
case cannot match CONNECTED_MODE=yes there. nic_cfg.sh returns before its dispatch when sourced.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
bash ignores errexit for a command inverted with "!", so a "! grep" that is not the last line of
a bats test never fails it. The ifcfg test wrote two of them, and the first could not report a
[user] section appearing in an ifcfg profile.
Both now call refute_grep, the helper that exists for this. Restoring the keyfile-only write
fails that test on both assertions.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
confignetwork_secondarynic_nicextraparams_updatenode fails on openEuler 20.03sp4, 22.03sp4 and
24.03sp4 and on el8, while its sibling confignetwork_secondarynic_updatenode passes everywhere.
The only failing check is CONNECTED_MODE=yes, twice. The run log shows configeth receives the
value -- "array_nic_params 0=CONNECTED_MODE=yes" -- and the profile NetworkManager reports,
/etc/sysconfig/network-scripts/ifcfg-xcat-ens4-1, does not contain it afterwards.
configeth writes nicextraparams in configipv4 and then activates the connection. The comment on
the late persistence block states why that write does not survive: the nmcli con reload/up in the
restart phase re-serializes the connection from NetworkManager's in-memory model and drops
anything NetworkManager does not model. The block repairs it after activation, but its gate
requires xcat_uses_nm_keyfile, so it runs for a keyfile store only. el8 never reaches that test,
and openEuler reaches it and fails it because NetworkManager there reports an ifcfg file. Those
are the two families on the ifcfg path and the two that fail.
The block now runs for both stores. xcat_nm_conn_file asks NetworkManager which file holds the
connection rather than composing the name, because NetworkManager suffixes it when a file of the
plain name exists. xcat_persist_nic_extra_param writes the key as the profile spells it -- under
[user] with an xcat. prefix in a keyfile, plain in an ifcfg file -- and appends only when the key
is absent.
xcatlib_nic_extra_param_persist.bats covers both stores with nmcli stubbed. Two behavioural
mutants were run against it: restoring the keyfile-only behaviour fails the three ifcfg
assertions and leaves the keyfile ones passing, and composing the filename fails only the
resolution assertion.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The specs are resolved with rpmspec and the Debian control files with
Dpkg::Deps for each architecture. xCAT and xCATsn require ipxe-xcat
and xnba-undi where they required xnba-undi, and ipxe-xcat alone on the
other architectures. The debs depend on ipxe-xcat and recommend
xnba-undi on every architecture.
The go-xcat install lists name both packages.
The riscv64 test copied riscv64_install_list out of go-xcat with awk and
ran it on its own copy of the install list. A change to the real lists
did not reach it.
The BATS test sources go-xcat and reads the install, riscv64 and
uninstall lists for the rpm and deb names. It checks that riscv64 drops
only the legacy Genesis, adds the OpenEmbedded Genesis and keeps the
x86 boot loaders, and that the uninstall list covers every package
go-xcat installs.
fix/service-node-el fixed the management-node guard as commit 0f708b96a, with the same
correction this branch made: the log must hold lines, not gain them. The two differ
only in the comment and in the error message, so the file no longer merges as one
change.
The wording here now matches fix/service-node-el. The remaining difference in this file
is the doubled-slash path filter, which that branch does not carry.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
check_provisioning_source.sh --baseline wrote one message for two conditions:
"no access log to baseline on ${MN_BASE:+$SN}${MN_BASE:-$MN}". When the management
node has a log and the service node does not, the second expansion returns the
management node's baseline spec, not a host name. On xcat42 the message read "no
access log to baseline on nosuchnode-xyz/var/log/httpd/access_log:881". That is the
message an operator reads when xdsh cannot reach the service node, so it has to name
the host that has no log.
Each condition now has its own test and its own message.
check_provisioning_source.bats covers the service-node condition. The case also
asserts the management node's log path is absent from the message, because the node
name alone matches the old text as a substring.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
(cherry picked from commit 89ac4558a0)
check_provisioning_source.bats held a case that required the check to FAIL when the
management node logs nothing after the baseline. That is the hierarchical result, not
a broken log: the management node provisions the service node before the baseline and
then serves the compute node nothing. The suite passed 14 of 14 while the check could
not pass on a real cluster.
Two more cases cover the path filter. The compute node fetches the root image with
wget from a URL that carries a double slash, so the path reaches the log as
//install/netboot/..., and the filter matched neither direction: the request counted
as no boot payload on the service node, and a flat provision that served it went
unreported.
All four cases fail against the current script.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
check_provisioning_source.sh counted every request the compute node had ever
made, for any path. Both directions were wrong. A flat run left
management-node requests for the same address, so a later hierarchical run
read them as its own and failed. A service-node entry from an earlier run, or
a 404 for /favicon.ico, satisfied the positive check without any boot payload
being served.
The script now takes a baseline. --baseline records how many lines each access
log holds on the management node and on the service node, before provisioning,
and the check counts only lines after that point. It counts only requests
under /install or /tftpboot, which are the trees httpd serves a boot payload
from. A log shorter than its baseline was rotated, so it is read from its
first line. Without a baseline the check refuses to answer instead of reading
the whole log. The three hierarchy cases call --baseline before nodeset.
check_provisioning_source.bats covers both regressions: a management-node
request before the baseline no longer fails the run, a service-node request
before the baseline no longer satisfies it, and a request that carries no boot
payload does neither. Removing the baseline comparison fails the first two.
Removing the path filter fails the other two.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
After `apt-get remove`, dpkg reports a package as "deinstall ok
config-files". uninstall_xcat_completely() selects packages with
installed_packages(), which accepts only "install ok installed". A
complete uninstall after an ordinary uninstall therefore does not purge
the xCAT packages, and their configuration files stay on the node.
The new bats test runs both uninstall paths against a dpkg-query stub.
The two purge tests fail on this tree. The ordinary uninstall keeps its
installed-only filter.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The Perl tests of the OpenEmbedded Genesis image ran its shell scripts
and C programs through system(), qx, pipes and fork, wrote fake
commands through copied read_file and write_file helpers, and matched
regexes against shell and C source. Many rejections checked only a
non-zero exit, so an earlier, unrelated failure also passed. The
quality gate judged 91% of their 762 checks bad.
The shell and C checks move to bats in xCAT-test/bats. The C programs
are compiled there and run: the plain console, test drivers for the
shared status view and the shell launcher, and genesis-udp-send against
a loopback listener. Each rejection now matches the message of its own
guard. The .t files keep the artifact checks (kas, recipes, kernel
fragments, units, README), read with XCAT::Test::File. Test files left
with no checks are deleted.
Claim map (file -> where its claims went):
- go_xcat_uninstall_installed_only.t -> go_xcat_uninstall_installed_only.bats
- genesis_readme_paths.t, genesis_base_sources_staged.t -> kept in the
.t (README read by slurp_repo_file, tarball read by Archive::Tar)
- genesis_openembedded_{export,hardware,runtime}.t -> the bats file of
the same name
- genesis_openembedded_{bmcsetup,metrics,extensions,qeth,actions,
discovery}.t -> scripts to the bats file of the same name, recipes
and units kept in the .t
- genesis_openembedded_console.t -> built and run checks, state.c,
plain_ui.c and shell.c to genesis_openembedded_console.bats; Newt
source checks kept in the .t
- genesis_openembedded.t -> oe/build to genesis_openembedded_build.bats;
shell matches to the runtime, hardware and extensions bats; console
matches to the console bats or the console .t; artifacts kept
Red proof: every moved or new test was run against a mutated copy of
the code it covers and went red, for example: rpm -q not checked,
tar without --mtime, VEX report not exported, support dir after PATH,
digest check off, layer2 forced, wildcard device allowed, arguments
eval'd, tcp probe skipped, SOURCE_PORT 302, AF_INET only, stage timer
compared, launch error ignored, kas run twice, status written in place.
One mutation stayed green: tar without --owner=0 when the test runs as
root.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Master added a workflow and several tests after this branch renamed
xCAT-genesis-builder to xCAT-genesis-base. They still name files under
xCAT-genesis-builder/, which no longer exists, so they fail or read
nothing. The OpenEmbedded workflow, six bats files and four unit tests
now use the new paths.
Two of those unit tests could reach the code only as shell.
genesis_ubuntu_build_root.t cut the package list out of
builddeb-genesis-base and evaluated it through bash.
genesis_payload_verification.t ran the bash verifier and parsed its
stderr.
XCAT::GenesisBuildRoot::required_packages() now returns the build-root
package list, and builddeb-genesis-base calls it. XCAT::GenesisPayload
holds the mandatory-command list and the payload check, and
verify-genesis-payload runs its main() with the same arguments, exit
codes and messages. stage_genesis_base_sources() stages the module
beside the script. Both modules use core Perl only.
The two tests now call these functions with chosen inputs and assert
the exact results. genesis_base_sources_staged.t expects the module in
the staged tarball.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Master fixed debuild-xcat-genesis-base and added a bats test for it. The
branch deleted that script as unused. The merge keeps the script and
moves it to xCAT-genesis-base with the other Genesis base files.
Master also stages verify-genesis-payload for the spec %install check.
stage_genesis_base_sources() now stages it too, and
genesis_base_sources_staged.t expects it in the tarball.
The workflow, builds.rst and xcat_probe_package_payload.t take the
master version.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The error-command test ended with
[ ! -e /run/testnode-logs.tar ]
meant to show the archive had not gone to the default path. It cannot show that.
The file may exist for reasons that have nothing to do with this test, in which case
the assertion fails while nothing is wrong; and its absence would be equally true if
the override had never worked at all. It answers a question about the host, not
about the run.
The positive assertion above it already carries the proof: the tar shadow writes a
marker, and the test greps for that marker in the path it passed. The output being
there is what shows the redirection went there.
Removing it changes nothing about what the test catches. With the override taken out
of the template, so the archive path is hard-coded again, the remaining assertion
still fails.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
site.cleanupdiskfullxcatpost names the node type it applies to, and site.5.rst
says "on the diskfull nodes". xcatdsklspost serves every type: updatenode calls
it for a diskless or statelite node as well, and the site value is one row that
reaches all of them. With cleanupdiskfullxcatpost=yes and cleanupxcatpost=no,
append_xcatpost_cleanup still appended the delete for those nodes.
The diskful branch now reads NODESETSTATE from the generated mypostscript and
appends nothing for netboot or statelite. That is the signal remoteshell,
hardeths, configeth, otherpkgs and servicenode already use for this question,
and mypostscript.tmpl exports it.
cleanupxcatpost is unchanged. It names no node type and keeps applying to every
one, which xcatdsklspost_xcatpost_cleanup.bats now asserts so a guard added to
the wrong branch shows up.
Four cases cover it: a netboot node and a statelite node keep their postscripts,
a diskful node is still cleaned, and cleanupxcatpost still empties a netboot
node. The first two fail against the parent commit.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
diskless_flat_vmothersetting_machine.bats checked the restore with a substring
match and checked the cleanup only for the absence of "unary operator
expected". A cleanup that writes the machine type back, or leaves it in place,
passed both.
The test now reads the value chdef receives. The restore must write exactly
machine:<type>, and must keep a setting the node already carries. The cleanup
must write an empty value when the machine type is all there is, and must leave
the other setting behind when there is one. The chdef stub brackets its
arguments so an empty value is not the same as no call.
ppc64le is red on the cleanup: the restore ladder writes
machine:pseries-rhel7.6.0 and the cleanup ladder removes machine:pseries-7.6.0,
so the node keeps the machine type. x86_64 and riscv64 pass.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
run_zypper_local_repo extracted six lines of the zypper branch of otherpkgs and
evaluated each one in turn. The branch it measures is an if/else: the repository
is deleted only when the refresh fails. Evaluating the lines separately ran the
delete every time, so the test showed that zypper sd is reachable and never that
a repository which refreshes is kept.
The helper now evaluates the whole branch, from the #use zypper comment to the
apt branch that follows, with pmatch lifted from the same file. The success case
asserts rc=0 and no zypper sd; the failure case keeps its delete assertion.
Flipping the refresh test in otherpkgs to "if [ $? -ne 0 ]" turns both cases
red. The same mutation left the previous helper green, because it never
evaluated that line.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The first subiquity error command redirects tar's output into
/run/#HOSTNAME#-logs.tar. The test shadows tar, but the redirection belongs to
the shell and happens whether or not tar runs:
$ bash -c 'tar() { :; }; ( tar ... >/run/testnode-logs.tar 2>/dev/null; exit 0 )'
bash: line 1: /run/testnode-logs.tar: Permission denied
exit=0
Unprivileged the open fails, "exit 0" swallows it and the test passes. CI runs
as root, where the same line creates or truncates that file on the host.
The archive path now comes from the environment, the way XCAT_ERROR_CONSOLE
already does in the same block, and points inside BATS_TEST_TMPDIR. The tar
shadow emits a marker so the test can tell the output reached the file it
named, and a second assertion says the default path under /run was not touched.
The new case fails until the template reads that variable.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Four cases in xCAT-test/bats/postscripts_otherpkgs.bats fail on the integrated
tree. extract_shell_if_block reports that the upgrade anchor occurs 4 times and
the url repository anchor occurs 2 times, where 1 is expected.
"if [ $hasyum -eq 1 ]; then" opens the upgrade block and also the yum branch of
the preremove, the install and the postremove blocks. "OTHERPKGDIR_INTERNET"
matches the guard that opens the url repository block and an assignment inside
that block. The earlier helper took the first match and said nothing.
otherpkgs_block now passes NTH and TOTAL through to extract_shell_if_block. The
two upgrade cases take occurrence 1 of 4, so a fifth identical line fails the
test instead of moving it. The two url repository cases anchor on the guard
line itself, which occurs once.
Both blocks the earlier helper took are the blocks the assertions describe, so
no case measured the wrong code. On the integrated tree bats -r xCAT-test/bats
gives 113 ok and 0 not ok. On this branch, where the helper is not yet
hardened, the file gives 18 ok.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
A failed package install through the rpm or dpkg fallback leaves no
record in syslog. The same holds for the four package removal branches
that run after the install. The postscript sets its return code, but a
reader of the node log sees nothing, because the package manager output
goes to the console only when VERBOSE is set.
In xCAT/postscripts/otherpkgs the fallback install block, and the yum,
zypper, apt and rpm removal blocks, set RETURNVAL on a non-zero status
and stop there. Only the three repository install branches send a
message with local4.err.
This change adds the same local4.err message to the five branches that
have none. Each message names the command that ran, as the repository
install branches do.
xCAT-test/bats/postscripts_otherpkgs.bats drives each block with a
package manager that returns a failure. Two cases assert the message is
present, and both fail on the parent commit. Two more assert the
success path still logs "installed." and "removed." alone.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The otherpkgs postscript writes two diagnostic lines for each package
sublist: the split package array with its size, and the detected package
manager. On a list with tens of entries these lines fill the updatenode
output and the node log, and hide the install results.
The two echo commands sit in xCAT/postscripts/otherpkgs, after the
IFS split that builds pkgsarray, and no condition guards them. Every
other diagnostic in the script runs only when VERBOSE is set.
This change puts the two lines in the same "if [ $VERBOSE ]" block the
rest of the script uses.
xCAT-test/bats/postscripts_otherpkgs.bats runs the split and the lines
that follow it. One case asserts no output when VERBOSE is empty, and
fails on the parent commit. A second case asserts both lines are still
printed when VERBOSE is set.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The local otherpkgs repository is written with the alias xcat-otherpkgs$localrepoindex, where
localrepoindex is urlrepoindex plus the array index. Lines 833 and 843 of
xCAT/postscripts/otherpkgs then refresh and delete xcat-otherpkgs$index. With http OTHERPKGDIR
entries present urlrepoindex is not zero, so zypper names a repository that block did not add:
the refresh fails for a repository that exists, and the delete removes another one.
Both lines now use $localrepoindex. The log and the echo of a failed SDK repository add name
$bname, which is what zypper ar used; they said bname without the $.
postscripts_otherpkgs.bats drives the add, the refresh and the delete with zypper shadowed and
urlrepoindex set to 2, and reads the alias back out of the repository file the script wrote.
Against the unfixed script the refresh and the delete name xcat-otherpkgs0 while the file says
xcat-otherpkgs2, and the SDK log carries no repository name.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Vinícius Ferrão
Daniel Hilst