reg_linux_diskless_installation_flat fails on aarch64 at the check
"lsdef $$CN | grep vmothersetting" matched against "machine". The case
stops there, so the diskless provisioning it exists to measure is never
asserted.
The case sets vmothersetting to machine:invalid, confirms the node still
provisions, then replaces it with a valid machine type and confirms
again. The second step selects that type by architecture and knows only
ppc64, x86_64 and riscv64. On aarch64 the variable stays empty, chdef
clears vmothersetting, and the check that looks for the word "machine"
returns 1.
Give aarch64 the machine type QEMU uses for it, which is virt, the same
value riscv64 already gets. The other three architectures keep the value
they had.
Measured on xcat-ci/xcat-test-el10-aarch64 #9: the case reported Failed
at this check, 08:36:51Z, while genimage and packimage returned 0 and
the node reached status=booted with rootfs on / type tmpfs and
IMAGENAME alma10.2-aarch64-netboot-compute.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
blackbox_prov_failing_at_one_place fails about 1 run in 30 on the
GitHub runner: missing-ptr/getdestiny expects discover and gets the
state of provtestcn, because xcatd still resolves 10.99.1.11 to
provtestcn.
xcatd names a client with gethostbyaddr, which on the runner reaches
systemd-resolved. systemd-resolved rereads /etc/hosts at most every 2
seconds, so for up to 2 seconds after makehosts -d it still returns the
removed entry. do_run_ordering sent getdestiny in that window.
do_run_ordering now waits until getent hosts no longer names the node
before it runs missing-ptr, and fails the stage after 10 seconds.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The case failed on openeuler20.03sp4, openeuler22.03sp4 and openeuler24.03sp4, 3 occurrences
in xcat-ci build 89. The diff check failed, not the extraction.
encrypted_passwd_openssl_diskfull is the only case in passwd/case0 that selects the extraction
by distribution. It greps /etc/*release for SUSE, then for "Red Hat|CentOS|Rocky|Alma", then for
Ubuntu. On an openEuler management node all three miss and openEuler matches, so the case takes
the final arm, prints "Sorry,this is not supported os", writes nothing to
/tmp/instcryptedpasswd, and the diff against the password table reports rc=2.
openEuler now takes the Red Hat arm. compute.openeuler.tmpl writes
"rootpw --iscrypted #CRYPT:passwd:key=system,username=root:password#", which is byte-identical
to every EL template, so field 3 holds the hash and the EL cells already pass with that arm.
The diff check is unchanged, so the case still fails when the extraction reads the wrong field:
field 3 of the expanded line gives diff rc=0 and field 4 gives rc=1.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
(cherry picked from commit a1b9372c413f54d18ba6a60d3839e9f75d7b8261)
confignetwork_secondarynic_nicextraparams_updatenode fails on EL8 because the dump
nic_cfg.sh prints carries no extra params, although configeth wrote CONNECTED_MODE and the
node has it.
nm_show resolves the connection's backend file by globbing
/etc/NetworkManager/system-connections for the UUID. The keyfile plugin keeps a connection
there, and on EL9 and EL10 it is the only plugin. On EL8 the ifcfg-rh plugin owns the
connection and keeps it under /etc/sysconfig/network-scripts, so the glob matches nothing and
no file is dumped. Every check that greps the dump for a value nm_show does not normalize
then sees nothing.
Ask NetworkManager for the path through the UUID,FILENAME fields and keep the glob as a
fallback. Read the MTU from either spelling, because a keyfile writes "mtu=" and an ifcfg
file "MTU=".
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The persist test sourced only xcatlib.sh and never reached the gate in configeth that skipped an
ifcfg profile, so putting that gate back left every test green. Its refusal test also passed on a
127 from a missing function.
The test now calls the repair through xcat_nm_persist_nic_extra_params with nmcli stubbed to
report an ifcfg file and a keyfile. It also captures three faults of xcat_persist_nic_extra_param:
it keeps an old value, it joins the key to a last line with no newline, and it appends a keyfile
key after a section that follows [user].
nic_cfg.bats captures that nm_show prints no profile when NM reports an ifcfg file, so the xcattest
case cannot match CONNECTED_MODE=yes there. nic_cfg.sh returns before its dispatch when sourced.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
xcat-ws.conf keeps its https redirect inside an IfModule mod_rewrite.c
block. No test starts Apache with the fragment, so nothing checks what the
REST aliases answer when that module is not loaded.
wsapi_apache_access.t starts a private Apache on a free port with the
fragment from the source tree. Without mod_rewrite it expects 403 from
/xcatws and /xcatwsv2. With mod_rewrite it expects the 302 to https. In
both cases /install is still served.
The Debian packages do not install xcat-ws.conf, so the GitHub action runs
the test from the source tree after install. The installed copy skips.
The test fails without mod_rewrite: both aliases answer 200 and run the CGI.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The packet test runs dhcpd and kea-dhcp4 in a network namespace with the
rules that BootPolicy renders, and sends relayed DHCPDISCOVER packets
from a second namespace. It checks the boot file of firmware PXE, xNBA
and iPXE clients, with each iPXE feature present or missing, for an xnba
node and an ipxe node that install, for both methods with a SAN disk,
and for an unknown node. On ISC it also covers an iSCSI node of each
method in the boot state, and repeats the node rows after a restart,
when dhcpd reads the host statements back from dhcpd.leases. On Kea it
repeats the rows of the ipxe nodes with a configuration that makedhcp
upgraded, which keeps the xNBA global classes of the older makedhcp -n.
It also checks that the UEFI clients of an xnba node get xNBA when the
server lacks the xNBA UEFI file.
The Kea validation test checks the x86 classes of xnba and ipxe nodes
with kea-dhcp4 -t. XCAT::Test::DHCP builds the packets, and the
integration case expects seven files.
The global Kea classes give the upstream loader to an x86 client that
cannot fetch its script, and the Etherboot class gives it the BIOS
loader. The network classes and the ISC x86 lines give the Genesis
script under xcat/ipxe/nets to a client with the iPXE features of its
path, and the Kea subnet lists those classes. Without an upstream loader
file on the server, neither backend names it or the second stage that it
would fetch, and the ISC lines stop such a client before the fallback.
The xNBA files name no global class.
An xnba node gets xNBA from its own first-stage classes when the server
has the xNBA BIOS file, and still xcat/xnba.efi without the local UEFI
file. Without the xNBA BIOS file it has no first-stage classes, as it
has no ISC host statements. An ipxe node without a SAN disk carries the
upstream loader in its own first-stage classes.
The ISC configuration test also checks these lines in configurations
that makedhcp upgraded, one of them with indented declarations.
The DHCP wire cases now expect the upstream loader for an x86 client
with no reservation, and its network script for a second stage that
reports the features in option 175. A second stage without them gets the
loader again, and an xnba node still gets xNBA.
Replace the dispatch trace fixture with an installed-node xcattest case.
Check the stored password, masked dispatch output, and absence of the
unmasked password in syslog. Restore the debug setting and remove the
temporary node on exit. Keep the redactor unit checks unchanged.
dhcpd keeps the host statements that makedhcp gives it through OMAPI in
dhcpd.leases, where it saves them without parentheses, and reads them
back when it restarts.
The lease test stores the statements of netboot=ipxe nodes through OMAPI
in a private network namespace, checks the saved leases with dhcpd -t,
and restarts dhcpd with them. The configuration test checks the iPXE
feature declarations of the ISC header with dhcpd -t.
start_daemon can now keep the identity of the caller, and the
integration case expects six files.
fix/service-node-el fixed the management-node guard as commit 0f708b96a, with the same
correction this branch made: the log must hold lines, not gain them. The two differ
only in the comment and in the error message, so the file no longer merges as one
change.
The wording here now matches fix/service-node-el. The remaining difference in this file
is the doubled-slash path filter, which that branch does not carry.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
local_logs added XCAT_HTTPD_ACCESS_LOG to the list of candidate logs and then
read the system paths as well. On a build host that runs apache, the check
counted the host's own /var/log/apache2/access.log beside the file the test
pointed at, so a test could not control what it measured. The empty-log case
read 64 lines it did not write.
An explicit XCAT_HTTPD_ACCESS_LOG now replaces the search instead of extending
it. Production behaviour is unchanged: nothing sets that variable there.
Without this change the bats case for an empty management-node log fails on a
host that serves apache, and passes on one that does not.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
(cherry picked from commit bcb4a77f1a)
check_provisioning_source.sh --baseline wrote one message for two conditions:
"no access log to baseline on ${MN_BASE:+$SN}${MN_BASE:-$MN}". When the management
node has a log and the service node does not, the second expansion returns the
management node's baseline spec, not a host name. On xcat42 the message read "no
access log to baseline on nosuchnode-xyz/var/log/httpd/access_log:881". That is the
message an operator reads when xdsh cannot reach the service node, so it has to name
the host that has no log.
Each condition now has its own test and its own message.
check_provisioning_source.bats covers the service-node condition. The case also
asserts the management node's log path is absent from the message, because the node
name alone matches the old text as a substring.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
(cherry picked from commit 89ac4558a0)
On the Ubuntu 24.04 hierarchy cell the check reported the right counts and then
failed: "xcat22-sn served xcat22-cn 277 boot-payload request(s)", "xcat22-mn-dhilst
served xcat22-cn 0", and then "no httpd access log with new entries could be read on
xcat22-mn-dhilst". The guard required the management node log to gain lines after the
baseline. It does not: the management node provisions the service node in
SN_setup_case, before the baseline the hierarchy case takes, and it is then idle. Its
silence is the result the check exists to find.
The guard now asks that the log hold at least one line, which is what makes a count of
0 mean something, and no longer asks for activity in the measured window.
The path filter also missed a doubled leading slash. The compute node fetches the root
image with wget as //install/netboot/<os>/<arch>/compute/rootimg.cpio.gz, which the
service node logged and the filter did not count.
check_provisioning_source.bats covers both. The four cases added in the commit before
this one fail without this change.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
check_provisioning_source.sh counted every request the compute node had ever
made, for any path. Both directions were wrong. A flat run left
management-node requests for the same address, so a later hierarchical run
read them as its own and failed. A service-node entry from an earlier run, or
a 404 for /favicon.ico, satisfied the positive check without any boot payload
being served.
The script now takes a baseline. --baseline records how many lines each access
log holds on the management node and on the service node, before provisioning,
and the check counts only lines after that point. It counts only requests
under /install or /tftpboot, which are the trees httpd serves a boot payload
from. A log shorter than its baseline was rotated, so it is read from its
first line. Without a baseline the check refuses to answer instead of reading
the whole log. The three hierarchy cases call --baseline before nodeset.
check_provisioning_source.bats covers both regressions: a management-node
request before the baseline no longer fails the run, a service-node request
before the baseline no longer satisfies it, and a request that carries no boot
payload does neither. Removing the baseline comparison fails the first two.
Removing the path filter fails the other two.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
A Windows UEFI install deferred to proxyDHCP must be offered option 60
PXEClient and no boot file, or the firmware boots the offered loader
instead of asking proxydhcp-xcat on port 4011.
proxydhcp.conf sends a UEFI DISCOVER with option 60 PXEClient and
asserts the offer carries PXEClient and no boot file. dhcpfixture.sh
run-proxydhcp starts proxydhcp-xcat when it is not running, defines a
win2022 node in chain.currstate install, and stops the daemon at
teardown. blackbox_dhcp_boot_file_conditions runs it.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Merge current master into the openEuler support branch. Keep native
kernel, DHCP, locale and timezone requirements with the upstream payload
verifier and EL10 package paths. Preserve signed openEuler installation.
Update the native fixtures to check the merged payload contract.
The provision-chain fixture defined grub2 nodes for x86_64 only. A mixed
cluster boots ppc64le and riscv64 nodes from the same management node,
and grub2.pm writes their configs against a loader named by the node's
architecture, so a regression there passed CI.
The fixture now defines a grub2 node for ppc64le (grub2.ppc) and one for
riscv64 (grub2.riscv64), set with plain grub2 over TFTP, and runs the
grub2 scenarios against each: loader, per-node config by IP and by MAC,
kernel and initrd, and the boot file DHCP names. A missing loader fails
setup. Both loaders ship in grub2-xcat.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
A case passed when its fixture could not run it, and a fixture passed a
stage it left out. provfixture skipped the HTTP stage when the web server
answered 404 for the xCAT aliases, which is the regression P-32 to P-39
exist to catch. It ran the discovery and petitboot scenarios only when
the file under test already existed. CI prints a passing case's output
nowhere, so none of this was visible.
A failed check, a skipped stage and a missing discovery or petitboot
file now fail the case. The DHCP phase fails when only one backend is
installed, since both are shipped.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
dhcptest and provtest carried two copies of the config loader, the
assertion language, the $step.field substitution, the TAP reporter, the
error types and the CLI. The copies had already drifted: list equality,
multi-line regex and CIDR membership behaved differently in each.
blackboxtest keeps one of each. A step type table in steps.py declares
the keys, required keys and reply fields of every type, and the runner
dispatches to the DHCP client or to the dig, tftp, curl and xcatd
clients. One scenario can now use a DHCP reply in a TFTP step.
The shipped scenarios move unchanged to conf/dhcp/ and conf/prov/. Their
scenario, step and assertion lists are identical to the old ones. The
wire cases and fixtures move to autotest/testcase/blackboxtest/, and CI
runs one offline phase instead of two.
Removed because no scenario or fixture used them: the pretty and json
reports, --pcap, --mac, the discover subcommand, the decline, inform
and sleep steps, assert_all, select, and the keys dhcptest accepted and
ignored.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
reg_linux_diskless_installation_flat corrupts the KVM machine type, proves the
node fails to boot, restores the machine type and then removes it again. On
ppc64 the two ladders name different machine types: the restore writes
machine:pseries-rhel7.6.0 and the cleanup removes machine:pseries-7.6.0. The sed
matches nothing, so the node keeps machine:pseries-rhel7.6.0 after the case ends
and the next case runs against a node the previous one changed.
The cleanup ladder now names the machine type the restore ladder writes.
diskless_flat_vmothersetting_machine.bats covers it: the ppc64le cleanup cases
are red on the commit before this one.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Run the complete postscripts in private mount namespaces. Verify generated
repository scope, native key import, missing trust and repository failures.
Check keeprepo behavior and existing EL upgrade scope. Register both
integration tests.
Thirteen test files this branch adds call BAIL_OUT at fifty-one places:
an extraction that stopped matching, a fixture that is not there, a
harness that wrote no log. prove stops every remaining file on a
bail-out, not only the file that called it, so one of them hides the
results of every test that would have run after it. die is just as loud
and costs only its own file.
Fifteen comments the branch added also carried the incident rather than
the constraint. Three pasted an error transcript, five traced a failure
from a macro or a missing file out to a node that never boots, and the
rest counted call sites, package sizes or dracut build numbers. Each now
states the one fact the reader cannot re-derive from the code.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
reg_linux_diskless_installation_flat fails on an EL 10 riscv64 compute node after the node has
provisioned correctly. The check that fails is "vmothersetting contains machine", and lsdef shows
the attribute empty.
The case corrupts the KVM machine type, checks that the node does not boot, and restores it. The
restore reads str3 from a ladder that names ppc64 and x86_64 only, so on riscv64 str3 is empty and
chdef writes an empty vmothersetting. The remove path later compares str1 against that same empty
str3, which bash reports as "unary operator expected".
Both ladders gain a riscv64 arm giving machine:virt, the machine a riscv64 guest runs and the one
guest_arch_profile in kvm.pm sets. The two comparisons now quote their operands, so an empty value
compares instead of ending the command.
diskless_flat_vmothersetting_machine.t fails five of fifteen assertions without this change.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
master moved 149 commits ahead of the branch point and four files needed a
decision.
xCAT/debian/control and xCATsn/debian/control: master moved nmap and
ipmitool-xcat into Depends, raised the ipmitool version and added the s390x
OpenEmbedded Genesis recommendation. The branch made the genesis-scripts
dependency per architecture. Both are kept, so the ppc64el metapackage depends
on xcat-genesis-scripts-ppc64el and no longer on the amd64 package.
build-utils/lib/XCAT/BuildUtils.pm and xCAT-test/unit/build_utils.t: master
replaced @DEB_ARCHES plus the branch's %NO_RISCV64 exception list with
%ARCH_PACKAGES, which carries the architecture list per package.
deb_package_arches returns the same answer for every package, so master's form
is kept and %NO_RISCV64 is dropped.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
credentials.pm:138-143 signs nothing unless the callback port named in
the request is below 1024, because only root can bind one -- a high
port would let any user on a node collect the cluster's keys.
The suite asserted the closed-port half of that and not this one. The
new scenario puts a listener up on a high port and asks to be called
back there, so a callback arriving is one xcatd chose to make rather
than one the client failed to answer.
P-55 chooses between four sources for imgserver and every one of them
held the same address, so the assertion could not say which won. The
node now has a tftpserver of its own on a second address, and imgserver
is asserted to be it and not site.master.
chain-advances asserted only that two answers differ; it now names the
state, boot, which nodeset leaves in chain.currchain (destiny.pm:550).
And the postscript is read for NODE, MASTER, SITEMASTER and DOMAIN --
four renderings from three sources -- and for any directive that
survived into the body.
The kickstart template carried no directive, so mkinstall rendered it
correctly by doing nothing; it now carries a #TABLE: per node and an
#INCLUDE:, and the fetched autoinst is asserted to hold the expanded
values and no directive or #INCLUDEBAD:.
The postscripts listing is walked as a node walks it: a name taken out
of the index, then that leaf fetched -- Indexes and the alias are
separate directives, and a tree listed but unreadable fails every
postscript at once.
And the kernel and initrd the config names are fetched over both
transports and compared by sha256.
The two loops that undefine the netboot and extra nodes read their list on
standard input, and the xCAT clients called inside them read standard input
too: makedhcp swallowed the rest of the file on the first iteration, so the
loop ended after one name. A full run left fourteen node definitions on the
machine, and the state directory was deleted regardless, so nothing
recorded what had been left.
Both loops read on file descriptor 3, as the provtest teardown already
does. Each removal is checked and teardown keeps its state directory when
one fails. Verified on both backends: no dhcptest node survives a run.
The CI runner is Ubuntu, where the program is grub-mkimage and not
grub2-mkimage, so the fixture built no loader; grub2.pm writes no
configuration at all for a node whose loader is missing, setup died on the
node every later stage needs, and all eight wire cases failed for that one
reason.
Both names are tried now, and the EFI and BIOS module trees under either
/usr/lib/grub or /usr/lib/grub2. Where none of that exists the node is set
with xnba instead, and the stages that read a grub2 config say which
assertions they left out. The workflow installs grub-common and
grub-efi-amd64-bin so the runner tests the grub2 path itself.
check answered only yes or no, so a machine holding an earlier run's nodes,
interface or namespace passed every wire case without running one -- and
went on doing that until someone looked. It now answers three ways: run,
cannot run and the case passes, or refuse, which fails the case. Leftovers
and a cluster with nodes of its own are refusals; PROVTEST_ALLOW_LIVE and
DHCPTEST_ALLOW_LIVE opt in.
The empty-cluster message from lsdef is no longer counted as a node. ss is
required rather than assumed, since without it every port probe answered
yes. Each skipped stage is counted and the total printed. The DNS removal
stage puts the name back and proves it resolves before asserting that it
does not.
Teardown removed nodes, restored tables and unpacked the saved zone tree
without checking any of it, then deleted the state directory regardless.
A restore that half-failed left the machine wrong and the record of what
to put back gone.
Every removal and restore is checked now. The zone archive is verified
when written and listed again before the live tree is removed, so an
unreadable archive leaves the tree alone. Anything that fails keeps the
state directory and returns non-zero. The kickstarts mkinstall renders
under autoinst are removed by name, and the loader moved aside during the
absent-loader stage stays in tftpdir rather than under /tmp.
makedns looks up the server of each zone it writes and qualifies it with
site.domain. On a cluster whose networks name their server as
<xcatmaster> that name is this machine's own, inside the fixture's
domain, which nothing resolves -- so makedns failed and every wire case
died in setup. It passed where the other networks name no server and the
lookup fell back to site.nameservers, an address.
Add the qualified name to /etc/hosts, which teardown already restores.
Both fixtures record what they are about to change so teardown can put
it back, and both truncated that record on entry. A setup run while one
was already up therefore saved the fixture's own site table, dhcpd.conf
and /etc/hosts as the originals, and the veth and namespace it no longer
knew about survived the teardown that followed.
Refuse when the state directory is there, naming the teardown to run.
P-70 asserts that the name DHCP sends can be fetched, but the fixture
passed the architecture's loader as that name. DHCP hands a grub2 node
the per-node name instead, so deleting it left the whole suite green
while a real node would TFTP a file that is not there.
Pass boot/grub2/grub2-<node>, which is what dhcptest asserts is sent.
The scenario stays out of the run where no loader is on disk, since the
per-node name is a link to it.
The branch's own comments had grown to explanations of the reasoning
behind each decision. Trim them to the fact and its consequence: why a
line exists, and what breaks without it. Long blocks come down to a
hundred words, most to twenty or fewer, and the file headers keep only
their usage tables and the invariant each suite rests on.
No code changes. Every unit test, scenario validation and syntax check
still passes.
Four repetitions, each of them a place for a later edit to be made in
one copy and not the others:
- the settings every scenario is given, retyped at nineteen calls
beside a helper that built them and was never called;
- three unit-name lookups with identical bodies;
- the node reset two stages open with;
- the eight settings the three ordering scenarios share.
tftpdir and installdir are asked of xcatd once rather than nine times
in a stage, and the name encodings now call provtest's own netutil
rather than a second implementation in shell that no test covers.
Daniel Hilst
Vinícius Ferrão