xCAT-test/debian/install did not list lib/xCAT/Test/OS.pm, so the
Debian package shipped xcattest without the module it loads, and
xcattest stopped at startup with "Can't locate xCAT/Test/OS.pm in @INC".
Install the module to opt/xcat/lib/perl/xCAT/Test, as the RPM spec does.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
dhcptest and provtest carried two copies of the config loader, the
assertion language, the $step.field substitution, the TAP reporter, the
error types and the CLI. The copies had already drifted: list equality,
multi-line regex and CIDR membership behaved differently in each.
blackboxtest keeps one of each. A step type table in steps.py declares
the keys, required keys and reply fields of every type, and the runner
dispatches to the DHCP client or to the dig, tftp, curl and xcatd
clients. One scenario can now use a DHCP reply in a TFTP step.
The shipped scenarios move unchanged to conf/dhcp/ and conf/prov/. Their
scenario, step and assertion lists are identical to the old ones. The
wire cases and fixtures move to autotest/testcase/blackboxtest/, and CI
runs one offline phase instead of two.
Removed because no scenario or fixture used them: the pretty and json
reports, --pcap, --mac, the discover subcommand, the decline, inform
and sleep steps, assert_all, select, and the keys dhcptest accepted and
ignored.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Ten cases: two offline ones labelled ci_test, the unit suite and a parse
of every shipped scenario, and eight wire cases labelled prov_wire, one
per stage of the chain a node walks -- DNS, loader and config, install
tree, genesis and discovery artefacts, discovery requests, the xcatd
protocols, the install monitor, and the three failures that look alike
from a console.
The wire cases are deliberately not ci_test: while one runs the
management node has an extra veth pair and namespace, four nodes, a
rewritten zone and possibly a moved http port. Each restores all of it
from a trap, and each passes with a message when the host cannot host it.
provfixture.sh holds every xCAT command; the scenarios read none of it.
xCAT's DHCP behaviour is tested from two directions today, neither of which
touches the wire: xCAT-test/unit/dhcp_*.t assert on generated config text, and
xCAT-test/integration/dhcp_{isc,kea}_config_validation.t feed that config to a
real daemon and check it parses. Both stop at "the server accepted our
config". Nothing verifies that a client sending option 93 = 0x000b actually
gets an aarch64 loader back.
The one existing wire tool, xCAT-probe/subcmds/detect_dhcpd, hand-packs a
fixed DISCOVER carrying no option 60, 77 or 93 (so it never reaches any arch
branch), parses replies by regexing tcpdump output, and binds an
IO::Socket::INET to a local address, so it cannot run on a provisioning NIC
that has no address yet.
dhcptest fills that gap. It is a Python 3 tool using only the standard library
plus Scapy, speaking raw Layer 2, driving DHCP transactions declared in INI
files and asserting on what came back:
dhcptest run -i eth1 --set net=10.0.0.0/24 conf/full-lease.conf
dhcptest validate conf/*.conf # no root, no network, no scapy
dhcptest list conf/*.conf
dhcptest discover -i eth1 # ad-hoc, no conf file
Scenarios cover DISCOVER/OFFER, the full lease, renew and rebind, reserved
versus pooled addresses, deliberate silence, the PXE architecture matrix and
the iPXE user-class split. Assertions are a small DSL -- target, operator,
value -- over message type, BOOTP header fields, options by number or name,
subnet membership, and the boot file a client would actually use (option 67
or the header, since servers differ on which they fill).
Two boundaries are enforced by unit test rather than by convention:
- The tool is implementation-agnostic. It exchanges packets and checks
fields; it never reads the xCAT database, never runs an xCAT command, and
names no DHCP implementation. Anything that differs between servers is
expressed by whoever writes the .conf.
- Only runner.py imports scapy, so validate and list stay usable in CI on a
host with neither scapy nor root.
Host networking is never modified. The client MAC defaults to a synthetic
locally-administered address, everything goes over a raw L2 socket, no leased
address is ever configured, and the ARP responder answers only for addresses
the session was actually granted.
Variables in .conf files are configparser's own BasicInterpolation, %(name)s,
fed from [vars] and --set. $offer.address and friends are resolved separately
at step-execution time, because they name a reply that has not arrived when
the file is read; BasicInterpolation gives $ no meaning, so the two coexist
without escaping.
Packaging and CI:
- xCAT-test.spec installs dhcptest alongside unit/ and integration/, and
only recommends python3-scapy: it lives in EPEL on EL, and a hard Requires
would make xCAT-test uninstallable on a management node without EPEL.
The deb depends on it outright.
- github_action_xcat_test.pl runs the unit tests and `dhcptest validate`
last, after the fast regression.
- autotest/testcase/dhcptest/cases0 carries the two offline cases plus a
wire case that stands down with an explanation when no provisioning NIC
is available.
Verified against a real DHCP server in a network namespace: 34 lease, renew,
rebind, reservation and silence assertions and 22 PXE/iPXE assertions all
pass, and host networking is untouched before and after.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Three of the files under xCAT-test/unit are not unit tests. They need an
installed management node rather than a checkout: copycds_packages_integrity.t
wants an /install populated by a real copycds, dhcp_kea_config_validation.t
wants a kea-dhcp4 binary that can read the config it generates, and
dhcp_kea_control_agent_smoke.t wants live kea-dhcp4 and kea-ctrl-agent
daemons running as root.
On a GitHub runner none of that exists, so all three plan skip_all. They were
the only three files skipping in the pull request run, which is not a
coincidence -- the skip is the symptom of them being filed in the wrong place.
A skipped test reports neither pass nor fail, so leaving them mixed in with
the unit tests trains the reader to scroll past skips in a directory where a
skip should mean something is wrong.
Move them to xCAT-test/integration, ship that directory alongside unit in
both the rpm and the deb, and drive it from a new xcattest testcase that
proves the installed copy on an MN. The case is deliberately not labelled
ci_test: the pull request workflow has no management node and must not pick
it up. check:rc==0 is the right gate for it -- prove exits non-zero on a real
failure, exits 0 when a test legitimately skips on a node without Kea, and
exits 2 if the directory is missing entirely, so a packaging regression still
fails the case.
Add a README.md to each directory recording which side of the line a new test
belongs on and how each suite is run.
xCAT-test/unit is now 44 files and 802 assertions with no skips at all; the
assertion count is unchanged, confirming the three moved files were
contributing nothing but skips.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
- refine the xcatperftest tool to make it be easily integrated into xcattest
- add testcase/performance/.. to trigger the performance testcase
- fix manpage missing issue for xcatperftest
* Feature #2681, to add a script in xCAT-test to get the performance baseline for node range operation.
* adopt review comments in PR#2916:
1, add the rst file for xcatperftest (#2681)
2, using /bin/bash as script executor
3, FILES in manpage should not be in the same line
4, minor adjustment if PERF_NOCREATE/PERF_DRYRUN is set
from the builds. Also remove the changelog because that file is going
to be ignored from the source tree. Ubuntu builds are being done
in the xcat-core directory and causing issues when we are switching
between releases. Should probably change the builds scripts as a longer
term solution
Daniel Hilst
Vinícius Ferrão
GONG Jie
Bin Xu
Bin Xu
Victor Hu
Victor Hu
xq2005
arifali
adalbas