xcat-ws.conf keeps its https redirect inside an IfModule mod_rewrite.c
block. No test starts Apache with the fragment, so nothing checks what the
REST aliases answer when that module is not loaded.
wsapi_apache_access.t starts a private Apache on a free port with the
fragment from the source tree. Without mod_rewrite it expects 403 from
/xcatws and /xcatwsv2. With mod_rewrite it expects the 302 to https. In
both cases /install is still served.
The Debian packages do not install xcat-ws.conf, so the GitHub action runs
the test from the source tree after install. The installed copy skips.
The test fails without mod_rewrite: both aliases answer 200 and run the CGI.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The GitHub check of master installs the xcat-dep packages of the latest
channel, which serves the stable release. A master change that needs a
new xcat-dep package then fails the check until that package is in a
stable release. The check now reads the devel channel, which carries
the xcat-dep packages of the next release. The 2.19 branch keeps the
latest channel.
A case passed when its fixture could not run it, and a fixture passed a
stage it left out. provfixture skipped the HTTP stage when the web server
answered 404 for the xCAT aliases, which is the regression P-32 to P-39
exist to catch. It ran the discovery and petitboot scenarios only when
the file under test already existed. CI prints a passing case's output
nowhere, so none of this was visible.
A failed check, a skipped stage and a missing discovery or petitboot
file now fail the case. The DHCP phase fails when only one backend is
installed, since both are shipped.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
dhcptest and provtest carried two copies of the config loader, the
assertion language, the $step.field substitution, the TAP reporter, the
error types and the CLI. The copies had already drifted: list equality,
multi-line regex and CIDR membership behaved differently in each.
blackboxtest keeps one of each. A step type table in steps.py declares
the keys, required keys and reply fields of every type, and the runner
dispatches to the DHCP client or to the dig, tftp, curl and xcatd
clients. One scenario can now use a DHCP reply in a TFTP step.
The shipped scenarios move unchanged to conf/dhcp/ and conf/prov/. Their
scenario, step and assertion lists are identical to the old ones. The
wire cases and fixtures move to autotest/testcase/blackboxtest/, and CI
runs one offline phase instead of two.
Removed because no scenario or fixture used them: the pretty and json
reports, --pcap, --mac, the discover subcommand, the decline, inform
and sleep steps, assert_all, select, and the keys dhcptest accepted and
ignored.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Thirteen test files this branch adds call BAIL_OUT at fifty-one places:
an extraction that stopped matching, a fixture that is not there, a
harness that wrote no log. prove stops every remaining file on a
bail-out, not only the file that called it, so one of them hides the
results of every test that would have run after it. die is just as loud
and costs only its own file.
Fifteen comments the branch added also carried the incident rather than
the constraint. Three pasted an error transcript, five traced a failure
from a macro or a missing file out to a node that never boots, and the
rest counted call sites, package sizes or dracut build numbers. Each now
states the one fact the reader cannot re-derive from the code.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
dhcp_ddns_policy.t and dhcp_isc_expression_grouping.t called BAIL_OUT
when a routine was missing or the plugin could not be read. prove stops
every remaining file on a bail-out, so one of them hides the results of
every test that would have run after it. die is just as loud and costs
only its own file.
Three facts in this branch were each written out in four places. That
SIGHUP reports success before Kea reads the file appears in Kea.pm twice
and in dhcp.pm again; that an installed node netboots when no boot file
reaches it appears in BootPolicy.pm and three times in dhcp.pm; the
dhcpd grouping rule appears in BootPolicy.pm and in the test header. Each
now stands once, without the symptom narration and the defect history
around it.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
master moved 149 commits ahead of the branch point and four files needed a
decision.
xCAT/debian/control and xCATsn/debian/control: master moved nmap and
ipmitool-xcat into Depends, raised the ipmitool version and added the s390x
OpenEmbedded Genesis recommendation. The branch made the genesis-scripts
dependency per architecture. Both are kept, so the ppc64el metapackage depends
on xcat-genesis-scripts-ppc64el and no longer on the amd64 package.
build-utils/lib/XCAT/BuildUtils.pm and xCAT-test/unit/build_utils.t: master
replaced @DEB_ARCHES plus the branch's %NO_RISCV64 exception list with
%ARCH_PACKAGES, which carries the architecture list per package.
deb_package_arches returns the same answer for every package, so master's form
is kept and %NO_RISCV64 is dropped.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
run_cases counts the Passed and Failed lines a run prints, so a case that
crashed before either one was counted nowhere and the phase reported
success over cases that never ran. Each phase now knows how many cases it
asked for and fails when the verdicts do not add up.
A failed query for the case list returned an empty list, which read as a
phase with nothing to do; it is a failure now. A backend whose setup failed
runs its teardown before the next one starts, rather than leaving the
machine configured for a backend that is no longer under test.
The branch's own comments had grown to explanations of the reasoning
behind each decision. Trim them to the fact and its consequence: why a
line exists, and what breaks without it. Long blocks come down to a
hundred words, most to twenty or fewer, and the file headers keep only
their usage tables and the invariant each suite rests on.
No code changes. Every unit test, scenario validation and syntax check
still passes.
The comment named only the DHCP cases; the provisioning ones are kept
out of that phase for the same kind of reason -- they add an
interface, nodes, a network and a rewritten zone.
The provisioning cases go where a node meets them: after the whole
ci_test set, so they are not reconfiguring the machine underneath it, and
before the DHCP phases.
run_provtest_unit_tests runs the offline half first -- the Python suite
and a parse of every scenario -- so a file that does not even load is
reported as that rather than as a case failure, and skips rather than
fails when python3 is absent. run_prov_wire_test then runs the prov_wire
cases once, not once per backend: nothing in them depends on which DHCP
server is installed, and what a node is told over DHCP is dhcptest's
subject.
The CI run is now three phases in order: the perl and bash unit tests,
the ci_test set, then the DHCP on-wire cases.
The wire cases carry dhcp_wire and deliberately not ci_test. They are the
only cases that do not leave the machine alone while they run -- a veth
pair appears, site.dhcpinterfaces and site.dhcpbackend change, the DHCP
daemon is stopped and started -- and although each one restores all of it
from a trap, a ci_test case running in between would be sharing a
management node that is mid-reconfiguration. Holding them out of the set
is also what stops them being run twice.
run_dhcp_wire_test runs the whole set once per backend installed: against
ISC, then against Kea, each pass bracketed by `dhcpfixture.sh
backend-setup` and `backend-teardown`. It reports separately from the
fast regression test and counts case runs rather than cases, since each
case is run once per backend and each failure carries the backend name.
The phase writes its own xcattest configuration without a dhcpbackend
key: xcattest applies [Table_site] before every case, so the regression
conf's `dhcpbackend=isc` would have reset the backend case by case and
left the run silently testing ISC twice.
Which DHCP backend a management node runs is an implementation default of
its distro -- kea on the newer releases, isc on the older ones -- so a
node booting on the same network has to be told the same things either
way. Until now CI exercised whichever backend the runner happened to
configure, which proves half of that and hides every drift between the
two.
The wire cases now carry a dhcp_wire label, and the CI driver holds them
back from the main pass: it asks dhcpfixture.sh which backends are
installed, then runs the whole set against one and again against the
other. Each pass is bracketed by backend-setup, which points
site.dhcpbackend at the backend and stops the other daemon -- two servers
on one wire both answer the same DISCOVER -- and backend-teardown, which
restores the site table and restarts what was running before.
Running every case under one backend and then every case under the other,
rather than switching inside each case, reconfigures the daemon once per
pass instead of once per case, and gives each failure in the summary the
backend name it belongs to. The summary counts case runs rather than
cases, since a wire case is run once per backend.
The check gate is relaxed from [ -x ] to [ -f ]: the fixture invokes
dhcptest as `python3 src/dhcptest`, so the execute bit only matters to
someone running it directly, and testing it made every wire case skip on
Debian.
dhcptest_backend_switch is dropped, along with the fixture's backend and
alt-backend actions: with every case running under both backends there is
nothing left for a case that switches one.
xCAT's DHCP behaviour is tested from two directions today, neither of which
touches the wire: xCAT-test/unit/dhcp_*.t assert on generated config text, and
xCAT-test/integration/dhcp_{isc,kea}_config_validation.t feed that config to a
real daemon and check it parses. Both stop at "the server accepted our
config". Nothing verifies that a client sending option 93 = 0x000b actually
gets an aarch64 loader back.
The one existing wire tool, xCAT-probe/subcmds/detect_dhcpd, hand-packs a
fixed DISCOVER carrying no option 60, 77 or 93 (so it never reaches any arch
branch), parses replies by regexing tcpdump output, and binds an
IO::Socket::INET to a local address, so it cannot run on a provisioning NIC
that has no address yet.
dhcptest fills that gap. It is a Python 3 tool using only the standard library
plus Scapy, speaking raw Layer 2, driving DHCP transactions declared in INI
files and asserting on what came back:
dhcptest run -i eth1 --set net=10.0.0.0/24 conf/full-lease.conf
dhcptest validate conf/*.conf # no root, no network, no scapy
dhcptest list conf/*.conf
dhcptest discover -i eth1 # ad-hoc, no conf file
Scenarios cover DISCOVER/OFFER, the full lease, renew and rebind, reserved
versus pooled addresses, deliberate silence, the PXE architecture matrix and
the iPXE user-class split. Assertions are a small DSL -- target, operator,
value -- over message type, BOOTP header fields, options by number or name,
subnet membership, and the boot file a client would actually use (option 67
or the header, since servers differ on which they fill).
Two boundaries are enforced by unit test rather than by convention:
- The tool is implementation-agnostic. It exchanges packets and checks
fields; it never reads the xCAT database, never runs an xCAT command, and
names no DHCP implementation. Anything that differs between servers is
expressed by whoever writes the .conf.
- Only runner.py imports scapy, so validate and list stay usable in CI on a
host with neither scapy nor root.
Host networking is never modified. The client MAC defaults to a synthetic
locally-administered address, everything goes over a raw L2 socket, no leased
address is ever configured, and the ARP responder answers only for addresses
the session was actually granted.
Variables in .conf files are configparser's own BasicInterpolation, %(name)s,
fed from [vars] and --set. $offer.address and friends are resolved separately
at step-execution time, because they name a reply that has not arrived when
the file is read; BasicInterpolation gives $ no meaning, so the two coexist
without escaping.
Packaging and CI:
- xCAT-test.spec installs dhcptest alongside unit/ and integration/, and
only recommends python3-scapy: it lives in EPEL on EL, and a hard Requires
would make xCAT-test uninstallable on a management node without EPEL.
The deb depends on it outright.
- github_action_xcat_test.pl runs the unit tests and `dhcptest validate`
last, after the fast regression.
- autotest/testcase/dhcptest/cases0 carries the two offline cases plus a
wire case that stands down with an explanation when no provisioning NIC
is available.
Verified against a real DHCP server in a network namespace: 34 lease, renew,
rebind, reservation and silence assertions and 22 PXE/iPXE assertions all
pass, and host networking is untouched before and after.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Shell unit tests were introduced under xCAT-test/autotest/bats, but the existing source-tree unit suite already lives directly under xCAT-test/unit. Keeping the BATS suite under xCAT-test/bats makes the unit-test layout consistent and keeps autotest reserved for xcattest-driven functional cases.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The go-xcat shell behavior tests were written as Perl harnesses, which made the shell assertions harder to read and kept shell-specific setup outside a native shell test framework.
Add BATS to the GitHub Actions dependency set, run BATS tests from the same preserved source tree as the Perl unit suite, and move the go-xcat repository checks into xCAT-test/autotest/bats.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
builddebs.pl replaced build-ubunturepo, and both CD pipelines prefer it: the
fallback to build-ubunturepo fires only for refs that predate builddebs.pl, and
such a ref carries its own copy. Nothing on this branch runs the script, so its
presence only invites edits that never reach a build. The developer guide said
it was kept as a differential oracle until the CD pipelines moved over. They
have.
Remove the script, and record the removal in the build guide beside the
buildcore.sh, makerpm and buildlocal.sh entries.
xcat_probe_package_payload.t asserted the Debian staging by matching a `cp -f`
line in build-ubunturepo. builddebs.pl stages the helpers through
XCAT::BuildUtils::stage_probe_helpers, so the test now calls that function and
checks the files it produced. Verified by making stage_probe_helpers skip a
helper: the assertion fails.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Four fixes from @viniciusferrao's review plus the CI break his review predates.
orig tarball version. dpkg looks for <source>_<upstream>.orig.tar.gz with no
Debian revision, and the call site passed the full Version-Release. The rule now
lives in BuildUtils::upstream_version and orig_tarball_name applies it, so the
call site cannot get it wrong whichever string it is handed. Currently dormant --
every package is Format: 1.0, so the quilt branch does not run, which is why the
differential build did not catch it.
--dest could write to the filesystem root. Cwd::abs_path returns undef when a
PARENT component is missing (a missing leaf is fine), and the caller interpolated
that, so `--dest /no/such/parent/out` became `/debs` and `/xcat-core` at /.
Replaced with BuildUtils::resolve_dest, which is rel2abs and purely lexical --
correct for an output directory that does not exist yet.
Generated debian/control left behind. xCAT-genesis-scripts has no debian/control
of its own; it is generated from control-<arch>. The cleanup restored only files
that already existed, so the generated one stayed. Worse than dirty: ppc64el ran
last, so the restore put back the amd64 BACKUP and the leftover was the wrong
architecture's control, which a later single-arch build would have started from.
with_prepared_tree now records created files and removes them. Verified by a real
build: the checkout is byte-clean afterwards, matching the oracle.
CI install step. build-ubunturepo wrote its repo to $curdir/../../xcat-core,
which under GitHub's work/<repo>/<repo> layout IS $RUNNER_WORKSPACE, so
install_xcat's `./mklocalrepo.sh` happened to be in the directory it chdir'd to.
builddebs.pl writes inside the checkout instead -- that outside-the-checkout path
is what used to rm -rf the tree -- so install_xcat now names the script by its
real location and fails with a clear message if the build produced no repository.
This is what reddened xcat_pr_test at 2m13s; the builder itself was fine (the
exact CI invocation, `./builddebs.pl --force` with no --dest, returns 0 with all
14 packages).
The executable bit was already fixed before the review landed.
Both new helpers are tested and mutation-verified: not stripping the revision
reddens 3 assertions, swapping rel2abs back to abs_path reddens 2. Equivalence
re-measured after these changes -- all 14 packages identical to build-ubunturepo
in control and in every non-changelog file by md5.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
build-ubunturepo was 710 lines of shell doing the Debian half of what
buildrpms.pl does for rpms, with no code in common and a different CLI. It also
carried paths that are dead: GSA uploads, the PROMOTE/PREGA release flows, and a
-d mode that built an xcat-dep repository from a different project's packages.
builddebs.pl replaces it and mirrors buildrpms.pl -- Getopt::Long options, one
package list, build then index then sign -- so the two builders read the same way
and share BuildUtils.pm.
The design rests on one fact: xcat-core debs are Perl. They are byte-identical
for every Ubuntu release, so they are built ONCE and the same files are published
into every codename. Only xCAT, xCATsn and xCAT-genesis-scripts carry an
architecture, and there the difference is packaging metadata, not compiled
output. That is why this needs no sbuild and no per-codename chroot -- unlike
xcat-dep, whose packages are compiled and genuinely differ per release.
BuildUtils.pm holds what both builders need and what was worth making testable:
the Version-Release derivation from the commit time, the xCAT-probe helper
staging, the deb arch and dist tables, the debian/control version pinning, the
changelog rewrite, the reprepro conf generation, and the build lock. Every
function is pure or takes its side effect as an argument, so build_utils.t (45
assertions) drives each one rather than grepping a builder for evidence that it
is called. Verified by mutation: shrinking the arch table reddens 1, dropping
the /g from the control pin reddens 2.
The env-var CLI maps to options: BUILDALL=1 -> --force, GPGSIGN=1 -> --gpg-sign,
GPG_HOME -> --gpg-home, DEST -> --dest, DISTS -> --dist (repeatable). UP=0 has no
equivalent because uploading is gone -- the CD pipeline's deploy step publishes.
Callers updated: github_action_xcat_test.pl and travis.pl. The comment in
github_action_xcat_test.pl explaining why CI copies the tree before building is
corrected -- build-ubunturepo rm -rf'd $curdir/../../xcat-core, which under
GitHub's work/<repo>/<repo> layout is the checkout's own parent; builddebs.pl
writes under dist/debs inside the checkout and restores every file it edits, so
the copy is now only isolating the tests from build residue.
Two tests moved with it. build_ubunturepo_lock.t extracted the lock out of the
shell with a regex and ran that; the lock is now a function, so builddebs_lock.t
calls it -- and asserts what actually matters, that two builds of one checkout
fail fast while two builds of different checkouts run concurrently.
ubuntu_2604_pkglist.t asserted that resolute appeared in a shell fragment of
build-ubunturepo's source; it now asks BuildUtils for the release list and checks
a resolute stanza reaches conf/distributions. That assertion would have passed on
any file containing the fragment and broken on a reflow that changed nothing.
Verified: prove -r xCAT-test/unit fails on 6 files here against 7 on
upstream/master, the difference being apache_config_sources.t, fixed by the
preceding commit. The remaining 6 are missing DB modules on the machine that ran
it and are identical on both.
NOT done here, and required before this can merge: the Ubuntu core CD pipelines
still invoke ./build-ubunturepo (ci/ubuntu/Jenkinsfile.core-ubuntu-{devel,stable}
in VersatusHPC/xcat-core-ci-cd, and the inline script in each live Jenkins job).
Those must be switched to builddebs.pl in the same change window.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The PR test job ran apt-get against the xcat.org repository with no
time limit of its own. When the repository stalled, the job hung on the
update step until the sixty minute workflow limit canceled the run, and
the log gave no reason.
Wrap the apt-get update and install steps in a timeout command and give
apt a transfer timeout with retries. A stalled repository now fails the
step in minutes, the driver prints its install error report, and a
rerun is possible at once.
run_fast_regression_test() prints a case's output only when it fails.
For 250 cases that is the right default, but it leaves no way to tell
whether a passing case did real work or skipped everything. That is not
academic for cases wrapping prove: prove exits 0 both when tests pass
and when every test skips, so integration_tests reports green either
way and the log cannot distinguish them.
Add @verbose_cases. A case named there has its output printed on a pass
as well, and the failure branch no longer prints a second copy. Seed it
with integration_tests to find out which of the three integration tests
actually run on a runner -- in particular whether
dhcp_kea_config_validation.t validates from /etc/kea now that the case
runs as root, or still skips. Emptying the list restores the previous
behaviour exactly.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The unit test stage failed on the first CI run with
Cannot detect source of 'xCAT-test/unit/*.t'
Files=0, Tests=0
Result: NOTESTS
the glob reached prove unexpanded because it matched nothing. The source
tree is gone by the time the tests run: build-ubunturepo sets
local_core_repo_path="$curdir/../../xcat-core"
and rm -rf's it before creating the apt repository there. GitHub checks
out into work/<repo>/<repo>, so for /home/runner/work/xcat-core/xcat-core
that path resolves to the checkout's own parent and the build wipes the
checkout, leaving an empty directory of the same name behind. The cd
still succeeds, which is why prove was handed a literal glob rather than
failing outright. This is also why every testcase that predates this
change proves /opt/xcat/share/xcat/tools/autotest/unit: after the build
the installed copy is the only one left.
Copy the checkout aside in preserve_source_tree() before the build and
prove that copy, so FindBin still resolves to a real source tree. Switch
to `prove -r xCAT-test/unit` as well, so a missing directory fails loudly
instead of silently degrading to a no-op the way an unmatched glob does.
Reproduced and verified by replaying the build under GitHub's directory
layout: the checkout drops to 0 test files, the preserved copy keeps all
47 and proves clean.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The 47 test files under xCAT-test/unit/ were shipped but almost never
executed on a pull request. Only three hand-written xcattest cases
reached them -- dhcp_unit, ipmi_unit and xcatprobe_unit -- and each
proved a single glob against the installed copy, so the majority of the
suite had never run at all. Real drift went unnoticed as a result:
ubuntu_subiquity_template.t still asserted the pre-86e77bcd7 shape of
compute.subiquity.tmpl and failed against the current template.
Run `prove xCAT-test/unit/*.t` directly from github_action_xcat_test.pl.
The tests resolve xCAT modules and fixture files relative to the repo
root through FindBin, so they must be proved from the checkout and not
from /opt/xcat/share/xcat/tools/autotest/unit; install_xcat() chdir's
away, hence the getcwd() captured up front. The step runs after the
install because the suite needs the perl dependencies xCAT pulls in
(Net::DNS, XML::Simple) and a usable xCAT database.
Drop the three prove testcases so their tests do not run twice, and
refresh the two stale ubuntu_subiquity_template.t assertions: the
identity section is now intentional (86e77bcd7) and the MAC
normalization gained cut filters ahead of the tr (c6e38483f), which the
loosened regex plus a new assertion for the suffix stripping now cover.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The xcat-dep APT repo now publishes only the noble (24.04) codename at
.../apt/latest/xcat-dep; the bionic/devel dist no longer exists, so
install_xcat failed on apt-get update/install. Run the workflow on
ubuntu-24.04 and point the apt source at latest/xcat-dep noble main.
Drop the apt-key step: apt-key is removed on Ubuntu 24.04 and the legacy
apt.key is gone from the xcat-dep dir; the repo already installs via
allow-insecure-repositories/allow-unauthenticated.
Daniel Hilst
Vinícius Ferrão
Mark Gurevich