The case gated only on rc==0, but prove exits 0 both when tests pass and
when every test skips, so the case could report green having run nothing
at all. That is not hypothetical: all three tests carry environment
guards, and on the pull request run the harness discards a passing
case's output, so there was no way to tell from the log whether anything
had executed.
Add check:output=~Files=3, which proves prove found all three files. It
still tolerates a legitimate skip on a node without Kea, and it catches
a packaging regression or a rename. A missing directory was already
caught, since prove -r on a nonexistent path exits 2.
The count has to be maintained by hand when tests are added here, which
the README now says.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
The integration testcase was added without the ci_test label on the
grounds that the pull request workflow has no management node. That was
wrong: github_action_xcat_test.pl builds xCAT, installs it, runs
setup-local-client.sh and chtab, and leaves xcatd running, which makes
the runner a single node management node. Running the ci_test cases
against it is exactly what the fast regression stage already does.
Label the case ci_test so the integration tests get pull request
coverage as well. This also unlocks a test that had been skipping for a
fixable reason: each case is invoked through sudo, and
dhcp_kea_config_validation.t skips only because /etc/kea is not writable
by the unprivileged runner user, so as root it validates instead of
skipping.
Running the integration suite as root and the unit suite unprivileged is
the right way round. Integration tests legitimately need to write to
places like /etc/kea, whereas running the unit tests as root would let
permission-related assertions pass for the wrong reason.
Correct both READMEs, which carried the mistaken claim that the
integration tests do not run in CI.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Three of the files under xCAT-test/unit are not unit tests. They need an
installed management node rather than a checkout: copycds_packages_integrity.t
wants an /install populated by a real copycds, dhcp_kea_config_validation.t
wants a kea-dhcp4 binary that can read the config it generates, and
dhcp_kea_control_agent_smoke.t wants live kea-dhcp4 and kea-ctrl-agent
daemons running as root.
On a GitHub runner none of that exists, so all three plan skip_all. They were
the only three files skipping in the pull request run, which is not a
coincidence -- the skip is the symptom of them being filed in the wrong place.
A skipped test reports neither pass nor fail, so leaving them mixed in with
the unit tests trains the reader to scroll past skips in a directory where a
skip should mean something is wrong.
Move them to xCAT-test/integration, ship that directory alongside unit in
both the rpm and the deb, and drive it from a new xcattest testcase that
proves the installed copy on an MN. The case is deliberately not labelled
ci_test: the pull request workflow has no management node and must not pick
it up. check:rc==0 is the right gate for it -- prove exits non-zero on a real
failure, exits 0 when a test legitimately skips on a node without Kea, and
exits 2 if the directory is missing entirely, so a packaging regression still
fails the case.
Add a README.md to each directory recording which side of the line a new test
belongs on and how each suite is run.
xCAT-test/unit is now 44 files and 802 assertions with no skips at all; the
assertion count is unchanged, confirming the three moved files were
contributing nothing but skips.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Vinícius Ferrão
Daniel Hilst