mirror of
https://github.com/xcat2/xcat-core.git
synced 2026-10-21 23:28:19 +00:00
Daniel Hilst
c6fc094c93
The Perl tests of the OpenEmbedded Genesis image ran its shell scripts
and C programs through system(), qx, pipes and fork, wrote fake
commands through copied read_file and write_file helpers, and matched
regexes against shell and C source. Many rejections checked only a
non-zero exit, so an earlier, unrelated failure also passed. The
quality gate judged 91% of their 762 checks bad.
The shell and C checks move to bats in xCAT-test/bats. The C programs
are compiled there and run: the plain console, test drivers for the
shared status view and the shell launcher, and genesis-udp-send against
a loopback listener. Each rejection now matches the message of its own
guard. The .t files keep the artifact checks (kas, recipes, kernel
fragments, units, README), read with XCAT::Test::File. Test files left
with no checks are deleted.
Claim map (file -> where its claims went):
- go_xcat_uninstall_installed_only.t -> go_xcat_uninstall_installed_only.bats
- genesis_readme_paths.t, genesis_base_sources_staged.t -> kept in the
.t (README read by slurp_repo_file, tarball read by Archive::Tar)
- genesis_openembedded_{export,hardware,runtime}.t -> the bats file of
the same name
- genesis_openembedded_{bmcsetup,metrics,extensions,qeth,actions,
discovery}.t -> scripts to the bats file of the same name, recipes
and units kept in the .t
- genesis_openembedded_console.t -> built and run checks, state.c,
plain_ui.c and shell.c to genesis_openembedded_console.bats; Newt
source checks kept in the .t
- genesis_openembedded.t -> oe/build to genesis_openembedded_build.bats;
shell matches to the runtime, hardware and extensions bats; console
matches to the console bats or the console .t; artifacts kept
Red proof: every moved or new test was run against a mutated copy of
the code it covers and went red, for example: rpm -q not checked,
tar without --mtime, VEX report not exported, support dir after PATH,
digest check off, layer2 forced, wildcard device allowed, arguments
eval'd, tcp probe skipped, SOURCE_PORT 302, AF_INET only, stage timer
compared, launch error ignored, kas run twice, status written in place.
One mutation stayed green: tar without --owner=0 when the test runs as
root.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
42 lines
1.6 KiB
Perl
Executable File
42 lines
1.6 KiB
Perl
Executable File
#!/usr/bin/env perl
|
|
use strict;
|
|
use warnings;
|
|
|
|
use FindBin;
|
|
use lib "$FindBin::Bin/../lib";
|
|
use Test::More;
|
|
|
|
use XCAT::Test::File qw(repo_path slurp_repo_file);
|
|
|
|
# The Genesis README is the build instructions: every command in it is meant to be pasted. A
|
|
# path that no longer exists is the whole document failing, and a rename is what breaks it --
|
|
# the directory was xCAT-genesis-builder until 2.19 and every path in the README carried that
|
|
# name. The file is the artifact here, so reading it is the test.
|
|
|
|
my $README = 'xCAT-genesis-base/README.md';
|
|
plan skip_all => "$README not found" unless -r repo_path($README);
|
|
|
|
my $text = slurp_repo_file($README);
|
|
|
|
# Every in-tree path the README names, whatever directory it names -- a rename that half
|
|
# happened leaves the old name here, and a pattern matching only the new one would collect
|
|
# nothing and pass. .work is the build's own output directory and is not in the checkout.
|
|
my %named = map { $_ => 1 } grep { !m{/\.work/} }
|
|
($text =~ m{(xCAT-[A-Za-z0-9._-]+/[A-Za-z0-9._/-]+)}g);
|
|
my @named = sort keys %named;
|
|
|
|
# A fixed floor, so a pattern that stops matching cannot shrink the plan to nothing and pass.
|
|
my $EXPECTED_AT_LEAST = 5;
|
|
die sprintf("the README names %d in-tree paths, fewer than the %d it should\n",
|
|
scalar(@named), $EXPECTED_AT_LEAST)
|
|
if @named < $EXPECTED_AT_LEAST;
|
|
|
|
ok(-e repo_path($_), "the README names $_, and it is there") for @named;
|
|
|
|
my ($link) = $text =~ m{\]\((\.\./docs/[^)]+)\)};
|
|
ok($link, 'the README links to the architecture plan');
|
|
ok($link && -e repo_path("docs/" . ($link =~ s{^\.\./docs/}{}r)),
|
|
'and the link resolves from the directory the README sits in');
|
|
|
|
done_testing();
|