xcattest reports "Test case <name> has an invalid OS option - Linux" and then
"There is no valid case to run", so every case is skipped on openEuler.
Recorded on an openEuler 24.03 LTS SP4 management node with xCAT 2.20.0: both
flat provisioning cases were rejected and the run measured nothing.
Two causes, both in the os: comparison in xCAT-test/xcattest. get_current_os
takes the /etc/os-release branch, finds no sles, and falls off the end of that
branch, so it returns the exit status of the failed grep, 256, which names no
distribution. And a case marked os:Linux expands to exactly rhels, sles and
ubuntu, so openEuler would be rejected even by a correct get_current_os.
Both halves move into xCAT::Test::OS. xcattest finds it beside itself and the
package installs it at <prefix>/lib/perl. current_os takes a path prefix, so a
test describes a system by writing release files into a scratch tree. openEuler
is named in both halves and nothing else changes.
xCAT-test/unit/xcattest_os_selection.t covers it. Removing openeuler from
linux_aliases fails two checks, and removing the openEuler branch from
current_os fails three. A further case gives a root both /etc/redhat-release
and an openEuler /etc/os-release and asserts rhels10, so the precedence is
named: openEuler ships neither that file nor /etc/lsb-release.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
Thirteen test files this branch adds call BAIL_OUT at fifty-one places:
an extraction that stopped matching, a fixture that is not there, a
harness that wrote no log. prove stops every remaining file on a
bail-out, not only the file that called it, so one of them hides the
results of every test that would have run after it. die is just as loud
and costs only its own file.
Fifteen comments the branch added also carried the incident rather than
the constraint. Three pasted an error transcript, five traced a failure
from a macro or a missing file out to a node that never boots, and the
rest counted call sites, package sizes or dracut build numbers. Each now
states the one fact the reader cannot re-derive from the code.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
A failing case named one failed check and no result at all for the checks after it,
although the commands of the case kept running. The first [Failed] line was read as
the cause of the failure three times this week, and each time the real fault was a
later check: a riscv64 cell reported a makedns check 160 lines before rpower could
not start the domain.
run_case in xCAT-test/xcattest used one variable, $failflag, for two facts: the
result of the case, and the result of the check being reported. Every branch read
$failflag to decide whether to print [Pass] or [Failed], so a check that ran after a
failed one always read as failed. The guard "last if ($failflag)" at the top of the
check loop hid that, and hid every later check with it.
The result of a check is now $checkfail, set and read inside one iteration. A
continue block carries it into $failflag, which keeps the result of the case. The
guard and the per-branch "last" statements are gone, so each check reports what it
found. The output ~~ branch no longer clears $failflag on a match, which without the
guard would have turned a failed case into a passing one.
xCAT-test/unit/xcattest_report_every_check.t runs the harness over a fixture case and
asserts on the CHECK lines it writes. Without this change it reports two of four
checks, and one of two failed checks. A case whose checks all pass logs the same text
before and after: no truncation could happen while $failflag stayed 0.
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
xcattest's runcmd ran each RUN: command via Perl backticks, i.e. /bin/sh.
On Ubuntu /bin/sh is dash, so testcases using bashisms ([ x == y ],
[[ ... ]]) error with "unexpected operator": the [ exits non-zero, the
enclosing if is taken as false, and the case's distro-specific branch is
silently skipped while the case still reports Passed -- a false pass. Run
the command through bash via list-form open() so no intervening /bin/sh
re-parses or re-expands the command's embedded quotes and backticks. EL
is unaffected (its /bin/sh is already bash).
Signed-off-by: Daniel Hilst <392820+dhilst@users.noreply.github.com>
When a test case file is missing the end marker and the last line is a
cmd: directive, $newcmdstart stays set. The parser then appends lines
from the next file into the current command. If that next file is a
shell script (like simulatorctl.sh in the testcase tree), thousands of
lines of Perl/bash get concatenated and executed.
Reset $newcmdstart after closing each file to prevent state leaking
between files.
Fixes#5255
xcattest will replace vars in testcases.
But vars with same prefix will lead to a replacement failure.
Issue:
XCATTEST_PORT=500 XCATTEST_CN=xcat-com001 XCATTEST_PDU=xmount XCATTEST_PDUIP=192.168.200.100 xcattest -t mkdef_pdu_object
cmd:mkdef $$PDU groups=pdu ip=$$PDUIP mgt=pdu nodetype=pdu
will be:
RUN:mkdef xmount groups=pdu ip=xmountIP mgt=pdu nodetype=pdu
$$PDUIP had no chance to be substituted.
Reported-by: Yang Jian <fun_yang@foxmail.com>
Signed-off-by: Chen Hanxiao <chen_han_xiao@126.com>
- to support customize the testcase dir with environment variable XCATTEST_CASEDIR
- to allow to print more debug info with environment variable XCATTEST_DEBUG
- to add a new func to get general object attribute (__GETOBJECTATTR(...)__)
2, allow xcattest get 'var' type config from environment variable when no config file specified.
For example, when given 'XCATTEST_CN=fake1', xcattest will set $$CN=fake1
Daniel Hilst
Vinícius Ferrão
Mark Gurevich
besawn
Wai Yee Wong
Chen Hanxiao
cxhong
Weihua Hu
GONG Jie
robin2008
Victor Hu