202 lines
5.5 KiB
ReStructuredText
202 lines
5.5 KiB
ReStructuredText
Running Charm Tests
|
|
===================
|
|
|
|
The end-to-end tests of a charm are divided into distinct phases. Each phase
|
|
can be run in isolation and tests shared between charms.
|
|
|
|
Running a suite of deployments and tests
|
|
----------------------------------------
|
|
|
|
**functest-run-suite** will read the charms tests.yaml and execute the
|
|
deployments and tests outlined there. However, each phase can be run
|
|
independently.
|
|
|
|
Charm Test Phases
|
|
-----------------
|
|
|
|
Charms should ship with bundles that deploy the charm with different
|
|
application versions, topologies or config options. functest-run-suite will
|
|
run through each phase listed below in order for each bundle that is to be
|
|
tested.
|
|
|
|
0) Environment Variables
|
|
~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
Optionally setting the **MODEL_SETTINGS** environment variable allows model
|
|
settings to be applied to the models created by zaza to run tests in. The
|
|
settings will be applied on top of those set
|
|
**charm_lifecycle.prepare.MODEL_DEFAULTS** so it can be used to override any
|
|
default setting.
|
|
|
|
**MODEL_SETTINGS** should be a list of key/value pairs delimited by
|
|
semicolon e.g.::
|
|
|
|
export MODEL_SETTINGS="no-proxy=jujucharms.com"
|
|
export MODEL_CONSTRAINTS="virt-type=kvm"
|
|
|
|
|
|
1) Prepare
|
|
~~~~~~~~~~
|
|
|
|
Prepare the environment ready for a deployment. At a minimum create a model
|
|
to run the deployment in.
|
|
|
|
To run manually::
|
|
|
|
$ functest-prepare --help
|
|
usage: functest-prepare [-h] -m MODEL_NAME [--log LOGLEVEL]
|
|
|
|
optional arguments:
|
|
-h, --help show this help message and exit
|
|
-m MODEL_NAME, --model-name MODEL_NAME
|
|
Name of new model
|
|
--log LOGLEVEL Loglevel [DEBUG|INFO|WARN|ERROR|CRITICAL]
|
|
|
|
2) Deploy
|
|
~~~~~~~~~
|
|
|
|
Deploy the target bundle and wait for it to complete. **functest-run-suite**
|
|
will look at the list of bundles in the tests.yaml in the charm to determine
|
|
the bundle.
|
|
|
|
In addition to the specified bundle the overlay template directory will be
|
|
searched for a corresponding template (\<bundle\_name\>.j2). If one is found
|
|
then the overlay will be rendered using environment variables a specific set
|
|
of environment variables as context. Currently these are:
|
|
|
|
* FIP\_RANGE
|
|
* GATEWAY
|
|
* NAME\_SERVER
|
|
* NET\_ID
|
|
* OS\_\*
|
|
* VIP\_RANGE
|
|
|
|
The rendered overlay will be used on top of the specified bundle at deploy time.
|
|
|
|
To run manually::
|
|
|
|
$ functest-deploy --help
|
|
usage: functest-deploy [-h] -m MODEL -b BUNDLE [--no-wait] [--log LOGLEVEL]
|
|
|
|
optional arguments:
|
|
-h, --help show this help message and exit
|
|
-m MODEL, --model MODEL
|
|
Model to deploy to
|
|
-b BUNDLE, --bundle BUNDLE
|
|
Bundle name (excluding file ext)
|
|
--no-wait Do not wait for deployment to settle
|
|
--log LOGLEVEL Loglevel [DEBUG|INFO|WARN|ERROR|CRITICAL]
|
|
|
|
|
|
3) Configure
|
|
~~~~~~~~~~~~
|
|
|
|
Post-deployment configuration, for example create network, tenant, image, etc.
|
|
Any necessary post-deploy actions go here. **functest-run-suite** will look
|
|
for a list of functions that should be run in tests.yaml and execute each
|
|
in turn.
|
|
|
|
To run manually::
|
|
|
|
$ functest-configure --help
|
|
usage: functest-configure [-h] [-c CONFIGFUNCS [CONFIGFUNCS ...]] [--log LOGLEVEL]
|
|
|
|
optional arguments:
|
|
-h, --help
|
|
show this help message and exit
|
|
-c CONFIGFUNCS, --configfuncs CONFIGFUNCS
|
|
Space separated list of config functions
|
|
--log LOGLEVEL Loglevel [DEBUG|INFO|WARN|ERROR|CRITICAL]
|
|
|
|
|
|
4) Test
|
|
~~~~~~~
|
|
|
|
Run tests. These maybe tests in zaza or a wrapper around another testing
|
|
framework like rally or tempest. **functest-run-suite** will look for a list
|
|
of test classes that should be run in tests.yaml and execute each in turn.
|
|
|
|
To run manually::
|
|
|
|
$ functest-test --help
|
|
usage: functest-test [-h] [-t TESTS [TESTS ...]] [--log LOGLEVEL]
|
|
|
|
optional arguments:
|
|
-h, --help show this help message and exit
|
|
-t TESTS, --tests TESTS
|
|
Space separated list of test classes
|
|
--log LOGLEVEL Loglevel [DEBUG|INFO|WARN|ERROR|CRITICAL]
|
|
|
|
|
|
5) Collect
|
|
~~~~~~~~~~
|
|
|
|
Collect artifacts useful for debugging any failures or useful for trend
|
|
analysis like deprecation warning or deployment time.
|
|
|
|
|
|
6) Destroy
|
|
~~~~~~~~~~
|
|
|
|
Destroy the model::
|
|
|
|
|
|
$ functest-destroy --help
|
|
usage: functest-destroy [-h] -m MODEL_NAME [--log LOGLEVEL]
|
|
|
|
optional arguments:
|
|
-h, --help show this help message and exit
|
|
-m MODEL_NAME, --model-name MODEL_NAME
|
|
Name of model to remove
|
|
--log LOGLEVEL Loglevel [DEBUG|INFO|WARN|ERROR|CRITICAL]
|
|
|
|
Example
|
|
-------
|
|
|
|
First, grab the charm in question from the charm store::
|
|
|
|
charm pull cs:~openstack-charmers-next/vault
|
|
cd vault
|
|
|
|
Run tests via tox
|
|
~~~~~~~~~~~~~~~~~~
|
|
|
|
To run all the charms functional tests::
|
|
|
|
tox -e func
|
|
|
|
or just the smoke test::
|
|
|
|
tox -e func-smoke
|
|
|
|
Run tests directly with functest commands
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
Setup the virtualenv needs to be created::
|
|
|
|
tox -e func-noop
|
|
source .tox/func-noop/bin/activate
|
|
|
|
All the phases can be run with a single command for a specific bundle::
|
|
|
|
$ functest-run-suite -b xenial-mysql
|
|
|
|
OR each phase can be run by hand,
|
|
|
|
Prepare phase::
|
|
|
|
$ functest-prepare -m testmodel
|
|
|
|
Pick a specific bundle to test::
|
|
|
|
$ functest-deploy -m testmodel -b tests/bundles/xenial-mysql.yaml
|
|
|
|
Run the configure script to prepare the environment for running tests::
|
|
|
|
$ functest-configure -m testmodel
|
|
|
|
Destroy the environment::
|
|
|
|
$ functest-destroy -m testmodel
|
|
|