Sometimes the slave RGW takes a little time to re-sync periods
and data from the new master RGW after a promotion.
Increase the number of attempts to retrieve the data written
to the master RGW from the slave RGW units.
* Expose resource_reaches_status' retry options
resource_reaches_status is used in many places to monitor many
different kinds of OpenStack resource. It is reasonable to expect a
flavor to be created in a few seconds where as an image based instance
may take minutes to become active. With that in mind this change
exposes the retry options used by tenacity allowing the caller to
set reasonable expectations for the resource state change.
In addition the image based instance creation call is updated
to retry for longer as this has been timing out.
* Remove swap files
Adapt existing CephRGW test case to deal with multisite
deployment testing.
Validate services across both standard ceph-radosgw and
slave-ceph-radosgw applications if discovered.
Upload test data to source site, validate replication and
delete on target site.
Switch masakari tests to use pacemaker-remote. For this the method
of simulating a host failure has to change to sending the process a
sigterm as shuting down pacemaker-remote any other way is seen as an
orderly shutdown and does not trigger things like stonith.
Also add check to ensure all nodes appear to be online from a
crm pov before starting failover test.
When running the file ready function each unit would download into the
same tmp directory. For files with restrictive permissions this would
lead to a permission denied and false negative on the second unit. i.e.
file is 444 and the subsequent scp fails due to the inability to
overwrite the first file.
Use a separate tmp dir for each unit to avoid this problem.
Centralize the check if the cerfiticates relation to vault exists. If
it does, update authentication information to include the CA
certificate and https as the protocol.
