arif/zaza-openstack-tests
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add test for keystone-openidc (#925)

Browse Source 
* Add keystone-openidc setup code.

The keystone-openidc charm requires 2 configuration steps:

1) Configure the oidc-client-id, oidc-client-secret and
   oidc-provider-metadata-url, this information is tightly related to
   the Identity Provider configured, which for testing purposes this is
   the openidc-test-fixture charm, the setup function
   zaza.openstack.charm_tests.openidc.setup.configure_keystone_openidc
   takes care of setting these values once the fixture charm is ready
   for service.
2) Create the OpenStack objects to correctly configure the federation,
   this is made by the setup function
   zaza.openstack.charm_tests.openidc.setup.keystone_federation_setup_site1
   which will create and configure the following resources:
   - Create a domain named 'federated_domain'.
   - Create a group named 'federated_users'.
   - Grant the 'Member' role to users in the 'federated_users' group.
   - Create an identity provider named 'openid'.
   - Create a mapping named 'openid_mapping'.
   - Create a federation protocol named 'openid' that relates the mapping
     and the identity provider.

* Add support for v3oidcpassword auth plugin.

get_keystone_session() uses the v3.OidcPassword class when the
OS_AUTH_TYPE is set to v3oidcpassword, this class expects the following
extra configuration options:

- OS_IDENTITY_PROVIDER
- OS_PROTOCOL
- OS_CLIENT_ID
- OS_CLIENT_SECRET
- OS_ACCESS_TOKEN_ENDPOINT (optional)
- OS_DISCOVERY_ENDPOINT (optional)

* Add test for keystone-openidc

This patch introduces a new testing class named CharmKeystoneOpenIDCTest
which interacts with keystone using users provided by
openidc-test-fixture via OpenID Connect.

* Add keystone_session argument to launch instances.

Adding the option to pass a keystone session allows callers to use
credentials different from the ones provided by
get_overcloud_keystone_session(), this is helpful when testing non
default keystone configurations (e.g. Federation).

* Add zaza.openstack.charm_tests.openidc.tests.TestLaunchInstance

This testing class configures a private network in the user's project defined by the mapping
rules during the setUpClass stage. Specifically this test performs the following steps:

- Create keypair named 'zaza' in the user's project
- Create a router for the project
- Attach the router to the external network
- Create a network
- Create a subnet attached to the previously create network
- Connect the subnet to the project's router

The testing method launches an instance using a keystone session
associated with a user backed by OpenID Connect.
This commit is contained in:
Felipe Reyes
2022-10-05 09:34:18 -03:00
committed by GitHub
parent 99186a6651
commit a55f320c2a
8 changed files with 516 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
zaza/openstack/charm_tests/keystone_federation/__init__.py Normal file
View File

@@ -0,0 +1,14 @@
# Copyright 2022 Canonical Ltd.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Collection of code to setup Keystone Federation."""

101
zaza/openstack/charm_tests/keystone_federation/utils.py Normal file
View File

@@ -0,0 +1,101 @@
# Copyright 2022 Canonical Ltd.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Code for setting up a Keystone Federation Provider."""
import json
import logging
import keystoneauth1
from zaza.openstack.utilities import (
cli as cli_utils,
openstack as openstack_utils,
)
def keystone_federation_setup(federated_domain: str,
federated_group: str,
idp_name: str,
idp_remote_id: str,
protocol_name: str,
map_template: str,
role_name: str = 'Member',
):
"""Configure Keystone Federation."""
cli_utils.setup_logging()
keystone_session = openstack_utils.get_overcloud_keystone_session()
keystone_client = openstack_utils.get_keystone_session_client(
keystone_session)
try:
domain = keystone_client.domains.find(name=federated_domain)
logging.info('Reusing domain %s with id %s',
federated_domain, domain.id)
except keystoneauth1.exceptions.http.NotFound:
logging.info('Creating domain %s', federated_domain)
domain = keystone_client.domains.create(
federated_domain,
description="Federated Domain",
enabled=True)
try:
group = keystone_client.groups.find(
name=federated_group, domain=domain)
logging.info('Reusing group %s with id %s', federated_group, group.id)
except keystoneauth1.exceptions.http.NotFound:
logging.info('Creating group %s', federated_group)
group = keystone_client.groups.create(
federated_group,
domain=domain,
enabled=True)
role = keystone_client.roles.find(name=role_name)
assert role is not None, 'Role %s not found' % role_name
logging.info('Granting %s role to group %s on domain %s',
role.name, group.name, domain.name)
keystone_client.roles.grant(role, group=group, domain=domain)
try:
idp = keystone_client.federation.identity_providers.get(idp_name)
logging.info('Reusing identity provider %s with id %s',
idp_name, idp.id)
except keystoneauth1.exceptions.http.NotFound:
logging.info('Creating identity provider %s', idp_name)
idp = keystone_client.federation.identity_providers.create(
idp_name,
remote_ids=[idp_remote_id],
domain_id=domain.id,
enabled=True)
JSON_RULES = json.loads(map_template.format(
domain_id=domain.id, group_id=group.id, role_name=role_name))
map_name = "{}_mapping".format(idp_name)
try:
keystone_client.federation.mappings.get(map_name)
logging.info('Reusing mapping %s', map_name)
except keystoneauth1.exceptions.http.NotFound:
logging.info('Creating mapping %s', map_name)
keystone_client.federation.mappings.create(
map_name, rules=JSON_RULES)
try:
keystone_client.federation.protocols.get(idp_name, protocol_name)
logging.info('Reusing protocol %s from identity provider %s',
protocol_name, idp_name)
except keystoneauth1.exceptions.http.NotFound:
logging.info(('Creating protocol %s for identity provider %s with '
'mapping %s'), protocol_name, idp_name, map_name)
keystone_client.federation.protocols.create(
protocol_name, mapping=map_name, identity_provider=idp)

15
zaza/openstack/charm_tests/openidc/__init__.py Normal file
View File

@@ -0,0 +1,15 @@
# Copyright 2022 Canonical Ltd.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Collection of code for setting up and testing Keystone OpenID Connect."""

178
zaza/openstack/charm_tests/openidc/setup.py Normal file
View File

@@ -0,0 +1,178 @@
# Copyright 2022 Canonical Ltd.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Code for setting up Keystone OpenID Connect federation."""
import logging
import zaza.model
from zaza.charm_lifecycle import utils as lifecycle_utils
from zaza.openstack.charm_tests.keystone_federation.utils import (
keystone_federation_setup,
)
from zaza.openstack.utilities import (
cli as cli_utils,
openstack as openstack_utils,
)
APP_NAME = 'keystone-openidc'
FEDERATED_DOMAIN = "federated_domain"
FEDERATED_GROUP = "federated_users"
MEMBER = "Member"
IDP = "openid"
LOCAL_IDP_REMOTE_ID = 'https://{}:8443/realms/demorealm'
REMOTE_ID = "http://openidc"
PROTOCOL_NAME = "openid"
MAP_TEMPLATE = '''
[{{
"local": [
{{
"user": {{
"name": "{{1}}",
"email": "{{2}}"
}},
"group": {{
"name": "{group_id}",
"domain": {{
"id": "{domain_id}"
}}
}},
"projects": [
{{
"name": "{{1}}_project",
"roles": [
{{
"name": "{role_name}"
}}
]
}}
]
}}
],
"remote": [
{{
"type": "HTTP_OIDC_SUB"
}},
{{
"type": "HTTP_OIDC_USERNAME"
}},
{{
"type": "HTTP_OIDC_EMAIL"
}}
]
}}]
'''
REQUIRED_KEYS_MSG = 'required keys: oidc_client_id, oidc_provider_metadata_url'
# Default objects created by openidc-test-fixture charm
DEFAULT_CLIENT_ID = 'keystone'
DEFAULT_CLIENT_SECRET = 'ubuntu11'
DEFAULT_REALM = 'demorealm'
OPENIDC_TEST_FIXTURE = 'openidc-test-fixture' # app's name
# NOTE(freyes): workaround for bug http://pad.lv/1982948
def relate_keystone_openidc():
"""Add relation between keystone and keystone-openidc.
.. note: This is a workaround for the bug http://pad.lv/1982948
"""
cli_utils.setup_logging()
relations_added = False
if not zaza.model.get_relation_id(APP_NAME, 'keystone'):
logging.info('Adding relation keystone-openidc -> keystone')
zaza.model.add_relation(APP_NAME,
'keystone-fid-service-provider',
'keystone:keystone-fid-service-provider')
relations_added = True
if not zaza.model.get_relation_id(APP_NAME, 'openstack-dashboard'):
logging.info('Adding relation keystone-openidc -> openstack-dashboard')
zaza.model.add_relation(
APP_NAME,
'websso-fid-service-provider',
'openstack-dashboard:websso-fid-service-provider'
)
relations_added = True
if relations_added:
zaza.model.wait_for_agent_status()
# NOTE: the test bundle has been deployed with a non-related
# keystone-opendic subordinate application, and thus Zaza is expecting no
# unit from this application. We are now relating it to a principal
# keystone application with 3 units. We now need to make sure we wait for
# the units to get fully deployed before proceeding:
test_config = lifecycle_utils.get_charm_config(fatal=False)
target_deploy_status = test_config.get('target_deploy_status', {})
try:
# this is a HA deployment
target_deploy_status['keystone-openidc']['num-expected-units'] = 3
opts = {
'workload-status-message-prefix': REQUIRED_KEYS_MSG,
'workload-status': 'blocked',
}
target_deploy_status['keystone-openidc'].update(opts)
except KeyError:
# num-expected-units wasn't set to 0, no expectation to be
# fixed, let's move on.
pass
zaza.model.wait_for_application_states(
states=target_deploy_status)
def configure_keystone_openidc():
"""Configure OpenIDC testing fixture certificate."""
units = zaza.model.get_units(OPENIDC_TEST_FIXTURE)
assert len(units) > 0, 'openidc-test-fixture units not found'
ip = zaza.model.get_unit_public_address(units[0])
url = 'https://{ip}:8443/realms/{realm}/.well-known/openid-configuration'
cfg = {'oidc-client-id': DEFAULT_CLIENT_ID,
'oidc-client-secret': DEFAULT_CLIENT_SECRET,
'oidc-provider-metadata-url': url.format(ip=ip,
realm=DEFAULT_REALM)}
zaza.model.set_application_config(APP_NAME, cfg)
zaza.model.wait_for_agent_status()
test_config = lifecycle_utils.get_charm_config(fatal=False)
target_deploy_status = test_config.get('target_deploy_status', {})
target_deploy_status.update({
'keystone-openidc': {
'workload-status': 'active',
'workload-status-message': 'Unit is ready'
},
})
zaza.model.wait_for_application_states(states=target_deploy_status)
def keystone_federation_setup_site1():
"""Configure Keystone Federation for the local IdP #1."""
idp_unit = zaza.model.get_units("openidc-test-fixture")[0]
idp_remote_id = LOCAL_IDP_REMOTE_ID.format(
zaza.model.get_unit_public_address(idp_unit))
keystone_session = openstack_utils.get_overcloud_keystone_session()
keystone_client = openstack_utils.get_keystone_session_client(
keystone_session)
role = keystone_client.roles.find(name=MEMBER)
logging.info('Using role name %s with id %s', role.name, role.id)
keystone_federation_setup(
federated_domain=FEDERATED_DOMAIN,
federated_group=FEDERATED_GROUP,
idp_name=IDP,
idp_remote_id=idp_remote_id,
protocol_name=PROTOCOL_NAME,
map_template=MAP_TEMPLATE,
role_name=role.name,
)

174
zaza/openstack/charm_tests/openidc/tests.py Normal file
View File

@@ -0,0 +1,174 @@
# Copyright 2022 Canonical Ltd.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Keystone OpenID Connect Testing."""
import copy
import logging
import pprint
import zaza.model
from zaza.openstack.charm_tests.glance.setup import CIRROS_IMAGE_NAME
from zaza.openstack.charm_tests.keystone import BaseKeystoneTest
from zaza.openstack.charm_tests.neutron.setup import (
OVERCLOUD_NETWORK_CONFIG,
DEFAULT_UNDERCLOUD_NETWORK_CONFIG,
)
from zaza.openstack.charm_tests.nova.setup import manage_ssh_key
from zaza.openstack.charm_tests.openidc.setup import (
FEDERATED_DOMAIN,
IDP,
PROTOCOL_NAME,
)
from zaza.openstack.utilities import (
generic as generic_utils,
openstack as openstack_utils,
)
# static users created by openidc-test-fixture charm
OIDC_TEST_USER = 'johndoe'
OIDC_TEST_USER_PASSWORD = 'f00bar'
class BaseCharmKeystoneOpenIDC(BaseKeystoneTest):
"""Charm Keystone OpenID Connect tests."""
run_resource_cleanup = True
RESOURCE_PREFIX = 'zaza-openidc'
@classmethod
def setUpClass(cls):
"""Define openrc credentials for OIDC_TEST_USER."""
super().setUpClass()
charm_config = zaza.model.get_application_config('keystone-openidc')
client_id = charm_config['oidc-client-id']['value']
client_secret = charm_config['oidc-client-secret']['value']
metadata_url = charm_config['oidc-provider-metadata-url']['value']
cls.oidc_test_openrc = {
'API_VERSION': 3,
'OS_USERNAME': OIDC_TEST_USER,
'OS_PASSWORD': OIDC_TEST_USER_PASSWORD,
# using the first keystone ip by default, for environments with
# HA+TLS enabled this is the virtual IP, otherwise it will be one
# of the keystone units.
'OS_AUTH_URL': 'https://{}:5000/v3'.format(cls.keystone_ips[0]),
'OS_PROJECT_DOMAIN_NAME': FEDERATED_DOMAIN,
'OS_PROJECT_NAME': '{}_project'.format(OIDC_TEST_USER),
'OS_CACERT': openstack_utils.get_cacert(),
# openid specific info
'OS_AUTH_TYPE': 'v3oidcpassword',
'OS_DISCOVERY_ENDPOINT': metadata_url,
'OS_OPENID_SCOPE': 'openid email profile',
'OS_CLIENT_ID': client_id,
'OS_CLIENT_SECRET': client_secret,
'OS_IDENTITY_PROVIDER': IDP,
'OS_PROTOCOL': PROTOCOL_NAME,
}
logging.info('openrc: %s', pprint.pformat(cls.oidc_test_openrc))
class TestToken(BaseCharmKeystoneOpenIDC):
"""Test tokens for user's backed by OpenID Connect via Federation."""
def test_token_issue(self):
"""Test token issue with a federated user via openidc."""
openrc = copy.deepcopy(self.oidc_test_openrc)
with self.v3_keystone_preferred():
for ip in self.keystone_ips:
logging.info('keystone IP %s', ip)
openrc['AUTH_URL'] = 'https://{}:5000/v3'.format(ip)
keystone_session = openstack_utils.get_keystone_session(
openrc, scope='PROJECT')
logging.info('Retrieving token for federated user')
token = keystone_session.get_token()
logging.info('Token: %s', token)
self.assertIsNotNone(token)
logging.info('OK')
class TestLaunchInstance(BaseCharmKeystoneOpenIDC):
"""Test instance launching in a project defined by Federation mapping."""
@classmethod
def setUpClass(cls):
"""Configure user's project network backed by OpenID Connect."""
super().setUpClass()
# Get network configuration settings
network_config = {"private_net_cidr": "192.168.21.0/24"}
# Declared overcloud settings
network_config.update(OVERCLOUD_NETWORK_CONFIG)
# Default undercloud settings
network_config.update(DEFAULT_UNDERCLOUD_NETWORK_CONFIG)
# Environment specific settings
network_config.update(generic_utils.get_undercloud_env_vars())
ip_version = network_config.get("ip_version") or 4
keystone_session = openstack_utils.get_keystone_session(
cls.oidc_test_openrc, scope='PROJECT')
# find user's project id
project_id = keystone_session.get_project_id()
# Get authenticated clients
neutron_client = openstack_utils.get_neutron_session_client(
keystone_session)
nova_client = openstack_utils.get_nova_session_client(
keystone_session)
# create 'zaza' key in user's project
manage_ssh_key(nova_client)
# create a router attached to the external network
ext_net_name = network_config["external_net_name"]
networks = neutron_client.list_networks(name=ext_net_name)
ext_network = networks['networks'][0]
provider_router = openstack_utils.create_provider_router(
neutron_client, project_id)
openstack_utils.plug_extnet_into_router(
neutron_client,
provider_router,
ext_network)
# create project's private network
project_network = openstack_utils.create_project_network(
neutron_client,
project_id,
shared=False,
network_type=network_config["network_type"],
net_name=network_config["project_net_name"])
project_subnet = openstack_utils.create_project_subnet(
neutron_client,
project_id,
project_network,
network_config["private_net_cidr"],
ip_version=ip_version,
subnet_name=network_config["project_subnet_name"])
openstack_utils.update_subnet_dns(
neutron_client,
project_subnet,
network_config["external_dns"])
openstack_utils.plug_subnet_into_router(
neutron_client,
provider_router['name'],
project_network,
project_subnet)
openstack_utils.add_neutron_secgroup_rules(neutron_client, project_id)
def test_20_launch_instance(self):
"""Test launching an instance in a project defined by mapping rules."""
keystone_session = openstack_utils.get_keystone_session(
self.oidc_test_openrc, scope='PROJECT')
self.launch_guest('test-42',
instance_key=CIRROS_IMAGE_NAME,
keystone_session=keystone_session)

8
zaza/openstack/charm_tests/test_utils.py
View File

@@ -752,7 +752,8 @@ class OpenStackBaseTest(BaseCharmTest):
def launch_guest(self, guest_name, userdata=None, use_boot_volume=False,
instance_key=None, flavor_name=None,
attach_to_external_network=False):
attach_to_external_network=False,
keystone_session=None):
"""Launch one guest to use in tests.
Note that it is up to the caller to have set the RESOURCE_PREFIX class
@@ -772,6 +773,8 @@ class OpenStackBaseTest(BaseCharmTest):
:param attach_to_external_network: Attach instance directly to external
network.
:type attach_to_external_network: bool
:param keystone_session: Keystone session to use.
:type keystone_session: Optional[keystoneauth1.session.Session]
:returns: Nova instance objects
:rtype: Server
"""
@@ -801,7 +804,8 @@ class OpenStackBaseTest(BaseCharmTest):
use_boot_volume=use_boot_volume,
userdata=userdata,
flavor_name=flavor_name,
attach_to_external_network=attach_to_external_network)
attach_to_external_network=attach_to_external_network,
keystone_session=keystone_session)
def launch_guests(self, userdata=None, attach_to_external_network=False,
flavor_name=None):

9
zaza/openstack/configure/guest.py
View File

@@ -94,7 +94,8 @@ def launch_instance_retryer(instance_key, **kwargs):
def launch_instance(instance_key, use_boot_volume=False, vm_name=None,
private_network_name=None, image_name=None,
flavor_name=None, external_network_name=None, meta=None,
userdata=None, attach_to_external_network=False):
userdata=None, attach_to_external_network=False,
keystone_session=None):
"""Launch an instance.
:param instance_key: Key to collect associated config data with.
@@ -120,10 +121,14 @@ def launch_instance(instance_key, use_boot_volume=False, vm_name=None,
:param attach_to_external_network: Attach instance directly to external
network.
:type attach_to_external_network: bool
:param keystone_session: Keystone session to use.
:type keystone_session: Optional[keystoneauth1.session.Session]
:returns: the created instance
:rtype: novaclient.Server
"""
keystone_session = openstack_utils.get_overcloud_keystone_session()
if not keystone_session:
keystone_session = openstack_utils.get_overcloud_keystone_session()
nova_client = openstack_utils.get_nova_session_client(keystone_session)
neutron_client = openstack_utils.get_neutron_session_client(
keystone_session)

23
zaza/openstack/utilities/openstack.py
View File

@@ -299,10 +299,26 @@ def get_ks_creds(cloud_creds, scope='PROJECT'):
'username': cloud_creds['OS_USERNAME'],
'password': cloud_creds['OS_PASSWORD'],
'auth_url': cloud_creds['OS_AUTH_URL'],
'user_domain_name': cloud_creds['OS_USER_DOMAIN_NAME'],
'project_domain_name': cloud_creds['OS_PROJECT_DOMAIN_NAME'],
'project_name': cloud_creds['OS_PROJECT_NAME'],
}
# the FederationBaseAuth class doesn't support the
# 'user_domain_name' argument, so only setting it in the 'auth'
# dict when it's passed in the cloud_creds.
if cloud_creds.get('OS_USER_DOMAIN_NAME'):
auth['user_domain_name'] = cloud_creds['OS_USER_DOMAIN_NAME']
if cloud_creds.get('OS_AUTH_TYPE') == 'v3oidcpassword':
auth.update({
'identity_provider': cloud_creds['OS_IDENTITY_PROVIDER'],
'protocol': cloud_creds['OS_PROTOCOL'],
'client_id': cloud_creds['OS_CLIENT_ID'],
'client_secret': cloud_creds['OS_CLIENT_SECRET'],
# optional configuration options:
'access_token_endpoint': cloud_creds.get(
'OS_ACCESS_TOKEN_ENDPOINT'),
'discovery_endpoint': cloud_creds.get('OS_DISCOVERY_ENDPOINT')
})
return auth
@@ -511,7 +527,10 @@ def get_keystone_session(openrc_creds, scope='PROJECT', verify=None):
if openrc_creds.get('API_VERSION', 2) == 2:
auth = v2.Password(**keystone_creds)
else:
auth = v3.Password(**keystone_creds)
if openrc_creds.get('OS_AUTH_TYPE') == 'v3oidcpassword':
auth = v3.OidcPassword(**keystone_creds)
else:
auth = v3.Password(**keystone_creds)
return session.Session(auth=auth, verify=verify)