OPNFV IPv6 Compliance Test Plan

OPNFV IPv6 Compliance Test Plan

Introduction

The IPv6 compliance test plan outlines the method for testing IPv6 Tenant Network feature compliance with the OPNFV platform.

Scope

This test, and other tests in the test suite, are designed to verify an entire SUT, and not any individual component of the system.

Test suite scope and procedures

The IPv6 compliance test suite will evaluate the ability for a SUT to support IPv6 Tenant Network features and functionality provided by OPNFV platform.

Please refer to the complete list of the test cases for details.

Test suite execution

Please refer to each test case for specific setup and execution procedure.

.._[1]: http://www.opnfv.org

IPv6 test procedure

Draft to be patched this week, someone feel free to work on this in parallel.

Test specification - Service VM as IPv6 vRouter

Draft to be worked on, this represents the YardStick test but I would suggest we need to break this into a set of tests which provide more details per action with boundary validation.

Test Item

TBD -> IPv6 Ping...

Identify the items or features to be tested by this test case. The item description and definition can be referenced from any one of several sources, depending on the level of the test case specification. It may be a good idea to reference the source documents as well.

Environmental requirements

For ipv6 Test Case 18-25, those test cases are scenario tests, they need to boot virtual machines and ping6 in addition to test APIs, ping6 to vRouter is not supported by SDN controller yet, such as Opendaylight (Boron and previous releases), so they are scenario dependent, i.e., currently ipv6 Test Case 18-25 can only run on scenario os-nosdn-nofeature.

Preconditions and procedural requirements

TBD

Input Specifications

TBD

Output Specifications

TBD

Test Reporting

The test report for this test case will be generated with links to relevant data sources. This section can be updated once we have a template for the report in place.

http://testresults.opnfv.org/grafana/dashboard/db/yardstick-tc027

IPv6 test design specification

This document outlines the approach and method for testing IPv6 in the OPNFV compliance test suite. Providing a brief outline of the features to be tested, the methodology for testing, schema’s and criteria.

Features to be tested

The IPv6 compliance test plan outlines the method for testing IPv6 compliance to the OPNFV platform behaviours and features of IPv6 enabled VNFi platforms. The specific features to be tested by the IPv6 compliance test suite is outlined in the following table.

Features / Requirements Tests available Test Cases
All topologies work in a multi-tenant environment No  
IPv6 VM to VM only No  
IPv6 external L2 VLAN directly attached to a VM No  

IPv6 subnet routed via L3 agent to an external IPv6 network

  1. Both VLAN and overlay (e.g. GRE, VXLAN) subnet attached to VMs;
  2. Must be able to support multiple L3 agents for a given external network to support scaling (neutron scheduler to assign vRouters to the L3 agents)
 No  

Ability for a NIC to support both IPv4 and IPv6 (dual stack) address.

  1. VM with a single interface associated with a network, which is then associated with two subnets.
  2. VM with two different interfaces associated with two different networks and two different subnets.
 No  

Support IPv6 Address assignment modes.

  1. SLAAC
  2. DHCPv6 Stateless
  3. DHCPv6 Stateful
 No  
Ability to create a port on an IPv6 DHCPv6 Stateful subnet and assign a specific IPv6 address to the port and have it taken out of the DHCP address pool. No  
Full support for IPv6 matching (i.e., IPv6, ICMPv6, TCP, UDP) in security groups. Ability to control and manage all IPv6 security group capabilities via Neutron/Nova API (REST and CLI) as well as via Horizon. No  
During network/subnet/router create, there should be an option to allow user to specify the type of address management they would like. This includes all options including those low priority if implemented (e.g., toggle on/off router and address prefix advertisements); It must be supported via Neutron API (REST and CLI) as well as via Horizon No  
Security groups anti-spoofing: Prevent VM from using a source IPv6/MAC address which is not assigned to the VM No  
Protect tenant and provider network from rogue RAs No  
Support the ability to assign multiple IPv6 addresses to an interface; both for Neutron router interfaces and VM interfaces. No  
Ability for a VM to support a mix of multiple IPv4 and IPv6 networks, including multiples of the same type. No  
Support for IPv6 Prefix Delegation. No  
IPv6 First-Hop Security, IPv6 ND spoofing No  
IPv6 support in Neutron Layer3 High Availability (keepalived+VRRP). No  

Test approach for IPv6

The most common approach for testing IPv6 capabilities in the test suite is through interaction with the SUT control plane. In this instance the test framework will exercise the NBI provided by the VIM to configure and leverage IPv6 related features in the platform, instantiate workloads, and invoke behaviours in the platform. The suite may also interact directly with the data plane to exercise platform capabilities and further invoke helper functions on the platform for the same purpose.

Test result analysis

All functional tests in the IPv6 test suite will provide a pass/fail result on completion of the test. In addition test logs and relevant additional information will be provided as part of the test log, available on test suite completion.

Some tests in the compliance suite measure such metrics as latency and performance. At this time these tests are intended to provide a feature based pass/fail metric not related to system performance. These tests may however provide detailed results of performance and latency in the ‘test report’_ document.

Test identification

TBD: WE need to identify the test naming scheme we will use in DoveTail in order that we can cross reference to the test projects and maintain our suite effectively. This naming scheme needs to be externally relevant to non-OPNFV consumers and as such some consideration is required on the selection.

Pass Fail Criteria

This section requires some further work with the test teams to identify how and where we generate, store and provide results.

Dovetail IPv6 tc001 specification - Bulk Creation and Deletion of IPv6 Networks, Ports and Subnets

test case name Bulk creation and deletion of IPv6 networks, ports and subnets
id dovetail.ipv6.tc001
objective To verify that platform is able to create/delete networks, ports and subnets in bulk operation
test items tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_network {idempotent_id(‘d4f9024d-1e28-4fc1-a6b1-25dbc6fa11e2’)} tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_port {idempotent_id(‘48037ff2-e889-4c3b-b86a-8e3f34d2d060’)} tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_subnet {idempotent_id(‘8936533b-c0aa-4f29-8e53-6cc873aec489’)}
environmental requirements & preconditions environment can be deployed on bare metal of virtualized infrastructure deployment can be HA or non-HA
scenario dependencies NA
procedural requirements
Step 1: create/delete network:
create 2 networks in one request asserting that the networks are found in the list after creation
Step 2: create/delete subnet:
create 2 subnets in one request asserting that the subnets are found in the list after creation
Step 3: create/delete port:
create 2 ports in one request asserting that the ports are found in the list after creation
input specifications The parameters needed to execute Neutron network APIs. Refer to Neutron Networking API v2.0 [1] [2]
output specifications The responses after executing Network network APIs. Refer to Neutron Networking API v2.0 [1] [2]
pass/fail criteria If normal response code 200 is returned, the test passes. Otherwise, the test fails with various error codes. Refer to Neutron Networking API v2.0 [1] [2]
test report TBD

Dovetail IPv6 tc026 specification - Service VM as IPv6 vRouter

test case name Service VM as IPv6 vRouter
id dovetail.ipv6.tc026
objective IPv6 connnectivity, service VM as IPv6 vRouter
modules under test neutron, nova, etc
dependent test project yardstick
test items yardstick_tc027
environmental requirements & preconditions OpenStack-only environment environment can be deplyed on bare metal of virtualized infrastructure deployment can be HA or non-HA test case image needs to be installed into Glance with ping6 included
scenario dependencies nosdn
procedural requirements
step 1: to setup IPv6 testing environment
1.1 disable security group 1.2 create (ipv6, ipv4) router, network and subnet 1.3 create vRouter, VM1, VM2
step 2: to run ping6 to verify IPv6 connectivity
2.1 ssh to VM1 2.2 ping6 to ipv6 router from VM1 2.3 get the result and store the logs
step 3: to teardown IPv6 testing environment
3.1 delete vRouter, VM1, VM2 3.2 delete (ipv6, ipv4) router, network and subnet 3.3 enable security group
input specifications packetsize: 56 ping_count: 5
output specifications output includes max_rtt, min_rtt, average_rtt
pass/fail criteria ping6 connectivity success, no SLA
test report dovetail dashboard DB here

IPv6 Compliance Testing Methodology and Test Cases

IPv6 Compliance Testing focuses on overlay IPv6 capabilities, i.e. to validate that IPv6 capability is supported in tenant networks, subnets and routers. Both Tempest API testing and Tempest Scenario testing are reused as much as we can in IPv6 Compliance Testing. In addition, Yardstick Test Case 027 is also used to validate a specific use case of using a Service VM as an IPv6 vRouter.

IPv6 Compliance Testing test cases are described as follows:

Test Case 1: Create and Delete an IPv6 Network, Port and Subnet

tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_network
tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_port
tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_subnet

Test Case 2: Create, Update and Delete an IPv6 Network and Subnet

tempest.api.network.test_networks.NetworksIpV6Test.test_create_update_delete_network_subnet

Test Case 3: Check External Network Visibility

tempest.api.network.test_networks.NetworksIpV6Test.test_external_network_visibility

Test Case 4: List IPv6 Networks and Subnets of a Tenant

tempest.api.network.test_networks.NetworksIpV6Test.test_list_networks
tempest.api.network.test_networks.NetworksIpV6Test.test_list_subnets

Test Case 5: Show Information of an IPv6 Network and Subnet

tempest.api.network.test_networks.NetworksIpV6Test.test_show_network
tempest.api.network.test_networks.NetworksIpV6Test.test_show_subnet

Test Case 6: Create an IPv6 Port in Allowed Allocation Pools

tempest.api.network.test_ports.PortsIpV6TestJSON.test_create_port_in_allowed_allocation_pools

Test Case 7: Create an IPv6 Port without Security Groups

tempest.api.network.test_ports.PortsIpV6TestJSON.test_create_port_with_no_securitygroups

Test Case 8: Create, Update and Delete an IPv6 Port

tempest.api.network.test_ports.PortsIpV6TestJSON.test_create_update_delete_port

Test Case 9: List IPv6 Ports of a Tenant

tempest.api.network.test_ports.PortsIpV6TestJSON.test_list_ports

Test Case 10: Show Information of an IPv6 Port

tempest.api.network.test_ports.PortsIpV6TestJSON.test_show_port

Test Case 11: Add Multiple Interfaces for an IPv6 Router

tempest.api.network.test_routers.RoutersIpV6Test.test_add_multiple_router_interfaces

Test Case 12: Add and Remove an IPv6 Router Interface with port_id

tempest.api.network.test_routers.RoutersIpV6Test.test_add_remove_router_interface_with_port_id

Test Case 13: Add and Remove an IPv6 Router Interface with subnet_id

tempest.api.network.test_routers.RoutersIpV6Test.test_add_remove_router_interface_with_subnet_id

Test Case 14: Create, Update, Delete, List and Show an IPv6 Router

tempest.api.network.test_routers.RoutersIpV6Test.test_create_show_list_update_delete_router

Test Case 15: Create, Update, Delete, List and Show an IPv6 Security Group

tempest.api.network.test_security_groups.SecGroupIPv6Test.test_create_list_update_show_delete_security_group

Test Case 16: Create, Delete and Show Security Group Rules

tempest.api.network.test_security_groups.SecGroupIPv6Test.test_create_show_delete_security_group_rule

Test Case 17: List All Security Groups

tempest.api.network.test_security_groups.SecGroupIPv6Test.test_list_security_groups

Test Case 18: IPv6 Address Assignment - DHCPv6 Stateless

tempest.scenario.test_network_v6.TestGettingAddress.test_dhcp6_stateless_from_os

Test Case 19: IPv6 Address Assignment - Dual Stack, DHCPv6 Stateless

tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_dhcp6_stateless_from_os

Test Case 20: IPv6 Address Assignment - Multiple Prefixes, DHCPv6 Stateless

tempest.scenario.test_network_v6.TestGettingAddress.test_multi_prefix_dhcpv6_stateless

Test Case 21: IPv6 Address Assignment - Dual Stack, Multiple Prefixes, DHCPv6 Stateless

tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_multi_prefix_dhcpv6_stateless

Test Case 22: IPv6 Address Assignment - SLAAC

tempest.scenario.test_network_v6.TestGettingAddress.test_slaac_from_os

Test Case 23: IPv6 Address Assignment - Dual Stack, SLAAC

tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_slaac_from_os

Test Case 24: IPv6 Address Assignment - Multiple Prefixes, SLAAC

tempest.scenario.test_network_v6.TestGettingAddress.test_multi_prefix_slaac

Test Case 25: IPv6 Address Assignment - Dual Stack, Multiple Prefixes, SLAAC

tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_multi_prefix_slaac

Test Case 26: Service VM as an IPv6 vRouter

# Refer to Yardstick Test Case 027
# Instruction: http://artifacts.opnfv.org/ipv6/docs/configurationguide/index.html
# Step 1: Set up Service VM as an IPv6 vRouter
#    1.1: Install OPNFV and Preparation
#    1.2: Disable Security Groups in OpenStack ML2 Setup
#    1.3: Create IPv4 and IPv6 Neutron routers, networks and subnets
#    1.4: Boot vRouter VM, and Guest VM1 and Guest VM2
# Step 2: Verify IPv6 Connectivity
#    2.1: ssh to Guest VM1
#    2.2: Ping6 from Guest VM1 to Guest VM2
#    2.3: Ping6 from Guest VM1 to vRouter VM
#    2.4: Ping6 from Guest VM1 to Neutron IPv6 Router Namespace
# Step 3: Tear down Setup
#    3.1: Delete Guest VM1, Guest VM2 and vRouter VM
#    3.2: Delete IPv4 and IPv6 Neutron routers, networks and subnets
#    3.3: Enable Security Groups