Functest Release Notes

1. License

OPNFV Euphrates release note for Functest Docs are licensed under a Creative Commons Attribution 4.0 International License. You should have received a copy of the license along with this. If not, see <http://creativecommons.org/licenses/by/4.0/>.

2. OPNFV Euphrates 5.0 release note for Functest

2.1. Abstract

This document contains the release notes of the Functest project.

2.2. OPNFV Euphrates Release

Functest deals with functional testing of the OPNFV solution. It includes test cases developed within the project, test cases developed in other OPNFV projects and it also intgrates test cases from other upstream communities.

The internal test cases are:

  • connection_check
  • api_check
  • snaps_health_check
  • vping_ssh
  • vping_userdata
  • tempest_smoke_serial
  • refstack_defcore
  • snaps_smoke
  • rally_sanity
  • odl
  • tempest_full_parallel
  • rally_full
  • cloudify_ims
  • vyos_vrouter

The OPNFV projects integrated into Functest framework for automation are:

  • barometer
  • bgpvpn
  • doctor
  • domino
  • fds
  • odl-sfc
  • odl-netvirt
  • parser
  • promise
  • orchestra_openims
  • orchestra_clearwaterims

2.3. Release Data

Project functest
Repo/tag opnfv-5.0.0
Release designation Euphrates initial release
Release date October 20th 2017
Purpose of the delivery Euphrates first release

2.5. Version change

Functest now delivers light-weigth Docker images based on Alpine 3.6. The test cases are grouped into several categories or tiers and must be run from the corresponding container. For example, to run the test case healthcheck, the image opnfv/functest-healthcheck shall be used. The tiers and the tests within them are explained in detail in the User Guide.

For ARM (aarch64), the former Ubuntu image opnfv/functest shall be used since there are not any Alpine images built for this architecture yet. It will probably be supported for Euphrates 5.1.

The Parser test case has its own dedicated Docker image since it requires libraries released for OpenStack Pike and Euphrates is based on Ocata.

The Docker images do not contain OS images (Cirros, Ubuntu, Centos, ..) anymore. A script has been created under the ci directory (download_images.sh) which contains all the needed images for all the tests. This file can be modified by the user since not all the images might be used. It must be executed before starting Functest and attach the needed images as a Docker volume. See Configuration Guide for more information.

The requirements have been split into 3 files:
  • requirements.txt : lists all abstract dependencies of the OPNFV packages
  • test-requirements.txt : lists all abstract dependencies required by Functest Unit Tests
  • upper-constraints.txt : lists all concrete upstream dependencies required by Functest Docker container

OPNFV (test-)requirements.txt have been updated according to stable/ocata global-requirements.txt. Functest uses (and completes) stable/ocata upper-constraints.txt in Dockerfiles and tox configuration. The project relies on pbr, which injects requirements into the install_requires, tests_require and/or dependency_links arguments to setup. It also supports conditional dependencies which can be added to the requirements (e.g. dnspython>=1.14.0;python_version==‘2.7’)

The way to manage logging has been centralized to a configuration file (logging.ini) which might be modified by the user. By default, the output of executing the test cases is redirected to log files and is not displayed on the console, only result messages and summary tables are displayed.

The framework has been refactored and all the test cases inherit from a core class TestCase. For Feature projects who develop test cases, 2 sub-classes have been created:

  • Feature: it implements all the needed functions and the developer must only overwrite the method “execute” (e.g. Barometer)
  • BashFeature: it is used if the third party test case is a shell script. This way, the execution command must be specified in

testcases.yaml as the argument (e.g. Domino, Doctor)

An internal REST API has been introduced in Euphrates. The goal is to trigger Functest operations through an API in addition of the CLI. This could be considered as a first step towards a pseudo micro services approach where the different test projects could expose and consume APIs to the other test projects.

2.6. Euphrates known restrictions/issues

Installer Scenario Issue
fuel@aarch64 any Alpine containers not supported yet for ARM The former Ubuntu Docker image shall be still used for this architecture.
fuel@aarch64 any VNF tier not supported yet.
any any The test cases belonging to the VNF tier have been only tested on os-nosdn-nofeature scenarios and baremetal deployments.
Joid

Compass

 k8 Functest does not offer test suites for Kubernetes scenarios yet.

2.8. Test results

The Functest scenario status on October 20, 2017 can be seen on http://testresults.opnfv.org/functest/euphrates/

Test logs are available in:

2.9. Open JIRA tickets

JIRA Description
   

All the tickets that are not blocking have been fixed or postponed the next release.

Revision:

Author:Jose Lausuch (jalausuch@suse.com)

Build date: October 20, 2017