FuncTest test result document overview

Functest project is described in Functest user guide. The user guide details the different test cases as well as the possible errors. A developer guide and a configuration guide are also available.

Functest is run systematically at the end of an OPNFV fresh installation. All the runnable tests are run sequentially. The installer and the scenario are considered to evaluate whether the test case can be run or not. That is why all the number of test cases may vary from 1 installer to another and from 1 scenario to another.

The list of scenario supported by installer can be described as follows:

+================+=========+=========+=========+=========+
|    Scenario    |  Apex   | Compass |  Fuel   |   Joid  |
+================+=========+=========+=========+=========+
|   odl_l2       |    X    |    X    |    X    |     X   |
+----------------+---------+---------+---------+---------+
|   odl_l3       |    X    |         |    X    |         |
+----------------+---------+---------+---------+---------+
|   onos         |    X    |    X    |    X    |     X   |
+----------------+---------+---------+---------+---------+
|   nosdn        |         |    X    |    X    |     X   |
+----------------+---------+---------+---------+---------+
|   odl_l2-sfc   |    X    |         |         |         |
+----------------+---------+---------+---------+---------+
|  odl_l2-bgpvpn |         |         |    X    |         |
+----------------+---------+---------+---------+---------+
|   ovs (dpdk)   |         |         |    X    |         |
+----------------+---------+---------+---------+---------+

The matrix below details the Functest runnable tests versus the installer and the scenario:

+================+=============+=============+=============+=============+
|  Test cases    |    Apex     |   Compass   |    Fuel     |     Joid    |
+================+=============+=============+=============+=============+
|   vPing        |all scenario |     all     |     all     |     all     |
+----------------+-------------+-------------+-------------+-------------+
| vPing userdata | all except  | all except  | all except  | all except  |
|                |   ONOS      |    ONOS     |    ONOS     |    ONOS     |
+----------------+-------------+-------------+-------------+-------------+
|   Tempest      |     all     |     all     |     all     |     all     |
+----------------+-------------+-------------+-------------+-------------+
|   Rally        |     all     |     all     |     all     |     all     |
+----------------+-------------+-------------+-------------+-------------+
|   ODL          |    all ODL  |  all ODL    |  all ODL    |  all ODL    |
+----------------+-------------+-------------+-------------+-------------+
|   ONOS         |    ONOS     |    ONOS     |  ONOS       |     ONOS    |
+----------------+-------------+-------------+-------------+-------------+
|   Promise      |     no      |     no      |    all      |   all       |
+----------------+-------------+-------------+-------------+-------------+
|     vIMS       | all except  | all except  | all except  | all except  |
|                |   ONOS      |    ONOS     |    ONOS     |    ONOS     |
+----------------+-------------+-------------+-------------+-------------+
|   Doctor       |     all     |     no      |    no       |   no        |
+----------------+-------------+-------------+-------------+-------------+
|   SDNVPN       |     no      |     no      |   bgpvpn    |   no        |
+----------------+-------------+-------------+-------------+-------------+
Functest results from continuous integration can be found in:

Rally html pages

Additional test result assests and information

The Functest deals with 2 dashboards:

  • The Test Dashboard provides an overview of all the projects

  • The Functest Dashboard dedicated to Functest real-time test results

    • is used to provide a quick overview of the different testcases according to installer and scenario.