2018-08-16 13:36:26.045 11431 INFO rally.api [-] Starting verification (UUID=ae1a2f14-6797-438f-bb50-3433e328d6ea) for deployment 'opnfv-rally' (UUID=3e6d0f86-cfdb-4138-9cd2-5cf14f99b3f1) by verifier 'opnfv-tempest' (UUID=fe1faab8-065d-496a-9707-352093dfc452).
2018-08-16 13:36:26.905 11431 WARNING oslo_config.cfg [-] Option "swift_operator_role" from group "tempest" is deprecated. Use option "swift_operator_role" from group "openstack".
2018-08-16 13:36:27.327 11431 WARNING oslo_config.cfg [-] Option "img_name_regex" from group "tempest" is deprecated. Use option "img_name_regex" from group "openstack".
2018-08-16 13:36:29.134 11431 INFO rally.task.context [-] Verification fe1faab8-065d-496a-9707-352093dfc452 | Context tempest@default setup()  finished in 2.56 sec
2018-08-16 13:36:29.137 11431 INFO rally.task.context [-] Verification fe1faab8-065d-496a-9707-352093dfc452 | Context testr@default setup()  finished in 2.02 msec
2018-08-16 13:36:42.707 11431 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_associate_disassociate_network ... success [5.031s]
2018-08-16 13:36:48.965 11431 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_associate_disassociate_router ... success [6.257s]
2018-08-16 13:36:50.513 11431 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_associate_invalid_network ... success [1.548s]
2018-08-16 13:36:58.507 11431 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_attach_associated_subnet_to_associated_router ... success [7.993s]
2018-08-16 13:36:59.021 11431 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_create_bgpvpn ... success [0.515s]
2018-08-16 13:36:59.036 11431 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_create_bgpvpn_as_non_admin_fail ... success [0.014s]
2018-08-16 13:36:59.841 11431 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_create_bgpvpn_with_invalid_routetargets ... success [0.805s]
2018-08-16 13:37:01.287 11431 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_delete_bgpvpn ... success [1.445s]
2018-08-16 13:37:01.687 11431 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_delete_bgpvpn_as_non_admin_fail ... success [0.400s]
2018-08-16 13:37:04.063 11431 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_disassociate_invalid_network ... success [2.376s]
2018-08-16 13:37:05.193 11431 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_list_bgpvpn ... success [1.130s]
2018-08-16 13:37:06.308 11431 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_list_bgpvpn_as_non_owner_fail ... success [1.115s]
2018-08-16 13:37:08.180 11431 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_list_netassoc_as_non_owner_fail ... success [1.870s]
2018-08-16 13:37:09.826 11431 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_list_show_network_association ... success [1.646s]
2018-08-16 13:37:12.242 11431 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_list_show_router_association ... success [2.415s]
2018-08-16 13:37:12.783 11431 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_show_bgpvpn ... success [0.541s]
2018-08-16 13:37:13.464 11431 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_show_bgpvpn_as_non_owner_fail ... success [0.681s]
2018-08-16 13:37:15.079 11431 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_show_netassoc_as_non_owner_fail ... success [1.614s]
2018-08-16 13:37:15.992 11431 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_update_bgpvpn_invalid_routetargets ... success [0.913s]
2018-08-16 13:37:16.210 11431 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_update_route_target ... fail [0.213s]
2018-08-16 13:37:16.407 11431 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_update_route_target_non_admin_fail ... success [0.200s]
2018-08-16 13:42:57.689 11431 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_basic ... fail [380.366s]
2018-08-16 13:49:19.653 11431 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_negative_delete_bgpvpn ... fail [381.950s]
2018-08-16 13:55:28.037 11431 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_negative_delete_net_association ... fail [368.364s]
2018-08-16 14:02:00.933 11431 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_negative_delete_router_association ... fail [392.880s]
2018-08-16 14:03:23.625 11431 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_negative_disjoint_import_export ... success [82.681s]
2018-08-16 14:04:30.054 11431 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_negative_ping_to_unassociated_net ... success [66.421s]
2018-08-16 14:10:46.621 11431 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_negative_update_to_disjoint_import_export ... fail [376.549s]
2018-08-16 14:16:56.281 11431 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_negative_update_to_empty_rt_list ... fail [369.644s]
2018-08-16 14:16:59.731 11431 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_port_association_create_and_delete ... skip [3.452s]: Skipped because network extension: bgpvpn-routes-control is not enabled
2018-08-16 14:17:02.416 11431 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_port_association_create_and_update ... skip [2.683s]: Skipped because network extension: bgpvpn-routes-control is not enabled
2018-08-16 14:17:05.083 11431 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_port_association_local_pref ... skip [2.665s]: Skipped because network extension: bgpvpn-routes-control is not enabled
2018-08-16 14:20:45.638 11431 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_tenant_separation_and_local_connectivity ... success [220.525s]
2018-08-16 14:22:04.561 11431 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_update_route_targets_and_unassociated_net ... fail [78.904s]
2018-08-16 14:28:27.040 11431 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_update_route_targets_common_target ... fail [382.463s]
2018-08-16 14:34:48.637 11431 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_update_route_targets_disjoint_targets ... fail [381.582s]
2018-08-16 14:37:33.574 11431 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_update_rt_and_keep_local_connectivity_variant1 ... fail [164.909s]
2018-08-16 14:42:57.678 11431 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_update_rt_and_keep_local_connectivity_variant2 ... fail [324.052s]
2018-08-16 14:49:22.911 11431 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_variant1 ... fail [385.214s]
2018-08-16 14:55:51.029 11431 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_variant2 ... fail [388.101s]
2018-08-16 14:56:15.598 11431 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_variant3 ... fail [24.564s]
2018-08-16 15:02:42.993 11431 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_variant4 ... fail [387.380s]
2018-08-16 15:09:04.630 11431 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_variant5 ... fail [381.622s]
2018-08-16 15:15:34.193 11431 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_variant6 ... fail [389.549s]
2018-08-16 15:15:55.237 11431 INFO rally.task.context [-] Verification fe1faab8-065d-496a-9707-352093dfc452 | Context testr@default cleanup() started
2018-08-16 15:15:55.238 11431 INFO rally.task.context [-] Verification fe1faab8-065d-496a-9707-352093dfc452 | Context testr@default cleanup() finished in 1.40 msec
2018-08-16 15:15:55.239 11431 INFO rally.task.context [-] Verification fe1faab8-065d-496a-9707-352093dfc452 | Context tempest@default cleanup() started
2018-08-16 15:15:56.404 11431 INFO rally.task.context [-] Verification fe1faab8-065d-496a-9707-352093dfc452 | Context tempest@default cleanup() finished in 1.17 sec
2018-08-16 15:15:56.484 11431 INFO rally.api [-] Verification (UUID=ae1a2f14-6797-438f-bb50-3433e328d6ea) has been successfully finished for deployment 'opnfv-rally' (UUID=3e6d0f86-cfdb-4138-9cd2-5cf14f99b3f1)!

======
Totals
======

Ran: 44 tests in 5942.497 sec.
 - Success: 23
 - Skipped: 3
 - Expected failures: 0
 - Unexpected success: 0
 - Failures: 18

Using verification (UUID=ae1a2f14-6797-438f-bb50-3433e328d6ea) as the default verification for the future operations.