2018-08-24 17:29:09.766 11588 INFO rally.api [-] Starting verification (UUID=00caaac7-44eb-4c79-91f3-ba1d6b3d5520) for deployment 'opnfv-rally' (UUID=001301d3-edb5-4f50-b028-6d64be2ac819) by verifier 'opnfv-tempest' (UUID=05ac9ef3-9f5e-4fd0-8424-1e0a837f1bf9).
2018-08-24 17:29:10.620 11588 WARNING oslo_config.cfg [-] Option "swift_operator_role" from group "tempest" is deprecated. Use option "swift_operator_role" from group "openstack".
2018-08-24 17:29:10.729 11588 WARNING oslo_config.cfg [-] Option "img_name_regex" from group "tempest" is deprecated. Use option "img_name_regex" from group "openstack".
2018-08-24 17:29:12.071 11588 INFO rally.task.context [-] Verification 05ac9ef3-9f5e-4fd0-8424-1e0a837f1bf9 | Context tempest@default setup()  finished in 1.78 sec
2018-08-24 17:29:12.074 11588 INFO rally.task.context [-] Verification 05ac9ef3-9f5e-4fd0-8424-1e0a837f1bf9 | Context testr@default setup()  finished in 2.58 msec
2018-08-24 17:29:23.614 11588 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_associate_disassociate_network ... success [3.601s]
2018-08-24 17:29:28.776 11588 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_associate_disassociate_router ... success [5.161s]
2018-08-24 17:29:30.531 11588 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_associate_invalid_network ... success [1.753s]
2018-08-24 17:29:35.507 11588 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_attach_associated_subnet_to_associated_router ... success [4.975s]
2018-08-24 17:29:35.728 11588 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_create_bgpvpn ... success [0.222s]
2018-08-24 17:29:35.746 11588 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_create_bgpvpn_as_non_admin_fail ... success [0.018s]
2018-08-24 17:29:36.079 11588 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_create_bgpvpn_with_invalid_routetargets ... success [0.333s]
2018-08-24 17:29:36.533 11588 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_delete_bgpvpn ... success [0.453s]
2018-08-24 17:29:37.139 11588 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_delete_bgpvpn_as_non_admin_fail ... success [0.605s]
2018-08-24 17:29:38.923 11588 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_disassociate_invalid_network ... success [1.782s]
2018-08-24 17:29:39.463 11588 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_list_bgpvpn ... success [0.541s]
2018-08-24 17:29:40.135 11588 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_list_bgpvpn_as_non_owner_fail ... success [0.670s]
2018-08-24 17:29:41.040 11588 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_list_netassoc_as_non_owner_fail ... success [0.904s]
2018-08-24 17:29:41.978 11588 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_list_show_network_association ... success [0.937s]
2018-08-24 17:29:42.782 11588 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_list_show_router_association ... success [0.802s]
2018-08-24 17:29:42.941 11588 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_show_bgpvpn ... success [0.160s]
2018-08-24 17:29:43.052 11588 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_show_bgpvpn_as_non_owner_fail ... success [0.111s]
2018-08-24 17:29:44.214 11588 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_show_netassoc_as_non_owner_fail ... success [1.160s]
2018-08-24 17:29:44.541 11588 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_update_bgpvpn_invalid_routetargets ... success [0.327s]
2018-08-24 17:29:45.302 11588 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_update_route_target ... fail [0.755s]
2018-08-24 17:29:45.444 11588 INFO opnfv-tempest [-] {1} networking_bgpvpn_tempest.tests.api.test_bgpvpn.BgpvpnTest.test_update_route_target_non_admin_fail ... success [0.146s]
2018-08-24 17:35:19.015 11588 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_basic ... fail [359.118s]
2018-08-24 17:41:25.712 11588 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_negative_delete_bgpvpn ... fail [366.679s]
2018-08-24 17:47:30.815 11588 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_negative_delete_net_association ... fail [365.085s]
2018-08-24 17:53:32.512 11588 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_negative_delete_router_association ... fail [361.680s]
2018-08-24 17:54:39.773 11588 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_negative_disjoint_import_export ... success [67.257s]
2018-08-24 17:55:33.378 11588 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_negative_ping_to_unassociated_net ... success [53.594s]
2018-08-24 18:01:25.940 11588 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_negative_update_to_disjoint_import_export ... fail [352.547s]
2018-08-24 18:07:21.349 11588 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_negative_update_to_empty_rt_list ... fail [355.396s]
2018-08-24 18:07:23.234 11588 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_port_association_create_and_delete ... skip [1.885s]: Skipped because network extension: bgpvpn-routes-control is not enabled
2018-08-24 18:07:25.089 11588 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_port_association_create_and_update ... skip [1.853s]: Skipped because network extension: bgpvpn-routes-control is not enabled
2018-08-24 18:07:26.888 11588 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_port_association_local_pref ... skip [1.798s]: Skipped because network extension: bgpvpn-routes-control is not enabled
2018-08-24 18:10:11.151 11588 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_tenant_separation_and_local_connectivity ... success [164.231s]
2018-08-24 18:11:16.884 11588 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_update_route_targets_and_unassociated_net ... fail [65.717s]
2018-08-24 18:17:12.450 11588 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_update_route_targets_common_target ... fail [355.551s]
2018-08-24 18:23:09.158 11588 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_update_route_targets_disjoint_targets ... fail [356.687s]
2018-08-24 18:25:04.568 11588 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_update_rt_and_keep_local_connectivity_variant1 ... fail [115.385s]
2018-08-24 18:29:56.928 11588 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_update_rt_and_keep_local_connectivity_variant2 ... fail [292.311s]
2018-08-24 18:36:03.473 11588 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_variant1 ... fail [366.527s]
2018-08-24 18:42:08.031 11588 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_variant2 ... fail [364.543s]
2018-08-24 18:48:22.614 11588 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_variant3 ... fail [374.562s]
2018-08-24 18:54:35.718 11588 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_variant4 ... fail [373.085s]
2018-08-24 19:00:46.207 11588 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_variant5 ... fail [370.470s]
2018-08-24 19:06:54.091 11588 INFO opnfv-tempest [-] {0} networking_bgpvpn_tempest.tests.scenario.test_bgpvpn_basic.TestBGPVPNBasic.test_bgpvpn_variant6 ... fail [367.867s]
2018-08-24 19:07:13.229 11588 INFO rally.task.context [-] Verification 05ac9ef3-9f5e-4fd0-8424-1e0a837f1bf9 | Context testr@default cleanup() started
2018-08-24 19:07:13.230 11588 INFO rally.task.context [-] Verification 05ac9ef3-9f5e-4fd0-8424-1e0a837f1bf9 | Context testr@default cleanup() finished in 0.71 msec
2018-08-24 19:07:13.230 11588 INFO rally.task.context [-] Verification 05ac9ef3-9f5e-4fd0-8424-1e0a837f1bf9 | Context tempest@default cleanup() started
2018-08-24 19:07:14.136 11588 INFO rally.task.context [-] Verification 05ac9ef3-9f5e-4fd0-8424-1e0a837f1bf9 | Context tempest@default cleanup() finished in 0.91 sec
2018-08-24 19:07:14.192 11588 INFO rally.api [-] Verification (UUID=00caaac7-44eb-4c79-91f3-ba1d6b3d5520) has been successfully finished for deployment 'opnfv-rally' (UUID=001301d3-edb5-4f50-b028-6d64be2ac819)!

======
Totals
======

Ran: 44 tests in 5859.368 sec.
 - Success: 23
 - Skipped: 3
 - Expected failures: 0
 - Unexpected success: 0
 - Failures: 18

Using verification (UUID=00caaac7-44eb-4c79-91f3-ba1d6b3d5520) as the default verification for the future operations.