2018-04-08 20:43:29,654 - baro_tests.barometer - INFO - =================== Starting barometer tests suite =================== 2018-04-08 20:43:29,654 - baro_tests.barometer - INFO - RSA key file /root/.ssh/id_rsa exists. 2018-04-08 20:43:33,915 - baro_tests.barometer - INFO - ==== Display of Control and Compute nodes available in the set up ==== 2018-04-08 20:43:33,915 - baro_tests.barometer - INFO - controllers: ['overcloud-controller-0: 192.30.9.6'] 2018-04-08 20:43:33,915 - baro_tests.barometer - INFO - computes: ['overcloud-novacompute-0: 192.30.9.4', 'overcloud-novacompute-1: 192.30.9.10'] 2018-04-08 20:43:33,916 - baro_tests.barometer - INFO - ========== Enabling mcelog and OVS bridges on compute nodes ========== 2018-04-08 20:43:46,556 - baro_tests.barometer - INFO - Checking if mcelog is enabled on node-overcloud-novacompute-0... 2018-04-08 20:43:46,606 - baro_tests.barometer - INFO - Mcelog will be enabled on node-overcloud-novacompute-0... 2018-04-08 20:43:46,965 - baro_tests.barometer - INFO - Mcelog was installed on node-overcloud-novacompute-0. 2018-04-08 20:43:46,978 - baro_tests.barometer - INFO - Checking if mcelog is enabled on node-overcloud-novacompute-1... 2018-04-08 20:43:47,028 - baro_tests.barometer - INFO - Mcelog will be enabled on node-overcloud-novacompute-1... 2018-04-08 20:43:47,411 - baro_tests.barometer - INFO - Mcelog was installed on node-overcloud-novacompute-1. 2018-04-08 20:43:58,959 - baro_tests.barometer - INFO - Creating OVS bridges on computes nodes 2018-04-08 20:43:59,249 - baro_tests.barometer - INFO - OVS Bridges created on compute nodes 2018-04-08 20:43:59,250 - baro_tests.barometer - INFO - ============= Testing Gnocchi and AODH plugins on nodes ============== 2018-04-08 20:44:39,804 - baro_tests.barometer - INFO - Gnocchi plugin available in compute node overcloud-novacompute-0 2018-04-08 20:44:51,694 - baro_tests.barometer - INFO - AODH plugin present in compute node overcloud-novacompute-0 2018-04-08 20:44:51,694 - baro_tests.barometer - INFO - === NODE overcloud-novacompute-0: Enabling Test Plug-in and Test case execution === 2018-04-08 20:46:09,528 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Hugepages Test case execution === 2018-04-08 20:46:32,835 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-04-08 20:47:09,962 - baro_tests.barometer - INFO - Plugin Interval is 10 2018-04-08 20:47:09,962 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-04-08 20:47:09,963 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on Hugepages. 2018-04-08 20:47:55,322 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Intel RDT Test case execution === 2018-04-08 20:48:29,100 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-04-08 20:49:06,630 - baro_tests.barometer - INFO - Plugin Interval is 10 2018-04-08 20:49:06,630 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-04-08 20:49:06,631 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on Intel RDT. 2018-04-08 20:49:53,315 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-04-08 20:50:14,914 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-04-08 20:50:52,049 - baro_tests.barometer - INFO - Plugin Interval is 1 2018-04-08 20:50:52,050 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-04-08 20:50:52,050 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on Mcelog. 2018-04-08 20:51:38,485 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-04-08 20:52:00,896 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-04-08 20:52:38,189 - baro_tests.barometer - INFO - Plugin Interval is 1 2018-04-08 20:52:38,190 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-04-08 20:52:38,190 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on OVS events. 2018-04-08 20:53:24,181 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS stats Test case execution === 2018-04-08 20:53:47,540 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-04-08 20:54:24,689 - baro_tests.barometer - INFO - Plugin Interval is 1 2018-04-08 20:54:24,689 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-04-08 20:54:24,690 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on OVS stats. 2018-04-08 20:55:11,667 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-04-08 20:55:34,675 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-0 2018-04-08 20:55:53,571 - baro_tests.barometer - INFO - Data not updated after interval of 12 seconds 2018-04-08 20:55:53,572 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with AODH FAILED on Mcelog. 2018-04-08 20:56:39,834 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-04-08 20:57:03,232 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-0 2018-04-08 20:57:05,507 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with AODH FAILED on OVS events. 2018-04-08 20:57:05,508 - baro_tests.barometer - INFO - ======= Node overcloud-novacompute-0: Executing CSV Testcases ======== 2018-04-08 20:57:05,508 - baro_tests.barometer - INFO - Restarting collectd for CSV tests 2018-04-08 20:57:17,027 - baro_tests.barometer - INFO - Stopping collectd service... 2018-04-08 20:57:27,813 - baro_tests.barometer - INFO - Starting collectd service... 2018-04-08 20:57:38,108 - baro_tests.barometer - INFO - Sleeping for 10 seconds after collectd restart... 2018-04-08 20:58:35,136 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Hugepages Test case execution === 2018-04-08 20:58:58,486 - baro_tests.barometer - INFO - Getting CSV metrics of plugin hugepages on compute node 670ca4aa-5f08-4e10-815c-5d2971d7a693... 2018-04-08 20:58:58,486 - baro_tests.barometer - DEBUG - Interval: 10 2018-04-08 20:58:58,487 - baro_tests.barometer - DEBUG - Plugin subdirs: ['hugepages-mm-2048Kb', 'hugepages-node0-2048Kb'] 2018-04-08 20:58:58,487 - baro_tests.barometer - DEBUG - Plugin meter categories: ['vmpage_number-free', 'vmpage_number-used'] 2018-04-08 20:59:09,645 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-08 20:59:09,646 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1523220740.27 ... 2018-04-08 20:59:09,646 - baro_tests.barometer - DEBUG - OK 2018-04-08 20:59:09,646 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1523220740.27 ... 2018-04-08 20:59:09,646 - baro_tests.barometer - DEBUG - OK 2018-04-08 20:59:09,647 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1523220740.27 ... 2018-04-08 20:59:09,647 - baro_tests.barometer - DEBUG - OK 2018-04-08 20:59:09,647 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1523220740.27 ... 2018-04-08 20:59:09,647 - baro_tests.barometer - DEBUG - OK 2018-04-08 20:59:09,647 - baro_tests.barometer - INFO - OK 2018-04-08 20:59:09,647 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-04-08 20:59:21,654 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-08 20:59:33,553 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-08 20:59:33,554 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1523220740.267, 1523220750.267), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1523220740.267, 1523220750.267), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1523220740.267, 1523220750.267), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1523220740.267, 1523220750.267)] 2018-04-08 20:59:33,554 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1523220760.267, 1523220770.267), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1523220760.267, 1523220770.267), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1523220760.267, 1523220770.267), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1523220760.267, 1523220770.267)] 2018-04-08 20:59:33,555 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1523220740.27 - hugepages-mm-2048Kb vmpage_number-free 1523220760.27 ... 2018-04-08 20:59:33,555 - baro_tests.barometer - DEBUG - OK 2018-04-08 20:59:33,556 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1523220740.27 - hugepages-mm-2048Kb vmpage_number-used 1523220760.27 ... 2018-04-08 20:59:33,556 - baro_tests.barometer - DEBUG - OK 2018-04-08 20:59:33,556 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1523220740.27 - hugepages-node0-2048Kb vmpage_number-free 1523220760.27 ... 2018-04-08 20:59:33,556 - baro_tests.barometer - DEBUG - OK 2018-04-08 20:59:33,556 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1523220740.27 - hugepages-node0-2048Kb vmpage_number-used 1523220760.27 ... 2018-04-08 20:59:33,556 - baro_tests.barometer - DEBUG - OK 2018-04-08 20:59:33,556 - baro_tests.barometer - INFO - OK 2018-04-08 20:59:33,556 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Hugepages. 2018-04-08 21:00:18,471 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Intel RDT Test case execution === 2018-04-08 21:00:52,131 - baro_tests.barometer - INFO - Getting CSV metrics of plugin intel_rdt on compute node 670ca4aa-5f08-4e10-815c-5d2971d7a693... 2018-04-08 21:00:52,132 - baro_tests.barometer - DEBUG - Interval: 10 2018-04-08 21:00:52,132 - baro_tests.barometer - DEBUG - Plugin subdirs: ['intel_rdt-0-2'] 2018-04-08 21:00:52,132 - baro_tests.barometer - DEBUG - Plugin meter categories: ['bytes-llc', 'ipc'] 2018-04-08 21:01:03,302 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-08 21:01:03,303 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1523220850.27 ... 2018-04-08 21:01:03,303 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:01:03,303 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1523220850.27 ... 2018-04-08 21:01:03,304 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:01:03,304 - baro_tests.barometer - INFO - OK 2018-04-08 21:01:03,304 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-04-08 21:01:15,315 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-08 21:01:27,169 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-08 21:01:27,170 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1523220850.267, 1523220860.267), ('intel_rdt-0-2', 'ipc', 1523220850.267, 1523220860.267)] 2018-04-08 21:01:27,170 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1523220870.267, 1523220880.267), ('intel_rdt-0-2', 'ipc', 1523220870.267, 1523220880.267)] 2018-04-08 21:01:27,170 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1523220850.27 - intel_rdt-0-2 bytes-llc 1523220870.27 ... 2018-04-08 21:01:27,170 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:01:27,171 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1523220850.27 - intel_rdt-0-2 ipc 1523220870.27 ... 2018-04-08 21:01:27,171 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:01:27,171 - baro_tests.barometer - INFO - OK 2018-04-08 21:01:27,171 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Intel RDT. 2018-04-08 21:02:12,862 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-04-08 21:02:35,268 - baro_tests.barometer - INFO - Getting CSV metrics of plugin mcelog on compute node 670ca4aa-5f08-4e10-815c-5d2971d7a693... 2018-04-08 21:02:35,268 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-08 21:02:35,269 - baro_tests.barometer - DEBUG - Plugin subdirs: ['mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'mcelog-SOCKET_0_CHANNEL_any_DIMM_any'] 2018-04-08 21:02:35,269 - baro_tests.barometer - DEBUG - Plugin meter categories: ['errors-corrected_memory_errors', 'errors-uncorrected_memory_errors'] 2018-04-08 21:02:46,149 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-08 21:02:46,149 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1523220966.65 ... 2018-04-08 21:02:46,150 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:02:46,150 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1523220966.65 ... 2018-04-08 21:02:46,150 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:02:46,150 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1523220966.65 ... 2018-04-08 21:02:46,151 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:02:46,151 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1523220966.65 ... 2018-04-08 21:02:46,151 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:02:46,151 - baro_tests.barometer - INFO - OK 2018-04-08 21:02:46,151 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-08 21:02:49,152 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-08 21:03:00,725 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-08 21:03:00,726 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1523220966.646, 1523220967.646), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1523220966.646, 1523220967.646), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1523220966.646, 1523220967.646), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1523220966.646, 1523220967.646)] 2018-04-08 21:03:00,726 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1523220980.646, 1523220981.646), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1523220980.646, 1523220982.646), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1523220981.646, 1523220982.646), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1523220981.646, 1523220982.646)] 2018-04-08 21:03:00,726 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1523220966.65 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1523220980.65 ... 2018-04-08 21:03:00,727 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:03:00,727 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1523220966.65 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1523220980.65 ... 2018-04-08 21:03:00,727 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:03:00,727 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1523220966.65 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1523220981.65 ... 2018-04-08 21:03:00,727 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:03:00,727 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1523220966.65 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1523220981.65 ... 2018-04-08 21:03:00,727 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:03:00,728 - baro_tests.barometer - INFO - OK 2018-04-08 21:03:00,728 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Mcelog. 2018-04-08 21:03:45,001 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-04-08 21:04:07,630 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_events on compute node 670ca4aa-5f08-4e10-815c-5d2971d7a693... 2018-04-08 21:04:07,631 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-08 21:04:07,631 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_events-br0'] 2018-04-08 21:04:07,631 - baro_tests.barometer - DEBUG - Plugin meter categories: ['gauge-link_status'] 2018-04-08 21:04:19,201 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-08 21:04:19,201 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1523221059.65 ... 2018-04-08 21:04:19,202 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:04:19,202 - baro_tests.barometer - INFO - OK 2018-04-08 21:04:19,202 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-08 21:04:22,205 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-08 21:04:33,183 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-08 21:04:33,183 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1523221059.646, 1523221060.647)] 2018-04-08 21:04:33,184 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1523221073.646, 1523221074.647)] 2018-04-08 21:04:33,184 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1523221059.65 - ovs_events-br0 gauge-link_status 1523221073.65 ... 2018-04-08 21:04:33,184 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:04:33,184 - baro_tests.barometer - INFO - OK 2018-04-08 21:04:33,185 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on OVS events. 2018-04-08 21:05:15,142 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS stats Test case execution === 2018-04-08 21:05:37,249 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_stats on compute node 670ca4aa-5f08-4e10-815c-5d2971d7a693... 2018-04-08 21:05:37,250 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-08 21:05:37,250 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_stats-br0.br0'] 2018-04-08 21:05:37,250 - baro_tests.barometer - DEBUG - Plugin meter categories: ['if_dropped', 'if_errors', 'if_packets'] 2018-04-08 21:05:48,432 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-08 21:05:48,432 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1523221148.65 ... 2018-04-08 21:05:48,433 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:05:48,433 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1523221148.65 ... 2018-04-08 21:05:48,433 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:05:48,434 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1523221148.65 ... 2018-04-08 21:05:48,434 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:05:48,434 - baro_tests.barometer - INFO - OK 2018-04-08 21:05:48,434 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-08 21:05:51,437 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-08 21:06:02,521 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-08 21:06:02,521 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1523221148.646, 1523221149.646), ('ovs_stats-br0.br0', 'if_errors', 1523221148.646, 1523221149.646), ('ovs_stats-br0.br0', 'if_packets', 1523221148.646, 1523221149.646)] 2018-04-08 21:06:02,522 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1523221162.646, 1523221163.646), ('ovs_stats-br0.br0', 'if_errors', 1523221162.646, 1523221163.646), ('ovs_stats-br0.br0', 'if_packets', 1523221163.646, 1523221164.646)] 2018-04-08 21:06:02,522 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1523221148.65 - ovs_stats-br0.br0 if_dropped 1523221162.65 ... 2018-04-08 21:06:02,522 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:06:02,522 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1523221148.65 - ovs_stats-br0.br0 if_errors 1523221162.65 ... 2018-04-08 21:06:02,522 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:06:02,523 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1523221148.65 - ovs_stats-br0.br0 if_packets 1523221163.65 ... 2018-04-08 21:06:02,523 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:06:02,523 - baro_tests.barometer - INFO - OK 2018-04-08 21:06:02,523 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on OVS stats. 2018-04-08 21:06:13,307 - baro_tests.barometer - INFO - Gnocchi plugin available in compute node overcloud-novacompute-1 2018-04-08 21:06:24,461 - baro_tests.barometer - INFO - AODH plugin present in compute node overcloud-novacompute-1 2018-04-08 21:06:24,461 - baro_tests.barometer - INFO - === NODE overcloud-novacompute-1: Enabling Test Plug-in and Test case execution === 2018-04-08 21:07:34,454 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Hugepages Test case execution === 2018-04-08 21:07:57,084 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-04-08 21:08:35,131 - baro_tests.barometer - INFO - PASS 2018-04-08 21:08:35,132 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with Gnocchi PASSED on Hugepages. 2018-04-08 21:09:21,325 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Intel RDT Test case execution === 2018-04-08 21:09:55,054 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-04-08 21:10:32,226 - baro_tests.barometer - INFO - Plugin Interval is 10 2018-04-08 21:10:32,226 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-04-08 21:10:32,227 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on Intel RDT. 2018-04-08 21:11:18,990 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-04-08 21:11:41,733 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-04-08 21:12:19,154 - baro_tests.barometer - INFO - Plugin Interval is 1 2018-04-08 21:12:19,154 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-04-08 21:12:19,155 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on Mcelog. 2018-04-08 21:13:06,369 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-04-08 21:13:28,678 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-04-08 21:14:05,936 - baro_tests.barometer - INFO - Plugin Interval is 1 2018-04-08 21:14:05,937 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-04-08 21:14:05,937 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on OVS events. 2018-04-08 21:14:52,156 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS stats Test case execution === 2018-04-08 21:15:15,564 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-04-08 21:15:53,144 - baro_tests.barometer - INFO - Plugin Interval is 1 2018-04-08 21:15:53,144 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-04-08 21:15:53,145 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on OVS stats. 2018-04-08 21:16:38,787 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-04-08 21:17:01,488 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-0 2018-04-08 21:17:20,324 - baro_tests.barometer - INFO - Data not updated after interval of 12 seconds 2018-04-08 21:17:20,325 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with AODH FAILED on Mcelog. 2018-04-08 21:18:06,117 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-04-08 21:18:27,801 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-0 2018-04-08 21:18:30,127 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with AODH FAILED on OVS events. 2018-04-08 21:18:30,129 - baro_tests.barometer - INFO - ======= Node overcloud-novacompute-1: Executing CSV Testcases ======== 2018-04-08 21:18:30,129 - baro_tests.barometer - INFO - Restarting collectd for CSV tests 2018-04-08 21:18:41,471 - baro_tests.barometer - INFO - Stopping collectd service... 2018-04-08 21:18:52,318 - baro_tests.barometer - INFO - Starting collectd service... 2018-04-08 21:19:02,620 - baro_tests.barometer - INFO - Sleeping for 10 seconds after collectd restart... 2018-04-08 21:19:58,140 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Hugepages Test case execution === 2018-04-08 21:20:20,839 - baro_tests.barometer - INFO - Getting CSV metrics of plugin hugepages on compute node 8a15593e-4bab-4f8e-bb43-9907274ee885... 2018-04-08 21:20:20,840 - baro_tests.barometer - DEBUG - Interval: 10 2018-04-08 21:20:20,841 - baro_tests.barometer - DEBUG - Plugin subdirs: ['hugepages-mm-2048Kb', 'hugepages-node0-2048Kb'] 2018-04-08 21:20:20,841 - baro_tests.barometer - DEBUG - Plugin meter categories: ['vmpage_number-free', 'vmpage_number-used'] 2018-04-08 21:20:32,049 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-08 21:20:32,050 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1523222027.86 ... 2018-04-08 21:20:32,050 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:20:32,050 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1523222027.86 ... 2018-04-08 21:20:32,050 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:20:32,051 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1523222027.86 ... 2018-04-08 21:20:32,051 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:20:32,051 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1523222027.86 ... 2018-04-08 21:20:32,051 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:20:32,051 - baro_tests.barometer - INFO - OK 2018-04-08 21:20:32,051 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-04-08 21:20:44,059 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-08 21:20:55,887 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-08 21:20:55,887 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1523222027.864, 1523222037.864), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1523222027.864, 1523222037.864), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1523222027.864, 1523222037.865), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1523222027.864, 1523222037.865)] 2018-04-08 21:20:55,888 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1523222057.864, 1523222067.864), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1523222057.864, 1523222067.864), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1523222057.864, 1523222067.865), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1523222057.864, 1523222067.865)] 2018-04-08 21:20:55,888 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1523222027.86 - hugepages-mm-2048Kb vmpage_number-free 1523222057.86 ... 2018-04-08 21:20:55,888 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:20:55,888 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1523222027.86 - hugepages-mm-2048Kb vmpage_number-used 1523222057.86 ... 2018-04-08 21:20:55,889 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:20:55,889 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1523222027.86 - hugepages-node0-2048Kb vmpage_number-free 1523222057.86 ... 2018-04-08 21:20:55,889 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:20:55,889 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1523222027.86 - hugepages-node0-2048Kb vmpage_number-used 1523222057.86 ... 2018-04-08 21:20:55,889 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:20:55,889 - baro_tests.barometer - INFO - OK 2018-04-08 21:20:55,890 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Hugepages. 2018-04-08 21:21:41,416 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Intel RDT Test case execution === 2018-04-08 21:22:15,847 - baro_tests.barometer - INFO - Getting CSV metrics of plugin intel_rdt on compute node 8a15593e-4bab-4f8e-bb43-9907274ee885... 2018-04-08 21:22:15,847 - baro_tests.barometer - DEBUG - Interval: 10 2018-04-08 21:22:15,847 - baro_tests.barometer - DEBUG - Plugin subdirs: ['intel_rdt-0-2'] 2018-04-08 21:22:15,848 - baro_tests.barometer - DEBUG - Plugin meter categories: ['bytes-llc', 'ipc'] 2018-04-08 21:22:26,617 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-08 21:22:26,618 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1523222147.86 ... 2018-04-08 21:22:26,618 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:22:26,619 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1523222147.86 ... 2018-04-08 21:22:26,619 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:22:26,619 - baro_tests.barometer - INFO - OK 2018-04-08 21:22:26,620 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-04-08 21:22:38,628 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-08 21:22:49,784 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-08 21:22:49,784 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1523222147.864, 1523222157.864), ('intel_rdt-0-2', 'ipc', 1523222147.864, 1523222157.864)] 2018-04-08 21:22:49,785 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1523222167.865, 1523222177.864), ('intel_rdt-0-2', 'ipc', 1523222167.865, 1523222177.864)] 2018-04-08 21:22:49,785 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1523222147.86 - intel_rdt-0-2 bytes-llc 1523222167.87 ... 2018-04-08 21:22:49,785 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:22:49,785 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1523222147.86 - intel_rdt-0-2 ipc 1523222167.87 ... 2018-04-08 21:22:49,786 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:22:49,786 - baro_tests.barometer - INFO - OK 2018-04-08 21:22:49,788 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Intel RDT. 2018-04-08 21:23:36,009 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-04-08 21:23:57,768 - baro_tests.barometer - INFO - Getting CSV metrics of plugin mcelog on compute node 8a15593e-4bab-4f8e-bb43-9907274ee885... 2018-04-08 21:23:57,768 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-08 21:23:57,768 - baro_tests.barometer - DEBUG - Plugin subdirs: ['mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'mcelog-SOCKET_0_CHANNEL_any_DIMM_any'] 2018-04-08 21:23:57,769 - baro_tests.barometer - DEBUG - Plugin meter categories: ['errors-corrected_memory_errors', 'errors-uncorrected_memory_errors'] 2018-04-08 21:24:09,870 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-08 21:24:09,871 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1523222262.87 ... 2018-04-08 21:24:09,871 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:24:09,871 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1523222262.87 ... 2018-04-08 21:24:09,871 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:24:09,872 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1523222263.87 ... 2018-04-08 21:24:09,872 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:24:09,872 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1523222263.87 ... 2018-04-08 21:24:09,872 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:24:09,872 - baro_tests.barometer - INFO - OK 2018-04-08 21:24:09,872 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-08 21:24:12,875 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-08 21:24:23,927 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-08 21:24:23,928 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1523222262.866, 1523222263.866), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1523222262.866, 1523222264.866), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1523222263.866, 1523222264.866), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1523222263.866, 1523222264.866)] 2018-04-08 21:24:23,929 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1523222276.866, 1523222278.866), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1523222277.866, 1523222278.866), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1523222277.866, 1523222278.866), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1523222277.866, 1523222278.866)] 2018-04-08 21:24:23,929 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1523222262.87 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1523222276.87 ... 2018-04-08 21:24:23,929 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:24:23,929 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1523222262.87 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1523222277.87 ... 2018-04-08 21:24:23,930 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:24:23,930 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1523222263.87 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1523222277.87 ... 2018-04-08 21:24:23,930 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:24:23,930 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1523222263.87 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1523222277.87 ... 2018-04-08 21:24:23,930 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:24:23,930 - baro_tests.barometer - INFO - OK 2018-04-08 21:24:23,930 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Mcelog. 2018-04-08 21:25:10,273 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-04-08 21:25:32,617 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_events on compute node 8a15593e-4bab-4f8e-bb43-9907274ee885... 2018-04-08 21:25:32,618 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-08 21:25:32,618 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_events-br0'] 2018-04-08 21:25:32,618 - baro_tests.barometer - DEBUG - Plugin meter categories: ['gauge-link_status'] 2018-04-08 21:25:44,017 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-08 21:25:44,018 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1523222357.87 ... 2018-04-08 21:25:44,018 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:25:44,018 - baro_tests.barometer - INFO - OK 2018-04-08 21:25:44,019 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-08 21:25:47,022 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-08 21:25:57,486 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-08 21:25:57,487 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1523222357.866, 1523222358.866)] 2018-04-08 21:25:57,487 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1523222370.866, 1523222371.866)] 2018-04-08 21:25:57,488 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1523222357.87 - ovs_events-br0 gauge-link_status 1523222370.87 ... 2018-04-08 21:25:57,489 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:25:57,490 - baro_tests.barometer - INFO - OK 2018-04-08 21:25:57,496 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on OVS events. 2018-04-08 21:26:43,808 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS stats Test case execution === 2018-04-08 21:27:07,464 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_stats on compute node 8a15593e-4bab-4f8e-bb43-9907274ee885... 2018-04-08 21:27:07,464 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-08 21:27:07,465 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_stats-br0.br0'] 2018-04-08 21:27:07,465 - baro_tests.barometer - DEBUG - Plugin meter categories: ['if_dropped', 'if_errors', 'if_packets'] 2018-04-08 21:27:19,279 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-08 21:27:19,279 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1523222452.87 ... 2018-04-08 21:27:19,280 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:27:19,280 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1523222452.87 ... 2018-04-08 21:27:19,280 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:27:19,281 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1523222452.87 ... 2018-04-08 21:27:19,281 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:27:19,281 - baro_tests.barometer - INFO - OK 2018-04-08 21:27:19,281 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-08 21:27:22,282 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-08 21:27:32,982 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-08 21:27:32,982 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1523222452.866, 1523222453.866), ('ovs_stats-br0.br0', 'if_errors', 1523222452.866, 1523222453.866), ('ovs_stats-br0.br0', 'if_packets', 1523222452.866, 1523222453.866)] 2018-04-08 21:27:32,982 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1523222465.866, 1523222466.866), ('ovs_stats-br0.br0', 'if_errors', 1523222465.866, 1523222466.866), ('ovs_stats-br0.br0', 'if_packets', 1523222465.866, 1523222466.866)] 2018-04-08 21:27:32,983 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1523222452.87 - ovs_stats-br0.br0 if_dropped 1523222465.87 ... 2018-04-08 21:27:32,983 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:27:32,983 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1523222452.87 - ovs_stats-br0.br0 if_errors 1523222465.87 ... 2018-04-08 21:27:32,983 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:27:32,983 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1523222452.87 - ovs_stats-br0.br0 if_packets 1523222465.87 ... 2018-04-08 21:27:32,983 - baro_tests.barometer - DEBUG - OK 2018-04-08 21:27:32,983 - baro_tests.barometer - INFO - OK 2018-04-08 21:27:32,983 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on OVS stats. 2018-04-08 21:27:44,513 - baro_tests.barometer - INFO - Mcelog is deleted from all compute nodes 2018-04-08 21:27:44,514 - baro_tests.barometer - INFO - ====================================================================== 2018-04-08 21:27:44,514 - baro_tests.barometer - INFO - +----------------------------------+ 2018-04-08 21:27:44,515 - baro_tests.barometer - INFO - | OVERALL SUMMARY | 2018-04-08 21:27:44,515 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-08 21:27:44,517 - baro_tests.barometer - INFO - | Test | Node-0 | Node-1 | 2018-04-08 21:27:44,517 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-08 21:27:44,518 - baro_tests.barometer - INFO - | OUT:Gnocchi | PASS | PASS | 2018-04-08 21:27:44,518 - baro_tests.barometer - INFO - | IN:Hugepages | FAIL | PASS | 2018-04-08 21:27:44,518 - baro_tests.barometer - INFO - | IN:Intel RDT | FAIL | FAIL | 2018-04-08 21:27:44,519 - baro_tests.barometer - INFO - | IN:Mcelog | FAIL | FAIL | 2018-04-08 21:27:44,519 - baro_tests.barometer - INFO - | IN:OVS events | FAIL | FAIL | 2018-04-08 21:27:44,519 - baro_tests.barometer - INFO - | IN:OVS stats | FAIL | FAIL | 2018-04-08 21:27:44,520 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-08 21:27:44,520 - baro_tests.barometer - INFO - | OUT:AODH | PASS | PASS | 2018-04-08 21:27:44,521 - baro_tests.barometer - INFO - | IN:Mcelog | FAIL | FAIL | 2018-04-08 21:27:44,522 - baro_tests.barometer - INFO - | IN:OVS events | SKIP | SKIP | 2018-04-08 21:27:44,523 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-08 21:27:44,523 - baro_tests.barometer - INFO - | OUT:CSV | PASS | PASS | 2018-04-08 21:27:44,523 - baro_tests.barometer - INFO - | IN:Hugepages | PASS | PASS | 2018-04-08 21:27:44,523 - baro_tests.barometer - INFO - | IN:Intel RDT | PASS | PASS | 2018-04-08 21:27:44,523 - baro_tests.barometer - INFO - | IN:Mcelog | PASS | PASS | 2018-04-08 21:27:44,523 - baro_tests.barometer - INFO - | IN:OVS events | PASS | PASS | 2018-04-08 21:27:44,523 - baro_tests.barometer - INFO - | IN:OVS stats | PASS | PASS | 2018-04-08 21:27:44,523 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-08 21:27:44,524 - baro_tests.barometer - INFO - ====================================================================== 2018-04-08 21:27:44,524 - baro_tests.barometer - ERROR - Some tests have failed or have not been executed 2018-04-08 21:27:44,524 - baro_tests.barometer - ERROR - Overall Result is Fail