2018-05-17 12:35:56,543 - baro_tests.barometer - INFO - =================== Starting barometer tests suite =================== 2018-05-17 12:35:56,543 - baro_tests.barometer - INFO - RSA key file /root/.ssh/id_rsa exists. 2018-05-17 12:36:01,152 - baro_tests.barometer - INFO - ==== Display of Control and Compute nodes available in the set up ==== 2018-05-17 12:36:01,152 - baro_tests.barometer - INFO - controllers: ['overcloud-controller-0: 192.30.9.8', 'overcloud-controller-1: 192.30.9.5', 'overcloud-controller-2: 192.30.9.10'] 2018-05-17 12:36:01,152 - baro_tests.barometer - INFO - computes: ['overcloud-novacompute-0: 192.30.9.9', 'overcloud-novacompute-1: 192.30.9.4'] 2018-05-17 12:36:01,153 - baro_tests.barometer - INFO - ========== Enabling mcelog and OVS bridges on compute nodes ========== 2018-05-17 12:36:15,853 - baro_tests.barometer - INFO - Checking if mcelog is enabled on node-overcloud-novacompute-1... 2018-05-17 12:36:15,921 - baro_tests.barometer - INFO - Mcelog will be enabled on node-overcloud-novacompute-1... 2018-05-17 12:36:16,424 - baro_tests.barometer - INFO - Mcelog was installed on node-overcloud-novacompute-1. 2018-05-17 12:36:16,455 - baro_tests.barometer - INFO - Checking if mcelog is enabled on node-overcloud-novacompute-0... 2018-05-17 12:36:16,523 - baro_tests.barometer - INFO - Mcelog will be enabled on node-overcloud-novacompute-0... 2018-05-17 12:36:17,030 - baro_tests.barometer - INFO - Mcelog was installed on node-overcloud-novacompute-0. 2018-05-17 12:36:29,912 - baro_tests.barometer - INFO - Creating OVS bridges on computes nodes 2018-05-17 12:36:30,236 - baro_tests.barometer - INFO - OVS Bridges created on compute nodes 2018-05-17 12:36:30,237 - baro_tests.barometer - INFO - ============= Testing Gnocchi and AODH plugins on nodes ============== 2018-05-17 12:38:19,737 - baro_tests.barometer - INFO - Gnocchi plugin available in compute node overcloud-novacompute-0 2018-05-17 12:38:33,651 - baro_tests.barometer - INFO - AODH plugin present in compute node overcloud-novacompute-0 2018-05-17 12:38:33,651 - baro_tests.barometer - INFO - === NODE overcloud-novacompute-0: Enabling Test Plug-in and Test case execution === 2018-05-17 12:39:49,018 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Hugepages Test case execution === 2018-05-17 12:40:14,900 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-17 12:40:18,125 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-05-17 12:40:20,772 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-1 2018-05-17 12:40:23,677 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on Hugepages. 2018-05-17 12:41:18,675 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Intel RDT Test case execution === 2018-05-17 12:41:58,692 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-17 12:42:36,001 - baro_tests.barometer - INFO - Plugin Interval is 10 2018-05-17 12:42:36,001 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-05-17 12:42:36,001 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on Intel RDT. 2018-05-17 12:43:29,548 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-05-17 12:43:54,830 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-17 12:44:31,997 - baro_tests.barometer - INFO - PASS 2018-05-17 12:44:31,998 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with Gnocchi PASSED on Mcelog. 2018-05-17 12:45:26,597 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-05-17 12:45:53,179 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-17 12:46:30,509 - baro_tests.barometer - INFO - PASS 2018-05-17 12:46:30,510 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with Gnocchi PASSED on OVS events. 2018-05-17 12:47:22,334 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS stats Test case execution === 2018-05-17 12:47:50,010 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-17 12:48:27,164 - baro_tests.barometer - INFO - Plugin Interval is 1 2018-05-17 12:48:27,165 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-05-17 12:48:27,166 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on OVS stats. 2018-05-17 12:49:20,577 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-05-17 12:49:45,810 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-05-17 12:50:05,620 - baro_tests.barometer - INFO - Data not updated after interval of 12 seconds 2018-05-17 12:50:05,621 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with AODH FAILED on Mcelog. 2018-05-17 12:50:58,573 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-05-17 12:51:24,005 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-05-17 12:51:26,471 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-0 2018-05-17 12:51:29,011 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-1 2018-05-17 12:51:31,425 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with AODH FAILED on OVS events. 2018-05-17 12:51:31,426 - baro_tests.barometer - INFO - ======= Node overcloud-novacompute-0: Executing CSV Testcases ======== 2018-05-17 12:51:31,426 - baro_tests.barometer - INFO - Restarting collectd for CSV tests 2018-05-17 12:51:44,471 - baro_tests.barometer - INFO - Stopping collectd service... 2018-05-17 12:51:56,178 - baro_tests.barometer - INFO - Starting collectd service... 2018-05-17 12:52:06,507 - baro_tests.barometer - INFO - Sleeping for 10 seconds after collectd restart... 2018-05-17 12:53:09,082 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Hugepages Test case execution === 2018-05-17 12:53:35,101 - baro_tests.barometer - INFO - Getting CSV metrics of plugin hugepages on compute node 8ff325fb-a79e-4f0a-ba6a-9538291f745b... 2018-05-17 12:53:35,102 - baro_tests.barometer - DEBUG - Interval: 10 2018-05-17 12:53:35,102 - baro_tests.barometer - DEBUG - Plugin subdirs: ['hugepages-mm-2048Kb', 'hugepages-node0-2048Kb'] 2018-05-17 12:53:35,102 - baro_tests.barometer - DEBUG - Plugin meter categories: ['vmpage_number-free', 'vmpage_number-used'] 2018-05-17 12:53:48,638 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-17 12:53:48,638 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1526561180.41 ... 2018-05-17 12:53:48,639 - baro_tests.barometer - DEBUG - OK 2018-05-17 12:53:48,639 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1526561180.41 ... 2018-05-17 12:53:48,639 - baro_tests.barometer - DEBUG - OK 2018-05-17 12:53:48,639 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1526561180.41 ... 2018-05-17 12:53:48,639 - baro_tests.barometer - DEBUG - OK 2018-05-17 12:53:48,640 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1526561180.41 ... 2018-05-17 12:53:48,640 - baro_tests.barometer - DEBUG - OK 2018-05-17 12:53:48,640 - baro_tests.barometer - INFO - OK 2018-05-17 12:53:48,640 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-05-17 12:54:00,653 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-05-17 12:54:14,509 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-17 12:54:14,510 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1526561180.409, 1526561190.409), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1526561180.409, 1526561190.409), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1526561180.409, 1526561190.409), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1526561180.409, 1526561190.409)] 2018-05-17 12:54:14,512 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1526561200.409, 1526561210.409), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1526561200.409, 1526561210.409), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1526561200.409, 1526561210.409), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1526561200.409, 1526561210.409)] 2018-05-17 12:54:14,512 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1526561180.41 - hugepages-mm-2048Kb vmpage_number-free 1526561200.41 ... 2018-05-17 12:54:14,514 - baro_tests.barometer - DEBUG - OK 2018-05-17 12:54:14,515 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1526561180.41 - hugepages-mm-2048Kb vmpage_number-used 1526561200.41 ... 2018-05-17 12:54:14,516 - baro_tests.barometer - DEBUG - OK 2018-05-17 12:54:14,517 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1526561180.41 - hugepages-node0-2048Kb vmpage_number-free 1526561200.41 ... 2018-05-17 12:54:14,518 - baro_tests.barometer - DEBUG - OK 2018-05-17 12:54:14,518 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1526561180.41 - hugepages-node0-2048Kb vmpage_number-used 1526561200.41 ... 2018-05-17 12:54:14,518 - baro_tests.barometer - DEBUG - OK 2018-05-17 12:54:14,518 - baro_tests.barometer - INFO - OK 2018-05-17 12:54:14,518 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Hugepages. 2018-05-17 12:55:08,768 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Intel RDT Test case execution === 2018-05-17 12:55:48,630 - baro_tests.barometer - INFO - Getting CSV metrics of plugin intel_rdt on compute node 8ff325fb-a79e-4f0a-ba6a-9538291f745b... 2018-05-17 12:55:48,630 - baro_tests.barometer - DEBUG - Interval: 10 2018-05-17 12:55:48,631 - baro_tests.barometer - DEBUG - Plugin subdirs: ['intel_rdt-0-2'] 2018-05-17 12:55:48,631 - baro_tests.barometer - DEBUG - Plugin meter categories: ['bytes-llc', 'ipc'] 2018-05-17 12:56:01,429 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-17 12:56:01,429 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1526561310.41 ... 2018-05-17 12:56:01,430 - baro_tests.barometer - DEBUG - OK 2018-05-17 12:56:01,430 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1526561310.41 ... 2018-05-17 12:56:01,430 - baro_tests.barometer - DEBUG - OK 2018-05-17 12:56:01,431 - baro_tests.barometer - INFO - OK 2018-05-17 12:56:01,431 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-05-17 12:56:13,442 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-05-17 12:56:26,459 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-17 12:56:26,459 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1526561310.409, 1526561320.409), ('intel_rdt-0-2', 'ipc', 1526561310.409, 1526561320.409)] 2018-05-17 12:56:26,459 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1526561330.409, 1526561340.409), ('intel_rdt-0-2', 'ipc', 1526561330.409, 1526561340.409)] 2018-05-17 12:56:26,460 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1526561310.41 - intel_rdt-0-2 bytes-llc 1526561330.41 ... 2018-05-17 12:56:26,460 - baro_tests.barometer - DEBUG - OK 2018-05-17 12:56:26,460 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1526561310.41 - intel_rdt-0-2 ipc 1526561330.41 ... 2018-05-17 12:56:26,460 - baro_tests.barometer - DEBUG - OK 2018-05-17 12:56:26,461 - baro_tests.barometer - INFO - OK 2018-05-17 12:56:26,461 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Intel RDT. 2018-05-17 12:57:20,120 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-05-17 12:57:45,809 - baro_tests.barometer - INFO - Getting CSV metrics of plugin mcelog on compute node 8ff325fb-a79e-4f0a-ba6a-9538291f745b... 2018-05-17 12:57:45,809 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-17 12:57:45,810 - baro_tests.barometer - DEBUG - Plugin subdirs: ['mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'mcelog-SOCKET_0_CHANNEL_any_DIMM_any'] 2018-05-17 12:57:45,811 - baro_tests.barometer - DEBUG - Plugin meter categories: ['errors-corrected_memory_errors', 'errors-uncorrected_memory_errors'] 2018-05-17 12:57:59,709 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-17 12:57:59,710 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1526561441.41 ... 2018-05-17 12:57:59,711 - baro_tests.barometer - DEBUG - OK 2018-05-17 12:57:59,711 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1526561441.41 ... 2018-05-17 12:57:59,711 - baro_tests.barometer - DEBUG - OK 2018-05-17 12:57:59,712 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1526561442.41 ... 2018-05-17 12:57:59,712 - baro_tests.barometer - DEBUG - OK 2018-05-17 12:57:59,712 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1526561442.41 ... 2018-05-17 12:57:59,712 - baro_tests.barometer - DEBUG - OK 2018-05-17 12:57:59,712 - baro_tests.barometer - INFO - OK 2018-05-17 12:57:59,712 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-17 12:58:02,716 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-05-17 12:58:15,637 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-17 12:58:15,638 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1526561441.41, 1526561442.41), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1526561441.41, 1526561442.41), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1526561442.41, 1526561443.41), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1526561442.41, 1526561443.41)] 2018-05-17 12:58:15,638 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1526561457.41, 1526561458.41), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1526561457.41, 1526561458.41), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1526561457.41, 1526561459.41), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1526561458.41, 1526561459.41)] 2018-05-17 12:58:15,639 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1526561441.41 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1526561457.41 ... 2018-05-17 12:58:15,639 - baro_tests.barometer - DEBUG - OK 2018-05-17 12:58:15,639 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1526561441.41 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1526561457.41 ... 2018-05-17 12:58:15,639 - baro_tests.barometer - DEBUG - OK 2018-05-17 12:58:15,640 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1526561442.41 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1526561457.41 ... 2018-05-17 12:58:15,640 - baro_tests.barometer - DEBUG - OK 2018-05-17 12:58:15,640 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1526561442.41 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1526561458.41 ... 2018-05-17 12:58:15,640 - baro_tests.barometer - DEBUG - OK 2018-05-17 12:58:15,640 - baro_tests.barometer - INFO - OK 2018-05-17 12:58:15,641 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Mcelog. 2018-05-17 12:59:07,867 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-05-17 12:59:33,694 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_events on compute node 8ff325fb-a79e-4f0a-ba6a-9538291f745b... 2018-05-17 12:59:33,694 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-17 12:59:33,695 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_events-br0'] 2018-05-17 12:59:33,696 - baro_tests.barometer - DEBUG - Plugin meter categories: ['gauge-link_status'] 2018-05-17 12:59:46,490 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-17 12:59:46,492 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1526561548.41 ... 2018-05-17 12:59:46,492 - baro_tests.barometer - DEBUG - OK 2018-05-17 12:59:46,493 - baro_tests.barometer - INFO - OK 2018-05-17 12:59:46,493 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-17 12:59:49,496 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-05-17 13:00:03,334 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-17 13:00:03,334 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1526561548.41, 1526561549.41)] 2018-05-17 13:00:03,335 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1526561565.41, 1526561566.41)] 2018-05-17 13:00:03,336 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1526561548.41 - ovs_events-br0 gauge-link_status 1526561565.41 ... 2018-05-17 13:00:03,336 - baro_tests.barometer - DEBUG - OK 2018-05-17 13:00:03,338 - baro_tests.barometer - INFO - OK 2018-05-17 13:00:03,339 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on OVS events. 2018-05-17 13:00:55,088 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS stats Test case execution === 2018-05-17 13:01:20,047 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_stats on compute node 8ff325fb-a79e-4f0a-ba6a-9538291f745b... 2018-05-17 13:01:20,048 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-17 13:01:20,049 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_stats-br0.br0'] 2018-05-17 13:01:20,050 - baro_tests.barometer - DEBUG - Plugin meter categories: ['if_dropped', 'if_errors', 'if_packets'] 2018-05-17 13:01:32,857 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-17 13:01:32,857 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1526561655.41 ... 2018-05-17 13:01:32,857 - baro_tests.barometer - DEBUG - OK 2018-05-17 13:01:32,858 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1526561655.41 ... 2018-05-17 13:01:32,858 - baro_tests.barometer - DEBUG - OK 2018-05-17 13:01:32,858 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1526561655.41 ... 2018-05-17 13:01:32,858 - baro_tests.barometer - DEBUG - OK 2018-05-17 13:01:32,858 - baro_tests.barometer - INFO - OK 2018-05-17 13:01:32,858 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-17 13:01:35,859 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-05-17 13:01:50,485 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-17 13:01:50,485 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1526561655.41, 1526561656.41), ('ovs_stats-br0.br0', 'if_errors', 1526561655.41, 1526561656.41), ('ovs_stats-br0.br0', 'if_packets', 1526561655.41, 1526561656.41)] 2018-05-17 13:01:50,486 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1526561672.41, 1526561673.41), ('ovs_stats-br0.br0', 'if_errors', 1526561672.41, 1526561673.41), ('ovs_stats-br0.br0', 'if_packets', 1526561672.41, 1526561673.41)] 2018-05-17 13:01:50,486 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1526561655.41 - ovs_stats-br0.br0 if_dropped 1526561672.41 ... 2018-05-17 13:01:50,486 - baro_tests.barometer - DEBUG - OK 2018-05-17 13:01:50,487 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1526561655.41 - ovs_stats-br0.br0 if_errors 1526561672.41 ... 2018-05-17 13:01:50,487 - baro_tests.barometer - DEBUG - OK 2018-05-17 13:01:50,487 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1526561655.41 - ovs_stats-br0.br0 if_packets 1526561672.41 ... 2018-05-17 13:01:50,487 - baro_tests.barometer - DEBUG - OK 2018-05-17 13:01:50,487 - baro_tests.barometer - INFO - OK 2018-05-17 13:01:50,487 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on OVS stats. 2018-05-17 13:02:03,463 - baro_tests.barometer - INFO - Gnocchi plugin available in compute node overcloud-novacompute-1 2018-05-17 13:02:16,110 - baro_tests.barometer - INFO - AODH plugin present in compute node overcloud-novacompute-1 2018-05-17 13:02:16,111 - baro_tests.barometer - INFO - === NODE overcloud-novacompute-1: Enabling Test Plug-in and Test case execution === 2018-05-17 13:03:31,964 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Hugepages Test case execution === 2018-05-17 13:03:56,851 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-17 13:04:34,223 - baro_tests.barometer - INFO - PASS 2018-05-17 13:04:34,227 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with Gnocchi PASSED on Hugepages. 2018-05-17 13:05:27,494 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Intel RDT Test case execution === 2018-05-17 13:06:05,574 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-17 13:06:42,781 - baro_tests.barometer - INFO - Plugin Interval is 10 2018-05-17 13:06:42,781 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-05-17 13:06:42,781 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on Intel RDT. 2018-05-17 13:07:36,750 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-05-17 13:08:03,435 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-17 13:08:40,897 - baro_tests.barometer - INFO - Plugin Interval is 1 2018-05-17 13:08:40,899 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-05-17 13:08:40,899 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on Mcelog. 2018-05-17 13:09:34,138 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-05-17 13:09:59,114 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-17 13:10:01,723 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-05-17 13:10:04,346 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-1 2018-05-17 13:10:07,096 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on OVS events. 2018-05-17 13:10:58,161 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS stats Test case execution === 2018-05-17 13:11:24,865 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-17 13:11:27,498 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-05-17 13:11:30,100 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-1 2018-05-17 13:11:32,697 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on OVS stats. 2018-05-17 13:12:23,903 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-05-17 13:12:49,508 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-05-17 13:13:08,370 - baro_tests.barometer - INFO - Data not updated after interval of 12 seconds 2018-05-17 13:13:08,372 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with AODH FAILED on Mcelog. 2018-05-17 13:13:58,414 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-05-17 13:14:23,697 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-05-17 13:14:26,094 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-0 2018-05-17 13:14:28,527 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-1 2018-05-17 13:14:30,894 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with AODH FAILED on OVS events. 2018-05-17 13:14:30,895 - baro_tests.barometer - INFO - ======= Node overcloud-novacompute-1: Executing CSV Testcases ======== 2018-05-17 13:14:30,895 - baro_tests.barometer - INFO - Restarting collectd for CSV tests 2018-05-17 13:14:43,196 - baro_tests.barometer - INFO - Stopping collectd service... 2018-05-17 13:14:53,473 - baro_tests.barometer - INFO - Starting collectd service... 2018-05-17 13:15:04,539 - baro_tests.barometer - INFO - Sleeping for 10 seconds after collectd restart... 2018-05-17 13:16:06,696 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Hugepages Test case execution === 2018-05-17 13:16:33,359 - baro_tests.barometer - INFO - Getting CSV metrics of plugin hugepages on compute node 646835b1-c2de-4084-81b9-769910ac4d97... 2018-05-17 13:16:33,359 - baro_tests.barometer - DEBUG - Interval: 10 2018-05-17 13:16:33,359 - baro_tests.barometer - DEBUG - Plugin subdirs: ['hugepages-mm-2048Kb', 'hugepages-node0-2048Kb'] 2018-05-17 13:16:33,360 - baro_tests.barometer - DEBUG - Plugin meter categories: ['vmpage_number-free', 'vmpage_number-used'] 2018-05-17 13:16:46,733 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-17 13:16:46,733 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1526562551.45 ... 2018-05-17 13:16:46,733 - baro_tests.barometer - DEBUG - OK 2018-05-17 13:16:46,734 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1526562551.45 ... 2018-05-17 13:16:46,734 - baro_tests.barometer - DEBUG - OK 2018-05-17 13:16:46,735 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1526562551.45 ... 2018-05-17 13:16:46,735 - baro_tests.barometer - DEBUG - OK 2018-05-17 13:16:46,735 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1526562551.45 ... 2018-05-17 13:16:46,736 - baro_tests.barometer - DEBUG - OK 2018-05-17 13:16:46,736 - baro_tests.barometer - INFO - OK 2018-05-17 13:16:46,736 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-05-17 13:16:58,744 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-05-17 13:17:11,163 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-17 13:17:11,165 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1526562551.455, 1526562561.455), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1526562551.455, 1526562561.455), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1526562551.455, 1526562561.455), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1526562551.455, 1526562561.455)] 2018-05-17 13:17:11,167 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1526562571.455, 1526562581.455), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1526562571.455, 1526562581.455), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1526562571.455, 1526562581.455), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1526562571.455, 1526562581.455)] 2018-05-17 13:17:11,168 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1526562551.45 - hugepages-mm-2048Kb vmpage_number-free 1526562571.45 ... 2018-05-17 13:17:11,168 - baro_tests.barometer - DEBUG - OK 2018-05-17 13:17:11,169 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1526562551.45 - hugepages-mm-2048Kb vmpage_number-used 1526562571.45 ... 2018-05-17 13:17:11,169 - baro_tests.barometer - DEBUG - OK 2018-05-17 13:17:11,171 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1526562551.45 - hugepages-node0-2048Kb vmpage_number-free 1526562571.45 ... 2018-05-17 13:17:11,172 - baro_tests.barometer - DEBUG - OK 2018-05-17 13:17:11,173 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1526562551.45 - hugepages-node0-2048Kb vmpage_number-used 1526562571.45 ... 2018-05-17 13:17:11,174 - baro_tests.barometer - DEBUG - OK 2018-05-17 13:17:11,174 - baro_tests.barometer - INFO - OK 2018-05-17 13:17:11,174 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Hugepages. 2018-05-17 13:18:01,373 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Intel RDT Test case execution === 2018-05-17 13:18:39,895 - baro_tests.barometer - INFO - Getting CSV metrics of plugin intel_rdt on compute node 646835b1-c2de-4084-81b9-769910ac4d97... 2018-05-17 13:18:39,895 - baro_tests.barometer - DEBUG - Interval: 10 2018-05-17 13:18:39,896 - baro_tests.barometer - DEBUG - Plugin subdirs: ['intel_rdt-0-2'] 2018-05-17 13:18:39,896 - baro_tests.barometer - DEBUG - Plugin meter categories: ['bytes-llc', 'ipc'] 2018-05-17 13:18:52,598 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-17 13:18:52,598 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1526562671.45 ... 2018-05-17 13:18:52,600 - baro_tests.barometer - DEBUG - OK 2018-05-17 13:18:52,600 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1526562671.45 ... 2018-05-17 13:18:52,602 - baro_tests.barometer - DEBUG - OK 2018-05-17 13:18:52,603 - baro_tests.barometer - INFO - OK 2018-05-17 13:18:52,603 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-05-17 13:19:04,611 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-05-17 13:19:17,603 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-17 13:19:17,604 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1526562671.455, 1526562681.455), ('intel_rdt-0-2', 'ipc', 1526562671.455, 1526562681.455)] 2018-05-17 13:19:17,604 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1526562701.455, 1526562711.455), ('intel_rdt-0-2', 'ipc', 1526562701.455, 1526562711.455)] 2018-05-17 13:19:17,604 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1526562671.45 - intel_rdt-0-2 bytes-llc 1526562701.45 ... 2018-05-17 13:19:17,604 - baro_tests.barometer - DEBUG - OK 2018-05-17 13:19:17,605 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1526562671.45 - intel_rdt-0-2 ipc 1526562701.45 ... 2018-05-17 13:19:17,605 - baro_tests.barometer - DEBUG - OK 2018-05-17 13:19:17,605 - baro_tests.barometer - INFO - OK 2018-05-17 13:19:17,611 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Intel RDT. 2018-05-17 13:20:05,820 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-05-17 13:20:30,413 - baro_tests.barometer - INFO - Getting CSV metrics of plugin mcelog on compute node 646835b1-c2de-4084-81b9-769910ac4d97... 2018-05-17 13:20:30,413 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-17 13:20:30,413 - baro_tests.barometer - DEBUG - Plugin subdirs: ['mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'mcelog-SOCKET_0_CHANNEL_any_DIMM_any'] 2018-05-17 13:20:30,414 - baro_tests.barometer - DEBUG - Plugin meter categories: ['errors-corrected_memory_errors', 'errors-uncorrected_memory_errors'] 2018-05-17 13:20:43,159 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-17 13:20:43,160 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1526562798.46 ... 2018-05-17 13:20:43,160 - baro_tests.barometer - DEBUG - OK 2018-05-17 13:20:43,160 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1526562798.46 ... 2018-05-17 13:20:43,162 - baro_tests.barometer - DEBUG - OK 2018-05-17 13:20:43,163 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1526562798.46 ... 2018-05-17 13:20:43,163 - baro_tests.barometer - DEBUG - OK 2018-05-17 13:20:43,163 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1526562798.46 ... 2018-05-17 13:20:43,164 - baro_tests.barometer - DEBUG - OK 2018-05-17 13:20:43,164 - baro_tests.barometer - INFO - OK 2018-05-17 13:20:43,165 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-17 13:20:46,169 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-05-17 13:20:59,795 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-17 13:20:59,796 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1526562798.456, 1526562799.456), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1526562798.456, 1526562799.456), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1526562798.456, 1526562799.456), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1526562798.456, 1526562799.456)] 2018-05-17 13:20:59,796 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1526562814.456, 1526562815.456), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1526562814.456, 1526562816.456), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1526562815.456, 1526562816.456), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1526562815.456, 1526562816.456)] 2018-05-17 13:20:59,796 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1526562798.46 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1526562814.46 ... 2018-05-17 13:20:59,797 - baro_tests.barometer - DEBUG - OK 2018-05-17 13:20:59,798 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1526562798.46 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1526562814.46 ... 2018-05-17 13:20:59,799 - baro_tests.barometer - DEBUG - OK 2018-05-17 13:20:59,799 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1526562798.46 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1526562815.46 ... 2018-05-17 13:20:59,803 - baro_tests.barometer - DEBUG - OK 2018-05-17 13:20:59,803 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1526562798.46 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1526562815.46 ... 2018-05-17 13:20:59,806 - baro_tests.barometer - DEBUG - OK 2018-05-17 13:20:59,806 - baro_tests.barometer - INFO - OK 2018-05-17 13:20:59,807 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Mcelog. 2018-05-17 13:21:49,750 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-05-17 13:22:15,061 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_events on compute node 646835b1-c2de-4084-81b9-769910ac4d97... 2018-05-17 13:22:15,061 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-17 13:22:15,061 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_events-br0'] 2018-05-17 13:22:15,062 - baro_tests.barometer - DEBUG - Plugin meter categories: ['gauge-link_status'] 2018-05-17 13:22:27,260 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-17 13:22:27,261 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1526562902.46 ... 2018-05-17 13:22:27,261 - baro_tests.barometer - DEBUG - OK 2018-05-17 13:22:27,261 - baro_tests.barometer - INFO - OK 2018-05-17 13:22:27,261 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-17 13:22:30,262 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-05-17 13:22:42,058 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-17 13:22:42,059 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1526562902.456, 1526562903.456)] 2018-05-17 13:22:42,060 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1526562917.456, 1526562918.456)] 2018-05-17 13:22:42,060 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1526562902.46 - ovs_events-br0 gauge-link_status 1526562917.46 ... 2018-05-17 13:22:42,061 - baro_tests.barometer - DEBUG - OK 2018-05-17 13:22:42,062 - baro_tests.barometer - INFO - OK 2018-05-17 13:22:42,067 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on OVS events. 2018-05-17 13:23:31,634 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS stats Test case execution === 2018-05-17 13:23:57,169 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_stats on compute node 646835b1-c2de-4084-81b9-769910ac4d97... 2018-05-17 13:23:57,170 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-17 13:23:57,170 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_stats-br0.br0'] 2018-05-17 13:23:57,170 - baro_tests.barometer - DEBUG - Plugin meter categories: ['if_dropped', 'if_errors', 'if_packets'] 2018-05-17 13:24:10,268 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-17 13:24:10,269 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1526563005.46 ... 2018-05-17 13:24:10,269 - baro_tests.barometer - DEBUG - OK 2018-05-17 13:24:10,269 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1526563005.46 ... 2018-05-17 13:24:10,269 - baro_tests.barometer - DEBUG - OK 2018-05-17 13:24:10,269 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1526563005.46 ... 2018-05-17 13:24:10,269 - baro_tests.barometer - DEBUG - OK 2018-05-17 13:24:10,269 - baro_tests.barometer - INFO - OK 2018-05-17 13:24:10,269 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-17 13:24:13,271 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-05-17 13:24:26,827 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-17 13:24:26,828 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1526563005.456, 1526563006.456), ('ovs_stats-br0.br0', 'if_errors', 1526563005.456, 1526563006.456), ('ovs_stats-br0.br0', 'if_packets', 1526563005.456, 1526563006.456)] 2018-05-17 13:24:26,828 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1526563021.456, 1526563023.456), ('ovs_stats-br0.br0', 'if_errors', 1526563022.456, 1526563023.456), ('ovs_stats-br0.br0', 'if_packets', 1526563022.456, 1526563023.456)] 2018-05-17 13:24:26,828 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1526563005.46 - ovs_stats-br0.br0 if_dropped 1526563021.46 ... 2018-05-17 13:24:26,829 - baro_tests.barometer - DEBUG - OK 2018-05-17 13:24:26,829 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1526563005.46 - ovs_stats-br0.br0 if_errors 1526563022.46 ... 2018-05-17 13:24:26,829 - baro_tests.barometer - DEBUG - OK 2018-05-17 13:24:26,829 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1526563005.46 - ovs_stats-br0.br0 if_packets 1526563022.46 ... 2018-05-17 13:24:26,830 - baro_tests.barometer - DEBUG - OK 2018-05-17 13:24:26,830 - baro_tests.barometer - INFO - OK 2018-05-17 13:24:26,830 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on OVS stats. 2018-05-17 13:24:39,662 - baro_tests.barometer - INFO - Mcelog is deleted from all compute nodes 2018-05-17 13:24:39,663 - baro_tests.barometer - INFO - ====================================================================== 2018-05-17 13:24:39,663 - baro_tests.barometer - INFO - +----------------------------------+ 2018-05-17 13:24:39,664 - baro_tests.barometer - INFO - | OVERALL SUMMARY | 2018-05-17 13:24:39,664 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-05-17 13:24:39,664 - baro_tests.barometer - INFO - | Test | Node-0 | Node-1 | 2018-05-17 13:24:39,664 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-05-17 13:24:39,665 - baro_tests.barometer - INFO - | OUT:Gnocchi | PASS | PASS | 2018-05-17 13:24:39,665 - baro_tests.barometer - INFO - | IN:Hugepages | FAIL | PASS | 2018-05-17 13:24:39,665 - baro_tests.barometer - INFO - | IN:Intel RDT | FAIL | FAIL | 2018-05-17 13:24:39,665 - baro_tests.barometer - INFO - | IN:Mcelog | PASS | FAIL | 2018-05-17 13:24:39,666 - baro_tests.barometer - INFO - | IN:OVS events | PASS | FAIL | 2018-05-17 13:24:39,666 - baro_tests.barometer - INFO - | IN:OVS stats | FAIL | FAIL | 2018-05-17 13:24:39,666 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-05-17 13:24:39,666 - baro_tests.barometer - INFO - | OUT:AODH | PASS | PASS | 2018-05-17 13:24:39,666 - baro_tests.barometer - INFO - | IN:Mcelog | FAIL | FAIL | 2018-05-17 13:24:39,666 - baro_tests.barometer - INFO - | IN:OVS events | SKIP | SKIP | 2018-05-17 13:24:39,666 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-05-17 13:24:39,666 - baro_tests.barometer - INFO - | OUT:CSV | PASS | PASS | 2018-05-17 13:24:39,666 - baro_tests.barometer - INFO - | IN:Hugepages | PASS | PASS | 2018-05-17 13:24:39,666 - baro_tests.barometer - INFO - | IN:Intel RDT | PASS | PASS | 2018-05-17 13:24:39,666 - baro_tests.barometer - INFO - | IN:Mcelog | PASS | PASS | 2018-05-17 13:24:39,666 - baro_tests.barometer - INFO - | IN:OVS events | PASS | PASS | 2018-05-17 13:24:39,667 - baro_tests.barometer - INFO - | IN:OVS stats | PASS | PASS | 2018-05-17 13:24:39,667 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-05-17 13:24:39,667 - baro_tests.barometer - INFO - ====================================================================== 2018-05-17 13:24:39,667 - baro_tests.barometer - ERROR - Some tests have failed or have not been executed 2018-05-17 13:24:39,667 - baro_tests.barometer - ERROR - Overall Result is Fail