2018-05-02 19:32:31,685 - baro_tests.barometer - INFO - =================== Starting barometer tests suite =================== 2018-05-02 19:32:31,686 - baro_tests.barometer - INFO - RSA key file /root/.ssh/id_rsa exists. 2018-05-02 19:32:36,067 - baro_tests.barometer - INFO - ==== Display of Control and Compute nodes available in the set up ==== 2018-05-02 19:32:36,067 - baro_tests.barometer - INFO - controllers: ['overcloud-controller-0: 192.30.9.6', 'overcloud-controller-1: 192.30.9.4', 'overcloud-controller-2: 192.30.9.7'] 2018-05-02 19:32:36,067 - baro_tests.barometer - INFO - computes: ['overcloud-novacompute-0: 192.30.9.3', 'overcloud-novacompute-1: 192.30.9.9'] 2018-05-02 19:32:36,067 - baro_tests.barometer - INFO - ========== Enabling mcelog and OVS bridges on compute nodes ========== 2018-05-02 19:32:49,545 - baro_tests.barometer - INFO - Checking if mcelog is enabled on node-overcloud-novacompute-0... 2018-05-02 19:32:49,613 - baro_tests.barometer - INFO - Mcelog will be enabled on node-overcloud-novacompute-0... 2018-05-02 19:32:50,113 - baro_tests.barometer - INFO - Mcelog was installed on node-overcloud-novacompute-0. 2018-05-02 19:32:50,145 - baro_tests.barometer - INFO - Checking if mcelog is enabled on node-overcloud-novacompute-1... 2018-05-02 19:32:50,213 - baro_tests.barometer - INFO - Mcelog will be enabled on node-overcloud-novacompute-1... 2018-05-02 19:32:50,724 - baro_tests.barometer - INFO - Mcelog was installed on node-overcloud-novacompute-1. 2018-05-02 19:33:05,248 - baro_tests.barometer - INFO - Creating OVS bridges on computes nodes 2018-05-02 19:33:05,566 - baro_tests.barometer - INFO - OVS Bridges created on compute nodes 2018-05-02 19:33:05,566 - baro_tests.barometer - INFO - ============= Testing Gnocchi and AODH plugins on nodes ============== 2018-05-02 19:34:55,598 - baro_tests.barometer - INFO - Gnocchi plugin available in compute node overcloud-novacompute-0 2018-05-02 19:35:09,949 - baro_tests.barometer - INFO - AODH plugin present in compute node overcloud-novacompute-0 2018-05-02 19:35:09,950 - baro_tests.barometer - INFO - === NODE overcloud-novacompute-0: Enabling Test Plug-in and Test case execution === 2018-05-02 19:36:43,614 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Hugepages Test case execution === 2018-05-02 19:37:11,786 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-02 19:37:49,029 - baro_tests.barometer - INFO - Plugin Interval is 10 2018-05-02 19:37:49,029 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-05-02 19:37:49,029 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on Hugepages. 2018-05-02 19:38:45,547 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Intel RDT Test case execution === 2018-05-02 19:39:26,302 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-02 19:40:03,618 - baro_tests.barometer - INFO - PASS 2018-05-02 19:40:03,619 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with Gnocchi PASSED on Intel RDT. 2018-05-02 19:40:58,994 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-05-02 19:41:26,858 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-02 19:42:04,187 - baro_tests.barometer - INFO - Plugin Interval is 1 2018-05-02 19:42:04,188 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-05-02 19:42:04,188 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on Mcelog. 2018-05-02 19:42:59,103 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-05-02 19:43:28,037 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-02 19:43:30,620 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-05-02 19:43:33,349 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-1 2018-05-02 19:43:36,123 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on OVS events. 2018-05-02 19:44:31,315 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS stats Test case execution === 2018-05-02 19:44:58,940 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-02 19:45:36,231 - baro_tests.barometer - INFO - PASS 2018-05-02 19:45:36,231 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with Gnocchi PASSED on OVS stats. 2018-05-02 19:46:31,695 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-05-02 19:46:58,998 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-05-02 19:47:18,864 - baro_tests.barometer - INFO - Data not updated after interval of 12 seconds 2018-05-02 19:47:18,865 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with AODH FAILED on Mcelog. 2018-05-02 19:48:13,521 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-05-02 19:48:40,498 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-05-02 19:48:42,830 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-0 2018-05-02 19:48:45,649 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-1 2018-05-02 19:48:48,151 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with AODH FAILED on OVS events. 2018-05-02 19:48:48,152 - baro_tests.barometer - INFO - ======= Node overcloud-novacompute-0: Executing CSV Testcases ======== 2018-05-02 19:48:48,152 - baro_tests.barometer - INFO - Restarting collectd for CSV tests 2018-05-02 19:49:01,216 - baro_tests.barometer - INFO - Stopping collectd service... 2018-05-02 19:49:11,949 - baro_tests.barometer - INFO - Starting collectd service... 2018-05-02 19:49:22,259 - baro_tests.barometer - INFO - Sleeping for 10 seconds after collectd restart... 2018-05-02 19:50:26,582 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Hugepages Test case execution === 2018-05-02 19:50:54,427 - baro_tests.barometer - INFO - Getting CSV metrics of plugin hugepages on compute node 3dfbfa1b-eba4-4364-86c5-5edb79ae2bf5... 2018-05-02 19:50:54,427 - baro_tests.barometer - DEBUG - Interval: 10 2018-05-02 19:50:54,428 - baro_tests.barometer - DEBUG - Plugin subdirs: ['hugepages-mm-2048Kb', 'hugepages-node0-2048Kb'] 2018-05-02 19:50:54,428 - baro_tests.barometer - DEBUG - Plugin meter categories: ['vmpage_number-free', 'vmpage_number-used'] 2018-05-02 19:51:08,598 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-02 19:51:08,598 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1525290231.92 ... 2018-05-02 19:51:08,600 - baro_tests.barometer - DEBUG - OK 2018-05-02 19:51:08,608 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1525290231.92 ... 2018-05-02 19:51:08,621 - baro_tests.barometer - DEBUG - OK 2018-05-02 19:51:08,631 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1525290231.92 ... 2018-05-02 19:51:08,632 - baro_tests.barometer - DEBUG - OK 2018-05-02 19:51:08,632 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1525290231.92 ... 2018-05-02 19:51:08,632 - baro_tests.barometer - DEBUG - OK 2018-05-02 19:51:08,632 - baro_tests.barometer - INFO - OK 2018-05-02 19:51:08,632 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-05-02 19:51:20,637 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-05-02 19:51:35,088 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-02 19:51:35,089 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1525290231.919, 1525290241.919), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1525290231.919, 1525290241.919), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1525290231.919, 1525290241.919), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1525290231.919, 1525290241.919)] 2018-05-02 19:51:35,089 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1525290261.918, 1525290271.919), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1525290261.918, 1525290271.919), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1525290261.919, 1525290271.919), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1525290261.919, 1525290271.919)] 2018-05-02 19:51:35,090 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1525290231.92 - hugepages-mm-2048Kb vmpage_number-free 1525290261.92 ... 2018-05-02 19:51:35,090 - baro_tests.barometer - DEBUG - OK 2018-05-02 19:51:35,090 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1525290231.92 - hugepages-mm-2048Kb vmpage_number-used 1525290261.92 ... 2018-05-02 19:51:35,090 - baro_tests.barometer - DEBUG - OK 2018-05-02 19:51:35,090 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1525290231.92 - hugepages-node0-2048Kb vmpage_number-free 1525290261.92 ... 2018-05-02 19:51:35,090 - baro_tests.barometer - DEBUG - OK 2018-05-02 19:51:35,090 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1525290231.92 - hugepages-node0-2048Kb vmpage_number-used 1525290261.92 ... 2018-05-02 19:51:35,090 - baro_tests.barometer - DEBUG - OK 2018-05-02 19:51:35,090 - baro_tests.barometer - INFO - OK 2018-05-02 19:51:35,090 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Hugepages. 2018-05-02 19:52:31,400 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Intel RDT Test case execution === 2018-05-02 19:53:12,913 - baro_tests.barometer - INFO - Getting CSV metrics of plugin intel_rdt on compute node 3dfbfa1b-eba4-4364-86c5-5edb79ae2bf5... 2018-05-02 19:53:12,913 - baro_tests.barometer - DEBUG - Interval: 10 2018-05-02 19:53:12,914 - baro_tests.barometer - DEBUG - Plugin subdirs: ['intel_rdt-0-2'] 2018-05-02 19:53:12,914 - baro_tests.barometer - DEBUG - Plugin meter categories: ['bytes-llc', 'ipc'] 2018-05-02 19:53:26,350 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-02 19:53:26,352 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1525290371.92 ... 2018-05-02 19:53:26,354 - baro_tests.barometer - DEBUG - OK 2018-05-02 19:53:26,355 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1525290371.92 ... 2018-05-02 19:53:26,356 - baro_tests.barometer - DEBUG - OK 2018-05-02 19:53:26,357 - baro_tests.barometer - INFO - OK 2018-05-02 19:53:26,358 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-05-02 19:53:38,370 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-05-02 19:53:52,804 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-02 19:53:52,805 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1525290371.919, 1525290381.919), ('intel_rdt-0-2', 'ipc', 1525290371.919, 1525290381.919)] 2018-05-02 19:53:52,805 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1525290401.918, 1525290411.919), ('intel_rdt-0-2', 'ipc', 1525290401.918, 1525290411.919)] 2018-05-02 19:53:52,805 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1525290371.92 - intel_rdt-0-2 bytes-llc 1525290401.92 ... 2018-05-02 19:53:52,806 - baro_tests.barometer - DEBUG - OK 2018-05-02 19:53:52,806 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1525290371.92 - intel_rdt-0-2 ipc 1525290401.92 ... 2018-05-02 19:53:52,806 - baro_tests.barometer - DEBUG - OK 2018-05-02 19:53:52,806 - baro_tests.barometer - INFO - OK 2018-05-02 19:53:52,808 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Intel RDT. 2018-05-02 19:54:47,597 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-05-02 19:55:13,754 - baro_tests.barometer - INFO - Getting CSV metrics of plugin mcelog on compute node 3dfbfa1b-eba4-4364-86c5-5edb79ae2bf5... 2018-05-02 19:55:13,755 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-02 19:55:13,755 - baro_tests.barometer - DEBUG - Plugin subdirs: ['mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'mcelog-SOCKET_0_CHANNEL_any_DIMM_any'] 2018-05-02 19:55:13,756 - baro_tests.barometer - DEBUG - Plugin meter categories: ['errors-corrected_memory_errors', 'errors-uncorrected_memory_errors'] 2018-05-02 19:55:27,434 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-02 19:55:27,435 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1525290504.92 ... 2018-05-02 19:55:27,435 - baro_tests.barometer - DEBUG - OK 2018-05-02 19:55:27,435 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1525290504.92 ... 2018-05-02 19:55:27,436 - baro_tests.barometer - DEBUG - OK 2018-05-02 19:55:27,436 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1525290504.92 ... 2018-05-02 19:55:27,437 - baro_tests.barometer - DEBUG - OK 2018-05-02 19:55:27,437 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1525290505.92 ... 2018-05-02 19:55:27,437 - baro_tests.barometer - DEBUG - OK 2018-05-02 19:55:27,437 - baro_tests.barometer - INFO - OK 2018-05-02 19:55:27,437 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-02 19:55:30,438 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-05-02 19:55:44,930 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-02 19:55:44,930 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1525290504.92, 1525290505.92), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1525290504.92, 1525290505.92), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1525290504.92, 1525290506.92), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1525290505.92, 1525290506.92)] 2018-05-02 19:55:44,931 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1525290522.92, 1525290523.92), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1525290522.92, 1525290523.92), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1525290522.92, 1525290523.92), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1525290522.92, 1525290523.92)] 2018-05-02 19:55:44,932 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1525290504.92 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1525290522.92 ... 2018-05-02 19:55:44,933 - baro_tests.barometer - DEBUG - OK 2018-05-02 19:55:44,933 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1525290504.92 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1525290522.92 ... 2018-05-02 19:55:44,933 - baro_tests.barometer - DEBUG - OK 2018-05-02 19:55:44,933 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1525290504.92 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1525290522.92 ... 2018-05-02 19:55:44,933 - baro_tests.barometer - DEBUG - OK 2018-05-02 19:55:44,933 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1525290505.92 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1525290522.92 ... 2018-05-02 19:55:44,934 - baro_tests.barometer - DEBUG - OK 2018-05-02 19:55:44,934 - baro_tests.barometer - INFO - OK 2018-05-02 19:55:44,934 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Mcelog. 2018-05-02 19:56:39,115 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-05-02 19:57:06,584 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_events on compute node 3dfbfa1b-eba4-4364-86c5-5edb79ae2bf5... 2018-05-02 19:57:06,585 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-02 19:57:06,585 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_events-br0'] 2018-05-02 19:57:06,585 - baro_tests.barometer - DEBUG - Plugin meter categories: ['gauge-link_status'] 2018-05-02 19:57:20,595 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-02 19:57:20,595 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1525290618.92 ... 2018-05-02 19:57:20,596 - baro_tests.barometer - DEBUG - OK 2018-05-02 19:57:20,596 - baro_tests.barometer - INFO - OK 2018-05-02 19:57:20,596 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-02 19:57:23,599 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-05-02 19:57:37,089 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-02 19:57:37,089 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1525290618.92, 1525290619.92)] 2018-05-02 19:57:37,089 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1525290634.92, 1525290635.92)] 2018-05-02 19:57:37,090 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1525290618.92 - ovs_events-br0 gauge-link_status 1525290634.92 ... 2018-05-02 19:57:37,090 - baro_tests.barometer - DEBUG - OK 2018-05-02 19:57:37,091 - baro_tests.barometer - INFO - OK 2018-05-02 19:57:37,095 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on OVS events. 2018-05-02 19:58:32,402 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS stats Test case execution === 2018-05-02 19:58:59,798 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_stats on compute node 3dfbfa1b-eba4-4364-86c5-5edb79ae2bf5... 2018-05-02 19:58:59,799 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-02 19:58:59,799 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_stats-br0.br0'] 2018-05-02 19:58:59,799 - baro_tests.barometer - DEBUG - Plugin meter categories: ['if_dropped', 'if_errors', 'if_packets'] 2018-05-02 19:59:13,034 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-02 19:59:13,035 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1525290730.92 ... 2018-05-02 19:59:13,035 - baro_tests.barometer - DEBUG - OK 2018-05-02 19:59:13,035 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1525290730.92 ... 2018-05-02 19:59:13,036 - baro_tests.barometer - DEBUG - OK 2018-05-02 19:59:13,036 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1525290730.92 ... 2018-05-02 19:59:13,036 - baro_tests.barometer - DEBUG - OK 2018-05-02 19:59:13,036 - baro_tests.barometer - INFO - OK 2018-05-02 19:59:13,037 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-02 19:59:16,038 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-05-02 19:59:29,234 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-02 19:59:29,234 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1525290730.92, 1525290731.92), ('ovs_stats-br0.br0', 'if_errors', 1525290730.92, 1525290731.92), ('ovs_stats-br0.br0', 'if_packets', 1525290730.92, 1525290731.92)] 2018-05-02 19:59:29,235 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1525290746.92, 1525290747.92), ('ovs_stats-br0.br0', 'if_errors', 1525290746.92, 1525290747.92), ('ovs_stats-br0.br0', 'if_packets', 1525290746.92, 1525290747.92)] 2018-05-02 19:59:29,235 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1525290730.92 - ovs_stats-br0.br0 if_dropped 1525290746.92 ... 2018-05-02 19:59:29,235 - baro_tests.barometer - DEBUG - OK 2018-05-02 19:59:29,235 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1525290730.92 - ovs_stats-br0.br0 if_errors 1525290746.92 ... 2018-05-02 19:59:29,236 - baro_tests.barometer - DEBUG - OK 2018-05-02 19:59:29,236 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1525290730.92 - ovs_stats-br0.br0 if_packets 1525290746.92 ... 2018-05-02 19:59:29,237 - baro_tests.barometer - DEBUG - OK 2018-05-02 19:59:29,237 - baro_tests.barometer - INFO - OK 2018-05-02 19:59:29,237 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on OVS stats. 2018-05-02 19:59:41,830 - baro_tests.barometer - INFO - Gnocchi plugin available in compute node overcloud-novacompute-1 2018-05-02 19:59:54,566 - baro_tests.barometer - INFO - AODH plugin present in compute node overcloud-novacompute-1 2018-05-02 19:59:54,567 - baro_tests.barometer - INFO - === NODE overcloud-novacompute-1: Enabling Test Plug-in and Test case execution === 2018-05-02 20:01:23,073 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Hugepages Test case execution === 2018-05-02 20:01:51,908 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-02 20:02:29,273 - baro_tests.barometer - INFO - Plugin Interval is 10 2018-05-02 20:02:29,273 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-05-02 20:02:29,274 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on Hugepages. 2018-05-02 20:03:22,143 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Intel RDT Test case execution === 2018-05-02 20:04:02,150 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-02 20:04:40,037 - baro_tests.barometer - INFO - Plugin Interval is 10 2018-05-02 20:04:40,037 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-05-02 20:04:40,037 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on Intel RDT. 2018-05-02 20:05:34,375 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-05-02 20:05:59,641 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-02 20:06:37,375 - baro_tests.barometer - INFO - PASS 2018-05-02 20:06:37,382 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with Gnocchi PASSED on Mcelog. 2018-05-02 20:07:33,299 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-05-02 20:07:59,646 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-02 20:08:37,170 - baro_tests.barometer - INFO - Plugin Interval is 1 2018-05-02 20:08:37,171 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-05-02 20:08:37,171 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on OVS events. 2018-05-02 20:09:33,494 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS stats Test case execution === 2018-05-02 20:10:00,849 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-02 20:10:38,566 - baro_tests.barometer - INFO - PASS 2018-05-02 20:10:38,566 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with Gnocchi PASSED on OVS stats. 2018-05-02 20:11:29,430 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-05-02 20:11:57,038 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-05-02 20:12:16,022 - baro_tests.barometer - INFO - Data not updated after interval of 12 seconds 2018-05-02 20:12:16,023 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with AODH FAILED on Mcelog. 2018-05-02 20:13:10,165 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-05-02 20:13:36,134 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-05-02 20:13:38,537 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-0 2018-05-02 20:13:40,894 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-1 2018-05-02 20:13:43,376 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with AODH FAILED on OVS events. 2018-05-02 20:13:43,377 - baro_tests.barometer - INFO - ======= Node overcloud-novacompute-1: Executing CSV Testcases ======== 2018-05-02 20:13:43,377 - baro_tests.barometer - INFO - Restarting collectd for CSV tests 2018-05-02 20:13:56,405 - baro_tests.barometer - INFO - Stopping collectd service... 2018-05-02 20:14:07,058 - baro_tests.barometer - INFO - Starting collectd service... 2018-05-02 20:14:17,392 - baro_tests.barometer - INFO - Sleeping for 10 seconds after collectd restart... 2018-05-02 20:15:20,288 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Hugepages Test case execution === 2018-05-02 20:15:46,652 - baro_tests.barometer - INFO - Getting CSV metrics of plugin hugepages on compute node 393a9ab2-bc7c-4ad4-b4fa-0a4941d251f8... 2018-05-02 20:15:46,654 - baro_tests.barometer - DEBUG - Interval: 10 2018-05-02 20:15:46,655 - baro_tests.barometer - DEBUG - Plugin subdirs: ['hugepages-mm-2048Kb', 'hugepages-node0-2048Kb'] 2018-05-02 20:15:46,658 - baro_tests.barometer - DEBUG - Plugin meter categories: ['vmpage_number-free', 'vmpage_number-used'] 2018-05-02 20:16:00,388 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-02 20:16:00,388 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1525291722.2 ... 2018-05-02 20:16:00,389 - baro_tests.barometer - DEBUG - OK 2018-05-02 20:16:00,390 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1525291722.2 ... 2018-05-02 20:16:00,390 - baro_tests.barometer - DEBUG - OK 2018-05-02 20:16:00,390 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1525291722.2 ... 2018-05-02 20:16:00,390 - baro_tests.barometer - DEBUG - OK 2018-05-02 20:16:00,391 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1525291722.2 ... 2018-05-02 20:16:00,391 - baro_tests.barometer - DEBUG - OK 2018-05-02 20:16:00,391 - baro_tests.barometer - INFO - OK 2018-05-02 20:16:00,391 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-05-02 20:16:12,404 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-05-02 20:16:26,048 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-02 20:16:26,049 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1525291722.203, 1525291732.203), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1525291722.203, 1525291732.203), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1525291722.203, 1525291732.203), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1525291722.203, 1525291732.203)] 2018-05-02 20:16:26,052 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1525291742.203, 1525291752.203), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1525291742.203, 1525291752.203), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1525291742.203, 1525291752.203), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1525291742.203, 1525291752.203)] 2018-05-02 20:16:26,054 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1525291722.2 - hugepages-mm-2048Kb vmpage_number-free 1525291742.2 ... 2018-05-02 20:16:26,054 - baro_tests.barometer - DEBUG - OK 2018-05-02 20:16:26,054 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1525291722.2 - hugepages-mm-2048Kb vmpage_number-used 1525291742.2 ... 2018-05-02 20:16:26,054 - baro_tests.barometer - DEBUG - OK 2018-05-02 20:16:26,054 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1525291722.2 - hugepages-node0-2048Kb vmpage_number-free 1525291742.2 ... 2018-05-02 20:16:26,054 - baro_tests.barometer - DEBUG - OK 2018-05-02 20:16:26,055 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1525291722.2 - hugepages-node0-2048Kb vmpage_number-used 1525291742.2 ... 2018-05-02 20:16:26,055 - baro_tests.barometer - DEBUG - OK 2018-05-02 20:16:26,055 - baro_tests.barometer - INFO - OK 2018-05-02 20:16:26,056 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Hugepages. 2018-05-02 20:17:17,791 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Intel RDT Test case execution === 2018-05-02 20:17:57,570 - baro_tests.barometer - INFO - Getting CSV metrics of plugin intel_rdt on compute node 393a9ab2-bc7c-4ad4-b4fa-0a4941d251f8... 2018-05-02 20:17:57,570 - baro_tests.barometer - DEBUG - Interval: 10 2018-05-02 20:17:57,571 - baro_tests.barometer - DEBUG - Plugin subdirs: ['intel_rdt-0-2'] 2018-05-02 20:17:57,571 - baro_tests.barometer - DEBUG - Plugin meter categories: ['bytes-llc', 'ipc'] 2018-05-02 20:18:11,183 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-02 20:18:11,183 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1525291852.2 ... 2018-05-02 20:18:11,185 - baro_tests.barometer - DEBUG - OK 2018-05-02 20:18:11,185 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1525291852.2 ... 2018-05-02 20:18:11,186 - baro_tests.barometer - DEBUG - OK 2018-05-02 20:18:11,186 - baro_tests.barometer - INFO - OK 2018-05-02 20:18:11,187 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-05-02 20:18:23,196 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-05-02 20:18:37,311 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-02 20:18:37,311 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1525291852.203, 1525291862.203), ('intel_rdt-0-2', 'ipc', 1525291852.203, 1525291862.203)] 2018-05-02 20:18:37,312 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1525291872.203, 1525291882.203), ('intel_rdt-0-2', 'ipc', 1525291872.203, 1525291882.203)] 2018-05-02 20:18:37,312 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1525291852.2 - intel_rdt-0-2 bytes-llc 1525291872.2 ... 2018-05-02 20:18:37,313 - baro_tests.barometer - DEBUG - OK 2018-05-02 20:18:37,313 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1525291852.2 - intel_rdt-0-2 ipc 1525291872.2 ... 2018-05-02 20:18:37,313 - baro_tests.barometer - DEBUG - OK 2018-05-02 20:18:37,313 - baro_tests.barometer - INFO - OK 2018-05-02 20:18:37,314 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Intel RDT. 2018-05-02 20:19:29,173 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-05-02 20:19:54,960 - baro_tests.barometer - INFO - Getting CSV metrics of plugin mcelog on compute node 393a9ab2-bc7c-4ad4-b4fa-0a4941d251f8... 2018-05-02 20:19:54,960 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-02 20:19:54,961 - baro_tests.barometer - DEBUG - Plugin subdirs: ['mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'mcelog-SOCKET_0_CHANNEL_any_DIMM_any'] 2018-05-02 20:19:54,961 - baro_tests.barometer - DEBUG - Plugin meter categories: ['errors-corrected_memory_errors', 'errors-uncorrected_memory_errors'] 2018-05-02 20:20:08,735 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-02 20:20:08,736 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1525291981.2 ... 2018-05-02 20:20:08,736 - baro_tests.barometer - DEBUG - OK 2018-05-02 20:20:08,737 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1525291982.2 ... 2018-05-02 20:20:08,738 - baro_tests.barometer - DEBUG - OK 2018-05-02 20:20:08,738 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1525291982.2 ... 2018-05-02 20:20:08,740 - baro_tests.barometer - DEBUG - OK 2018-05-02 20:20:08,740 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1525291982.2 ... 2018-05-02 20:20:08,740 - baro_tests.barometer - DEBUG - OK 2018-05-02 20:20:08,740 - baro_tests.barometer - INFO - OK 2018-05-02 20:20:08,740 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-02 20:20:11,743 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-05-02 20:20:26,455 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-02 20:20:26,455 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1525291981.204, 1525291982.204), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1525291982.204, 1525291983.204), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1525291982.204, 1525291983.204), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1525291982.204, 1525291983.204)] 2018-05-02 20:20:26,456 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1525291999.204, 1525292000.204), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1525291999.204, 1525292000.204), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1525291999.204, 1525292000.204), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1525292000.204, 1525292001.205)] 2018-05-02 20:20:26,456 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1525291981.2 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1525291999.2 ... 2018-05-02 20:20:26,456 - baro_tests.barometer - DEBUG - OK 2018-05-02 20:20:26,456 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1525291982.2 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1525291999.2 ... 2018-05-02 20:20:26,457 - baro_tests.barometer - DEBUG - OK 2018-05-02 20:20:26,457 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1525291982.2 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1525291999.2 ... 2018-05-02 20:20:26,457 - baro_tests.barometer - DEBUG - OK 2018-05-02 20:20:26,457 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1525291982.2 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1525292000.2 ... 2018-05-02 20:20:26,457 - baro_tests.barometer - DEBUG - OK 2018-05-02 20:20:26,457 - baro_tests.barometer - INFO - OK 2018-05-02 20:20:26,458 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Mcelog. 2018-05-02 20:21:18,626 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-05-02 20:21:43,763 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_events on compute node 393a9ab2-bc7c-4ad4-b4fa-0a4941d251f8... 2018-05-02 20:21:43,763 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-02 20:21:43,763 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_events-br0'] 2018-05-02 20:21:43,764 - baro_tests.barometer - DEBUG - Plugin meter categories: ['gauge-link_status'] 2018-05-02 20:21:57,095 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-02 20:21:57,095 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1525292090.2 ... 2018-05-02 20:21:57,096 - baro_tests.barometer - DEBUG - OK 2018-05-02 20:21:57,096 - baro_tests.barometer - INFO - OK 2018-05-02 20:21:57,096 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-02 20:22:00,097 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-05-02 20:22:13,996 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-02 20:22:13,997 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1525292090.204, 1525292091.204)] 2018-05-02 20:22:13,998 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1525292107.204, 1525292108.204)] 2018-05-02 20:22:13,998 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1525292090.2 - ovs_events-br0 gauge-link_status 1525292107.2 ... 2018-05-02 20:22:13,999 - baro_tests.barometer - DEBUG - OK 2018-05-02 20:22:14,000 - baro_tests.barometer - INFO - OK 2018-05-02 20:22:14,002 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on OVS events. 2018-05-02 20:23:05,684 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS stats Test case execution === 2018-05-02 20:23:32,300 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_stats on compute node 393a9ab2-bc7c-4ad4-b4fa-0a4941d251f8... 2018-05-02 20:23:32,300 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-02 20:23:32,302 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_stats-br0.br0'] 2018-05-02 20:23:32,304 - baro_tests.barometer - DEBUG - Plugin meter categories: ['if_dropped', 'if_errors', 'if_packets'] 2018-05-02 20:23:46,537 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-02 20:23:46,537 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1525292199.2 ... 2018-05-02 20:23:46,538 - baro_tests.barometer - DEBUG - OK 2018-05-02 20:23:46,539 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1525292199.2 ... 2018-05-02 20:23:46,540 - baro_tests.barometer - DEBUG - OK 2018-05-02 20:23:46,541 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1525292200.2 ... 2018-05-02 20:23:46,541 - baro_tests.barometer - DEBUG - OK 2018-05-02 20:23:46,542 - baro_tests.barometer - INFO - OK 2018-05-02 20:23:46,542 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-02 20:23:49,545 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-05-02 20:24:03,398 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-02 20:24:03,398 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1525292199.204, 1525292200.204), ('ovs_stats-br0.br0', 'if_errors', 1525292199.204, 1525292201.204), ('ovs_stats-br0.br0', 'if_packets', 1525292200.204, 1525292201.204)] 2018-05-02 20:24:03,399 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1525292216.204, 1525292217.204), ('ovs_stats-br0.br0', 'if_errors', 1525292216.204, 1525292217.204), ('ovs_stats-br0.br0', 'if_packets', 1525292216.204, 1525292218.204)] 2018-05-02 20:24:03,399 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1525292199.2 - ovs_stats-br0.br0 if_dropped 1525292216.2 ... 2018-05-02 20:24:03,399 - baro_tests.barometer - DEBUG - OK 2018-05-02 20:24:03,400 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1525292199.2 - ovs_stats-br0.br0 if_errors 1525292216.2 ... 2018-05-02 20:24:03,401 - baro_tests.barometer - DEBUG - OK 2018-05-02 20:24:03,401 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1525292200.2 - ovs_stats-br0.br0 if_packets 1525292216.2 ... 2018-05-02 20:24:03,401 - baro_tests.barometer - DEBUG - OK 2018-05-02 20:24:03,402 - baro_tests.barometer - INFO - OK 2018-05-02 20:24:03,404 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on OVS stats. 2018-05-02 20:24:16,499 - baro_tests.barometer - INFO - Mcelog is deleted from all compute nodes 2018-05-02 20:24:16,500 - baro_tests.barometer - INFO - ====================================================================== 2018-05-02 20:24:16,500 - baro_tests.barometer - INFO - +----------------------------------+ 2018-05-02 20:24:16,500 - baro_tests.barometer - INFO - | OVERALL SUMMARY | 2018-05-02 20:24:16,501 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-05-02 20:24:16,501 - baro_tests.barometer - INFO - | Test | Node-0 | Node-1 | 2018-05-02 20:24:16,501 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-05-02 20:24:16,501 - baro_tests.barometer - INFO - | OUT:Gnocchi | PASS | PASS | 2018-05-02 20:24:16,502 - baro_tests.barometer - INFO - | IN:Hugepages | FAIL | FAIL | 2018-05-02 20:24:16,505 - baro_tests.barometer - INFO - | IN:Intel RDT | PASS | FAIL | 2018-05-02 20:24:16,508 - baro_tests.barometer - INFO - | IN:Mcelog | FAIL | PASS | 2018-05-02 20:24:16,509 - baro_tests.barometer - INFO - | IN:OVS events | FAIL | FAIL | 2018-05-02 20:24:16,509 - baro_tests.barometer - INFO - | IN:OVS stats | PASS | PASS | 2018-05-02 20:24:16,509 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-05-02 20:24:16,509 - baro_tests.barometer - INFO - | OUT:AODH | PASS | PASS | 2018-05-02 20:24:16,509 - baro_tests.barometer - INFO - | IN:Mcelog | FAIL | FAIL | 2018-05-02 20:24:16,509 - baro_tests.barometer - INFO - | IN:OVS events | SKIP | SKIP | 2018-05-02 20:24:16,510 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-05-02 20:24:16,511 - baro_tests.barometer - INFO - | OUT:CSV | PASS | PASS | 2018-05-02 20:24:16,511 - baro_tests.barometer - INFO - | IN:Hugepages | PASS | PASS | 2018-05-02 20:24:16,511 - baro_tests.barometer - INFO - | IN:Intel RDT | PASS | PASS | 2018-05-02 20:24:16,512 - baro_tests.barometer - INFO - | IN:Mcelog | PASS | PASS | 2018-05-02 20:24:16,512 - baro_tests.barometer - INFO - | IN:OVS events | PASS | PASS | 2018-05-02 20:24:16,512 - baro_tests.barometer - INFO - | IN:OVS stats | PASS | PASS | 2018-05-02 20:24:16,512 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-05-02 20:24:16,512 - baro_tests.barometer - INFO - ====================================================================== 2018-05-02 20:24:16,512 - baro_tests.barometer - ERROR - Some tests have failed or have not been executed 2018-05-02 20:24:16,512 - baro_tests.barometer - ERROR - Overall Result is Fail