2018-04-02 21:18:46,221 - baro_tests.barometer - INFO - =================== Starting barometer tests suite =================== 2018-04-02 21:18:46,221 - baro_tests.barometer - INFO - RSA key file /root/.ssh/id_rsa exists. 2018-04-02 21:18:50,561 - baro_tests.barometer - INFO - ==== Display of Control and Compute nodes available in the set up ==== 2018-04-02 21:18:50,561 - baro_tests.barometer - INFO - controllers: ['overcloud-controller-0: 192.30.9.8'] 2018-04-02 21:18:50,561 - baro_tests.barometer - INFO - computes: ['overcloud-novacompute-0: 192.30.9.3', 'overcloud-novacompute-1: 192.30.9.10'] 2018-04-02 21:18:50,562 - baro_tests.barometer - INFO - ========== Enabling mcelog and OVS bridges on compute nodes ========== 2018-04-02 21:19:02,398 - baro_tests.barometer - INFO - Checking if mcelog is enabled on node-overcloud-novacompute-0... 2018-04-02 21:19:02,447 - baro_tests.barometer - INFO - Mcelog will be enabled on node-overcloud-novacompute-0... 2018-04-02 21:19:02,815 - baro_tests.barometer - INFO - Mcelog was installed on node-overcloud-novacompute-0. 2018-04-02 21:19:02,828 - baro_tests.barometer - INFO - Checking if mcelog is enabled on node-overcloud-novacompute-1... 2018-04-02 21:19:02,878 - baro_tests.barometer - INFO - Mcelog will be enabled on node-overcloud-novacompute-1... 2018-04-02 21:19:03,242 - baro_tests.barometer - INFO - Mcelog was installed on node-overcloud-novacompute-1. 2018-04-02 21:19:15,062 - baro_tests.barometer - INFO - Creating OVS bridges on computes nodes 2018-04-02 21:19:15,314 - baro_tests.barometer - INFO - OVS Bridges created on compute nodes 2018-04-02 21:19:15,314 - baro_tests.barometer - INFO - ============= Testing Gnocchi and AODH plugins on nodes ============== 2018-04-02 21:19:53,968 - baro_tests.barometer - INFO - Gnocchi plugin available in compute node overcloud-novacompute-0 2018-04-02 21:20:05,819 - baro_tests.barometer - INFO - AODH plugin present in compute node overcloud-novacompute-0 2018-04-02 21:20:05,820 - baro_tests.barometer - INFO - === NODE overcloud-novacompute-0: Enabling Test Plug-in and Test case execution === 2018-04-02 21:21:17,839 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Hugepages Test case execution === 2018-04-02 21:21:42,052 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-04-02 21:22:19,241 - baro_tests.barometer - INFO - PASS 2018-04-02 21:22:19,241 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with Gnocchi PASSED on Hugepages. 2018-04-02 21:23:08,254 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Intel RDT Test case execution === 2018-04-02 21:23:42,294 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-04-02 21:24:19,194 - baro_tests.barometer - INFO - Plugin Interval is 10 2018-04-02 21:24:19,194 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-04-02 21:24:19,195 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on Intel RDT. 2018-04-02 21:25:07,332 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-04-02 21:25:30,044 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-04-02 21:26:07,026 - baro_tests.barometer - INFO - Plugin Interval is 1 2018-04-02 21:26:07,027 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-04-02 21:26:07,027 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on Mcelog. 2018-04-02 21:26:55,806 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-04-02 21:27:18,202 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-04-02 21:27:20,894 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on OVS events. 2018-04-02 21:28:06,750 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS stats Test case execution === 2018-04-02 21:28:30,786 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-04-02 21:29:07,809 - baro_tests.barometer - INFO - PASS 2018-04-02 21:29:07,809 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with Gnocchi PASSED on OVS stats. 2018-04-02 21:29:55,578 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-04-02 21:30:18,902 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-0 2018-04-02 21:30:37,817 - baro_tests.barometer - INFO - Data not updated after interval of 12 seconds 2018-04-02 21:30:37,818 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with AODH FAILED on Mcelog. 2018-04-02 21:31:25,899 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-04-02 21:31:49,756 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-0 2018-04-02 21:31:52,042 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with AODH FAILED on OVS events. 2018-04-02 21:31:52,042 - baro_tests.barometer - INFO - ======= Node overcloud-novacompute-0: Executing CSV Testcases ======== 2018-04-02 21:31:52,043 - baro_tests.barometer - INFO - Restarting collectd for CSV tests 2018-04-02 21:32:04,430 - baro_tests.barometer - INFO - Stopping collectd service... 2018-04-02 21:32:15,619 - baro_tests.barometer - INFO - Starting collectd service... 2018-04-02 21:32:25,910 - baro_tests.barometer - INFO - Sleeping for 10 seconds after collectd restart... 2018-04-02 21:33:21,571 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Hugepages Test case execution === 2018-04-02 21:33:44,894 - baro_tests.barometer - INFO - Getting CSV metrics of plugin hugepages on compute node 37826fed-fb0c-4a85-9b16-92befc83dd5f... 2018-04-02 21:33:44,894 - baro_tests.barometer - DEBUG - Interval: 10 2018-04-02 21:33:44,895 - baro_tests.barometer - DEBUG - Plugin subdirs: ['hugepages-mm-2048Kb', 'hugepages-node0-2048Kb'] 2018-04-02 21:33:44,895 - baro_tests.barometer - DEBUG - Plugin meter categories: ['vmpage_number-free', 'vmpage_number-used'] 2018-04-02 21:33:56,594 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-02 21:33:56,595 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1522704424.38 ... 2018-04-02 21:33:56,595 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:33:56,595 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1522704434.38 ... 2018-04-02 21:33:56,595 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:33:56,595 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1522704434.38 ... 2018-04-02 21:33:56,595 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:33:56,596 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1522704434.38 ... 2018-04-02 21:33:56,596 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:33:56,596 - baro_tests.barometer - INFO - OK 2018-04-02 21:33:56,596 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-04-02 21:34:08,601 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-02 21:34:20,036 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-02 21:34:20,037 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1522704424.379, 1522704434.379), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1522704434.379, 1522704444.379), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1522704434.379, 1522704444.379), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1522704434.379, 1522704444.379)] 2018-04-02 21:34:20,037 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1522704454.379, 1522704464.379), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1522704454.379, 1522704464.379), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1522704454.379, 1522704464.379), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1522704454.379, 1522704464.379)] 2018-04-02 21:34:20,038 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1522704424.38 - hugepages-mm-2048Kb vmpage_number-free 1522704454.38 ... 2018-04-02 21:34:20,038 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:34:20,038 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1522704434.38 - hugepages-mm-2048Kb vmpage_number-used 1522704454.38 ... 2018-04-02 21:34:20,038 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:34:20,038 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1522704434.38 - hugepages-node0-2048Kb vmpage_number-free 1522704454.38 ... 2018-04-02 21:34:20,038 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:34:20,038 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1522704434.38 - hugepages-node0-2048Kb vmpage_number-used 1522704454.38 ... 2018-04-02 21:34:20,038 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:34:20,038 - baro_tests.barometer - INFO - OK 2018-04-02 21:34:20,039 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Hugepages. 2018-04-02 21:35:05,640 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Intel RDT Test case execution === 2018-04-02 21:35:39,128 - baro_tests.barometer - INFO - Getting CSV metrics of plugin intel_rdt on compute node 37826fed-fb0c-4a85-9b16-92befc83dd5f... 2018-04-02 21:35:39,128 - baro_tests.barometer - DEBUG - Interval: 10 2018-04-02 21:35:39,128 - baro_tests.barometer - DEBUG - Plugin subdirs: ['intel_rdt-0-2'] 2018-04-02 21:35:39,128 - baro_tests.barometer - DEBUG - Plugin meter categories: ['bytes-llc', 'ipc'] 2018-04-02 21:35:50,381 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-02 21:35:50,382 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1522704544.38 ... 2018-04-02 21:35:50,382 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:35:50,383 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1522704544.38 ... 2018-04-02 21:35:50,383 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:35:50,383 - baro_tests.barometer - INFO - OK 2018-04-02 21:35:50,383 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-04-02 21:36:02,386 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-02 21:36:13,054 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-02 21:36:13,054 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1522704544.379, 1522704554.379), ('intel_rdt-0-2', 'ipc', 1522704544.379, 1522704554.379)] 2018-04-02 21:36:13,054 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1522704564.379, 1522704574.379), ('intel_rdt-0-2', 'ipc', 1522704564.379, 1522704574.379)] 2018-04-02 21:36:13,055 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1522704544.38 - intel_rdt-0-2 bytes-llc 1522704564.38 ... 2018-04-02 21:36:13,055 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:36:13,055 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1522704544.38 - intel_rdt-0-2 ipc 1522704564.38 ... 2018-04-02 21:36:13,055 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:36:13,055 - baro_tests.barometer - INFO - OK 2018-04-02 21:36:13,056 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Intel RDT. 2018-04-02 21:36:58,538 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-04-02 21:37:21,746 - baro_tests.barometer - INFO - Getting CSV metrics of plugin mcelog on compute node 37826fed-fb0c-4a85-9b16-92befc83dd5f... 2018-04-02 21:37:21,748 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-02 21:37:21,749 - baro_tests.barometer - DEBUG - Plugin subdirs: ['mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'mcelog-SOCKET_0_CHANNEL_any_DIMM_any'] 2018-04-02 21:37:21,749 - baro_tests.barometer - DEBUG - Plugin meter categories: ['errors-corrected_memory_errors', 'errors-uncorrected_memory_errors'] 2018-04-02 21:37:32,345 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-02 21:37:32,345 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1522704659.38 ... 2018-04-02 21:37:32,346 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:37:32,346 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1522704659.38 ... 2018-04-02 21:37:32,346 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:37:32,347 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1522704659.38 ... 2018-04-02 21:37:32,347 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:37:32,347 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1522704659.38 ... 2018-04-02 21:37:32,347 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:37:32,347 - baro_tests.barometer - INFO - OK 2018-04-02 21:37:32,347 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-02 21:37:35,350 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-02 21:37:47,207 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-02 21:37:47,208 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1522704659.38, 1522704660.38), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1522704659.38, 1522704660.38), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1522704659.38, 1522704660.38), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1522704659.38, 1522704660.38)] 2018-04-02 21:37:47,208 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1522704673.38, 1522704675.38), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1522704674.38, 1522704675.38), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1522704674.38, 1522704675.38), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1522704674.38, 1522704675.38)] 2018-04-02 21:37:47,208 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1522704659.38 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1522704673.38 ... 2018-04-02 21:37:47,209 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:37:47,209 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1522704659.38 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1522704674.38 ... 2018-04-02 21:37:47,209 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:37:47,209 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1522704659.38 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1522704674.38 ... 2018-04-02 21:37:47,209 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:37:47,210 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1522704659.38 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1522704674.38 ... 2018-04-02 21:37:47,210 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:37:47,210 - baro_tests.barometer - INFO - OK 2018-04-02 21:37:47,210 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Mcelog. 2018-04-02 21:38:33,899 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-04-02 21:38:55,956 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_events on compute node 37826fed-fb0c-4a85-9b16-92befc83dd5f... 2018-04-02 21:38:55,957 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-02 21:38:55,957 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_events-br0'] 2018-04-02 21:38:55,957 - baro_tests.barometer - DEBUG - Plugin meter categories: ['gauge-link_status'] 2018-04-02 21:39:06,956 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-02 21:39:06,957 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1522704753.38 ... 2018-04-02 21:39:06,957 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:39:06,957 - baro_tests.barometer - INFO - OK 2018-04-02 21:39:06,957 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-02 21:39:09,958 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-02 21:39:21,022 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-02 21:39:21,022 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1522704753.38, 1522704755.38)] 2018-04-02 21:39:21,023 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1522704768.38, 1522704769.38)] 2018-04-02 21:39:21,023 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1522704753.38 - ovs_events-br0 gauge-link_status 1522704768.38 ... 2018-04-02 21:39:21,023 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:39:21,023 - baro_tests.barometer - INFO - OK 2018-04-02 21:39:21,023 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on OVS events. 2018-04-02 21:40:06,136 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS stats Test case execution === 2018-04-02 21:40:27,511 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_stats on compute node 37826fed-fb0c-4a85-9b16-92befc83dd5f... 2018-04-02 21:40:27,511 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-02 21:40:27,512 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_stats-br0.br0'] 2018-04-02 21:40:27,512 - baro_tests.barometer - DEBUG - Plugin meter categories: ['if_dropped', 'if_errors', 'if_packets'] 2018-04-02 21:40:38,785 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-02 21:40:38,785 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1522704845.38 ... 2018-04-02 21:40:38,786 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:40:38,786 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1522704845.38 ... 2018-04-02 21:40:38,787 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:40:38,788 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1522704845.38 ... 2018-04-02 21:40:38,789 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:40:38,790 - baro_tests.barometer - INFO - OK 2018-04-02 21:40:38,791 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-02 21:40:41,795 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-02 21:40:52,723 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-02 21:40:52,723 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1522704845.38, 1522704846.38), ('ovs_stats-br0.br0', 'if_errors', 1522704845.38, 1522704846.38), ('ovs_stats-br0.br0', 'if_packets', 1522704845.38, 1522704846.38)] 2018-04-02 21:40:52,723 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1522704859.38, 1522704860.38), ('ovs_stats-br0.br0', 'if_errors', 1522704859.38, 1522704860.38), ('ovs_stats-br0.br0', 'if_packets', 1522704859.38, 1522704860.38)] 2018-04-02 21:40:52,724 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1522704845.38 - ovs_stats-br0.br0 if_dropped 1522704859.38 ... 2018-04-02 21:40:52,725 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:40:52,726 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1522704845.38 - ovs_stats-br0.br0 if_errors 1522704859.38 ... 2018-04-02 21:40:52,726 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:40:52,726 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1522704845.38 - ovs_stats-br0.br0 if_packets 1522704859.38 ... 2018-04-02 21:40:52,726 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:40:52,726 - baro_tests.barometer - INFO - OK 2018-04-02 21:40:52,727 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on OVS stats. 2018-04-02 21:41:04,135 - baro_tests.barometer - INFO - Gnocchi plugin available in compute node overcloud-novacompute-1 2018-04-02 21:41:15,155 - baro_tests.barometer - INFO - AODH plugin present in compute node overcloud-novacompute-1 2018-04-02 21:41:15,156 - baro_tests.barometer - INFO - === NODE overcloud-novacompute-1: Enabling Test Plug-in and Test case execution === 2018-04-02 21:42:24,208 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Hugepages Test case execution === 2018-04-02 21:42:47,311 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-04-02 21:43:24,541 - baro_tests.barometer - INFO - Plugin Interval is 10 2018-04-02 21:43:24,542 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-04-02 21:43:24,542 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on Hugepages. 2018-04-02 21:44:11,792 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Intel RDT Test case execution === 2018-04-02 21:44:46,316 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-04-02 21:45:23,630 - baro_tests.barometer - INFO - PASS 2018-04-02 21:45:23,630 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with Gnocchi PASSED on Intel RDT. 2018-04-02 21:46:07,612 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-04-02 21:46:28,796 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-04-02 21:47:06,324 - baro_tests.barometer - INFO - Plugin Interval is 1 2018-04-02 21:47:06,324 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-04-02 21:47:06,324 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on Mcelog. 2018-04-02 21:47:53,007 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-04-02 21:48:15,883 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-04-02 21:48:53,121 - baro_tests.barometer - INFO - Plugin Interval is 1 2018-04-02 21:48:53,121 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-04-02 21:48:53,122 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on OVS events. 2018-04-02 21:49:39,980 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS stats Test case execution === 2018-04-02 21:50:01,364 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-04-02 21:50:38,710 - baro_tests.barometer - INFO - PASS 2018-04-02 21:50:38,711 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with Gnocchi PASSED on OVS stats. 2018-04-02 21:51:24,789 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-04-02 21:51:46,981 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-0 2018-04-02 21:52:05,852 - baro_tests.barometer - INFO - Data not updated after interval of 12 seconds 2018-04-02 21:52:05,855 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with AODH FAILED on Mcelog. 2018-04-02 21:52:47,911 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-04-02 21:53:09,608 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-0 2018-04-02 21:53:11,975 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with AODH FAILED on OVS events. 2018-04-02 21:53:11,975 - baro_tests.barometer - INFO - ======= Node overcloud-novacompute-1: Executing CSV Testcases ======== 2018-04-02 21:53:11,975 - baro_tests.barometer - INFO - Restarting collectd for CSV tests 2018-04-02 21:53:21,562 - baro_tests.barometer - INFO - Stopping collectd service... 2018-04-02 21:53:32,450 - baro_tests.barometer - INFO - Starting collectd service... 2018-04-02 21:53:42,738 - baro_tests.barometer - INFO - Sleeping for 10 seconds after collectd restart... 2018-04-02 21:54:34,689 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Hugepages Test case execution === 2018-04-02 21:54:56,224 - baro_tests.barometer - INFO - Getting CSV metrics of plugin hugepages on compute node 9fab6820-47cc-453f-9b54-71fcc1717e59... 2018-04-02 21:54:56,224 - baro_tests.barometer - DEBUG - Interval: 10 2018-04-02 21:54:56,224 - baro_tests.barometer - DEBUG - Plugin subdirs: ['hugepages-mm-2048Kb', 'hugepages-node0-2048Kb'] 2018-04-02 21:54:56,225 - baro_tests.barometer - DEBUG - Plugin meter categories: ['vmpage_number-free', 'vmpage_number-used'] 2018-04-02 21:55:08,445 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-02 21:55:08,446 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1522705714.13 ... 2018-04-02 21:55:08,446 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:55:08,446 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1522705714.13 ... 2018-04-02 21:55:08,447 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:55:08,447 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1522705714.13 ... 2018-04-02 21:55:08,447 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:55:08,448 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1522705714.13 ... 2018-04-02 21:55:08,448 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:55:08,449 - baro_tests.barometer - INFO - OK 2018-04-02 21:55:08,449 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-04-02 21:55:20,460 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-02 21:55:31,905 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-02 21:55:31,905 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1522705714.128, 1522705724.128), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1522705714.128, 1522705724.128), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1522705714.128, 1522705724.128), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1522705714.128, 1522705724.128)] 2018-04-02 21:55:31,905 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1522705734.128, 1522705744.128), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1522705734.128, 1522705744.128), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1522705734.128, 1522705744.128), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1522705734.128, 1522705744.128)] 2018-04-02 21:55:31,906 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1522705714.13 - hugepages-mm-2048Kb vmpage_number-free 1522705734.13 ... 2018-04-02 21:55:31,906 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:55:31,906 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1522705714.13 - hugepages-mm-2048Kb vmpage_number-used 1522705734.13 ... 2018-04-02 21:55:31,906 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:55:31,907 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1522705714.13 - hugepages-node0-2048Kb vmpage_number-free 1522705734.13 ... 2018-04-02 21:55:31,907 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:55:31,907 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1522705714.13 - hugepages-node0-2048Kb vmpage_number-used 1522705734.13 ... 2018-04-02 21:55:31,907 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:55:31,907 - baro_tests.barometer - INFO - OK 2018-04-02 21:55:31,907 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Hugepages. 2018-04-02 21:56:16,849 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Intel RDT Test case execution === 2018-04-02 21:56:49,521 - baro_tests.barometer - INFO - Getting CSV metrics of plugin intel_rdt on compute node 9fab6820-47cc-453f-9b54-71fcc1717e59... 2018-04-02 21:56:49,521 - baro_tests.barometer - DEBUG - Interval: 10 2018-04-02 21:56:49,521 - baro_tests.barometer - DEBUG - Plugin subdirs: ['intel_rdt-0-2'] 2018-04-02 21:56:49,521 - baro_tests.barometer - DEBUG - Plugin meter categories: ['bytes-llc', 'ipc'] 2018-04-02 21:57:00,812 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-02 21:57:00,812 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1522705824.13 ... 2018-04-02 21:57:00,813 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:57:00,813 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1522705824.13 ... 2018-04-02 21:57:00,813 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:57:00,813 - baro_tests.barometer - INFO - OK 2018-04-02 21:57:00,813 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-04-02 21:57:12,825 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-02 21:57:24,509 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-02 21:57:24,509 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1522705824.128, 1522705834.128), ('intel_rdt-0-2', 'ipc', 1522705824.128, 1522705834.128)] 2018-04-02 21:57:24,509 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1522705854.128, 1522705864.128), ('intel_rdt-0-2', 'ipc', 1522705854.128, 1522705864.128)] 2018-04-02 21:57:24,510 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1522705824.13 - intel_rdt-0-2 bytes-llc 1522705854.13 ... 2018-04-02 21:57:24,510 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:57:24,510 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1522705824.13 - intel_rdt-0-2 ipc 1522705854.13 ... 2018-04-02 21:57:24,510 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:57:24,510 - baro_tests.barometer - INFO - OK 2018-04-02 21:57:24,510 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Intel RDT. 2018-04-02 21:58:07,980 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-04-02 21:58:29,211 - baro_tests.barometer - INFO - Getting CSV metrics of plugin mcelog on compute node 9fab6820-47cc-453f-9b54-71fcc1717e59... 2018-04-02 21:58:29,211 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-02 21:58:29,212 - baro_tests.barometer - DEBUG - Plugin subdirs: ['mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'mcelog-SOCKET_0_CHANNEL_any_DIMM_any'] 2018-04-02 21:58:29,212 - baro_tests.barometer - DEBUG - Plugin meter categories: ['errors-corrected_memory_errors', 'errors-uncorrected_memory_errors'] 2018-04-02 21:58:39,719 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-02 21:58:39,719 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1522705939.13 ... 2018-04-02 21:58:39,720 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:58:39,720 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1522705939.13 ... 2018-04-02 21:58:39,720 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:58:39,721 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1522705939.13 ... 2018-04-02 21:58:39,721 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:58:39,721 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1522705939.13 ... 2018-04-02 21:58:39,721 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:58:39,721 - baro_tests.barometer - INFO - OK 2018-04-02 21:58:39,721 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-02 21:58:42,724 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-02 21:58:54,609 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-02 21:58:54,610 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1522705939.129, 1522705940.129), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1522705939.129, 1522705940.129), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1522705939.129, 1522705940.129), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1522705939.129, 1522705941.129)] 2018-04-02 21:58:54,610 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1522705954.129, 1522705955.129), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1522705954.129, 1522705955.129), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1522705954.129, 1522705955.129), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1522705954.129, 1522705955.129)] 2018-04-02 21:58:54,611 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1522705939.13 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1522705954.13 ... 2018-04-02 21:58:54,611 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:58:54,611 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1522705939.13 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1522705954.13 ... 2018-04-02 21:58:54,612 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:58:54,615 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1522705939.13 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1522705954.13 ... 2018-04-02 21:58:54,616 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:58:54,619 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1522705939.13 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1522705954.13 ... 2018-04-02 21:58:54,620 - baro_tests.barometer - DEBUG - OK 2018-04-02 21:58:54,621 - baro_tests.barometer - INFO - OK 2018-04-02 21:58:54,622 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Mcelog. 2018-04-02 21:59:40,682 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-04-02 22:00:03,966 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_events on compute node 9fab6820-47cc-453f-9b54-71fcc1717e59... 2018-04-02 22:00:03,966 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-02 22:00:03,966 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_events-br0'] 2018-04-02 22:00:03,966 - baro_tests.barometer - DEBUG - Plugin meter categories: ['gauge-link_status'] 2018-04-02 22:00:15,241 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-02 22:00:15,242 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1522706035.13 ... 2018-04-02 22:00:15,242 - baro_tests.barometer - DEBUG - OK 2018-04-02 22:00:15,242 - baro_tests.barometer - INFO - OK 2018-04-02 22:00:15,243 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-02 22:00:18,246 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-02 22:00:30,147 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-02 22:00:30,147 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1522706035.129, 1522706036.129)] 2018-04-02 22:00:30,147 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1522706050.129, 1522706051.129)] 2018-04-02 22:00:30,148 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1522706035.13 - ovs_events-br0 gauge-link_status 1522706050.13 ... 2018-04-02 22:00:30,148 - baro_tests.barometer - DEBUG - OK 2018-04-02 22:00:30,148 - baro_tests.barometer - INFO - OK 2018-04-02 22:00:30,149 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on OVS events. 2018-04-02 22:01:17,009 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS stats Test case execution === 2018-04-02 22:01:39,131 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_stats on compute node 9fab6820-47cc-453f-9b54-71fcc1717e59... 2018-04-02 22:01:39,131 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-02 22:01:39,132 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_stats-br0.br0'] 2018-04-02 22:01:39,132 - baro_tests.barometer - DEBUG - Plugin meter categories: ['if_dropped', 'if_errors', 'if_packets'] 2018-04-02 22:01:49,982 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-02 22:01:49,983 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1522706129.13 ... 2018-04-02 22:01:49,983 - baro_tests.barometer - DEBUG - OK 2018-04-02 22:01:49,983 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1522706130.13 ... 2018-04-02 22:01:49,983 - baro_tests.barometer - DEBUG - OK 2018-04-02 22:01:49,983 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1522706130.13 ... 2018-04-02 22:01:49,983 - baro_tests.barometer - DEBUG - OK 2018-04-02 22:01:49,983 - baro_tests.barometer - INFO - OK 2018-04-02 22:01:49,984 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-02 22:01:52,987 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-02 22:02:04,777 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-02 22:02:04,778 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1522706129.129, 1522706131.129), ('ovs_stats-br0.br0', 'if_errors', 1522706130.129, 1522706131.129), ('ovs_stats-br0.br0', 'if_packets', 1522706130.129, 1522706131.129)] 2018-04-02 22:02:04,778 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1522706144.129, 1522706145.129), ('ovs_stats-br0.br0', 'if_errors', 1522706144.129, 1522706145.129), ('ovs_stats-br0.br0', 'if_packets', 1522706144.129, 1522706146.129)] 2018-04-02 22:02:04,778 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1522706129.13 - ovs_stats-br0.br0 if_dropped 1522706144.13 ... 2018-04-02 22:02:04,779 - baro_tests.barometer - DEBUG - OK 2018-04-02 22:02:04,779 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1522706130.13 - ovs_stats-br0.br0 if_errors 1522706144.13 ... 2018-04-02 22:02:04,779 - baro_tests.barometer - DEBUG - OK 2018-04-02 22:02:04,779 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1522706130.13 - ovs_stats-br0.br0 if_packets 1522706144.13 ... 2018-04-02 22:02:04,779 - baro_tests.barometer - DEBUG - OK 2018-04-02 22:02:04,779 - baro_tests.barometer - INFO - OK 2018-04-02 22:02:04,780 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on OVS stats. 2018-04-02 22:02:15,568 - baro_tests.barometer - INFO - Mcelog is deleted from all compute nodes 2018-04-02 22:02:15,569 - baro_tests.barometer - INFO - ====================================================================== 2018-04-02 22:02:15,570 - baro_tests.barometer - INFO - +----------------------------------+ 2018-04-02 22:02:15,571 - baro_tests.barometer - INFO - | OVERALL SUMMARY | 2018-04-02 22:02:15,573 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-02 22:02:15,575 - baro_tests.barometer - INFO - | Test | Node-0 | Node-1 | 2018-04-02 22:02:15,577 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-02 22:02:15,577 - baro_tests.barometer - INFO - | OUT:Gnocchi | PASS | PASS | 2018-04-02 22:02:15,577 - baro_tests.barometer - INFO - | IN:Hugepages | PASS | FAIL | 2018-04-02 22:02:15,577 - baro_tests.barometer - INFO - | IN:Intel RDT | FAIL | PASS | 2018-04-02 22:02:15,577 - baro_tests.barometer - INFO - | IN:Mcelog | FAIL | FAIL | 2018-04-02 22:02:15,578 - baro_tests.barometer - INFO - | IN:OVS events | FAIL | FAIL | 2018-04-02 22:02:15,578 - baro_tests.barometer - INFO - | IN:OVS stats | PASS | PASS | 2018-04-02 22:02:15,578 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-02 22:02:15,578 - baro_tests.barometer - INFO - | OUT:AODH | PASS | PASS | 2018-04-02 22:02:15,578 - baro_tests.barometer - INFO - | IN:Mcelog | FAIL | FAIL | 2018-04-02 22:02:15,578 - baro_tests.barometer - INFO - | IN:OVS events | SKIP | SKIP | 2018-04-02 22:02:15,578 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-02 22:02:15,579 - baro_tests.barometer - INFO - | OUT:CSV | PASS | PASS | 2018-04-02 22:02:15,579 - baro_tests.barometer - INFO - | IN:Hugepages | PASS | PASS | 2018-04-02 22:02:15,580 - baro_tests.barometer - INFO - | IN:Intel RDT | PASS | PASS | 2018-04-02 22:02:15,580 - baro_tests.barometer - INFO - | IN:Mcelog | PASS | PASS | 2018-04-02 22:02:15,580 - baro_tests.barometer - INFO - | IN:OVS events | PASS | PASS | 2018-04-02 22:02:15,580 - baro_tests.barometer - INFO - | IN:OVS stats | PASS | PASS | 2018-04-02 22:02:15,580 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-02 22:02:15,580 - baro_tests.barometer - INFO - ====================================================================== 2018-04-02 22:02:15,580 - baro_tests.barometer - ERROR - Some tests have failed or have not been executed 2018-04-02 22:02:15,580 - baro_tests.barometer - ERROR - Overall Result is Fail