2018-05-17 19:28:47,233 - baro_tests.barometer - INFO - =================== Starting barometer tests suite =================== 2018-05-17 19:28:47,233 - baro_tests.barometer - INFO - RSA key file /root/.ssh/id_rsa exists. 2018-05-17 19:28:51,043 - baro_tests.barometer - INFO - ==== Display of Control and Compute nodes available in the set up ==== 2018-05-17 19:28:51,044 - baro_tests.barometer - INFO - controllers: ['overcloud-controller-0: 192.30.9.5'] 2018-05-17 19:28:51,044 - baro_tests.barometer - INFO - computes: ['overcloud-novacompute-0: 192.30.9.7', 'overcloud-novacompute-1: 192.30.9.10'] 2018-05-17 19:28:51,044 - baro_tests.barometer - INFO - ========== Enabling mcelog and OVS bridges on compute nodes ========== 2018-05-17 19:29:03,031 - baro_tests.barometer - INFO - Checking if mcelog is enabled on node-overcloud-novacompute-0... 2018-05-17 19:29:03,100 - baro_tests.barometer - INFO - Mcelog will be enabled on node-overcloud-novacompute-0... 2018-05-17 19:29:03,624 - baro_tests.barometer - INFO - Mcelog was installed on node-overcloud-novacompute-0. 2018-05-17 19:29:03,655 - baro_tests.barometer - INFO - Checking if mcelog is enabled on node-overcloud-novacompute-1... 2018-05-17 19:29:03,724 - baro_tests.barometer - INFO - Mcelog will be enabled on node-overcloud-novacompute-1... 2018-05-17 19:29:04,226 - baro_tests.barometer - INFO - Mcelog was installed on node-overcloud-novacompute-1. 2018-05-17 19:29:16,129 - baro_tests.barometer - INFO - Creating OVS bridges on computes nodes 2018-05-17 19:29:16,452 - baro_tests.barometer - INFO - OVS Bridges created on compute nodes 2018-05-17 19:29:16,452 - baro_tests.barometer - INFO - ============= Testing Gnocchi and AODH plugins on nodes ============== 2018-05-17 19:29:54,936 - baro_tests.barometer - INFO - Gnocchi plugin available in compute node overcloud-novacompute-0 2018-05-17 19:30:07,296 - baro_tests.barometer - INFO - AODH plugin present in compute node overcloud-novacompute-0 2018-05-17 19:30:07,297 - baro_tests.barometer - INFO - === NODE overcloud-novacompute-0: Enabling Test Plug-in and Test case execution === 2018-05-17 19:31:19,015 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Hugepages Test case execution === 2018-05-17 19:31:43,702 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-05-17 19:32:20,928 - baro_tests.barometer - INFO - Plugin Interval is 10 2018-05-17 19:32:20,928 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-05-17 19:32:20,929 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on Hugepages. 2018-05-17 19:33:09,832 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Intel RDT Test case execution === 2018-05-17 19:33:45,574 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-05-17 19:34:22,822 - baro_tests.barometer - INFO - Plugin Interval is 10 2018-05-17 19:34:22,823 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-05-17 19:34:22,824 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on Intel RDT. 2018-05-17 19:35:07,450 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-05-17 19:35:30,773 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-05-17 19:36:07,912 - baro_tests.barometer - INFO - Plugin Interval is 1 2018-05-17 19:36:07,912 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-05-17 19:36:07,913 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on Mcelog. 2018-05-17 19:36:51,778 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-05-17 19:37:14,103 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-05-17 19:37:16,685 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on OVS events. 2018-05-17 19:38:01,614 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS stats Test case execution === 2018-05-17 19:38:25,569 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-05-17 19:39:02,748 - baro_tests.barometer - INFO - PASS 2018-05-17 19:39:02,749 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with Gnocchi PASSED on OVS stats. 2018-05-17 19:39:52,208 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-05-17 19:40:14,723 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-0 2018-05-17 19:40:33,799 - baro_tests.barometer - INFO - Data not updated after interval of 12 seconds 2018-05-17 19:40:33,800 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with AODH FAILED on Mcelog. 2018-05-17 19:41:21,299 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-05-17 19:41:44,594 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-0 2018-05-17 19:41:46,915 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with AODH FAILED on OVS events. 2018-05-17 19:41:46,916 - baro_tests.barometer - INFO - ======= Node overcloud-novacompute-0: Executing CSV Testcases ======== 2018-05-17 19:41:46,916 - baro_tests.barometer - INFO - Restarting collectd for CSV tests 2018-05-17 19:41:58,583 - baro_tests.barometer - INFO - Stopping collectd service... 2018-05-17 19:42:09,810 - baro_tests.barometer - INFO - Starting collectd service... 2018-05-17 19:42:20,116 - baro_tests.barometer - INFO - Sleeping for 10 seconds after collectd restart... 2018-05-17 19:43:16,927 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Hugepages Test case execution === 2018-05-17 19:43:39,663 - baro_tests.barometer - INFO - Getting CSV metrics of plugin hugepages on compute node 54f79356-1128-4998-ac0e-c26d2c0ee9cd... 2018-05-17 19:43:39,664 - baro_tests.barometer - DEBUG - Interval: 10 2018-05-17 19:43:39,664 - baro_tests.barometer - DEBUG - Plugin subdirs: ['hugepages-mm-2048Kb', 'hugepages-node0-2048Kb'] 2018-05-17 19:43:39,664 - baro_tests.barometer - DEBUG - Plugin meter categories: ['vmpage_number-free', 'vmpage_number-used'] 2018-05-17 19:43:52,144 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-17 19:43:52,144 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1526585776.92 ... 2018-05-17 19:43:52,145 - baro_tests.barometer - DEBUG - OK 2018-05-17 19:43:52,145 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1526585776.92 ... 2018-05-17 19:43:52,145 - baro_tests.barometer - DEBUG - OK 2018-05-17 19:43:52,145 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1526585776.92 ... 2018-05-17 19:43:52,146 - baro_tests.barometer - DEBUG - OK 2018-05-17 19:43:52,146 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1526585776.92 ... 2018-05-17 19:43:52,146 - baro_tests.barometer - DEBUG - OK 2018-05-17 19:43:52,146 - baro_tests.barometer - INFO - OK 2018-05-17 19:43:52,146 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-05-17 19:44:04,159 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-05-17 19:44:16,019 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-17 19:44:16,019 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1526585776.918, 1526585786.918), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1526585776.918, 1526585786.918), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1526585776.918, 1526585786.918), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1526585776.918, 1526585786.918)] 2018-05-17 19:44:16,020 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1526585796.918, 1526585806.918), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1526585796.918, 1526585806.918), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1526585796.918, 1526585806.918), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1526585796.918, 1526585806.918)] 2018-05-17 19:44:16,020 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1526585776.92 - hugepages-mm-2048Kb vmpage_number-free 1526585796.92 ... 2018-05-17 19:44:16,020 - baro_tests.barometer - DEBUG - OK 2018-05-17 19:44:16,021 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1526585776.92 - hugepages-mm-2048Kb vmpage_number-used 1526585796.92 ... 2018-05-17 19:44:16,021 - baro_tests.barometer - DEBUG - OK 2018-05-17 19:44:16,021 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1526585776.92 - hugepages-node0-2048Kb vmpage_number-free 1526585796.92 ... 2018-05-17 19:44:16,021 - baro_tests.barometer - DEBUG - OK 2018-05-17 19:44:16,021 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1526585776.92 - hugepages-node0-2048Kb vmpage_number-used 1526585796.92 ... 2018-05-17 19:44:16,022 - baro_tests.barometer - DEBUG - OK 2018-05-17 19:44:16,022 - baro_tests.barometer - INFO - OK 2018-05-17 19:44:16,022 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Hugepages. 2018-05-17 19:45:02,569 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Intel RDT Test case execution === 2018-05-17 19:45:36,383 - baro_tests.barometer - INFO - Getting CSV metrics of plugin intel_rdt on compute node 54f79356-1128-4998-ac0e-c26d2c0ee9cd... 2018-05-17 19:45:36,383 - baro_tests.barometer - DEBUG - Interval: 10 2018-05-17 19:45:36,384 - baro_tests.barometer - DEBUG - Plugin subdirs: ['intel_rdt-0-2'] 2018-05-17 19:45:36,384 - baro_tests.barometer - DEBUG - Plugin meter categories: ['bytes-llc', 'ipc'] 2018-05-17 19:45:48,019 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-17 19:45:48,019 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1526585886.92 ... 2018-05-17 19:45:48,019 - baro_tests.barometer - DEBUG - OK 2018-05-17 19:45:48,020 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1526585886.92 ... 2018-05-17 19:45:48,020 - baro_tests.barometer - DEBUG - OK 2018-05-17 19:45:48,020 - baro_tests.barometer - INFO - OK 2018-05-17 19:45:48,020 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-05-17 19:46:00,027 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-05-17 19:46:10,643 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-17 19:46:10,644 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1526585886.918, 1526585896.918), ('intel_rdt-0-2', 'ipc', 1526585886.918, 1526585896.918)] 2018-05-17 19:46:10,644 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1526585906.918, 1526585926.918), ('intel_rdt-0-2', 'ipc', 1526585916.918, 1526585926.918)] 2018-05-17 19:46:10,644 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1526585886.92 - intel_rdt-0-2 bytes-llc 1526585906.92 ... 2018-05-17 19:46:10,644 - baro_tests.barometer - DEBUG - OK 2018-05-17 19:46:10,644 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1526585886.92 - intel_rdt-0-2 ipc 1526585916.92 ... 2018-05-17 19:46:10,645 - baro_tests.barometer - DEBUG - OK 2018-05-17 19:46:10,645 - baro_tests.barometer - INFO - OK 2018-05-17 19:46:10,645 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Intel RDT. 2018-05-17 19:46:53,878 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-05-17 19:47:15,595 - baro_tests.barometer - INFO - Getting CSV metrics of plugin mcelog on compute node 54f79356-1128-4998-ac0e-c26d2c0ee9cd... 2018-05-17 19:47:15,595 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-17 19:47:15,596 - baro_tests.barometer - DEBUG - Plugin subdirs: ['mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'mcelog-SOCKET_0_CHANNEL_any_DIMM_any'] 2018-05-17 19:47:15,596 - baro_tests.barometer - DEBUG - Plugin meter categories: ['errors-corrected_memory_errors', 'errors-uncorrected_memory_errors'] 2018-05-17 19:47:28,287 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-17 19:47:28,288 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1526586002.92 ... 2018-05-17 19:47:28,288 - baro_tests.barometer - DEBUG - OK 2018-05-17 19:47:28,288 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1526586002.92 ... 2018-05-17 19:47:28,288 - baro_tests.barometer - DEBUG - OK 2018-05-17 19:47:28,289 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1526586003.92 ... 2018-05-17 19:47:28,289 - baro_tests.barometer - DEBUG - OK 2018-05-17 19:47:28,289 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1526586003.92 ... 2018-05-17 19:47:28,289 - baro_tests.barometer - DEBUG - OK 2018-05-17 19:47:28,289 - baro_tests.barometer - INFO - OK 2018-05-17 19:47:28,289 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-17 19:47:31,292 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-05-17 19:47:44,409 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-17 19:47:44,409 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1526586002.919, 1526586003.919), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1526586002.919, 1526586003.919), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1526586003.919, 1526586004.919), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1526586003.919, 1526586004.919)] 2018-05-17 19:47:44,409 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1526586018.919, 1526586019.919), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1526586018.919, 1526586020.919), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1526586019.919, 1526586020.919), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1526586019.919, 1526586020.919)] 2018-05-17 19:47:44,410 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1526586002.92 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1526586018.92 ... 2018-05-17 19:47:44,410 - baro_tests.barometer - DEBUG - OK 2018-05-17 19:47:44,410 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1526586002.92 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1526586018.92 ... 2018-05-17 19:47:44,411 - baro_tests.barometer - DEBUG - OK 2018-05-17 19:47:44,411 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1526586003.92 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1526586019.92 ... 2018-05-17 19:47:44,411 - baro_tests.barometer - DEBUG - OK 2018-05-17 19:47:44,411 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1526586003.92 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1526586019.92 ... 2018-05-17 19:47:44,411 - baro_tests.barometer - DEBUG - OK 2018-05-17 19:47:44,411 - baro_tests.barometer - INFO - OK 2018-05-17 19:47:44,412 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Mcelog. 2018-05-17 19:48:30,253 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-05-17 19:48:53,736 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_events on compute node 54f79356-1128-4998-ac0e-c26d2c0ee9cd... 2018-05-17 19:48:53,736 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-17 19:48:53,737 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_events-br0'] 2018-05-17 19:48:53,737 - baro_tests.barometer - DEBUG - Plugin meter categories: ['gauge-link_status'] 2018-05-17 19:49:04,818 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-17 19:49:04,819 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1526586099.92 ... 2018-05-17 19:49:04,819 - baro_tests.barometer - DEBUG - OK 2018-05-17 19:49:04,819 - baro_tests.barometer - INFO - OK 2018-05-17 19:49:04,820 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-17 19:49:07,823 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-05-17 19:49:19,614 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-17 19:49:19,615 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1526586099.919, 1526586100.919)] 2018-05-17 19:49:19,615 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1526586114.919, 1526586115.92)] 2018-05-17 19:49:19,616 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1526586099.92 - ovs_events-br0 gauge-link_status 1526586114.92 ... 2018-05-17 19:49:19,616 - baro_tests.barometer - DEBUG - OK 2018-05-17 19:49:19,616 - baro_tests.barometer - INFO - OK 2018-05-17 19:49:19,616 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on OVS events. 2018-05-17 19:50:08,109 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS stats Test case execution === 2018-05-17 19:50:31,586 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_stats on compute node 54f79356-1128-4998-ac0e-c26d2c0ee9cd... 2018-05-17 19:50:31,586 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-17 19:50:31,586 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_stats-br0.br0'] 2018-05-17 19:50:31,587 - baro_tests.barometer - DEBUG - Plugin meter categories: ['if_dropped', 'if_errors', 'if_packets'] 2018-05-17 19:50:43,941 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-17 19:50:43,941 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1526586198.92 ... 2018-05-17 19:50:43,942 - baro_tests.barometer - DEBUG - OK 2018-05-17 19:50:43,942 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1526586198.92 ... 2018-05-17 19:50:43,943 - baro_tests.barometer - DEBUG - OK 2018-05-17 19:50:43,943 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1526586198.92 ... 2018-05-17 19:50:43,943 - baro_tests.barometer - DEBUG - OK 2018-05-17 19:50:43,944 - baro_tests.barometer - INFO - OK 2018-05-17 19:50:43,944 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-17 19:50:46,948 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-05-17 19:50:58,515 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-17 19:50:58,515 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1526586198.919, 1526586199.919), ('ovs_stats-br0.br0', 'if_errors', 1526586198.919, 1526586199.919), ('ovs_stats-br0.br0', 'if_packets', 1526586198.919, 1526586199.919)] 2018-05-17 19:50:58,516 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1526586213.919, 1526586214.919), ('ovs_stats-br0.br0', 'if_errors', 1526586213.919, 1526586214.919), ('ovs_stats-br0.br0', 'if_packets', 1526586213.919, 1526586214.919)] 2018-05-17 19:50:58,517 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1526586198.92 - ovs_stats-br0.br0 if_dropped 1526586213.92 ... 2018-05-17 19:50:58,517 - baro_tests.barometer - DEBUG - OK 2018-05-17 19:50:58,517 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1526586198.92 - ovs_stats-br0.br0 if_errors 1526586213.92 ... 2018-05-17 19:50:58,518 - baro_tests.barometer - DEBUG - OK 2018-05-17 19:50:58,518 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1526586198.92 - ovs_stats-br0.br0 if_packets 1526586213.92 ... 2018-05-17 19:50:58,520 - baro_tests.barometer - DEBUG - OK 2018-05-17 19:50:58,521 - baro_tests.barometer - INFO - OK 2018-05-17 19:50:58,521 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on OVS stats. 2018-05-17 19:51:09,143 - baro_tests.barometer - INFO - Gnocchi plugin available in compute node overcloud-novacompute-1 2018-05-17 19:51:20,194 - baro_tests.barometer - INFO - AODH plugin present in compute node overcloud-novacompute-1 2018-05-17 19:51:20,195 - baro_tests.barometer - INFO - === NODE overcloud-novacompute-1: Enabling Test Plug-in and Test case execution === 2018-05-17 19:52:30,525 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Hugepages Test case execution === 2018-05-17 19:52:53,278 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-05-17 19:53:30,493 - baro_tests.barometer - INFO - Plugin Interval is 10 2018-05-17 19:53:30,493 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-05-17 19:53:30,494 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on Hugepages. 2018-05-17 19:54:17,128 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Intel RDT Test case execution === 2018-05-17 19:54:52,072 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-05-17 19:55:29,172 - baro_tests.barometer - INFO - Plugin Interval is 10 2018-05-17 19:55:29,172 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-05-17 19:55:29,173 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on Intel RDT. 2018-05-17 19:56:16,706 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-05-17 19:56:39,952 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-05-17 19:56:42,591 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on Mcelog. 2018-05-17 19:57:26,446 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-05-17 19:57:48,777 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-05-17 19:57:51,562 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on OVS events. 2018-05-17 19:58:39,550 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS stats Test case execution === 2018-05-17 19:59:02,905 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-05-17 19:59:40,322 - baro_tests.barometer - INFO - Plugin Interval is 1 2018-05-17 19:59:40,323 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-05-17 19:59:40,323 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on OVS stats. 2018-05-17 20:00:25,836 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-05-17 20:00:48,627 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-0 2018-05-17 20:01:07,672 - baro_tests.barometer - INFO - Data not updated after interval of 12 seconds 2018-05-17 20:01:07,673 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with AODH FAILED on Mcelog. 2018-05-17 20:01:51,997 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-05-17 20:02:14,820 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-0 2018-05-17 20:02:17,177 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with AODH FAILED on OVS events. 2018-05-17 20:02:17,178 - baro_tests.barometer - INFO - ======= Node overcloud-novacompute-1: Executing CSV Testcases ======== 2018-05-17 20:02:17,178 - baro_tests.barometer - INFO - Restarting collectd for CSV tests 2018-05-17 20:02:28,689 - baro_tests.barometer - INFO - Stopping collectd service... 2018-05-17 20:02:40,323 - baro_tests.barometer - INFO - Starting collectd service... 2018-05-17 20:02:50,645 - baro_tests.barometer - INFO - Sleeping for 10 seconds after collectd restart... 2018-05-17 20:03:48,775 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Hugepages Test case execution === 2018-05-17 20:04:11,648 - baro_tests.barometer - INFO - Getting CSV metrics of plugin hugepages on compute node 0ada9e3f-6739-490c-a2d6-e317e1b83dad... 2018-05-17 20:04:11,648 - baro_tests.barometer - DEBUG - Interval: 10 2018-05-17 20:04:11,649 - baro_tests.barometer - DEBUG - Plugin subdirs: ['hugepages-mm-2048Kb', 'hugepages-node0-2048Kb'] 2018-05-17 20:04:11,650 - baro_tests.barometer - DEBUG - Plugin meter categories: ['vmpage_number-free', 'vmpage_number-used'] 2018-05-17 20:04:22,855 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-17 20:04:22,856 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1526587014.21 ... 2018-05-17 20:04:22,856 - baro_tests.barometer - DEBUG - OK 2018-05-17 20:04:22,856 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1526587014.21 ... 2018-05-17 20:04:22,856 - baro_tests.barometer - DEBUG - OK 2018-05-17 20:04:22,857 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1526587014.21 ... 2018-05-17 20:04:22,857 - baro_tests.barometer - DEBUG - OK 2018-05-17 20:04:22,858 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1526587014.21 ... 2018-05-17 20:04:22,858 - baro_tests.barometer - DEBUG - OK 2018-05-17 20:04:22,858 - baro_tests.barometer - INFO - OK 2018-05-17 20:04:22,858 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-05-17 20:04:34,870 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-05-17 20:04:46,701 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-17 20:04:46,701 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1526587014.209, 1526587024.209), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1526587014.209, 1526587024.209), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1526587014.209, 1526587024.209), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1526587014.209, 1526587024.209)] 2018-05-17 20:04:46,701 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1526587034.209, 1526587044.209), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1526587034.209, 1526587044.209), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1526587034.209, 1526587044.209), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1526587034.209, 1526587044.209)] 2018-05-17 20:04:46,702 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1526587014.21 - hugepages-mm-2048Kb vmpage_number-free 1526587034.21 ... 2018-05-17 20:04:46,702 - baro_tests.barometer - DEBUG - OK 2018-05-17 20:04:46,702 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1526587014.21 - hugepages-mm-2048Kb vmpage_number-used 1526587034.21 ... 2018-05-17 20:04:46,702 - baro_tests.barometer - DEBUG - OK 2018-05-17 20:04:46,703 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1526587014.21 - hugepages-node0-2048Kb vmpage_number-free 1526587034.21 ... 2018-05-17 20:04:46,703 - baro_tests.barometer - DEBUG - OK 2018-05-17 20:04:46,703 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1526587014.21 - hugepages-node0-2048Kb vmpage_number-used 1526587034.21 ... 2018-05-17 20:04:46,704 - baro_tests.barometer - DEBUG - OK 2018-05-17 20:04:46,704 - baro_tests.barometer - INFO - OK 2018-05-17 20:04:46,704 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Hugepages. 2018-05-17 20:05:30,295 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Intel RDT Test case execution === 2018-05-17 20:06:03,149 - baro_tests.barometer - INFO - Getting CSV metrics of plugin intel_rdt on compute node 0ada9e3f-6739-490c-a2d6-e317e1b83dad... 2018-05-17 20:06:03,149 - baro_tests.barometer - DEBUG - Interval: 10 2018-05-17 20:06:03,150 - baro_tests.barometer - DEBUG - Plugin subdirs: ['intel_rdt-0-2'] 2018-05-17 20:06:03,150 - baro_tests.barometer - DEBUG - Plugin meter categories: ['bytes-llc', 'ipc'] 2018-05-17 20:06:14,873 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-17 20:06:14,875 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1526587124.21 ... 2018-05-17 20:06:14,877 - baro_tests.barometer - DEBUG - OK 2018-05-17 20:06:14,880 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1526587124.21 ... 2018-05-17 20:06:14,883 - baro_tests.barometer - DEBUG - OK 2018-05-17 20:06:14,883 - baro_tests.barometer - INFO - OK 2018-05-17 20:06:14,884 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-05-17 20:06:26,897 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-05-17 20:06:38,425 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-17 20:06:38,426 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1526587124.209, 1526587134.209), ('intel_rdt-0-2', 'ipc', 1526587124.209, 1526587134.209)] 2018-05-17 20:06:38,427 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1526587144.209, 1526587154.209), ('intel_rdt-0-2', 'ipc', 1526587144.209, 1526587154.209)] 2018-05-17 20:06:38,428 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1526587124.21 - intel_rdt-0-2 bytes-llc 1526587144.21 ... 2018-05-17 20:06:38,429 - baro_tests.barometer - DEBUG - OK 2018-05-17 20:06:38,430 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1526587124.21 - intel_rdt-0-2 ipc 1526587144.21 ... 2018-05-17 20:06:38,430 - baro_tests.barometer - DEBUG - OK 2018-05-17 20:06:38,431 - baro_tests.barometer - INFO - OK 2018-05-17 20:06:38,437 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Intel RDT. 2018-05-17 20:07:24,500 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-05-17 20:07:48,241 - baro_tests.barometer - INFO - Getting CSV metrics of plugin mcelog on compute node 0ada9e3f-6739-490c-a2d6-e317e1b83dad... 2018-05-17 20:07:48,241 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-17 20:07:48,242 - baro_tests.barometer - DEBUG - Plugin subdirs: ['mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'mcelog-SOCKET_0_CHANNEL_any_DIMM_any'] 2018-05-17 20:07:48,243 - baro_tests.barometer - DEBUG - Plugin meter categories: ['errors-corrected_memory_errors', 'errors-uncorrected_memory_errors'] 2018-05-17 20:08:00,096 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-17 20:08:00,096 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1526587242.21 ... 2018-05-17 20:08:00,097 - baro_tests.barometer - DEBUG - OK 2018-05-17 20:08:00,097 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1526587242.21 ... 2018-05-17 20:08:00,097 - baro_tests.barometer - DEBUG - OK 2018-05-17 20:08:00,097 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1526587242.21 ... 2018-05-17 20:08:00,097 - baro_tests.barometer - DEBUG - OK 2018-05-17 20:08:00,098 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1526587242.21 ... 2018-05-17 20:08:00,098 - baro_tests.barometer - DEBUG - OK 2018-05-17 20:08:00,098 - baro_tests.barometer - INFO - OK 2018-05-17 20:08:00,098 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-17 20:08:03,099 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-05-17 20:08:15,226 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-17 20:08:15,226 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1526587242.21, 1526587243.21), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1526587242.21, 1526587243.21), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1526587242.21, 1526587243.21), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1526587242.21, 1526587243.21)] 2018-05-17 20:08:15,226 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1526587257.21, 1526587258.21), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1526587257.21, 1526587258.21), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1526587257.21, 1526587258.21), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1526587257.21, 1526587258.21)] 2018-05-17 20:08:15,227 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1526587242.21 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1526587257.21 ... 2018-05-17 20:08:15,227 - baro_tests.barometer - DEBUG - OK 2018-05-17 20:08:15,227 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1526587242.21 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1526587257.21 ... 2018-05-17 20:08:15,227 - baro_tests.barometer - DEBUG - OK 2018-05-17 20:08:15,228 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1526587242.21 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1526587257.21 ... 2018-05-17 20:08:15,228 - baro_tests.barometer - DEBUG - OK 2018-05-17 20:08:15,228 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1526587242.21 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1526587257.21 ... 2018-05-17 20:08:15,228 - baro_tests.barometer - DEBUG - OK 2018-05-17 20:08:15,228 - baro_tests.barometer - INFO - OK 2018-05-17 20:08:15,228 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Mcelog. 2018-05-17 20:08:59,671 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-05-17 20:09:23,203 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_events on compute node 0ada9e3f-6739-490c-a2d6-e317e1b83dad... 2018-05-17 20:09:23,208 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-17 20:09:23,209 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_events-br0'] 2018-05-17 20:09:23,211 - baro_tests.barometer - DEBUG - Plugin meter categories: ['gauge-link_status'] 2018-05-17 20:09:34,335 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-17 20:09:34,335 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1526587336.21 ... 2018-05-17 20:09:34,335 - baro_tests.barometer - DEBUG - OK 2018-05-17 20:09:34,337 - baro_tests.barometer - INFO - OK 2018-05-17 20:09:34,338 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-17 20:09:37,341 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-05-17 20:09:48,673 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-17 20:09:48,673 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1526587336.21, 1526587337.21)] 2018-05-17 20:09:48,674 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1526587350.21, 1526587352.21)] 2018-05-17 20:09:48,674 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1526587336.21 - ovs_events-br0 gauge-link_status 1526587350.21 ... 2018-05-17 20:09:48,675 - baro_tests.barometer - DEBUG - OK 2018-05-17 20:09:48,675 - baro_tests.barometer - INFO - OK 2018-05-17 20:09:48,675 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on OVS events. 2018-05-17 20:10:34,362 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS stats Test case execution === 2018-05-17 20:10:58,344 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_stats on compute node 0ada9e3f-6739-490c-a2d6-e317e1b83dad... 2018-05-17 20:10:58,344 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-17 20:10:58,345 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_stats-br0.br0'] 2018-05-17 20:10:58,345 - baro_tests.barometer - DEBUG - Plugin meter categories: ['if_dropped', 'if_errors', 'if_packets'] 2018-05-17 20:11:09,915 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-17 20:11:09,916 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1526587431.21 ... 2018-05-17 20:11:09,918 - baro_tests.barometer - DEBUG - OK 2018-05-17 20:11:09,918 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1526587432.21 ... 2018-05-17 20:11:09,919 - baro_tests.barometer - DEBUG - OK 2018-05-17 20:11:09,919 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1526587432.21 ... 2018-05-17 20:11:09,919 - baro_tests.barometer - DEBUG - OK 2018-05-17 20:11:09,919 - baro_tests.barometer - INFO - OK 2018-05-17 20:11:09,920 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-17 20:11:12,923 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-05-17 20:11:24,355 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-17 20:11:24,356 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1526587431.21, 1526587432.21), ('ovs_stats-br0.br0', 'if_errors', 1526587432.21, 1526587433.21), ('ovs_stats-br0.br0', 'if_packets', 1526587432.21, 1526587433.21)] 2018-05-17 20:11:24,356 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1526587446.21, 1526587447.21), ('ovs_stats-br0.br0', 'if_errors', 1526587446.21, 1526587447.21), ('ovs_stats-br0.br0', 'if_packets', 1526587446.21, 1526587447.21)] 2018-05-17 20:11:24,358 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1526587431.21 - ovs_stats-br0.br0 if_dropped 1526587446.21 ... 2018-05-17 20:11:24,358 - baro_tests.barometer - DEBUG - OK 2018-05-17 20:11:24,360 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1526587432.21 - ovs_stats-br0.br0 if_errors 1526587446.21 ... 2018-05-17 20:11:24,360 - baro_tests.barometer - DEBUG - OK 2018-05-17 20:11:24,360 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1526587432.21 - ovs_stats-br0.br0 if_packets 1526587446.21 ... 2018-05-17 20:11:24,361 - baro_tests.barometer - DEBUG - OK 2018-05-17 20:11:24,361 - baro_tests.barometer - INFO - OK 2018-05-17 20:11:24,361 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on OVS stats. 2018-05-17 20:11:35,260 - baro_tests.barometer - INFO - Mcelog is deleted from all compute nodes 2018-05-17 20:11:35,261 - baro_tests.barometer - INFO - ====================================================================== 2018-05-17 20:11:35,261 - baro_tests.barometer - INFO - +----------------------------------+ 2018-05-17 20:11:35,262 - baro_tests.barometer - INFO - | OVERALL SUMMARY | 2018-05-17 20:11:35,264 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-05-17 20:11:35,264 - baro_tests.barometer - INFO - | Test | Node-0 | Node-1 | 2018-05-17 20:11:35,265 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-05-17 20:11:35,265 - baro_tests.barometer - INFO - | OUT:Gnocchi | PASS | PASS | 2018-05-17 20:11:35,266 - baro_tests.barometer - INFO - | IN:Hugepages | FAIL | FAIL | 2018-05-17 20:11:35,266 - baro_tests.barometer - INFO - | IN:Intel RDT | FAIL | FAIL | 2018-05-17 20:11:35,266 - baro_tests.barometer - INFO - | IN:Mcelog | FAIL | FAIL | 2018-05-17 20:11:35,266 - baro_tests.barometer - INFO - | IN:OVS events | FAIL | FAIL | 2018-05-17 20:11:35,267 - baro_tests.barometer - INFO - | IN:OVS stats | PASS | FAIL | 2018-05-17 20:11:35,268 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-05-17 20:11:35,269 - baro_tests.barometer - INFO - | OUT:AODH | PASS | PASS | 2018-05-17 20:11:35,269 - baro_tests.barometer - INFO - | IN:Mcelog | FAIL | FAIL | 2018-05-17 20:11:35,270 - baro_tests.barometer - INFO - | IN:OVS events | SKIP | SKIP | 2018-05-17 20:11:35,270 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-05-17 20:11:35,270 - baro_tests.barometer - INFO - | OUT:CSV | PASS | PASS | 2018-05-17 20:11:35,270 - baro_tests.barometer - INFO - | IN:Hugepages | PASS | PASS | 2018-05-17 20:11:35,270 - baro_tests.barometer - INFO - | IN:Intel RDT | PASS | PASS | 2018-05-17 20:11:35,270 - baro_tests.barometer - INFO - | IN:Mcelog | PASS | PASS | 2018-05-17 20:11:35,270 - baro_tests.barometer - INFO - | IN:OVS events | PASS | PASS | 2018-05-17 20:11:35,270 - baro_tests.barometer - INFO - | IN:OVS stats | PASS | PASS | 2018-05-17 20:11:35,270 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-05-17 20:11:35,271 - baro_tests.barometer - INFO - ====================================================================== 2018-05-17 20:11:35,271 - baro_tests.barometer - ERROR - Some tests have failed or have not been executed 2018-05-17 20:11:35,271 - baro_tests.barometer - ERROR - Overall Result is Fail