2018-04-24 04:19:44,214 - baro_tests.barometer - INFO - =================== Starting barometer tests suite =================== 2018-04-24 04:19:44,214 - baro_tests.barometer - INFO - RSA key file /root/.ssh/id_rsa exists. 2018-04-24 04:19:48,422 - baro_tests.barometer - INFO - ==== Display of Control and Compute nodes available in the set up ==== 2018-04-24 04:19:48,423 - baro_tests.barometer - INFO - controllers: ['overcloud-controller-0: 192.30.9.8', 'overcloud-controller-1: 192.30.9.7', 'overcloud-controller-2: 192.30.9.6'] 2018-04-24 04:19:48,423 - baro_tests.barometer - INFO - computes: ['overcloud-novacompute-0: 192.30.9.9', 'overcloud-novacompute-1: 192.30.9.4'] 2018-04-24 04:19:48,423 - baro_tests.barometer - INFO - ========== Enabling mcelog and OVS bridges on compute nodes ========== 2018-04-24 04:20:03,346 - baro_tests.barometer - INFO - Checking if mcelog is enabled on node-overcloud-novacompute-0... 2018-04-24 04:20:03,396 - baro_tests.barometer - INFO - Mcelog will be enabled on node-overcloud-novacompute-0... 2018-04-24 04:20:03,892 - baro_tests.barometer - INFO - Mcelog was installed on node-overcloud-novacompute-0. 2018-04-24 04:20:03,904 - baro_tests.barometer - INFO - Checking if mcelog is enabled on node-overcloud-novacompute-1... 2018-04-24 04:20:03,955 - baro_tests.barometer - INFO - Mcelog will be enabled on node-overcloud-novacompute-1... 2018-04-24 04:20:04,351 - baro_tests.barometer - INFO - Mcelog was installed on node-overcloud-novacompute-1. 2018-04-24 04:20:18,743 - baro_tests.barometer - INFO - Creating OVS bridges on computes nodes 2018-04-24 04:20:18,992 - baro_tests.barometer - INFO - OVS Bridges created on compute nodes 2018-04-24 04:20:18,993 - baro_tests.barometer - INFO - ============= Testing Gnocchi and AODH plugins on nodes ============== 2018-04-24 04:22:10,325 - baro_tests.barometer - INFO - Gnocchi plugin available in compute node overcloud-novacompute-0 2018-04-24 04:22:23,338 - baro_tests.barometer - INFO - AODH plugin present in compute node overcloud-novacompute-0 2018-04-24 04:22:23,338 - baro_tests.barometer - INFO - === NODE overcloud-novacompute-0: Enabling Test Plug-in and Test case execution === 2018-04-24 04:23:42,314 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Hugepages Test case execution === 2018-04-24 04:24:07,828 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-24 04:24:45,205 - baro_tests.barometer - INFO - PASS 2018-04-24 04:24:45,205 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with Gnocchi PASSED on Hugepages. 2018-04-24 04:25:37,356 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Intel RDT Test case execution === 2018-04-24 04:26:18,115 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-24 04:26:55,363 - baro_tests.barometer - INFO - PASS 2018-04-24 04:26:55,363 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with Gnocchi PASSED on Intel RDT. 2018-04-24 04:27:47,606 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-04-24 04:28:13,369 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-24 04:28:50,651 - baro_tests.barometer - INFO - Plugin Interval is 1 2018-04-24 04:28:50,652 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-04-24 04:28:50,652 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on Mcelog. 2018-04-24 04:29:44,405 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-04-24 04:30:09,282 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-24 04:30:46,461 - baro_tests.barometer - INFO - PASS 2018-04-24 04:30:46,462 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with Gnocchi PASSED on OVS events. 2018-04-24 04:31:39,475 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS stats Test case execution === 2018-04-24 04:32:05,163 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-24 04:32:42,265 - baro_tests.barometer - INFO - PASS 2018-04-24 04:32:42,266 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with Gnocchi PASSED on OVS stats. 2018-04-24 04:33:33,383 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-04-24 04:33:58,202 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-04-24 04:34:17,332 - baro_tests.barometer - INFO - Data not updated after interval of 12 seconds 2018-04-24 04:34:17,334 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with AODH FAILED on Mcelog. 2018-04-24 04:35:08,772 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-04-24 04:35:34,807 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-04-24 04:35:37,165 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-0 2018-04-24 04:35:39,554 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-1 2018-04-24 04:35:42,522 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with AODH FAILED on OVS events. 2018-04-24 04:35:42,523 - baro_tests.barometer - INFO - ======= Node overcloud-novacompute-0: Executing CSV Testcases ======== 2018-04-24 04:35:42,523 - baro_tests.barometer - INFO - Restarting collectd for CSV tests 2018-04-24 04:35:54,826 - baro_tests.barometer - INFO - Stopping collectd service... 2018-04-24 04:36:06,606 - baro_tests.barometer - INFO - Starting collectd service... 2018-04-24 04:36:16,910 - baro_tests.barometer - INFO - Sleeping for 10 seconds after collectd restart... 2018-04-24 04:37:19,676 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Hugepages Test case execution === 2018-04-24 04:37:45,885 - baro_tests.barometer - INFO - Getting CSV metrics of plugin hugepages on compute node 9279f0db-780f-4d8f-9d67-c1484622c9b8... 2018-04-24 04:37:45,885 - baro_tests.barometer - DEBUG - Interval: 10 2018-04-24 04:37:45,885 - baro_tests.barometer - DEBUG - Plugin subdirs: ['hugepages-mm-2048Kb', 'hugepages-node0-2048Kb'] 2018-04-24 04:37:45,886 - baro_tests.barometer - DEBUG - Plugin meter categories: ['vmpage_number-free', 'vmpage_number-used'] 2018-04-24 04:37:59,657 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-24 04:37:59,658 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1524544259.32 ... 2018-04-24 04:37:59,658 - baro_tests.barometer - DEBUG - OK 2018-04-24 04:37:59,658 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1524544259.32 ... 2018-04-24 04:37:59,659 - baro_tests.barometer - DEBUG - OK 2018-04-24 04:37:59,659 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1524544259.32 ... 2018-04-24 04:37:59,659 - baro_tests.barometer - DEBUG - OK 2018-04-24 04:37:59,659 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1524544259.32 ... 2018-04-24 04:37:59,659 - baro_tests.barometer - DEBUG - OK 2018-04-24 04:37:59,659 - baro_tests.barometer - INFO - OK 2018-04-24 04:37:59,660 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-04-24 04:38:11,670 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-24 04:38:25,356 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-24 04:38:25,357 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1524544259.32, 1524544269.32), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1524544259.32, 1524544269.32), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1524544259.32, 1524544269.32), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1524544259.32, 1524544269.32)] 2018-04-24 04:38:25,357 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1524544279.32, 1524544289.32), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1524544279.32, 1524544289.32), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1524544279.32, 1524544289.32), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1524544279.32, 1524544289.32)] 2018-04-24 04:38:25,358 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1524544259.32 - hugepages-mm-2048Kb vmpage_number-free 1524544279.32 ... 2018-04-24 04:38:25,358 - baro_tests.barometer - DEBUG - OK 2018-04-24 04:38:25,358 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1524544259.32 - hugepages-mm-2048Kb vmpage_number-used 1524544279.32 ... 2018-04-24 04:38:25,358 - baro_tests.barometer - DEBUG - OK 2018-04-24 04:38:25,359 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1524544259.32 - hugepages-node0-2048Kb vmpage_number-free 1524544279.32 ... 2018-04-24 04:38:25,359 - baro_tests.barometer - DEBUG - OK 2018-04-24 04:38:25,359 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1524544259.32 - hugepages-node0-2048Kb vmpage_number-used 1524544279.32 ... 2018-04-24 04:38:25,359 - baro_tests.barometer - DEBUG - OK 2018-04-24 04:38:25,359 - baro_tests.barometer - INFO - OK 2018-04-24 04:38:25,359 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Hugepages. 2018-04-24 04:39:16,523 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Intel RDT Test case execution === 2018-04-24 04:39:54,897 - baro_tests.barometer - INFO - Getting CSV metrics of plugin intel_rdt on compute node 9279f0db-780f-4d8f-9d67-c1484622c9b8... 2018-04-24 04:39:54,897 - baro_tests.barometer - DEBUG - Interval: 10 2018-04-24 04:39:54,898 - baro_tests.barometer - DEBUG - Plugin subdirs: ['intel_rdt-0-2'] 2018-04-24 04:39:54,898 - baro_tests.barometer - DEBUG - Plugin meter categories: ['bytes-llc', 'ipc'] 2018-04-24 04:40:07,903 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-24 04:40:07,904 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1524544389.32 ... 2018-04-24 04:40:07,904 - baro_tests.barometer - DEBUG - OK 2018-04-24 04:40:07,904 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1524544389.32 ... 2018-04-24 04:40:07,904 - baro_tests.barometer - DEBUG - OK 2018-04-24 04:40:07,905 - baro_tests.barometer - INFO - OK 2018-04-24 04:40:07,905 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-04-24 04:40:19,917 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-24 04:40:33,824 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-24 04:40:33,826 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1524544389.32, 1524544399.32), ('intel_rdt-0-2', 'ipc', 1524544389.32, 1524544399.32)] 2018-04-24 04:40:33,828 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1524544409.32, 1524544419.32), ('intel_rdt-0-2', 'ipc', 1524544409.32, 1524544419.32)] 2018-04-24 04:40:33,829 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1524544389.32 - intel_rdt-0-2 bytes-llc 1524544409.32 ... 2018-04-24 04:40:33,830 - baro_tests.barometer - DEBUG - OK 2018-04-24 04:40:33,832 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1524544389.32 - intel_rdt-0-2 ipc 1524544409.32 ... 2018-04-24 04:40:33,833 - baro_tests.barometer - DEBUG - OK 2018-04-24 04:40:33,833 - baro_tests.barometer - INFO - OK 2018-04-24 04:40:33,833 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Intel RDT. 2018-04-24 04:41:26,227 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-04-24 04:41:52,090 - baro_tests.barometer - INFO - Getting CSV metrics of plugin mcelog on compute node 9279f0db-780f-4d8f-9d67-c1484622c9b8... 2018-04-24 04:41:52,090 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-24 04:41:52,091 - baro_tests.barometer - DEBUG - Plugin subdirs: ['mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'mcelog-SOCKET_0_CHANNEL_any_DIMM_any'] 2018-04-24 04:41:52,091 - baro_tests.barometer - DEBUG - Plugin meter categories: ['errors-corrected_memory_errors', 'errors-uncorrected_memory_errors'] 2018-04-24 04:42:04,705 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-24 04:42:04,705 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1524544515.32 ... 2018-04-24 04:42:04,705 - baro_tests.barometer - DEBUG - OK 2018-04-24 04:42:04,706 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1524544515.32 ... 2018-04-24 04:42:04,706 - baro_tests.barometer - DEBUG - OK 2018-04-24 04:42:04,707 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1524544515.32 ... 2018-04-24 04:42:04,707 - baro_tests.barometer - DEBUG - OK 2018-04-24 04:42:04,707 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1524544515.32 ... 2018-04-24 04:42:04,707 - baro_tests.barometer - DEBUG - OK 2018-04-24 04:42:04,707 - baro_tests.barometer - INFO - OK 2018-04-24 04:42:04,707 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-24 04:42:07,709 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-24 04:42:19,806 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-24 04:42:19,806 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1524544515.321, 1524544516.321), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1524544515.321, 1524544516.321), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1524544515.321, 1524544516.321), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1524544515.321, 1524544516.321)] 2018-04-24 04:42:19,807 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1524544530.321, 1524544531.321), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1524544530.321, 1524544531.321), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1524544530.321, 1524544531.321), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1524544530.321, 1524544531.321)] 2018-04-24 04:42:19,809 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1524544515.32 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1524544530.32 ... 2018-04-24 04:42:19,809 - baro_tests.barometer - DEBUG - OK 2018-04-24 04:42:19,809 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1524544515.32 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1524544530.32 ... 2018-04-24 04:42:19,810 - baro_tests.barometer - DEBUG - OK 2018-04-24 04:42:19,812 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1524544515.32 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1524544530.32 ... 2018-04-24 04:42:19,812 - baro_tests.barometer - DEBUG - OK 2018-04-24 04:42:19,812 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1524544515.32 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1524544530.32 ... 2018-04-24 04:42:19,812 - baro_tests.barometer - DEBUG - OK 2018-04-24 04:42:19,812 - baro_tests.barometer - INFO - OK 2018-04-24 04:42:19,812 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Mcelog. 2018-04-24 04:43:10,270 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-04-24 04:43:34,612 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_events on compute node 9279f0db-780f-4d8f-9d67-c1484622c9b8... 2018-04-24 04:43:34,612 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-24 04:43:34,612 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_events-br0'] 2018-04-24 04:43:34,613 - baro_tests.barometer - DEBUG - Plugin meter categories: ['gauge-link_status'] 2018-04-24 04:43:48,309 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-24 04:43:48,309 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1524544619.32 ... 2018-04-24 04:43:48,309 - baro_tests.barometer - DEBUG - OK 2018-04-24 04:43:48,309 - baro_tests.barometer - INFO - OK 2018-04-24 04:43:48,310 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-24 04:43:51,311 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-24 04:44:04,419 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-24 04:44:04,420 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1524544619.321, 1524544620.321)] 2018-04-24 04:44:04,420 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1524544635.321, 1524544636.321)] 2018-04-24 04:44:04,420 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1524544619.32 - ovs_events-br0 gauge-link_status 1524544635.32 ... 2018-04-24 04:44:04,420 - baro_tests.barometer - DEBUG - OK 2018-04-24 04:44:04,421 - baro_tests.barometer - INFO - OK 2018-04-24 04:44:04,421 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on OVS events. 2018-04-24 04:44:52,912 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS stats Test case execution === 2018-04-24 04:45:16,881 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_stats on compute node 9279f0db-780f-4d8f-9d67-c1484622c9b8... 2018-04-24 04:45:16,881 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-24 04:45:16,882 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_stats-br0.br0'] 2018-04-24 04:45:16,882 - baro_tests.barometer - DEBUG - Plugin meter categories: ['if_dropped', 'if_errors', 'if_packets'] 2018-04-24 04:45:29,045 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-24 04:45:29,045 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1524544719.32 ... 2018-04-24 04:45:29,045 - baro_tests.barometer - DEBUG - OK 2018-04-24 04:45:29,045 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1524544719.32 ... 2018-04-24 04:45:29,046 - baro_tests.barometer - DEBUG - OK 2018-04-24 04:45:29,046 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1524544720.32 ... 2018-04-24 04:45:29,046 - baro_tests.barometer - DEBUG - OK 2018-04-24 04:45:29,046 - baro_tests.barometer - INFO - OK 2018-04-24 04:45:29,048 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-24 04:45:32,051 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-24 04:45:44,926 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-24 04:45:44,926 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1524544719.321, 1524544720.321), ('ovs_stats-br0.br0', 'if_errors', 1524544719.321, 1524544721.321), ('ovs_stats-br0.br0', 'if_packets', 1524544720.321, 1524544721.321)] 2018-04-24 04:45:44,927 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1524544735.321, 1524544736.321), ('ovs_stats-br0.br0', 'if_errors', 1524544735.321, 1524544736.321), ('ovs_stats-br0.br0', 'if_packets', 1524544735.321, 1524544737.321)] 2018-04-24 04:45:44,928 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1524544719.32 - ovs_stats-br0.br0 if_dropped 1524544735.32 ... 2018-04-24 04:45:44,929 - baro_tests.barometer - DEBUG - OK 2018-04-24 04:45:44,929 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1524544719.32 - ovs_stats-br0.br0 if_errors 1524544735.32 ... 2018-04-24 04:45:44,930 - baro_tests.barometer - DEBUG - OK 2018-04-24 04:45:44,930 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1524544720.32 - ovs_stats-br0.br0 if_packets 1524544735.32 ... 2018-04-24 04:45:44,930 - baro_tests.barometer - DEBUG - OK 2018-04-24 04:45:44,931 - baro_tests.barometer - INFO - OK 2018-04-24 04:45:44,931 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on OVS stats. 2018-04-24 04:45:58,070 - baro_tests.barometer - INFO - Gnocchi plugin available in compute node overcloud-novacompute-1 2018-04-24 04:46:10,807 - baro_tests.barometer - INFO - AODH plugin present in compute node overcloud-novacompute-1 2018-04-24 04:46:10,808 - baro_tests.barometer - INFO - === NODE overcloud-novacompute-1: Enabling Test Plug-in and Test case execution === 2018-04-24 04:47:28,662 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Hugepages Test case execution === 2018-04-24 04:47:53,061 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-24 04:48:30,255 - baro_tests.barometer - INFO - PASS 2018-04-24 04:48:30,257 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with Gnocchi PASSED on Hugepages. 2018-04-24 04:49:23,126 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Intel RDT Test case execution === 2018-04-24 04:50:00,740 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-24 04:50:38,140 - baro_tests.barometer - INFO - PASS 2018-04-24 04:50:38,141 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with Gnocchi PASSED on Intel RDT. 2018-04-24 04:51:30,399 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-04-24 04:51:56,428 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-24 04:52:33,892 - baro_tests.barometer - INFO - PASS 2018-04-24 04:52:33,893 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with Gnocchi PASSED on Mcelog. 2018-04-24 04:53:26,835 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-04-24 04:53:52,062 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-24 04:54:29,409 - baro_tests.barometer - INFO - PASS 2018-04-24 04:54:29,410 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with Gnocchi PASSED on OVS events. 2018-04-24 04:55:19,497 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS stats Test case execution === 2018-04-24 04:55:45,109 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-24 04:56:22,585 - baro_tests.barometer - INFO - Plugin Interval is 1 2018-04-24 04:56:22,586 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-04-24 04:56:22,587 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on OVS stats. 2018-04-24 04:57:14,033 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-04-24 04:57:40,316 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-04-24 04:57:59,197 - baro_tests.barometer - INFO - Data not updated after interval of 12 seconds 2018-04-24 04:57:59,198 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with AODH FAILED on Mcelog. 2018-04-24 04:58:50,164 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-04-24 04:59:15,662 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-04-24 04:59:18,088 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-0 2018-04-24 04:59:20,395 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-1 2018-04-24 04:59:22,795 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with AODH FAILED on OVS events. 2018-04-24 04:59:22,797 - baro_tests.barometer - INFO - ======= Node overcloud-novacompute-1: Executing CSV Testcases ======== 2018-04-24 04:59:22,797 - baro_tests.barometer - INFO - Restarting collectd for CSV tests 2018-04-24 04:59:34,781 - baro_tests.barometer - INFO - Stopping collectd service... 2018-04-24 04:59:45,331 - baro_tests.barometer - INFO - Starting collectd service... 2018-04-24 04:59:56,434 - baro_tests.barometer - INFO - Sleeping for 10 seconds after collectd restart... 2018-04-24 05:00:55,474 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Hugepages Test case execution === 2018-04-24 05:01:21,070 - baro_tests.barometer - INFO - Getting CSV metrics of plugin hugepages on compute node b88e1c7f-a395-42f2-84b7-11a4bf60557f... 2018-04-24 05:01:21,070 - baro_tests.barometer - DEBUG - Interval: 10 2018-04-24 05:01:21,071 - baro_tests.barometer - DEBUG - Plugin subdirs: ['hugepages-mm-2048Kb', 'hugepages-node0-2048Kb'] 2018-04-24 05:01:21,072 - baro_tests.barometer - DEBUG - Plugin meter categories: ['vmpage_number-free', 'vmpage_number-used'] 2018-04-24 05:01:33,016 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-24 05:01:33,017 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1524545662.39 ... 2018-04-24 05:01:33,022 - baro_tests.barometer - DEBUG - OK 2018-04-24 05:01:33,023 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1524545662.39 ... 2018-04-24 05:01:33,023 - baro_tests.barometer - DEBUG - OK 2018-04-24 05:01:33,023 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1524545662.39 ... 2018-04-24 05:01:33,023 - baro_tests.barometer - DEBUG - OK 2018-04-24 05:01:33,023 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1524545662.39 ... 2018-04-24 05:01:33,023 - baro_tests.barometer - DEBUG - OK 2018-04-24 05:01:33,024 - baro_tests.barometer - INFO - OK 2018-04-24 05:01:33,024 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-04-24 05:01:45,036 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-24 05:01:57,675 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-24 05:01:57,676 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1524545662.39, 1524545672.39), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1524545662.39, 1524545672.39), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1524545662.39, 1524545672.39), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1524545662.39, 1524545672.39)] 2018-04-24 05:01:57,676 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1524545692.39, 1524545702.39), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1524545692.39, 1524545702.39), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1524545692.39, 1524545702.39), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1524545692.39, 1524545702.39)] 2018-04-24 05:01:57,676 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1524545662.39 - hugepages-mm-2048Kb vmpage_number-free 1524545692.39 ... 2018-04-24 05:01:57,677 - baro_tests.barometer - DEBUG - OK 2018-04-24 05:01:57,677 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1524545662.39 - hugepages-mm-2048Kb vmpage_number-used 1524545692.39 ... 2018-04-24 05:01:57,677 - baro_tests.barometer - DEBUG - OK 2018-04-24 05:01:57,677 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1524545662.39 - hugepages-node0-2048Kb vmpage_number-free 1524545692.39 ... 2018-04-24 05:01:57,677 - baro_tests.barometer - DEBUG - OK 2018-04-24 05:01:57,677 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1524545662.39 - hugepages-node0-2048Kb vmpage_number-used 1524545692.39 ... 2018-04-24 05:01:57,678 - baro_tests.barometer - DEBUG - OK 2018-04-24 05:01:57,678 - baro_tests.barometer - INFO - OK 2018-04-24 05:01:57,681 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Hugepages. 2018-04-24 05:02:49,222 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Intel RDT Test case execution === 2018-04-24 05:03:26,952 - baro_tests.barometer - INFO - Getting CSV metrics of plugin intel_rdt on compute node b88e1c7f-a395-42f2-84b7-11a4bf60557f... 2018-04-24 05:03:26,954 - baro_tests.barometer - DEBUG - Interval: 10 2018-04-24 05:03:26,956 - baro_tests.barometer - DEBUG - Plugin subdirs: ['intel_rdt-0-2'] 2018-04-24 05:03:26,957 - baro_tests.barometer - DEBUG - Plugin meter categories: ['bytes-llc', 'ipc'] 2018-04-24 05:03:39,540 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-24 05:03:39,541 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1524545792.39 ... 2018-04-24 05:03:39,541 - baro_tests.barometer - DEBUG - OK 2018-04-24 05:03:39,541 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1524545792.39 ... 2018-04-24 05:03:39,541 - baro_tests.barometer - DEBUG - OK 2018-04-24 05:03:39,541 - baro_tests.barometer - INFO - OK 2018-04-24 05:03:39,542 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-04-24 05:03:51,554 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-24 05:04:04,481 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-24 05:04:04,482 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1524545792.39, 1524545802.39), ('intel_rdt-0-2', 'ipc', 1524545792.39, 1524545802.39)] 2018-04-24 05:04:04,482 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1524545812.39, 1524545822.39), ('intel_rdt-0-2', 'ipc', 1524545812.39, 1524545822.39)] 2018-04-24 05:04:04,483 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1524545792.39 - intel_rdt-0-2 bytes-llc 1524545812.39 ... 2018-04-24 05:04:04,483 - baro_tests.barometer - DEBUG - OK 2018-04-24 05:04:04,483 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1524545792.39 - intel_rdt-0-2 ipc 1524545812.39 ... 2018-04-24 05:04:04,483 - baro_tests.barometer - DEBUG - OK 2018-04-24 05:04:04,483 - baro_tests.barometer - INFO - OK 2018-04-24 05:04:04,484 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Intel RDT. 2018-04-24 05:04:54,309 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-04-24 05:05:18,213 - baro_tests.barometer - INFO - Getting CSV metrics of plugin mcelog on compute node b88e1c7f-a395-42f2-84b7-11a4bf60557f... 2018-04-24 05:05:18,213 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-24 05:05:18,213 - baro_tests.barometer - DEBUG - Plugin subdirs: ['mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'mcelog-SOCKET_0_CHANNEL_any_DIMM_any'] 2018-04-24 05:05:18,214 - baro_tests.barometer - DEBUG - Plugin meter categories: ['errors-corrected_memory_errors', 'errors-uncorrected_memory_errors'] 2018-04-24 05:05:31,236 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-24 05:05:31,236 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1524545915.39 ... 2018-04-24 05:05:31,236 - baro_tests.barometer - DEBUG - OK 2018-04-24 05:05:31,237 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1524545915.39 ... 2018-04-24 05:05:31,237 - baro_tests.barometer - DEBUG - OK 2018-04-24 05:05:31,241 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1524545915.39 ... 2018-04-24 05:05:31,241 - baro_tests.barometer - DEBUG - OK 2018-04-24 05:05:31,242 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1524545915.39 ... 2018-04-24 05:05:31,242 - baro_tests.barometer - DEBUG - OK 2018-04-24 05:05:31,243 - baro_tests.barometer - INFO - OK 2018-04-24 05:05:31,243 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-24 05:05:34,246 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-24 05:05:47,133 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-24 05:05:47,134 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1524545915.391, 1524545916.391), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1524545915.391, 1524545916.391), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1524545915.391, 1524545916.391), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1524545915.391, 1524545916.391)] 2018-04-24 05:05:47,135 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1524545930.391, 1524545931.391), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1524545931.391, 1524545932.391), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1524545931.391, 1524545932.391), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1524545931.391, 1524545932.391)] 2018-04-24 05:05:47,135 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1524545915.39 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1524545930.39 ... 2018-04-24 05:05:47,137 - baro_tests.barometer - DEBUG - OK 2018-04-24 05:05:47,137 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1524545915.39 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1524545931.39 ... 2018-04-24 05:05:47,137 - baro_tests.barometer - DEBUG - OK 2018-04-24 05:05:47,137 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1524545915.39 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1524545931.39 ... 2018-04-24 05:05:47,137 - baro_tests.barometer - DEBUG - OK 2018-04-24 05:05:47,138 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1524545915.39 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1524545931.39 ... 2018-04-24 05:05:47,138 - baro_tests.barometer - DEBUG - OK 2018-04-24 05:05:47,138 - baro_tests.barometer - INFO - OK 2018-04-24 05:05:47,138 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Mcelog. 2018-04-24 05:06:37,096 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-04-24 05:07:02,574 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_events on compute node b88e1c7f-a395-42f2-84b7-11a4bf60557f... 2018-04-24 05:07:02,574 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-24 05:07:02,574 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_events-br0'] 2018-04-24 05:07:02,575 - baro_tests.barometer - DEBUG - Plugin meter categories: ['gauge-link_status'] 2018-04-24 05:07:14,662 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-24 05:07:14,662 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1524546019.39 ... 2018-04-24 05:07:14,662 - baro_tests.barometer - DEBUG - OK 2018-04-24 05:07:14,663 - baro_tests.barometer - INFO - OK 2018-04-24 05:07:14,663 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-24 05:07:17,666 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-24 05:07:30,313 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-24 05:07:30,313 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1524546019.391, 1524546020.391)] 2018-04-24 05:07:30,314 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1524546034.391, 1524546035.391)] 2018-04-24 05:07:30,314 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1524546019.39 - ovs_events-br0 gauge-link_status 1524546034.39 ... 2018-04-24 05:07:30,314 - baro_tests.barometer - DEBUG - OK 2018-04-24 05:07:30,314 - baro_tests.barometer - INFO - OK 2018-04-24 05:07:30,319 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on OVS events. 2018-04-24 05:08:21,118 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS stats Test case execution === 2018-04-24 05:08:46,026 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_stats on compute node b88e1c7f-a395-42f2-84b7-11a4bf60557f... 2018-04-24 05:08:46,028 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-24 05:08:46,029 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_stats-br0.br0'] 2018-04-24 05:08:46,031 - baro_tests.barometer - DEBUG - Plugin meter categories: ['if_dropped', 'if_errors', 'if_packets'] 2018-04-24 05:08:58,717 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-24 05:08:58,717 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1524546122.39 ... 2018-04-24 05:08:58,717 - baro_tests.barometer - DEBUG - OK 2018-04-24 05:08:58,718 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1524546123.39 ... 2018-04-24 05:08:58,718 - baro_tests.barometer - DEBUG - OK 2018-04-24 05:08:58,718 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1524546123.39 ... 2018-04-24 05:08:58,719 - baro_tests.barometer - DEBUG - OK 2018-04-24 05:08:58,719 - baro_tests.barometer - INFO - OK 2018-04-24 05:08:58,719 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-24 05:09:01,722 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-24 05:09:15,263 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-24 05:09:15,264 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1524546122.391, 1524546124.391), ('ovs_stats-br0.br0', 'if_errors', 1524546123.391, 1524546124.391), ('ovs_stats-br0.br0', 'if_packets', 1524546123.391, 1524546124.391)] 2018-04-24 05:09:15,264 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1524546139.391, 1524546140.391), ('ovs_stats-br0.br0', 'if_errors', 1524546139.391, 1524546140.391), ('ovs_stats-br0.br0', 'if_packets', 1524546139.391, 1524546140.391)] 2018-04-24 05:09:15,265 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1524546122.39 - ovs_stats-br0.br0 if_dropped 1524546139.39 ... 2018-04-24 05:09:15,265 - baro_tests.barometer - DEBUG - OK 2018-04-24 05:09:15,266 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1524546123.39 - ovs_stats-br0.br0 if_errors 1524546139.39 ... 2018-04-24 05:09:15,267 - baro_tests.barometer - DEBUG - OK 2018-04-24 05:09:15,267 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1524546123.39 - ovs_stats-br0.br0 if_packets 1524546139.39 ... 2018-04-24 05:09:15,268 - baro_tests.barometer - DEBUG - OK 2018-04-24 05:09:15,270 - baro_tests.barometer - INFO - OK 2018-04-24 05:09:15,271 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on OVS stats. 2018-04-24 05:09:29,117 - baro_tests.barometer - INFO - Mcelog is deleted from all compute nodes 2018-04-24 05:09:29,118 - baro_tests.barometer - INFO - ====================================================================== 2018-04-24 05:09:29,120 - baro_tests.barometer - INFO - +----------------------------------+ 2018-04-24 05:09:29,121 - baro_tests.barometer - INFO - | OVERALL SUMMARY | 2018-04-24 05:09:29,123 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-24 05:09:29,124 - baro_tests.barometer - INFO - | Test | Node-0 | Node-1 | 2018-04-24 05:09:29,125 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-24 05:09:29,128 - baro_tests.barometer - INFO - | OUT:Gnocchi | PASS | PASS | 2018-04-24 05:09:29,129 - baro_tests.barometer - INFO - | IN:Hugepages | PASS | PASS | 2018-04-24 05:09:29,131 - baro_tests.barometer - INFO - | IN:Intel RDT | PASS | PASS | 2018-04-24 05:09:29,132 - baro_tests.barometer - INFO - | IN:Mcelog | FAIL | PASS | 2018-04-24 05:09:29,132 - baro_tests.barometer - INFO - | IN:OVS events | PASS | PASS | 2018-04-24 05:09:29,133 - baro_tests.barometer - INFO - | IN:OVS stats | PASS | FAIL | 2018-04-24 05:09:29,133 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-24 05:09:29,133 - baro_tests.barometer - INFO - | OUT:AODH | PASS | PASS | 2018-04-24 05:09:29,135 - baro_tests.barometer - INFO - | IN:Mcelog | FAIL | FAIL | 2018-04-24 05:09:29,135 - baro_tests.barometer - INFO - | IN:OVS events | SKIP | SKIP | 2018-04-24 05:09:29,137 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-24 05:09:29,138 - baro_tests.barometer - INFO - | OUT:CSV | PASS | PASS | 2018-04-24 05:09:29,138 - baro_tests.barometer - INFO - | IN:Hugepages | PASS | PASS | 2018-04-24 05:09:29,138 - baro_tests.barometer - INFO - | IN:Intel RDT | PASS | PASS | 2018-04-24 05:09:29,138 - baro_tests.barometer - INFO - | IN:Mcelog | PASS | PASS | 2018-04-24 05:09:29,138 - baro_tests.barometer - INFO - | IN:OVS events | PASS | PASS | 2018-04-24 05:09:29,138 - baro_tests.barometer - INFO - | IN:OVS stats | PASS | PASS | 2018-04-24 05:09:29,139 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-24 05:09:29,139 - baro_tests.barometer - INFO - ====================================================================== 2018-04-24 05:09:29,139 - baro_tests.barometer - ERROR - Some tests have failed or have not been executed 2018-04-24 05:09:29,139 - baro_tests.barometer - ERROR - Overall Result is Fail