2018-05-25 22:03:07,081 - baro_tests.barometer - INFO - =================== Starting barometer tests suite =================== 2018-05-25 22:03:07,081 - baro_tests.barometer - INFO - RSA key file /root/.ssh/id_rsa exists. 2018-05-25 22:03:10,465 - baro_tests.barometer - INFO - ==== Display of Control and Compute nodes available in the set up ==== 2018-05-25 22:03:10,465 - baro_tests.barometer - INFO - controllers: ['overcloud-controller-0: 192.30.9.4', 'overcloud-controller-1: 192.30.9.5', 'overcloud-controller-2: 192.30.9.7'] 2018-05-25 22:03:10,465 - baro_tests.barometer - INFO - computes: ['overcloud-novacompute-0: 192.30.9.10', 'overcloud-novacompute-1: 192.30.9.3'] 2018-05-25 22:03:10,465 - baro_tests.barometer - INFO - ========== Enabling mcelog and OVS bridges on compute nodes ========== 2018-05-25 22:03:23,877 - baro_tests.barometer - INFO - Checking if mcelog is enabled on node-overcloud-novacompute-0... 2018-05-25 22:03:23,945 - baro_tests.barometer - INFO - Mcelog will be enabled on node-overcloud-novacompute-0... 2018-05-25 22:03:24,578 - baro_tests.barometer - INFO - Mcelog was installed on node-overcloud-novacompute-0. 2018-05-25 22:03:24,609 - baro_tests.barometer - INFO - Checking if mcelog is enabled on node-overcloud-novacompute-1... 2018-05-25 22:03:24,678 - baro_tests.barometer - INFO - Mcelog will be enabled on node-overcloud-novacompute-1... 2018-05-25 22:03:25,603 - baro_tests.barometer - INFO - Mcelog was installed on node-overcloud-novacompute-1. 2018-05-25 22:03:37,587 - baro_tests.barometer - INFO - Creating OVS bridges on computes nodes 2018-05-25 22:03:37,908 - baro_tests.barometer - INFO - OVS Bridges created on compute nodes 2018-05-25 22:03:37,909 - baro_tests.barometer - INFO - ============= Testing Gnocchi and AODH plugins on nodes ============== 2018-05-25 22:05:21,638 - baro_tests.barometer - INFO - Gnocchi plugin available in compute node overcloud-novacompute-0 2018-05-25 22:05:34,001 - baro_tests.barometer - INFO - AODH plugin present in compute node overcloud-novacompute-0 2018-05-25 22:05:34,002 - baro_tests.barometer - INFO - === NODE overcloud-novacompute-0: Enabling Test Plug-in and Test case execution === 2018-05-25 22:06:42,996 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Hugepages Test case execution === 2018-05-25 22:07:07,360 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-25 22:07:44,707 - baro_tests.barometer - INFO - PASS 2018-05-25 22:07:44,708 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with Gnocchi PASSED on Hugepages. 2018-05-25 22:08:34,924 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Intel RDT Test case execution === 2018-05-25 22:09:12,735 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-25 22:09:50,002 - baro_tests.barometer - INFO - Plugin Interval is 10 2018-05-25 22:09:50,002 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-05-25 22:09:50,003 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on Intel RDT. 2018-05-25 22:10:41,290 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-05-25 22:11:07,977 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-25 22:11:45,302 - baro_tests.barometer - INFO - Plugin Interval is 1 2018-05-25 22:11:45,302 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-05-25 22:11:45,303 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on Mcelog. 2018-05-25 22:12:35,619 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-05-25 22:13:01,608 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-25 22:13:38,733 - baro_tests.barometer - INFO - Plugin Interval is 1 2018-05-25 22:13:38,733 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-05-25 22:13:38,734 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on OVS events. 2018-05-25 22:14:30,793 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS stats Test case execution === 2018-05-25 22:14:56,258 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-25 22:14:58,838 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-05-25 22:15:01,700 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-1 2018-05-25 22:15:04,497 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on OVS stats. 2018-05-25 22:15:55,946 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-05-25 22:16:20,181 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-05-25 22:16:39,911 - baro_tests.barometer - INFO - Data not updated after interval of 12 seconds 2018-05-25 22:16:39,913 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with AODH FAILED on Mcelog. 2018-05-25 22:17:30,515 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-05-25 22:17:55,422 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-05-25 22:17:57,834 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-0 2018-05-25 22:18:01,030 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-1 2018-05-25 22:18:04,574 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with AODH FAILED on OVS events. 2018-05-25 22:18:04,575 - baro_tests.barometer - INFO - ======= Node overcloud-novacompute-0: Executing CSV Testcases ======== 2018-05-25 22:18:04,575 - baro_tests.barometer - INFO - Restarting collectd for CSV tests 2018-05-25 22:18:17,637 - baro_tests.barometer - INFO - Stopping collectd service... 2018-05-25 22:18:28,293 - baro_tests.barometer - INFO - Starting collectd service... 2018-05-25 22:18:38,634 - baro_tests.barometer - INFO - Sleeping for 10 seconds after collectd restart... 2018-05-25 22:19:40,488 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Hugepages Test case execution === 2018-05-25 22:20:05,178 - baro_tests.barometer - INFO - Getting CSV metrics of plugin hugepages on compute node 4c17c528-7ad2-49ce-83b8-5f4f13fb21c2... 2018-05-25 22:20:05,178 - baro_tests.barometer - DEBUG - Interval: 10 2018-05-25 22:20:05,180 - baro_tests.barometer - DEBUG - Plugin subdirs: ['hugepages-mm-2048Kb', 'hugepages-node0-2048Kb'] 2018-05-25 22:20:05,182 - baro_tests.barometer - DEBUG - Plugin meter categories: ['vmpage_number-free', 'vmpage_number-used'] 2018-05-25 22:20:17,916 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-25 22:20:17,918 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1527286351.88 ... 2018-05-25 22:20:17,919 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:20:17,919 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1527286351.88 ... 2018-05-25 22:20:17,920 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:20:17,920 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1527286351.88 ... 2018-05-25 22:20:17,920 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:20:17,920 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1527286351.88 ... 2018-05-25 22:20:17,920 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:20:17,921 - baro_tests.barometer - INFO - OK 2018-05-25 22:20:17,921 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-05-25 22:20:29,932 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-05-25 22:20:43,270 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-25 22:20:43,270 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1527286351.878, 1527286361.878), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1527286351.878, 1527286361.878), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1527286351.878, 1527286361.878), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1527286351.878, 1527286361.878)] 2018-05-25 22:20:43,270 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1527286381.878, 1527286391.878), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1527286381.878, 1527286391.878), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1527286381.878, 1527286391.878), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1527286381.878, 1527286391.878)] 2018-05-25 22:20:43,271 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1527286351.88 - hugepages-mm-2048Kb vmpage_number-free 1527286381.88 ... 2018-05-25 22:20:43,271 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:20:43,271 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1527286351.88 - hugepages-mm-2048Kb vmpage_number-used 1527286381.88 ... 2018-05-25 22:20:43,271 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:20:43,271 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1527286351.88 - hugepages-node0-2048Kb vmpage_number-free 1527286381.88 ... 2018-05-25 22:20:43,271 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:20:43,272 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1527286351.88 - hugepages-node0-2048Kb vmpage_number-used 1527286381.88 ... 2018-05-25 22:20:43,272 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:20:43,272 - baro_tests.barometer - INFO - OK 2018-05-25 22:20:43,272 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Hugepages. 2018-05-25 22:21:33,912 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Intel RDT Test case execution === 2018-05-25 22:22:10,151 - baro_tests.barometer - INFO - Getting CSV metrics of plugin intel_rdt on compute node 4c17c528-7ad2-49ce-83b8-5f4f13fb21c2... 2018-05-25 22:22:10,152 - baro_tests.barometer - DEBUG - Interval: 10 2018-05-25 22:22:10,152 - baro_tests.barometer - DEBUG - Plugin subdirs: ['intel_rdt-0-2'] 2018-05-25 22:22:10,153 - baro_tests.barometer - DEBUG - Plugin meter categories: ['bytes-llc', 'ipc'] 2018-05-25 22:22:23,207 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-25 22:22:23,208 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1527286481.88 ... 2018-05-25 22:22:23,208 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:22:23,208 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1527286481.88 ... 2018-05-25 22:22:23,209 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:22:23,209 - baro_tests.barometer - INFO - OK 2018-05-25 22:22:23,209 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-05-25 22:22:35,219 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-05-25 22:22:47,511 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-25 22:22:47,511 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1527286481.878, 1527286491.878), ('intel_rdt-0-2', 'ipc', 1527286481.878, 1527286491.878)] 2018-05-25 22:22:47,512 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1527286501.878, 1527286511.878), ('intel_rdt-0-2', 'ipc', 1527286501.878, 1527286511.878)] 2018-05-25 22:22:47,512 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1527286481.88 - intel_rdt-0-2 bytes-llc 1527286501.88 ... 2018-05-25 22:22:47,512 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:22:47,512 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1527286481.88 - intel_rdt-0-2 ipc 1527286501.88 ... 2018-05-25 22:22:47,512 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:22:47,512 - baro_tests.barometer - INFO - OK 2018-05-25 22:22:47,513 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Intel RDT. 2018-05-25 22:23:41,339 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-05-25 22:24:05,272 - baro_tests.barometer - INFO - Getting CSV metrics of plugin mcelog on compute node 4c17c528-7ad2-49ce-83b8-5f4f13fb21c2... 2018-05-25 22:24:05,272 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-25 22:24:05,273 - baro_tests.barometer - DEBUG - Plugin subdirs: ['mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'mcelog-SOCKET_0_CHANNEL_any_DIMM_any'] 2018-05-25 22:24:05,273 - baro_tests.barometer - DEBUG - Plugin meter categories: ['errors-corrected_memory_errors', 'errors-uncorrected_memory_errors'] 2018-05-25 22:24:18,832 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-25 22:24:18,833 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1527286609.88 ... 2018-05-25 22:24:18,833 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:24:18,834 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1527286609.88 ... 2018-05-25 22:24:18,834 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:24:18,834 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1527286609.88 ... 2018-05-25 22:24:18,834 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:24:18,834 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1527286609.88 ... 2018-05-25 22:24:18,834 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:24:18,835 - baro_tests.barometer - INFO - OK 2018-05-25 22:24:18,836 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-25 22:24:21,837 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-05-25 22:24:34,150 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-25 22:24:34,150 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1527286609.879, 1527286610.879), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1527286609.879, 1527286610.879), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1527286609.879, 1527286610.879), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1527286609.879, 1527286610.879)] 2018-05-25 22:24:34,150 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1527286625.879, 1527286626.879), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1527286625.879, 1527286626.879), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1527286625.879, 1527286626.879), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1527286625.879, 1527286626.879)] 2018-05-25 22:24:34,150 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1527286609.88 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1527286625.88 ... 2018-05-25 22:24:34,151 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:24:34,151 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1527286609.88 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1527286625.88 ... 2018-05-25 22:24:34,151 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:24:34,152 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1527286609.88 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1527286625.88 ... 2018-05-25 22:24:34,152 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:24:34,152 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1527286609.88 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1527286625.88 ... 2018-05-25 22:24:34,152 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:24:34,152 - baro_tests.barometer - INFO - OK 2018-05-25 22:24:34,152 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Mcelog. 2018-05-25 22:25:25,604 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-05-25 22:25:50,362 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_events on compute node 4c17c528-7ad2-49ce-83b8-5f4f13fb21c2... 2018-05-25 22:25:50,362 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-25 22:25:50,362 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_events-br0'] 2018-05-25 22:25:50,362 - baro_tests.barometer - DEBUG - Plugin meter categories: ['gauge-link_status'] 2018-05-25 22:26:03,573 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-25 22:26:03,574 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1527286714.88 ... 2018-05-25 22:26:03,574 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:26:03,574 - baro_tests.barometer - INFO - OK 2018-05-25 22:26:03,575 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-25 22:26:06,577 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-05-25 22:26:19,503 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-25 22:26:19,504 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1527286714.879, 1527286715.879)] 2018-05-25 22:26:19,504 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1527286730.879, 1527286731.879)] 2018-05-25 22:26:19,505 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1527286714.88 - ovs_events-br0 gauge-link_status 1527286730.88 ... 2018-05-25 22:26:19,505 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:26:19,505 - baro_tests.barometer - INFO - OK 2018-05-25 22:26:19,505 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on OVS events. 2018-05-25 22:27:10,093 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS stats Test case execution === 2018-05-25 22:27:36,514 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_stats on compute node 4c17c528-7ad2-49ce-83b8-5f4f13fb21c2... 2018-05-25 22:27:36,516 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-25 22:27:36,516 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_stats-br0.br0'] 2018-05-25 22:27:36,518 - baro_tests.barometer - DEBUG - Plugin meter categories: ['if_dropped', 'if_errors', 'if_packets'] 2018-05-25 22:27:50,775 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-25 22:27:50,775 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1527286821.88 ... 2018-05-25 22:27:50,775 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:27:50,776 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1527286821.88 ... 2018-05-25 22:27:50,776 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:27:50,776 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1527286822.88 ... 2018-05-25 22:27:50,777 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:27:50,777 - baro_tests.barometer - INFO - OK 2018-05-25 22:27:50,777 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-25 22:27:53,778 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-05-25 22:28:07,942 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-25 22:28:07,943 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1527286821.879, 1527286822.879), ('ovs_stats-br0.br0', 'if_errors', 1527286821.879, 1527286823.879), ('ovs_stats-br0.br0', 'if_packets', 1527286822.879, 1527286823.879)] 2018-05-25 22:28:07,943 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1527286839.879, 1527286840.879), ('ovs_stats-br0.br0', 'if_errors', 1527286839.879, 1527286840.879), ('ovs_stats-br0.br0', 'if_packets', 1527286839.879, 1527286840.879)] 2018-05-25 22:28:07,944 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1527286821.88 - ovs_stats-br0.br0 if_dropped 1527286839.88 ... 2018-05-25 22:28:07,945 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:28:07,945 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1527286821.88 - ovs_stats-br0.br0 if_errors 1527286839.88 ... 2018-05-25 22:28:07,946 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:28:07,946 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1527286822.88 - ovs_stats-br0.br0 if_packets 1527286839.88 ... 2018-05-25 22:28:07,951 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:28:07,952 - baro_tests.barometer - INFO - OK 2018-05-25 22:28:07,952 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on OVS stats. 2018-05-25 22:28:20,666 - baro_tests.barometer - INFO - Gnocchi plugin available in compute node overcloud-novacompute-1 2018-05-25 22:28:34,236 - baro_tests.barometer - INFO - AODH plugin present in compute node overcloud-novacompute-1 2018-05-25 22:28:34,236 - baro_tests.barometer - INFO - === NODE overcloud-novacompute-1: Enabling Test Plug-in and Test case execution === 2018-05-25 22:29:53,601 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Hugepages Test case execution === 2018-05-25 22:30:18,937 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-25 22:30:56,003 - baro_tests.barometer - INFO - Plugin Interval is 10 2018-05-25 22:30:56,003 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-05-25 22:30:56,003 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on Hugepages. 2018-05-25 22:31:50,206 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Intel RDT Test case execution === 2018-05-25 22:32:30,364 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-25 22:33:07,693 - baro_tests.barometer - INFO - PASS 2018-05-25 22:33:07,698 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with Gnocchi PASSED on Intel RDT. 2018-05-25 22:34:00,692 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-05-25 22:34:26,356 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-25 22:35:04,450 - baro_tests.barometer - INFO - Plugin Interval is 1 2018-05-25 22:35:04,451 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-05-25 22:35:04,451 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on Mcelog. 2018-05-25 22:35:56,910 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-05-25 22:36:22,955 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-25 22:36:25,680 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-05-25 22:36:28,282 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-1 2018-05-25 22:36:31,477 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on OVS events. 2018-05-25 22:37:22,158 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS stats Test case execution === 2018-05-25 22:37:47,605 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-25 22:38:25,111 - baro_tests.barometer - INFO - Plugin Interval is 1 2018-05-25 22:38:25,112 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-05-25 22:38:25,112 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on OVS stats. 2018-05-25 22:39:16,720 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-05-25 22:39:41,623 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-05-25 22:40:00,575 - baro_tests.barometer - INFO - Data not updated after interval of 12 seconds 2018-05-25 22:40:00,576 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with AODH FAILED on Mcelog. 2018-05-25 22:40:52,433 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-05-25 22:41:18,684 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-05-25 22:41:21,105 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-0 2018-05-25 22:41:23,526 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-1 2018-05-25 22:41:25,935 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with AODH FAILED on OVS events. 2018-05-25 22:41:25,936 - baro_tests.barometer - INFO - ======= Node overcloud-novacompute-1: Executing CSV Testcases ======== 2018-05-25 22:41:25,936 - baro_tests.barometer - INFO - Restarting collectd for CSV tests 2018-05-25 22:41:38,051 - baro_tests.barometer - INFO - Stopping collectd service... 2018-05-25 22:41:49,637 - baro_tests.barometer - INFO - Starting collectd service... 2018-05-25 22:41:59,963 - baro_tests.barometer - INFO - Sleeping for 10 seconds after collectd restart... 2018-05-25 22:43:02,488 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Hugepages Test case execution === 2018-05-25 22:43:28,662 - baro_tests.barometer - INFO - Getting CSV metrics of plugin hugepages on compute node 8e1565a9-50f7-4f4e-9635-3d3c423f49f9... 2018-05-25 22:43:28,663 - baro_tests.barometer - DEBUG - Interval: 10 2018-05-25 22:43:28,663 - baro_tests.barometer - DEBUG - Plugin subdirs: ['hugepages-mm-2048Kb', 'hugepages-node0-2048Kb'] 2018-05-25 22:43:28,664 - baro_tests.barometer - DEBUG - Plugin meter categories: ['vmpage_number-free', 'vmpage_number-used'] 2018-05-25 22:43:41,973 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-25 22:43:41,974 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1527287755.22 ... 2018-05-25 22:43:41,974 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:43:41,974 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1527287755.22 ... 2018-05-25 22:43:41,974 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:43:41,975 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1527287755.22 ... 2018-05-25 22:43:41,975 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:43:41,977 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1527287755.22 ... 2018-05-25 22:43:41,979 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:43:41,980 - baro_tests.barometer - INFO - OK 2018-05-25 22:43:41,980 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-05-25 22:43:53,992 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-05-25 22:44:06,959 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-25 22:44:06,959 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1527287755.224, 1527287765.224), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1527287755.224, 1527287765.224), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1527287755.224, 1527287765.224), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1527287755.224, 1527287765.224)] 2018-05-25 22:44:06,959 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1527287775.224, 1527287785.224), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1527287775.224, 1527287785.224), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1527287775.224, 1527287785.224), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1527287775.224, 1527287785.224)] 2018-05-25 22:44:06,960 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1527287755.22 - hugepages-mm-2048Kb vmpage_number-free 1527287775.22 ... 2018-05-25 22:44:06,960 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:44:06,960 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1527287755.22 - hugepages-mm-2048Kb vmpage_number-used 1527287775.22 ... 2018-05-25 22:44:06,961 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:44:06,961 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1527287755.22 - hugepages-node0-2048Kb vmpage_number-free 1527287775.22 ... 2018-05-25 22:44:06,961 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:44:06,961 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1527287755.22 - hugepages-node0-2048Kb vmpage_number-used 1527287775.22 ... 2018-05-25 22:44:06,961 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:44:06,961 - baro_tests.barometer - INFO - OK 2018-05-25 22:44:06,961 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Hugepages. 2018-05-25 22:44:58,088 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Intel RDT Test case execution === 2018-05-25 22:45:35,392 - baro_tests.barometer - INFO - Getting CSV metrics of plugin intel_rdt on compute node 8e1565a9-50f7-4f4e-9635-3d3c423f49f9... 2018-05-25 22:45:35,392 - baro_tests.barometer - DEBUG - Interval: 10 2018-05-25 22:45:35,392 - baro_tests.barometer - DEBUG - Plugin subdirs: ['intel_rdt-0-2'] 2018-05-25 22:45:35,393 - baro_tests.barometer - DEBUG - Plugin meter categories: ['bytes-llc', 'ipc'] 2018-05-25 22:45:49,643 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-25 22:45:49,643 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1527287875.22 ... 2018-05-25 22:45:49,643 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:45:49,644 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1527287875.22 ... 2018-05-25 22:45:49,644 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:45:49,644 - baro_tests.barometer - INFO - OK 2018-05-25 22:45:49,645 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-05-25 22:46:01,657 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-05-25 22:46:15,121 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-25 22:46:15,122 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1527287875.224, 1527287885.224), ('intel_rdt-0-2', 'ipc', 1527287875.224, 1527287885.224)] 2018-05-25 22:46:15,123 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1527287905.223, 1527287915.224), ('intel_rdt-0-2', 'ipc', 1527287905.223, 1527287915.224)] 2018-05-25 22:46:15,124 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1527287875.22 - intel_rdt-0-2 bytes-llc 1527287905.22 ... 2018-05-25 22:46:15,125 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:46:15,126 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1527287875.22 - intel_rdt-0-2 ipc 1527287905.22 ... 2018-05-25 22:46:15,126 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:46:15,126 - baro_tests.barometer - INFO - OK 2018-05-25 22:46:15,128 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Intel RDT. 2018-05-25 22:47:07,822 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-05-25 22:47:32,999 - baro_tests.barometer - INFO - Getting CSV metrics of plugin mcelog on compute node 8e1565a9-50f7-4f4e-9635-3d3c423f49f9... 2018-05-25 22:47:33,000 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-25 22:47:33,000 - baro_tests.barometer - DEBUG - Plugin subdirs: ['mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'mcelog-SOCKET_0_CHANNEL_any_DIMM_any'] 2018-05-25 22:47:33,000 - baro_tests.barometer - DEBUG - Plugin meter categories: ['errors-corrected_memory_errors', 'errors-uncorrected_memory_errors'] 2018-05-25 22:47:46,158 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-25 22:47:46,159 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1527288009.22 ... 2018-05-25 22:47:46,161 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:47:46,161 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1527288009.22 ... 2018-05-25 22:47:46,162 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:47:46,162 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1527288010.22 ... 2018-05-25 22:47:46,163 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:47:46,163 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1527288010.22 ... 2018-05-25 22:47:46,164 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:47:46,164 - baro_tests.barometer - INFO - OK 2018-05-25 22:47:46,164 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-25 22:47:49,167 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-05-25 22:48:03,365 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-25 22:48:03,366 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1527288009.224, 1527288010.225), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1527288009.224, 1527288010.225), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1527288010.225, 1527288011.225), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1527288010.225, 1527288011.225)] 2018-05-25 22:48:03,366 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1527288026.225, 1527288028.224), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1527288027.225, 1527288028.224), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1527288027.225, 1527288028.224), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1527288027.225, 1527288028.224)] 2018-05-25 22:48:03,366 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1527288009.22 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1527288026.22 ... 2018-05-25 22:48:03,366 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:48:03,366 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1527288009.22 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1527288027.22 ... 2018-05-25 22:48:03,367 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:48:03,367 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1527288010.22 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1527288027.22 ... 2018-05-25 22:48:03,368 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:48:03,370 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1527288010.22 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1527288027.22 ... 2018-05-25 22:48:03,370 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:48:03,371 - baro_tests.barometer - INFO - OK 2018-05-25 22:48:03,372 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Mcelog. 2018-05-25 22:48:54,540 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-05-25 22:49:18,661 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_events on compute node 8e1565a9-50f7-4f4e-9635-3d3c423f49f9... 2018-05-25 22:49:18,662 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-25 22:49:18,662 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_events-br0'] 2018-05-25 22:49:18,663 - baro_tests.barometer - DEBUG - Plugin meter categories: ['gauge-link_status'] 2018-05-25 22:49:31,161 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-25 22:49:31,161 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1527288115.22 ... 2018-05-25 22:49:31,161 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:49:31,161 - baro_tests.barometer - INFO - OK 2018-05-25 22:49:31,162 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-25 22:49:34,165 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-05-25 22:49:47,112 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-25 22:49:47,113 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1527288115.225, 1527288116.225)] 2018-05-25 22:49:47,113 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1527288131.225, 1527288132.225)] 2018-05-25 22:49:47,114 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1527288115.22 - ovs_events-br0 gauge-link_status 1527288131.22 ... 2018-05-25 22:49:47,114 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:49:47,114 - baro_tests.barometer - INFO - OK 2018-05-25 22:49:47,117 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on OVS events. 2018-05-25 22:50:38,998 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS stats Test case execution === 2018-05-25 22:51:04,325 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_stats on compute node 8e1565a9-50f7-4f4e-9635-3d3c423f49f9... 2018-05-25 22:51:04,330 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-25 22:51:04,332 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_stats-br0.br0'] 2018-05-25 22:51:04,333 - baro_tests.barometer - DEBUG - Plugin meter categories: ['if_dropped', 'if_errors', 'if_packets'] 2018-05-25 22:51:17,963 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-25 22:51:17,964 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1527288221.22 ... 2018-05-25 22:51:17,970 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:51:17,975 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1527288221.22 ... 2018-05-25 22:51:17,980 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:51:17,985 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1527288221.22 ... 2018-05-25 22:51:17,988 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:51:17,989 - baro_tests.barometer - INFO - OK 2018-05-25 22:51:17,991 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-25 22:51:20,993 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-05-25 22:51:33,509 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-25 22:51:33,510 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1527288221.225, 1527288222.225), ('ovs_stats-br0.br0', 'if_errors', 1527288221.225, 1527288222.225), ('ovs_stats-br0.br0', 'if_packets', 1527288221.225, 1527288223.225)] 2018-05-25 22:51:33,512 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1527288237.225, 1527288238.225), ('ovs_stats-br0.br0', 'if_errors', 1527288237.225, 1527288238.225), ('ovs_stats-br0.br0', 'if_packets', 1527288237.225, 1527288238.225)] 2018-05-25 22:51:33,515 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1527288221.22 - ovs_stats-br0.br0 if_dropped 1527288237.22 ... 2018-05-25 22:51:33,515 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:51:33,517 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1527288221.22 - ovs_stats-br0.br0 if_errors 1527288237.22 ... 2018-05-25 22:51:33,519 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:51:33,520 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1527288221.22 - ovs_stats-br0.br0 if_packets 1527288237.22 ... 2018-05-25 22:51:33,521 - baro_tests.barometer - DEBUG - OK 2018-05-25 22:51:33,523 - baro_tests.barometer - INFO - OK 2018-05-25 22:51:33,523 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on OVS stats. 2018-05-25 22:51:46,913 - baro_tests.barometer - INFO - Mcelog is deleted from all compute nodes 2018-05-25 22:51:46,914 - baro_tests.barometer - INFO - ====================================================================== 2018-05-25 22:51:46,916 - baro_tests.barometer - INFO - +----------------------------------+ 2018-05-25 22:51:46,917 - baro_tests.barometer - INFO - | OVERALL SUMMARY | 2018-05-25 22:51:46,917 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-05-25 22:51:46,920 - baro_tests.barometer - INFO - | Test | Node-0 | Node-1 | 2018-05-25 22:51:46,921 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-05-25 22:51:46,924 - baro_tests.barometer - INFO - | OUT:Gnocchi | PASS | PASS | 2018-05-25 22:51:46,926 - baro_tests.barometer - INFO - | IN:Hugepages | PASS | FAIL | 2018-05-25 22:51:46,928 - baro_tests.barometer - INFO - | IN:Intel RDT | FAIL | PASS | 2018-05-25 22:51:46,928 - baro_tests.barometer - INFO - | IN:Mcelog | FAIL | FAIL | 2018-05-25 22:51:46,929 - baro_tests.barometer - INFO - | IN:OVS events | FAIL | FAIL | 2018-05-25 22:51:46,929 - baro_tests.barometer - INFO - | IN:OVS stats | FAIL | FAIL | 2018-05-25 22:51:46,930 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-05-25 22:51:46,930 - baro_tests.barometer - INFO - | OUT:AODH | PASS | PASS | 2018-05-25 22:51:46,930 - baro_tests.barometer - INFO - | IN:Mcelog | FAIL | FAIL | 2018-05-25 22:51:46,930 - baro_tests.barometer - INFO - | IN:OVS events | SKIP | SKIP | 2018-05-25 22:51:46,931 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-05-25 22:51:46,931 - baro_tests.barometer - INFO - | OUT:CSV | PASS | PASS | 2018-05-25 22:51:46,931 - baro_tests.barometer - INFO - | IN:Hugepages | PASS | PASS | 2018-05-25 22:51:46,931 - baro_tests.barometer - INFO - | IN:Intel RDT | PASS | PASS | 2018-05-25 22:51:46,931 - baro_tests.barometer - INFO - | IN:Mcelog | PASS | PASS | 2018-05-25 22:51:46,931 - baro_tests.barometer - INFO - | IN:OVS events | PASS | PASS | 2018-05-25 22:51:46,931 - baro_tests.barometer - INFO - | IN:OVS stats | PASS | PASS | 2018-05-25 22:51:46,931 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-05-25 22:51:46,932 - baro_tests.barometer - INFO - ====================================================================== 2018-05-25 22:51:46,932 - baro_tests.barometer - ERROR - Some tests have failed or have not been executed 2018-05-25 22:51:46,932 - baro_tests.barometer - ERROR - Overall Result is Fail