2018-05-08 16:59:23,907 - baro_tests.barometer - INFO - =================== Starting barometer tests suite =================== 2018-05-08 16:59:23,907 - baro_tests.barometer - INFO - RSA key file /root/.ssh/id_rsa exists. 2018-05-08 16:59:28,916 - baro_tests.barometer - INFO - ==== Display of Control and Compute nodes available in the set up ==== 2018-05-08 16:59:28,917 - baro_tests.barometer - INFO - controllers: ['overcloud-controller-0: 192.30.9.3', 'overcloud-controller-1: 192.30.9.5', 'overcloud-controller-2: 192.30.9.6'] 2018-05-08 16:59:28,917 - baro_tests.barometer - INFO - computes: ['overcloud-novacompute-0: 192.30.9.7', 'overcloud-novacompute-1: 192.30.9.10'] 2018-05-08 16:59:28,917 - baro_tests.barometer - INFO - ========== Enabling mcelog and OVS bridges on compute nodes ========== 2018-05-08 16:59:42,638 - baro_tests.barometer - INFO - Checking if mcelog is enabled on node-overcloud-novacompute-0... 2018-05-08 16:59:42,705 - baro_tests.barometer - INFO - Mcelog will be enabled on node-overcloud-novacompute-0... 2018-05-08 16:59:43,228 - baro_tests.barometer - INFO - Mcelog was installed on node-overcloud-novacompute-0. 2018-05-08 16:59:43,260 - baro_tests.barometer - INFO - Checking if mcelog is enabled on node-overcloud-novacompute-1... 2018-05-08 16:59:43,328 - baro_tests.barometer - INFO - Mcelog will be enabled on node-overcloud-novacompute-1... 2018-05-08 16:59:43,826 - baro_tests.barometer - INFO - Mcelog was installed on node-overcloud-novacompute-1. 2018-05-08 16:59:58,726 - baro_tests.barometer - INFO - Creating OVS bridges on computes nodes 2018-05-08 16:59:59,049 - baro_tests.barometer - INFO - OVS Bridges created on compute nodes 2018-05-08 16:59:59,049 - baro_tests.barometer - INFO - ============= Testing Gnocchi and AODH plugins on nodes ============== 2018-05-08 17:01:53,815 - baro_tests.barometer - INFO - Gnocchi plugin available in compute node overcloud-novacompute-0 2018-05-08 17:02:07,790 - baro_tests.barometer - INFO - AODH plugin present in compute node overcloud-novacompute-0 2018-05-08 17:02:07,791 - baro_tests.barometer - INFO - === NODE overcloud-novacompute-0: Enabling Test Plug-in and Test case execution === 2018-05-08 17:03:36,990 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Hugepages Test case execution === 2018-05-08 17:04:04,987 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-08 17:04:42,419 - baro_tests.barometer - INFO - Plugin Interval is 10 2018-05-08 17:04:42,419 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-05-08 17:04:42,419 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on Hugepages. 2018-05-08 17:05:39,519 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Intel RDT Test case execution === 2018-05-08 17:06:21,447 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-08 17:06:58,565 - baro_tests.barometer - INFO - PASS 2018-05-08 17:06:58,566 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with Gnocchi PASSED on Intel RDT. 2018-05-08 17:07:53,307 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-05-08 17:08:20,315 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-08 17:08:57,588 - baro_tests.barometer - INFO - Plugin Interval is 1 2018-05-08 17:08:57,588 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-05-08 17:08:57,589 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on Mcelog. 2018-05-08 17:09:53,873 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-05-08 17:10:21,477 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-08 17:10:58,856 - baro_tests.barometer - INFO - Plugin Interval is 1 2018-05-08 17:10:58,856 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-05-08 17:10:58,857 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on OVS events. 2018-05-08 17:11:53,759 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS stats Test case execution === 2018-05-08 17:12:20,235 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-08 17:12:57,600 - baro_tests.barometer - INFO - PASS 2018-05-08 17:12:57,600 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with Gnocchi PASSED on OVS stats. 2018-05-08 17:13:50,446 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-05-08 17:14:16,552 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-05-08 17:14:35,680 - baro_tests.barometer - INFO - PASS 2018-05-08 17:14:35,682 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with AODH PASSED on Mcelog. 2018-05-08 17:15:28,239 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-05-08 17:15:55,036 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-05-08 17:16:14,055 - baro_tests.barometer - INFO - PASS 2018-05-08 17:16:14,056 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with AODH PASSED on OVS events. 2018-05-08 17:16:14,057 - baro_tests.barometer - INFO - ======= Node overcloud-novacompute-0: Executing CSV Testcases ======== 2018-05-08 17:16:14,057 - baro_tests.barometer - INFO - Restarting collectd for CSV tests 2018-05-08 17:16:27,761 - baro_tests.barometer - INFO - Stopping collectd service... 2018-05-08 17:16:39,107 - baro_tests.barometer - INFO - Starting collectd service... 2018-05-08 17:16:49,439 - baro_tests.barometer - INFO - Sleeping for 10 seconds after collectd restart... 2018-05-08 17:17:51,988 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Hugepages Test case execution === 2018-05-08 17:18:19,568 - baro_tests.barometer - INFO - Getting CSV metrics of plugin hugepages on compute node 02eaf396-82ac-4651-8a3c-8d85b1b80402... 2018-05-08 17:18:19,569 - baro_tests.barometer - DEBUG - Interval: 10 2018-05-08 17:18:19,569 - baro_tests.barometer - DEBUG - Plugin subdirs: ['hugepages-mm-2048Kb', 'hugepages-node0-2048Kb'] 2018-05-08 17:18:19,569 - baro_tests.barometer - DEBUG - Plugin meter categories: ['vmpage_number-free', 'vmpage_number-used'] 2018-05-08 17:18:34,048 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-08 17:18:34,048 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1525799471.37 ... 2018-05-08 17:18:34,049 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:18:34,051 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1525799471.37 ... 2018-05-08 17:18:34,052 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:18:34,052 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1525799471.37 ... 2018-05-08 17:18:34,053 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:18:34,054 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1525799471.37 ... 2018-05-08 17:18:34,057 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:18:34,058 - baro_tests.barometer - INFO - OK 2018-05-08 17:18:34,058 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-05-08 17:18:46,066 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-05-08 17:19:00,743 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-08 17:19:00,744 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1525799471.371, 1525799481.371), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1525799471.371, 1525799481.371), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1525799471.371, 1525799481.371), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1525799471.371, 1525799481.371)] 2018-05-08 17:19:00,744 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1525799501.371, 1525799511.37), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1525799501.371, 1525799511.37), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1525799501.371, 1525799511.371), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1525799501.371, 1525799511.371)] 2018-05-08 17:19:00,745 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1525799471.37 - hugepages-mm-2048Kb vmpage_number-free 1525799501.37 ... 2018-05-08 17:19:00,745 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:19:00,745 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1525799471.37 - hugepages-mm-2048Kb vmpage_number-used 1525799501.37 ... 2018-05-08 17:19:00,745 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:19:00,746 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1525799471.37 - hugepages-node0-2048Kb vmpage_number-free 1525799501.37 ... 2018-05-08 17:19:00,746 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:19:00,746 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1525799471.37 - hugepages-node0-2048Kb vmpage_number-used 1525799501.37 ... 2018-05-08 17:19:00,746 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:19:00,746 - baro_tests.barometer - INFO - OK 2018-05-08 17:19:00,746 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Hugepages. 2018-05-08 17:19:56,385 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Intel RDT Test case execution === 2018-05-08 17:20:35,392 - baro_tests.barometer - INFO - Getting CSV metrics of plugin intel_rdt on compute node 02eaf396-82ac-4651-8a3c-8d85b1b80402... 2018-05-08 17:20:35,392 - baro_tests.barometer - DEBUG - Interval: 10 2018-05-08 17:20:35,393 - baro_tests.barometer - DEBUG - Plugin subdirs: ['intel_rdt-0-2'] 2018-05-08 17:20:35,393 - baro_tests.barometer - DEBUG - Plugin meter categories: ['bytes-llc', 'ipc'] 2018-05-08 17:20:49,477 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-08 17:20:49,478 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1525799601.37 ... 2018-05-08 17:20:49,478 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:20:49,478 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1525799601.37 ... 2018-05-08 17:20:49,479 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:20:49,480 - baro_tests.barometer - INFO - OK 2018-05-08 17:20:49,480 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-05-08 17:21:01,490 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-05-08 17:21:14,727 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-08 17:21:14,727 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1525799601.371, 1525799611.37), ('intel_rdt-0-2', 'ipc', 1525799601.371, 1525799611.37)] 2018-05-08 17:21:14,729 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1525799631.37, 1525799641.37), ('intel_rdt-0-2', 'ipc', 1525799631.37, 1525799641.37)] 2018-05-08 17:21:14,731 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1525799601.37 - intel_rdt-0-2 bytes-llc 1525799631.37 ... 2018-05-08 17:21:14,733 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:21:14,733 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1525799601.37 - intel_rdt-0-2 ipc 1525799631.37 ... 2018-05-08 17:21:14,734 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:21:14,734 - baro_tests.barometer - INFO - OK 2018-05-08 17:21:14,734 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Intel RDT. 2018-05-08 17:22:07,843 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-05-08 17:22:33,696 - baro_tests.barometer - INFO - Getting CSV metrics of plugin mcelog on compute node 02eaf396-82ac-4651-8a3c-8d85b1b80402... 2018-05-08 17:22:33,699 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-08 17:22:33,699 - baro_tests.barometer - DEBUG - Plugin subdirs: ['mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'mcelog-SOCKET_0_CHANNEL_any_DIMM_any'] 2018-05-08 17:22:33,701 - baro_tests.barometer - DEBUG - Plugin meter categories: ['errors-corrected_memory_errors', 'errors-uncorrected_memory_errors'] 2018-05-08 17:22:47,431 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-08 17:22:47,431 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1525799737.37 ... 2018-05-08 17:22:47,432 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:22:47,432 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1525799737.37 ... 2018-05-08 17:22:47,432 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:22:47,432 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1525799737.37 ... 2018-05-08 17:22:47,433 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:22:47,433 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1525799737.37 ... 2018-05-08 17:22:47,433 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:22:47,433 - baro_tests.barometer - INFO - OK 2018-05-08 17:22:47,433 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-08 17:22:50,434 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-05-08 17:23:04,190 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-08 17:23:04,190 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1525799737.372, 1525799738.372), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1525799737.372, 1525799738.372), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1525799737.372, 1525799738.372), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1525799737.372, 1525799739.371)] 2018-05-08 17:23:04,190 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1525799754.372, 1525799755.372), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1525799754.372, 1525799755.372), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1525799754.372, 1525799755.372), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1525799754.372, 1525799755.372)] 2018-05-08 17:23:04,191 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1525799737.37 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1525799754.37 ... 2018-05-08 17:23:04,192 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:23:04,192 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1525799737.37 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1525799754.37 ... 2018-05-08 17:23:04,193 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:23:04,193 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1525799737.37 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1525799754.37 ... 2018-05-08 17:23:04,193 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:23:04,193 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1525799737.37 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1525799754.37 ... 2018-05-08 17:23:04,193 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:23:04,193 - baro_tests.barometer - INFO - OK 2018-05-08 17:23:04,193 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Mcelog. 2018-05-08 17:23:56,648 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-05-08 17:24:23,593 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_events on compute node 02eaf396-82ac-4651-8a3c-8d85b1b80402... 2018-05-08 17:24:23,594 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-08 17:24:23,595 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_events-br0'] 2018-05-08 17:24:23,596 - baro_tests.barometer - DEBUG - Plugin meter categories: ['gauge-link_status'] 2018-05-08 17:24:37,190 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-08 17:24:37,190 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1525799847.37 ... 2018-05-08 17:24:37,191 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:24:37,191 - baro_tests.barometer - INFO - OK 2018-05-08 17:24:37,193 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-08 17:24:40,196 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-05-08 17:24:53,368 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-08 17:24:53,369 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1525799847.372, 1525799848.372)] 2018-05-08 17:24:53,369 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1525799863.372, 1525799864.372)] 2018-05-08 17:24:53,369 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1525799847.37 - ovs_events-br0 gauge-link_status 1525799863.37 ... 2018-05-08 17:24:53,370 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:24:53,370 - baro_tests.barometer - INFO - OK 2018-05-08 17:24:53,370 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on OVS events. 2018-05-08 17:25:46,002 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS stats Test case execution === 2018-05-08 17:26:12,501 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_stats on compute node 02eaf396-82ac-4651-8a3c-8d85b1b80402... 2018-05-08 17:26:12,502 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-08 17:26:12,502 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_stats-br0.br0'] 2018-05-08 17:26:12,502 - baro_tests.barometer - DEBUG - Plugin meter categories: ['if_dropped', 'if_errors', 'if_packets'] 2018-05-08 17:26:26,603 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-08 17:26:26,604 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1525799956.37 ... 2018-05-08 17:26:26,604 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:26:26,604 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1525799956.37 ... 2018-05-08 17:26:26,604 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:26:26,605 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1525799957.37 ... 2018-05-08 17:26:26,605 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:26:26,605 - baro_tests.barometer - INFO - OK 2018-05-08 17:26:26,605 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-08 17:26:29,608 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-05-08 17:26:44,513 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-08 17:26:44,515 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1525799956.372, 1525799957.372), ('ovs_stats-br0.br0', 'if_errors', 1525799956.372, 1525799958.372), ('ovs_stats-br0.br0', 'if_packets', 1525799957.372, 1525799958.372)] 2018-05-08 17:26:44,515 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1525799974.372, 1525799975.372), ('ovs_stats-br0.br0', 'if_errors', 1525799974.372, 1525799975.372), ('ovs_stats-br0.br0', 'if_packets', 1525799975.372, 1525799976.372)] 2018-05-08 17:26:44,515 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1525799956.37 - ovs_stats-br0.br0 if_dropped 1525799974.37 ... 2018-05-08 17:26:44,515 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:26:44,516 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1525799956.37 - ovs_stats-br0.br0 if_errors 1525799974.37 ... 2018-05-08 17:26:44,516 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:26:44,516 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1525799957.37 - ovs_stats-br0.br0 if_packets 1525799975.37 ... 2018-05-08 17:26:44,516 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:26:44,517 - baro_tests.barometer - INFO - OK 2018-05-08 17:26:44,518 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on OVS stats. 2018-05-08 17:26:58,230 - baro_tests.barometer - INFO - Gnocchi plugin available in compute node overcloud-novacompute-1 2018-05-08 17:27:11,695 - baro_tests.barometer - INFO - AODH plugin present in compute node overcloud-novacompute-1 2018-05-08 17:27:11,696 - baro_tests.barometer - INFO - === NODE overcloud-novacompute-1: Enabling Test Plug-in and Test case execution === 2018-05-08 17:28:40,107 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Hugepages Test case execution === 2018-05-08 17:29:07,113 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-08 17:29:44,312 - baro_tests.barometer - INFO - Plugin Interval is 10 2018-05-08 17:29:44,312 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-05-08 17:29:44,313 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on Hugepages. 2018-05-08 17:30:37,870 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Intel RDT Test case execution === 2018-05-08 17:31:18,699 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-08 17:31:56,074 - baro_tests.barometer - INFO - Plugin Interval is 10 2018-05-08 17:31:56,075 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-05-08 17:31:56,075 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on Intel RDT. 2018-05-08 17:32:51,044 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-05-08 17:33:17,065 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-08 17:33:54,559 - baro_tests.barometer - INFO - PASS 2018-05-08 17:33:54,560 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with Gnocchi PASSED on Mcelog. 2018-05-08 17:34:47,148 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-05-08 17:35:12,885 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-08 17:35:50,062 - baro_tests.barometer - INFO - PASS 2018-05-08 17:35:50,062 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with Gnocchi PASSED on OVS events. 2018-05-08 17:36:43,475 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS stats Test case execution === 2018-05-08 17:37:10,513 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-08 17:37:13,292 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-05-08 17:37:16,896 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-1 2018-05-08 17:37:19,767 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on OVS stats. 2018-05-08 17:38:13,632 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-05-08 17:38:38,793 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-05-08 17:38:57,776 - baro_tests.barometer - INFO - Data not updated after interval of 12 seconds 2018-05-08 17:38:57,777 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with AODH FAILED on Mcelog. 2018-05-08 17:39:52,510 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-05-08 17:40:20,666 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-05-08 17:40:23,070 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-0 2018-05-08 17:40:25,659 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-1 2018-05-08 17:40:29,088 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with AODH FAILED on OVS events. 2018-05-08 17:40:29,088 - baro_tests.barometer - INFO - ======= Node overcloud-novacompute-1: Executing CSV Testcases ======== 2018-05-08 17:40:29,089 - baro_tests.barometer - INFO - Restarting collectd for CSV tests 2018-05-08 17:40:42,526 - baro_tests.barometer - INFO - Stopping collectd service... 2018-05-08 17:40:53,562 - baro_tests.barometer - INFO - Starting collectd service... 2018-05-08 17:41:03,912 - baro_tests.barometer - INFO - Sleeping for 10 seconds after collectd restart... 2018-05-08 17:42:07,068 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Hugepages Test case execution === 2018-05-08 17:42:34,980 - baro_tests.barometer - INFO - Getting CSV metrics of plugin hugepages on compute node 696a8fa6-e3a3-48a6-b86a-fc25705e5a56... 2018-05-08 17:42:34,980 - baro_tests.barometer - DEBUG - Interval: 10 2018-05-08 17:42:34,981 - baro_tests.barometer - DEBUG - Plugin subdirs: ['hugepages-mm-2048Kb', 'hugepages-node0-2048Kb'] 2018-05-08 17:42:34,982 - baro_tests.barometer - DEBUG - Plugin meter categories: ['vmpage_number-free', 'vmpage_number-used'] 2018-05-08 17:42:49,198 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-08 17:42:49,199 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1525800922.47 ... 2018-05-08 17:42:49,200 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:42:49,201 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1525800922.47 ... 2018-05-08 17:42:49,201 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:42:49,202 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1525800922.47 ... 2018-05-08 17:42:49,203 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:42:49,205 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1525800922.47 ... 2018-05-08 17:42:49,205 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:42:49,205 - baro_tests.barometer - INFO - OK 2018-05-08 17:42:49,205 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-05-08 17:43:01,217 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-05-08 17:43:15,499 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-08 17:43:15,499 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1525800922.469, 1525800932.469), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1525800922.469, 1525800932.469), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1525800922.469, 1525800932.469), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1525800922.469, 1525800932.469)] 2018-05-08 17:43:15,499 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1525800952.469, 1525800962.469), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1525800952.469, 1525800962.469), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1525800952.469, 1525800962.469), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1525800952.469, 1525800962.469)] 2018-05-08 17:43:15,499 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1525800922.47 - hugepages-mm-2048Kb vmpage_number-free 1525800952.47 ... 2018-05-08 17:43:15,500 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:43:15,500 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1525800922.47 - hugepages-mm-2048Kb vmpage_number-used 1525800952.47 ... 2018-05-08 17:43:15,500 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:43:15,500 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1525800922.47 - hugepages-node0-2048Kb vmpage_number-free 1525800952.47 ... 2018-05-08 17:43:15,500 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:43:15,500 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1525800922.47 - hugepages-node0-2048Kb vmpage_number-used 1525800952.47 ... 2018-05-08 17:43:15,500 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:43:15,501 - baro_tests.barometer - INFO - OK 2018-05-08 17:43:15,501 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Hugepages. 2018-05-08 17:44:08,028 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Intel RDT Test case execution === 2018-05-08 17:44:47,617 - baro_tests.barometer - INFO - Getting CSV metrics of plugin intel_rdt on compute node 696a8fa6-e3a3-48a6-b86a-fc25705e5a56... 2018-05-08 17:44:47,618 - baro_tests.barometer - DEBUG - Interval: 10 2018-05-08 17:44:47,619 - baro_tests.barometer - DEBUG - Plugin subdirs: ['intel_rdt-0-2'] 2018-05-08 17:44:47,620 - baro_tests.barometer - DEBUG - Plugin meter categories: ['bytes-llc', 'ipc'] 2018-05-08 17:45:00,761 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-08 17:45:00,762 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1525801052.47 ... 2018-05-08 17:45:00,763 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:45:00,766 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1525801052.47 ... 2018-05-08 17:45:00,769 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:45:00,770 - baro_tests.barometer - INFO - OK 2018-05-08 17:45:00,772 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-05-08 17:45:12,783 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-05-08 17:45:26,829 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-08 17:45:26,831 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1525801052.469, 1525801062.469), ('intel_rdt-0-2', 'ipc', 1525801052.469, 1525801062.469)] 2018-05-08 17:45:26,833 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1525801082.469, 1525801092.469), ('intel_rdt-0-2', 'ipc', 1525801082.469, 1525801092.469)] 2018-05-08 17:45:26,834 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1525801052.47 - intel_rdt-0-2 bytes-llc 1525801082.47 ... 2018-05-08 17:45:26,835 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:45:26,835 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1525801052.47 - intel_rdt-0-2 ipc 1525801082.47 ... 2018-05-08 17:45:26,836 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:45:26,836 - baro_tests.barometer - INFO - OK 2018-05-08 17:45:26,837 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Intel RDT. 2018-05-08 17:46:16,892 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-05-08 17:46:42,457 - baro_tests.barometer - INFO - Getting CSV metrics of plugin mcelog on compute node 696a8fa6-e3a3-48a6-b86a-fc25705e5a56... 2018-05-08 17:46:42,458 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-08 17:46:42,458 - baro_tests.barometer - DEBUG - Plugin subdirs: ['mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'mcelog-SOCKET_0_CHANNEL_any_DIMM_any'] 2018-05-08 17:46:42,460 - baro_tests.barometer - DEBUG - Plugin meter categories: ['errors-corrected_memory_errors', 'errors-uncorrected_memory_errors'] 2018-05-08 17:46:56,771 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-08 17:46:56,772 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1525801183.47 ... 2018-05-08 17:46:56,772 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:46:56,772 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1525801183.47 ... 2018-05-08 17:46:56,773 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:46:56,773 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1525801183.47 ... 2018-05-08 17:46:56,773 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:46:56,774 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1525801183.47 ... 2018-05-08 17:46:56,774 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:46:56,774 - baro_tests.barometer - INFO - OK 2018-05-08 17:46:56,775 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-08 17:46:59,778 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-05-08 17:47:13,433 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-08 17:47:13,434 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1525801183.47, 1525801184.47), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1525801183.47, 1525801184.47), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1525801183.47, 1525801184.47), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1525801183.47, 1525801184.47)] 2018-05-08 17:47:13,434 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1525801200.47, 1525801201.47), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1525801200.47, 1525801201.47), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1525801200.47, 1525801201.47), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1525801200.47, 1525801201.47)] 2018-05-08 17:47:13,434 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1525801183.47 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1525801200.47 ... 2018-05-08 17:47:13,434 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:47:13,435 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1525801183.47 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1525801200.47 ... 2018-05-08 17:47:13,435 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:47:13,435 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1525801183.47 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1525801200.47 ... 2018-05-08 17:47:13,435 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:47:13,435 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1525801183.47 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1525801200.47 ... 2018-05-08 17:47:13,436 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:47:13,436 - baro_tests.barometer - INFO - OK 2018-05-08 17:47:13,436 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Mcelog. 2018-05-08 17:48:05,404 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-05-08 17:48:31,520 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_events on compute node 696a8fa6-e3a3-48a6-b86a-fc25705e5a56... 2018-05-08 17:48:31,521 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-08 17:48:31,522 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_events-br0'] 2018-05-08 17:48:31,522 - baro_tests.barometer - DEBUG - Plugin meter categories: ['gauge-link_status'] 2018-05-08 17:48:44,989 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-08 17:48:44,989 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1525801292.47 ... 2018-05-08 17:48:44,990 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:48:44,990 - baro_tests.barometer - INFO - OK 2018-05-08 17:48:44,990 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-08 17:48:47,994 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-05-08 17:49:00,927 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-08 17:49:00,927 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1525801292.47, 1525801293.47)] 2018-05-08 17:49:00,928 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1525801307.47, 1525801309.47)] 2018-05-08 17:49:00,928 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1525801292.47 - ovs_events-br0 gauge-link_status 1525801307.47 ... 2018-05-08 17:49:00,929 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:49:00,930 - baro_tests.barometer - INFO - OK 2018-05-08 17:49:00,930 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on OVS events. 2018-05-08 17:49:53,603 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS stats Test case execution === 2018-05-08 17:50:20,674 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_stats on compute node 696a8fa6-e3a3-48a6-b86a-fc25705e5a56... 2018-05-08 17:50:20,674 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-08 17:50:20,674 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_stats-br0.br0'] 2018-05-08 17:50:20,675 - baro_tests.barometer - DEBUG - Plugin meter categories: ['if_dropped', 'if_errors', 'if_packets'] 2018-05-08 17:50:34,672 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-08 17:50:34,672 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1525801401.47 ... 2018-05-08 17:50:34,673 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:50:34,673 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1525801401.47 ... 2018-05-08 17:50:34,675 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:50:34,675 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1525801401.47 ... 2018-05-08 17:50:34,675 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:50:34,675 - baro_tests.barometer - INFO - OK 2018-05-08 17:50:34,676 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-08 17:50:37,679 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-05-08 17:50:51,338 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-08 17:50:51,338 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1525801401.47, 1525801402.47), ('ovs_stats-br0.br0', 'if_errors', 1525801401.47, 1525801402.47), ('ovs_stats-br0.br0', 'if_packets', 1525801401.47, 1525801402.47)] 2018-05-08 17:50:51,339 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1525801418.47, 1525801419.47), ('ovs_stats-br0.br0', 'if_errors', 1525801418.47, 1525801419.47), ('ovs_stats-br0.br0', 'if_packets', 1525801418.47, 1525801419.47)] 2018-05-08 17:50:51,339 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1525801401.47 - ovs_stats-br0.br0 if_dropped 1525801418.47 ... 2018-05-08 17:50:51,339 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:50:51,339 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1525801401.47 - ovs_stats-br0.br0 if_errors 1525801418.47 ... 2018-05-08 17:50:51,339 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:50:51,340 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1525801401.47 - ovs_stats-br0.br0 if_packets 1525801418.47 ... 2018-05-08 17:50:51,340 - baro_tests.barometer - DEBUG - OK 2018-05-08 17:50:51,340 - baro_tests.barometer - INFO - OK 2018-05-08 17:50:51,340 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on OVS stats. 2018-05-08 17:51:05,208 - baro_tests.barometer - INFO - Mcelog is deleted from all compute nodes 2018-05-08 17:51:05,209 - baro_tests.barometer - INFO - ====================================================================== 2018-05-08 17:51:05,211 - baro_tests.barometer - INFO - +----------------------------------+ 2018-05-08 17:51:05,213 - baro_tests.barometer - INFO - | OVERALL SUMMARY | 2018-05-08 17:51:05,215 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-05-08 17:51:05,218 - baro_tests.barometer - INFO - | Test | Node-0 | Node-1 | 2018-05-08 17:51:05,219 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-05-08 17:51:05,222 - baro_tests.barometer - INFO - | OUT:Gnocchi | PASS | PASS | 2018-05-08 17:51:05,227 - baro_tests.barometer - INFO - | IN:Hugepages | FAIL | FAIL | 2018-05-08 17:51:05,230 - baro_tests.barometer - INFO - | IN:Intel RDT | PASS | FAIL | 2018-05-08 17:51:05,231 - baro_tests.barometer - INFO - | IN:Mcelog | FAIL | PASS | 2018-05-08 17:51:05,232 - baro_tests.barometer - INFO - | IN:OVS events | FAIL | PASS | 2018-05-08 17:51:05,236 - baro_tests.barometer - INFO - | IN:OVS stats | PASS | FAIL | 2018-05-08 17:51:05,237 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-05-08 17:51:05,238 - baro_tests.barometer - INFO - | OUT:AODH | PASS | PASS | 2018-05-08 17:51:05,239 - baro_tests.barometer - INFO - | IN:Mcelog | PASS | FAIL | 2018-05-08 17:51:05,240 - baro_tests.barometer - INFO - | IN:OVS events | PASS | SKIP | 2018-05-08 17:51:05,240 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-05-08 17:51:05,241 - baro_tests.barometer - INFO - | OUT:CSV | PASS | PASS | 2018-05-08 17:51:05,241 - baro_tests.barometer - INFO - | IN:Hugepages | PASS | PASS | 2018-05-08 17:51:05,242 - baro_tests.barometer - INFO - | IN:Intel RDT | PASS | PASS | 2018-05-08 17:51:05,242 - baro_tests.barometer - INFO - | IN:Mcelog | PASS | PASS | 2018-05-08 17:51:05,243 - baro_tests.barometer - INFO - | IN:OVS events | PASS | PASS | 2018-05-08 17:51:05,243 - baro_tests.barometer - INFO - | IN:OVS stats | PASS | PASS | 2018-05-08 17:51:05,244 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-05-08 17:51:05,244 - baro_tests.barometer - INFO - ====================================================================== 2018-05-08 17:51:05,245 - baro_tests.barometer - ERROR - Some tests have failed or have not been executed 2018-05-08 17:51:05,248 - baro_tests.barometer - ERROR - Overall Result is Fail