2018-04-27 00:28:42,734 - baro_tests.barometer - INFO - =================== Starting barometer tests suite =================== 2018-04-27 00:28:42,735 - baro_tests.barometer - INFO - RSA key file /root/.ssh/id_rsa exists. 2018-04-27 00:28:46,816 - baro_tests.barometer - INFO - ==== Display of Control and Compute nodes available in the set up ==== 2018-04-27 00:28:46,817 - baro_tests.barometer - INFO - controllers: ['overcloud-controller-0: 192.30.9.8', 'overcloud-controller-1: 192.30.9.6', 'overcloud-controller-2: 192.30.9.10'] 2018-04-27 00:28:46,817 - baro_tests.barometer - INFO - computes: ['overcloud-novacompute-0: 192.30.9.5', 'overcloud-novacompute-1: 192.30.9.4'] 2018-04-27 00:28:46,817 - baro_tests.barometer - INFO - ========== Enabling mcelog and OVS bridges on compute nodes ========== 2018-04-27 00:29:01,838 - baro_tests.barometer - INFO - Checking if mcelog is enabled on node-overcloud-novacompute-0... 2018-04-27 00:29:01,890 - baro_tests.barometer - INFO - Mcelog will be enabled on node-overcloud-novacompute-0... 2018-04-27 00:29:02,268 - baro_tests.barometer - INFO - Mcelog was installed on node-overcloud-novacompute-0. 2018-04-27 00:29:02,280 - baro_tests.barometer - INFO - Checking if mcelog is enabled on node-overcloud-novacompute-1... 2018-04-27 00:29:02,331 - baro_tests.barometer - INFO - Mcelog will be enabled on node-overcloud-novacompute-1... 2018-04-27 00:29:02,702 - baro_tests.barometer - INFO - Mcelog was installed on node-overcloud-novacompute-1. 2018-04-27 00:29:16,718 - baro_tests.barometer - INFO - Creating OVS bridges on computes nodes 2018-04-27 00:29:16,968 - baro_tests.barometer - INFO - OVS Bridges created on compute nodes 2018-04-27 00:29:16,968 - baro_tests.barometer - INFO - ============= Testing Gnocchi and AODH plugins on nodes ============== 2018-04-27 00:31:06,758 - baro_tests.barometer - INFO - Gnocchi plugin available in compute node overcloud-novacompute-0 2018-04-27 00:31:20,963 - baro_tests.barometer - INFO - AODH plugin present in compute node overcloud-novacompute-0 2018-04-27 00:31:20,963 - baro_tests.barometer - INFO - === NODE overcloud-novacompute-0: Enabling Test Plug-in and Test case execution === 2018-04-27 00:32:47,997 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Hugepages Test case execution === 2018-04-27 00:33:14,463 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-27 00:33:52,833 - baro_tests.barometer - INFO - PASS 2018-04-27 00:33:52,834 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with Gnocchi PASSED on Hugepages. 2018-04-27 00:34:46,175 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Intel RDT Test case execution === 2018-04-27 00:35:24,904 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-27 00:35:27,586 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-04-27 00:35:30,394 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-1 2018-04-27 00:35:33,076 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on Intel RDT. 2018-04-27 00:36:23,876 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-04-27 00:36:47,943 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-27 00:37:25,163 - baro_tests.barometer - INFO - PASS 2018-04-27 00:37:25,164 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with Gnocchi PASSED on Mcelog. 2018-04-27 00:38:18,576 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-04-27 00:38:44,831 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-27 00:39:21,962 - baro_tests.barometer - INFO - Plugin Interval is 1 2018-04-27 00:39:21,963 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-04-27 00:39:21,963 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on OVS events. 2018-04-27 00:40:14,289 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS stats Test case execution === 2018-04-27 00:40:40,055 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-27 00:40:42,768 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-04-27 00:40:45,355 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-1 2018-04-27 00:40:47,967 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on OVS stats. 2018-04-27 00:41:39,000 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-04-27 00:42:05,417 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-04-27 00:42:24,403 - baro_tests.barometer - INFO - Data not updated after interval of 12 seconds 2018-04-27 00:42:24,404 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with AODH FAILED on Mcelog. 2018-04-27 00:43:15,533 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-04-27 00:43:41,059 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-04-27 00:43:43,394 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-0 2018-04-27 00:43:45,808 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-1 2018-04-27 00:43:49,262 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with AODH FAILED on OVS events. 2018-04-27 00:43:49,263 - baro_tests.barometer - INFO - ======= Node overcloud-novacompute-0: Executing CSV Testcases ======== 2018-04-27 00:43:49,263 - baro_tests.barometer - INFO - Restarting collectd for CSV tests 2018-04-27 00:44:01,375 - baro_tests.barometer - INFO - Stopping collectd service... 2018-04-27 00:44:12,419 - baro_tests.barometer - INFO - Starting collectd service... 2018-04-27 00:44:23,160 - baro_tests.barometer - INFO - Sleeping for 10 seconds after collectd restart... 2018-04-27 00:45:26,231 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Hugepages Test case execution === 2018-04-27 00:45:51,251 - baro_tests.barometer - INFO - Getting CSV metrics of plugin hugepages on compute node 0c5c0b20-e05e-4d46-a47f-4a89608ea4db... 2018-04-27 00:45:51,251 - baro_tests.barometer - DEBUG - Interval: 10 2018-04-27 00:45:51,252 - baro_tests.barometer - DEBUG - Plugin subdirs: ['hugepages-mm-2048Kb', 'hugepages-node0-2048Kb'] 2018-04-27 00:45:51,252 - baro_tests.barometer - DEBUG - Plugin meter categories: ['vmpage_number-free', 'vmpage_number-used'] 2018-04-27 00:46:05,093 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-27 00:46:05,093 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1524789540.95 ... 2018-04-27 00:46:05,094 - baro_tests.barometer - DEBUG - OK 2018-04-27 00:46:05,094 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1524789540.95 ... 2018-04-27 00:46:05,094 - baro_tests.barometer - DEBUG - OK 2018-04-27 00:46:05,095 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1524789540.95 ... 2018-04-27 00:46:05,095 - baro_tests.barometer - DEBUG - OK 2018-04-27 00:46:05,095 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1524789540.95 ... 2018-04-27 00:46:05,095 - baro_tests.barometer - DEBUG - OK 2018-04-27 00:46:05,095 - baro_tests.barometer - INFO - OK 2018-04-27 00:46:05,096 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-04-27 00:46:17,106 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-27 00:46:30,812 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-27 00:46:30,813 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1524789540.95, 1524789550.951), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1524789540.95, 1524789550.951), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1524789540.951, 1524789550.951), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1524789540.951, 1524789550.951)] 2018-04-27 00:46:30,814 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1524789560.951, 1524789570.95), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1524789560.951, 1524789570.95), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1524789560.951, 1524789570.951), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1524789560.951, 1524789570.951)] 2018-04-27 00:46:30,814 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1524789540.95 - hugepages-mm-2048Kb vmpage_number-free 1524789560.95 ... 2018-04-27 00:46:30,814 - baro_tests.barometer - DEBUG - OK 2018-04-27 00:46:30,814 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1524789540.95 - hugepages-mm-2048Kb vmpage_number-used 1524789560.95 ... 2018-04-27 00:46:30,815 - baro_tests.barometer - DEBUG - OK 2018-04-27 00:46:30,815 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1524789540.95 - hugepages-node0-2048Kb vmpage_number-free 1524789560.95 ... 2018-04-27 00:46:30,815 - baro_tests.barometer - DEBUG - OK 2018-04-27 00:46:30,815 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1524789540.95 - hugepages-node0-2048Kb vmpage_number-used 1524789560.95 ... 2018-04-27 00:46:30,815 - baro_tests.barometer - DEBUG - OK 2018-04-27 00:46:30,815 - baro_tests.barometer - INFO - OK 2018-04-27 00:46:30,815 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Hugepages. 2018-04-27 00:47:21,340 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Intel RDT Test case execution === 2018-04-27 00:47:59,394 - baro_tests.barometer - INFO - Getting CSV metrics of plugin intel_rdt on compute node 0c5c0b20-e05e-4d46-a47f-4a89608ea4db... 2018-04-27 00:47:59,396 - baro_tests.barometer - DEBUG - Interval: 10 2018-04-27 00:47:59,397 - baro_tests.barometer - DEBUG - Plugin subdirs: ['intel_rdt-0-2'] 2018-04-27 00:47:59,397 - baro_tests.barometer - DEBUG - Plugin meter categories: ['bytes-llc', 'ipc'] 2018-04-27 00:48:12,785 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-27 00:48:12,785 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1524789660.95 ... 2018-04-27 00:48:12,785 - baro_tests.barometer - DEBUG - OK 2018-04-27 00:48:12,785 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1524789660.95 ... 2018-04-27 00:48:12,786 - baro_tests.barometer - DEBUG - OK 2018-04-27 00:48:12,786 - baro_tests.barometer - INFO - OK 2018-04-27 00:48:12,786 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-04-27 00:48:24,792 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-27 00:48:38,716 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-27 00:48:38,717 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1524789660.95, 1524789670.95), ('intel_rdt-0-2', 'ipc', 1524789660.95, 1524789670.95)] 2018-04-27 00:48:38,717 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1524789690.95, 1524789700.95), ('intel_rdt-0-2', 'ipc', 1524789690.95, 1524789700.95)] 2018-04-27 00:48:38,717 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1524789660.95 - intel_rdt-0-2 bytes-llc 1524789690.95 ... 2018-04-27 00:48:38,718 - baro_tests.barometer - DEBUG - OK 2018-04-27 00:48:38,718 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1524789660.95 - intel_rdt-0-2 ipc 1524789690.95 ... 2018-04-27 00:48:38,718 - baro_tests.barometer - DEBUG - OK 2018-04-27 00:48:38,718 - baro_tests.barometer - INFO - OK 2018-04-27 00:48:38,719 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Intel RDT. 2018-04-27 00:49:29,556 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-04-27 00:49:53,744 - baro_tests.barometer - INFO - Getting CSV metrics of plugin mcelog on compute node 0c5c0b20-e05e-4d46-a47f-4a89608ea4db... 2018-04-27 00:49:53,744 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-27 00:49:53,745 - baro_tests.barometer - DEBUG - Plugin subdirs: ['mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'mcelog-SOCKET_0_CHANNEL_any_DIMM_any'] 2018-04-27 00:49:53,745 - baro_tests.barometer - DEBUG - Plugin meter categories: ['errors-corrected_memory_errors', 'errors-uncorrected_memory_errors'] 2018-04-27 00:50:06,250 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-27 00:50:06,251 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1524789791.95 ... 2018-04-27 00:50:06,251 - baro_tests.barometer - DEBUG - OK 2018-04-27 00:50:06,251 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1524789791.95 ... 2018-04-27 00:50:06,252 - baro_tests.barometer - DEBUG - OK 2018-04-27 00:50:06,252 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1524789791.95 ... 2018-04-27 00:50:06,252 - baro_tests.barometer - DEBUG - OK 2018-04-27 00:50:06,252 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1524789792.95 ... 2018-04-27 00:50:06,252 - baro_tests.barometer - DEBUG - OK 2018-04-27 00:50:06,253 - baro_tests.barometer - INFO - OK 2018-04-27 00:50:06,253 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-27 00:50:09,256 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-27 00:50:22,053 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-27 00:50:22,053 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1524789791.951, 1524789792.951), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1524789791.951, 1524789792.951), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1524789791.951, 1524789792.951), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1524789792.951, 1524789793.952)] 2018-04-27 00:50:22,054 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1524789807.952, 1524789808.951), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1524789807.952, 1524789808.951), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1524789807.952, 1524789808.951), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1524789807.952, 1524789808.951)] 2018-04-27 00:50:22,054 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1524789791.95 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1524789807.95 ... 2018-04-27 00:50:22,054 - baro_tests.barometer - DEBUG - OK 2018-04-27 00:50:22,054 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1524789791.95 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1524789807.95 ... 2018-04-27 00:50:22,054 - baro_tests.barometer - DEBUG - OK 2018-04-27 00:50:22,054 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1524789791.95 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1524789807.95 ... 2018-04-27 00:50:22,054 - baro_tests.barometer - DEBUG - OK 2018-04-27 00:50:22,054 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1524789792.95 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1524789807.95 ... 2018-04-27 00:50:22,055 - baro_tests.barometer - DEBUG - OK 2018-04-27 00:50:22,055 - baro_tests.barometer - INFO - OK 2018-04-27 00:50:22,055 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Mcelog. 2018-04-27 00:51:11,719 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-04-27 00:51:37,202 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_events on compute node 0c5c0b20-e05e-4d46-a47f-4a89608ea4db... 2018-04-27 00:51:37,202 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-27 00:51:37,203 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_events-br0'] 2018-04-27 00:51:37,203 - baro_tests.barometer - DEBUG - Plugin meter categories: ['gauge-link_status'] 2018-04-27 00:51:49,179 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-27 00:51:49,180 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1524789894.95 ... 2018-04-27 00:51:49,180 - baro_tests.barometer - DEBUG - OK 2018-04-27 00:51:49,180 - baro_tests.barometer - INFO - OK 2018-04-27 00:51:49,181 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-27 00:51:52,184 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-27 00:52:05,177 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-27 00:52:05,178 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1524789894.952, 1524789895.952)] 2018-04-27 00:52:05,178 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1524789910.952, 1524789911.952)] 2018-04-27 00:52:05,178 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1524789894.95 - ovs_events-br0 gauge-link_status 1524789910.95 ... 2018-04-27 00:52:05,179 - baro_tests.barometer - DEBUG - OK 2018-04-27 00:52:05,179 - baro_tests.barometer - INFO - OK 2018-04-27 00:52:05,179 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on OVS events. 2018-04-27 00:52:54,507 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS stats Test case execution === 2018-04-27 00:53:19,358 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_stats on compute node 0c5c0b20-e05e-4d46-a47f-4a89608ea4db... 2018-04-27 00:53:19,358 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-27 00:53:19,358 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_stats-br0.br0'] 2018-04-27 00:53:19,359 - baro_tests.barometer - DEBUG - Plugin meter categories: ['if_dropped', 'if_errors', 'if_packets'] 2018-04-27 00:53:31,947 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-27 00:53:31,948 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1524789997.95 ... 2018-04-27 00:53:31,948 - baro_tests.barometer - DEBUG - OK 2018-04-27 00:53:31,948 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1524789997.95 ... 2018-04-27 00:53:31,948 - baro_tests.barometer - DEBUG - OK 2018-04-27 00:53:31,948 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1524789997.95 ... 2018-04-27 00:53:31,949 - baro_tests.barometer - DEBUG - OK 2018-04-27 00:53:31,949 - baro_tests.barometer - INFO - OK 2018-04-27 00:53:31,949 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-27 00:53:34,950 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-27 00:53:48,333 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-27 00:53:48,334 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1524789997.951, 1524789998.951), ('ovs_stats-br0.br0', 'if_errors', 1524789997.951, 1524789998.951), ('ovs_stats-br0.br0', 'if_packets', 1524789997.951, 1524789998.951)] 2018-04-27 00:53:48,334 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1524790013.952, 1524790014.952), ('ovs_stats-br0.br0', 'if_errors', 1524790014.952, 1524790015.951), ('ovs_stats-br0.br0', 'if_packets', 1524790014.952, 1524790015.951)] 2018-04-27 00:53:48,334 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1524789997.95 - ovs_stats-br0.br0 if_dropped 1524790013.95 ... 2018-04-27 00:53:48,334 - baro_tests.barometer - DEBUG - OK 2018-04-27 00:53:48,334 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1524789997.95 - ovs_stats-br0.br0 if_errors 1524790014.95 ... 2018-04-27 00:53:48,334 - baro_tests.barometer - DEBUG - OK 2018-04-27 00:53:48,335 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1524789997.95 - ovs_stats-br0.br0 if_packets 1524790014.95 ... 2018-04-27 00:53:48,335 - baro_tests.barometer - DEBUG - OK 2018-04-27 00:53:48,335 - baro_tests.barometer - INFO - OK 2018-04-27 00:53:48,335 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on OVS stats. 2018-04-27 00:54:00,990 - baro_tests.barometer - INFO - Gnocchi plugin available in compute node overcloud-novacompute-1 2018-04-27 00:54:13,694 - baro_tests.barometer - INFO - AODH plugin present in compute node overcloud-novacompute-1 2018-04-27 00:54:13,694 - baro_tests.barometer - INFO - === NODE overcloud-novacompute-1: Enabling Test Plug-in and Test case execution === 2018-04-27 00:55:28,100 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Hugepages Test case execution === 2018-04-27 00:55:52,682 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-27 00:56:30,037 - baro_tests.barometer - INFO - Plugin Interval is 10 2018-04-27 00:56:30,037 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-04-27 00:56:30,038 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on Hugepages. 2018-04-27 00:57:20,103 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Intel RDT Test case execution === 2018-04-27 00:57:58,483 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-27 00:58:01,161 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-04-27 00:58:03,833 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-1 2018-04-27 00:58:06,451 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on Intel RDT. 2018-04-27 00:58:59,059 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-04-27 00:59:25,346 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-27 00:59:28,138 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-04-27 00:59:30,785 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-1 2018-04-27 00:59:33,475 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on Mcelog. 2018-04-27 01:00:23,590 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-04-27 01:00:49,303 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-27 01:01:26,588 - baro_tests.barometer - INFO - Plugin Interval is 1 2018-04-27 01:01:26,589 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-04-27 01:01:26,590 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on OVS events. 2018-04-27 01:02:14,882 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS stats Test case execution === 2018-04-27 01:02:39,883 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-27 01:03:17,405 - baro_tests.barometer - INFO - Plugin Interval is 1 2018-04-27 01:03:17,405 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-04-27 01:03:17,406 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on OVS stats. 2018-04-27 01:04:09,740 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-04-27 01:04:36,010 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-04-27 01:04:54,950 - baro_tests.barometer - INFO - Data not updated after interval of 12 seconds 2018-04-27 01:04:54,952 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with AODH FAILED on Mcelog. 2018-04-27 01:05:46,271 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-04-27 01:06:10,497 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-04-27 01:06:12,852 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-0 2018-04-27 01:06:15,209 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-1 2018-04-27 01:06:17,500 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with AODH FAILED on OVS events. 2018-04-27 01:06:17,503 - baro_tests.barometer - INFO - ======= Node overcloud-novacompute-1: Executing CSV Testcases ======== 2018-04-27 01:06:17,504 - baro_tests.barometer - INFO - Restarting collectd for CSV tests 2018-04-27 01:06:29,984 - baro_tests.barometer - INFO - Stopping collectd service... 2018-04-27 01:06:41,033 - baro_tests.barometer - INFO - Starting collectd service... 2018-04-27 01:06:51,349 - baro_tests.barometer - INFO - Sleeping for 10 seconds after collectd restart... 2018-04-27 01:07:53,122 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Hugepages Test case execution === 2018-04-27 01:08:20,910 - baro_tests.barometer - INFO - Getting CSV metrics of plugin hugepages on compute node 34684d50-38e1-4d8b-897b-45a564858c02... 2018-04-27 01:08:20,911 - baro_tests.barometer - DEBUG - Interval: 10 2018-04-27 01:08:20,912 - baro_tests.barometer - DEBUG - Plugin subdirs: ['hugepages-mm-2048Kb', 'hugepages-node0-2048Kb'] 2018-04-27 01:08:20,913 - baro_tests.barometer - DEBUG - Plugin meter categories: ['vmpage_number-free', 'vmpage_number-used'] 2018-04-27 01:08:34,529 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-27 01:08:34,529 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1524790881.96 ... 2018-04-27 01:08:34,530 - baro_tests.barometer - DEBUG - OK 2018-04-27 01:08:34,530 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1524790881.96 ... 2018-04-27 01:08:34,530 - baro_tests.barometer - DEBUG - OK 2018-04-27 01:08:34,531 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1524790881.96 ... 2018-04-27 01:08:34,531 - baro_tests.barometer - DEBUG - OK 2018-04-27 01:08:34,531 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1524790881.96 ... 2018-04-27 01:08:34,531 - baro_tests.barometer - DEBUG - OK 2018-04-27 01:08:34,531 - baro_tests.barometer - INFO - OK 2018-04-27 01:08:34,532 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-04-27 01:08:46,539 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-27 01:09:00,110 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-27 01:09:00,112 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1524790881.959, 1524790891.959), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1524790881.959, 1524790891.959), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1524790881.959, 1524790891.959), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1524790881.959, 1524790891.959)] 2018-04-27 01:09:00,116 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1524790901.959, 1524790911.959), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1524790901.959, 1524790911.959), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1524790901.959, 1524790911.959), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1524790901.959, 1524790911.959)] 2018-04-27 01:09:00,117 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1524790881.96 - hugepages-mm-2048Kb vmpage_number-free 1524790901.96 ... 2018-04-27 01:09:00,118 - baro_tests.barometer - DEBUG - OK 2018-04-27 01:09:00,119 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1524790881.96 - hugepages-mm-2048Kb vmpage_number-used 1524790901.96 ... 2018-04-27 01:09:00,120 - baro_tests.barometer - DEBUG - OK 2018-04-27 01:09:00,121 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1524790881.96 - hugepages-node0-2048Kb vmpage_number-free 1524790901.96 ... 2018-04-27 01:09:00,122 - baro_tests.barometer - DEBUG - OK 2018-04-27 01:09:00,122 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1524790881.96 - hugepages-node0-2048Kb vmpage_number-used 1524790901.96 ... 2018-04-27 01:09:00,122 - baro_tests.barometer - DEBUG - OK 2018-04-27 01:09:00,122 - baro_tests.barometer - INFO - OK 2018-04-27 01:09:00,123 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Hugepages. 2018-04-27 01:09:53,992 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Intel RDT Test case execution === 2018-04-27 01:10:31,534 - baro_tests.barometer - INFO - Getting CSV metrics of plugin intel_rdt on compute node 34684d50-38e1-4d8b-897b-45a564858c02... 2018-04-27 01:10:31,535 - baro_tests.barometer - DEBUG - Interval: 10 2018-04-27 01:10:31,535 - baro_tests.barometer - DEBUG - Plugin subdirs: ['intel_rdt-0-2'] 2018-04-27 01:10:31,535 - baro_tests.barometer - DEBUG - Plugin meter categories: ['bytes-llc', 'ipc'] 2018-04-27 01:10:43,539 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-27 01:10:43,540 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1524791011.96 ... 2018-04-27 01:10:43,541 - baro_tests.barometer - DEBUG - OK 2018-04-27 01:10:43,541 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1524791011.96 ... 2018-04-27 01:10:43,542 - baro_tests.barometer - DEBUG - OK 2018-04-27 01:10:43,543 - baro_tests.barometer - INFO - OK 2018-04-27 01:10:43,544 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-04-27 01:10:55,557 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-27 01:11:07,326 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-27 01:11:07,327 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1524791011.959, 1524791021.959), ('intel_rdt-0-2', 'ipc', 1524791011.959, 1524791021.959)] 2018-04-27 01:11:07,328 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1524791031.959, 1524791041.959), ('intel_rdt-0-2', 'ipc', 1524791031.959, 1524791041.959)] 2018-04-27 01:11:07,328 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1524791011.96 - intel_rdt-0-2 bytes-llc 1524791031.96 ... 2018-04-27 01:11:07,329 - baro_tests.barometer - DEBUG - OK 2018-04-27 01:11:07,329 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1524791011.96 - intel_rdt-0-2 ipc 1524791031.96 ... 2018-04-27 01:11:07,329 - baro_tests.barometer - DEBUG - OK 2018-04-27 01:11:07,329 - baro_tests.barometer - INFO - OK 2018-04-27 01:11:07,329 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Intel RDT. 2018-04-27 01:11:58,520 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-04-27 01:12:22,795 - baro_tests.barometer - INFO - Getting CSV metrics of plugin mcelog on compute node 34684d50-38e1-4d8b-897b-45a564858c02... 2018-04-27 01:12:22,796 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-27 01:12:22,796 - baro_tests.barometer - DEBUG - Plugin subdirs: ['mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'mcelog-SOCKET_0_CHANNEL_any_DIMM_any'] 2018-04-27 01:12:22,796 - baro_tests.barometer - DEBUG - Plugin meter categories: ['errors-corrected_memory_errors', 'errors-uncorrected_memory_errors'] 2018-04-27 01:12:35,184 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-27 01:12:35,185 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1524791134.05 ... 2018-04-27 01:12:35,185 - baro_tests.barometer - DEBUG - OK 2018-04-27 01:12:35,185 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1524791134.05 ... 2018-04-27 01:12:35,185 - baro_tests.barometer - DEBUG - OK 2018-04-27 01:12:35,185 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1524791134.05 ... 2018-04-27 01:12:35,185 - baro_tests.barometer - DEBUG - OK 2018-04-27 01:12:35,185 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1524791134.05 ... 2018-04-27 01:12:35,186 - baro_tests.barometer - DEBUG - OK 2018-04-27 01:12:35,187 - baro_tests.barometer - INFO - OK 2018-04-27 01:12:35,188 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-27 01:12:38,192 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-27 01:12:50,458 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-27 01:12:50,458 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1524791134.053, 1524791135.053), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1524791134.053, 1524791135.053), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1524791134.053, 1524791135.052), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1524791134.053, 1524791135.052)] 2018-04-27 01:12:50,459 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1524791149.052, 1524791150.053), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1524791149.052, 1524791150.053), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1524791149.052, 1524791150.053), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1524791149.052, 1524791151.053)] 2018-04-27 01:12:50,459 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1524791134.05 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1524791149.05 ... 2018-04-27 01:12:50,459 - baro_tests.barometer - DEBUG - OK 2018-04-27 01:12:50,459 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1524791134.05 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1524791149.05 ... 2018-04-27 01:12:50,460 - baro_tests.barometer - DEBUG - OK 2018-04-27 01:12:50,460 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1524791134.05 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1524791149.05 ... 2018-04-27 01:12:50,460 - baro_tests.barometer - DEBUG - OK 2018-04-27 01:12:50,460 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1524791134.05 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1524791149.05 ... 2018-04-27 01:12:50,461 - baro_tests.barometer - DEBUG - OK 2018-04-27 01:12:50,462 - baro_tests.barometer - INFO - OK 2018-04-27 01:12:50,462 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Mcelog. 2018-04-27 01:13:41,847 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-04-27 01:14:06,193 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_events on compute node 34684d50-38e1-4d8b-897b-45a564858c02... 2018-04-27 01:14:06,193 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-27 01:14:06,193 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_events-br0'] 2018-04-27 01:14:06,194 - baro_tests.barometer - DEBUG - Plugin meter categories: ['gauge-link_status'] 2018-04-27 01:14:18,816 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-27 01:14:18,816 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1524791238.05 ... 2018-04-27 01:14:18,816 - baro_tests.barometer - DEBUG - OK 2018-04-27 01:14:18,817 - baro_tests.barometer - INFO - OK 2018-04-27 01:14:18,817 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-27 01:14:21,820 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-27 01:14:34,512 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-27 01:14:34,514 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1524791238.053, 1524791239.053)] 2018-04-27 01:14:34,516 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1524791254.053, 1524791255.053)] 2018-04-27 01:14:34,517 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1524791238.05 - ovs_events-br0 gauge-link_status 1524791254.05 ... 2018-04-27 01:14:34,517 - baro_tests.barometer - DEBUG - OK 2018-04-27 01:14:34,518 - baro_tests.barometer - INFO - OK 2018-04-27 01:14:34,521 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on OVS events. 2018-04-27 01:15:24,348 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS stats Test case execution === 2018-04-27 01:15:48,623 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_stats on compute node 34684d50-38e1-4d8b-897b-45a564858c02... 2018-04-27 01:15:48,624 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-27 01:15:48,627 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_stats-br0.br0'] 2018-04-27 01:15:48,629 - baro_tests.barometer - DEBUG - Plugin meter categories: ['if_dropped', 'if_errors', 'if_packets'] 2018-04-27 01:16:01,343 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-27 01:16:01,344 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1524791340.05 ... 2018-04-27 01:16:01,344 - baro_tests.barometer - DEBUG - OK 2018-04-27 01:16:01,344 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1524791340.05 ... 2018-04-27 01:16:01,345 - baro_tests.barometer - DEBUG - OK 2018-04-27 01:16:01,345 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1524791340.05 ... 2018-04-27 01:16:01,345 - baro_tests.barometer - DEBUG - OK 2018-04-27 01:16:01,346 - baro_tests.barometer - INFO - OK 2018-04-27 01:16:01,346 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-27 01:16:04,348 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-27 01:16:17,562 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-27 01:16:17,565 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1524791340.052, 1524791341.052), ('ovs_stats-br0.br0', 'if_errors', 1524791340.052, 1524791341.052), ('ovs_stats-br0.br0', 'if_packets', 1524791340.052, 1524791341.052)] 2018-04-27 01:16:17,566 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1524791356.052, 1524791357.052), ('ovs_stats-br0.br0', 'if_errors', 1524791357.053, 1524791358.052), ('ovs_stats-br0.br0', 'if_packets', 1524791357.053, 1524791358.053)] 2018-04-27 01:16:17,566 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1524791340.05 - ovs_stats-br0.br0 if_dropped 1524791356.05 ... 2018-04-27 01:16:17,567 - baro_tests.barometer - DEBUG - OK 2018-04-27 01:16:17,567 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1524791340.05 - ovs_stats-br0.br0 if_errors 1524791357.05 ... 2018-04-27 01:16:17,567 - baro_tests.barometer - DEBUG - OK 2018-04-27 01:16:17,567 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1524791340.05 - ovs_stats-br0.br0 if_packets 1524791357.05 ... 2018-04-27 01:16:17,567 - baro_tests.barometer - DEBUG - OK 2018-04-27 01:16:17,567 - baro_tests.barometer - INFO - OK 2018-04-27 01:16:17,568 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on OVS stats. 2018-04-27 01:16:30,405 - baro_tests.barometer - INFO - Mcelog is deleted from all compute nodes 2018-04-27 01:16:30,406 - baro_tests.barometer - INFO - ====================================================================== 2018-04-27 01:16:30,406 - baro_tests.barometer - INFO - +----------------------------------+ 2018-04-27 01:16:30,408 - baro_tests.barometer - INFO - | OVERALL SUMMARY | 2018-04-27 01:16:30,408 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-27 01:16:30,409 - baro_tests.barometer - INFO - | Test | Node-0 | Node-1 | 2018-04-27 01:16:30,409 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-27 01:16:30,409 - baro_tests.barometer - INFO - | OUT:Gnocchi | PASS | PASS | 2018-04-27 01:16:30,411 - baro_tests.barometer - INFO - | IN:Hugepages | PASS | FAIL | 2018-04-27 01:16:30,411 - baro_tests.barometer - INFO - | IN:Intel RDT | FAIL | FAIL | 2018-04-27 01:16:30,412 - baro_tests.barometer - INFO - | IN:Mcelog | PASS | FAIL | 2018-04-27 01:16:30,413 - baro_tests.barometer - INFO - | IN:OVS events | FAIL | FAIL | 2018-04-27 01:16:30,413 - baro_tests.barometer - INFO - | IN:OVS stats | FAIL | FAIL | 2018-04-27 01:16:30,414 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-27 01:16:30,415 - baro_tests.barometer - INFO - | OUT:AODH | PASS | PASS | 2018-04-27 01:16:30,415 - baro_tests.barometer - INFO - | IN:Mcelog | FAIL | FAIL | 2018-04-27 01:16:30,415 - baro_tests.barometer - INFO - | IN:OVS events | SKIP | SKIP | 2018-04-27 01:16:30,415 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-27 01:16:30,416 - baro_tests.barometer - INFO - | OUT:CSV | PASS | PASS | 2018-04-27 01:16:30,418 - baro_tests.barometer - INFO - | IN:Hugepages | PASS | PASS | 2018-04-27 01:16:30,418 - baro_tests.barometer - INFO - | IN:Intel RDT | PASS | PASS | 2018-04-27 01:16:30,419 - baro_tests.barometer - INFO - | IN:Mcelog | PASS | PASS | 2018-04-27 01:16:30,419 - baro_tests.barometer - INFO - | IN:OVS events | PASS | PASS | 2018-04-27 01:16:30,419 - baro_tests.barometer - INFO - | IN:OVS stats | PASS | PASS | 2018-04-27 01:16:30,419 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-27 01:16:30,420 - baro_tests.barometer - INFO - ====================================================================== 2018-04-27 01:16:30,421 - baro_tests.barometer - ERROR - Some tests have failed or have not been executed 2018-04-27 01:16:30,423 - baro_tests.barometer - ERROR - Overall Result is Fail