2018-05-20 11:38:33,544 - baro_tests.barometer - INFO - =================== Starting barometer tests suite =================== 2018-05-20 11:38:33,544 - baro_tests.barometer - INFO - RSA key file /root/.ssh/id_rsa exists. 2018-05-20 11:38:37,863 - baro_tests.barometer - INFO - ==== Display of Control and Compute nodes available in the set up ==== 2018-05-20 11:38:37,864 - baro_tests.barometer - INFO - controllers: ['overcloud-controller-0: 192.30.9.8', 'overcloud-controller-1: 192.30.9.9', 'overcloud-controller-2: 192.30.9.10'] 2018-05-20 11:38:37,864 - baro_tests.barometer - INFO - computes: ['overcloud-novacompute-0: 192.30.9.5', 'overcloud-novacompute-1: 192.30.9.7'] 2018-05-20 11:38:37,864 - baro_tests.barometer - INFO - ========== Enabling mcelog and OVS bridges on compute nodes ========== 2018-05-20 11:38:53,410 - baro_tests.barometer - INFO - Checking if mcelog is enabled on node-overcloud-novacompute-0... 2018-05-20 11:38:53,478 - baro_tests.barometer - INFO - Mcelog will be enabled on node-overcloud-novacompute-0... 2018-05-20 11:38:53,970 - baro_tests.barometer - INFO - Mcelog was installed on node-overcloud-novacompute-0. 2018-05-20 11:38:54,001 - baro_tests.barometer - INFO - Checking if mcelog is enabled on node-overcloud-novacompute-1... 2018-05-20 11:38:54,070 - baro_tests.barometer - INFO - Mcelog will be enabled on node-overcloud-novacompute-1... 2018-05-20 11:38:54,590 - baro_tests.barometer - INFO - Mcelog was installed on node-overcloud-novacompute-1. 2018-05-20 11:39:08,010 - baro_tests.barometer - INFO - Creating OVS bridges on computes nodes 2018-05-20 11:39:08,335 - baro_tests.barometer - INFO - OVS Bridges created on compute nodes 2018-05-20 11:39:08,336 - baro_tests.barometer - INFO - ============= Testing Gnocchi and AODH plugins on nodes ============== 2018-05-20 11:41:01,875 - baro_tests.barometer - INFO - Gnocchi plugin available in compute node overcloud-novacompute-0 2018-05-20 11:41:15,397 - baro_tests.barometer - INFO - AODH plugin present in compute node overcloud-novacompute-0 2018-05-20 11:41:15,398 - baro_tests.barometer - INFO - === NODE overcloud-novacompute-0: Enabling Test Plug-in and Test case execution === 2018-05-20 11:42:36,036 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Hugepages Test case execution === 2018-05-20 11:43:02,906 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-20 11:43:40,114 - baro_tests.barometer - INFO - PASS 2018-05-20 11:43:40,115 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with Gnocchi PASSED on Hugepages. 2018-05-20 11:44:34,274 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Intel RDT Test case execution === 2018-05-20 11:45:13,319 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-20 11:45:50,457 - baro_tests.barometer - INFO - Plugin Interval is 10 2018-05-20 11:45:50,457 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-05-20 11:45:50,458 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on Intel RDT. 2018-05-20 11:46:44,694 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-05-20 11:47:11,832 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-20 11:47:48,902 - baro_tests.barometer - INFO - PASS 2018-05-20 11:47:48,903 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with Gnocchi PASSED on Mcelog. 2018-05-20 11:48:42,381 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-05-20 11:49:09,181 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-20 11:49:47,009 - baro_tests.barometer - INFO - Plugin Interval is 1 2018-05-20 11:49:47,009 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-05-20 11:49:47,010 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on OVS events. 2018-05-20 11:50:41,259 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS stats Test case execution === 2018-05-20 11:51:07,317 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-20 11:51:44,459 - baro_tests.barometer - INFO - PASS 2018-05-20 11:51:44,459 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with Gnocchi PASSED on OVS stats. 2018-05-20 11:52:37,434 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-05-20 11:53:04,395 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-05-20 11:53:23,364 - baro_tests.barometer - INFO - Data not updated after interval of 12 seconds 2018-05-20 11:53:23,365 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with AODH FAILED on Mcelog. 2018-05-20 11:54:16,334 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-05-20 11:54:42,542 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-05-20 11:54:44,942 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-0 2018-05-20 11:54:48,415 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-1 2018-05-20 11:54:51,348 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with AODH FAILED on OVS events. 2018-05-20 11:54:51,349 - baro_tests.barometer - INFO - ======= Node overcloud-novacompute-0: Executing CSV Testcases ======== 2018-05-20 11:54:51,349 - baro_tests.barometer - INFO - Restarting collectd for CSV tests 2018-05-20 11:55:04,505 - baro_tests.barometer - INFO - Stopping collectd service... 2018-05-20 11:55:14,851 - baro_tests.barometer - INFO - Starting collectd service... 2018-05-20 11:55:25,184 - baro_tests.barometer - INFO - Sleeping for 10 seconds after collectd restart... 2018-05-20 11:56:28,864 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Hugepages Test case execution === 2018-05-20 11:56:55,514 - baro_tests.barometer - INFO - Getting CSV metrics of plugin hugepages on compute node 96bb816c-e6b4-4259-872c-ef98cf69cf56... 2018-05-20 11:56:55,514 - baro_tests.barometer - DEBUG - Interval: 10 2018-05-20 11:56:55,514 - baro_tests.barometer - DEBUG - Plugin subdirs: ['hugepages-mm-2048Kb', 'hugepages-node0-2048Kb'] 2018-05-20 11:56:55,514 - baro_tests.barometer - DEBUG - Plugin meter categories: ['vmpage_number-free', 'vmpage_number-used'] 2018-05-20 11:57:09,347 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-20 11:57:09,347 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1526816974.98 ... 2018-05-20 11:57:09,348 - baro_tests.barometer - DEBUG - OK 2018-05-20 11:57:09,348 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1526816974.98 ... 2018-05-20 11:57:09,348 - baro_tests.barometer - DEBUG - OK 2018-05-20 11:57:09,348 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1526816974.98 ... 2018-05-20 11:57:09,349 - baro_tests.barometer - DEBUG - OK 2018-05-20 11:57:09,349 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1526816974.98 ... 2018-05-20 11:57:09,349 - baro_tests.barometer - DEBUG - OK 2018-05-20 11:57:09,349 - baro_tests.barometer - INFO - OK 2018-05-20 11:57:09,350 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-05-20 11:57:21,362 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-05-20 11:57:35,104 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-20 11:57:35,104 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1526816974.984, 1526816984.984), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1526816974.984, 1526816984.984), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1526816974.984, 1526816984.985), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1526816974.984, 1526816984.985)] 2018-05-20 11:57:35,105 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1526816994.985, 1526817004.984), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1526816994.985, 1526817004.984), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1526816994.985, 1526817004.984), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1526816994.985, 1526817004.984)] 2018-05-20 11:57:35,105 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1526816974.98 - hugepages-mm-2048Kb vmpage_number-free 1526816994.98 ... 2018-05-20 11:57:35,105 - baro_tests.barometer - DEBUG - OK 2018-05-20 11:57:35,106 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1526816974.98 - hugepages-mm-2048Kb vmpage_number-used 1526816994.98 ... 2018-05-20 11:57:35,106 - baro_tests.barometer - DEBUG - OK 2018-05-20 11:57:35,106 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1526816974.98 - hugepages-node0-2048Kb vmpage_number-free 1526816994.98 ... 2018-05-20 11:57:35,106 - baro_tests.barometer - DEBUG - OK 2018-05-20 11:57:35,106 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1526816974.98 - hugepages-node0-2048Kb vmpage_number-used 1526816994.98 ... 2018-05-20 11:57:35,106 - baro_tests.barometer - DEBUG - OK 2018-05-20 11:57:35,107 - baro_tests.barometer - INFO - OK 2018-05-20 11:57:35,107 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Hugepages. 2018-05-20 11:58:28,158 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Intel RDT Test case execution === 2018-05-20 11:59:06,726 - baro_tests.barometer - INFO - Getting CSV metrics of plugin intel_rdt on compute node 96bb816c-e6b4-4259-872c-ef98cf69cf56... 2018-05-20 11:59:06,726 - baro_tests.barometer - DEBUG - Interval: 10 2018-05-20 11:59:06,727 - baro_tests.barometer - DEBUG - Plugin subdirs: ['intel_rdt-0-2'] 2018-05-20 11:59:06,727 - baro_tests.barometer - DEBUG - Plugin meter categories: ['bytes-llc', 'ipc'] 2018-05-20 11:59:20,571 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-20 11:59:20,572 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1526817104.98 ... 2018-05-20 11:59:20,572 - baro_tests.barometer - DEBUG - OK 2018-05-20 11:59:20,572 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1526817104.98 ... 2018-05-20 11:59:20,572 - baro_tests.barometer - DEBUG - OK 2018-05-20 11:59:20,573 - baro_tests.barometer - INFO - OK 2018-05-20 11:59:20,573 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-05-20 11:59:32,585 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-05-20 11:59:46,476 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-20 11:59:46,477 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1526817104.984, 1526817114.984), ('intel_rdt-0-2', 'ipc', 1526817104.984, 1526817114.984)] 2018-05-20 11:59:46,477 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1526817134.984, 1526817144.984), ('intel_rdt-0-2', 'ipc', 1526817134.984, 1526817144.984)] 2018-05-20 11:59:46,477 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1526817104.98 - intel_rdt-0-2 bytes-llc 1526817134.98 ... 2018-05-20 11:59:46,478 - baro_tests.barometer - DEBUG - OK 2018-05-20 11:59:46,478 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1526817104.98 - intel_rdt-0-2 ipc 1526817134.98 ... 2018-05-20 11:59:46,479 - baro_tests.barometer - DEBUG - OK 2018-05-20 11:59:46,480 - baro_tests.barometer - INFO - OK 2018-05-20 11:59:46,480 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Intel RDT. 2018-05-20 12:00:37,497 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-05-20 12:01:02,131 - baro_tests.barometer - INFO - Getting CSV metrics of plugin mcelog on compute node 96bb816c-e6b4-4259-872c-ef98cf69cf56... 2018-05-20 12:01:02,132 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-20 12:01:02,133 - baro_tests.barometer - DEBUG - Plugin subdirs: ['mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'mcelog-SOCKET_0_CHANNEL_any_DIMM_any'] 2018-05-20 12:01:02,133 - baro_tests.barometer - DEBUG - Plugin meter categories: ['errors-corrected_memory_errors', 'errors-uncorrected_memory_errors'] 2018-05-20 12:01:14,669 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-20 12:01:14,669 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1526817232.98 ... 2018-05-20 12:01:14,670 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:01:14,670 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1526817232.98 ... 2018-05-20 12:01:14,670 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:01:14,671 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1526817232.98 ... 2018-05-20 12:01:14,671 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:01:14,671 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1526817232.98 ... 2018-05-20 12:01:14,672 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:01:14,672 - baro_tests.barometer - INFO - OK 2018-05-20 12:01:14,672 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-20 12:01:17,675 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-05-20 12:01:30,339 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-20 12:01:30,339 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1526817232.985, 1526817233.985), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1526817232.985, 1526817233.985), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1526817232.985, 1526817233.985), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1526817232.985, 1526817233.985)] 2018-05-20 12:01:30,339 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1526817247.985, 1526817248.985), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1526817247.985, 1526817248.985), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1526817247.985, 1526817249.985), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1526817248.985, 1526817249.985)] 2018-05-20 12:01:30,340 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1526817232.98 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1526817247.98 ... 2018-05-20 12:01:30,340 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:01:30,340 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1526817232.98 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1526817247.98 ... 2018-05-20 12:01:30,340 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:01:30,340 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1526817232.98 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1526817247.98 ... 2018-05-20 12:01:30,340 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:01:30,341 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1526817232.98 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1526817248.98 ... 2018-05-20 12:01:30,341 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:01:30,341 - baro_tests.barometer - INFO - OK 2018-05-20 12:01:30,341 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Mcelog. 2018-05-20 12:02:21,965 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-05-20 12:02:47,014 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_events on compute node 96bb816c-e6b4-4259-872c-ef98cf69cf56... 2018-05-20 12:02:47,014 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-20 12:02:47,015 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_events-br0'] 2018-05-20 12:02:47,015 - baro_tests.barometer - DEBUG - Plugin meter categories: ['gauge-link_status'] 2018-05-20 12:02:59,857 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-20 12:02:59,858 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1526817337.98 ... 2018-05-20 12:02:59,858 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:02:59,858 - baro_tests.barometer - INFO - OK 2018-05-20 12:02:59,858 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-20 12:03:02,861 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-05-20 12:03:16,783 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-20 12:03:16,783 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1526817337.985, 1526817338.985)] 2018-05-20 12:03:16,783 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1526817354.985, 1526817355.985)] 2018-05-20 12:03:16,784 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1526817337.98 - ovs_events-br0 gauge-link_status 1526817354.98 ... 2018-05-20 12:03:16,785 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:03:16,785 - baro_tests.barometer - INFO - OK 2018-05-20 12:03:16,786 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on OVS events. 2018-05-20 12:04:07,461 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS stats Test case execution === 2018-05-20 12:04:32,098 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_stats on compute node 96bb816c-e6b4-4259-872c-ef98cf69cf56... 2018-05-20 12:04:32,099 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-20 12:04:32,099 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_stats-br0.br0'] 2018-05-20 12:04:32,099 - baro_tests.barometer - DEBUG - Plugin meter categories: ['if_dropped', 'if_errors', 'if_packets'] 2018-05-20 12:04:46,024 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-20 12:04:46,025 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1526817443.98 ... 2018-05-20 12:04:46,025 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:04:46,025 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1526817443.98 ... 2018-05-20 12:04:46,025 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:04:46,025 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1526817443.98 ... 2018-05-20 12:04:46,025 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:04:46,025 - baro_tests.barometer - INFO - OK 2018-05-20 12:04:46,026 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-20 12:04:49,028 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-05-20 12:05:01,645 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-20 12:05:01,645 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1526817443.985, 1526817444.985), ('ovs_stats-br0.br0', 'if_errors', 1526817443.985, 1526817444.985), ('ovs_stats-br0.br0', 'if_packets', 1526817443.985, 1526817444.985)] 2018-05-20 12:05:01,646 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1526817459.985, 1526817460.985), ('ovs_stats-br0.br0', 'if_errors', 1526817459.985, 1526817460.985), ('ovs_stats-br0.br0', 'if_packets', 1526817459.985, 1526817460.985)] 2018-05-20 12:05:01,646 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1526817443.98 - ovs_stats-br0.br0 if_dropped 1526817459.98 ... 2018-05-20 12:05:01,646 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:05:01,646 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1526817443.98 - ovs_stats-br0.br0 if_errors 1526817459.98 ... 2018-05-20 12:05:01,647 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:05:01,647 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1526817443.98 - ovs_stats-br0.br0 if_packets 1526817459.98 ... 2018-05-20 12:05:01,647 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:05:01,647 - baro_tests.barometer - INFO - OK 2018-05-20 12:05:01,647 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on OVS stats. 2018-05-20 12:05:14,903 - baro_tests.barometer - INFO - Gnocchi plugin available in compute node overcloud-novacompute-1 2018-05-20 12:05:28,567 - baro_tests.barometer - INFO - AODH plugin present in compute node overcloud-novacompute-1 2018-05-20 12:05:28,567 - baro_tests.barometer - INFO - === NODE overcloud-novacompute-1: Enabling Test Plug-in and Test case execution === 2018-05-20 12:06:44,587 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Hugepages Test case execution === 2018-05-20 12:07:10,938 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-20 12:07:48,216 - baro_tests.barometer - INFO - PASS 2018-05-20 12:07:48,216 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with Gnocchi PASSED on Hugepages. 2018-05-20 12:08:40,493 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Intel RDT Test case execution === 2018-05-20 12:09:19,368 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-20 12:09:56,581 - baro_tests.barometer - INFO - Plugin Interval is 10 2018-05-20 12:09:56,581 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-05-20 12:09:56,581 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on Intel RDT. 2018-05-20 12:10:47,614 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-05-20 12:11:12,751 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-20 12:11:50,172 - baro_tests.barometer - INFO - PASS 2018-05-20 12:11:50,172 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with Gnocchi PASSED on Mcelog. 2018-05-20 12:12:42,508 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-05-20 12:13:07,461 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-20 12:13:10,093 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-05-20 12:13:12,885 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-1 2018-05-20 12:13:15,601 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on OVS events. 2018-05-20 12:14:05,845 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS stats Test case execution === 2018-05-20 12:14:31,222 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-05-20 12:15:08,967 - baro_tests.barometer - INFO - Plugin Interval is 1 2018-05-20 12:15:08,970 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-05-20 12:15:08,972 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on OVS stats. 2018-05-20 12:16:01,477 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-05-20 12:16:27,330 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-05-20 12:16:46,216 - baro_tests.barometer - INFO - Data not updated after interval of 12 seconds 2018-05-20 12:16:46,217 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with AODH FAILED on Mcelog. 2018-05-20 12:17:37,020 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-05-20 12:18:03,870 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-05-20 12:18:06,214 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-0 2018-05-20 12:18:08,579 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-1 2018-05-20 12:18:10,978 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with AODH FAILED on OVS events. 2018-05-20 12:18:10,982 - baro_tests.barometer - INFO - ======= Node overcloud-novacompute-1: Executing CSV Testcases ======== 2018-05-20 12:18:10,983 - baro_tests.barometer - INFO - Restarting collectd for CSV tests 2018-05-20 12:18:24,482 - baro_tests.barometer - INFO - Stopping collectd service... 2018-05-20 12:18:35,208 - baro_tests.barometer - INFO - Starting collectd service... 2018-05-20 12:18:45,546 - baro_tests.barometer - INFO - Sleeping for 10 seconds after collectd restart... 2018-05-20 12:19:46,206 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Hugepages Test case execution === 2018-05-20 12:20:11,247 - baro_tests.barometer - INFO - Getting CSV metrics of plugin hugepages on compute node f601c85d-898f-43dd-9079-5d083e407bde... 2018-05-20 12:20:11,247 - baro_tests.barometer - DEBUG - Interval: 10 2018-05-20 12:20:11,248 - baro_tests.barometer - DEBUG - Plugin subdirs: ['hugepages-mm-2048Kb', 'hugepages-node0-2048Kb'] 2018-05-20 12:20:11,248 - baro_tests.barometer - DEBUG - Plugin meter categories: ['vmpage_number-free', 'vmpage_number-used'] 2018-05-20 12:20:24,461 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-20 12:20:24,462 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1526818358.74 ... 2018-05-20 12:20:24,463 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:20:24,467 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1526818358.74 ... 2018-05-20 12:20:24,467 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:20:24,468 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1526818358.74 ... 2018-05-20 12:20:24,469 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:20:24,469 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1526818358.74 ... 2018-05-20 12:20:24,469 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:20:24,469 - baro_tests.barometer - INFO - OK 2018-05-20 12:20:24,469 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-05-20 12:20:36,476 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-05-20 12:20:49,746 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-20 12:20:49,746 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1526818358.737, 1526818368.737), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1526818358.737, 1526818368.737), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1526818358.737, 1526818368.737), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1526818358.737, 1526818368.737)] 2018-05-20 12:20:49,747 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1526818388.737, 1526818398.737), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1526818388.737, 1526818398.737), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1526818388.737, 1526818398.737), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1526818388.737, 1526818398.737)] 2018-05-20 12:20:49,748 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1526818358.74 - hugepages-mm-2048Kb vmpage_number-free 1526818388.74 ... 2018-05-20 12:20:49,748 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:20:49,750 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1526818358.74 - hugepages-mm-2048Kb vmpage_number-used 1526818388.74 ... 2018-05-20 12:20:49,751 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:20:49,751 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1526818358.74 - hugepages-node0-2048Kb vmpage_number-free 1526818388.74 ... 2018-05-20 12:20:49,752 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:20:49,753 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1526818358.74 - hugepages-node0-2048Kb vmpage_number-used 1526818388.74 ... 2018-05-20 12:20:49,753 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:20:49,754 - baro_tests.barometer - INFO - OK 2018-05-20 12:20:49,756 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Hugepages. 2018-05-20 12:21:38,494 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Intel RDT Test case execution === 2018-05-20 12:22:14,669 - baro_tests.barometer - INFO - Getting CSV metrics of plugin intel_rdt on compute node f601c85d-898f-43dd-9079-5d083e407bde... 2018-05-20 12:22:14,670 - baro_tests.barometer - DEBUG - Interval: 10 2018-05-20 12:22:14,670 - baro_tests.barometer - DEBUG - Plugin subdirs: ['intel_rdt-0-2'] 2018-05-20 12:22:14,671 - baro_tests.barometer - DEBUG - Plugin meter categories: ['bytes-llc', 'ipc'] 2018-05-20 12:22:28,251 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-20 12:22:28,251 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1526818488.74 ... 2018-05-20 12:22:28,251 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:22:28,252 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1526818488.74 ... 2018-05-20 12:22:28,252 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:22:28,252 - baro_tests.barometer - INFO - OK 2018-05-20 12:22:28,253 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-05-20 12:22:40,264 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-05-20 12:22:52,739 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-20 12:22:52,740 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1526818488.737, 1526818498.737), ('intel_rdt-0-2', 'ipc', 1526818488.737, 1526818498.737)] 2018-05-20 12:22:52,740 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1526818508.737, 1526818518.737), ('intel_rdt-0-2', 'ipc', 1526818508.737, 1526818518.737)] 2018-05-20 12:22:52,740 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1526818488.74 - intel_rdt-0-2 bytes-llc 1526818508.74 ... 2018-05-20 12:22:52,741 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:22:52,741 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1526818488.74 - intel_rdt-0-2 ipc 1526818508.74 ... 2018-05-20 12:22:52,741 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:22:52,742 - baro_tests.barometer - INFO - OK 2018-05-20 12:22:52,743 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Intel RDT. 2018-05-20 12:23:42,776 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-05-20 12:24:08,056 - baro_tests.barometer - INFO - Getting CSV metrics of plugin mcelog on compute node f601c85d-898f-43dd-9079-5d083e407bde... 2018-05-20 12:24:08,056 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-20 12:24:08,056 - baro_tests.barometer - DEBUG - Plugin subdirs: ['mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'mcelog-SOCKET_0_CHANNEL_any_DIMM_any'] 2018-05-20 12:24:08,056 - baro_tests.barometer - DEBUG - Plugin meter categories: ['errors-corrected_memory_errors', 'errors-uncorrected_memory_errors'] 2018-05-20 12:24:21,161 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-20 12:24:21,162 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1526818612.54 ... 2018-05-20 12:24:21,163 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:24:21,163 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1526818612.54 ... 2018-05-20 12:24:21,163 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:24:21,163 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1526818612.54 ... 2018-05-20 12:24:21,164 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:24:21,164 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1526818612.54 ... 2018-05-20 12:24:21,164 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:24:21,164 - baro_tests.barometer - INFO - OK 2018-05-20 12:24:21,164 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-20 12:24:24,167 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-05-20 12:24:37,212 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-20 12:24:37,212 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1526818612.544, 1526818613.544), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1526818612.544, 1526818613.544), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1526818612.544, 1526818613.544), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1526818612.544, 1526818613.544)] 2018-05-20 12:24:37,213 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1526818628.544, 1526818629.544), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1526818628.544, 1526818629.544), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1526818628.544, 1526818629.544), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1526818628.544, 1526818629.544)] 2018-05-20 12:24:37,213 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1526818612.54 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1526818628.54 ... 2018-05-20 12:24:37,214 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:24:37,214 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1526818612.54 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1526818628.54 ... 2018-05-20 12:24:37,214 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:24:37,216 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1526818612.54 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1526818628.54 ... 2018-05-20 12:24:37,216 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:24:37,216 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1526818612.54 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1526818628.54 ... 2018-05-20 12:24:37,217 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:24:37,217 - baro_tests.barometer - INFO - OK 2018-05-20 12:24:37,218 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Mcelog. 2018-05-20 12:25:28,079 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-05-20 12:25:51,863 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_events on compute node f601c85d-898f-43dd-9079-5d083e407bde... 2018-05-20 12:25:51,863 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-20 12:25:51,863 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_events-br0'] 2018-05-20 12:25:51,864 - baro_tests.barometer - DEBUG - Plugin meter categories: ['gauge-link_status'] 2018-05-20 12:26:03,838 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-20 12:26:03,840 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1526818715.54 ... 2018-05-20 12:26:03,840 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:26:03,840 - baro_tests.barometer - INFO - OK 2018-05-20 12:26:03,841 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-20 12:26:06,844 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-05-20 12:26:18,822 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-20 12:26:18,823 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1526818715.544, 1526818716.544)] 2018-05-20 12:26:18,823 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1526818730.544, 1526818731.545)] 2018-05-20 12:26:18,824 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1526818715.54 - ovs_events-br0 gauge-link_status 1526818730.54 ... 2018-05-20 12:26:18,825 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:26:18,827 - baro_tests.barometer - INFO - OK 2018-05-20 12:26:18,837 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on OVS events. 2018-05-20 12:27:08,952 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS stats Test case execution === 2018-05-20 12:27:34,371 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_stats on compute node f601c85d-898f-43dd-9079-5d083e407bde... 2018-05-20 12:27:34,372 - baro_tests.barometer - DEBUG - Interval: 1 2018-05-20 12:27:34,375 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_stats-br0.br0'] 2018-05-20 12:27:34,376 - baro_tests.barometer - DEBUG - Plugin meter categories: ['if_dropped', 'if_errors', 'if_packets'] 2018-05-20 12:27:47,306 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-05-20 12:27:47,308 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1526818818.54 ... 2018-05-20 12:27:47,308 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:27:47,308 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1526818818.54 ... 2018-05-20 12:27:47,308 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:27:47,309 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1526818818.54 ... 2018-05-20 12:27:47,309 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:27:47,309 - baro_tests.barometer - INFO - OK 2018-05-20 12:27:47,309 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-05-20 12:27:50,312 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-05-20 12:28:03,194 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-05-20 12:28:03,195 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1526818818.544, 1526818819.544), ('ovs_stats-br0.br0', 'if_errors', 1526818818.544, 1526818819.544), ('ovs_stats-br0.br0', 'if_packets', 1526818818.544, 1526818820.544)] 2018-05-20 12:28:03,195 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1526818834.544, 1526818835.544), ('ovs_stats-br0.br0', 'if_errors', 1526818834.544, 1526818835.544), ('ovs_stats-br0.br0', 'if_packets', 1526818834.544, 1526818835.544)] 2018-05-20 12:28:03,195 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1526818818.54 - ovs_stats-br0.br0 if_dropped 1526818834.54 ... 2018-05-20 12:28:03,195 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:28:03,195 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1526818818.54 - ovs_stats-br0.br0 if_errors 1526818834.54 ... 2018-05-20 12:28:03,196 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:28:03,196 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1526818818.54 - ovs_stats-br0.br0 if_packets 1526818834.54 ... 2018-05-20 12:28:03,196 - baro_tests.barometer - DEBUG - OK 2018-05-20 12:28:03,196 - baro_tests.barometer - INFO - OK 2018-05-20 12:28:03,196 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on OVS stats. 2018-05-20 12:28:16,015 - baro_tests.barometer - INFO - Mcelog is deleted from all compute nodes 2018-05-20 12:28:16,015 - baro_tests.barometer - INFO - ====================================================================== 2018-05-20 12:28:16,016 - baro_tests.barometer - INFO - +----------------------------------+ 2018-05-20 12:28:16,016 - baro_tests.barometer - INFO - | OVERALL SUMMARY | 2018-05-20 12:28:16,016 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-05-20 12:28:16,016 - baro_tests.barometer - INFO - | Test | Node-0 | Node-1 | 2018-05-20 12:28:16,017 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-05-20 12:28:16,017 - baro_tests.barometer - INFO - | OUT:Gnocchi | PASS | PASS | 2018-05-20 12:28:16,017 - baro_tests.barometer - INFO - | IN:Hugepages | PASS | PASS | 2018-05-20 12:28:16,017 - baro_tests.barometer - INFO - | IN:Intel RDT | FAIL | FAIL | 2018-05-20 12:28:16,017 - baro_tests.barometer - INFO - | IN:Mcelog | PASS | PASS | 2018-05-20 12:28:16,017 - baro_tests.barometer - INFO - | IN:OVS events | FAIL | FAIL | 2018-05-20 12:28:16,017 - baro_tests.barometer - INFO - | IN:OVS stats | PASS | FAIL | 2018-05-20 12:28:16,017 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-05-20 12:28:16,017 - baro_tests.barometer - INFO - | OUT:AODH | PASS | PASS | 2018-05-20 12:28:16,017 - baro_tests.barometer - INFO - | IN:Mcelog | FAIL | FAIL | 2018-05-20 12:28:16,018 - baro_tests.barometer - INFO - | IN:OVS events | SKIP | SKIP | 2018-05-20 12:28:16,018 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-05-20 12:28:16,018 - baro_tests.barometer - INFO - | OUT:CSV | PASS | PASS | 2018-05-20 12:28:16,018 - baro_tests.barometer - INFO - | IN:Hugepages | PASS | PASS | 2018-05-20 12:28:16,018 - baro_tests.barometer - INFO - | IN:Intel RDT | PASS | PASS | 2018-05-20 12:28:16,018 - baro_tests.barometer - INFO - | IN:Mcelog | PASS | PASS | 2018-05-20 12:28:16,018 - baro_tests.barometer - INFO - | IN:OVS events | PASS | PASS | 2018-05-20 12:28:16,018 - baro_tests.barometer - INFO - | IN:OVS stats | PASS | PASS | 2018-05-20 12:28:16,018 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-05-20 12:28:16,018 - baro_tests.barometer - INFO - ====================================================================== 2018-05-20 12:28:16,018 - baro_tests.barometer - ERROR - Some tests have failed or have not been executed 2018-05-20 12:28:16,019 - baro_tests.barometer - ERROR - Overall Result is Fail