2018-04-11 23:51:19,612 - baro_tests.barometer - INFO - =================== Starting barometer tests suite =================== 2018-04-11 23:51:19,612 - baro_tests.barometer - INFO - RSA key file /root/.ssh/id_rsa exists. 2018-04-11 23:51:23,970 - baro_tests.barometer - INFO - ==== Display of Control and Compute nodes available in the set up ==== 2018-04-11 23:51:23,970 - baro_tests.barometer - INFO - controllers: ['overcloud-controller-0: 192.30.9.6', 'overcloud-controller-1: 192.30.9.9', 'overcloud-controller-2: 192.30.9.8'] 2018-04-11 23:51:23,971 - baro_tests.barometer - INFO - computes: ['overcloud-novacompute-0: 192.30.9.5', 'overcloud-novacompute-1: 192.30.9.4'] 2018-04-11 23:51:23,971 - baro_tests.barometer - INFO - ========== Enabling mcelog and OVS bridges on compute nodes ========== 2018-04-11 23:51:38,387 - baro_tests.barometer - INFO - Checking if mcelog is enabled on node-overcloud-novacompute-0... 2018-04-11 23:51:38,461 - baro_tests.barometer - INFO - Mcelog will be enabled on node-overcloud-novacompute-0... 2018-04-11 23:51:38,845 - baro_tests.barometer - INFO - Mcelog was installed on node-overcloud-novacompute-0. 2018-04-11 23:51:38,857 - baro_tests.barometer - INFO - Checking if mcelog is enabled on node-overcloud-novacompute-1... 2018-04-11 23:51:38,909 - baro_tests.barometer - INFO - Mcelog will be enabled on node-overcloud-novacompute-1... 2018-04-11 23:51:39,277 - baro_tests.barometer - INFO - Mcelog was installed on node-overcloud-novacompute-1. 2018-04-11 23:51:52,755 - baro_tests.barometer - INFO - Creating OVS bridges on computes nodes 2018-04-11 23:51:53,011 - baro_tests.barometer - INFO - OVS Bridges created on compute nodes 2018-04-11 23:51:53,012 - baro_tests.barometer - INFO - ============= Testing Gnocchi and AODH plugins on nodes ============== 2018-04-11 23:53:43,671 - baro_tests.barometer - INFO - Gnocchi plugin available in compute node overcloud-novacompute-0 2018-04-11 23:53:57,846 - baro_tests.barometer - INFO - AODH plugin present in compute node overcloud-novacompute-0 2018-04-11 23:53:57,846 - baro_tests.barometer - INFO - === NODE overcloud-novacompute-0: Enabling Test Plug-in and Test case execution === 2018-04-11 23:55:14,820 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Hugepages Test case execution === 2018-04-11 23:55:41,136 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-11 23:56:19,895 - baro_tests.barometer - INFO - Plugin Interval is 10 2018-04-11 23:56:19,895 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-04-11 23:56:19,896 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on Hugepages. 2018-04-11 23:57:13,747 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Intel RDT Test case execution === 2018-04-11 23:57:53,551 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-11 23:57:56,115 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-04-11 23:57:58,823 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-1 2018-04-11 23:58:01,652 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on Intel RDT. 2018-04-11 23:58:54,739 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-04-11 23:59:20,502 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-11 23:59:57,679 - baro_tests.barometer - INFO - PASS 2018-04-11 23:59:57,679 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with Gnocchi PASSED on Mcelog. 2018-04-12 00:00:50,310 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-04-12 00:01:15,906 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-12 00:01:53,069 - baro_tests.barometer - INFO - Plugin Interval is 1 2018-04-12 00:01:53,069 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-04-12 00:01:53,070 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on OVS events. 2018-04-12 00:02:46,768 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS stats Test case execution === 2018-04-12 00:03:13,679 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-12 00:03:51,317 - baro_tests.barometer - INFO - Plugin Interval is 1 2018-04-12 00:03:51,317 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-04-12 00:03:51,318 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on OVS stats. 2018-04-12 00:04:43,073 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-04-12 00:05:10,993 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-04-12 00:05:30,490 - baro_tests.barometer - INFO - Data not updated after interval of 12 seconds 2018-04-12 00:05:30,491 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with AODH FAILED on Mcelog. 2018-04-12 00:06:23,597 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-04-12 00:06:49,681 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-04-12 00:06:52,026 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-0 2018-04-12 00:06:54,536 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-1 2018-04-12 00:06:56,957 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with AODH FAILED on OVS events. 2018-04-12 00:06:56,958 - baro_tests.barometer - INFO - ======= Node overcloud-novacompute-0: Executing CSV Testcases ======== 2018-04-12 00:06:56,958 - baro_tests.barometer - INFO - Restarting collectd for CSV tests 2018-04-12 00:07:09,802 - baro_tests.barometer - INFO - Stopping collectd service... 2018-04-12 00:07:21,110 - baro_tests.barometer - INFO - Starting collectd service... 2018-04-12 00:07:31,429 - baro_tests.barometer - INFO - Sleeping for 10 seconds after collectd restart... 2018-04-12 00:08:31,231 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Hugepages Test case execution === 2018-04-12 00:08:56,652 - baro_tests.barometer - INFO - Getting CSV metrics of plugin hugepages on compute node cc45b91d-718c-4267-b632-be74bd508cc3... 2018-04-12 00:08:56,653 - baro_tests.barometer - DEBUG - Interval: 10 2018-04-12 00:08:56,653 - baro_tests.barometer - DEBUG - Plugin subdirs: ['hugepages-mm-2048Kb', 'hugepages-node0-2048Kb'] 2018-04-12 00:08:56,653 - baro_tests.barometer - DEBUG - Plugin meter categories: ['vmpage_number-free', 'vmpage_number-used'] 2018-04-12 00:09:10,875 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-12 00:09:10,875 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1523491330.01 ... 2018-04-12 00:09:10,876 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:09:10,876 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1523491330.01 ... 2018-04-12 00:09:10,876 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:09:10,876 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1523491330.01 ... 2018-04-12 00:09:10,876 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:09:10,877 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1523491330.01 ... 2018-04-12 00:09:10,877 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:09:10,877 - baro_tests.barometer - INFO - OK 2018-04-12 00:09:10,877 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-04-12 00:09:22,887 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-12 00:09:36,571 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-12 00:09:36,572 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1523491330.01, 1523491340.01), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1523491330.01, 1523491340.01), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1523491330.01, 1523491340.01), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1523491330.01, 1523491340.01)] 2018-04-12 00:09:36,572 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1523491360.01, 1523491370.01), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1523491360.01, 1523491370.01), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1523491360.01, 1523491370.01), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1523491360.01, 1523491370.01)] 2018-04-12 00:09:36,573 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1523491330.01 - hugepages-mm-2048Kb vmpage_number-free 1523491360.01 ... 2018-04-12 00:09:36,573 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:09:36,574 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1523491330.01 - hugepages-mm-2048Kb vmpage_number-used 1523491360.01 ... 2018-04-12 00:09:36,575 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:09:36,576 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1523491330.01 - hugepages-node0-2048Kb vmpage_number-free 1523491360.01 ... 2018-04-12 00:09:36,577 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:09:36,577 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1523491330.01 - hugepages-node0-2048Kb vmpage_number-used 1523491360.01 ... 2018-04-12 00:09:36,578 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:09:36,578 - baro_tests.barometer - INFO - OK 2018-04-12 00:09:36,580 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Hugepages. 2018-04-12 00:10:29,979 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Intel RDT Test case execution === 2018-04-12 00:11:10,390 - baro_tests.barometer - INFO - Getting CSV metrics of plugin intel_rdt on compute node cc45b91d-718c-4267-b632-be74bd508cc3... 2018-04-12 00:11:10,390 - baro_tests.barometer - DEBUG - Interval: 10 2018-04-12 00:11:10,391 - baro_tests.barometer - DEBUG - Plugin subdirs: ['intel_rdt-0-2'] 2018-04-12 00:11:10,391 - baro_tests.barometer - DEBUG - Plugin meter categories: ['bytes-llc', 'ipc'] 2018-04-12 00:11:23,926 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-12 00:11:23,927 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1523491470.01 ... 2018-04-12 00:11:23,927 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:11:23,927 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1523491470.01 ... 2018-04-12 00:11:23,928 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:11:23,928 - baro_tests.barometer - INFO - OK 2018-04-12 00:11:23,928 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-04-12 00:11:35,936 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-12 00:11:48,267 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-12 00:11:48,268 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1523491470.01, 1523491480.01), ('intel_rdt-0-2', 'ipc', 1523491470.01, 1523491480.01)] 2018-04-12 00:11:48,268 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1523491490.01, 1523491500.01), ('intel_rdt-0-2', 'ipc', 1523491490.01, 1523491500.01)] 2018-04-12 00:11:48,269 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1523491470.01 - intel_rdt-0-2 bytes-llc 1523491490.01 ... 2018-04-12 00:11:48,269 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:11:48,269 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1523491470.01 - intel_rdt-0-2 ipc 1523491490.01 ... 2018-04-12 00:11:48,269 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:11:48,269 - baro_tests.barometer - INFO - OK 2018-04-12 00:11:48,270 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Intel RDT. 2018-04-12 00:12:39,263 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-04-12 00:13:05,092 - baro_tests.barometer - INFO - Getting CSV metrics of plugin mcelog on compute node cc45b91d-718c-4267-b632-be74bd508cc3... 2018-04-12 00:13:05,092 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-12 00:13:05,092 - baro_tests.barometer - DEBUG - Plugin subdirs: ['mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'mcelog-SOCKET_0_CHANNEL_any_DIMM_any'] 2018-04-12 00:13:05,093 - baro_tests.barometer - DEBUG - Plugin meter categories: ['errors-corrected_memory_errors', 'errors-uncorrected_memory_errors'] 2018-04-12 00:13:17,923 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-12 00:13:17,924 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1523491595.01 ... 2018-04-12 00:13:17,924 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:13:17,924 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1523491595.01 ... 2018-04-12 00:13:17,924 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:13:17,925 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1523491595.01 ... 2018-04-12 00:13:17,925 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:13:17,925 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1523491595.01 ... 2018-04-12 00:13:17,925 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:13:17,925 - baro_tests.barometer - INFO - OK 2018-04-12 00:13:17,925 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-12 00:13:20,928 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-12 00:13:33,573 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-12 00:13:33,573 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1523491595.01, 1523491596.01), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1523491595.01, 1523491596.01), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1523491595.01, 1523491596.01), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1523491595.01, 1523491596.01)] 2018-04-12 00:13:33,574 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1523491610.01, 1523491611.01), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1523491610.01, 1523491611.01), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1523491611.01, 1523491612.01), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1523491611.01, 1523491612.01)] 2018-04-12 00:13:33,574 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1523491595.01 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1523491610.01 ... 2018-04-12 00:13:33,575 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:13:33,575 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1523491595.01 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1523491610.01 ... 2018-04-12 00:13:33,575 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:13:33,575 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1523491595.01 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1523491611.01 ... 2018-04-12 00:13:33,575 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:13:33,575 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1523491595.01 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1523491611.01 ... 2018-04-12 00:13:33,575 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:13:33,575 - baro_tests.barometer - INFO - OK 2018-04-12 00:13:33,576 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Mcelog. 2018-04-12 00:14:23,957 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-04-12 00:14:49,348 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_events on compute node cc45b91d-718c-4267-b632-be74bd508cc3... 2018-04-12 00:14:49,351 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-12 00:14:49,353 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_events-br0'] 2018-04-12 00:14:49,353 - baro_tests.barometer - DEBUG - Plugin meter categories: ['gauge-link_status'] 2018-04-12 00:15:03,046 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-12 00:15:03,046 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1523491700.01 ... 2018-04-12 00:15:03,046 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:15:03,046 - baro_tests.barometer - INFO - OK 2018-04-12 00:15:03,047 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-12 00:15:06,050 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-12 00:15:19,047 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-12 00:15:19,047 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1523491700.01, 1523491701.011)] 2018-04-12 00:15:19,047 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1523491716.01, 1523491717.011)] 2018-04-12 00:15:19,048 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1523491700.01 - ovs_events-br0 gauge-link_status 1523491716.01 ... 2018-04-12 00:15:19,048 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:15:19,048 - baro_tests.barometer - INFO - OK 2018-04-12 00:15:19,048 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on OVS events. 2018-04-12 00:16:09,233 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS stats Test case execution === 2018-04-12 00:16:33,878 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_stats on compute node cc45b91d-718c-4267-b632-be74bd508cc3... 2018-04-12 00:16:33,879 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-12 00:16:33,879 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_stats-br0.br0'] 2018-04-12 00:16:33,879 - baro_tests.barometer - DEBUG - Plugin meter categories: ['if_dropped', 'if_errors', 'if_packets'] 2018-04-12 00:16:46,288 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-12 00:16:46,288 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1523491803.01 ... 2018-04-12 00:16:46,288 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:16:46,289 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1523491803.01 ... 2018-04-12 00:16:46,289 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:16:46,289 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1523491803.01 ... 2018-04-12 00:16:46,289 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:16:46,290 - baro_tests.barometer - INFO - OK 2018-04-12 00:16:46,290 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-12 00:16:49,293 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-12 00:17:01,554 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-12 00:17:01,555 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1523491803.01, 1523491804.01), ('ovs_stats-br0.br0', 'if_errors', 1523491803.01, 1523491804.01), ('ovs_stats-br0.br0', 'if_packets', 1523491803.01, 1523491804.01)] 2018-04-12 00:17:01,555 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1523491818.01, 1523491819.01), ('ovs_stats-br0.br0', 'if_errors', 1523491818.01, 1523491819.01), ('ovs_stats-br0.br0', 'if_packets', 1523491818.01, 1523491819.01)] 2018-04-12 00:17:01,555 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1523491803.01 - ovs_stats-br0.br0 if_dropped 1523491818.01 ... 2018-04-12 00:17:01,556 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:17:01,556 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1523491803.01 - ovs_stats-br0.br0 if_errors 1523491818.01 ... 2018-04-12 00:17:01,556 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:17:01,557 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1523491803.01 - ovs_stats-br0.br0 if_packets 1523491818.01 ... 2018-04-12 00:17:01,557 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:17:01,559 - baro_tests.barometer - INFO - OK 2018-04-12 00:17:01,560 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on OVS stats. 2018-04-12 00:17:14,109 - baro_tests.barometer - INFO - Gnocchi plugin available in compute node overcloud-novacompute-1 2018-04-12 00:17:26,012 - baro_tests.barometer - INFO - AODH plugin present in compute node overcloud-novacompute-1 2018-04-12 00:17:26,014 - baro_tests.barometer - INFO - === NODE overcloud-novacompute-1: Enabling Test Plug-in and Test case execution === 2018-04-12 00:18:42,876 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Hugepages Test case execution === 2018-04-12 00:19:08,016 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-12 00:19:45,188 - baro_tests.barometer - INFO - PASS 2018-04-12 00:19:45,189 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with Gnocchi PASSED on Hugepages. 2018-04-12 00:20:36,524 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Intel RDT Test case execution === 2018-04-12 00:21:15,230 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-12 00:21:17,828 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-04-12 00:21:20,453 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-1 2018-04-12 00:21:23,164 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on Intel RDT. 2018-04-12 00:22:15,189 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-04-12 00:22:39,748 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-12 00:23:17,225 - baro_tests.barometer - INFO - PASS 2018-04-12 00:23:17,228 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with Gnocchi PASSED on Mcelog. 2018-04-12 00:24:07,804 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-04-12 00:24:33,374 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-12 00:25:10,885 - baro_tests.barometer - INFO - Plugin Interval is 1 2018-04-12 00:25:10,885 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-04-12 00:25:10,886 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on OVS events. 2018-04-12 00:26:02,286 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS stats Test case execution === 2018-04-12 00:26:27,364 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-12 00:27:04,850 - baro_tests.barometer - INFO - PASS 2018-04-12 00:27:04,862 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with Gnocchi PASSED on OVS stats. 2018-04-12 00:27:55,220 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-04-12 00:28:20,052 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-04-12 00:28:38,970 - baro_tests.barometer - INFO - Data not updated after interval of 12 seconds 2018-04-12 00:28:38,971 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with AODH FAILED on Mcelog. 2018-04-12 00:29:29,883 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-04-12 00:29:56,643 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-04-12 00:29:58,986 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-0 2018-04-12 00:30:01,413 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-1 2018-04-12 00:30:03,717 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with AODH FAILED on OVS events. 2018-04-12 00:30:03,718 - baro_tests.barometer - INFO - ======= Node overcloud-novacompute-1: Executing CSV Testcases ======== 2018-04-12 00:30:03,718 - baro_tests.barometer - INFO - Restarting collectd for CSV tests 2018-04-12 00:30:15,392 - baro_tests.barometer - INFO - Stopping collectd service... 2018-04-12 00:30:25,941 - baro_tests.barometer - INFO - Starting collectd service... 2018-04-12 00:30:36,250 - baro_tests.barometer - INFO - Sleeping for 10 seconds after collectd restart... 2018-04-12 00:31:39,698 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Hugepages Test case execution === 2018-04-12 00:32:05,345 - baro_tests.barometer - INFO - Getting CSV metrics of plugin hugepages on compute node 1d0be931-d113-49c5-b63f-cacf8a01cd13... 2018-04-12 00:32:05,345 - baro_tests.barometer - DEBUG - Interval: 10 2018-04-12 00:32:05,346 - baro_tests.barometer - DEBUG - Plugin subdirs: ['hugepages-mm-2048Kb', 'hugepages-node0-2048Kb'] 2018-04-12 00:32:05,346 - baro_tests.barometer - DEBUG - Plugin meter categories: ['vmpage_number-free', 'vmpage_number-used'] 2018-04-12 00:32:18,340 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-12 00:32:18,340 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1523492737.17 ... 2018-04-12 00:32:18,341 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:32:18,341 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1523492737.17 ... 2018-04-12 00:32:18,341 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:32:18,341 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1523492737.17 ... 2018-04-12 00:32:18,341 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:32:18,342 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1523492737.17 ... 2018-04-12 00:32:18,342 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:32:18,342 - baro_tests.barometer - INFO - OK 2018-04-12 00:32:18,342 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-04-12 00:32:30,354 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-12 00:32:42,593 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-12 00:32:42,594 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1523492737.171, 1523492747.171), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1523492737.171, 1523492747.171), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1523492737.171, 1523492747.171), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1523492737.171, 1523492747.171)] 2018-04-12 00:32:42,594 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1523492757.171, 1523492767.171), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1523492757.171, 1523492767.171), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1523492757.171, 1523492767.171), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1523492757.171, 1523492767.171)] 2018-04-12 00:32:42,594 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1523492737.17 - hugepages-mm-2048Kb vmpage_number-free 1523492757.17 ... 2018-04-12 00:32:42,595 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:32:42,595 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1523492737.17 - hugepages-mm-2048Kb vmpage_number-used 1523492757.17 ... 2018-04-12 00:32:42,595 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:32:42,595 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1523492737.17 - hugepages-node0-2048Kb vmpage_number-free 1523492757.17 ... 2018-04-12 00:32:42,595 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:32:42,595 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1523492737.17 - hugepages-node0-2048Kb vmpage_number-used 1523492757.17 ... 2018-04-12 00:32:42,595 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:32:42,595 - baro_tests.barometer - INFO - OK 2018-04-12 00:32:42,595 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Hugepages. 2018-04-12 00:33:33,979 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Intel RDT Test case execution === 2018-04-12 00:34:11,381 - baro_tests.barometer - INFO - Getting CSV metrics of plugin intel_rdt on compute node 1d0be931-d113-49c5-b63f-cacf8a01cd13... 2018-04-12 00:34:11,381 - baro_tests.barometer - DEBUG - Interval: 10 2018-04-12 00:34:11,381 - baro_tests.barometer - DEBUG - Plugin subdirs: ['intel_rdt-0-2'] 2018-04-12 00:34:11,382 - baro_tests.barometer - DEBUG - Plugin meter categories: ['bytes-llc', 'ipc'] 2018-04-12 00:34:23,538 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-12 00:34:23,538 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1523492857.17 ... 2018-04-12 00:34:23,538 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:34:23,539 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1523492857.17 ... 2018-04-12 00:34:23,539 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:34:23,540 - baro_tests.barometer - INFO - OK 2018-04-12 00:34:23,540 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-04-12 00:34:35,552 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-12 00:34:48,022 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-12 00:34:48,023 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1523492857.171, 1523492867.171), ('intel_rdt-0-2', 'ipc', 1523492857.171, 1523492867.171)] 2018-04-12 00:34:48,023 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1523492887.171, 1523492897.171), ('intel_rdt-0-2', 'ipc', 1523492887.171, 1523492897.171)] 2018-04-12 00:34:48,023 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1523492857.17 - intel_rdt-0-2 bytes-llc 1523492887.17 ... 2018-04-12 00:34:48,023 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:34:48,024 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1523492857.17 - intel_rdt-0-2 ipc 1523492887.17 ... 2018-04-12 00:34:48,024 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:34:48,024 - baro_tests.barometer - INFO - OK 2018-04-12 00:34:48,024 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Intel RDT. 2018-04-12 00:35:38,118 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-04-12 00:36:03,355 - baro_tests.barometer - INFO - Getting CSV metrics of plugin mcelog on compute node 1d0be931-d113-49c5-b63f-cacf8a01cd13... 2018-04-12 00:36:03,356 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-12 00:36:03,360 - baro_tests.barometer - DEBUG - Plugin subdirs: ['mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'mcelog-SOCKET_0_CHANNEL_any_DIMM_any'] 2018-04-12 00:36:03,361 - baro_tests.barometer - DEBUG - Plugin meter categories: ['errors-corrected_memory_errors', 'errors-uncorrected_memory_errors'] 2018-04-12 00:36:16,801 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-12 00:36:16,802 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1523492985.17 ... 2018-04-12 00:36:16,802 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:36:16,803 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1523492985.17 ... 2018-04-12 00:36:16,803 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:36:16,803 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1523492986.17 ... 2018-04-12 00:36:16,803 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:36:16,803 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1523492986.17 ... 2018-04-12 00:36:16,804 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:36:16,804 - baro_tests.barometer - INFO - OK 2018-04-12 00:36:16,804 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-12 00:36:19,807 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-12 00:36:32,651 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-12 00:36:32,651 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1523492985.172, 1523492986.172), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1523492985.172, 1523492986.172), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1523492986.172, 1523492987.172), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1523492986.172, 1523492987.172)] 2018-04-12 00:36:32,652 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1523493002.173, 1523493003.173), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1523493002.173, 1523493003.173), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1523493002.172, 1523493003.173), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1523493002.172, 1523493003.173)] 2018-04-12 00:36:32,652 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1523492985.17 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1523493002.17 ... 2018-04-12 00:36:32,652 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:36:32,652 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1523492985.17 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1523493002.17 ... 2018-04-12 00:36:32,653 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:36:32,653 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1523492986.17 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1523493002.17 ... 2018-04-12 00:36:32,654 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:36:32,655 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1523492986.17 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1523493002.17 ... 2018-04-12 00:36:32,655 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:36:32,656 - baro_tests.barometer - INFO - OK 2018-04-12 00:36:32,659 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Mcelog. 2018-04-12 00:37:24,231 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-04-12 00:37:49,511 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_events on compute node 1d0be931-d113-49c5-b63f-cacf8a01cd13... 2018-04-12 00:37:49,512 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-12 00:37:49,512 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_events-br0'] 2018-04-12 00:37:49,514 - baro_tests.barometer - DEBUG - Plugin meter categories: ['gauge-link_status'] 2018-04-12 00:38:02,221 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-12 00:38:02,222 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1523493091.17 ... 2018-04-12 00:38:02,223 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:38:02,224 - baro_tests.barometer - INFO - OK 2018-04-12 00:38:02,226 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-12 00:38:05,230 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-12 00:38:17,378 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-12 00:38:17,379 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1523493091.172, 1523493092.173)] 2018-04-12 00:38:17,379 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1523493107.173, 1523493108.173)] 2018-04-12 00:38:17,379 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1523493091.17 - ovs_events-br0 gauge-link_status 1523493107.17 ... 2018-04-12 00:38:17,380 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:38:17,380 - baro_tests.barometer - INFO - OK 2018-04-12 00:38:17,380 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on OVS events. 2018-04-12 00:39:05,396 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS stats Test case execution === 2018-04-12 00:39:31,078 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_stats on compute node 1d0be931-d113-49c5-b63f-cacf8a01cd13... 2018-04-12 00:39:31,079 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-12 00:39:31,079 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_stats-br0.br0'] 2018-04-12 00:39:31,079 - baro_tests.barometer - DEBUG - Plugin meter categories: ['if_dropped', 'if_errors', 'if_packets'] 2018-04-12 00:39:43,971 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-12 00:39:43,972 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1523493193.17 ... 2018-04-12 00:39:43,972 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:39:43,972 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1523493193.17 ... 2018-04-12 00:39:43,972 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:39:43,972 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1523493193.17 ... 2018-04-12 00:39:43,972 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:39:43,973 - baro_tests.barometer - INFO - OK 2018-04-12 00:39:43,973 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-12 00:39:46,976 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-12 00:39:59,707 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-12 00:39:59,707 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1523493193.172, 1523493194.172), ('ovs_stats-br0.br0', 'if_errors', 1523493193.172, 1523493194.172), ('ovs_stats-br0.br0', 'if_packets', 1523493193.172, 1523493194.172)] 2018-04-12 00:39:59,708 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1523493209.172, 1523493210.172), ('ovs_stats-br0.br0', 'if_errors', 1523493209.172, 1523493210.172), ('ovs_stats-br0.br0', 'if_packets', 1523493209.172, 1523493210.172)] 2018-04-12 00:39:59,708 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1523493193.17 - ovs_stats-br0.br0 if_dropped 1523493209.17 ... 2018-04-12 00:39:59,708 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:39:59,708 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1523493193.17 - ovs_stats-br0.br0 if_errors 1523493209.17 ... 2018-04-12 00:39:59,708 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:39:59,708 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1523493193.17 - ovs_stats-br0.br0 if_packets 1523493209.17 ... 2018-04-12 00:39:59,708 - baro_tests.barometer - DEBUG - OK 2018-04-12 00:39:59,708 - baro_tests.barometer - INFO - OK 2018-04-12 00:39:59,709 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on OVS stats. 2018-04-12 00:40:12,222 - baro_tests.barometer - INFO - Mcelog is deleted from all compute nodes 2018-04-12 00:40:12,223 - baro_tests.barometer - INFO - ====================================================================== 2018-04-12 00:40:12,223 - baro_tests.barometer - INFO - +----------------------------------+ 2018-04-12 00:40:12,223 - baro_tests.barometer - INFO - | OVERALL SUMMARY | 2018-04-12 00:40:12,224 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-12 00:40:12,224 - baro_tests.barometer - INFO - | Test | Node-0 | Node-1 | 2018-04-12 00:40:12,224 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-12 00:40:12,225 - baro_tests.barometer - INFO - | OUT:Gnocchi | PASS | PASS | 2018-04-12 00:40:12,225 - baro_tests.barometer - INFO - | IN:Hugepages | FAIL | PASS | 2018-04-12 00:40:12,225 - baro_tests.barometer - INFO - | IN:Intel RDT | FAIL | FAIL | 2018-04-12 00:40:12,225 - baro_tests.barometer - INFO - | IN:Mcelog | PASS | PASS | 2018-04-12 00:40:12,225 - baro_tests.barometer - INFO - | IN:OVS events | FAIL | FAIL | 2018-04-12 00:40:12,225 - baro_tests.barometer - INFO - | IN:OVS stats | FAIL | PASS | 2018-04-12 00:40:12,225 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-12 00:40:12,226 - baro_tests.barometer - INFO - | OUT:AODH | PASS | PASS | 2018-04-12 00:40:12,226 - baro_tests.barometer - INFO - | IN:Mcelog | FAIL | FAIL | 2018-04-12 00:40:12,226 - baro_tests.barometer - INFO - | IN:OVS events | SKIP | SKIP | 2018-04-12 00:40:12,226 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-12 00:40:12,226 - baro_tests.barometer - INFO - | OUT:CSV | PASS | PASS | 2018-04-12 00:40:12,226 - baro_tests.barometer - INFO - | IN:Hugepages | PASS | PASS | 2018-04-12 00:40:12,226 - baro_tests.barometer - INFO - | IN:Intel RDT | PASS | PASS | 2018-04-12 00:40:12,226 - baro_tests.barometer - INFO - | IN:Mcelog | PASS | PASS | 2018-04-12 00:40:12,226 - baro_tests.barometer - INFO - | IN:OVS events | PASS | PASS | 2018-04-12 00:40:12,226 - baro_tests.barometer - INFO - | IN:OVS stats | PASS | PASS | 2018-04-12 00:40:12,227 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-12 00:40:12,227 - baro_tests.barometer - INFO - ====================================================================== 2018-04-12 00:40:12,227 - baro_tests.barometer - ERROR - Some tests have failed or have not been executed 2018-04-12 00:40:12,227 - baro_tests.barometer - ERROR - Overall Result is Fail