2018-04-14 20:18:02,945 - baro_tests.barometer - INFO - =================== Starting barometer tests suite =================== 2018-04-14 20:18:02,945 - baro_tests.barometer - INFO - RSA key file /root/.ssh/id_rsa exists. 2018-04-14 20:18:07,192 - baro_tests.barometer - INFO - ==== Display of Control and Compute nodes available in the set up ==== 2018-04-14 20:18:07,193 - baro_tests.barometer - INFO - controllers: ['overcloud-controller-0: 192.30.9.9', 'overcloud-controller-1: 192.30.9.8', 'overcloud-controller-2: 192.30.9.3'] 2018-04-14 20:18:07,193 - baro_tests.barometer - INFO - computes: ['overcloud-novacompute-0: 192.30.9.7', 'overcloud-novacompute-1: 192.30.9.4'] 2018-04-14 20:18:07,193 - baro_tests.barometer - INFO - ========== Enabling mcelog and OVS bridges on compute nodes ========== 2018-04-14 20:18:20,717 - baro_tests.barometer - INFO - Checking if mcelog is enabled on node-overcloud-novacompute-0... 2018-04-14 20:18:20,767 - baro_tests.barometer - INFO - Mcelog will be enabled on node-overcloud-novacompute-0... 2018-04-14 20:18:21,135 - baro_tests.barometer - INFO - Mcelog was installed on node-overcloud-novacompute-0. 2018-04-14 20:18:21,147 - baro_tests.barometer - INFO - Checking if mcelog is enabled on node-overcloud-novacompute-1... 2018-04-14 20:18:21,198 - baro_tests.barometer - INFO - Mcelog will be enabled on node-overcloud-novacompute-1... 2018-04-14 20:18:21,580 - baro_tests.barometer - INFO - Mcelog was installed on node-overcloud-novacompute-1. 2018-04-14 20:18:35,330 - baro_tests.barometer - INFO - Creating OVS bridges on computes nodes 2018-04-14 20:18:35,581 - baro_tests.barometer - INFO - OVS Bridges created on compute nodes 2018-04-14 20:18:35,582 - baro_tests.barometer - INFO - ============= Testing Gnocchi and AODH plugins on nodes ============== 2018-04-14 20:20:27,726 - baro_tests.barometer - INFO - Gnocchi plugin available in compute node overcloud-novacompute-0 2018-04-14 20:20:40,361 - baro_tests.barometer - INFO - AODH plugin present in compute node overcloud-novacompute-0 2018-04-14 20:20:40,361 - baro_tests.barometer - INFO - === NODE overcloud-novacompute-0: Enabling Test Plug-in and Test case execution === 2018-04-14 20:21:59,732 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Hugepages Test case execution === 2018-04-14 20:22:26,052 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-14 20:22:28,730 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-04-14 20:22:31,625 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-1 2018-04-14 20:22:34,336 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on Hugepages. 2018-04-14 20:23:27,270 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Intel RDT Test case execution === 2018-04-14 20:24:06,688 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-14 20:24:43,773 - baro_tests.barometer - INFO - PASS 2018-04-14 20:24:43,773 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with Gnocchi PASSED on Intel RDT. 2018-04-14 20:25:36,932 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-04-14 20:26:04,460 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-14 20:26:41,627 - baro_tests.barometer - INFO - PASS 2018-04-14 20:26:41,628 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with Gnocchi PASSED on Mcelog. 2018-04-14 20:27:36,179 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-04-14 20:28:02,580 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-14 20:28:05,230 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-04-14 20:28:07,879 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-1 2018-04-14 20:28:10,576 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on OVS events. 2018-04-14 20:29:03,124 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS stats Test case execution === 2018-04-14 20:29:29,325 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-14 20:30:06,501 - baro_tests.barometer - INFO - Plugin Interval is 1 2018-04-14 20:30:06,501 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-04-14 20:30:06,502 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on OVS stats. 2018-04-14 20:30:59,892 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-04-14 20:31:25,773 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-04-14 20:31:45,153 - baro_tests.barometer - INFO - Data not updated after interval of 12 seconds 2018-04-14 20:31:45,154 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with AODH FAILED on Mcelog. 2018-04-14 20:32:37,691 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-04-14 20:33:02,652 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-04-14 20:33:05,000 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-0 2018-04-14 20:33:07,507 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-1 2018-04-14 20:33:10,609 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with AODH FAILED on OVS events. 2018-04-14 20:33:10,610 - baro_tests.barometer - INFO - ======= Node overcloud-novacompute-0: Executing CSV Testcases ======== 2018-04-14 20:33:10,611 - baro_tests.barometer - INFO - Restarting collectd for CSV tests 2018-04-14 20:33:23,044 - baro_tests.barometer - INFO - Stopping collectd service... 2018-04-14 20:33:33,791 - baro_tests.barometer - INFO - Starting collectd service... 2018-04-14 20:33:44,091 - baro_tests.barometer - INFO - Sleeping for 10 seconds after collectd restart... 2018-04-14 20:34:45,751 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Hugepages Test case execution === 2018-04-14 20:35:11,194 - baro_tests.barometer - INFO - Getting CSV metrics of plugin hugepages on compute node 9f0757bb-61b8-4592-8255-c80236ba04a9... 2018-04-14 20:35:11,195 - baro_tests.barometer - DEBUG - Interval: 10 2018-04-14 20:35:11,195 - baro_tests.barometer - DEBUG - Plugin subdirs: ['hugepages-mm-2048Kb', 'hugepages-node0-2048Kb'] 2018-04-14 20:35:11,196 - baro_tests.barometer - DEBUG - Plugin meter categories: ['vmpage_number-free', 'vmpage_number-used'] 2018-04-14 20:35:24,379 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-14 20:35:24,380 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1523737711.92 ... 2018-04-14 20:35:24,380 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:35:24,380 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1523737711.92 ... 2018-04-14 20:35:24,381 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:35:24,381 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1523737711.92 ... 2018-04-14 20:35:24,381 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:35:24,381 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1523737711.92 ... 2018-04-14 20:35:24,381 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:35:24,382 - baro_tests.barometer - INFO - OK 2018-04-14 20:35:24,382 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-04-14 20:35:36,387 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-14 20:35:50,281 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-14 20:35:50,281 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1523737711.918, 1523737721.918), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1523737711.918, 1523737721.918), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1523737711.918, 1523737721.918), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1523737711.918, 1523737721.918)] 2018-04-14 20:35:50,281 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1523737741.918, 1523737751.918), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1523737741.918, 1523737751.918), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1523737741.918, 1523737751.918), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1523737741.918, 1523737751.918)] 2018-04-14 20:35:50,282 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1523737711.92 - hugepages-mm-2048Kb vmpage_number-free 1523737741.92 ... 2018-04-14 20:35:50,283 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:35:50,283 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1523737711.92 - hugepages-mm-2048Kb vmpage_number-used 1523737741.92 ... 2018-04-14 20:35:50,283 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:35:50,283 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1523737711.92 - hugepages-node0-2048Kb vmpage_number-free 1523737741.92 ... 2018-04-14 20:35:50,283 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:35:50,283 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1523737711.92 - hugepages-node0-2048Kb vmpage_number-used 1523737741.92 ... 2018-04-14 20:35:50,283 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:35:50,283 - baro_tests.barometer - INFO - OK 2018-04-14 20:35:50,283 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Hugepages. 2018-04-14 20:36:45,617 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Intel RDT Test case execution === 2018-04-14 20:37:25,989 - baro_tests.barometer - INFO - Getting CSV metrics of plugin intel_rdt on compute node 9f0757bb-61b8-4592-8255-c80236ba04a9... 2018-04-14 20:37:25,990 - baro_tests.barometer - DEBUG - Interval: 10 2018-04-14 20:37:25,990 - baro_tests.barometer - DEBUG - Plugin subdirs: ['intel_rdt-0-2'] 2018-04-14 20:37:25,991 - baro_tests.barometer - DEBUG - Plugin meter categories: ['bytes-llc', 'ipc'] 2018-04-14 20:37:38,847 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-14 20:37:38,847 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1523737851.92 ... 2018-04-14 20:37:38,847 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:37:38,848 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1523737851.92 ... 2018-04-14 20:37:38,848 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:37:38,848 - baro_tests.barometer - INFO - OK 2018-04-14 20:37:38,848 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-04-14 20:37:50,856 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-14 20:38:04,057 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-14 20:38:04,057 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1523737851.918, 1523737861.918), ('intel_rdt-0-2', 'ipc', 1523737851.918, 1523737861.918)] 2018-04-14 20:38:04,058 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1523737871.918, 1523737881.918), ('intel_rdt-0-2', 'ipc', 1523737871.918, 1523737881.918)] 2018-04-14 20:38:04,058 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1523737851.92 - intel_rdt-0-2 bytes-llc 1523737871.92 ... 2018-04-14 20:38:04,058 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:38:04,059 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1523737851.92 - intel_rdt-0-2 ipc 1523737871.92 ... 2018-04-14 20:38:04,059 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:38:04,059 - baro_tests.barometer - INFO - OK 2018-04-14 20:38:04,060 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Intel RDT. 2018-04-14 20:38:54,987 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-04-14 20:39:19,930 - baro_tests.barometer - INFO - Getting CSV metrics of plugin mcelog on compute node 9f0757bb-61b8-4592-8255-c80236ba04a9... 2018-04-14 20:39:19,931 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-14 20:39:19,931 - baro_tests.barometer - DEBUG - Plugin subdirs: ['mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'mcelog-SOCKET_0_CHANNEL_any_DIMM_any'] 2018-04-14 20:39:19,931 - baro_tests.barometer - DEBUG - Plugin meter categories: ['errors-corrected_memory_errors', 'errors-uncorrected_memory_errors'] 2018-04-14 20:39:34,005 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-14 20:39:34,006 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1523737979.92 ... 2018-04-14 20:39:34,006 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:39:34,006 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1523737979.92 ... 2018-04-14 20:39:34,007 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:39:34,007 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1523737979.92 ... 2018-04-14 20:39:34,008 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:39:34,009 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1523737979.92 ... 2018-04-14 20:39:34,010 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:39:34,010 - baro_tests.barometer - INFO - OK 2018-04-14 20:39:34,013 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-14 20:39:37,017 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-14 20:39:49,907 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-14 20:39:49,907 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1523737979.919, 1523737980.919), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1523737979.919, 1523737980.919), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1523737979.919, 1523737980.919), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1523737979.919, 1523737980.919)] 2018-04-14 20:39:49,908 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1523737995.919, 1523737996.919), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1523737995.919, 1523737996.919), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1523737995.919, 1523737996.919), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1523737995.919, 1523737996.919)] 2018-04-14 20:39:49,908 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1523737979.92 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1523737995.92 ... 2018-04-14 20:39:49,909 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:39:49,909 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1523737979.92 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1523737995.92 ... 2018-04-14 20:39:49,910 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:39:49,910 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1523737979.92 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1523737995.92 ... 2018-04-14 20:39:49,911 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:39:49,911 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1523737979.92 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1523737995.92 ... 2018-04-14 20:39:49,911 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:39:49,911 - baro_tests.barometer - INFO - OK 2018-04-14 20:39:49,912 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Mcelog. 2018-04-14 20:40:41,126 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-04-14 20:41:06,590 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_events on compute node 9f0757bb-61b8-4592-8255-c80236ba04a9... 2018-04-14 20:41:06,591 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-14 20:41:06,591 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_events-br0'] 2018-04-14 20:41:06,591 - baro_tests.barometer - DEBUG - Plugin meter categories: ['gauge-link_status'] 2018-04-14 20:41:19,337 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-14 20:41:19,337 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1523738085.92 ... 2018-04-14 20:41:19,338 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:41:19,339 - baro_tests.barometer - INFO - OK 2018-04-14 20:41:19,339 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-14 20:41:22,342 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-14 20:41:36,260 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-14 20:41:36,260 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1523738085.919, 1523738086.919)] 2018-04-14 20:41:36,260 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1523738102.919, 1523738103.919)] 2018-04-14 20:41:36,260 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1523738085.92 - ovs_events-br0 gauge-link_status 1523738102.92 ... 2018-04-14 20:41:36,261 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:41:36,261 - baro_tests.barometer - INFO - OK 2018-04-14 20:41:36,262 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on OVS events. 2018-04-14 20:42:26,824 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS stats Test case execution === 2018-04-14 20:42:51,277 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_stats on compute node 9f0757bb-61b8-4592-8255-c80236ba04a9... 2018-04-14 20:42:51,277 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-14 20:42:51,278 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_stats-br0.br0'] 2018-04-14 20:42:51,278 - baro_tests.barometer - DEBUG - Plugin meter categories: ['if_dropped', 'if_errors', 'if_packets'] 2018-04-14 20:43:04,481 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-14 20:43:04,481 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1523738189.92 ... 2018-04-14 20:43:04,481 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:43:04,482 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1523738189.92 ... 2018-04-14 20:43:04,482 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:43:04,482 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1523738189.92 ... 2018-04-14 20:43:04,482 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:43:04,482 - baro_tests.barometer - INFO - OK 2018-04-14 20:43:04,483 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-14 20:43:07,483 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-14 20:43:20,025 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-14 20:43:20,025 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1523738189.919, 1523738190.919), ('ovs_stats-br0.br0', 'if_errors', 1523738189.919, 1523738190.919), ('ovs_stats-br0.br0', 'if_packets', 1523738189.919, 1523738190.919)] 2018-04-14 20:43:20,025 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1523738205.919, 1523738206.919), ('ovs_stats-br0.br0', 'if_errors', 1523738205.919, 1523738206.919), ('ovs_stats-br0.br0', 'if_packets', 1523738205.919, 1523738206.919)] 2018-04-14 20:43:20,025 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1523738189.92 - ovs_stats-br0.br0 if_dropped 1523738205.92 ... 2018-04-14 20:43:20,026 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:43:20,026 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1523738189.92 - ovs_stats-br0.br0 if_errors 1523738205.92 ... 2018-04-14 20:43:20,027 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:43:20,027 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1523738189.92 - ovs_stats-br0.br0 if_packets 1523738205.92 ... 2018-04-14 20:43:20,027 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:43:20,027 - baro_tests.barometer - INFO - OK 2018-04-14 20:43:20,027 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on OVS stats. 2018-04-14 20:43:32,082 - baro_tests.barometer - INFO - Gnocchi plugin available in compute node overcloud-novacompute-1 2018-04-14 20:43:45,258 - baro_tests.barometer - INFO - AODH plugin present in compute node overcloud-novacompute-1 2018-04-14 20:43:45,259 - baro_tests.barometer - INFO - === NODE overcloud-novacompute-1: Enabling Test Plug-in and Test case execution === 2018-04-14 20:45:04,622 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Hugepages Test case execution === 2018-04-14 20:45:28,706 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-14 20:46:05,897 - baro_tests.barometer - INFO - PASS 2018-04-14 20:46:05,905 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with Gnocchi PASSED on Hugepages. 2018-04-14 20:46:57,514 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Intel RDT Test case execution === 2018-04-14 20:47:35,827 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-14 20:48:13,152 - baro_tests.barometer - INFO - Plugin Interval is 10 2018-04-14 20:48:13,153 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-04-14 20:48:13,153 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on Intel RDT. 2018-04-14 20:49:06,137 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-04-14 20:49:32,482 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-14 20:50:09,958 - baro_tests.barometer - INFO - PASS 2018-04-14 20:50:09,959 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with Gnocchi PASSED on Mcelog. 2018-04-14 20:51:01,482 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-04-14 20:51:27,082 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-14 20:52:04,478 - baro_tests.barometer - INFO - PASS 2018-04-14 20:52:04,479 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with Gnocchi PASSED on OVS events. 2018-04-14 20:52:57,866 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS stats Test case execution === 2018-04-14 20:53:23,741 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-14 20:54:01,371 - baro_tests.barometer - INFO - PASS 2018-04-14 20:54:01,371 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with Gnocchi PASSED on OVS stats. 2018-04-14 20:54:53,547 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-04-14 20:55:18,654 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-04-14 20:55:37,462 - baro_tests.barometer - INFO - Data not updated after interval of 12 seconds 2018-04-14 20:55:37,464 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with AODH FAILED on Mcelog. 2018-04-14 20:56:28,344 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-04-14 20:56:52,189 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-04-14 20:56:54,571 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-0 2018-04-14 20:56:56,874 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-1 2018-04-14 20:56:59,177 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with AODH FAILED on OVS events. 2018-04-14 20:56:59,178 - baro_tests.barometer - INFO - ======= Node overcloud-novacompute-1: Executing CSV Testcases ======== 2018-04-14 20:56:59,179 - baro_tests.barometer - INFO - Restarting collectd for CSV tests 2018-04-14 20:57:11,279 - baro_tests.barometer - INFO - Stopping collectd service... 2018-04-14 20:57:22,903 - baro_tests.barometer - INFO - Starting collectd service... 2018-04-14 20:57:33,614 - baro_tests.barometer - INFO - Sleeping for 10 seconds after collectd restart... 2018-04-14 20:58:35,874 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Hugepages Test case execution === 2018-04-14 20:59:00,580 - baro_tests.barometer - INFO - Getting CSV metrics of plugin hugepages on compute node b100b454-9603-48bd-b4c5-600862a88e3d... 2018-04-14 20:59:00,580 - baro_tests.barometer - DEBUG - Interval: 10 2018-04-14 20:59:00,580 - baro_tests.barometer - DEBUG - Plugin subdirs: ['hugepages-mm-2048Kb', 'hugepages-node0-2048Kb'] 2018-04-14 20:59:00,581 - baro_tests.barometer - DEBUG - Plugin meter categories: ['vmpage_number-free', 'vmpage_number-used'] 2018-04-14 20:59:14,151 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-14 20:59:14,151 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1523739139.04 ... 2018-04-14 20:59:14,152 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:59:14,152 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1523739139.04 ... 2018-04-14 20:59:14,152 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:59:14,152 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1523739139.04 ... 2018-04-14 20:59:14,153 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:59:14,153 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1523739139.04 ... 2018-04-14 20:59:14,153 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:59:14,153 - baro_tests.barometer - INFO - OK 2018-04-14 20:59:14,153 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-04-14 20:59:26,159 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-14 20:59:39,762 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-14 20:59:39,762 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1523739139.044, 1523739149.044), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1523739139.044, 1523739149.044), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1523739139.044, 1523739149.044), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1523739139.044, 1523739149.044)] 2018-04-14 20:59:39,762 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1523739159.044, 1523739169.044), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1523739159.044, 1523739169.044), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1523739159.044, 1523739169.044), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1523739159.044, 1523739169.044)] 2018-04-14 20:59:39,763 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1523739139.04 - hugepages-mm-2048Kb vmpage_number-free 1523739159.04 ... 2018-04-14 20:59:39,763 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:59:39,763 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1523739139.04 - hugepages-mm-2048Kb vmpage_number-used 1523739159.04 ... 2018-04-14 20:59:39,764 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:59:39,764 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1523739139.04 - hugepages-node0-2048Kb vmpage_number-free 1523739159.04 ... 2018-04-14 20:59:39,765 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:59:39,765 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1523739139.04 - hugepages-node0-2048Kb vmpage_number-used 1523739159.04 ... 2018-04-14 20:59:39,765 - baro_tests.barometer - DEBUG - OK 2018-04-14 20:59:39,765 - baro_tests.barometer - INFO - OK 2018-04-14 20:59:39,765 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Hugepages. 2018-04-14 21:00:28,686 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Intel RDT Test case execution === 2018-04-14 21:01:05,324 - baro_tests.barometer - INFO - Getting CSV metrics of plugin intel_rdt on compute node b100b454-9603-48bd-b4c5-600862a88e3d... 2018-04-14 21:01:05,324 - baro_tests.barometer - DEBUG - Interval: 10 2018-04-14 21:01:05,325 - baro_tests.barometer - DEBUG - Plugin subdirs: ['intel_rdt-0-2'] 2018-04-14 21:01:05,325 - baro_tests.barometer - DEBUG - Plugin meter categories: ['bytes-llc', 'ipc'] 2018-04-14 21:01:17,373 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-14 21:01:17,374 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1523739259.04 ... 2018-04-14 21:01:17,374 - baro_tests.barometer - DEBUG - OK 2018-04-14 21:01:17,374 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1523739259.04 ... 2018-04-14 21:01:17,374 - baro_tests.barometer - DEBUG - OK 2018-04-14 21:01:17,375 - baro_tests.barometer - INFO - OK 2018-04-14 21:01:17,375 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-04-14 21:01:29,387 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-14 21:01:42,035 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-14 21:01:42,036 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1523739259.044, 1523739269.044), ('intel_rdt-0-2', 'ipc', 1523739259.044, 1523739269.044)] 2018-04-14 21:01:42,036 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1523739279.044, 1523739289.044), ('intel_rdt-0-2', 'ipc', 1523739279.044, 1523739289.044)] 2018-04-14 21:01:42,036 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1523739259.04 - intel_rdt-0-2 bytes-llc 1523739279.04 ... 2018-04-14 21:01:42,036 - baro_tests.barometer - DEBUG - OK 2018-04-14 21:01:42,036 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1523739259.04 - intel_rdt-0-2 ipc 1523739279.04 ... 2018-04-14 21:01:42,037 - baro_tests.barometer - DEBUG - OK 2018-04-14 21:01:42,037 - baro_tests.barometer - INFO - OK 2018-04-14 21:01:42,037 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Intel RDT. 2018-04-14 21:02:32,224 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-04-14 21:02:58,004 - baro_tests.barometer - INFO - Getting CSV metrics of plugin mcelog on compute node b100b454-9603-48bd-b4c5-600862a88e3d... 2018-04-14 21:02:58,005 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-14 21:02:58,005 - baro_tests.barometer - DEBUG - Plugin subdirs: ['mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'mcelog-SOCKET_0_CHANNEL_any_DIMM_any'] 2018-04-14 21:02:58,005 - baro_tests.barometer - DEBUG - Plugin meter categories: ['errors-corrected_memory_errors', 'errors-uncorrected_memory_errors'] 2018-04-14 21:03:10,761 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-14 21:03:10,761 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1523739384.05 ... 2018-04-14 21:03:10,761 - baro_tests.barometer - DEBUG - OK 2018-04-14 21:03:10,762 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1523739384.05 ... 2018-04-14 21:03:10,762 - baro_tests.barometer - DEBUG - OK 2018-04-14 21:03:10,762 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1523739385.05 ... 2018-04-14 21:03:10,762 - baro_tests.barometer - DEBUG - OK 2018-04-14 21:03:10,762 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1523739385.05 ... 2018-04-14 21:03:10,762 - baro_tests.barometer - DEBUG - OK 2018-04-14 21:03:10,762 - baro_tests.barometer - INFO - OK 2018-04-14 21:03:10,762 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-14 21:03:13,763 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-14 21:03:26,388 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-14 21:03:26,388 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1523739384.045, 1523739385.045), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1523739384.045, 1523739385.045), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1523739385.045, 1523739386.045), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1523739385.045, 1523739386.045)] 2018-04-14 21:03:26,389 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1523739400.045, 1523739401.045), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1523739400.045, 1523739401.045), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1523739400.045, 1523739401.045), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1523739400.045, 1523739401.045)] 2018-04-14 21:03:26,389 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1523739384.05 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1523739400.05 ... 2018-04-14 21:03:26,389 - baro_tests.barometer - DEBUG - OK 2018-04-14 21:03:26,389 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1523739384.05 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1523739400.05 ... 2018-04-14 21:03:26,389 - baro_tests.barometer - DEBUG - OK 2018-04-14 21:03:26,390 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1523739385.05 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1523739400.05 ... 2018-04-14 21:03:26,390 - baro_tests.barometer - DEBUG - OK 2018-04-14 21:03:26,390 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1523739385.05 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1523739400.05 ... 2018-04-14 21:03:26,390 - baro_tests.barometer - DEBUG - OK 2018-04-14 21:03:26,390 - baro_tests.barometer - INFO - OK 2018-04-14 21:03:26,390 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Mcelog. 2018-04-14 21:04:15,933 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-04-14 21:04:40,840 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_events on compute node b100b454-9603-48bd-b4c5-600862a88e3d... 2018-04-14 21:04:40,843 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-14 21:04:40,843 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_events-br0'] 2018-04-14 21:04:40,843 - baro_tests.barometer - DEBUG - Plugin meter categories: ['gauge-link_status'] 2018-04-14 21:04:52,779 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-14 21:04:52,780 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1523739487.05 ... 2018-04-14 21:04:52,780 - baro_tests.barometer - DEBUG - OK 2018-04-14 21:04:52,780 - baro_tests.barometer - INFO - OK 2018-04-14 21:04:52,780 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-14 21:04:55,783 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-14 21:05:07,956 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-14 21:05:07,956 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1523739487.045, 1523739488.045)] 2018-04-14 21:05:07,957 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1523739502.045, 1523739503.045)] 2018-04-14 21:05:07,957 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1523739487.05 - ovs_events-br0 gauge-link_status 1523739502.05 ... 2018-04-14 21:05:07,957 - baro_tests.barometer - DEBUG - OK 2018-04-14 21:05:07,958 - baro_tests.barometer - INFO - OK 2018-04-14 21:05:07,958 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on OVS events. 2018-04-14 21:06:00,024 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS stats Test case execution === 2018-04-14 21:06:25,352 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_stats on compute node b100b454-9603-48bd-b4c5-600862a88e3d... 2018-04-14 21:06:25,352 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-14 21:06:25,353 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_stats-br0.br0'] 2018-04-14 21:06:25,353 - baro_tests.barometer - DEBUG - Plugin meter categories: ['if_dropped', 'if_errors', 'if_packets'] 2018-04-14 21:06:37,505 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-14 21:06:37,505 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1523739591.05 ... 2018-04-14 21:06:37,505 - baro_tests.barometer - DEBUG - OK 2018-04-14 21:06:37,505 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1523739591.05 ... 2018-04-14 21:06:37,505 - baro_tests.barometer - DEBUG - OK 2018-04-14 21:06:37,506 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1523739591.05 ... 2018-04-14 21:06:37,506 - baro_tests.barometer - DEBUG - OK 2018-04-14 21:06:37,506 - baro_tests.barometer - INFO - OK 2018-04-14 21:06:37,506 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-14 21:06:40,508 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-14 21:06:54,436 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-14 21:06:54,437 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1523739591.045, 1523739592.045), ('ovs_stats-br0.br0', 'if_errors', 1523739591.045, 1523739592.045), ('ovs_stats-br0.br0', 'if_packets', 1523739591.045, 1523739592.045)] 2018-04-14 21:06:54,437 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1523739608.045, 1523739609.045), ('ovs_stats-br0.br0', 'if_errors', 1523739608.045, 1523739609.045), ('ovs_stats-br0.br0', 'if_packets', 1523739608.045, 1523739609.045)] 2018-04-14 21:06:54,437 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1523739591.05 - ovs_stats-br0.br0 if_dropped 1523739608.05 ... 2018-04-14 21:06:54,437 - baro_tests.barometer - DEBUG - OK 2018-04-14 21:06:54,438 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1523739591.05 - ovs_stats-br0.br0 if_errors 1523739608.05 ... 2018-04-14 21:06:54,438 - baro_tests.barometer - DEBUG - OK 2018-04-14 21:06:54,438 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1523739591.05 - ovs_stats-br0.br0 if_packets 1523739608.05 ... 2018-04-14 21:06:54,438 - baro_tests.barometer - DEBUG - OK 2018-04-14 21:06:54,438 - baro_tests.barometer - INFO - OK 2018-04-14 21:06:54,439 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on OVS stats. 2018-04-14 21:07:07,736 - baro_tests.barometer - INFO - Mcelog is deleted from all compute nodes 2018-04-14 21:07:07,736 - baro_tests.barometer - INFO - ====================================================================== 2018-04-14 21:07:07,737 - baro_tests.barometer - INFO - +----------------------------------+ 2018-04-14 21:07:07,737 - baro_tests.barometer - INFO - | OVERALL SUMMARY | 2018-04-14 21:07:07,737 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-14 21:07:07,737 - baro_tests.barometer - INFO - | Test | Node-0 | Node-1 | 2018-04-14 21:07:07,737 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-14 21:07:07,737 - baro_tests.barometer - INFO - | OUT:Gnocchi | PASS | PASS | 2018-04-14 21:07:07,737 - baro_tests.barometer - INFO - | IN:Hugepages | FAIL | PASS | 2018-04-14 21:07:07,737 - baro_tests.barometer - INFO - | IN:Intel RDT | PASS | FAIL | 2018-04-14 21:07:07,737 - baro_tests.barometer - INFO - | IN:Mcelog | PASS | PASS | 2018-04-14 21:07:07,738 - baro_tests.barometer - INFO - | IN:OVS events | FAIL | PASS | 2018-04-14 21:07:07,738 - baro_tests.barometer - INFO - | IN:OVS stats | FAIL | PASS | 2018-04-14 21:07:07,738 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-14 21:07:07,738 - baro_tests.barometer - INFO - | OUT:AODH | PASS | PASS | 2018-04-14 21:07:07,738 - baro_tests.barometer - INFO - | IN:Mcelog | FAIL | FAIL | 2018-04-14 21:07:07,738 - baro_tests.barometer - INFO - | IN:OVS events | SKIP | SKIP | 2018-04-14 21:07:07,738 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-14 21:07:07,738 - baro_tests.barometer - INFO - | OUT:CSV | PASS | PASS | 2018-04-14 21:07:07,738 - baro_tests.barometer - INFO - | IN:Hugepages | PASS | PASS | 2018-04-14 21:07:07,738 - baro_tests.barometer - INFO - | IN:Intel RDT | PASS | PASS | 2018-04-14 21:07:07,739 - baro_tests.barometer - INFO - | IN:Mcelog | PASS | PASS | 2018-04-14 21:07:07,739 - baro_tests.barometer - INFO - | IN:OVS events | PASS | PASS | 2018-04-14 21:07:07,739 - baro_tests.barometer - INFO - | IN:OVS stats | PASS | PASS | 2018-04-14 21:07:07,739 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-14 21:07:07,739 - baro_tests.barometer - INFO - ====================================================================== 2018-04-14 21:07:07,739 - baro_tests.barometer - ERROR - Some tests have failed or have not been executed 2018-04-14 21:07:07,739 - baro_tests.barometer - ERROR - Overall Result is Fail