2018-04-04 03:37:31,618 - baro_tests.barometer - INFO - =================== Starting barometer tests suite =================== 2018-04-04 03:37:31,618 - baro_tests.barometer - INFO - RSA key file /root/.ssh/id_rsa exists. 2018-04-04 03:37:35,575 - baro_tests.barometer - INFO - ==== Display of Control and Compute nodes available in the set up ==== 2018-04-04 03:37:35,575 - baro_tests.barometer - INFO - controllers: ['overcloud-controller-0: 192.30.9.9', 'overcloud-controller-1: 192.30.9.10', 'overcloud-controller-2: 192.30.9.7'] 2018-04-04 03:37:35,575 - baro_tests.barometer - INFO - computes: ['overcloud-novacompute-0: 192.30.9.6', 'overcloud-novacompute-1: 192.30.9.4'] 2018-04-04 03:37:35,576 - baro_tests.barometer - INFO - ========== Enabling mcelog and OVS bridges on compute nodes ========== 2018-04-04 03:37:48,620 - baro_tests.barometer - INFO - Checking if mcelog is enabled on node-overcloud-novacompute-0... 2018-04-04 03:37:48,670 - baro_tests.barometer - INFO - Mcelog will be enabled on node-overcloud-novacompute-0... 2018-04-04 03:37:49,062 - baro_tests.barometer - INFO - Mcelog was installed on node-overcloud-novacompute-0. 2018-04-04 03:37:49,074 - baro_tests.barometer - INFO - Checking if mcelog is enabled on node-overcloud-novacompute-1... 2018-04-04 03:37:49,124 - baro_tests.barometer - INFO - Mcelog will be enabled on node-overcloud-novacompute-1... 2018-04-04 03:37:49,849 - baro_tests.barometer - INFO - Mcelog was installed on node-overcloud-novacompute-1. 2018-04-04 03:38:04,231 - baro_tests.barometer - INFO - Creating OVS bridges on computes nodes 2018-04-04 03:38:04,485 - baro_tests.barometer - INFO - OVS Bridges created on compute nodes 2018-04-04 03:38:04,486 - baro_tests.barometer - INFO - ============= Testing Gnocchi and AODH plugins on nodes ============== 2018-04-04 03:39:57,646 - baro_tests.barometer - INFO - Gnocchi plugin available in compute node overcloud-novacompute-0 2018-04-04 03:40:12,298 - baro_tests.barometer - INFO - AODH plugin present in compute node overcloud-novacompute-0 2018-04-04 03:40:12,299 - baro_tests.barometer - INFO - === NODE overcloud-novacompute-0: Enabling Test Plug-in and Test case execution === 2018-04-04 03:41:38,337 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Hugepages Test case execution === 2018-04-04 03:42:04,787 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-04 03:42:41,948 - baro_tests.barometer - INFO - Plugin Interval is 10 2018-04-04 03:42:41,948 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-04-04 03:42:41,948 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on Hugepages. 2018-04-04 03:43:36,327 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Intel RDT Test case execution === 2018-04-04 03:44:16,786 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-04 03:44:55,285 - baro_tests.barometer - INFO - Plugin Interval is 10 2018-04-04 03:44:55,286 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-04-04 03:44:55,286 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on Intel RDT. 2018-04-04 03:45:48,310 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-04-04 03:46:14,316 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-04 03:46:52,102 - baro_tests.barometer - INFO - PASS 2018-04-04 03:46:52,102 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with Gnocchi PASSED on Mcelog. 2018-04-04 03:47:43,019 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-04-04 03:48:09,722 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-04 03:48:12,275 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-04-04 03:48:14,948 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-1 2018-04-04 03:48:17,680 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on OVS events. 2018-04-04 03:49:10,361 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS stats Test case execution === 2018-04-04 03:49:37,011 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-04 03:50:14,230 - baro_tests.barometer - INFO - PASS 2018-04-04 03:50:14,231 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with Gnocchi PASSED on OVS stats. 2018-04-04 03:51:06,850 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-04-04 03:51:31,796 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-04-04 03:51:51,055 - baro_tests.barometer - INFO - Data not updated after interval of 12 seconds 2018-04-04 03:51:51,056 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with AODH FAILED on Mcelog. 2018-04-04 03:52:43,649 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-04-04 03:53:10,134 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-04-04 03:53:12,409 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-0 2018-04-04 03:53:15,071 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-1 2018-04-04 03:53:17,540 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with AODH FAILED on OVS events. 2018-04-04 03:53:17,541 - baro_tests.barometer - INFO - ======= Node overcloud-novacompute-0: Executing CSV Testcases ======== 2018-04-04 03:53:17,541 - baro_tests.barometer - INFO - Restarting collectd for CSV tests 2018-04-04 03:53:30,980 - baro_tests.barometer - INFO - Stopping collectd service... 2018-04-04 03:53:41,906 - baro_tests.barometer - INFO - Starting collectd service... 2018-04-04 03:53:52,435 - baro_tests.barometer - INFO - Sleeping for 10 seconds after collectd restart... 2018-04-04 03:54:57,550 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Hugepages Test case execution === 2018-04-04 03:55:23,090 - baro_tests.barometer - INFO - Getting CSV metrics of plugin hugepages on compute node 0ab5a814-7694-43d2-bd67-9343046fada5... 2018-04-04 03:55:23,091 - baro_tests.barometer - DEBUG - Interval: 10 2018-04-04 03:55:23,091 - baro_tests.barometer - DEBUG - Plugin subdirs: ['hugepages-mm-2048Kb', 'hugepages-node0-2048Kb'] 2018-04-04 03:55:23,091 - baro_tests.barometer - DEBUG - Plugin meter categories: ['vmpage_number-free', 'vmpage_number-used'] 2018-04-04 03:55:36,381 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-04 03:55:36,383 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1522813727.89 ... 2018-04-04 03:55:36,387 - baro_tests.barometer - DEBUG - OK 2018-04-04 03:55:36,389 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1522813727.89 ... 2018-04-04 03:55:36,389 - baro_tests.barometer - DEBUG - OK 2018-04-04 03:55:36,390 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1522813727.89 ... 2018-04-04 03:55:36,390 - baro_tests.barometer - DEBUG - OK 2018-04-04 03:55:36,390 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1522813727.89 ... 2018-04-04 03:55:36,390 - baro_tests.barometer - DEBUG - OK 2018-04-04 03:55:36,390 - baro_tests.barometer - INFO - OK 2018-04-04 03:55:36,390 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-04-04 03:55:48,400 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-04 03:56:00,732 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-04 03:56:00,733 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1522813727.893, 1522813737.893), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1522813727.893, 1522813737.893), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1522813727.893, 1522813737.893), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1522813727.893, 1522813737.893)] 2018-04-04 03:56:00,733 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1522813747.893, 1522813757.893), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1522813747.893, 1522813757.893), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1522813747.893, 1522813757.893), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1522813747.893, 1522813757.893)] 2018-04-04 03:56:00,733 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1522813727.89 - hugepages-mm-2048Kb vmpage_number-free 1522813747.89 ... 2018-04-04 03:56:00,734 - baro_tests.barometer - DEBUG - OK 2018-04-04 03:56:00,734 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1522813727.89 - hugepages-mm-2048Kb vmpage_number-used 1522813747.89 ... 2018-04-04 03:56:00,734 - baro_tests.barometer - DEBUG - OK 2018-04-04 03:56:00,734 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1522813727.89 - hugepages-node0-2048Kb vmpage_number-free 1522813747.89 ... 2018-04-04 03:56:00,735 - baro_tests.barometer - DEBUG - OK 2018-04-04 03:56:00,735 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1522813727.89 - hugepages-node0-2048Kb vmpage_number-used 1522813747.89 ... 2018-04-04 03:56:00,735 - baro_tests.barometer - DEBUG - OK 2018-04-04 03:56:00,735 - baro_tests.barometer - INFO - OK 2018-04-04 03:56:00,735 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Hugepages. 2018-04-04 03:56:53,277 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Intel RDT Test case execution === 2018-04-04 03:57:32,456 - baro_tests.barometer - INFO - Getting CSV metrics of plugin intel_rdt on compute node 0ab5a814-7694-43d2-bd67-9343046fada5... 2018-04-04 03:57:32,456 - baro_tests.barometer - DEBUG - Interval: 10 2018-04-04 03:57:32,456 - baro_tests.barometer - DEBUG - Plugin subdirs: ['intel_rdt-0-2'] 2018-04-04 03:57:32,457 - baro_tests.barometer - DEBUG - Plugin meter categories: ['bytes-llc', 'ipc'] 2018-04-04 03:57:45,288 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-04 03:57:45,289 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1522813857.89 ... 2018-04-04 03:57:45,289 - baro_tests.barometer - DEBUG - OK 2018-04-04 03:57:45,289 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1522813857.89 ... 2018-04-04 03:57:45,289 - baro_tests.barometer - DEBUG - OK 2018-04-04 03:57:45,289 - baro_tests.barometer - INFO - OK 2018-04-04 03:57:45,289 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-04-04 03:57:57,302 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-04 03:58:10,197 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-04 03:58:10,198 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1522813857.893, 1522813867.893), ('intel_rdt-0-2', 'ipc', 1522813857.893, 1522813867.893)] 2018-04-04 03:58:10,198 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1522813877.893, 1522813887.893), ('intel_rdt-0-2', 'ipc', 1522813877.893, 1522813887.893)] 2018-04-04 03:58:10,199 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1522813857.89 - intel_rdt-0-2 bytes-llc 1522813877.89 ... 2018-04-04 03:58:10,199 - baro_tests.barometer - DEBUG - OK 2018-04-04 03:58:10,199 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1522813857.89 - intel_rdt-0-2 ipc 1522813877.89 ... 2018-04-04 03:58:10,199 - baro_tests.barometer - DEBUG - OK 2018-04-04 03:58:10,199 - baro_tests.barometer - INFO - OK 2018-04-04 03:58:10,199 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Intel RDT. 2018-04-04 03:59:02,585 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-04-04 03:59:27,414 - baro_tests.barometer - INFO - Getting CSV metrics of plugin mcelog on compute node 0ab5a814-7694-43d2-bd67-9343046fada5... 2018-04-04 03:59:27,415 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-04 03:59:27,416 - baro_tests.barometer - DEBUG - Plugin subdirs: ['mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'mcelog-SOCKET_0_CHANNEL_any_DIMM_any'] 2018-04-04 03:59:27,416 - baro_tests.barometer - DEBUG - Plugin meter categories: ['errors-corrected_memory_errors', 'errors-uncorrected_memory_errors'] 2018-04-04 03:59:40,274 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-04 03:59:40,275 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1522813983.89 ... 2018-04-04 03:59:40,275 - baro_tests.barometer - DEBUG - OK 2018-04-04 03:59:40,275 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1522813983.89 ... 2018-04-04 03:59:40,275 - baro_tests.barometer - DEBUG - OK 2018-04-04 03:59:40,276 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1522813983.89 ... 2018-04-04 03:59:40,276 - baro_tests.barometer - DEBUG - OK 2018-04-04 03:59:40,276 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1522813983.89 ... 2018-04-04 03:59:40,277 - baro_tests.barometer - DEBUG - OK 2018-04-04 03:59:40,277 - baro_tests.barometer - INFO - OK 2018-04-04 03:59:40,277 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-04 03:59:43,280 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-04 03:59:55,520 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-04 03:59:55,520 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1522813983.894, 1522813984.894), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1522813983.894, 1522813984.894), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1522813983.894, 1522813984.894), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1522813983.894, 1522813984.894)] 2018-04-04 03:59:55,521 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1522813998.894, 1522813999.894), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1522813998.894, 1522813999.894), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1522813998.894, 1522813999.894), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1522813999.894, 1522814000.894)] 2018-04-04 03:59:55,521 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1522813983.89 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1522813998.89 ... 2018-04-04 03:59:55,521 - baro_tests.barometer - DEBUG - OK 2018-04-04 03:59:55,521 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1522813983.89 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1522813998.89 ... 2018-04-04 03:59:55,522 - baro_tests.barometer - DEBUG - OK 2018-04-04 03:59:55,522 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1522813983.89 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1522813998.89 ... 2018-04-04 03:59:55,522 - baro_tests.barometer - DEBUG - OK 2018-04-04 03:59:55,522 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1522813983.89 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1522813999.89 ... 2018-04-04 03:59:55,522 - baro_tests.barometer - DEBUG - OK 2018-04-04 03:59:55,522 - baro_tests.barometer - INFO - OK 2018-04-04 03:59:55,522 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Mcelog. 2018-04-04 04:00:45,841 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-04-04 04:01:10,721 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_events on compute node 0ab5a814-7694-43d2-bd67-9343046fada5... 2018-04-04 04:01:10,722 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-04 04:01:10,722 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_events-br0'] 2018-04-04 04:01:10,722 - baro_tests.barometer - DEBUG - Plugin meter categories: ['gauge-link_status'] 2018-04-04 04:01:23,370 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-04 04:01:23,370 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1522814086.89 ... 2018-04-04 04:01:23,371 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:01:23,371 - baro_tests.barometer - INFO - OK 2018-04-04 04:01:23,372 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-04 04:01:26,375 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-04 04:01:39,861 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-04 04:01:39,861 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1522814086.894, 1522814087.894)] 2018-04-04 04:01:39,862 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1522814103.894, 1522814104.895)] 2018-04-04 04:01:39,862 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1522814086.89 - ovs_events-br0 gauge-link_status 1522814103.89 ... 2018-04-04 04:01:39,862 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:01:39,863 - baro_tests.barometer - INFO - OK 2018-04-04 04:01:39,863 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on OVS events. 2018-04-04 04:02:29,408 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS stats Test case execution === 2018-04-04 04:02:54,600 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_stats on compute node 0ab5a814-7694-43d2-bd67-9343046fada5... 2018-04-04 04:02:54,600 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-04 04:02:54,601 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_stats-br0.br0'] 2018-04-04 04:02:54,601 - baro_tests.barometer - DEBUG - Plugin meter categories: ['if_dropped', 'if_errors', 'if_packets'] 2018-04-04 04:03:07,432 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-04 04:03:07,433 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1522814190.89 ... 2018-04-04 04:03:07,433 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:03:07,433 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1522814190.89 ... 2018-04-04 04:03:07,433 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:03:07,434 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1522814190.89 ... 2018-04-04 04:03:07,434 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:03:07,434 - baro_tests.barometer - INFO - OK 2018-04-04 04:03:07,434 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-04 04:03:10,437 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-04 04:03:23,359 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-04 04:03:23,360 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1522814190.894, 1522814191.894), ('ovs_stats-br0.br0', 'if_errors', 1522814190.894, 1522814191.894), ('ovs_stats-br0.br0', 'if_packets', 1522814190.894, 1522814192.894)] 2018-04-04 04:03:23,360 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1522814206.896, 1522814207.894), ('ovs_stats-br0.br0', 'if_errors', 1522814206.896, 1522814207.894), ('ovs_stats-br0.br0', 'if_packets', 1522814206.896, 1522814207.894)] 2018-04-04 04:03:23,360 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1522814190.89 - ovs_stats-br0.br0 if_dropped 1522814206.9 ... 2018-04-04 04:03:23,360 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:03:23,361 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1522814190.89 - ovs_stats-br0.br0 if_errors 1522814206.9 ... 2018-04-04 04:03:23,361 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:03:23,361 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1522814190.89 - ovs_stats-br0.br0 if_packets 1522814206.9 ... 2018-04-04 04:03:23,361 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:03:23,361 - baro_tests.barometer - INFO - OK 2018-04-04 04:03:23,361 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on OVS stats. 2018-04-04 04:03:35,562 - baro_tests.barometer - INFO - Gnocchi plugin available in compute node overcloud-novacompute-1 2018-04-04 04:03:48,235 - baro_tests.barometer - INFO - AODH plugin present in compute node overcloud-novacompute-1 2018-04-04 04:03:48,236 - baro_tests.barometer - INFO - === NODE overcloud-novacompute-1: Enabling Test Plug-in and Test case execution === 2018-04-04 04:05:05,807 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Hugepages Test case execution === 2018-04-04 04:05:31,002 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-04 04:05:33,646 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-04-04 04:05:36,341 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-1 2018-04-04 04:05:39,056 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on Hugepages. 2018-04-04 04:06:31,096 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Intel RDT Test case execution === 2018-04-04 04:07:09,347 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-04 04:07:46,521 - baro_tests.barometer - INFO - PASS 2018-04-04 04:07:46,522 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with Gnocchi PASSED on Intel RDT. 2018-04-04 04:08:39,212 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-04-04 04:09:05,114 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-04 04:09:42,736 - baro_tests.barometer - INFO - PASS 2018-04-04 04:09:42,737 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with Gnocchi PASSED on Mcelog. 2018-04-04 04:10:36,257 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-04-04 04:11:00,444 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-04 04:11:37,686 - baro_tests.barometer - INFO - PASS 2018-04-04 04:11:37,687 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with Gnocchi PASSED on OVS events. 2018-04-04 04:12:28,624 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS stats Test case execution === 2018-04-04 04:12:53,060 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-04 04:13:30,399 - baro_tests.barometer - INFO - PASS 2018-04-04 04:13:30,400 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with Gnocchi PASSED on OVS stats. 2018-04-04 04:14:20,006 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-04-04 04:14:44,967 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-04-04 04:15:03,842 - baro_tests.barometer - INFO - Data not updated after interval of 12 seconds 2018-04-04 04:15:03,843 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with AODH FAILED on Mcelog. 2018-04-04 04:15:56,283 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-04-04 04:16:22,641 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-04-04 04:16:25,072 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-0 2018-04-04 04:16:27,401 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-1 2018-04-04 04:16:29,771 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with AODH FAILED on OVS events. 2018-04-04 04:16:29,776 - baro_tests.barometer - INFO - ======= Node overcloud-novacompute-1: Executing CSV Testcases ======== 2018-04-04 04:16:29,777 - baro_tests.barometer - INFO - Restarting collectd for CSV tests 2018-04-04 04:16:43,467 - baro_tests.barometer - INFO - Stopping collectd service... 2018-04-04 04:16:53,766 - baro_tests.barometer - INFO - Starting collectd service... 2018-04-04 04:17:04,065 - baro_tests.barometer - INFO - Sleeping for 10 seconds after collectd restart... 2018-04-04 04:18:05,154 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Hugepages Test case execution === 2018-04-04 04:18:30,433 - baro_tests.barometer - INFO - Getting CSV metrics of plugin hugepages on compute node 5f6ecc43-4550-4f6d-9eff-878b75b9a4e8... 2018-04-04 04:18:30,434 - baro_tests.barometer - DEBUG - Interval: 10 2018-04-04 04:18:30,434 - baro_tests.barometer - DEBUG - Plugin subdirs: ['hugepages-mm-2048Kb', 'hugepages-node0-2048Kb'] 2018-04-04 04:18:30,434 - baro_tests.barometer - DEBUG - Plugin meter categories: ['vmpage_number-free', 'vmpage_number-used'] 2018-04-04 04:18:43,986 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-04 04:18:43,987 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1522815123.65 ... 2018-04-04 04:18:43,987 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:18:43,987 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1522815123.65 ... 2018-04-04 04:18:43,988 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:18:43,989 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1522815123.65 ... 2018-04-04 04:18:43,990 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:18:43,990 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1522815123.65 ... 2018-04-04 04:18:43,991 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:18:43,991 - baro_tests.barometer - INFO - OK 2018-04-04 04:18:43,991 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-04-04 04:18:56,002 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-04 04:19:09,077 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-04 04:19:09,077 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1522815123.649, 1522815133.649), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1522815123.649, 1522815133.649), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1522815123.649, 1522815133.649), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1522815123.649, 1522815133.649)] 2018-04-04 04:19:09,077 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1522815153.649, 1522815163.649), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1522815153.649, 1522815163.649), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1522815153.649, 1522815163.649), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1522815153.649, 1522815163.649)] 2018-04-04 04:19:09,078 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1522815123.65 - hugepages-mm-2048Kb vmpage_number-free 1522815153.65 ... 2018-04-04 04:19:09,078 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:19:09,078 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1522815123.65 - hugepages-mm-2048Kb vmpage_number-used 1522815153.65 ... 2018-04-04 04:19:09,078 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:19:09,079 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1522815123.65 - hugepages-node0-2048Kb vmpage_number-free 1522815153.65 ... 2018-04-04 04:19:09,079 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:19:09,079 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1522815123.65 - hugepages-node0-2048Kb vmpage_number-used 1522815153.65 ... 2018-04-04 04:19:09,079 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:19:09,079 - baro_tests.barometer - INFO - OK 2018-04-04 04:19:09,079 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Hugepages. 2018-04-04 04:19:59,865 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Intel RDT Test case execution === 2018-04-04 04:20:37,703 - baro_tests.barometer - INFO - Getting CSV metrics of plugin intel_rdt on compute node 5f6ecc43-4550-4f6d-9eff-878b75b9a4e8... 2018-04-04 04:20:37,704 - baro_tests.barometer - DEBUG - Interval: 10 2018-04-04 04:20:37,704 - baro_tests.barometer - DEBUG - Plugin subdirs: ['intel_rdt-0-2'] 2018-04-04 04:20:37,704 - baro_tests.barometer - DEBUG - Plugin meter categories: ['bytes-llc', 'ipc'] 2018-04-04 04:20:50,390 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-04 04:20:50,391 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1522815253.65 ... 2018-04-04 04:20:50,391 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:20:50,392 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1522815253.65 ... 2018-04-04 04:20:50,392 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:20:50,392 - baro_tests.barometer - INFO - OK 2018-04-04 04:20:50,392 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-04-04 04:21:02,404 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-04 04:21:15,164 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-04 04:21:15,166 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1522815253.649, 1522815263.649), ('intel_rdt-0-2', 'ipc', 1522815253.649, 1522815263.649)] 2018-04-04 04:21:15,167 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1522815283.649, 1522815293.649), ('intel_rdt-0-2', 'ipc', 1522815283.649, 1522815293.649)] 2018-04-04 04:21:15,169 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1522815253.65 - intel_rdt-0-2 bytes-llc 1522815283.65 ... 2018-04-04 04:21:15,170 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:21:15,171 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1522815253.65 - intel_rdt-0-2 ipc 1522815283.65 ... 2018-04-04 04:21:15,172 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:21:15,173 - baro_tests.barometer - INFO - OK 2018-04-04 04:21:15,182 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Intel RDT. 2018-04-04 04:22:04,675 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-04-04 04:22:28,077 - baro_tests.barometer - INFO - Getting CSV metrics of plugin mcelog on compute node 5f6ecc43-4550-4f6d-9eff-878b75b9a4e8... 2018-04-04 04:22:28,079 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-04 04:22:28,080 - baro_tests.barometer - DEBUG - Plugin subdirs: ['mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'mcelog-SOCKET_0_CHANNEL_any_DIMM_any'] 2018-04-04 04:22:28,082 - baro_tests.barometer - DEBUG - Plugin meter categories: ['errors-corrected_memory_errors', 'errors-uncorrected_memory_errors'] 2018-04-04 04:22:39,760 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-04 04:22:39,760 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1522815377.65 ... 2018-04-04 04:22:39,760 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:22:39,760 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1522815377.65 ... 2018-04-04 04:22:39,760 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:22:39,760 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1522815377.65 ... 2018-04-04 04:22:39,760 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:22:39,761 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1522815377.65 ... 2018-04-04 04:22:39,761 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:22:39,761 - baro_tests.barometer - INFO - OK 2018-04-04 04:22:39,761 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-04 04:22:42,762 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-04 04:22:55,056 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-04 04:22:55,056 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1522815377.65, 1522815378.651), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1522815377.65, 1522815378.651), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1522815377.65, 1522815378.651), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1522815377.65, 1522815378.651)] 2018-04-04 04:22:55,057 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1522815392.65, 1522815393.65), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1522815392.65, 1522815393.65), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1522815392.65, 1522815393.65), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1522815392.65, 1522815394.65)] 2018-04-04 04:22:55,057 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1522815377.65 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1522815392.65 ... 2018-04-04 04:22:55,057 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:22:55,058 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1522815377.65 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1522815392.65 ... 2018-04-04 04:22:55,058 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:22:55,058 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1522815377.65 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1522815392.65 ... 2018-04-04 04:22:55,058 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:22:55,058 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1522815377.65 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1522815392.65 ... 2018-04-04 04:22:55,058 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:22:55,058 - baro_tests.barometer - INFO - OK 2018-04-04 04:22:55,058 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Mcelog. 2018-04-04 04:23:44,652 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-04-04 04:24:09,645 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_events on compute node 5f6ecc43-4550-4f6d-9eff-878b75b9a4e8... 2018-04-04 04:24:09,645 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-04 04:24:09,646 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_events-br0'] 2018-04-04 04:24:09,646 - baro_tests.barometer - DEBUG - Plugin meter categories: ['gauge-link_status'] 2018-04-04 04:24:21,941 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-04 04:24:21,941 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1522815479.65 ... 2018-04-04 04:24:21,941 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:24:21,942 - baro_tests.barometer - INFO - OK 2018-04-04 04:24:21,942 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-04 04:24:24,944 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-04 04:24:38,441 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-04 04:24:38,443 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1522815479.65, 1522815480.651)] 2018-04-04 04:24:38,444 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1522815496.651, 1522815497.651)] 2018-04-04 04:24:38,444 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1522815479.65 - ovs_events-br0 gauge-link_status 1522815496.65 ... 2018-04-04 04:24:38,444 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:24:38,444 - baro_tests.barometer - INFO - OK 2018-04-04 04:24:38,445 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on OVS events. 2018-04-04 04:25:28,853 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS stats Test case execution === 2018-04-04 04:25:53,725 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_stats on compute node 5f6ecc43-4550-4f6d-9eff-878b75b9a4e8... 2018-04-04 04:25:53,727 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-04 04:25:53,727 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_stats-br0.br0'] 2018-04-04 04:25:53,729 - baro_tests.barometer - DEBUG - Plugin meter categories: ['if_dropped', 'if_errors', 'if_packets'] 2018-04-04 04:26:07,099 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-04 04:26:07,100 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1522815584.65 ... 2018-04-04 04:26:07,100 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:26:07,100 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1522815584.65 ... 2018-04-04 04:26:07,101 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:26:07,101 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1522815584.65 ... 2018-04-04 04:26:07,101 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:26:07,101 - baro_tests.barometer - INFO - OK 2018-04-04 04:26:07,101 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-04 04:26:10,105 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-04 04:26:22,644 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-04 04:26:22,644 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1522815584.65, 1522815585.65), ('ovs_stats-br0.br0', 'if_errors', 1522815584.65, 1522815585.65), ('ovs_stats-br0.br0', 'if_packets', 1522815584.65, 1522815586.65)] 2018-04-04 04:26:22,645 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1522815600.65, 1522815601.65), ('ovs_stats-br0.br0', 'if_errors', 1522815600.65, 1522815601.65), ('ovs_stats-br0.br0', 'if_packets', 1522815600.65, 1522815601.65)] 2018-04-04 04:26:22,645 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1522815584.65 - ovs_stats-br0.br0 if_dropped 1522815600.65 ... 2018-04-04 04:26:22,645 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:26:22,646 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1522815584.65 - ovs_stats-br0.br0 if_errors 1522815600.65 ... 2018-04-04 04:26:22,646 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:26:22,646 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1522815584.65 - ovs_stats-br0.br0 if_packets 1522815600.65 ... 2018-04-04 04:26:22,646 - baro_tests.barometer - DEBUG - OK 2018-04-04 04:26:22,646 - baro_tests.barometer - INFO - OK 2018-04-04 04:26:22,647 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on OVS stats. 2018-04-04 04:26:35,245 - baro_tests.barometer - INFO - Mcelog is deleted from all compute nodes 2018-04-04 04:26:35,246 - baro_tests.barometer - INFO - ====================================================================== 2018-04-04 04:26:35,246 - baro_tests.barometer - INFO - +----------------------------------+ 2018-04-04 04:26:35,246 - baro_tests.barometer - INFO - | OVERALL SUMMARY | 2018-04-04 04:26:35,247 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-04 04:26:35,247 - baro_tests.barometer - INFO - | Test | Node-0 | Node-1 | 2018-04-04 04:26:35,247 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-04 04:26:35,247 - baro_tests.barometer - INFO - | OUT:Gnocchi | PASS | PASS | 2018-04-04 04:26:35,248 - baro_tests.barometer - INFO - | IN:Hugepages | FAIL | FAIL | 2018-04-04 04:26:35,248 - baro_tests.barometer - INFO - | IN:Intel RDT | FAIL | PASS | 2018-04-04 04:26:35,248 - baro_tests.barometer - INFO - | IN:Mcelog | PASS | PASS | 2018-04-04 04:26:35,248 - baro_tests.barometer - INFO - | IN:OVS events | FAIL | PASS | 2018-04-04 04:26:35,248 - baro_tests.barometer - INFO - | IN:OVS stats | PASS | PASS | 2018-04-04 04:26:35,248 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-04 04:26:35,248 - baro_tests.barometer - INFO - | OUT:AODH | PASS | PASS | 2018-04-04 04:26:35,248 - baro_tests.barometer - INFO - | IN:Mcelog | FAIL | FAIL | 2018-04-04 04:26:35,248 - baro_tests.barometer - INFO - | IN:OVS events | SKIP | SKIP | 2018-04-04 04:26:35,248 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-04 04:26:35,249 - baro_tests.barometer - INFO - | OUT:CSV | PASS | PASS | 2018-04-04 04:26:35,249 - baro_tests.barometer - INFO - | IN:Hugepages | PASS | PASS | 2018-04-04 04:26:35,249 - baro_tests.barometer - INFO - | IN:Intel RDT | PASS | PASS | 2018-04-04 04:26:35,249 - baro_tests.barometer - INFO - | IN:Mcelog | PASS | PASS | 2018-04-04 04:26:35,249 - baro_tests.barometer - INFO - | IN:OVS events | PASS | PASS | 2018-04-04 04:26:35,249 - baro_tests.barometer - INFO - | IN:OVS stats | PASS | PASS | 2018-04-04 04:26:35,249 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-04 04:26:35,249 - baro_tests.barometer - INFO - ====================================================================== 2018-04-04 04:26:35,249 - baro_tests.barometer - ERROR - Some tests have failed or have not been executed 2018-04-04 04:26:35,249 - baro_tests.barometer - ERROR - Overall Result is Fail