2018-04-02 15:02:52,137 - baro_tests.barometer - INFO - =================== Starting barometer tests suite =================== 2018-04-02 15:02:52,138 - baro_tests.barometer - INFO - RSA key file /root/.ssh/id_rsa exists. 2018-04-02 15:02:56,207 - baro_tests.barometer - INFO - ==== Display of Control and Compute nodes available in the set up ==== 2018-04-02 15:02:56,207 - baro_tests.barometer - INFO - controllers: ['overcloud-controller-0: 192.30.9.5', 'overcloud-controller-1: 192.30.9.8', 'overcloud-controller-2: 192.30.9.6'] 2018-04-02 15:02:56,207 - baro_tests.barometer - INFO - computes: ['overcloud-novacompute-0: 192.30.9.7', 'overcloud-novacompute-1: 192.30.9.3'] 2018-04-02 15:02:56,208 - baro_tests.barometer - INFO - ========== Enabling mcelog and OVS bridges on compute nodes ========== 2018-04-02 15:03:09,626 - baro_tests.barometer - INFO - Checking if mcelog is enabled on node-overcloud-novacompute-0... 2018-04-02 15:03:09,677 - baro_tests.barometer - INFO - Mcelog will be enabled on node-overcloud-novacompute-0... 2018-04-02 15:03:10,070 - baro_tests.barometer - INFO - Mcelog was installed on node-overcloud-novacompute-0. 2018-04-02 15:03:10,083 - baro_tests.barometer - INFO - Checking if mcelog is enabled on node-overcloud-novacompute-1... 2018-04-02 15:03:10,133 - baro_tests.barometer - INFO - Mcelog will be enabled on node-overcloud-novacompute-1... 2018-04-02 15:03:10,501 - baro_tests.barometer - INFO - Mcelog was installed on node-overcloud-novacompute-1. 2018-04-02 15:03:23,862 - baro_tests.barometer - INFO - Creating OVS bridges on computes nodes 2018-04-02 15:03:24,115 - baro_tests.barometer - INFO - OVS Bridges created on compute nodes 2018-04-02 15:03:24,116 - baro_tests.barometer - INFO - ============= Testing Gnocchi and AODH plugins on nodes ============== 2018-04-02 15:05:14,824 - baro_tests.barometer - INFO - Gnocchi plugin available in compute node overcloud-novacompute-0 2018-04-02 15:05:28,038 - baro_tests.barometer - INFO - AODH plugin present in compute node overcloud-novacompute-0 2018-04-02 15:05:28,038 - baro_tests.barometer - INFO - === NODE overcloud-novacompute-0: Enabling Test Plug-in and Test case execution === 2018-04-02 15:06:44,529 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Hugepages Test case execution === 2018-04-02 15:07:10,696 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-02 15:07:48,766 - baro_tests.barometer - INFO - Plugin Interval is 10 2018-04-02 15:07:48,767 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-04-02 15:07:48,767 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on Hugepages. 2018-04-02 15:08:41,377 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Intel RDT Test case execution === 2018-04-02 15:09:20,182 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-02 15:09:57,857 - baro_tests.barometer - INFO - PASS 2018-04-02 15:09:57,857 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with Gnocchi PASSED on Intel RDT. 2018-04-02 15:10:51,096 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-04-02 15:11:17,173 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-02 15:11:54,275 - baro_tests.barometer - INFO - PASS 2018-04-02 15:11:54,276 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with Gnocchi PASSED on Mcelog. 2018-04-02 15:12:45,417 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-04-02 15:13:11,102 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-02 15:13:13,790 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-04-02 15:13:16,465 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-1 2018-04-02 15:13:20,402 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on OVS events. 2018-04-02 15:14:12,105 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS stats Test case execution === 2018-04-02 15:14:37,748 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-02 15:15:15,756 - baro_tests.barometer - INFO - PASS 2018-04-02 15:15:15,756 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with Gnocchi PASSED on OVS stats. 2018-04-02 15:16:08,383 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-04-02 15:16:34,153 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-04-02 15:16:53,167 - baro_tests.barometer - INFO - Data not updated after interval of 12 seconds 2018-04-02 15:16:53,168 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with AODH FAILED on Mcelog. 2018-04-02 15:17:45,000 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-04-02 15:18:11,031 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-04-02 15:18:13,416 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-0 2018-04-02 15:18:16,121 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-1 2018-04-02 15:18:19,272 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with AODH FAILED on OVS events. 2018-04-02 15:18:19,273 - baro_tests.barometer - INFO - ======= Node overcloud-novacompute-0: Executing CSV Testcases ======== 2018-04-02 15:18:19,274 - baro_tests.barometer - INFO - Restarting collectd for CSV tests 2018-04-02 15:18:32,114 - baro_tests.barometer - INFO - Stopping collectd service... 2018-04-02 15:18:43,471 - baro_tests.barometer - INFO - Starting collectd service... 2018-04-02 15:18:53,772 - baro_tests.barometer - INFO - Sleeping for 10 seconds after collectd restart... 2018-04-02 15:19:54,961 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Hugepages Test case execution === 2018-04-02 15:20:19,722 - baro_tests.barometer - INFO - Getting CSV metrics of plugin hugepages on compute node 69f51022-bf88-4446-b1b9-4d1e206dc8a8... 2018-04-02 15:20:19,722 - baro_tests.barometer - DEBUG - Interval: 10 2018-04-02 15:20:19,722 - baro_tests.barometer - DEBUG - Plugin subdirs: ['hugepages-mm-2048Kb', 'hugepages-node0-2048Kb'] 2018-04-02 15:20:19,722 - baro_tests.barometer - DEBUG - Plugin meter categories: ['vmpage_number-free', 'vmpage_number-used'] 2018-04-02 15:20:33,081 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-02 15:20:33,082 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1522682035.31 ... 2018-04-02 15:20:33,082 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:20:33,083 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1522682035.31 ... 2018-04-02 15:20:33,083 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:20:33,083 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1522682035.31 ... 2018-04-02 15:20:33,083 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:20:33,083 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1522682035.31 ... 2018-04-02 15:20:33,083 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:20:33,083 - baro_tests.barometer - INFO - OK 2018-04-02 15:20:33,083 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-04-02 15:20:45,095 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-02 15:20:58,528 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-02 15:20:58,528 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1522682035.309, 1522682045.31), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1522682035.309, 1522682045.31), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1522682035.31, 1522682045.31), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1522682035.31, 1522682045.31)] 2018-04-02 15:20:58,529 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1522682065.309, 1522682075.31), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1522682065.309, 1522682075.31), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1522682065.31, 1522682075.31), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1522682065.31, 1522682075.31)] 2018-04-02 15:20:58,529 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1522682035.31 - hugepages-mm-2048Kb vmpage_number-free 1522682065.31 ... 2018-04-02 15:20:58,529 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:20:58,529 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1522682035.31 - hugepages-mm-2048Kb vmpage_number-used 1522682065.31 ... 2018-04-02 15:20:58,529 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:20:58,530 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1522682035.31 - hugepages-node0-2048Kb vmpage_number-free 1522682065.31 ... 2018-04-02 15:20:58,530 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:20:58,530 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1522682035.31 - hugepages-node0-2048Kb vmpage_number-used 1522682065.31 ... 2018-04-02 15:20:58,530 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:20:58,530 - baro_tests.barometer - INFO - OK 2018-04-02 15:20:58,530 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Hugepages. 2018-04-02 15:21:51,100 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Intel RDT Test case execution === 2018-04-02 15:22:29,575 - baro_tests.barometer - INFO - Getting CSV metrics of plugin intel_rdt on compute node 69f51022-bf88-4446-b1b9-4d1e206dc8a8... 2018-04-02 15:22:29,575 - baro_tests.barometer - DEBUG - Interval: 10 2018-04-02 15:22:29,576 - baro_tests.barometer - DEBUG - Plugin subdirs: ['intel_rdt-0-2'] 2018-04-02 15:22:29,576 - baro_tests.barometer - DEBUG - Plugin meter categories: ['bytes-llc', 'ipc'] 2018-04-02 15:22:43,246 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-02 15:22:43,246 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1522682165.31 ... 2018-04-02 15:22:43,246 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:22:43,247 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1522682165.31 ... 2018-04-02 15:22:43,247 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:22:43,247 - baro_tests.barometer - INFO - OK 2018-04-02 15:22:43,247 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-04-02 15:22:55,253 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-02 15:23:08,453 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-02 15:23:08,453 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1522682165.309, 1522682175.31), ('intel_rdt-0-2', 'ipc', 1522682165.309, 1522682175.31)] 2018-04-02 15:23:08,453 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1522682195.309, 1522682205.31), ('intel_rdt-0-2', 'ipc', 1522682195.309, 1522682205.31)] 2018-04-02 15:23:08,454 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1522682165.31 - intel_rdt-0-2 bytes-llc 1522682195.31 ... 2018-04-02 15:23:08,454 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:23:08,454 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1522682165.31 - intel_rdt-0-2 ipc 1522682195.31 ... 2018-04-02 15:23:08,454 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:23:08,455 - baro_tests.barometer - INFO - OK 2018-04-02 15:23:08,455 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Intel RDT. 2018-04-02 15:23:58,610 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-04-02 15:24:24,412 - baro_tests.barometer - INFO - Getting CSV metrics of plugin mcelog on compute node 69f51022-bf88-4446-b1b9-4d1e206dc8a8... 2018-04-02 15:24:24,413 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-02 15:24:24,413 - baro_tests.barometer - DEBUG - Plugin subdirs: ['mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'mcelog-SOCKET_0_CHANNEL_any_DIMM_any'] 2018-04-02 15:24:24,413 - baro_tests.barometer - DEBUG - Plugin meter categories: ['errors-corrected_memory_errors', 'errors-uncorrected_memory_errors'] 2018-04-02 15:24:37,824 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-02 15:24:37,825 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1522682297.31 ... 2018-04-02 15:24:37,825 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:24:37,826 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1522682297.31 ... 2018-04-02 15:24:37,826 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:24:37,826 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1522682297.31 ... 2018-04-02 15:24:37,826 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:24:37,827 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1522682298.31 ... 2018-04-02 15:24:37,827 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:24:37,827 - baro_tests.barometer - INFO - OK 2018-04-02 15:24:37,827 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-02 15:24:40,830 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-02 15:24:53,482 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-02 15:24:53,482 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1522682297.311, 1522682298.311), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1522682297.311, 1522682298.311), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1522682297.311, 1522682298.311), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1522682298.311, 1522682299.311)] 2018-04-02 15:24:53,483 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1522682313.311, 1522682314.311), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1522682313.311, 1522682314.311), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1522682313.311, 1522682314.311), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1522682313.311, 1522682314.311)] 2018-04-02 15:24:53,483 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1522682297.31 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1522682313.31 ... 2018-04-02 15:24:53,483 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:24:53,484 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1522682297.31 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1522682313.31 ... 2018-04-02 15:24:53,484 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:24:53,484 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1522682297.31 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1522682313.31 ... 2018-04-02 15:24:53,484 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:24:53,484 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1522682298.31 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1522682313.31 ... 2018-04-02 15:24:53,484 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:24:53,484 - baro_tests.barometer - INFO - OK 2018-04-02 15:24:53,485 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Mcelog. 2018-04-02 15:25:45,781 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-04-02 15:26:11,252 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_events on compute node 69f51022-bf88-4446-b1b9-4d1e206dc8a8... 2018-04-02 15:26:11,252 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-02 15:26:11,253 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_events-br0'] 2018-04-02 15:26:11,253 - baro_tests.barometer - DEBUG - Plugin meter categories: ['gauge-link_status'] 2018-04-02 15:26:23,813 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-02 15:26:23,813 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1522682403.31 ... 2018-04-02 15:26:23,814 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:26:23,814 - baro_tests.barometer - INFO - OK 2018-04-02 15:26:23,814 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-02 15:26:26,818 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-02 15:26:39,607 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-02 15:26:39,608 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1522682403.311, 1522682404.311)] 2018-04-02 15:26:39,608 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1522682419.311, 1522682420.311)] 2018-04-02 15:26:39,608 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1522682403.31 - ovs_events-br0 gauge-link_status 1522682419.31 ... 2018-04-02 15:26:39,609 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:26:39,609 - baro_tests.barometer - INFO - OK 2018-04-02 15:26:39,610 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on OVS events. 2018-04-02 15:27:31,520 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS stats Test case execution === 2018-04-02 15:27:55,882 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_stats on compute node 69f51022-bf88-4446-b1b9-4d1e206dc8a8... 2018-04-02 15:27:55,883 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-02 15:27:55,883 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_stats-br0.br0'] 2018-04-02 15:27:55,883 - baro_tests.barometer - DEBUG - Plugin meter categories: ['if_dropped', 'if_errors', 'if_packets'] 2018-04-02 15:28:09,227 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-02 15:28:09,228 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1522682509.31 ... 2018-04-02 15:28:09,228 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:28:09,228 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1522682509.31 ... 2018-04-02 15:28:09,228 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:28:09,228 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1522682509.31 ... 2018-04-02 15:28:09,229 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:28:09,229 - baro_tests.barometer - INFO - OK 2018-04-02 15:28:09,229 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-02 15:28:12,229 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-04-02 15:28:25,994 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-02 15:28:25,995 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1522682509.311, 1522682510.311), ('ovs_stats-br0.br0', 'if_errors', 1522682509.311, 1522682510.311), ('ovs_stats-br0.br0', 'if_packets', 1522682509.311, 1522682510.311)] 2018-04-02 15:28:25,995 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1522682525.311, 1522682527.311), ('ovs_stats-br0.br0', 'if_errors', 1522682526.311, 1522682527.311), ('ovs_stats-br0.br0', 'if_packets', 1522682526.311, 1522682527.311)] 2018-04-02 15:28:25,996 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1522682509.31 - ovs_stats-br0.br0 if_dropped 1522682525.31 ... 2018-04-02 15:28:25,996 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:28:25,998 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1522682509.31 - ovs_stats-br0.br0 if_errors 1522682526.31 ... 2018-04-02 15:28:25,998 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:28:25,999 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1522682509.31 - ovs_stats-br0.br0 if_packets 1522682526.31 ... 2018-04-02 15:28:25,999 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:28:26,000 - baro_tests.barometer - INFO - OK 2018-04-02 15:28:26,003 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on OVS stats. 2018-04-02 15:28:39,241 - baro_tests.barometer - INFO - Gnocchi plugin available in compute node overcloud-novacompute-1 2018-04-02 15:28:51,539 - baro_tests.barometer - INFO - AODH plugin present in compute node overcloud-novacompute-1 2018-04-02 15:28:51,540 - baro_tests.barometer - INFO - === NODE overcloud-novacompute-1: Enabling Test Plug-in and Test case execution === 2018-04-02 15:30:06,662 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Hugepages Test case execution === 2018-04-02 15:30:32,100 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-02 15:31:09,297 - baro_tests.barometer - INFO - Plugin Interval is 10 2018-04-02 15:31:09,298 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-04-02 15:31:09,298 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on Hugepages. 2018-04-02 15:32:01,648 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Intel RDT Test case execution === 2018-04-02 15:32:39,011 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-02 15:33:16,354 - baro_tests.barometer - INFO - PASS 2018-04-02 15:33:16,355 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with Gnocchi PASSED on Intel RDT. 2018-04-02 15:34:08,184 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-04-02 15:34:32,187 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-02 15:34:34,886 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-04-02 15:34:37,490 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-1 2018-04-02 15:34:40,181 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on Mcelog. 2018-04-02 15:35:28,881 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-04-02 15:35:53,133 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-02 15:36:30,364 - baro_tests.barometer - INFO - Plugin Interval is 1 2018-04-02 15:36:30,365 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-04-02 15:36:30,365 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on OVS events. 2018-04-02 15:37:20,861 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS stats Test case execution === 2018-04-02 15:37:45,052 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-04-02 15:38:22,602 - baro_tests.barometer - INFO - PASS 2018-04-02 15:38:22,603 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with Gnocchi PASSED on OVS stats. 2018-04-02 15:39:11,874 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-04-02 15:39:37,216 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-04-02 15:39:56,177 - baro_tests.barometer - INFO - Data not updated after interval of 12 seconds 2018-04-02 15:39:56,178 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with AODH FAILED on Mcelog. 2018-04-02 15:40:46,123 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-04-02 15:41:10,983 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-04-02 15:41:13,330 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-0 2018-04-02 15:41:15,638 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-1 2018-04-02 15:41:17,942 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with AODH FAILED on OVS events. 2018-04-02 15:41:17,943 - baro_tests.barometer - INFO - ======= Node overcloud-novacompute-1: Executing CSV Testcases ======== 2018-04-02 15:41:17,943 - baro_tests.barometer - INFO - Restarting collectd for CSV tests 2018-04-02 15:41:31,044 - baro_tests.barometer - INFO - Stopping collectd service... 2018-04-02 15:41:42,716 - baro_tests.barometer - INFO - Starting collectd service... 2018-04-02 15:41:53,329 - baro_tests.barometer - INFO - Sleeping for 10 seconds after collectd restart... 2018-04-02 15:42:54,614 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Hugepages Test case execution === 2018-04-02 15:43:19,661 - baro_tests.barometer - INFO - Getting CSV metrics of plugin hugepages on compute node 97adb5ea-d197-4e7c-bed9-03a17b269d46... 2018-04-02 15:43:19,661 - baro_tests.barometer - DEBUG - Interval: 10 2018-04-02 15:43:19,662 - baro_tests.barometer - DEBUG - Plugin subdirs: ['hugepages-mm-2048Kb', 'hugepages-node0-2048Kb'] 2018-04-02 15:43:19,662 - baro_tests.barometer - DEBUG - Plugin meter categories: ['vmpage_number-free', 'vmpage_number-used'] 2018-04-02 15:43:33,274 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-02 15:43:33,275 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1522683401.65 ... 2018-04-02 15:43:33,277 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:43:33,278 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1522683401.65 ... 2018-04-02 15:43:33,279 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:43:33,280 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1522683401.65 ... 2018-04-02 15:43:33,284 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:43:33,286 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1522683401.65 ... 2018-04-02 15:43:33,289 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:43:33,289 - baro_tests.barometer - INFO - OK 2018-04-02 15:43:33,289 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-04-02 15:43:45,301 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-02 15:43:58,858 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-02 15:43:58,858 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1522683401.65, 1522683411.65), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1522683401.65, 1522683411.65), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1522683401.65, 1522683411.65), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1522683401.65, 1522683411.65)] 2018-04-02 15:43:58,859 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1522683431.65, 1522683441.649), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1522683431.65, 1522683441.649), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1522683431.65, 1522683441.65), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1522683431.65, 1522683441.65)] 2018-04-02 15:43:58,859 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1522683401.65 - hugepages-mm-2048Kb vmpage_number-free 1522683431.65 ... 2018-04-02 15:43:58,859 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:43:58,860 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1522683401.65 - hugepages-mm-2048Kb vmpage_number-used 1522683431.65 ... 2018-04-02 15:43:58,860 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:43:58,860 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1522683401.65 - hugepages-node0-2048Kb vmpage_number-free 1522683431.65 ... 2018-04-02 15:43:58,860 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:43:58,860 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1522683401.65 - hugepages-node0-2048Kb vmpage_number-used 1522683431.65 ... 2018-04-02 15:43:58,860 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:43:58,860 - baro_tests.barometer - INFO - OK 2018-04-02 15:43:58,861 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Hugepages. 2018-04-02 15:44:49,178 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Intel RDT Test case execution === 2018-04-02 15:45:26,504 - baro_tests.barometer - INFO - Getting CSV metrics of plugin intel_rdt on compute node 97adb5ea-d197-4e7c-bed9-03a17b269d46... 2018-04-02 15:45:26,504 - baro_tests.barometer - DEBUG - Interval: 10 2018-04-02 15:45:26,505 - baro_tests.barometer - DEBUG - Plugin subdirs: ['intel_rdt-0-2'] 2018-04-02 15:45:26,506 - baro_tests.barometer - DEBUG - Plugin meter categories: ['bytes-llc', 'ipc'] 2018-04-02 15:45:39,036 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-02 15:45:39,036 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1522683531.65 ... 2018-04-02 15:45:39,036 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:45:39,037 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1522683531.65 ... 2018-04-02 15:45:39,038 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:45:39,038 - baro_tests.barometer - INFO - OK 2018-04-02 15:45:39,038 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-04-02 15:45:51,051 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-02 15:46:03,375 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-02 15:46:03,375 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1522683531.65, 1522683541.65), ('intel_rdt-0-2', 'ipc', 1522683531.65, 1522683541.65)] 2018-04-02 15:46:03,375 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1522683551.65, 1522683561.65), ('intel_rdt-0-2', 'ipc', 1522683551.65, 1522683561.65)] 2018-04-02 15:46:03,376 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1522683531.65 - intel_rdt-0-2 bytes-llc 1522683551.65 ... 2018-04-02 15:46:03,376 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:46:03,377 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1522683531.65 - intel_rdt-0-2 ipc 1522683551.65 ... 2018-04-02 15:46:03,377 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:46:03,377 - baro_tests.barometer - INFO - OK 2018-04-02 15:46:03,379 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Intel RDT. 2018-04-02 15:46:53,065 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-04-02 15:47:17,415 - baro_tests.barometer - INFO - Getting CSV metrics of plugin mcelog on compute node 97adb5ea-d197-4e7c-bed9-03a17b269d46... 2018-04-02 15:47:17,416 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-02 15:47:17,416 - baro_tests.barometer - DEBUG - Plugin subdirs: ['mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'mcelog-SOCKET_0_CHANNEL_any_DIMM_any'] 2018-04-02 15:47:17,416 - baro_tests.barometer - DEBUG - Plugin meter categories: ['errors-corrected_memory_errors', 'errors-uncorrected_memory_errors'] 2018-04-02 15:47:31,142 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-02 15:47:31,142 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1522683657.65 ... 2018-04-02 15:47:31,143 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:47:31,143 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1522683657.65 ... 2018-04-02 15:47:31,143 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:47:31,143 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1522683657.65 ... 2018-04-02 15:47:31,144 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:47:31,144 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1522683657.65 ... 2018-04-02 15:47:31,144 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:47:31,144 - baro_tests.barometer - INFO - OK 2018-04-02 15:47:31,144 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-02 15:47:34,145 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-02 15:47:46,681 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-02 15:47:46,681 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1522683657.651, 1522683658.651), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1522683657.651, 1522683658.651), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1522683657.651, 1522683658.651), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1522683657.651, 1522683658.651)] 2018-04-02 15:47:46,681 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1522683673.651, 1522683674.651), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1522683673.651, 1522683674.651), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1522683673.651, 1522683674.651), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1522683673.651, 1522683674.651)] 2018-04-02 15:47:46,682 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1522683657.65 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1522683673.65 ... 2018-04-02 15:47:46,683 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:47:46,683 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1522683657.65 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1522683673.65 ... 2018-04-02 15:47:46,684 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:47:46,685 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1522683657.65 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1522683673.65 ... 2018-04-02 15:47:46,685 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:47:46,685 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1522683657.65 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1522683673.65 ... 2018-04-02 15:47:46,685 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:47:46,685 - baro_tests.barometer - INFO - OK 2018-04-02 15:47:46,685 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Mcelog. 2018-04-02 15:48:35,923 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-04-02 15:49:00,653 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_events on compute node 97adb5ea-d197-4e7c-bed9-03a17b269d46... 2018-04-02 15:49:00,654 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-02 15:49:00,654 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_events-br0'] 2018-04-02 15:49:00,654 - baro_tests.barometer - DEBUG - Plugin meter categories: ['gauge-link_status'] 2018-04-02 15:49:13,101 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-02 15:49:13,101 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1522683759.65 ... 2018-04-02 15:49:13,102 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:49:13,102 - baro_tests.barometer - INFO - OK 2018-04-02 15:49:13,102 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-02 15:49:16,104 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-02 15:49:28,442 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-02 15:49:28,444 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1522683759.651, 1522683760.651)] 2018-04-02 15:49:28,445 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1522683775.651, 1522683776.651)] 2018-04-02 15:49:28,446 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1522683759.65 - ovs_events-br0 gauge-link_status 1522683775.65 ... 2018-04-02 15:49:28,446 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:49:28,446 - baro_tests.barometer - INFO - OK 2018-04-02 15:49:28,447 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on OVS events. 2018-04-02 15:50:17,859 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS stats Test case execution === 2018-04-02 15:50:42,541 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_stats on compute node 97adb5ea-d197-4e7c-bed9-03a17b269d46... 2018-04-02 15:50:42,542 - baro_tests.barometer - DEBUG - Interval: 1 2018-04-02 15:50:42,542 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_stats-br0.br0'] 2018-04-02 15:50:42,542 - baro_tests.barometer - DEBUG - Plugin meter categories: ['if_dropped', 'if_errors', 'if_packets'] 2018-04-02 15:50:55,362 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-04-02 15:50:55,362 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1522683861.65 ... 2018-04-02 15:50:55,363 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:50:55,364 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1522683861.65 ... 2018-04-02 15:50:55,364 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:50:55,364 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1522683861.65 ... 2018-04-02 15:50:55,365 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:50:55,365 - baro_tests.barometer - INFO - OK 2018-04-02 15:50:55,365 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-04-02 15:50:58,368 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-04-02 15:51:11,036 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-04-02 15:51:11,036 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1522683861.651, 1522683862.651), ('ovs_stats-br0.br0', 'if_errors', 1522683861.651, 1522683862.651), ('ovs_stats-br0.br0', 'if_packets', 1522683861.651, 1522683863.651)] 2018-04-02 15:51:11,037 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1522683877.651, 1522683878.651), ('ovs_stats-br0.br0', 'if_errors', 1522683877.651, 1522683878.651), ('ovs_stats-br0.br0', 'if_packets', 1522683877.651, 1522683878.651)] 2018-04-02 15:51:11,039 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1522683861.65 - ovs_stats-br0.br0 if_dropped 1522683877.65 ... 2018-04-02 15:51:11,040 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:51:11,040 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1522683861.65 - ovs_stats-br0.br0 if_errors 1522683877.65 ... 2018-04-02 15:51:11,040 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:51:11,040 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1522683861.65 - ovs_stats-br0.br0 if_packets 1522683877.65 ... 2018-04-02 15:51:11,040 - baro_tests.barometer - DEBUG - OK 2018-04-02 15:51:11,040 - baro_tests.barometer - INFO - OK 2018-04-02 15:51:11,040 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on OVS stats. 2018-04-02 15:51:23,406 - baro_tests.barometer - INFO - Mcelog is deleted from all compute nodes 2018-04-02 15:51:23,409 - baro_tests.barometer - INFO - ====================================================================== 2018-04-02 15:51:23,410 - baro_tests.barometer - INFO - +----------------------------------+ 2018-04-02 15:51:23,412 - baro_tests.barometer - INFO - | OVERALL SUMMARY | 2018-04-02 15:51:23,413 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-02 15:51:23,413 - baro_tests.barometer - INFO - | Test | Node-0 | Node-1 | 2018-04-02 15:51:23,414 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-02 15:51:23,415 - baro_tests.barometer - INFO - | OUT:Gnocchi | PASS | PASS | 2018-04-02 15:51:23,415 - baro_tests.barometer - INFO - | IN:Hugepages | FAIL | FAIL | 2018-04-02 15:51:23,417 - baro_tests.barometer - INFO - | IN:Intel RDT | PASS | PASS | 2018-04-02 15:51:23,417 - baro_tests.barometer - INFO - | IN:Mcelog | PASS | FAIL | 2018-04-02 15:51:23,417 - baro_tests.barometer - INFO - | IN:OVS events | FAIL | FAIL | 2018-04-02 15:51:23,417 - baro_tests.barometer - INFO - | IN:OVS stats | PASS | PASS | 2018-04-02 15:51:23,417 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-02 15:51:23,417 - baro_tests.barometer - INFO - | OUT:AODH | PASS | PASS | 2018-04-02 15:51:23,418 - baro_tests.barometer - INFO - | IN:Mcelog | FAIL | FAIL | 2018-04-02 15:51:23,418 - baro_tests.barometer - INFO - | IN:OVS events | SKIP | SKIP | 2018-04-02 15:51:23,418 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-02 15:51:23,418 - baro_tests.barometer - INFO - | OUT:CSV | PASS | PASS | 2018-04-02 15:51:23,418 - baro_tests.barometer - INFO - | IN:Hugepages | PASS | PASS | 2018-04-02 15:51:23,418 - baro_tests.barometer - INFO - | IN:Intel RDT | PASS | PASS | 2018-04-02 15:51:23,418 - baro_tests.barometer - INFO - | IN:Mcelog | PASS | PASS | 2018-04-02 15:51:23,418 - baro_tests.barometer - INFO - | IN:OVS events | PASS | PASS | 2018-04-02 15:51:23,418 - baro_tests.barometer - INFO - | IN:OVS stats | PASS | PASS | 2018-04-02 15:51:23,418 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-04-02 15:51:23,419 - baro_tests.barometer - INFO - ====================================================================== 2018-04-02 15:51:23,419 - baro_tests.barometer - ERROR - Some tests have failed or have not been executed 2018-04-02 15:51:23,419 - baro_tests.barometer - ERROR - Overall Result is Fail