2018-03-31 09:30:47,916 - baro_tests.barometer - INFO - =================== Starting barometer tests suite =================== 2018-03-31 09:30:47,916 - baro_tests.barometer - INFO - RSA key file /root/.ssh/id_rsa exists. 2018-03-31 09:30:51,947 - baro_tests.barometer - INFO - ==== Display of Control and Compute nodes available in the set up ==== 2018-03-31 09:30:51,947 - baro_tests.barometer - INFO - controllers: ['overcloud-controller-0: 192.30.9.7', 'overcloud-controller-1: 192.30.9.8', 'overcloud-controller-2: 192.30.9.9'] 2018-03-31 09:30:51,948 - baro_tests.barometer - INFO - computes: ['overcloud-novacompute-0: 192.30.9.6', 'overcloud-novacompute-1: 192.30.9.10'] 2018-03-31 09:30:51,948 - baro_tests.barometer - INFO - ========== Enabling mcelog and OVS bridges on compute nodes ========== 2018-03-31 09:31:05,794 - baro_tests.barometer - INFO - Checking if mcelog is enabled on node-overcloud-novacompute-0... 2018-03-31 09:31:05,845 - baro_tests.barometer - INFO - Mcelog will be enabled on node-overcloud-novacompute-0... 2018-03-31 09:31:06,215 - baro_tests.barometer - INFO - Mcelog was installed on node-overcloud-novacompute-0. 2018-03-31 09:31:06,227 - baro_tests.barometer - INFO - Checking if mcelog is enabled on node-overcloud-novacompute-1... 2018-03-31 09:31:06,279 - baro_tests.barometer - INFO - Mcelog will be enabled on node-overcloud-novacompute-1... 2018-03-31 09:31:06,643 - baro_tests.barometer - INFO - Mcelog was installed on node-overcloud-novacompute-1. 2018-03-31 09:31:21,610 - baro_tests.barometer - INFO - Creating OVS bridges on computes nodes 2018-03-31 09:31:21,866 - baro_tests.barometer - INFO - OVS Bridges created on compute nodes 2018-03-31 09:31:21,866 - baro_tests.barometer - INFO - ============= Testing Gnocchi and AODH plugins on nodes ============== 2018-03-31 09:33:15,502 - baro_tests.barometer - INFO - Gnocchi plugin available in compute node overcloud-novacompute-0 2018-03-31 09:33:28,751 - baro_tests.barometer - INFO - AODH plugin present in compute node overcloud-novacompute-0 2018-03-31 09:33:28,751 - baro_tests.barometer - INFO - === NODE overcloud-novacompute-0: Enabling Test Plug-in and Test case execution === 2018-03-31 09:34:49,027 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Hugepages Test case execution === 2018-03-31 09:35:17,198 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-03-31 09:35:54,693 - baro_tests.barometer - INFO - PASS 2018-03-31 09:35:54,694 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with Gnocchi PASSED on Hugepages. 2018-03-31 09:36:47,971 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Intel RDT Test case execution === 2018-03-31 09:37:29,523 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-03-31 09:38:07,829 - baro_tests.barometer - INFO - Plugin Interval is 10 2018-03-31 09:38:07,830 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-03-31 09:38:07,831 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on Intel RDT. 2018-03-31 09:39:00,853 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-03-31 09:39:27,628 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-03-31 09:40:04,740 - baro_tests.barometer - INFO - PASS 2018-03-31 09:40:04,741 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with Gnocchi PASSED on Mcelog. 2018-03-31 09:40:59,449 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-03-31 09:41:26,274 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-03-31 09:41:28,924 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-03-31 09:41:31,583 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-1 2018-03-31 09:41:34,378 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on OVS events. 2018-03-31 09:42:26,730 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS stats Test case execution === 2018-03-31 09:42:52,532 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-03-31 09:43:29,605 - baro_tests.barometer - INFO - Plugin Interval is 1 2018-03-31 09:43:29,605 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-03-31 09:43:29,606 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with Gnocchi FAILED on OVS stats. 2018-03-31 09:44:22,885 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-03-31 09:44:48,847 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-03-31 09:45:08,089 - baro_tests.barometer - INFO - Data not updated after interval of 12 seconds 2018-03-31 09:45:08,090 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with AODH FAILED on Mcelog. 2018-03-31 09:46:01,794 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-03-31 09:46:27,670 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-03-31 09:46:30,050 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-0 2018-03-31 09:46:32,488 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-1 2018-03-31 09:46:36,077 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-0 with AODH FAILED on OVS events. 2018-03-31 09:46:36,078 - baro_tests.barometer - INFO - ======= Node overcloud-novacompute-0: Executing CSV Testcases ======== 2018-03-31 09:46:36,078 - baro_tests.barometer - INFO - Restarting collectd for CSV tests 2018-03-31 09:46:49,223 - baro_tests.barometer - INFO - Stopping collectd service... 2018-03-31 09:47:00,011 - baro_tests.barometer - INFO - Starting collectd service... 2018-03-31 09:47:10,312 - baro_tests.barometer - INFO - Sleeping for 10 seconds after collectd restart... 2018-03-31 09:48:13,071 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Hugepages Test case execution === 2018-03-31 09:48:39,472 - baro_tests.barometer - INFO - Getting CSV metrics of plugin hugepages on compute node 3a3fa4e9-7c94-48fe-b6a0-0ce667fe4b5b... 2018-03-31 09:48:39,472 - baro_tests.barometer - DEBUG - Interval: 10 2018-03-31 09:48:39,472 - baro_tests.barometer - DEBUG - Plugin subdirs: ['hugepages-mm-2048Kb', 'hugepages-node0-2048Kb'] 2018-03-31 09:48:39,473 - baro_tests.barometer - DEBUG - Plugin meter categories: ['vmpage_number-free', 'vmpage_number-used'] 2018-03-31 09:48:53,405 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-03-31 09:48:53,405 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1522489329.78 ... 2018-03-31 09:48:53,405 - baro_tests.barometer - DEBUG - OK 2018-03-31 09:48:53,406 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1522489329.78 ... 2018-03-31 09:48:53,406 - baro_tests.barometer - DEBUG - OK 2018-03-31 09:48:53,406 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1522489329.78 ... 2018-03-31 09:48:53,406 - baro_tests.barometer - DEBUG - OK 2018-03-31 09:48:53,406 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1522489329.78 ... 2018-03-31 09:48:53,406 - baro_tests.barometer - DEBUG - OK 2018-03-31 09:48:53,407 - baro_tests.barometer - INFO - OK 2018-03-31 09:48:53,407 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-03-31 09:49:05,409 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-03-31 09:49:19,745 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-03-31 09:49:19,746 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1522489329.781, 1522489339.781), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1522489329.781, 1522489339.781), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1522489329.781, 1522489339.781), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1522489329.781, 1522489339.781)] 2018-03-31 09:49:19,746 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1522489349.781, 1522489359.781), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1522489349.781, 1522489359.781), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1522489349.781, 1522489359.781), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1522489349.781, 1522489359.781)] 2018-03-31 09:49:19,746 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1522489329.78 - hugepages-mm-2048Kb vmpage_number-free 1522489349.78 ... 2018-03-31 09:49:19,746 - baro_tests.barometer - DEBUG - OK 2018-03-31 09:49:19,747 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1522489329.78 - hugepages-mm-2048Kb vmpage_number-used 1522489349.78 ... 2018-03-31 09:49:19,747 - baro_tests.barometer - DEBUG - OK 2018-03-31 09:49:19,747 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1522489329.78 - hugepages-node0-2048Kb vmpage_number-free 1522489349.78 ... 2018-03-31 09:49:19,747 - baro_tests.barometer - DEBUG - OK 2018-03-31 09:49:19,747 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1522489329.78 - hugepages-node0-2048Kb vmpage_number-used 1522489349.78 ... 2018-03-31 09:49:19,747 - baro_tests.barometer - DEBUG - OK 2018-03-31 09:49:19,748 - baro_tests.barometer - INFO - OK 2018-03-31 09:49:19,748 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Hugepages. 2018-03-31 09:50:11,348 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Intel RDT Test case execution === 2018-03-31 09:50:48,852 - baro_tests.barometer - INFO - Getting CSV metrics of plugin intel_rdt on compute node 3a3fa4e9-7c94-48fe-b6a0-0ce667fe4b5b... 2018-03-31 09:50:48,852 - baro_tests.barometer - DEBUG - Interval: 10 2018-03-31 09:50:48,852 - baro_tests.barometer - DEBUG - Plugin subdirs: ['intel_rdt-0-2'] 2018-03-31 09:50:48,853 - baro_tests.barometer - DEBUG - Plugin meter categories: ['bytes-llc', 'ipc'] 2018-03-31 09:51:01,542 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-03-31 09:51:01,543 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1522489459.78 ... 2018-03-31 09:51:01,543 - baro_tests.barometer - DEBUG - OK 2018-03-31 09:51:01,544 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1522489459.78 ... 2018-03-31 09:51:01,544 - baro_tests.barometer - DEBUG - OK 2018-03-31 09:51:01,544 - baro_tests.barometer - INFO - OK 2018-03-31 09:51:01,544 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-03-31 09:51:13,547 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-03-31 09:51:27,108 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-03-31 09:51:27,109 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1522489459.781, 1522489469.781), ('intel_rdt-0-2', 'ipc', 1522489459.781, 1522489469.781)] 2018-03-31 09:51:27,109 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1522489479.781, 1522489489.781), ('intel_rdt-0-2', 'ipc', 1522489479.781, 1522489489.781)] 2018-03-31 09:51:27,109 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1522489459.78 - intel_rdt-0-2 bytes-llc 1522489479.78 ... 2018-03-31 09:51:27,110 - baro_tests.barometer - DEBUG - OK 2018-03-31 09:51:27,110 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1522489459.78 - intel_rdt-0-2 ipc 1522489479.78 ... 2018-03-31 09:51:27,110 - baro_tests.barometer - DEBUG - OK 2018-03-31 09:51:27,110 - baro_tests.barometer - INFO - OK 2018-03-31 09:51:27,110 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Intel RDT. 2018-03-31 09:52:18,210 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in Mcelog Test case execution === 2018-03-31 09:52:44,018 - baro_tests.barometer - INFO - Getting CSV metrics of plugin mcelog on compute node 3a3fa4e9-7c94-48fe-b6a0-0ce667fe4b5b... 2018-03-31 09:52:44,019 - baro_tests.barometer - DEBUG - Interval: 1 2018-03-31 09:52:44,019 - baro_tests.barometer - DEBUG - Plugin subdirs: ['mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'mcelog-SOCKET_0_CHANNEL_any_DIMM_any'] 2018-03-31 09:52:44,019 - baro_tests.barometer - DEBUG - Plugin meter categories: ['errors-corrected_memory_errors', 'errors-uncorrected_memory_errors'] 2018-03-31 09:52:56,893 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-03-31 09:52:56,893 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1522489584.78 ... 2018-03-31 09:52:56,893 - baro_tests.barometer - DEBUG - OK 2018-03-31 09:52:56,894 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1522489584.78 ... 2018-03-31 09:52:56,894 - baro_tests.barometer - DEBUG - OK 2018-03-31 09:52:56,894 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1522489584.78 ... 2018-03-31 09:52:56,894 - baro_tests.barometer - DEBUG - OK 2018-03-31 09:52:56,895 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1522489584.78 ... 2018-03-31 09:52:56,895 - baro_tests.barometer - DEBUG - OK 2018-03-31 09:52:56,895 - baro_tests.barometer - INFO - OK 2018-03-31 09:52:56,895 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-03-31 09:52:59,899 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-03-31 09:53:12,338 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-03-31 09:53:12,338 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1522489584.782, 1522489585.782), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1522489584.782, 1522489585.782), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1522489584.782, 1522489585.782), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1522489584.782, 1522489585.782)] 2018-03-31 09:53:12,340 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1522489599.782, 1522489600.782), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1522489599.782, 1522489600.782), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1522489599.782, 1522489600.782), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1522489599.782, 1522489601.782)] 2018-03-31 09:53:12,340 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1522489584.78 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1522489599.78 ... 2018-03-31 09:53:12,341 - baro_tests.barometer - DEBUG - OK 2018-03-31 09:53:12,341 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1522489584.78 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1522489599.78 ... 2018-03-31 09:53:12,342 - baro_tests.barometer - DEBUG - OK 2018-03-31 09:53:12,342 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1522489584.78 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1522489599.78 ... 2018-03-31 09:53:12,344 - baro_tests.barometer - DEBUG - OK 2018-03-31 09:53:12,345 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1522489584.78 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1522489599.78 ... 2018-03-31 09:53:12,346 - baro_tests.barometer - DEBUG - OK 2018-03-31 09:53:12,346 - baro_tests.barometer - INFO - OK 2018-03-31 09:53:12,352 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on Mcelog. 2018-03-31 09:54:03,949 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS events Test case execution === 2018-03-31 09:54:28,786 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_events on compute node 3a3fa4e9-7c94-48fe-b6a0-0ce667fe4b5b... 2018-03-31 09:54:28,787 - baro_tests.barometer - DEBUG - Interval: 1 2018-03-31 09:54:28,788 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_events-br0'] 2018-03-31 09:54:28,790 - baro_tests.barometer - DEBUG - Plugin meter categories: ['gauge-link_status'] 2018-03-31 09:54:40,974 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-03-31 09:54:40,974 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1522489688.78 ... 2018-03-31 09:54:40,974 - baro_tests.barometer - DEBUG - OK 2018-03-31 09:54:40,974 - baro_tests.barometer - INFO - OK 2018-03-31 09:54:40,974 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-03-31 09:54:43,978 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-03-31 09:54:55,579 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-03-31 09:54:55,579 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1522489688.782, 1522489689.782)] 2018-03-31 09:54:55,580 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1522489703.782, 1522489704.782)] 2018-03-31 09:54:55,580 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1522489688.78 - ovs_events-br0 gauge-link_status 1522489703.78 ... 2018-03-31 09:54:55,580 - baro_tests.barometer - DEBUG - OK 2018-03-31 09:54:55,581 - baro_tests.barometer - INFO - OK 2018-03-31 09:54:55,581 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on OVS events. 2018-03-31 09:55:47,689 - baro_tests.barometer - INFO - === Node overcloud-novacompute-0: Plug-in OVS stats Test case execution === 2018-03-31 09:56:12,224 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_stats on compute node 3a3fa4e9-7c94-48fe-b6a0-0ce667fe4b5b... 2018-03-31 09:56:12,225 - baro_tests.barometer - DEBUG - Interval: 1 2018-03-31 09:56:12,225 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_stats-br0.br0'] 2018-03-31 09:56:12,225 - baro_tests.barometer - DEBUG - Plugin meter categories: ['if_dropped', 'if_errors', 'if_packets'] 2018-03-31 09:56:24,897 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-03-31 09:56:24,897 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1522489792.78 ... 2018-03-31 09:56:24,897 - baro_tests.barometer - DEBUG - OK 2018-03-31 09:56:24,898 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1522489792.78 ... 2018-03-31 09:56:24,898 - baro_tests.barometer - DEBUG - OK 2018-03-31 09:56:24,898 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1522489792.78 ... 2018-03-31 09:56:24,899 - baro_tests.barometer - DEBUG - OK 2018-03-31 09:56:24,900 - baro_tests.barometer - INFO - OK 2018-03-31 09:56:24,901 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-03-31 09:56:27,904 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-0... 2018-03-31 09:56:41,289 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-03-31 09:56:41,289 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1522489792.782, 1522489793.782), ('ovs_stats-br0.br0', 'if_errors', 1522489792.782, 1522489793.782), ('ovs_stats-br0.br0', 'if_packets', 1522489792.782, 1522489793.782)] 2018-03-31 09:56:41,290 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1522489808.782, 1522489809.782), ('ovs_stats-br0.br0', 'if_errors', 1522489808.782, 1522489809.782), ('ovs_stats-br0.br0', 'if_packets', 1522489808.782, 1522489809.782)] 2018-03-31 09:56:41,290 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1522489792.78 - ovs_stats-br0.br0 if_dropped 1522489808.78 ... 2018-03-31 09:56:41,290 - baro_tests.barometer - DEBUG - OK 2018-03-31 09:56:41,291 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1522489792.78 - ovs_stats-br0.br0 if_errors 1522489808.78 ... 2018-03-31 09:56:41,291 - baro_tests.barometer - DEBUG - OK 2018-03-31 09:56:41,291 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1522489792.78 - ovs_stats-br0.br0 if_packets 1522489808.78 ... 2018-03-31 09:56:41,292 - baro_tests.barometer - DEBUG - OK 2018-03-31 09:56:41,293 - baro_tests.barometer - INFO - OK 2018-03-31 09:56:41,294 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-0 with CSV PASSED on OVS stats. 2018-03-31 09:56:53,264 - baro_tests.barometer - INFO - Gnocchi plugin available in compute node overcloud-novacompute-1 2018-03-31 09:57:06,037 - baro_tests.barometer - INFO - AODH plugin present in compute node overcloud-novacompute-1 2018-03-31 09:57:06,038 - baro_tests.barometer - INFO - === NODE overcloud-novacompute-1: Enabling Test Plug-in and Test case execution === 2018-03-31 09:58:26,154 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Hugepages Test case execution === 2018-03-31 09:58:53,883 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-03-31 09:59:31,151 - baro_tests.barometer - INFO - Plugin Interval is 10 2018-03-31 09:59:31,151 - baro_tests.barometer - INFO - Data not updated after 30 seconds 2018-03-31 09:59:31,155 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on Hugepages. 2018-03-31 10:00:22,673 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Intel RDT Test case execution === 2018-03-31 10:00:58,624 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-03-31 10:01:01,220 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-03-31 10:01:03,813 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-1 2018-03-31 10:01:06,392 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on Intel RDT. 2018-03-31 10:01:57,618 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-03-31 10:02:23,000 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-03-31 10:02:25,664 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-03-31 10:02:28,431 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-1 2018-03-31 10:02:31,125 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on Mcelog. 2018-03-31 10:03:22,894 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-03-31 10:03:47,238 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-03-31 10:04:24,526 - baro_tests.barometer - INFO - PASS 2018-03-31 10:04:24,526 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with Gnocchi PASSED on OVS events. 2018-03-31 10:05:14,727 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS stats Test case execution === 2018-03-31 10:05:38,744 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-2 2018-03-31 10:05:41,300 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-0 2018-03-31 10:05:44,105 - baro_tests.barometer - INFO - Getting gnocchi metric list on overcloud-controller-1 2018-03-31 10:05:46,858 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with Gnocchi FAILED on OVS stats. 2018-03-31 10:06:35,410 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-03-31 10:06:59,497 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-03-31 10:07:18,255 - baro_tests.barometer - INFO - Data not updated after interval of 12 seconds 2018-03-31 10:07:18,256 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with AODH FAILED on Mcelog. 2018-03-31 10:08:09,082 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-03-31 10:08:33,962 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-2 2018-03-31 10:08:36,358 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-0 2018-03-31 10:08:38,686 - baro_tests.barometer - INFO - Getting AODH Alarm list on overcloud-controller-1 2018-03-31 10:08:41,049 - baro_tests.barometer - ERROR - Test case for overcloud-novacompute-1 with AODH FAILED on OVS events. 2018-03-31 10:08:41,049 - baro_tests.barometer - INFO - ======= Node overcloud-novacompute-1: Executing CSV Testcases ======== 2018-03-31 10:08:41,049 - baro_tests.barometer - INFO - Restarting collectd for CSV tests 2018-03-31 10:08:54,084 - baro_tests.barometer - INFO - Stopping collectd service... 2018-03-31 10:09:05,120 - baro_tests.barometer - INFO - Starting collectd service... 2018-03-31 10:09:15,432 - baro_tests.barometer - INFO - Sleeping for 10 seconds after collectd restart... 2018-03-31 10:10:18,245 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Hugepages Test case execution === 2018-03-31 10:10:43,571 - baro_tests.barometer - INFO - Getting CSV metrics of plugin hugepages on compute node 3d0da409-4f77-45d0-810c-1ebcc6bcdfb7... 2018-03-31 10:10:43,571 - baro_tests.barometer - DEBUG - Interval: 10 2018-03-31 10:10:43,572 - baro_tests.barometer - DEBUG - Plugin subdirs: ['hugepages-mm-2048Kb', 'hugepages-node0-2048Kb'] 2018-03-31 10:10:43,572 - baro_tests.barometer - DEBUG - Plugin meter categories: ['vmpage_number-free', 'vmpage_number-used'] 2018-03-31 10:10:57,381 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-03-31 10:10:57,381 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1522490667.89 ... 2018-03-31 10:10:57,382 - baro_tests.barometer - DEBUG - OK 2018-03-31 10:10:57,382 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1522490667.89 ... 2018-03-31 10:10:57,382 - baro_tests.barometer - DEBUG - OK 2018-03-31 10:10:57,383 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1522490667.89 ... 2018-03-31 10:10:57,383 - baro_tests.barometer - DEBUG - OK 2018-03-31 10:10:57,383 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1522490667.89 ... 2018-03-31 10:10:57,383 - baro_tests.barometer - DEBUG - OK 2018-03-31 10:10:57,383 - baro_tests.barometer - INFO - OK 2018-03-31 10:10:57,383 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-03-31 10:11:09,395 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-03-31 10:11:22,155 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-03-31 10:11:22,155 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1522490667.89, 1522490677.89), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1522490667.89, 1522490677.89), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1522490667.89, 1522490677.89), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1522490667.89, 1522490677.89)] 2018-03-31 10:11:22,156 - baro_tests.barometer - DEBUG - [('hugepages-mm-2048Kb', 'vmpage_number-free', 1522490687.89, 1522490697.89), ('hugepages-mm-2048Kb', 'vmpage_number-used', 1522490687.89, 1522490697.89), ('hugepages-node0-2048Kb', 'vmpage_number-free', 1522490687.89, 1522490697.89), ('hugepages-node0-2048Kb', 'vmpage_number-used', 1522490687.89, 1522490697.89)] 2018-03-31 10:11:22,156 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-free 1522490667.89 - hugepages-mm-2048Kb vmpage_number-free 1522490687.89 ... 2018-03-31 10:11:22,156 - baro_tests.barometer - DEBUG - OK 2018-03-31 10:11:22,157 - baro_tests.barometer - DEBUG - hugepages-mm-2048Kb vmpage_number-used 1522490667.89 - hugepages-mm-2048Kb vmpage_number-used 1522490687.89 ... 2018-03-31 10:11:22,158 - baro_tests.barometer - DEBUG - OK 2018-03-31 10:11:22,161 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-free 1522490667.89 - hugepages-node0-2048Kb vmpage_number-free 1522490687.89 ... 2018-03-31 10:11:22,163 - baro_tests.barometer - DEBUG - OK 2018-03-31 10:11:22,165 - baro_tests.barometer - DEBUG - hugepages-node0-2048Kb vmpage_number-used 1522490667.89 - hugepages-node0-2048Kb vmpage_number-used 1522490687.89 ... 2018-03-31 10:11:22,167 - baro_tests.barometer - DEBUG - OK 2018-03-31 10:11:22,168 - baro_tests.barometer - INFO - OK 2018-03-31 10:11:22,170 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Hugepages. 2018-03-31 10:12:13,376 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Intel RDT Test case execution === 2018-03-31 10:12:49,963 - baro_tests.barometer - INFO - Getting CSV metrics of plugin intel_rdt on compute node 3d0da409-4f77-45d0-810c-1ebcc6bcdfb7... 2018-03-31 10:12:49,968 - baro_tests.barometer - DEBUG - Interval: 10 2018-03-31 10:12:49,971 - baro_tests.barometer - DEBUG - Plugin subdirs: ['intel_rdt-0-2'] 2018-03-31 10:12:49,971 - baro_tests.barometer - DEBUG - Plugin meter categories: ['bytes-llc', 'ipc'] 2018-03-31 10:13:02,585 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-03-31 10:13:02,586 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1522490787.89 ... 2018-03-31 10:13:02,586 - baro_tests.barometer - DEBUG - OK 2018-03-31 10:13:02,587 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1522490787.89 ... 2018-03-31 10:13:02,588 - baro_tests.barometer - DEBUG - OK 2018-03-31 10:13:02,588 - baro_tests.barometer - INFO - OK 2018-03-31 10:13:02,589 - baro_tests.barometer - INFO - Sleeping for 12 seconds to get updated entries (interval is 10 sec)... 2018-03-31 10:13:14,600 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-03-31 10:13:27,320 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-03-31 10:13:27,321 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1522490787.89, 1522490797.89), ('intel_rdt-0-2', 'ipc', 1522490787.89, 1522490797.89)] 2018-03-31 10:13:27,321 - baro_tests.barometer - DEBUG - [('intel_rdt-0-2', 'bytes-llc', 1522490817.89, 1522490827.89), ('intel_rdt-0-2', 'ipc', 1522490817.89, 1522490827.89)] 2018-03-31 10:13:27,322 - baro_tests.barometer - DEBUG - intel_rdt-0-2 bytes-llc 1522490787.89 - intel_rdt-0-2 bytes-llc 1522490817.89 ... 2018-03-31 10:13:27,322 - baro_tests.barometer - DEBUG - OK 2018-03-31 10:13:27,323 - baro_tests.barometer - DEBUG - intel_rdt-0-2 ipc 1522490787.89 - intel_rdt-0-2 ipc 1522490817.89 ... 2018-03-31 10:13:27,323 - baro_tests.barometer - DEBUG - OK 2018-03-31 10:13:27,324 - baro_tests.barometer - INFO - OK 2018-03-31 10:13:27,324 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Intel RDT. 2018-03-31 10:14:17,111 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in Mcelog Test case execution === 2018-03-31 10:14:40,793 - baro_tests.barometer - INFO - Getting CSV metrics of plugin mcelog on compute node 3d0da409-4f77-45d0-810c-1ebcc6bcdfb7... 2018-03-31 10:14:40,794 - baro_tests.barometer - DEBUG - Interval: 1 2018-03-31 10:14:40,794 - baro_tests.barometer - DEBUG - Plugin subdirs: ['mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'mcelog-SOCKET_0_CHANNEL_any_DIMM_any'] 2018-03-31 10:14:40,795 - baro_tests.barometer - DEBUG - Plugin meter categories: ['errors-corrected_memory_errors', 'errors-uncorrected_memory_errors'] 2018-03-31 10:14:53,461 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-03-31 10:14:53,462 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1522490914.4 ... 2018-03-31 10:14:53,462 - baro_tests.barometer - DEBUG - OK 2018-03-31 10:14:53,462 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1522490914.4 ... 2018-03-31 10:14:53,462 - baro_tests.barometer - DEBUG - OK 2018-03-31 10:14:53,463 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1522490914.4 ... 2018-03-31 10:14:53,463 - baro_tests.barometer - DEBUG - OK 2018-03-31 10:14:53,463 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1522490914.4 ... 2018-03-31 10:14:53,463 - baro_tests.barometer - DEBUG - OK 2018-03-31 10:14:53,463 - baro_tests.barometer - INFO - OK 2018-03-31 10:14:53,463 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-03-31 10:14:56,466 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-03-31 10:15:08,879 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-03-31 10:15:08,879 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1522490914.398, 1522490915.398), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1522490914.398, 1522490915.398), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1522490914.398, 1522490915.398), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1522490914.398, 1522490915.398)] 2018-03-31 10:15:08,880 - baro_tests.barometer - DEBUG - [('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-corrected_memory_errors', 1522490929.398, 1522490930.398), ('mcelog-SOCKET_0_CHANNEL_0_DIMM_any', 'errors-uncorrected_memory_errors', 1522490929.398, 1522490930.398), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-corrected_memory_errors', 1522490929.398, 1522490930.398), ('mcelog-SOCKET_0_CHANNEL_any_DIMM_any', 'errors-uncorrected_memory_errors', 1522490929.398, 1522490930.398)] 2018-03-31 10:15:08,880 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1522490914.4 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-corrected_memory_errors 1522490929.4 ... 2018-03-31 10:15:08,880 - baro_tests.barometer - DEBUG - OK 2018-03-31 10:15:08,880 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1522490914.4 - mcelog-SOCKET_0_CHANNEL_0_DIMM_any errors-uncorrected_memory_errors 1522490929.4 ... 2018-03-31 10:15:08,881 - baro_tests.barometer - DEBUG - OK 2018-03-31 10:15:08,881 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1522490914.4 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-corrected_memory_errors 1522490929.4 ... 2018-03-31 10:15:08,882 - baro_tests.barometer - DEBUG - OK 2018-03-31 10:15:08,882 - baro_tests.barometer - DEBUG - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1522490914.4 - mcelog-SOCKET_0_CHANNEL_any_DIMM_any errors-uncorrected_memory_errors 1522490929.4 ... 2018-03-31 10:15:08,882 - baro_tests.barometer - DEBUG - OK 2018-03-31 10:15:08,882 - baro_tests.barometer - INFO - OK 2018-03-31 10:15:08,883 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on Mcelog. 2018-03-31 10:15:59,212 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS events Test case execution === 2018-03-31 10:16:25,322 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_events on compute node 3d0da409-4f77-45d0-810c-1ebcc6bcdfb7... 2018-03-31 10:16:25,322 - baro_tests.barometer - DEBUG - Interval: 1 2018-03-31 10:16:25,323 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_events-br0'] 2018-03-31 10:16:25,324 - baro_tests.barometer - DEBUG - Plugin meter categories: ['gauge-link_status'] 2018-03-31 10:16:38,473 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-03-31 10:16:38,474 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1522491019.4 ... 2018-03-31 10:16:38,475 - baro_tests.barometer - DEBUG - OK 2018-03-31 10:16:38,475 - baro_tests.barometer - INFO - OK 2018-03-31 10:16:38,476 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-03-31 10:16:41,481 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-03-31 10:16:53,509 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-03-31 10:16:53,512 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1522491019.398, 1522491020.398)] 2018-03-31 10:16:53,513 - baro_tests.barometer - DEBUG - [('ovs_events-br0', 'gauge-link_status', 1522491034.398, 1522491035.398)] 2018-03-31 10:16:53,514 - baro_tests.barometer - DEBUG - ovs_events-br0 gauge-link_status 1522491019.4 - ovs_events-br0 gauge-link_status 1522491034.4 ... 2018-03-31 10:16:53,515 - baro_tests.barometer - DEBUG - OK 2018-03-31 10:16:53,516 - baro_tests.barometer - INFO - OK 2018-03-31 10:16:53,517 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on OVS events. 2018-03-31 10:17:43,369 - baro_tests.barometer - INFO - === Node overcloud-novacompute-1: Plug-in OVS stats Test case execution === 2018-03-31 10:18:09,552 - baro_tests.barometer - INFO - Getting CSV metrics of plugin ovs_stats on compute node 3d0da409-4f77-45d0-810c-1ebcc6bcdfb7... 2018-03-31 10:18:09,552 - baro_tests.barometer - DEBUG - Interval: 1 2018-03-31 10:18:09,553 - baro_tests.barometer - DEBUG - Plugin subdirs: ['ovs_stats-br0.br0'] 2018-03-31 10:18:09,553 - baro_tests.barometer - DEBUG - Plugin meter categories: ['if_dropped', 'if_errors', 'if_packets'] 2018-03-31 10:18:21,909 - baro_tests.barometer - INFO - Checking that last two entries in metrics are correspondingto interval... 2018-03-31 10:18:21,911 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1522491122.4 ... 2018-03-31 10:18:21,913 - baro_tests.barometer - DEBUG - OK 2018-03-31 10:18:21,915 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1522491122.4 ... 2018-03-31 10:18:21,915 - baro_tests.barometer - DEBUG - OK 2018-03-31 10:18:21,915 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1522491122.4 ... 2018-03-31 10:18:21,915 - baro_tests.barometer - DEBUG - OK 2018-03-31 10:18:21,915 - baro_tests.barometer - INFO - OK 2018-03-31 10:18:21,916 - baro_tests.barometer - INFO - Sleeping for 3 seconds to get updated entries (interval is 1 sec)... 2018-03-31 10:18:24,919 - baro_tests.barometer - INFO - Getting new metrics of compute node overcloud-novacompute-1... 2018-03-31 10:18:37,946 - baro_tests.barometer - INFO - Comparing old and new metrics... 2018-03-31 10:18:37,947 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1522491122.398, 1522491123.398), ('ovs_stats-br0.br0', 'if_errors', 1522491122.398, 1522491123.398), ('ovs_stats-br0.br0', 'if_packets', 1522491122.398, 1522491124.398)] 2018-03-31 10:18:37,947 - baro_tests.barometer - DEBUG - [('ovs_stats-br0.br0', 'if_dropped', 1522491138.398, 1522491139.398), ('ovs_stats-br0.br0', 'if_errors', 1522491138.398, 1522491139.398), ('ovs_stats-br0.br0', 'if_packets', 1522491138.398, 1522491140.398)] 2018-03-31 10:18:37,948 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_dropped 1522491122.4 - ovs_stats-br0.br0 if_dropped 1522491138.4 ... 2018-03-31 10:18:37,949 - baro_tests.barometer - DEBUG - OK 2018-03-31 10:18:37,950 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_errors 1522491122.4 - ovs_stats-br0.br0 if_errors 1522491138.4 ... 2018-03-31 10:18:37,952 - baro_tests.barometer - DEBUG - OK 2018-03-31 10:18:37,953 - baro_tests.barometer - DEBUG - ovs_stats-br0.br0 if_packets 1522491122.4 - ovs_stats-br0.br0 if_packets 1522491138.4 ... 2018-03-31 10:18:37,955 - baro_tests.barometer - DEBUG - OK 2018-03-31 10:18:37,956 - baro_tests.barometer - INFO - OK 2018-03-31 10:18:37,960 - baro_tests.barometer - INFO - Test case for overcloud-novacompute-1 with CSV PASSED on OVS stats. 2018-03-31 10:18:51,721 - baro_tests.barometer - INFO - Mcelog is deleted from all compute nodes 2018-03-31 10:18:51,721 - baro_tests.barometer - INFO - ====================================================================== 2018-03-31 10:18:51,722 - baro_tests.barometer - INFO - +----------------------------------+ 2018-03-31 10:18:51,722 - baro_tests.barometer - INFO - | OVERALL SUMMARY | 2018-03-31 10:18:51,722 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-03-31 10:18:51,722 - baro_tests.barometer - INFO - | Test | Node-0 | Node-1 | 2018-03-31 10:18:51,723 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-03-31 10:18:51,724 - baro_tests.barometer - INFO - | OUT:Gnocchi | PASS | PASS | 2018-03-31 10:18:51,724 - baro_tests.barometer - INFO - | IN:Hugepages | PASS | FAIL | 2018-03-31 10:18:51,724 - baro_tests.barometer - INFO - | IN:Intel RDT | FAIL | FAIL | 2018-03-31 10:18:51,725 - baro_tests.barometer - INFO - | IN:Mcelog | PASS | FAIL | 2018-03-31 10:18:51,725 - baro_tests.barometer - INFO - | IN:OVS events | FAIL | PASS | 2018-03-31 10:18:51,725 - baro_tests.barometer - INFO - | IN:OVS stats | FAIL | FAIL | 2018-03-31 10:18:51,725 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-03-31 10:18:51,725 - baro_tests.barometer - INFO - | OUT:AODH | PASS | PASS | 2018-03-31 10:18:51,725 - baro_tests.barometer - INFO - | IN:Mcelog | FAIL | FAIL | 2018-03-31 10:18:51,725 - baro_tests.barometer - INFO - | IN:OVS events | SKIP | SKIP | 2018-03-31 10:18:51,725 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-03-31 10:18:51,725 - baro_tests.barometer - INFO - | OUT:CSV | PASS | PASS | 2018-03-31 10:18:51,725 - baro_tests.barometer - INFO - | IN:Hugepages | PASS | PASS | 2018-03-31 10:18:51,725 - baro_tests.barometer - INFO - | IN:Intel RDT | PASS | PASS | 2018-03-31 10:18:51,726 - baro_tests.barometer - INFO - | IN:Mcelog | PASS | PASS | 2018-03-31 10:18:51,726 - baro_tests.barometer - INFO - | IN:OVS events | PASS | PASS | 2018-03-31 10:18:51,726 - baro_tests.barometer - INFO - | IN:OVS stats | PASS | PASS | 2018-03-31 10:18:51,726 - baro_tests.barometer - INFO - +----------------+--------+--------+ 2018-03-31 10:18:51,726 - baro_tests.barometer - INFO - ====================================================================== 2018-03-31 10:18:51,726 - baro_tests.barometer - ERROR - Some tests have failed or have not been executed 2018-03-31 10:18:51,726 - baro_tests.barometer - ERROR - Overall Result is Fail