2020-09-29 12:51:52,723 - xtesting.ci.run_tests - INFO - Deployment description: +-------------------------+------------------------------------------------------------+ | ENV VAR | VALUE | +-------------------------+------------------------------------------------------------+ | CI_LOOP | daily | | DEBUG | true | | DEPLOY_SCENARIO | k8-nosdn-nofeature-noha | | INSTALLER_TYPE | unknown | | BUILD_TAG | 0I3XDBUQ27AT | | NODE_NAME | lf-virtual1-4 | | TEST_DB_URL | http://testresults.opnfv.org/test/api/v1/results | | TEST_DB_EXT_URL | http://testresults.opnfv.org/test/api/v1/results | | S3_ENDPOINT_URL | https://storage.googleapis.com | | S3_DST_URL | s3://artifacts.opnfv.org/functest- | | | kubernetes/0I3XDBUQ27AT/functest-kubernetes-opnfv- | | | functest-kubernetes-cnf-kali-helm_vims-run-2 | | HTTP_DST_URL | http://artifacts.opnfv.org/functest- | | | kubernetes/0I3XDBUQ27AT/functest-kubernetes-opnfv- | | | functest-kubernetes-cnf-kali-helm_vims-run-2 | +-------------------------+------------------------------------------------------------+ 2020-09-29 12:51:52,728 - xtesting.ci.run_tests - DEBUG - No env file /var/lib/xtesting/conf/env_file found 2020-09-29 12:51:52,728 - xtesting.ci.run_tests - DEBUG - Test args: helm_vims 2020-09-29 12:51:52,739 - xtesting.ci.run_tests - INFO - Loading test case 'helm_vims'... 2020-09-29 12:51:53,078 - xtesting.ci.run_tests - INFO - Running test case 'helm_vims'... 2020-09-29 12:51:53,105 - kubernetes.client.rest - DEBUG - response body: {"kind":"Namespace","apiVersion":"v1","metadata":{"name":"ims-cgn86","generateName":"ims-","selfLink":"/api/v1/namespaces/ims-cgn86","uid":"8d2154e8-9081-4f92-8cc1-0a71dcef344e","resourceVersion":"1641078","creationTimestamp":"2020-09-29T12:51:53Z","managedFields":[{"manager":"OpenAPI-Generator","operation":"Update","apiVersion":"v1","time":"2020-09-29T12:51:53Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:generateName":{}},"f:status":{"f:phase":{}}}}]},"spec":{"finalizers":["kubernetes"]},"status":{"phase":"Active"}} 2020-09-29 12:51:53,107 - functest_kubernetes.ims.ims - DEBUG - create_namespace: {'api_version': 'v1', 'kind': 'Namespace', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 51, 53, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'ims-', 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields': None, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2020, 9, 29, 12, 51, 53, tzinfo=tzlocal())}], 'name': 'ims-cgn86', 'namespace': None, 'owner_references': None, 'resource_version': '1641078', 'self_link': '/api/v1/namespaces/ims-cgn86', 'uid': '8d2154e8-9081-4f92-8cc1-0a71dcef344e'}, 'spec': {'finalizers': ['kubernetes']}, 'status': {'phase': 'Active'}} 2020-09-29 12:51:53,150 - kubernetes.client.rest - DEBUG - response body: {"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"env-vars","namespace":"ims-cgn86","selfLink":"/api/v1/namespaces/ims-cgn86/configmaps/env-vars","uid":"4df669de-3fb7-4700-9467-d63c11040302","resourceVersion":"1641080","creationTimestamp":"2020-09-29T12:51:53Z","managedFields":[{"manager":"OpenAPI-Generator","operation":"Update","apiVersion":"v1","time":"2020-09-29T12:51:53Z","fieldsType":"FieldsV1","fieldsV1":{"f:data":{".":{},"f:ADDITIONAL_SHARED_CONFIG":{},"f:ZONE":{}}}}]},"data":{"ADDITIONAL_SHARED_CONFIG":"","ZONE":"ims-cgn86.svc.cluster.local"}} 2020-09-29 12:51:53,151 - functest_kubernetes.ims.ims - DEBUG - create_namespaced_config_map: {'api_version': 'v1', 'binary_data': None, 'data': {'ADDITIONAL_SHARED_CONFIG': '', 'ZONE': 'ims-cgn86.svc.cluster.local'}, 'kind': 'ConfigMap', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 51, 53, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields': None, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2020, 9, 29, 12, 51, 53, tzinfo=tzlocal())}], 'name': 'env-vars', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641080', 'self_link': '/api/v1/namespaces/ims-cgn86/configmaps/env-vars', 'uid': '4df669de-3fb7-4700-9467-d63c11040302'}} 2020-09-29 12:52:06,467 - functest_kubernetes.ims.ims - DEBUG - NAME: clearwater LAST DEPLOYED: Tue Sep 29 12:52:03 2020 NAMESPACE: ims-cgn86 STATUS: deployed REVISION: 1 TEST SUITE: None 2020-09-29 12:52:06,599 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'homestead', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641341', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/homestead', 'uid': 'f1a2a027-9c0f-452c-b7c5-e46bba0d7433'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'homestead'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'homestead', 'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-homestead:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 60, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'homestead', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 8888, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/homestead', 'mount_propagation': None, 'name': 'homesteadlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/homestead/homestead_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/homestead', 'mount_propagation': None, 'name': 'homesteadlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'cassandra; ' 'do echo ' 'waiting for ' 'cassandra to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'astaire; do ' 'echo waiting ' 'for astaire ' 'to start ' '...; sleep ' '2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-astaire', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'homesteadlogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': None, 'observed_generation': None, 'ready_replicas': None, 'replicas': None, 'unavailable_replicas': None, 'updated_replicas': None}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'homestead', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/homestead', 'uid': 'f1a2a027-9c0f-452c-b7c5-e46bba0d7433', 'resourceVersion': '1641341', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'homestead'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'homestead', 'snmp': 'enabled'}}, 'spec': {'volumes': [{'name': 'homesteadlogs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'wait-on-astaire', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup astaire; do echo waiting for astaire to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'homestead', 'image': 'ollivier/clearwater-homestead:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 8888, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'homesteadlogs', 'mountPath': '/var/log/homestead'}], 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'initialDelaySeconds': 60, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'tailer', 'image': 'busybox:1.28', 'command': ['tail', '-F', '/var/log/homestead/homestead_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'homesteadlogs', 'mountPath': '/var/log/homestead'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {}}} 2020-09-29 12:52:06,619 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'ellis', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641333', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/ellis', 'uid': 'ab455ae9-328e-4b0e-a10d-b2ed1d3658e9'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'ellis'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'ellis'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-ellis:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 80}, 'timeout_seconds': 1}, 'name': 'ellis', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 80, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 80}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'homestead-prov; ' 'do echo ' 'waiting for ' 'homestead-prov ' 'to start ' '...; sleep ' '2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-homestead-prov', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'sprout; do ' 'echo waiting ' 'for sprout ' 'to start ' '...; sleep ' '2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-sprout', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': None, 'observed_generation': None, 'ready_replicas': None, 'replicas': None, 'unavailable_replicas': None, 'updated_replicas': None}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'ellis', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/ellis', 'uid': 'ab455ae9-328e-4b0e-a10d-b2ed1d3658e9', 'resourceVersion': '1641333', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'ellis'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'ellis'}}, 'spec': {'initContainers': [{'name': 'wait-on-homestead-prov', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup homestead-prov; do echo waiting for homestead-prov to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'wait-on-sprout', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup sprout; do echo waiting for sprout to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'ellis', 'image': 'ollivier/clearwater-ellis:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 80, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'livenessProbe': {'tcpSocket': {'port': 80}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'tcpSocket': {'port': 80}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {}}} 2020-09-29 12:52:06,629 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'homestead-prov', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641335', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/homestead-prov', 'uid': '9e6bead5-f8b8-446c-a528-fbe3bf8ba7cd'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'homestead-prov'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'homestead-prov', 'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-homestead-prov:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 60, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'homestead-prov', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 8889, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'homestead; ' 'do echo ' 'waiting for ' 'homestead to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-homestead', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': None, 'observed_generation': None, 'ready_replicas': None, 'replicas': None, 'unavailable_replicas': None, 'updated_replicas': None}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'homestead-prov', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/homestead-prov', 'uid': '9e6bead5-f8b8-446c-a528-fbe3bf8ba7cd', 'resourceVersion': '1641335', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'homestead-prov'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'homestead-prov', 'snmp': 'enabled'}}, 'spec': {'initContainers': [{'name': 'wait-on-homestead', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup homestead; do echo waiting for homestead to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'homestead-prov', 'image': 'ollivier/clearwater-homestead-prov:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 8889, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'initialDelaySeconds': 60, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {}}} 2020-09-29 12:52:06,660 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'bono', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641336', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/bono', 'uid': '39b3eef8-15d5-4e88-ac4d-ce0d8a783d4d'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'bono'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'bono', 'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-bono:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 ' '5060 ' '5062']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'bono', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 3478, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 5060, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 5062, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 5060, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'UDP'}, {'container_port': 5062, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'UDP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 ' '5060 ' '5062']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/bono', 'mount_propagation': None, 'name': 'bonologs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/bono/bono_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/bono', 'mount_propagation': None, 'name': 'bonologs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'sprout; do ' 'echo waiting ' 'for sprout ' 'to start ' '...; sleep ' '2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-sprout', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'bonologs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': None, 'observed_generation': None, 'ready_replicas': None, 'replicas': None, 'unavailable_replicas': None, 'updated_replicas': None}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'bono', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/bono', 'uid': '39b3eef8-15d5-4e88-ac4d-ce0d8a783d4d', 'resourceVersion': '1641336', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'bono'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'bono', 'snmp': 'enabled'}}, 'spec': {'volumes': [{'name': 'bonologs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-sprout', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup sprout; do echo waiting for sprout to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'bono', 'image': 'ollivier/clearwater-bono:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 3478, 'protocol': 'TCP'}, {'containerPort': 5060, 'protocol': 'TCP'}, {'containerPort': 5062, 'protocol': 'TCP'}, {'containerPort': 5060, 'protocol': 'UDP'}, {'containerPort': 5062, 'protocol': 'UDP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'bonologs', 'mountPath': '/var/log/bono'}], 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 5060 5062']}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 5060 5062']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'tailer', 'image': 'busybox:1.28', 'command': ['tail', '-F', '/var/log/bono/bono_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'bonologs', 'mountPath': '/var/log/bono'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {}}} 2020-09-29 12:52:06,680 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'etcd', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641337', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/etcd', 'uid': 'da6efd4b-a7fc-4f36-9abf-8ca3a644590f'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'instance-type': 'etcd-pod'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'instance-type': 'etcd-pod'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': ['--name', '$(MY_POD_NAME)', '--advertise-client-urls', 'http://$(MY_POD_IP):2379,http://$(MY_POD_IP):4001', '--listen-client-urls', 'http://0.0.0.0:2379,http://0.0.0.0:4001', '--initial-advertise-peer-urls', 'http://$(MY_POD_IP):2380', '--listen-peer-urls', 'http://0.0.0.0:2380', '--initial-cluster', '$(MY_POD_NAME)=http://$(MY_POD_IP):2380', '--initial-cluster-state', 'new'], 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}, {'name': 'MY_POD_NAME', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'metadata.name'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': None, 'image': 'quay.io/coreos/etcd:v2.2.5', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 300, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 4001}, 'timeout_seconds': 1}, 'name': 'etcd', 'ports': [{'container_port': 2379, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 2380, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 4001, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 4001}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': None, 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': None, 'observed_generation': None, 'ready_replicas': None, 'replicas': None, 'unavailable_replicas': None, 'updated_replicas': None}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'etcd', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/etcd', 'uid': 'da6efd4b-a7fc-4f36-9abf-8ca3a644590f', 'resourceVersion': '1641337', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'instance-type': 'etcd-pod'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'instance-type': 'etcd-pod'}}, 'spec': {'containers': [{'name': 'etcd', 'image': 'quay.io/coreos/etcd:v2.2.5', 'args': ['--name', '$(MY_POD_NAME)', '--advertise-client-urls', 'http://$(MY_POD_IP):2379,http://$(MY_POD_IP):4001', '--listen-client-urls', 'http://0.0.0.0:2379,http://0.0.0.0:4001', '--initial-advertise-peer-urls', 'http://$(MY_POD_IP):2380', '--listen-peer-urls', 'http://0.0.0.0:2380', '--initial-cluster', '$(MY_POD_NAME)=http://$(MY_POD_IP):2380', '--initial-cluster-state', 'new'], 'ports': [{'containerPort': 2379, 'protocol': 'TCP'}, {'containerPort': 2380, 'protocol': 'TCP'}, {'containerPort': 4001, 'protocol': 'TCP'}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}, {'name': 'MY_POD_NAME', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'metadata.name'}}}], 'resources': {}, 'livenessProbe': {'tcpSocket': {'port': 4001}, 'initialDelaySeconds': 300, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'tcpSocket': {'port': 4001}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {}}} 2020-09-29 12:52:06,688 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'cassandra', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641340', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/cassandra', 'uid': '578317be-b028-4475-a59c-b55a7432b679'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'cassandra'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'cassandra'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-cassandra:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 ' '9042 ' '9160']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 600, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'cassandra', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 7001, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 9042, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 9160, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 ' '9042 ' '9160']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'etcd; do ' 'echo waiting ' 'for etcd to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-etcd', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': None, 'observed_generation': None, 'ready_replicas': None, 'replicas': None, 'unavailable_replicas': None, 'updated_replicas': None}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'cassandra', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/cassandra', 'uid': '578317be-b028-4475-a59c-b55a7432b679', 'resourceVersion': '1641340', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'cassandra'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'cassandra'}}, 'spec': {'initContainers': [{'name': 'wait-on-etcd', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup etcd; do echo waiting for etcd to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'cassandra', 'image': 'ollivier/clearwater-cassandra:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 7001, 'protocol': 'TCP'}, {'containerPort': 9042, 'protocol': 'TCP'}, {'containerPort': 9160, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 9042 9160']}, 'initialDelaySeconds': 600, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 9042 9160']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {}}} 2020-09-29 12:52:06,697 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'astaire', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641331', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/astaire', 'uid': '5854d3fa-058f-4e83-8159-1cc01de9d297'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'astaire'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'astaire'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-astaire:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': {'post_start': None, 'pre_stop': {'_exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}, 'http_get': None, 'tcp_socket': None}}, 'liveness_probe': {'_exec': None, 'failure_threshold': 9, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 11311}, 'timeout_seconds': 1}, 'name': 'astaire', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 11211, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 11311, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 11311}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/astaire', 'mount_propagation': None, 'name': 'astairelogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/astaire/astaire_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/astaire', 'mount_propagation': None, 'name': 'astairelogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'cassandra; ' 'do echo ' 'waiting for ' 'cassandra to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 120, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'astairelogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': None, 'observed_generation': None, 'ready_replicas': None, 'replicas': None, 'unavailable_replicas': None, 'updated_replicas': None}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'astaire', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/astaire', 'uid': '5854d3fa-058f-4e83-8159-1cc01de9d297', 'resourceVersion': '1641331', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'astaire'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'astaire'}}, 'spec': {'volumes': [{'name': 'astairelogs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'astaire', 'image': 'ollivier/clearwater-astaire:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 11211, 'protocol': 'TCP'}, {'containerPort': 11311, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'astairelogs', 'mountPath': '/var/log/astaire'}], 'livenessProbe': {'tcpSocket': {'port': 11311}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 9}, 'readinessProbe': {'tcpSocket': {'port': 11311}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'lifecycle': {'preStop': {'exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}}}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'tailer', 'image': 'busybox:1.28', 'command': ['tail', '-F', '/var/log/astaire/astaire_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'astairelogs', 'mountPath': '/var/log/astaire'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 120, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {}}} 2020-09-29 12:52:06,709 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'sprout', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641334', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/sprout', 'uid': 'd933e2f6-a77a-4051-ae84-52c606034448'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'sprout'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'sprout', 'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-sprout:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 ' '5054']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 3, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'sprout', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 ' '5054']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/sprout', 'mount_propagation': None, 'name': 'sproutlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/sprout/sprout_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/sprout', 'mount_propagation': None, 'name': 'sproutlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'ralf; do ' 'echo waiting ' 'for ralf to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-ralf', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'chronos; do ' 'echo waiting ' 'for chronos ' 'to start ' '...; sleep ' '2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-chronos', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'homestead; ' 'do echo ' 'waiting for ' 'homestead to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-homestead', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'sproutlogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': None, 'observed_generation': None, 'ready_replicas': None, 'replicas': None, 'unavailable_replicas': None, 'updated_replicas': None}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'sprout', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/sprout', 'uid': 'd933e2f6-a77a-4051-ae84-52c606034448', 'resourceVersion': '1641334', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'sprout'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'sprout', 'snmp': 'enabled'}}, 'spec': {'volumes': [{'name': 'sproutlogs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-ralf', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup ralf; do echo waiting for ralf to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'wait-on-chronos', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup chronos; do echo waiting for chronos to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'wait-on-homestead', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup homestead; do echo waiting for homestead to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'sprout', 'image': 'ollivier/clearwater-sprout:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'sproutlogs', 'mountPath': '/var/log/sprout'}], 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 5054']}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 3, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 5054']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'tailer', 'image': 'busybox:1.28', 'command': ['tail', '-F', '/var/log/sprout/sprout_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'sproutlogs', 'mountPath': '/var/log/sprout'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {}}} 2020-09-29 12:52:06,721 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'homer', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641332', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/homer', 'uid': '7f83b4e3-5af2-4d79-b532-6c16cad4773d'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'homer'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'homer'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-homer:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7888}, 'timeout_seconds': 1}, 'name': 'homer', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 7888, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7888}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'cassandra; ' 'do echo ' 'waiting for ' 'cassandra to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': None, 'observed_generation': None, 'ready_replicas': None, 'replicas': None, 'unavailable_replicas': None, 'updated_replicas': None}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'homer', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/homer', 'uid': '7f83b4e3-5af2-4d79-b532-6c16cad4773d', 'resourceVersion': '1641332', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'homer'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'homer'}}, 'spec': {'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'homer', 'image': 'ollivier/clearwater-homer:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 7888, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'livenessProbe': {'tcpSocket': {'port': 7888}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'tcpSocket': {'port': 7888}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {}}} 2020-09-29 12:52:06,729 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm', 'service': 'chronos'}, 'managed_fields': None, 'name': 'chronos', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641338', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/chronos', 'uid': '52d3b182-86dc-451c-9cd5-b61a4e65f189'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'chronos'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'chronos'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-chronos:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': {'post_start': None, 'pre_stop': {'_exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}, 'http_get': None, 'tcp_socket': None}}, 'liveness_probe': {'_exec': None, 'failure_threshold': 9, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7253}, 'timeout_seconds': 1}, 'name': 'chronos', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 7253, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7253}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/chronos', 'mount_propagation': None, 'name': 'chronoslogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/chronos/chronos_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/chronos', 'mount_propagation': None, 'name': 'chronoslogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'cassandra; ' 'do echo ' 'waiting for ' 'cassandra to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 120, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'chronoslogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': None, 'observed_generation': None, 'ready_replicas': None, 'replicas': None, 'unavailable_replicas': None, 'updated_replicas': None}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'chronos', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/chronos', 'uid': '52d3b182-86dc-451c-9cd5-b61a4e65f189', 'resourceVersion': '1641338', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm', 'service': 'chronos'}, 'annotations': {'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'chronos'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'chronos'}}, 'spec': {'volumes': [{'name': 'chronoslogs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'chronos', 'image': 'ollivier/clearwater-chronos:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 7253, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'chronoslogs', 'mountPath': '/var/log/chronos'}], 'livenessProbe': {'tcpSocket': {'port': 7253}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 9}, 'readinessProbe': {'tcpSocket': {'port': 7253}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'lifecycle': {'preStop': {'exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}}}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'tailer', 'image': 'busybox:1.28', 'command': ['tail', '-F', '/var/log/chronos/chronos_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'chronoslogs', 'mountPath': '/var/log/chronos'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 120, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {}}} 2020-09-29 12:52:06,740 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'ralf', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641339', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/ralf', 'uid': 'b88a9a7f-0df4-4fd5-b389-2bf6e218f350'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'ralf'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'ralf', 'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-ralf:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 10888}, 'timeout_seconds': 1}, 'name': 'ralf', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 10888, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 10888}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/ralf', 'mount_propagation': None, 'name': 'ralflogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/ralf/ralf_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/ralf', 'mount_propagation': None, 'name': 'ralflogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'cassandra; ' 'do echo ' 'waiting for ' 'cassandra to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'ralflogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': None, 'observed_generation': None, 'ready_replicas': None, 'replicas': None, 'unavailable_replicas': None, 'updated_replicas': None}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'ralf', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/ralf', 'uid': 'b88a9a7f-0df4-4fd5-b389-2bf6e218f350', 'resourceVersion': '1641339', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'ralf'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'ralf', 'snmp': 'enabled'}}, 'spec': {'volumes': [{'name': 'ralflogs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'ralf', 'image': 'ollivier/clearwater-ralf:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 10888, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'ralflogs', 'mountPath': '/var/log/ralf'}], 'livenessProbe': {'tcpSocket': {'port': 10888}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'tcpSocket': {'port': 10888}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'tailer', 'image': 'busybox:1.28', 'command': ['tail', '-F', '/var/log/ralf/ralf_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'ralflogs', 'mountPath': '/var/log/ralf'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {}}} 2020-09-29 12:52:06,790 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'sprout', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641361', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/sprout', 'uid': 'd933e2f6-a77a-4051-ae84-52c606034448'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'sprout'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'sprout', 'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-sprout:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 ' '5054']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 3, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'sprout', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 ' '5054']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/sprout', 'mount_propagation': None, 'name': 'sproutlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/sprout/sprout_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/sprout', 'mount_propagation': None, 'name': 'sproutlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'ralf; do ' 'echo waiting ' 'for ralf to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-ralf', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'chronos; do ' 'echo waiting ' 'for chronos ' 'to start ' '...; sleep ' '2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-chronos', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'homestead; ' 'do echo ' 'waiting for ' 'homestead to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-homestead', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'sproutlogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 6, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 6, tzinfo=tzlocal()), 'message': 'Created new replica set ' '"sprout-768bcbd967"', 'reason': 'NewReplicaSetCreated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': None, 'ready_replicas': None, 'replicas': None, 'unavailable_replicas': None, 'updated_replicas': None}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'sprout', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/sprout', 'uid': 'd933e2f6-a77a-4051-ae84-52c606034448', 'resourceVersion': '1641361', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'sprout'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'sprout', 'snmp': 'enabled'}}, 'spec': {'volumes': [{'name': 'sproutlogs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-ralf', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup ralf; do echo waiting for ralf to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'wait-on-chronos', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup chronos; do echo waiting for chronos to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'wait-on-homestead', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup homestead; do echo waiting for homestead to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'sprout', 'image': 'ollivier/clearwater-sprout:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'sproutlogs', 'mountPath': '/var/log/sprout'}], 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 5054']}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 3, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 5054']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'tailer', 'image': 'busybox:1.28', 'command': ['tail', '-F', '/var/log/sprout/sprout_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'sproutlogs', 'mountPath': '/var/log/sprout'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'conditions': [{'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:52:06Z', 'lastTransitionTime': '2020-09-29T12:52:06Z', 'reason': 'NewReplicaSetCreated', 'message': 'Created new replica set "sprout-768bcbd967"'}]}}} 2020-09-29 12:52:06,861 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'homestead-prov', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641362', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/homestead-prov', 'uid': '9e6bead5-f8b8-446c-a528-fbe3bf8ba7cd'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'homestead-prov'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'homestead-prov', 'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-homestead-prov:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 60, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'homestead-prov', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 8889, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'homestead; ' 'do echo ' 'waiting for ' 'homestead to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-homestead', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 6, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 6, tzinfo=tzlocal()), 'message': 'Created new replica set ' '"homestead-prov-d67ff8f9b"', 'reason': 'NewReplicaSetCreated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': None, 'ready_replicas': None, 'replicas': None, 'unavailable_replicas': None, 'updated_replicas': None}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'homestead-prov', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/homestead-prov', 'uid': '9e6bead5-f8b8-446c-a528-fbe3bf8ba7cd', 'resourceVersion': '1641362', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'homestead-prov'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'homestead-prov', 'snmp': 'enabled'}}, 'spec': {'initContainers': [{'name': 'wait-on-homestead', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup homestead; do echo waiting for homestead to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'homestead-prov', 'image': 'ollivier/clearwater-homestead-prov:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 8889, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'initialDelaySeconds': 60, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'conditions': [{'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:52:06Z', 'lastTransitionTime': '2020-09-29T12:52:06Z', 'reason': 'NewReplicaSetCreated', 'message': 'Created new replica set "homestead-prov-d67ff8f9b"'}]}}} 2020-09-29 12:52:06,870 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'ellis', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641364', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/ellis', 'uid': 'ab455ae9-328e-4b0e-a10d-b2ed1d3658e9'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'ellis'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'ellis'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-ellis:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 80}, 'timeout_seconds': 1}, 'name': 'ellis', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 80, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 80}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'homestead-prov; ' 'do echo ' 'waiting for ' 'homestead-prov ' 'to start ' '...; sleep ' '2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-homestead-prov', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'sprout; do ' 'echo waiting ' 'for sprout ' 'to start ' '...; sleep ' '2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-sprout', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 6, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 6, tzinfo=tzlocal()), 'message': 'Created new replica set ' '"ellis-54c6475b79"', 'reason': 'NewReplicaSetCreated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': None, 'ready_replicas': None, 'replicas': None, 'unavailable_replicas': None, 'updated_replicas': None}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'ellis', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/ellis', 'uid': 'ab455ae9-328e-4b0e-a10d-b2ed1d3658e9', 'resourceVersion': '1641364', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'ellis'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'ellis'}}, 'spec': {'initContainers': [{'name': 'wait-on-homestead-prov', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup homestead-prov; do echo waiting for homestead-prov to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'wait-on-sprout', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup sprout; do echo waiting for sprout to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'ellis', 'image': 'ollivier/clearwater-ellis:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 80, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'livenessProbe': {'tcpSocket': {'port': 80}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'tcpSocket': {'port': 80}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'conditions': [{'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:52:06Z', 'lastTransitionTime': '2020-09-29T12:52:06Z', 'reason': 'NewReplicaSetCreated', 'message': 'Created new replica set "ellis-54c6475b79"'}]}}} 2020-09-29 12:52:06,881 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'homer', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641365', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/homer', 'uid': '7f83b4e3-5af2-4d79-b532-6c16cad4773d'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'homer'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'homer'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-homer:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7888}, 'timeout_seconds': 1}, 'name': 'homer', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 7888, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7888}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'cassandra; ' 'do echo ' 'waiting for ' 'cassandra to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 6, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 6, tzinfo=tzlocal()), 'message': 'Created new replica set ' '"homer-767485c9dc"', 'reason': 'NewReplicaSetCreated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': None, 'ready_replicas': None, 'replicas': None, 'unavailable_replicas': None, 'updated_replicas': None}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'homer', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/homer', 'uid': '7f83b4e3-5af2-4d79-b532-6c16cad4773d', 'resourceVersion': '1641365', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'homer'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'homer'}}, 'spec': {'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'homer', 'image': 'ollivier/clearwater-homer:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 7888, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'livenessProbe': {'tcpSocket': {'port': 7888}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'tcpSocket': {'port': 7888}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'conditions': [{'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:52:06Z', 'lastTransitionTime': '2020-09-29T12:52:06Z', 'reason': 'NewReplicaSetCreated', 'message': 'Created new replica set "homer-767485c9dc"'}]}}} 2020-09-29 12:52:06,891 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'astaire', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641367', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/astaire', 'uid': '5854d3fa-058f-4e83-8159-1cc01de9d297'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'astaire'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'astaire'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-astaire:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': {'post_start': None, 'pre_stop': {'_exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}, 'http_get': None, 'tcp_socket': None}}, 'liveness_probe': {'_exec': None, 'failure_threshold': 9, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 11311}, 'timeout_seconds': 1}, 'name': 'astaire', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 11211, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 11311, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 11311}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/astaire', 'mount_propagation': None, 'name': 'astairelogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/astaire/astaire_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/astaire', 'mount_propagation': None, 'name': 'astairelogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'cassandra; ' 'do echo ' 'waiting for ' 'cassandra to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 120, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'astairelogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 6, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 6, tzinfo=tzlocal()), 'message': 'Created new replica set ' '"astaire-65dfc8d888"', 'reason': 'NewReplicaSetCreated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': None, 'ready_replicas': None, 'replicas': None, 'unavailable_replicas': None, 'updated_replicas': None}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'astaire', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/astaire', 'uid': '5854d3fa-058f-4e83-8159-1cc01de9d297', 'resourceVersion': '1641367', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'astaire'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'astaire'}}, 'spec': {'volumes': [{'name': 'astairelogs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'astaire', 'image': 'ollivier/clearwater-astaire:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 11211, 'protocol': 'TCP'}, {'containerPort': 11311, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'astairelogs', 'mountPath': '/var/log/astaire'}], 'livenessProbe': {'tcpSocket': {'port': 11311}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 9}, 'readinessProbe': {'tcpSocket': {'port': 11311}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'lifecycle': {'preStop': {'exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}}}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'tailer', 'image': 'busybox:1.28', 'command': ['tail', '-F', '/var/log/astaire/astaire_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'astairelogs', 'mountPath': '/var/log/astaire'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 120, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'conditions': [{'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:52:06Z', 'lastTransitionTime': '2020-09-29T12:52:06Z', 'reason': 'NewReplicaSetCreated', 'message': 'Created new replica set "astaire-65dfc8d888"'}]}}} 2020-09-29 12:52:07,706 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'bono', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641408', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/bono', 'uid': '39b3eef8-15d5-4e88-ac4d-ce0d8a783d4d'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'bono'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'bono', 'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-bono:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 ' '5060 ' '5062']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'bono', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 3478, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 5060, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 5062, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 5060, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'UDP'}, {'container_port': 5062, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'UDP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 ' '5060 ' '5062']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/bono', 'mount_propagation': None, 'name': 'bonologs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/bono/bono_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/bono', 'mount_propagation': None, 'name': 'bonologs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'sprout; do ' 'echo waiting ' 'for sprout ' 'to start ' '...; sleep ' '2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-sprout', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'bonologs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'message': 'Created new replica set ' '"bono-649d6c88df"', 'reason': 'NewReplicaSetCreated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': None, 'ready_replicas': None, 'replicas': None, 'unavailable_replicas': None, 'updated_replicas': None}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'bono', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/bono', 'uid': '39b3eef8-15d5-4e88-ac4d-ce0d8a783d4d', 'resourceVersion': '1641408', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'bono'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'bono', 'snmp': 'enabled'}}, 'spec': {'volumes': [{'name': 'bonologs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-sprout', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup sprout; do echo waiting for sprout to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'bono', 'image': 'ollivier/clearwater-bono:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 3478, 'protocol': 'TCP'}, {'containerPort': 5060, 'protocol': 'TCP'}, {'containerPort': 5062, 'protocol': 'TCP'}, {'containerPort': 5060, 'protocol': 'UDP'}, {'containerPort': 5062, 'protocol': 'UDP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'bonologs', 'mountPath': '/var/log/bono'}], 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 5060 5062']}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 5060 5062']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'tailer', 'image': 'busybox:1.28', 'command': ['tail', '-F', '/var/log/bono/bono_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'bonologs', 'mountPath': '/var/log/bono'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'conditions': [{'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:52:07Z', 'lastTransitionTime': '2020-09-29T12:52:07Z', 'reason': 'NewReplicaSetCreated', 'message': 'Created new replica set "bono-649d6c88df"'}]}}} 2020-09-29 12:52:07,769 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'etcd', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641409', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/etcd', 'uid': 'da6efd4b-a7fc-4f36-9abf-8ca3a644590f'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'instance-type': 'etcd-pod'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'instance-type': 'etcd-pod'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': ['--name', '$(MY_POD_NAME)', '--advertise-client-urls', 'http://$(MY_POD_IP):2379,http://$(MY_POD_IP):4001', '--listen-client-urls', 'http://0.0.0.0:2379,http://0.0.0.0:4001', '--initial-advertise-peer-urls', 'http://$(MY_POD_IP):2380', '--listen-peer-urls', 'http://0.0.0.0:2380', '--initial-cluster', '$(MY_POD_NAME)=http://$(MY_POD_IP):2380', '--initial-cluster-state', 'new'], 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}, {'name': 'MY_POD_NAME', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'metadata.name'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': None, 'image': 'quay.io/coreos/etcd:v2.2.5', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 300, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 4001}, 'timeout_seconds': 1}, 'name': 'etcd', 'ports': [{'container_port': 2379, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 2380, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 4001, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 4001}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': None, 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'message': 'Created new replica set ' '"etcd-744b4d9f98"', 'reason': 'NewReplicaSetCreated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': None, 'ready_replicas': None, 'replicas': None, 'unavailable_replicas': None, 'updated_replicas': None}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'etcd', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/etcd', 'uid': 'da6efd4b-a7fc-4f36-9abf-8ca3a644590f', 'resourceVersion': '1641409', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'instance-type': 'etcd-pod'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'instance-type': 'etcd-pod'}}, 'spec': {'containers': [{'name': 'etcd', 'image': 'quay.io/coreos/etcd:v2.2.5', 'args': ['--name', '$(MY_POD_NAME)', '--advertise-client-urls', 'http://$(MY_POD_IP):2379,http://$(MY_POD_IP):4001', '--listen-client-urls', 'http://0.0.0.0:2379,http://0.0.0.0:4001', '--initial-advertise-peer-urls', 'http://$(MY_POD_IP):2380', '--listen-peer-urls', 'http://0.0.0.0:2380', '--initial-cluster', '$(MY_POD_NAME)=http://$(MY_POD_IP):2380', '--initial-cluster-state', 'new'], 'ports': [{'containerPort': 2379, 'protocol': 'TCP'}, {'containerPort': 2380, 'protocol': 'TCP'}, {'containerPort': 4001, 'protocol': 'TCP'}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}, {'name': 'MY_POD_NAME', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'metadata.name'}}}], 'resources': {}, 'livenessProbe': {'tcpSocket': {'port': 4001}, 'initialDelaySeconds': 300, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'tcpSocket': {'port': 4001}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'conditions': [{'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:52:07Z', 'lastTransitionTime': '2020-09-29T12:52:07Z', 'reason': 'NewReplicaSetCreated', 'message': 'Created new replica set "etcd-744b4d9f98"'}]}}} 2020-09-29 12:52:07,784 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm', 'service': 'chronos'}, 'managed_fields': None, 'name': 'chronos', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641410', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/chronos', 'uid': '52d3b182-86dc-451c-9cd5-b61a4e65f189'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'chronos'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'chronos'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-chronos:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': {'post_start': None, 'pre_stop': {'_exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}, 'http_get': None, 'tcp_socket': None}}, 'liveness_probe': {'_exec': None, 'failure_threshold': 9, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7253}, 'timeout_seconds': 1}, 'name': 'chronos', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 7253, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7253}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/chronos', 'mount_propagation': None, 'name': 'chronoslogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/chronos/chronos_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/chronos', 'mount_propagation': None, 'name': 'chronoslogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'cassandra; ' 'do echo ' 'waiting for ' 'cassandra to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 120, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'chronoslogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'message': 'Created new replica set ' '"chronos-6dff6b5ff8"', 'reason': 'NewReplicaSetCreated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': None, 'ready_replicas': None, 'replicas': None, 'unavailable_replicas': None, 'updated_replicas': None}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'chronos', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/chronos', 'uid': '52d3b182-86dc-451c-9cd5-b61a4e65f189', 'resourceVersion': '1641410', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm', 'service': 'chronos'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'chronos'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'chronos'}}, 'spec': {'volumes': [{'name': 'chronoslogs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'chronos', 'image': 'ollivier/clearwater-chronos:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 7253, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'chronoslogs', 'mountPath': '/var/log/chronos'}], 'livenessProbe': {'tcpSocket': {'port': 7253}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 9}, 'readinessProbe': {'tcpSocket': {'port': 7253}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'lifecycle': {'preStop': {'exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}}}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'tailer', 'image': 'busybox:1.28', 'command': ['tail', '-F', '/var/log/chronos/chronos_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'chronoslogs', 'mountPath': '/var/log/chronos'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 120, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'conditions': [{'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:52:07Z', 'lastTransitionTime': '2020-09-29T12:52:07Z', 'reason': 'NewReplicaSetCreated', 'message': 'Created new replica set "chronos-6dff6b5ff8"'}]}}} 2020-09-29 12:52:07,796 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'ralf', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641412', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/ralf', 'uid': 'b88a9a7f-0df4-4fd5-b389-2bf6e218f350'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'ralf'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'ralf', 'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-ralf:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 10888}, 'timeout_seconds': 1}, 'name': 'ralf', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 10888, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 10888}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/ralf', 'mount_propagation': None, 'name': 'ralflogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/ralf/ralf_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/ralf', 'mount_propagation': None, 'name': 'ralflogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'cassandra; ' 'do echo ' 'waiting for ' 'cassandra to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'ralflogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'message': 'Created new replica set ' '"ralf-7449977d5c"', 'reason': 'NewReplicaSetCreated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': None, 'ready_replicas': None, 'replicas': None, 'unavailable_replicas': None, 'updated_replicas': None}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'ralf', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/ralf', 'uid': 'b88a9a7f-0df4-4fd5-b389-2bf6e218f350', 'resourceVersion': '1641412', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'ralf'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'ralf', 'snmp': 'enabled'}}, 'spec': {'volumes': [{'name': 'ralflogs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'ralf', 'image': 'ollivier/clearwater-ralf:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 10888, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'ralflogs', 'mountPath': '/var/log/ralf'}], 'livenessProbe': {'tcpSocket': {'port': 10888}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'tcpSocket': {'port': 10888}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'tailer', 'image': 'busybox:1.28', 'command': ['tail', '-F', '/var/log/ralf/ralf_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'ralflogs', 'mountPath': '/var/log/ralf'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'conditions': [{'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:52:07Z', 'lastTransitionTime': '2020-09-29T12:52:07Z', 'reason': 'NewReplicaSetCreated', 'message': 'Created new replica set "ralf-7449977d5c"'}]}}} 2020-09-29 12:52:07,808 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'cassandra', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641420', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/cassandra', 'uid': '578317be-b028-4475-a59c-b55a7432b679'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'cassandra'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'cassandra'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-cassandra:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 ' '9042 ' '9160']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 600, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'cassandra', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 7001, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 9042, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 9160, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 ' '9042 ' '9160']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'etcd; do ' 'echo waiting ' 'for etcd to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-etcd', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'message': 'Created new replica set ' '"cassandra-864f6d8f65"', 'reason': 'NewReplicaSetCreated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': None, 'ready_replicas': None, 'replicas': None, 'unavailable_replicas': None, 'updated_replicas': None}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'cassandra', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/cassandra', 'uid': '578317be-b028-4475-a59c-b55a7432b679', 'resourceVersion': '1641420', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'cassandra'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'cassandra'}}, 'spec': {'initContainers': [{'name': 'wait-on-etcd', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup etcd; do echo waiting for etcd to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'cassandra', 'image': 'ollivier/clearwater-cassandra:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 7001, 'protocol': 'TCP'}, {'containerPort': 9042, 'protocol': 'TCP'}, {'containerPort': 9160, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 9042 9160']}, 'initialDelaySeconds': 600, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 9042 9160']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'conditions': [{'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:52:07Z', 'lastTransitionTime': '2020-09-29T12:52:07Z', 'reason': 'NewReplicaSetCreated', 'message': 'Created new replica set "cassandra-864f6d8f65"'}]}}} 2020-09-29 12:52:08,196 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'ellis', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641469', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/ellis', 'uid': 'ab455ae9-328e-4b0e-a10d-b2ed1d3658e9'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'ellis'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'ellis'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-ellis:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 80}, 'timeout_seconds': 1}, 'name': 'ellis', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 80, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 80}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'homestead-prov; ' 'do echo ' 'waiting for ' 'homestead-prov ' 'to start ' '...; sleep ' '2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-homestead-prov', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'sprout; do ' 'echo waiting ' 'for sprout ' 'to start ' '...; sleep ' '2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-sprout', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum ' 'availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 6, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'message': 'ReplicaSet "ellis-54c6475b79" is ' 'progressing.', 'reason': 'ReplicaSetUpdated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': 1, 'unavailable_replicas': 1, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'ellis', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/ellis', 'uid': 'ab455ae9-328e-4b0e-a10d-b2ed1d3658e9', 'resourceVersion': '1641469', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'ellis'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'ellis'}}, 'spec': {'initContainers': [{'name': 'wait-on-homestead-prov', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup homestead-prov; do echo waiting for homestead-prov to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'wait-on-sprout', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup sprout; do echo waiting for sprout to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'ellis', 'image': 'ollivier/clearwater-ellis:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 80, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'livenessProbe': {'tcpSocket': {'port': 80}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'tcpSocket': {'port': 80}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'unavailableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'False', 'lastUpdateTime': '2020-09-29T12:52:08Z', 'lastTransitionTime': '2020-09-29T12:52:08Z', 'reason': 'MinimumReplicasUnavailable', 'message': 'Deployment does not have minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:52:08Z', 'lastTransitionTime': '2020-09-29T12:52:06Z', 'reason': 'ReplicaSetUpdated', 'message': 'ReplicaSet "ellis-54c6475b79" is progressing.'}]}}} 2020-09-29 12:52:08,210 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'homer', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641470', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/homer', 'uid': '7f83b4e3-5af2-4d79-b532-6c16cad4773d'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'homer'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'homer'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-homer:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7888}, 'timeout_seconds': 1}, 'name': 'homer', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 7888, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7888}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'cassandra; ' 'do echo ' 'waiting for ' 'cassandra to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum ' 'availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 6, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'message': 'ReplicaSet "homer-767485c9dc" is ' 'progressing.', 'reason': 'ReplicaSetUpdated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': 1, 'unavailable_replicas': 1, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'homer', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/homer', 'uid': '7f83b4e3-5af2-4d79-b532-6c16cad4773d', 'resourceVersion': '1641470', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'homer'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'homer'}}, 'spec': {'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'homer', 'image': 'ollivier/clearwater-homer:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 7888, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'livenessProbe': {'tcpSocket': {'port': 7888}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'tcpSocket': {'port': 7888}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'unavailableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'False', 'lastUpdateTime': '2020-09-29T12:52:08Z', 'lastTransitionTime': '2020-09-29T12:52:08Z', 'reason': 'MinimumReplicasUnavailable', 'message': 'Deployment does not have minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:52:08Z', 'lastTransitionTime': '2020-09-29T12:52:06Z', 'reason': 'ReplicaSetUpdated', 'message': 'ReplicaSet "homer-767485c9dc" is progressing.'}]}}} 2020-09-29 12:52:08,236 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'homestead-prov', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641471', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/homestead-prov', 'uid': '9e6bead5-f8b8-446c-a528-fbe3bf8ba7cd'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'homestead-prov'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'homestead-prov', 'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-homestead-prov:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 60, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'homestead-prov', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 8889, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'homestead; ' 'do echo ' 'waiting for ' 'homestead to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-homestead', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum ' 'availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 6, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'message': 'ReplicaSet "homestead-prov-d67ff8f9b" ' 'is progressing.', 'reason': 'ReplicaSetUpdated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': 1, 'unavailable_replicas': 1, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'homestead-prov', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/homestead-prov', 'uid': '9e6bead5-f8b8-446c-a528-fbe3bf8ba7cd', 'resourceVersion': '1641471', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'homestead-prov'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'homestead-prov', 'snmp': 'enabled'}}, 'spec': {'initContainers': [{'name': 'wait-on-homestead', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup homestead; do echo waiting for homestead to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'homestead-prov', 'image': 'ollivier/clearwater-homestead-prov:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 8889, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'initialDelaySeconds': 60, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'unavailableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'False', 'lastUpdateTime': '2020-09-29T12:52:08Z', 'lastTransitionTime': '2020-09-29T12:52:08Z', 'reason': 'MinimumReplicasUnavailable', 'message': 'Deployment does not have minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:52:08Z', 'lastTransitionTime': '2020-09-29T12:52:06Z', 'reason': 'ReplicaSetUpdated', 'message': 'ReplicaSet "homestead-prov-d67ff8f9b" is progressing.'}]}}} 2020-09-29 12:52:08,247 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'astaire', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641472', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/astaire', 'uid': '5854d3fa-058f-4e83-8159-1cc01de9d297'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'astaire'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'astaire'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-astaire:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': {'post_start': None, 'pre_stop': {'_exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}, 'http_get': None, 'tcp_socket': None}}, 'liveness_probe': {'_exec': None, 'failure_threshold': 9, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 11311}, 'timeout_seconds': 1}, 'name': 'astaire', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 11211, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 11311, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 11311}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/astaire', 'mount_propagation': None, 'name': 'astairelogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/astaire/astaire_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/astaire', 'mount_propagation': None, 'name': 'astairelogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'cassandra; ' 'do echo ' 'waiting for ' 'cassandra to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 120, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'astairelogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum ' 'availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 6, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'message': 'ReplicaSet "astaire-65dfc8d888" is ' 'progressing.', 'reason': 'ReplicaSetUpdated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': 1, 'unavailable_replicas': 1, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'astaire', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/astaire', 'uid': '5854d3fa-058f-4e83-8159-1cc01de9d297', 'resourceVersion': '1641472', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'astaire'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'astaire'}}, 'spec': {'volumes': [{'name': 'astairelogs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'astaire', 'image': 'ollivier/clearwater-astaire:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 11211, 'protocol': 'TCP'}, {'containerPort': 11311, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'astairelogs', 'mountPath': '/var/log/astaire'}], 'livenessProbe': {'tcpSocket': {'port': 11311}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 9}, 'readinessProbe': {'tcpSocket': {'port': 11311}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'lifecycle': {'preStop': {'exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}}}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'tailer', 'image': 'busybox:1.28', 'command': ['tail', '-F', '/var/log/astaire/astaire_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'astairelogs', 'mountPath': '/var/log/astaire'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 120, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'unavailableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'False', 'lastUpdateTime': '2020-09-29T12:52:08Z', 'lastTransitionTime': '2020-09-29T12:52:08Z', 'reason': 'MinimumReplicasUnavailable', 'message': 'Deployment does not have minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:52:08Z', 'lastTransitionTime': '2020-09-29T12:52:06Z', 'reason': 'ReplicaSetUpdated', 'message': 'ReplicaSet "astaire-65dfc8d888" is progressing.'}]}}} 2020-09-29 12:52:08,296 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'sprout', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641479', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/sprout', 'uid': 'd933e2f6-a77a-4051-ae84-52c606034448'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'sprout'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'sprout', 'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-sprout:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 ' '5054']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 3, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'sprout', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 ' '5054']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/sprout', 'mount_propagation': None, 'name': 'sproutlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/sprout/sprout_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/sprout', 'mount_propagation': None, 'name': 'sproutlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'ralf; do ' 'echo waiting ' 'for ralf to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-ralf', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'chronos; do ' 'echo waiting ' 'for chronos ' 'to start ' '...; sleep ' '2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-chronos', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'homestead; ' 'do echo ' 'waiting for ' 'homestead to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-homestead', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'sproutlogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum ' 'availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 6, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'message': 'ReplicaSet "sprout-768bcbd967" is ' 'progressing.', 'reason': 'ReplicaSetUpdated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': 1, 'unavailable_replicas': 1, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'sprout', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/sprout', 'uid': 'd933e2f6-a77a-4051-ae84-52c606034448', 'resourceVersion': '1641479', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'sprout'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'sprout', 'snmp': 'enabled'}}, 'spec': {'volumes': [{'name': 'sproutlogs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-ralf', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup ralf; do echo waiting for ralf to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'wait-on-chronos', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup chronos; do echo waiting for chronos to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'wait-on-homestead', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup homestead; do echo waiting for homestead to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'sprout', 'image': 'ollivier/clearwater-sprout:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'sproutlogs', 'mountPath': '/var/log/sprout'}], 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 5054']}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 3, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 5054']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'tailer', 'image': 'busybox:1.28', 'command': ['tail', '-F', '/var/log/sprout/sprout_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'sproutlogs', 'mountPath': '/var/log/sprout'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'unavailableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'False', 'lastUpdateTime': '2020-09-29T12:52:08Z', 'lastTransitionTime': '2020-09-29T12:52:08Z', 'reason': 'MinimumReplicasUnavailable', 'message': 'Deployment does not have minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:52:08Z', 'lastTransitionTime': '2020-09-29T12:52:06Z', 'reason': 'ReplicaSetUpdated', 'message': 'ReplicaSet "sprout-768bcbd967" is progressing.'}]}}} 2020-09-29 12:52:08,316 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'ralf', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641480', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/ralf', 'uid': 'b88a9a7f-0df4-4fd5-b389-2bf6e218f350'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'ralf'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'ralf', 'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-ralf:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 10888}, 'timeout_seconds': 1}, 'name': 'ralf', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 10888, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 10888}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/ralf', 'mount_propagation': None, 'name': 'ralflogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/ralf/ralf_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/ralf', 'mount_propagation': None, 'name': 'ralflogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'cassandra; ' 'do echo ' 'waiting for ' 'cassandra to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'ralflogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'message': 'Created new replica set ' '"ralf-7449977d5c"', 'reason': 'NewReplicaSetCreated', 'status': 'True', 'type': 'Progressing'}, {'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum ' 'availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': None, 'unavailable_replicas': 1, 'updated_replicas': None}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'ralf', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/ralf', 'uid': 'b88a9a7f-0df4-4fd5-b389-2bf6e218f350', 'resourceVersion': '1641480', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'ralf'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'ralf', 'snmp': 'enabled'}}, 'spec': {'volumes': [{'name': 'ralflogs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'ralf', 'image': 'ollivier/clearwater-ralf:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 10888, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'ralflogs', 'mountPath': '/var/log/ralf'}], 'livenessProbe': {'tcpSocket': {'port': 10888}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'tcpSocket': {'port': 10888}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'tailer', 'image': 'busybox:1.28', 'command': ['tail', '-F', '/var/log/ralf/ralf_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'ralflogs', 'mountPath': '/var/log/ralf'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'unavailableReplicas': 1, 'conditions': [{'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:52:07Z', 'lastTransitionTime': '2020-09-29T12:52:07Z', 'reason': 'NewReplicaSetCreated', 'message': 'Created new replica set "ralf-7449977d5c"'}, {'type': 'Available', 'status': 'False', 'lastUpdateTime': '2020-09-29T12:52:08Z', 'lastTransitionTime': '2020-09-29T12:52:08Z', 'reason': 'MinimumReplicasUnavailable', 'message': 'Deployment does not have minimum availability.'}]}}} 2020-09-29 12:52:08,330 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'homestead', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641482', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/homestead', 'uid': 'f1a2a027-9c0f-452c-b7c5-e46bba0d7433'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'homestead'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'homestead', 'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-homestead:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 60, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'homestead', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 8888, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/homestead', 'mount_propagation': None, 'name': 'homesteadlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/homestead/homestead_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/homestead', 'mount_propagation': None, 'name': 'homesteadlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'cassandra; ' 'do echo ' 'waiting for ' 'cassandra to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'astaire; do ' 'echo waiting ' 'for astaire ' 'to start ' '...; sleep ' '2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-astaire', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'homesteadlogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'message': 'Created new replica set ' '"homestead-86d688d85d"', 'reason': 'NewReplicaSetCreated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': None, 'ready_replicas': None, 'replicas': None, 'unavailable_replicas': None, 'updated_replicas': None}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'homestead', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/homestead', 'uid': 'f1a2a027-9c0f-452c-b7c5-e46bba0d7433', 'resourceVersion': '1641482', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'homestead'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'homestead', 'snmp': 'enabled'}}, 'spec': {'volumes': [{'name': 'homesteadlogs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'wait-on-astaire', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup astaire; do echo waiting for astaire to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'homestead', 'image': 'ollivier/clearwater-homestead:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 8888, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'homesteadlogs', 'mountPath': '/var/log/homestead'}], 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'initialDelaySeconds': 60, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'tailer', 'image': 'busybox:1.28', 'command': ['tail', '-F', '/var/log/homestead/homestead_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'homesteadlogs', 'mountPath': '/var/log/homestead'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'conditions': [{'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:52:08Z', 'lastTransitionTime': '2020-09-29T12:52:08Z', 'reason': 'NewReplicaSetCreated', 'message': 'Created new replica set "homestead-86d688d85d"'}]}}} 2020-09-29 12:52:08,389 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'cassandra', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641487', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/cassandra', 'uid': '578317be-b028-4475-a59c-b55a7432b679'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'cassandra'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'cassandra'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-cassandra:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 ' '9042 ' '9160']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 600, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'cassandra', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 7001, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 9042, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 9160, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 ' '9042 ' '9160']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'etcd; do ' 'echo waiting ' 'for etcd to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-etcd', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'message': 'Created new replica set ' '"cassandra-864f6d8f65"', 'reason': 'NewReplicaSetCreated', 'status': 'True', 'type': 'Progressing'}, {'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum ' 'availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': None, 'unavailable_replicas': 1, 'updated_replicas': None}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'cassandra', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/cassandra', 'uid': '578317be-b028-4475-a59c-b55a7432b679', 'resourceVersion': '1641487', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'cassandra'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'cassandra'}}, 'spec': {'initContainers': [{'name': 'wait-on-etcd', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup etcd; do echo waiting for etcd to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'cassandra', 'image': 'ollivier/clearwater-cassandra:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 7001, 'protocol': 'TCP'}, {'containerPort': 9042, 'protocol': 'TCP'}, {'containerPort': 9160, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 9042 9160']}, 'initialDelaySeconds': 600, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 9042 9160']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'unavailableReplicas': 1, 'conditions': [{'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:52:07Z', 'lastTransitionTime': '2020-09-29T12:52:07Z', 'reason': 'NewReplicaSetCreated', 'message': 'Created new replica set "cassandra-864f6d8f65"'}, {'type': 'Available', 'status': 'False', 'lastUpdateTime': '2020-09-29T12:52:08Z', 'lastTransitionTime': '2020-09-29T12:52:08Z', 'reason': 'MinimumReplicasUnavailable', 'message': 'Deployment does not have minimum availability.'}]}}} 2020-09-29 12:52:08,406 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'etcd', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641488', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/etcd', 'uid': 'da6efd4b-a7fc-4f36-9abf-8ca3a644590f'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'instance-type': 'etcd-pod'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'instance-type': 'etcd-pod'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': ['--name', '$(MY_POD_NAME)', '--advertise-client-urls', 'http://$(MY_POD_IP):2379,http://$(MY_POD_IP):4001', '--listen-client-urls', 'http://0.0.0.0:2379,http://0.0.0.0:4001', '--initial-advertise-peer-urls', 'http://$(MY_POD_IP):2380', '--listen-peer-urls', 'http://0.0.0.0:2380', '--initial-cluster', '$(MY_POD_NAME)=http://$(MY_POD_IP):2380', '--initial-cluster-state', 'new'], 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}, {'name': 'MY_POD_NAME', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'metadata.name'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': None, 'image': 'quay.io/coreos/etcd:v2.2.5', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 300, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 4001}, 'timeout_seconds': 1}, 'name': 'etcd', 'ports': [{'container_port': 2379, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 2380, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 4001, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 4001}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': None, 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum ' 'availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'message': 'ReplicaSet "etcd-744b4d9f98" is ' 'progressing.', 'reason': 'ReplicaSetUpdated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': 1, 'unavailable_replicas': 1, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'etcd', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/etcd', 'uid': 'da6efd4b-a7fc-4f36-9abf-8ca3a644590f', 'resourceVersion': '1641488', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'instance-type': 'etcd-pod'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'instance-type': 'etcd-pod'}}, 'spec': {'containers': [{'name': 'etcd', 'image': 'quay.io/coreos/etcd:v2.2.5', 'args': ['--name', '$(MY_POD_NAME)', '--advertise-client-urls', 'http://$(MY_POD_IP):2379,http://$(MY_POD_IP):4001', '--listen-client-urls', 'http://0.0.0.0:2379,http://0.0.0.0:4001', '--initial-advertise-peer-urls', 'http://$(MY_POD_IP):2380', '--listen-peer-urls', 'http://0.0.0.0:2380', '--initial-cluster', '$(MY_POD_NAME)=http://$(MY_POD_IP):2380', '--initial-cluster-state', 'new'], 'ports': [{'containerPort': 2379, 'protocol': 'TCP'}, {'containerPort': 2380, 'protocol': 'TCP'}, {'containerPort': 4001, 'protocol': 'TCP'}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}, {'name': 'MY_POD_NAME', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'metadata.name'}}}], 'resources': {}, 'livenessProbe': {'tcpSocket': {'port': 4001}, 'initialDelaySeconds': 300, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'tcpSocket': {'port': 4001}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'unavailableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'False', 'lastUpdateTime': '2020-09-29T12:52:08Z', 'lastTransitionTime': '2020-09-29T12:52:08Z', 'reason': 'MinimumReplicasUnavailable', 'message': 'Deployment does not have minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:52:08Z', 'lastTransitionTime': '2020-09-29T12:52:07Z', 'reason': 'ReplicaSetUpdated', 'message': 'ReplicaSet "etcd-744b4d9f98" is progressing.'}]}}} 2020-09-29 12:52:08,416 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm', 'service': 'chronos'}, 'managed_fields': None, 'name': 'chronos', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641494', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/chronos', 'uid': '52d3b182-86dc-451c-9cd5-b61a4e65f189'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'chronos'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'chronos'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-chronos:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': {'post_start': None, 'pre_stop': {'_exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}, 'http_get': None, 'tcp_socket': None}}, 'liveness_probe': {'_exec': None, 'failure_threshold': 9, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7253}, 'timeout_seconds': 1}, 'name': 'chronos', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 7253, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7253}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/chronos', 'mount_propagation': None, 'name': 'chronoslogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/chronos/chronos_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/chronos', 'mount_propagation': None, 'name': 'chronoslogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'cassandra; ' 'do echo ' 'waiting for ' 'cassandra to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 120, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'chronoslogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum ' 'availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'message': 'ReplicaSet "chronos-6dff6b5ff8" is ' 'progressing.', 'reason': 'ReplicaSetUpdated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': 1, 'unavailable_replicas': 1, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'chronos', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/chronos', 'uid': '52d3b182-86dc-451c-9cd5-b61a4e65f189', 'resourceVersion': '1641494', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm', 'service': 'chronos'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'chronos'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'chronos'}}, 'spec': {'volumes': [{'name': 'chronoslogs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'chronos', 'image': 'ollivier/clearwater-chronos:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 7253, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'chronoslogs', 'mountPath': '/var/log/chronos'}], 'livenessProbe': {'tcpSocket': {'port': 7253}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 9}, 'readinessProbe': {'tcpSocket': {'port': 7253}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'lifecycle': {'preStop': {'exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}}}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'tailer', 'image': 'busybox:1.28', 'command': ['tail', '-F', '/var/log/chronos/chronos_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'chronoslogs', 'mountPath': '/var/log/chronos'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 120, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'unavailableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'False', 'lastUpdateTime': '2020-09-29T12:52:08Z', 'lastTransitionTime': '2020-09-29T12:52:08Z', 'reason': 'MinimumReplicasUnavailable', 'message': 'Deployment does not have minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:52:08Z', 'lastTransitionTime': '2020-09-29T12:52:07Z', 'reason': 'ReplicaSetUpdated', 'message': 'ReplicaSet "chronos-6dff6b5ff8" is progressing.'}]}}} 2020-09-29 12:52:08,440 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'bono', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641499', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/bono', 'uid': '39b3eef8-15d5-4e88-ac4d-ce0d8a783d4d'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'bono'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'bono', 'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-bono:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 ' '5060 ' '5062']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'bono', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 3478, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 5060, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 5062, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 5060, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'UDP'}, {'container_port': 5062, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'UDP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 ' '5060 ' '5062']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/bono', 'mount_propagation': None, 'name': 'bonologs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/bono/bono_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/bono', 'mount_propagation': None, 'name': 'bonologs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'sprout; do ' 'echo waiting ' 'for sprout ' 'to start ' '...; sleep ' '2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-sprout', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'bonologs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum ' 'availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'message': 'ReplicaSet "bono-649d6c88df" is ' 'progressing.', 'reason': 'ReplicaSetUpdated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': 1, 'unavailable_replicas': 1, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'bono', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/bono', 'uid': '39b3eef8-15d5-4e88-ac4d-ce0d8a783d4d', 'resourceVersion': '1641499', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'bono'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'bono', 'snmp': 'enabled'}}, 'spec': {'volumes': [{'name': 'bonologs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-sprout', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup sprout; do echo waiting for sprout to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'bono', 'image': 'ollivier/clearwater-bono:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 3478, 'protocol': 'TCP'}, {'containerPort': 5060, 'protocol': 'TCP'}, {'containerPort': 5062, 'protocol': 'TCP'}, {'containerPort': 5060, 'protocol': 'UDP'}, {'containerPort': 5062, 'protocol': 'UDP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'bonologs', 'mountPath': '/var/log/bono'}], 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 5060 5062']}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 5060 5062']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'tailer', 'image': 'busybox:1.28', 'command': ['tail', '-F', '/var/log/bono/bono_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'bonologs', 'mountPath': '/var/log/bono'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'unavailableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'False', 'lastUpdateTime': '2020-09-29T12:52:08Z', 'lastTransitionTime': '2020-09-29T12:52:08Z', 'reason': 'MinimumReplicasUnavailable', 'message': 'Deployment does not have minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:52:08Z', 'lastTransitionTime': '2020-09-29T12:52:07Z', 'reason': 'ReplicaSetUpdated', 'message': 'ReplicaSet "bono-649d6c88df" is progressing.'}]}}} 2020-09-29 12:52:08,459 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'cassandra', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641500', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/cassandra', 'uid': '578317be-b028-4475-a59c-b55a7432b679'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'cassandra'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'cassandra'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-cassandra:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 ' '9042 ' '9160']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 600, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'cassandra', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 7001, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 9042, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 9160, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 ' '9042 ' '9160']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'etcd; do ' 'echo waiting ' 'for etcd to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-etcd', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum ' 'availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'message': 'ReplicaSet "cassandra-864f6d8f65" is ' 'progressing.', 'reason': 'ReplicaSetUpdated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': 1, 'unavailable_replicas': 1, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'cassandra', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/cassandra', 'uid': '578317be-b028-4475-a59c-b55a7432b679', 'resourceVersion': '1641500', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'cassandra'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'cassandra'}}, 'spec': {'initContainers': [{'name': 'wait-on-etcd', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup etcd; do echo waiting for etcd to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'cassandra', 'image': 'ollivier/clearwater-cassandra:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 7001, 'protocol': 'TCP'}, {'containerPort': 9042, 'protocol': 'TCP'}, {'containerPort': 9160, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 9042 9160']}, 'initialDelaySeconds': 600, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 9042 9160']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'unavailableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'False', 'lastUpdateTime': '2020-09-29T12:52:08Z', 'lastTransitionTime': '2020-09-29T12:52:08Z', 'reason': 'MinimumReplicasUnavailable', 'message': 'Deployment does not have minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:52:08Z', 'lastTransitionTime': '2020-09-29T12:52:07Z', 'reason': 'ReplicaSetUpdated', 'message': 'ReplicaSet "cassandra-864f6d8f65" is progressing.'}]}}} 2020-09-29 12:52:08,470 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'ralf', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641505', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/ralf', 'uid': 'b88a9a7f-0df4-4fd5-b389-2bf6e218f350'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'ralf'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'ralf', 'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-ralf:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 10888}, 'timeout_seconds': 1}, 'name': 'ralf', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 10888, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 10888}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/ralf', 'mount_propagation': None, 'name': 'ralflogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/ralf/ralf_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/ralf', 'mount_propagation': None, 'name': 'ralflogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'cassandra; ' 'do echo ' 'waiting for ' 'cassandra to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'ralflogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum ' 'availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'message': 'ReplicaSet "ralf-7449977d5c" is ' 'progressing.', 'reason': 'ReplicaSetUpdated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': 1, 'unavailable_replicas': 1, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'ralf', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/ralf', 'uid': 'b88a9a7f-0df4-4fd5-b389-2bf6e218f350', 'resourceVersion': '1641505', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'ralf'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'ralf', 'snmp': 'enabled'}}, 'spec': {'volumes': [{'name': 'ralflogs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'ralf', 'image': 'ollivier/clearwater-ralf:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 10888, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'ralflogs', 'mountPath': '/var/log/ralf'}], 'livenessProbe': {'tcpSocket': {'port': 10888}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'tcpSocket': {'port': 10888}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'tailer', 'image': 'busybox:1.28', 'command': ['tail', '-F', '/var/log/ralf/ralf_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'ralflogs', 'mountPath': '/var/log/ralf'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'unavailableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'False', 'lastUpdateTime': '2020-09-29T12:52:08Z', 'lastTransitionTime': '2020-09-29T12:52:08Z', 'reason': 'MinimumReplicasUnavailable', 'message': 'Deployment does not have minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:52:08Z', 'lastTransitionTime': '2020-09-29T12:52:07Z', 'reason': 'ReplicaSetUpdated', 'message': 'ReplicaSet "ralf-7449977d5c" is progressing.'}]}}} 2020-09-29 12:52:08,541 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'homestead', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641511', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/homestead', 'uid': 'f1a2a027-9c0f-452c-b7c5-e46bba0d7433'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'homestead'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'homestead', 'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-homestead:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 60, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'homestead', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 8888, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/homestead', 'mount_propagation': None, 'name': 'homesteadlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/homestead/homestead_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/homestead', 'mount_propagation': None, 'name': 'homesteadlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'cassandra; ' 'do echo ' 'waiting for ' 'cassandra to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'astaire; do ' 'echo waiting ' 'for astaire ' 'to start ' '...; sleep ' '2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-astaire', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'homesteadlogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'message': 'Created new replica set ' '"homestead-86d688d85d"', 'reason': 'NewReplicaSetCreated', 'status': 'True', 'type': 'Progressing'}, {'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum ' 'availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': None, 'unavailable_replicas': 1, 'updated_replicas': None}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'homestead', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/homestead', 'uid': 'f1a2a027-9c0f-452c-b7c5-e46bba0d7433', 'resourceVersion': '1641511', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'homestead'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'homestead', 'snmp': 'enabled'}}, 'spec': {'volumes': [{'name': 'homesteadlogs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'wait-on-astaire', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup astaire; do echo waiting for astaire to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'homestead', 'image': 'ollivier/clearwater-homestead:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 8888, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'homesteadlogs', 'mountPath': '/var/log/homestead'}], 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'initialDelaySeconds': 60, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'tailer', 'image': 'busybox:1.28', 'command': ['tail', '-F', '/var/log/homestead/homestead_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'homesteadlogs', 'mountPath': '/var/log/homestead'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'unavailableReplicas': 1, 'conditions': [{'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:52:08Z', 'lastTransitionTime': '2020-09-29T12:52:08Z', 'reason': 'NewReplicaSetCreated', 'message': 'Created new replica set "homestead-86d688d85d"'}, {'type': 'Available', 'status': 'False', 'lastUpdateTime': '2020-09-29T12:52:08Z', 'lastTransitionTime': '2020-09-29T12:52:08Z', 'reason': 'MinimumReplicasUnavailable', 'message': 'Deployment does not have minimum availability.'}]}}} 2020-09-29 12:52:08,582 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'homestead', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641515', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/homestead', 'uid': 'f1a2a027-9c0f-452c-b7c5-e46bba0d7433'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'homestead'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'homestead', 'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-homestead:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 60, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'homestead', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 8888, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/homestead', 'mount_propagation': None, 'name': 'homesteadlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/homestead/homestead_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/homestead', 'mount_propagation': None, 'name': 'homesteadlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'cassandra; ' 'do echo ' 'waiting for ' 'cassandra to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'astaire; do ' 'echo waiting ' 'for astaire ' 'to start ' '...; sleep ' '2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-astaire', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'homesteadlogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum ' 'availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'message': 'ReplicaSet "homestead-86d688d85d" is ' 'progressing.', 'reason': 'ReplicaSetUpdated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': 1, 'unavailable_replicas': 1, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'homestead', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/homestead', 'uid': 'f1a2a027-9c0f-452c-b7c5-e46bba0d7433', 'resourceVersion': '1641515', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'homestead'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'homestead', 'snmp': 'enabled'}}, 'spec': {'volumes': [{'name': 'homesteadlogs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'wait-on-astaire', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup astaire; do echo waiting for astaire to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'homestead', 'image': 'ollivier/clearwater-homestead:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 8888, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'homesteadlogs', 'mountPath': '/var/log/homestead'}], 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'initialDelaySeconds': 60, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'tailer', 'image': 'busybox:1.28', 'command': ['tail', '-F', '/var/log/homestead/homestead_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'homesteadlogs', 'mountPath': '/var/log/homestead'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'unavailableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'False', 'lastUpdateTime': '2020-09-29T12:52:08Z', 'lastTransitionTime': '2020-09-29T12:52:08Z', 'reason': 'MinimumReplicasUnavailable', 'message': 'Deployment does not have minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:52:08Z', 'lastTransitionTime': '2020-09-29T12:52:08Z', 'reason': 'ReplicaSetUpdated', 'message': 'ReplicaSet "homestead-86d688d85d" is progressing.'}]}}} 2020-09-29 12:52:23,648 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'etcd', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1641689', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/etcd', 'uid': 'da6efd4b-a7fc-4f36-9abf-8ca3a644590f'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'instance-type': 'etcd-pod'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'instance-type': 'etcd-pod'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': ['--name', '$(MY_POD_NAME)', '--advertise-client-urls', 'http://$(MY_POD_IP):2379,http://$(MY_POD_IP):4001', '--listen-client-urls', 'http://0.0.0.0:2379,http://0.0.0.0:4001', '--initial-advertise-peer-urls', 'http://$(MY_POD_IP):2380', '--listen-peer-urls', 'http://0.0.0.0:2380', '--initial-cluster', '$(MY_POD_NAME)=http://$(MY_POD_IP):2380', '--initial-cluster-state', 'new'], 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}, {'name': 'MY_POD_NAME', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'metadata.name'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': None, 'image': 'quay.io/coreos/etcd:v2.2.5', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 300, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 4001}, 'timeout_seconds': 1}, 'name': 'etcd', 'ports': [{'container_port': 2379, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 2380, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 4001, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 4001}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': None, 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': 1, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 23, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 23, tzinfo=tzlocal()), 'message': 'Deployment has minimum availability.', 'reason': 'MinimumReplicasAvailable', 'status': 'True', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 52, 23, tzinfo=tzlocal()), 'message': 'ReplicaSet "etcd-744b4d9f98" has ' 'successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': 1, 'replicas': 1, 'unavailable_replicas': None, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'etcd', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/etcd', 'uid': 'da6efd4b-a7fc-4f36-9abf-8ca3a644590f', 'resourceVersion': '1641689', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'instance-type': 'etcd-pod'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'instance-type': 'etcd-pod'}}, 'spec': {'containers': [{'name': 'etcd', 'image': 'quay.io/coreos/etcd:v2.2.5', 'args': ['--name', '$(MY_POD_NAME)', '--advertise-client-urls', 'http://$(MY_POD_IP):2379,http://$(MY_POD_IP):4001', '--listen-client-urls', 'http://0.0.0.0:2379,http://0.0.0.0:4001', '--initial-advertise-peer-urls', 'http://$(MY_POD_IP):2380', '--listen-peer-urls', 'http://0.0.0.0:2380', '--initial-cluster', '$(MY_POD_NAME)=http://$(MY_POD_IP):2380', '--initial-cluster-state', 'new'], 'ports': [{'containerPort': 2379, 'protocol': 'TCP'}, {'containerPort': 2380, 'protocol': 'TCP'}, {'containerPort': 4001, 'protocol': 'TCP'}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}, {'name': 'MY_POD_NAME', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'metadata.name'}}}], 'resources': {}, 'livenessProbe': {'tcpSocket': {'port': 4001}, 'initialDelaySeconds': 300, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'tcpSocket': {'port': 4001}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'readyReplicas': 1, 'availableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:52:23Z', 'lastTransitionTime': '2020-09-29T12:52:23Z', 'reason': 'MinimumReplicasAvailable', 'message': 'Deployment has minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:52:23Z', 'lastTransitionTime': '2020-09-29T12:52:07Z', 'reason': 'NewReplicaSetAvailable', 'message': 'ReplicaSet "etcd-744b4d9f98" has successfully progressed.'}]}}} 2020-09-29 12:52:23,657 - functest_kubernetes.ims.ims - INFO - etcd started in 30.58 sec 2020-09-29 12:54:01,237 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'cassandra', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1642227', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/cassandra', 'uid': '578317be-b028-4475-a59c-b55a7432b679'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'cassandra'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'cassandra'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-cassandra:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 ' '9042 ' '9160']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 600, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'cassandra', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 7001, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 9042, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 9160, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 ' '9042 ' '9160']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'etcd; do ' 'echo waiting ' 'for etcd to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-etcd', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': 1, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 54, 1, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 54, 1, tzinfo=tzlocal()), 'message': 'Deployment has minimum availability.', 'reason': 'MinimumReplicasAvailable', 'status': 'True', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 54, 1, tzinfo=tzlocal()), 'message': 'ReplicaSet "cassandra-864f6d8f65" has ' 'successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': 1, 'replicas': 1, 'unavailable_replicas': None, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'cassandra', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/cassandra', 'uid': '578317be-b028-4475-a59c-b55a7432b679', 'resourceVersion': '1642227', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'cassandra'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'cassandra'}}, 'spec': {'initContainers': [{'name': 'wait-on-etcd', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup etcd; do echo waiting for etcd to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'cassandra', 'image': 'ollivier/clearwater-cassandra:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 7001, 'protocol': 'TCP'}, {'containerPort': 9042, 'protocol': 'TCP'}, {'containerPort': 9160, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 9042 9160']}, 'initialDelaySeconds': 600, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 9042 9160']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'readyReplicas': 1, 'availableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:54:01Z', 'lastTransitionTime': '2020-09-29T12:54:01Z', 'reason': 'MinimumReplicasAvailable', 'message': 'Deployment has minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:54:01Z', 'lastTransitionTime': '2020-09-29T12:52:07Z', 'reason': 'NewReplicaSetAvailable', 'message': 'ReplicaSet "cassandra-864f6d8f65" has successfully progressed.'}]}}} 2020-09-29 12:54:01,248 - functest_kubernetes.ims.ims - INFO - cassandra started in 128.17 sec 2020-09-29 12:54:34,122 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'homer', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1642422', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/homer', 'uid': '7f83b4e3-5af2-4d79-b532-6c16cad4773d'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'homer'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'homer'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-homer:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7888}, 'timeout_seconds': 1}, 'name': 'homer', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 7888, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7888}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'cassandra; ' 'do echo ' 'waiting for ' 'cassandra to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': 1, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 54, 34, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 54, 34, tzinfo=tzlocal()), 'message': 'Deployment has minimum availability.', 'reason': 'MinimumReplicasAvailable', 'status': 'True', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 6, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 54, 34, tzinfo=tzlocal()), 'message': 'ReplicaSet "homer-767485c9dc" has ' 'successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': 1, 'replicas': 1, 'unavailable_replicas': None, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'homer', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/homer', 'uid': '7f83b4e3-5af2-4d79-b532-6c16cad4773d', 'resourceVersion': '1642422', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'homer'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'homer'}}, 'spec': {'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'homer', 'image': 'ollivier/clearwater-homer:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 7888, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'livenessProbe': {'tcpSocket': {'port': 7888}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'tcpSocket': {'port': 7888}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'readyReplicas': 1, 'availableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:54:34Z', 'lastTransitionTime': '2020-09-29T12:54:34Z', 'reason': 'MinimumReplicasAvailable', 'message': 'Deployment has minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:54:34Z', 'lastTransitionTime': '2020-09-29T12:52:06Z', 'reason': 'NewReplicaSetAvailable', 'message': 'ReplicaSet "homer-767485c9dc" has successfully progressed.'}]}}} 2020-09-29 12:54:34,134 - functest_kubernetes.ims.ims - INFO - homer started in 161.06 sec 2020-09-29 12:54:37,625 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'ralf', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1642445', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/ralf', 'uid': 'b88a9a7f-0df4-4fd5-b389-2bf6e218f350'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'ralf'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'ralf', 'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-ralf:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 10888}, 'timeout_seconds': 1}, 'name': 'ralf', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 10888, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 10888}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/ralf', 'mount_propagation': None, 'name': 'ralflogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/ralf/ralf_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/ralf', 'mount_propagation': None, 'name': 'ralflogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'cassandra; ' 'do echo ' 'waiting for ' 'cassandra to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'ralflogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': 1, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 54, 37, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 54, 37, tzinfo=tzlocal()), 'message': 'Deployment has minimum availability.', 'reason': 'MinimumReplicasAvailable', 'status': 'True', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 54, 37, tzinfo=tzlocal()), 'message': 'ReplicaSet "ralf-7449977d5c" has ' 'successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': 1, 'replicas': 1, 'unavailable_replicas': None, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'ralf', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/ralf', 'uid': 'b88a9a7f-0df4-4fd5-b389-2bf6e218f350', 'resourceVersion': '1642445', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'ralf'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'ralf', 'snmp': 'enabled'}}, 'spec': {'volumes': [{'name': 'ralflogs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'ralf', 'image': 'ollivier/clearwater-ralf:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 10888, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'ralflogs', 'mountPath': '/var/log/ralf'}], 'livenessProbe': {'tcpSocket': {'port': 10888}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'tcpSocket': {'port': 10888}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'tailer', 'image': 'busybox:1.28', 'command': ['tail', '-F', '/var/log/ralf/ralf_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'ralflogs', 'mountPath': '/var/log/ralf'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'readyReplicas': 1, 'availableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:54:37Z', 'lastTransitionTime': '2020-09-29T12:54:37Z', 'reason': 'MinimumReplicasAvailable', 'message': 'Deployment has minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:54:37Z', 'lastTransitionTime': '2020-09-29T12:52:07Z', 'reason': 'NewReplicaSetAvailable', 'message': 'ReplicaSet "ralf-7449977d5c" has successfully progressed.'}]}}} 2020-09-29 12:54:37,639 - functest_kubernetes.ims.ims - INFO - ralf started in 164.56 sec 2020-09-29 12:54:39,950 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm', 'service': 'chronos'}, 'managed_fields': None, 'name': 'chronos', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1642456', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/chronos', 'uid': '52d3b182-86dc-451c-9cd5-b61a4e65f189'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'chronos'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'chronos'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-chronos:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': {'post_start': None, 'pre_stop': {'_exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}, 'http_get': None, 'tcp_socket': None}}, 'liveness_probe': {'_exec': None, 'failure_threshold': 9, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7253}, 'timeout_seconds': 1}, 'name': 'chronos', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 7253, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7253}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/chronos', 'mount_propagation': None, 'name': 'chronoslogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/chronos/chronos_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/chronos', 'mount_propagation': None, 'name': 'chronoslogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'cassandra; ' 'do echo ' 'waiting for ' 'cassandra to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 120, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'chronoslogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': 1, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 54, 39, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 54, 39, tzinfo=tzlocal()), 'message': 'Deployment has minimum availability.', 'reason': 'MinimumReplicasAvailable', 'status': 'True', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 54, 39, tzinfo=tzlocal()), 'message': 'ReplicaSet "chronos-6dff6b5ff8" has ' 'successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': 1, 'replicas': 1, 'unavailable_replicas': None, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'chronos', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/chronos', 'uid': '52d3b182-86dc-451c-9cd5-b61a4e65f189', 'resourceVersion': '1642456', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm', 'service': 'chronos'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'chronos'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'chronos'}}, 'spec': {'volumes': [{'name': 'chronoslogs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'chronos', 'image': 'ollivier/clearwater-chronos:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 7253, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'chronoslogs', 'mountPath': '/var/log/chronos'}], 'livenessProbe': {'tcpSocket': {'port': 7253}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 9}, 'readinessProbe': {'tcpSocket': {'port': 7253}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'lifecycle': {'preStop': {'exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}}}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'tailer', 'image': 'busybox:1.28', 'command': ['tail', '-F', '/var/log/chronos/chronos_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'chronoslogs', 'mountPath': '/var/log/chronos'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 120, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'readyReplicas': 1, 'availableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:54:39Z', 'lastTransitionTime': '2020-09-29T12:54:39Z', 'reason': 'MinimumReplicasAvailable', 'message': 'Deployment has minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:54:39Z', 'lastTransitionTime': '2020-09-29T12:52:07Z', 'reason': 'NewReplicaSetAvailable', 'message': 'ReplicaSet "chronos-6dff6b5ff8" has successfully progressed.'}]}}} 2020-09-29 12:54:39,965 - functest_kubernetes.ims.ims - INFO - chronos started in 166.89 sec 2020-09-29 12:55:12,311 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'astaire', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1642640', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/astaire', 'uid': '5854d3fa-058f-4e83-8159-1cc01de9d297'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'astaire'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'astaire'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-astaire:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': {'post_start': None, 'pre_stop': {'_exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}, 'http_get': None, 'tcp_socket': None}}, 'liveness_probe': {'_exec': None, 'failure_threshold': 9, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 11311}, 'timeout_seconds': 1}, 'name': 'astaire', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 11211, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 11311, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 11311}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/astaire', 'mount_propagation': None, 'name': 'astairelogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/astaire/astaire_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/astaire', 'mount_propagation': None, 'name': 'astairelogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'cassandra; ' 'do echo ' 'waiting for ' 'cassandra to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 120, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'astairelogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': 1, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 55, 12, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 55, 12, tzinfo=tzlocal()), 'message': 'Deployment has minimum availability.', 'reason': 'MinimumReplicasAvailable', 'status': 'True', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 6, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 55, 12, tzinfo=tzlocal()), 'message': 'ReplicaSet "astaire-65dfc8d888" has ' 'successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': 1, 'replicas': 1, 'unavailable_replicas': None, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'astaire', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/astaire', 'uid': '5854d3fa-058f-4e83-8159-1cc01de9d297', 'resourceVersion': '1642640', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'astaire'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'astaire'}}, 'spec': {'volumes': [{'name': 'astairelogs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'astaire', 'image': 'ollivier/clearwater-astaire:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 11211, 'protocol': 'TCP'}, {'containerPort': 11311, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'astairelogs', 'mountPath': '/var/log/astaire'}], 'livenessProbe': {'tcpSocket': {'port': 11311}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 9}, 'readinessProbe': {'tcpSocket': {'port': 11311}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'lifecycle': {'preStop': {'exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}}}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'tailer', 'image': 'busybox:1.28', 'command': ['tail', '-F', '/var/log/astaire/astaire_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'astairelogs', 'mountPath': '/var/log/astaire'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 120, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'readyReplicas': 1, 'availableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:55:12Z', 'lastTransitionTime': '2020-09-29T12:55:12Z', 'reason': 'MinimumReplicasAvailable', 'message': 'Deployment has minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:55:12Z', 'lastTransitionTime': '2020-09-29T12:52:06Z', 'reason': 'NewReplicaSetAvailable', 'message': 'ReplicaSet "astaire-65dfc8d888" has successfully progressed.'}]}}} 2020-09-29 12:55:12,324 - functest_kubernetes.ims.ims - INFO - astaire started in 199.25 sec 2020-09-29 12:55:44,253 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'homestead', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1642768', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/homestead', 'uid': 'f1a2a027-9c0f-452c-b7c5-e46bba0d7433'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'homestead'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'homestead', 'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-homestead:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 60, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'homestead', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 8888, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/homestead', 'mount_propagation': None, 'name': 'homesteadlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/homestead/homestead_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/homestead', 'mount_propagation': None, 'name': 'homesteadlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'cassandra; ' 'do echo ' 'waiting for ' 'cassandra to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'astaire; do ' 'echo waiting ' 'for astaire ' 'to start ' '...; sleep ' '2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-astaire', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'homesteadlogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': 1, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 55, 44, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 55, 44, tzinfo=tzlocal()), 'message': 'Deployment has minimum availability.', 'reason': 'MinimumReplicasAvailable', 'status': 'True', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 55, 44, tzinfo=tzlocal()), 'message': 'ReplicaSet "homestead-86d688d85d" has ' 'successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': 1, 'replicas': 1, 'unavailable_replicas': None, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'homestead', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/homestead', 'uid': 'f1a2a027-9c0f-452c-b7c5-e46bba0d7433', 'resourceVersion': '1642768', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'homestead'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'homestead', 'snmp': 'enabled'}}, 'spec': {'volumes': [{'name': 'homesteadlogs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'wait-on-astaire', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup astaire; do echo waiting for astaire to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'homestead', 'image': 'ollivier/clearwater-homestead:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 8888, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'homesteadlogs', 'mountPath': '/var/log/homestead'}], 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'initialDelaySeconds': 60, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'tailer', 'image': 'busybox:1.28', 'command': ['tail', '-F', '/var/log/homestead/homestead_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'homesteadlogs', 'mountPath': '/var/log/homestead'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'readyReplicas': 1, 'availableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:55:44Z', 'lastTransitionTime': '2020-09-29T12:55:44Z', 'reason': 'MinimumReplicasAvailable', 'message': 'Deployment has minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:55:44Z', 'lastTransitionTime': '2020-09-29T12:52:08Z', 'reason': 'NewReplicaSetAvailable', 'message': 'ReplicaSet "homestead-86d688d85d" has successfully progressed.'}]}}} 2020-09-29 12:55:44,266 - functest_kubernetes.ims.ims - INFO - homestead started in 231.19 sec 2020-09-29 12:56:11,017 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'sprout', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1642888', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/sprout', 'uid': 'd933e2f6-a77a-4051-ae84-52c606034448'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'sprout'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'sprout', 'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-sprout:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 ' '5054']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 3, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'sprout', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 ' '5054']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/sprout', 'mount_propagation': None, 'name': 'sproutlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/sprout/sprout_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/sprout', 'mount_propagation': None, 'name': 'sproutlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'ralf; do ' 'echo waiting ' 'for ralf to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-ralf', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'chronos; do ' 'echo waiting ' 'for chronos ' 'to start ' '...; sleep ' '2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-chronos', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'homestead; ' 'do echo ' 'waiting for ' 'homestead to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-homestead', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'sproutlogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': 1, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 56, 10, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 56, 10, tzinfo=tzlocal()), 'message': 'Deployment has minimum availability.', 'reason': 'MinimumReplicasAvailable', 'status': 'True', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 6, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 56, 10, tzinfo=tzlocal()), 'message': 'ReplicaSet "sprout-768bcbd967" has ' 'successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': 1, 'replicas': 1, 'unavailable_replicas': None, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'sprout', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/sprout', 'uid': 'd933e2f6-a77a-4051-ae84-52c606034448', 'resourceVersion': '1642888', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'sprout'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'sprout', 'snmp': 'enabled'}}, 'spec': {'volumes': [{'name': 'sproutlogs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-ralf', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup ralf; do echo waiting for ralf to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'wait-on-chronos', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup chronos; do echo waiting for chronos to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'wait-on-homestead', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup homestead; do echo waiting for homestead to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'sprout', 'image': 'ollivier/clearwater-sprout:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'sproutlogs', 'mountPath': '/var/log/sprout'}], 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 5054']}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 3, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 5054']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'tailer', 'image': 'busybox:1.28', 'command': ['tail', '-F', '/var/log/sprout/sprout_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'sproutlogs', 'mountPath': '/var/log/sprout'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'readyReplicas': 1, 'availableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:56:10Z', 'lastTransitionTime': '2020-09-29T12:56:10Z', 'reason': 'MinimumReplicasAvailable', 'message': 'Deployment has minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:56:10Z', 'lastTransitionTime': '2020-09-29T12:52:06Z', 'reason': 'NewReplicaSetAvailable', 'message': 'ReplicaSet "sprout-768bcbd967" has successfully progressed.'}]}}} 2020-09-29 12:56:11,034 - functest_kubernetes.ims.ims - INFO - sprout started in 257.95 sec 2020-09-29 12:56:15,744 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'homestead-prov', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1642907', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/homestead-prov', 'uid': '9e6bead5-f8b8-446c-a528-fbe3bf8ba7cd'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'homestead-prov'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'homestead-prov', 'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-homestead-prov:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 60, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'homestead-prov', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 8889, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'homestead; ' 'do echo ' 'waiting for ' 'homestead to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-homestead', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': 1, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 56, 15, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 56, 15, tzinfo=tzlocal()), 'message': 'Deployment has minimum availability.', 'reason': 'MinimumReplicasAvailable', 'status': 'True', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 6, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 56, 15, tzinfo=tzlocal()), 'message': 'ReplicaSet "homestead-prov-d67ff8f9b" ' 'has successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': 1, 'replicas': 1, 'unavailable_replicas': None, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'homestead-prov', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/homestead-prov', 'uid': '9e6bead5-f8b8-446c-a528-fbe3bf8ba7cd', 'resourceVersion': '1642907', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'homestead-prov'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'homestead-prov', 'snmp': 'enabled'}}, 'spec': {'initContainers': [{'name': 'wait-on-homestead', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup homestead; do echo waiting for homestead to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'homestead-prov', 'image': 'ollivier/clearwater-homestead-prov:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 8889, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'initialDelaySeconds': 60, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'readyReplicas': 1, 'availableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:56:15Z', 'lastTransitionTime': '2020-09-29T12:56:15Z', 'reason': 'MinimumReplicasAvailable', 'message': 'Deployment has minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:56:15Z', 'lastTransitionTime': '2020-09-29T12:52:06Z', 'reason': 'NewReplicaSetAvailable', 'message': 'ReplicaSet "homestead-prov-d67ff8f9b" has successfully progressed.'}]}}} 2020-09-29 12:56:15,755 - functest_kubernetes.ims.ims - INFO - homestead-prov started in 262.68 sec 2020-09-29 12:56:29,219 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'bono', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1642959', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/bono', 'uid': '39b3eef8-15d5-4e88-ac4d-ce0d8a783d4d'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'bono'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'bono', 'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-bono:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 ' '5060 ' '5062']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'bono', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 3478, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 5060, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 5062, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 5060, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'UDP'}, {'container_port': 5062, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'UDP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 ' '5060 ' '5062']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/bono', 'mount_propagation': None, 'name': 'bonologs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/bono/bono_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/bono', 'mount_propagation': None, 'name': 'bonologs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'sprout; do ' 'echo waiting ' 'for sprout ' 'to start ' '...; sleep ' '2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-sprout', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'bonologs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': 1, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 56, 29, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 56, 29, tzinfo=tzlocal()), 'message': 'Deployment has minimum availability.', 'reason': 'MinimumReplicasAvailable', 'status': 'True', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 56, 29, tzinfo=tzlocal()), 'message': 'ReplicaSet "bono-649d6c88df" has ' 'successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': 1, 'replicas': 1, 'unavailable_replicas': None, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'bono', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/bono', 'uid': '39b3eef8-15d5-4e88-ac4d-ce0d8a783d4d', 'resourceVersion': '1642959', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'bono'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'bono', 'snmp': 'enabled'}}, 'spec': {'volumes': [{'name': 'bonologs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-sprout', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup sprout; do echo waiting for sprout to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'bono', 'image': 'ollivier/clearwater-bono:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 3478, 'protocol': 'TCP'}, {'containerPort': 5060, 'protocol': 'TCP'}, {'containerPort': 5062, 'protocol': 'TCP'}, {'containerPort': 5060, 'protocol': 'UDP'}, {'containerPort': 5062, 'protocol': 'UDP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'bonologs', 'mountPath': '/var/log/bono'}], 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 5060 5062']}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 5060 5062']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'tailer', 'image': 'busybox:1.28', 'command': ['tail', '-F', '/var/log/bono/bono_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'bonologs', 'mountPath': '/var/log/bono'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'readyReplicas': 1, 'availableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:56:29Z', 'lastTransitionTime': '2020-09-29T12:56:29Z', 'reason': 'MinimumReplicasAvailable', 'message': 'Deployment has minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:56:29Z', 'lastTransitionTime': '2020-09-29T12:52:07Z', 'reason': 'NewReplicaSetAvailable', 'message': 'ReplicaSet "bono-649d6c88df" has successfully progressed.'}]}}} 2020-09-29 12:56:29,236 - functest_kubernetes.ims.ims - INFO - bono started in 276.16 sec 2020-09-29 12:56:59,482 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 5, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'ellis', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1643062', 'self_link': '/apis/apps/v1/namespaces/ims-cgn86/deployments/ellis', 'uid': 'ab455ae9-328e-4b0e-a10d-b2ed1d3658e9'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'ellis'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'ellis'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-ellis:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 80}, 'timeout_seconds': 1}, 'name': 'ellis', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 80, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 80}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'homestead-prov; ' 'do echo ' 'waiting for ' 'homestead-prov ' 'to start ' '...; sleep ' '2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-homestead-prov', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'sprout; do ' 'echo waiting ' 'for sprout ' 'to start ' '...; sleep ' '2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-sprout', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': 1, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 9, 29, 12, 56, 59, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 56, 59, tzinfo=tzlocal()), 'message': 'Deployment has minimum availability.', 'reason': 'MinimumReplicasAvailable', 'status': 'True', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 6, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 9, 29, 12, 56, 59, tzinfo=tzlocal()), 'message': 'ReplicaSet "ellis-54c6475b79" has ' 'successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': 1, 'replicas': 1, 'unavailable_replicas': None, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'ellis', 'namespace': 'ims-cgn86', 'selfLink': '/apis/apps/v1/namespaces/ims-cgn86/deployments/ellis', 'uid': 'ab455ae9-328e-4b0e-a10d-b2ed1d3658e9', 'resourceVersion': '1643062', 'generation': 1, 'creationTimestamp': '2020-09-29T12:52:05Z', 'labels': {'app.kubernetes.io/managed-by': 'Helm'}, 'annotations': {'deployment.kubernetes.io/revision': '1', 'meta.helm.sh/release-name': 'clearwater', 'meta.helm.sh/release-namespace': 'ims-cgn86'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'ellis'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'ellis'}}, 'spec': {'initContainers': [{'name': 'wait-on-homestead-prov', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup homestead-prov; do echo waiting for homestead-prov to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'wait-on-sprout', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup sprout; do echo waiting for sprout to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'ellis', 'image': 'ollivier/clearwater-ellis:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 80, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'livenessProbe': {'tcpSocket': {'port': 80}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'tcpSocket': {'port': 80}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'readyReplicas': 1, 'availableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:56:59Z', 'lastTransitionTime': '2020-09-29T12:56:59Z', 'reason': 'MinimumReplicasAvailable', 'message': 'Deployment has minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-09-29T12:56:59Z', 'lastTransitionTime': '2020-09-29T12:52:06Z', 'reason': 'NewReplicaSetAvailable', 'message': 'ReplicaSet "ellis-54c6475b79" has successfully progressed.'}]}}} 2020-09-29 12:56:59,494 - functest_kubernetes.ims.ims - INFO - ellis started in 306.42 sec 2020-09-29 12:58:59,609 - kubernetes.client.rest - DEBUG - response body: {"kind":"Pod","apiVersion":"v1","metadata":{"name":"live-test","namespace":"ims-cgn86","selfLink":"/api/v1/namespaces/ims-cgn86/pods/live-test","uid":"bebe78f5-b78c-4b02-8673-4230b3f7569f","resourceVersion":"1643416","creationTimestamp":"2020-09-29T12:58:59Z","managedFields":[{"manager":"OpenAPI-Generator","operation":"Update","apiVersion":"v1","time":"2020-09-29T12:58:59Z","fieldsType":"FieldsV1","fieldsV1":{"f:spec":{"f:containers":{"k:{\"name\":\"live-test\"}":{".":{},"f:command":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{},"f:terminationGracePeriodSeconds":{}}}}]},"spec":{"volumes":[{"name":"default-token-qcbq9","secret":{"secretName":"default-token-qcbq9","defaultMode":420}}],"containers":[{"name":"live-test","image":"ollivier/clearwater-live-test:hunter","command":["rake","test[ims-cgn86.svc.cluster.local]","PROXY=bono.ims-cgn86.svc.cluster.local","ELLIS=ellis.ims-cgn86.svc.cluster.local","SIGNUP_CODE=secret","--trace"],"resources":{},"volumeMounts":[{"name":"default-token-qcbq9","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Never","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}],"priority":0,"enableServiceLinks":true,"preemptionPolicy":"PreemptLowerPriority"},"status":{"phase":"Pending","qosClass":"BestEffort"}} 2020-09-29 12:58:59,619 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'bono-649d6c88df-', 'generation': None, 'initializers': None, 'labels': {'pod-template-hash': '649d6c88df', 'service': 'bono', 'snmp': 'enabled'}, 'managed_fields': None, 'name': 'bono-649d6c88df-zgj5f', 'namespace': 'ims-cgn86', 'owner_references': [{'api_version': 'apps/v1', 'block_owner_deletion': True, 'controller': True, 'kind': 'ReplicaSet', 'name': 'bono-649d6c88df', 'uid': 'f1075cd9-6967-4094-82d1-b7e708a16a7f'}], 'resource_version': '1642955', 'self_link': '/api/v1/namespaces/ims-cgn86/pods/bono-649d6c88df-zgj5f', 'uid': '72822478-8799-4589-b6ca-a88fe2a70ec3'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-bono:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 5060 ' '5062']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'bono', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 3478, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 5060, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 5062, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 5060, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'UDP'}, {'container_port': 5062, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'UDP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 5060 ' '5062']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/bono', 'mount_propagation': None, 'name': 'bonologs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-qcbq9', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/bono/bono_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/bono', 'mount_propagation': None, 'name': 'bonologs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-qcbq9', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup sprout; do echo ' 'waiting for sprout to start ...; ' 'sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-sprout', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-qcbq9', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'node_name': 'kali-worker2', 'node_selector': None, 'preemption_policy': 'PreemptLowerPriority', 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'bonologs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}, {'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-qcbq9', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-qcbq9'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 56, 17, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 56, 29, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 56, 29, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://33212dc4c95e162f0762d2f39a1c47e613e47848925f1ef9afe47b02f89bc5c0', 'image': 'docker.io/ollivier/clearwater-bono:hunter', 'image_id': 'docker.io/ollivier/clearwater-bono@sha256:0aa1f9ff35f9d5682b1b48a3ceb95acce0166fbf17401af2f86efd61ea91e760', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'bono', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 9, 29, 12, 56, 19, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}, {'container_id': 'containerd://033e5f4efbd30663006606490873df2a12fadbfaefa9e66398ea6187abdaba43', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'tailer', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 9, 29, 12, 56, 21, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.13', 'init_container_statuses': [{'container_id': 'containerd://143b676e6e233d8d218e6ea21c83e73754137c615b39fafb90379f64ebf15de2', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-sprout', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://143b676e6e233d8d218e6ea21c83e73754137c615b39fafb90379f64ebf15de2', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 9, 29, 12, 56, 15, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 9, 29, 12, 52, 23, tzinfo=tzlocal())}, 'waiting': None}}], 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.1.127', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal())}}, 'raw_object': {'kind': 'Pod', 'apiVersion': 'v1', 'metadata': {'name': 'bono-649d6c88df-zgj5f', 'generateName': 'bono-649d6c88df-', 'namespace': 'ims-cgn86', 'selfLink': '/api/v1/namespaces/ims-cgn86/pods/bono-649d6c88df-zgj5f', 'uid': '72822478-8799-4589-b6ca-a88fe2a70ec3', 'resourceVersion': '1642955', 'creationTimestamp': '2020-09-29T12:52:07Z', 'labels': {'pod-template-hash': '649d6c88df', 'service': 'bono', 'snmp': 'enabled'}, 'ownerReferences': [{'apiVersion': 'apps/v1', 'kind': 'ReplicaSet', 'name': 'bono-649d6c88df', 'uid': 'f1075cd9-6967-4094-82d1-b7e708a16a7f', 'controller': True, 'blockOwnerDeletion': True}]}, 'spec': {'volumes': [{'name': 'bonologs', 'emptyDir': {}}, {'name': 'default-token-qcbq9', 'secret': {'secretName': 'default-token-qcbq9', 'defaultMode': 420}}], 'initContainers': [{'name': 'wait-on-sprout', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup sprout; do echo waiting for sprout to start ...; sleep 2; done'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-qcbq9', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'bono', 'image': 'ollivier/clearwater-bono:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 3478, 'protocol': 'TCP'}, {'containerPort': 5060, 'protocol': 'TCP'}, {'containerPort': 5062, 'protocol': 'TCP'}, {'containerPort': 5060, 'protocol': 'UDP'}, {'containerPort': 5062, 'protocol': 'UDP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'bonologs', 'mountPath': '/var/log/bono'}, {'name': 'default-token-qcbq9', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 5060 5062']}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 5060 5062']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'tailer', 'image': 'busybox:1.28', 'command': ['tail', '-F', '/var/log/bono/bono_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'bonologs', 'mountPath': '/var/log/bono'}, {'name': 'default-token-qcbq9', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'serviceAccountName': 'default', 'serviceAccount': 'default', 'nodeName': 'kali-worker2', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler', 'tolerations': [{'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}, {'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}], 'priority': 0, 'enableServiceLinks': True, 'preemptionPolicy': 'PreemptLowerPriority'}, 'status': {'phase': 'Running', 'conditions': [{'type': 'Initialized', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:56:17Z'}, {'type': 'Ready', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:56:29Z'}, {'type': 'ContainersReady', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:56:29Z'}, {'type': 'PodScheduled', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:52:07Z'}], 'hostIP': '172.18.0.13', 'podIP': '10.244.1.127', 'podIPs': [{'ip': '10.244.1.127'}], 'startTime': '2020-09-29T12:52:08Z', 'initContainerStatuses': [{'name': 'wait-on-sprout', 'state': {'terminated': {'exitCode': 0, 'reason': 'Completed', 'startedAt': '2020-09-29T12:52:23Z', 'finishedAt': '2020-09-29T12:56:15Z', 'containerID': 'containerd://143b676e6e233d8d218e6ea21c83e73754137c615b39fafb90379f64ebf15de2'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:1.28', 'imageID': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'containerID': 'containerd://143b676e6e233d8d218e6ea21c83e73754137c615b39fafb90379f64ebf15de2'}], 'containerStatuses': [{'name': 'bono', 'state': {'running': {'startedAt': '2020-09-29T12:56:19Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/ollivier/clearwater-bono:hunter', 'imageID': 'docker.io/ollivier/clearwater-bono@sha256:0aa1f9ff35f9d5682b1b48a3ceb95acce0166fbf17401af2f86efd61ea91e760', 'containerID': 'containerd://33212dc4c95e162f0762d2f39a1c47e613e47848925f1ef9afe47b02f89bc5c0', 'started': True}, {'name': 'tailer', 'state': {'running': {'startedAt': '2020-09-29T12:56:21Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:1.28', 'imageID': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'containerID': 'containerd://033e5f4efbd30663006606490873df2a12fadbfaefa9e66398ea6187abdaba43', 'started': True}], 'qosClass': 'BestEffort'}}} 2020-09-29 12:58:59,652 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'homestead-86d688d85d-', 'generation': None, 'initializers': None, 'labels': {'pod-template-hash': '86d688d85d', 'service': 'homestead', 'snmp': 'enabled'}, 'managed_fields': None, 'name': 'homestead-86d688d85d-5cvzl', 'namespace': 'ims-cgn86', 'owner_references': [{'api_version': 'apps/v1', 'block_owner_deletion': True, 'controller': True, 'kind': 'ReplicaSet', 'name': 'homestead-86d688d85d', 'uid': 'f2dfb54f-5bac-42a1-8c02-f5623d1b4736'}], 'resource_version': '1642764', 'self_link': '/api/v1/namespaces/ims-cgn86/pods/homestead-86d688d85d-5cvzl', 'uid': '271a3fd5-cf19-46f8-834e-c5b6ef5ad219'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-homestead:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 60, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'homestead', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 8888, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/homestead', 'mount_propagation': None, 'name': 'homesteadlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-qcbq9', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/homestead/homestead_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/homestead', 'mount_propagation': None, 'name': 'homesteadlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-qcbq9', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup cassandra; do echo ' 'waiting for cassandra to start ' '...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-qcbq9', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until nslookup astaire; do echo ' 'waiting for astaire to start ...; ' 'sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-astaire', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-qcbq9', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'node_name': 'kali-worker2', 'node_selector': None, 'preemption_policy': 'PreemptLowerPriority', 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'homesteadlogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}, {'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-qcbq9', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-qcbq9'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 55, 31, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 55, 44, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 55, 44, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://fde411da19be5280fc637457874095d9ec375fa8e1d281df18a0ec721103d394', 'image': 'docker.io/ollivier/clearwater-homestead:hunter', 'image_id': 'docker.io/ollivier/clearwater-homestead@sha256:709052509ff00074676a8d53840c92e45004d4bc0c1615ff4ede6bfccf90437d', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'homestead', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 9, 29, 12, 55, 33, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}, {'container_id': 'containerd://825a91c9a382faab8c82887284ec23bb006379b2cafdf954f6e64a132d949b1c', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'tailer', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 9, 29, 12, 55, 35, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.13', 'init_container_statuses': [{'container_id': 'containerd://50da718d1c1913040774867547041834622d07a61f5d1752a4b087c5f0b3ffdf', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-cassandra', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://50da718d1c1913040774867547041834622d07a61f5d1752a4b087c5f0b3ffdf', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 9, 29, 12, 54, 17, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 9, 29, 12, 52, 23, tzinfo=tzlocal())}, 'waiting': None}}, {'container_id': 'containerd://422a2a1a835afa33d2bd84e10c3264365aa60acb4ffe3be0a95aa416f0bec9a7', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-astaire', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://422a2a1a835afa33d2bd84e10c3264365aa60acb4ffe3be0a95aa416f0bec9a7', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 9, 29, 12, 55, 29, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 9, 29, 12, 54, 25, tzinfo=tzlocal())}, 'waiting': None}}], 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.1.129', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 9, 29, 12, 52, 9, tzinfo=tzlocal())}}, 'raw_object': {'kind': 'Pod', 'apiVersion': 'v1', 'metadata': {'name': 'homestead-86d688d85d-5cvzl', 'generateName': 'homestead-86d688d85d-', 'namespace': 'ims-cgn86', 'selfLink': '/api/v1/namespaces/ims-cgn86/pods/homestead-86d688d85d-5cvzl', 'uid': '271a3fd5-cf19-46f8-834e-c5b6ef5ad219', 'resourceVersion': '1642764', 'creationTimestamp': '2020-09-29T12:52:08Z', 'labels': {'pod-template-hash': '86d688d85d', 'service': 'homestead', 'snmp': 'enabled'}, 'ownerReferences': [{'apiVersion': 'apps/v1', 'kind': 'ReplicaSet', 'name': 'homestead-86d688d85d', 'uid': 'f2dfb54f-5bac-42a1-8c02-f5623d1b4736', 'controller': True, 'blockOwnerDeletion': True}]}, 'spec': {'volumes': [{'name': 'homesteadlogs', 'emptyDir': {}}, {'name': 'default-token-qcbq9', 'secret': {'secretName': 'default-token-qcbq9', 'defaultMode': 420}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-qcbq9', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'wait-on-astaire', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup astaire; do echo waiting for astaire to start ...; sleep 2; done'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-qcbq9', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'homestead', 'image': 'ollivier/clearwater-homestead:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 8888, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'homesteadlogs', 'mountPath': '/var/log/homestead'}, {'name': 'default-token-qcbq9', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'initialDelaySeconds': 60, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'tailer', 'image': 'busybox:1.28', 'command': ['tail', '-F', '/var/log/homestead/homestead_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'homesteadlogs', 'mountPath': '/var/log/homestead'}, {'name': 'default-token-qcbq9', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'serviceAccountName': 'default', 'serviceAccount': 'default', 'nodeName': 'kali-worker2', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler', 'tolerations': [{'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}, {'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}], 'priority': 0, 'enableServiceLinks': True, 'preemptionPolicy': 'PreemptLowerPriority'}, 'status': {'phase': 'Running', 'conditions': [{'type': 'Initialized', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:55:31Z'}, {'type': 'Ready', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:55:44Z'}, {'type': 'ContainersReady', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:55:44Z'}, {'type': 'PodScheduled', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:52:08Z'}], 'hostIP': '172.18.0.13', 'podIP': '10.244.1.129', 'podIPs': [{'ip': '10.244.1.129'}], 'startTime': '2020-09-29T12:52:09Z', 'initContainerStatuses': [{'name': 'wait-on-cassandra', 'state': {'terminated': {'exitCode': 0, 'reason': 'Completed', 'startedAt': '2020-09-29T12:52:23Z', 'finishedAt': '2020-09-29T12:54:17Z', 'containerID': 'containerd://50da718d1c1913040774867547041834622d07a61f5d1752a4b087c5f0b3ffdf'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:1.28', 'imageID': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'containerID': 'containerd://50da718d1c1913040774867547041834622d07a61f5d1752a4b087c5f0b3ffdf'}, {'name': 'wait-on-astaire', 'state': {'terminated': {'exitCode': 0, 'reason': 'Completed', 'startedAt': '2020-09-29T12:54:25Z', 'finishedAt': '2020-09-29T12:55:29Z', 'containerID': 'containerd://422a2a1a835afa33d2bd84e10c3264365aa60acb4ffe3be0a95aa416f0bec9a7'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:1.28', 'imageID': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'containerID': 'containerd://422a2a1a835afa33d2bd84e10c3264365aa60acb4ffe3be0a95aa416f0bec9a7'}], 'containerStatuses': [{'name': 'homestead', 'state': {'running': {'startedAt': '2020-09-29T12:55:33Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/ollivier/clearwater-homestead:hunter', 'imageID': 'docker.io/ollivier/clearwater-homestead@sha256:709052509ff00074676a8d53840c92e45004d4bc0c1615ff4ede6bfccf90437d', 'containerID': 'containerd://fde411da19be5280fc637457874095d9ec375fa8e1d281df18a0ec721103d394', 'started': True}, {'name': 'tailer', 'state': {'running': {'startedAt': '2020-09-29T12:55:35Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:1.28', 'imageID': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'containerID': 'containerd://825a91c9a382faab8c82887284ec23bb006379b2cafdf954f6e64a132d949b1c', 'started': True}], 'qosClass': 'BestEffort'}}} 2020-09-29 12:58:59,673 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 6, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'astaire-65dfc8d888-', 'generation': None, 'initializers': None, 'labels': {'pod-template-hash': '65dfc8d888', 'service': 'astaire'}, 'managed_fields': None, 'name': 'astaire-65dfc8d888-8hl8d', 'namespace': 'ims-cgn86', 'owner_references': [{'api_version': 'apps/v1', 'block_owner_deletion': True, 'controller': True, 'kind': 'ReplicaSet', 'name': 'astaire-65dfc8d888', 'uid': 'c28df95d-1552-4aae-ac29-dae0690bd9cd'}], 'resource_version': '1642636', 'self_link': '/api/v1/namespaces/ims-cgn86/pods/astaire-65dfc8d888-8hl8d', 'uid': 'e2eed0a4-99cf-4ead-9271-1bdb3fbef7d5'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-astaire:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': {'post_start': None, 'pre_stop': {'_exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}, 'http_get': None, 'tcp_socket': None}}, 'liveness_probe': {'_exec': None, 'failure_threshold': 9, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 11311}, 'timeout_seconds': 1}, 'name': 'astaire', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 11211, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 11311, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 11311}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/astaire', 'mount_propagation': None, 'name': 'astairelogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-qcbq9', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/astaire/astaire_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/astaire', 'mount_propagation': None, 'name': 'astairelogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-qcbq9', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup cassandra; do echo ' 'waiting for cassandra to start ' '...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-qcbq9', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'node_name': 'kali-worker2', 'node_selector': None, 'preemption_policy': 'PreemptLowerPriority', 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 120, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'astairelogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}, {'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-qcbq9', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-qcbq9'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 54, 18, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 55, 12, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 55, 12, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 6, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://73de0aa87a928bfcb28b3140d2cae29b7a8502ac978c4d248dd7e38943ed2b6d', 'image': 'docker.io/ollivier/clearwater-astaire:hunter', 'image_id': 'docker.io/ollivier/clearwater-astaire@sha256:952f8fa55e26181c90aad327c5a9ec98661ef933decbc3f547836c03447a4c3c', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'astaire', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 9, 29, 12, 54, 25, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}, {'container_id': 'containerd://84d672e22caad2cef479f155f125b40beede5ed020e4e1077beb9c9ad26b79c2', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'tailer', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 9, 29, 12, 54, 28, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.13', 'init_container_statuses': [{'container_id': 'containerd://bcac4a55620a2213692699d49526093e8d023ead3b7e1574625909dd917c2e70', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-cassandra', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://bcac4a55620a2213692699d49526093e8d023ead3b7e1574625909dd917c2e70', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 9, 29, 12, 54, 17, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 9, 29, 12, 52, 16, tzinfo=tzlocal())}, 'waiting': None}}], 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.1.123', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal())}}, 'raw_object': {'kind': 'Pod', 'apiVersion': 'v1', 'metadata': {'name': 'astaire-65dfc8d888-8hl8d', 'generateName': 'astaire-65dfc8d888-', 'namespace': 'ims-cgn86', 'selfLink': '/api/v1/namespaces/ims-cgn86/pods/astaire-65dfc8d888-8hl8d', 'uid': 'e2eed0a4-99cf-4ead-9271-1bdb3fbef7d5', 'resourceVersion': '1642636', 'creationTimestamp': '2020-09-29T12:52:06Z', 'labels': {'pod-template-hash': '65dfc8d888', 'service': 'astaire'}, 'ownerReferences': [{'apiVersion': 'apps/v1', 'kind': 'ReplicaSet', 'name': 'astaire-65dfc8d888', 'uid': 'c28df95d-1552-4aae-ac29-dae0690bd9cd', 'controller': True, 'blockOwnerDeletion': True}]}, 'spec': {'volumes': [{'name': 'astairelogs', 'emptyDir': {}}, {'name': 'default-token-qcbq9', 'secret': {'secretName': 'default-token-qcbq9', 'defaultMode': 420}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-qcbq9', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'astaire', 'image': 'ollivier/clearwater-astaire:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 11211, 'protocol': 'TCP'}, {'containerPort': 11311, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'astairelogs', 'mountPath': '/var/log/astaire'}, {'name': 'default-token-qcbq9', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'livenessProbe': {'tcpSocket': {'port': 11311}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 9}, 'readinessProbe': {'tcpSocket': {'port': 11311}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'lifecycle': {'preStop': {'exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}}}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'tailer', 'image': 'busybox:1.28', 'command': ['tail', '-F', '/var/log/astaire/astaire_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'astairelogs', 'mountPath': '/var/log/astaire'}, {'name': 'default-token-qcbq9', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 120, 'dnsPolicy': 'ClusterFirst', 'serviceAccountName': 'default', 'serviceAccount': 'default', 'nodeName': 'kali-worker2', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler', 'tolerations': [{'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}, {'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}], 'priority': 0, 'enableServiceLinks': True, 'preemptionPolicy': 'PreemptLowerPriority'}, 'status': {'phase': 'Running', 'conditions': [{'type': 'Initialized', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:54:18Z'}, {'type': 'Ready', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:55:12Z'}, {'type': 'ContainersReady', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:55:12Z'}, {'type': 'PodScheduled', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:52:06Z'}], 'hostIP': '172.18.0.13', 'podIP': '10.244.1.123', 'podIPs': [{'ip': '10.244.1.123'}], 'startTime': '2020-09-29T12:52:08Z', 'initContainerStatuses': [{'name': 'wait-on-cassandra', 'state': {'terminated': {'exitCode': 0, 'reason': 'Completed', 'startedAt': '2020-09-29T12:52:16Z', 'finishedAt': '2020-09-29T12:54:17Z', 'containerID': 'containerd://bcac4a55620a2213692699d49526093e8d023ead3b7e1574625909dd917c2e70'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:1.28', 'imageID': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'containerID': 'containerd://bcac4a55620a2213692699d49526093e8d023ead3b7e1574625909dd917c2e70'}], 'containerStatuses': [{'name': 'astaire', 'state': {'running': {'startedAt': '2020-09-29T12:54:25Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/ollivier/clearwater-astaire:hunter', 'imageID': 'docker.io/ollivier/clearwater-astaire@sha256:952f8fa55e26181c90aad327c5a9ec98661ef933decbc3f547836c03447a4c3c', 'containerID': 'containerd://73de0aa87a928bfcb28b3140d2cae29b7a8502ac978c4d248dd7e38943ed2b6d', 'started': True}, {'name': 'tailer', 'state': {'running': {'startedAt': '2020-09-29T12:54:28Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:1.28', 'imageID': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'containerID': 'containerd://84d672e22caad2cef479f155f125b40beede5ed020e4e1077beb9c9ad26b79c2', 'started': True}], 'qosClass': 'BestEffort'}}} 2020-09-29 12:58:59,688 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'etcd-744b4d9f98-', 'generation': None, 'initializers': None, 'labels': {'instance-type': 'etcd-pod', 'pod-template-hash': '744b4d9f98'}, 'managed_fields': None, 'name': 'etcd-744b4d9f98-xsvtv', 'namespace': 'ims-cgn86', 'owner_references': [{'api_version': 'apps/v1', 'block_owner_deletion': True, 'controller': True, 'kind': 'ReplicaSet', 'name': 'etcd-744b4d9f98', 'uid': 'd2dc3cf6-e1d7-431c-b42c-a3522e20b65a'}], 'resource_version': '1641685', 'self_link': '/api/v1/namespaces/ims-cgn86/pods/etcd-744b4d9f98-xsvtv', 'uid': 'e0b74bf5-24a0-492c-b571-216abb618710'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': ['--name', '$(MY_POD_NAME)', '--advertise-client-urls', 'http://$(MY_POD_IP):2379,http://$(MY_POD_IP):4001', '--listen-client-urls', 'http://0.0.0.0:2379,http://0.0.0.0:4001', '--initial-advertise-peer-urls', 'http://$(MY_POD_IP):2380', '--listen-peer-urls', 'http://0.0.0.0:2380', '--initial-cluster', '$(MY_POD_NAME)=http://$(MY_POD_IP):2380', '--initial-cluster-state', 'new'], 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}, {'name': 'MY_POD_NAME', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'metadata.name'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': None, 'image': 'quay.io/coreos/etcd:v2.2.5', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 300, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 4001}, 'timeout_seconds': 1}, 'name': 'etcd', 'ports': [{'container_port': 2379, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 2380, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 4001, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 4001}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-qcbq9', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': None, 'node_name': 'kali-worker', 'node_selector': None, 'preemption_policy': 'PreemptLowerPriority', 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-qcbq9', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-qcbq9'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 23, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 23, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://9464a49fcd3bd4bd751473ed8bee239322b9971cedd5caa11c4a6469aa6a0875', 'image': 'quay.io/coreos/etcd:v2.2.5', 'image_id': 'sha256:66590e2dd05422771324d8ffe9ad35165ba35db0842ac5e0f1668c2d56733860', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'etcd', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 9, 29, 12, 52, 17, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.12', 'init_container_statuses': None, 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.2.119', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal())}}, 'raw_object': {'kind': 'Pod', 'apiVersion': 'v1', 'metadata': {'name': 'etcd-744b4d9f98-xsvtv', 'generateName': 'etcd-744b4d9f98-', 'namespace': 'ims-cgn86', 'selfLink': '/api/v1/namespaces/ims-cgn86/pods/etcd-744b4d9f98-xsvtv', 'uid': 'e0b74bf5-24a0-492c-b571-216abb618710', 'resourceVersion': '1641685', 'creationTimestamp': '2020-09-29T12:52:07Z', 'labels': {'instance-type': 'etcd-pod', 'pod-template-hash': '744b4d9f98'}, 'ownerReferences': [{'apiVersion': 'apps/v1', 'kind': 'ReplicaSet', 'name': 'etcd-744b4d9f98', 'uid': 'd2dc3cf6-e1d7-431c-b42c-a3522e20b65a', 'controller': True, 'blockOwnerDeletion': True}]}, 'spec': {'volumes': [{'name': 'default-token-qcbq9', 'secret': {'secretName': 'default-token-qcbq9', 'defaultMode': 420}}], 'containers': [{'name': 'etcd', 'image': 'quay.io/coreos/etcd:v2.2.5', 'args': ['--name', '$(MY_POD_NAME)', '--advertise-client-urls', 'http://$(MY_POD_IP):2379,http://$(MY_POD_IP):4001', '--listen-client-urls', 'http://0.0.0.0:2379,http://0.0.0.0:4001', '--initial-advertise-peer-urls', 'http://$(MY_POD_IP):2380', '--listen-peer-urls', 'http://0.0.0.0:2380', '--initial-cluster', '$(MY_POD_NAME)=http://$(MY_POD_IP):2380', '--initial-cluster-state', 'new'], 'ports': [{'containerPort': 2379, 'protocol': 'TCP'}, {'containerPort': 2380, 'protocol': 'TCP'}, {'containerPort': 4001, 'protocol': 'TCP'}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}, {'name': 'MY_POD_NAME', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'metadata.name'}}}], 'resources': {}, 'volumeMounts': [{'name': 'default-token-qcbq9', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'livenessProbe': {'tcpSocket': {'port': 4001}, 'initialDelaySeconds': 300, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'tcpSocket': {'port': 4001}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'serviceAccountName': 'default', 'serviceAccount': 'default', 'nodeName': 'kali-worker', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler', 'tolerations': [{'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}, {'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}], 'priority': 0, 'enableServiceLinks': True, 'preemptionPolicy': 'PreemptLowerPriority'}, 'status': {'phase': 'Running', 'conditions': [{'type': 'Initialized', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:52:08Z'}, {'type': 'Ready', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:52:23Z'}, {'type': 'ContainersReady', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:52:23Z'}, {'type': 'PodScheduled', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:52:07Z'}], 'hostIP': '172.18.0.12', 'podIP': '10.244.2.119', 'podIPs': [{'ip': '10.244.2.119'}], 'startTime': '2020-09-29T12:52:08Z', 'containerStatuses': [{'name': 'etcd', 'state': {'running': {'startedAt': '2020-09-29T12:52:17Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'quay.io/coreos/etcd:v2.2.5', 'imageID': 'sha256:66590e2dd05422771324d8ffe9ad35165ba35db0842ac5e0f1668c2d56733860', 'containerID': 'containerd://9464a49fcd3bd4bd751473ed8bee239322b9971cedd5caa11c4a6469aa6a0875', 'started': True}], 'qosClass': 'BestEffort'}}} 2020-09-29 12:58:59,699 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 6, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'homer-767485c9dc-', 'generation': None, 'initializers': None, 'labels': {'pod-template-hash': '767485c9dc', 'service': 'homer'}, 'managed_fields': None, 'name': 'homer-767485c9dc-lnd5m', 'namespace': 'ims-cgn86', 'owner_references': [{'api_version': 'apps/v1', 'block_owner_deletion': True, 'controller': True, 'kind': 'ReplicaSet', 'name': 'homer-767485c9dc', 'uid': '4e454cbf-ddac-4c5f-b032-4a58f6b1f90d'}], 'resource_version': '1642418', 'self_link': '/api/v1/namespaces/ims-cgn86/pods/homer-767485c9dc-lnd5m', 'uid': '4ca40c00-b48e-4581-9fd8-79d44701bc80'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-homer:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7888}, 'timeout_seconds': 1}, 'name': 'homer', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 7888, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7888}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-qcbq9', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup cassandra; do echo ' 'waiting for cassandra to start ' '...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-qcbq9', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'node_name': 'kali-worker2', 'node_selector': None, 'preemption_policy': 'PreemptLowerPriority', 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-qcbq9', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-qcbq9'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 54, 17, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 54, 34, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 54, 34, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://97af7bf268fa91fe3b32bedee9b4ea466b4543448d255d749669ebe605bb862d', 'image': 'docker.io/ollivier/clearwater-homer:hunter', 'image_id': 'docker.io/ollivier/clearwater-homer@sha256:63a6cb127f0e57ecd4f7d066ae194be6d8f021e96f4b6022bf9daacee0f1ff5a', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'homer', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 9, 29, 12, 54, 23, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.13', 'init_container_statuses': [{'container_id': 'containerd://a2eac0cf558e31be0b04395183cc2b93e73e2d6e4bab764535d3cdf6edcc5661', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-cassandra', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://a2eac0cf558e31be0b04395183cc2b93e73e2d6e4bab764535d3cdf6edcc5661', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 9, 29, 12, 54, 17, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 9, 29, 12, 52, 20, tzinfo=tzlocal())}, 'waiting': None}}], 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.1.125', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal())}}, 'raw_object': {'kind': 'Pod', 'apiVersion': 'v1', 'metadata': {'name': 'homer-767485c9dc-lnd5m', 'generateName': 'homer-767485c9dc-', 'namespace': 'ims-cgn86', 'selfLink': '/api/v1/namespaces/ims-cgn86/pods/homer-767485c9dc-lnd5m', 'uid': '4ca40c00-b48e-4581-9fd8-79d44701bc80', 'resourceVersion': '1642418', 'creationTimestamp': '2020-09-29T12:52:06Z', 'labels': {'pod-template-hash': '767485c9dc', 'service': 'homer'}, 'ownerReferences': [{'apiVersion': 'apps/v1', 'kind': 'ReplicaSet', 'name': 'homer-767485c9dc', 'uid': '4e454cbf-ddac-4c5f-b032-4a58f6b1f90d', 'controller': True, 'blockOwnerDeletion': True}]}, 'spec': {'volumes': [{'name': 'default-token-qcbq9', 'secret': {'secretName': 'default-token-qcbq9', 'defaultMode': 420}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-qcbq9', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'homer', 'image': 'ollivier/clearwater-homer:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 7888, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'default-token-qcbq9', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'livenessProbe': {'tcpSocket': {'port': 7888}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'tcpSocket': {'port': 7888}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'serviceAccountName': 'default', 'serviceAccount': 'default', 'nodeName': 'kali-worker2', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler', 'tolerations': [{'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}, {'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}], 'priority': 0, 'enableServiceLinks': True, 'preemptionPolicy': 'PreemptLowerPriority'}, 'status': {'phase': 'Running', 'conditions': [{'type': 'Initialized', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:54:17Z'}, {'type': 'Ready', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:54:34Z'}, {'type': 'ContainersReady', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:54:34Z'}, {'type': 'PodScheduled', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:52:07Z'}], 'hostIP': '172.18.0.13', 'podIP': '10.244.1.125', 'podIPs': [{'ip': '10.244.1.125'}], 'startTime': '2020-09-29T12:52:08Z', 'initContainerStatuses': [{'name': 'wait-on-cassandra', 'state': {'terminated': {'exitCode': 0, 'reason': 'Completed', 'startedAt': '2020-09-29T12:52:20Z', 'finishedAt': '2020-09-29T12:54:17Z', 'containerID': 'containerd://a2eac0cf558e31be0b04395183cc2b93e73e2d6e4bab764535d3cdf6edcc5661'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:1.28', 'imageID': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'containerID': 'containerd://a2eac0cf558e31be0b04395183cc2b93e73e2d6e4bab764535d3cdf6edcc5661'}], 'containerStatuses': [{'name': 'homer', 'state': {'running': {'startedAt': '2020-09-29T12:54:23Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/ollivier/clearwater-homer:hunter', 'imageID': 'docker.io/ollivier/clearwater-homer@sha256:63a6cb127f0e57ecd4f7d066ae194be6d8f021e96f4b6022bf9daacee0f1ff5a', 'containerID': 'containerd://97af7bf268fa91fe3b32bedee9b4ea466b4543448d255d749669ebe605bb862d', 'started': True}], 'qosClass': 'BestEffort'}}} 2020-09-29 12:58:59,711 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'cassandra-864f6d8f65-', 'generation': None, 'initializers': None, 'labels': {'pod-template-hash': '864f6d8f65', 'service': 'cassandra'}, 'managed_fields': None, 'name': 'cassandra-864f6d8f65-bfhdb', 'namespace': 'ims-cgn86', 'owner_references': [{'api_version': 'apps/v1', 'block_owner_deletion': True, 'controller': True, 'kind': 'ReplicaSet', 'name': 'cassandra-864f6d8f65', 'uid': 'dbf6fbe0-b9bd-436c-a805-202c03eeb52f'}], 'resource_version': '1642222', 'self_link': '/api/v1/namespaces/ims-cgn86/pods/cassandra-864f6d8f65-bfhdb', 'uid': '590768ea-7018-40d4-819f-8f631db81c13'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-cassandra:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 9042 ' '9160']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 600, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'cassandra', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 7001, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 9042, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 9160, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 9042 ' '9160']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-qcbq9', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup etcd; do echo ' 'waiting for etcd to start ...; ' 'sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-etcd', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-qcbq9', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'node_name': 'kali-worker2', 'node_selector': None, 'preemption_policy': 'PreemptLowerPriority', 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-qcbq9', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-qcbq9'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 25, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 54, 1, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 54, 1, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://75f9bc37bb7b772607b26f432cc899c3e5918ac40de8bd73bfc8720ceef9fa1c', 'image': 'docker.io/ollivier/clearwater-cassandra:hunter', 'image_id': 'docker.io/ollivier/clearwater-cassandra@sha256:e0a6ca94460ea7a08848fca34ac5379e12427b63083a6cd409f4b4b94ccddb57', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'cassandra', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 9, 29, 12, 52, 27, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.13', 'init_container_statuses': [{'container_id': 'containerd://a94c86375c0f7852ed64b1431b6e3f41d474970bb3a1f57cf31653e5816d7573', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-etcd', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://a94c86375c0f7852ed64b1431b6e3f41d474970bb3a1f57cf31653e5816d7573', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 9, 29, 12, 52, 24, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 9, 29, 12, 52, 22, tzinfo=tzlocal())}, 'waiting': None}}], 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.1.128', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 9, 29, 12, 52, 9, tzinfo=tzlocal())}}, 'raw_object': {'kind': 'Pod', 'apiVersion': 'v1', 'metadata': {'name': 'cassandra-864f6d8f65-bfhdb', 'generateName': 'cassandra-864f6d8f65-', 'namespace': 'ims-cgn86', 'selfLink': '/api/v1/namespaces/ims-cgn86/pods/cassandra-864f6d8f65-bfhdb', 'uid': '590768ea-7018-40d4-819f-8f631db81c13', 'resourceVersion': '1642222', 'creationTimestamp': '2020-09-29T12:52:08Z', 'labels': {'pod-template-hash': '864f6d8f65', 'service': 'cassandra'}, 'ownerReferences': [{'apiVersion': 'apps/v1', 'kind': 'ReplicaSet', 'name': 'cassandra-864f6d8f65', 'uid': 'dbf6fbe0-b9bd-436c-a805-202c03eeb52f', 'controller': True, 'blockOwnerDeletion': True}]}, 'spec': {'volumes': [{'name': 'default-token-qcbq9', 'secret': {'secretName': 'default-token-qcbq9', 'defaultMode': 420}}], 'initContainers': [{'name': 'wait-on-etcd', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup etcd; do echo waiting for etcd to start ...; sleep 2; done'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-qcbq9', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'cassandra', 'image': 'ollivier/clearwater-cassandra:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 7001, 'protocol': 'TCP'}, {'containerPort': 9042, 'protocol': 'TCP'}, {'containerPort': 9160, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'default-token-qcbq9', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 9042 9160']}, 'initialDelaySeconds': 600, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 9042 9160']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'serviceAccountName': 'default', 'serviceAccount': 'default', 'nodeName': 'kali-worker2', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler', 'tolerations': [{'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}, {'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}], 'priority': 0, 'enableServiceLinks': True, 'preemptionPolicy': 'PreemptLowerPriority'}, 'status': {'phase': 'Running', 'conditions': [{'type': 'Initialized', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:52:25Z'}, {'type': 'Ready', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:54:01Z'}, {'type': 'ContainersReady', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:54:01Z'}, {'type': 'PodScheduled', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:52:08Z'}], 'hostIP': '172.18.0.13', 'podIP': '10.244.1.128', 'podIPs': [{'ip': '10.244.1.128'}], 'startTime': '2020-09-29T12:52:09Z', 'initContainerStatuses': [{'name': 'wait-on-etcd', 'state': {'terminated': {'exitCode': 0, 'reason': 'Completed', 'startedAt': '2020-09-29T12:52:22Z', 'finishedAt': '2020-09-29T12:52:24Z', 'containerID': 'containerd://a94c86375c0f7852ed64b1431b6e3f41d474970bb3a1f57cf31653e5816d7573'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:1.28', 'imageID': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'containerID': 'containerd://a94c86375c0f7852ed64b1431b6e3f41d474970bb3a1f57cf31653e5816d7573'}], 'containerStatuses': [{'name': 'cassandra', 'state': {'running': {'startedAt': '2020-09-29T12:52:27Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/ollivier/clearwater-cassandra:hunter', 'imageID': 'docker.io/ollivier/clearwater-cassandra@sha256:e0a6ca94460ea7a08848fca34ac5379e12427b63083a6cd409f4b4b94ccddb57', 'containerID': 'containerd://75f9bc37bb7b772607b26f432cc899c3e5918ac40de8bd73bfc8720ceef9fa1c', 'started': True}], 'qosClass': 'BestEffort'}}} 2020-09-29 12:58:59,723 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'ralf-7449977d5c-', 'generation': None, 'initializers': None, 'labels': {'pod-template-hash': '7449977d5c', 'service': 'ralf', 'snmp': 'enabled'}, 'managed_fields': None, 'name': 'ralf-7449977d5c-7wzg9', 'namespace': 'ims-cgn86', 'owner_references': [{'api_version': 'apps/v1', 'block_owner_deletion': True, 'controller': True, 'kind': 'ReplicaSet', 'name': 'ralf-7449977d5c', 'uid': 'ffac4b0d-d48a-4b96-97c9-6b7bb73fd898'}], 'resource_version': '1642441', 'self_link': '/api/v1/namespaces/ims-cgn86/pods/ralf-7449977d5c-7wzg9', 'uid': '57b8f5cd-1138-4779-959b-a268c34474e4'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-ralf:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 10888}, 'timeout_seconds': 1}, 'name': 'ralf', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 10888, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 10888}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/ralf', 'mount_propagation': None, 'name': 'ralflogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-qcbq9', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/ralf/ralf_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/ralf', 'mount_propagation': None, 'name': 'ralflogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-qcbq9', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup cassandra; do echo ' 'waiting for cassandra to start ' '...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-qcbq9', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'node_name': 'kali-worker', 'node_selector': None, 'preemption_policy': 'PreemptLowerPriority', 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'ralflogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}, {'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-qcbq9', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-qcbq9'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 54, 20, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 54, 37, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 54, 37, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://02ca372c4039d0f652f0137538a8ffd234c2743bfe892df946fbf52307ab1f99', 'image': 'docker.io/ollivier/clearwater-ralf:hunter', 'image_id': 'docker.io/ollivier/clearwater-ralf@sha256:5dfc81029d63fae2fba5c896a0ba94abd85c745e66e6efa4fc3b0bfec8c820d7', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'ralf', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 9, 29, 12, 54, 25, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}, {'container_id': 'containerd://0a6e85a83be91f1292dfc868c9e5e41a64cc71853a1331eb4696b61a01d4c8e1', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'tailer', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 9, 29, 12, 54, 28, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.12', 'init_container_statuses': [{'container_id': 'containerd://e1339f408d9f316ee0a1490fe8ea8f4ca03ded03796bdeb4f58a6401d62cfdad', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-cassandra', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://e1339f408d9f316ee0a1490fe8ea8f4ca03ded03796bdeb4f58a6401d62cfdad', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 9, 29, 12, 54, 19, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 9, 29, 12, 52, 18, tzinfo=tzlocal())}, 'waiting': None}}], 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.2.120', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal())}}, 'raw_object': {'kind': 'Pod', 'apiVersion': 'v1', 'metadata': {'name': 'ralf-7449977d5c-7wzg9', 'generateName': 'ralf-7449977d5c-', 'namespace': 'ims-cgn86', 'selfLink': '/api/v1/namespaces/ims-cgn86/pods/ralf-7449977d5c-7wzg9', 'uid': '57b8f5cd-1138-4779-959b-a268c34474e4', 'resourceVersion': '1642441', 'creationTimestamp': '2020-09-29T12:52:07Z', 'labels': {'pod-template-hash': '7449977d5c', 'service': 'ralf', 'snmp': 'enabled'}, 'ownerReferences': [{'apiVersion': 'apps/v1', 'kind': 'ReplicaSet', 'name': 'ralf-7449977d5c', 'uid': 'ffac4b0d-d48a-4b96-97c9-6b7bb73fd898', 'controller': True, 'blockOwnerDeletion': True}]}, 'spec': {'volumes': [{'name': 'ralflogs', 'emptyDir': {}}, {'name': 'default-token-qcbq9', 'secret': {'secretName': 'default-token-qcbq9', 'defaultMode': 420}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-qcbq9', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'ralf', 'image': 'ollivier/clearwater-ralf:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 10888, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'ralflogs', 'mountPath': '/var/log/ralf'}, {'name': 'default-token-qcbq9', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'livenessProbe': {'tcpSocket': {'port': 10888}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'tcpSocket': {'port': 10888}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'tailer', 'image': 'busybox:1.28', 'command': ['tail', '-F', '/var/log/ralf/ralf_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'ralflogs', 'mountPath': '/var/log/ralf'}, {'name': 'default-token-qcbq9', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'serviceAccountName': 'default', 'serviceAccount': 'default', 'nodeName': 'kali-worker', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler', 'tolerations': [{'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}, {'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}], 'priority': 0, 'enableServiceLinks': True, 'preemptionPolicy': 'PreemptLowerPriority'}, 'status': {'phase': 'Running', 'conditions': [{'type': 'Initialized', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:54:20Z'}, {'type': 'Ready', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:54:37Z'}, {'type': 'ContainersReady', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:54:37Z'}, {'type': 'PodScheduled', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:52:07Z'}], 'hostIP': '172.18.0.12', 'podIP': '10.244.2.120', 'podIPs': [{'ip': '10.244.2.120'}], 'startTime': '2020-09-29T12:52:08Z', 'initContainerStatuses': [{'name': 'wait-on-cassandra', 'state': {'terminated': {'exitCode': 0, 'reason': 'Completed', 'startedAt': '2020-09-29T12:52:18Z', 'finishedAt': '2020-09-29T12:54:19Z', 'containerID': 'containerd://e1339f408d9f316ee0a1490fe8ea8f4ca03ded03796bdeb4f58a6401d62cfdad'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:1.28', 'imageID': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'containerID': 'containerd://e1339f408d9f316ee0a1490fe8ea8f4ca03ded03796bdeb4f58a6401d62cfdad'}], 'containerStatuses': [{'name': 'ralf', 'state': {'running': {'startedAt': '2020-09-29T12:54:25Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/ollivier/clearwater-ralf:hunter', 'imageID': 'docker.io/ollivier/clearwater-ralf@sha256:5dfc81029d63fae2fba5c896a0ba94abd85c745e66e6efa4fc3b0bfec8c820d7', 'containerID': 'containerd://02ca372c4039d0f652f0137538a8ffd234c2743bfe892df946fbf52307ab1f99', 'started': True}, {'name': 'tailer', 'state': {'running': {'startedAt': '2020-09-29T12:54:28Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:1.28', 'imageID': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'containerID': 'containerd://0a6e85a83be91f1292dfc868c9e5e41a64cc71853a1331eb4696b61a01d4c8e1', 'started': True}], 'qosClass': 'BestEffort'}}} 2020-09-29 12:58:59,739 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 6, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'sprout-768bcbd967-', 'generation': None, 'initializers': None, 'labels': {'pod-template-hash': '768bcbd967', 'service': 'sprout', 'snmp': 'enabled'}, 'managed_fields': None, 'name': 'sprout-768bcbd967-mgkf9', 'namespace': 'ims-cgn86', 'owner_references': [{'api_version': 'apps/v1', 'block_owner_deletion': True, 'controller': True, 'kind': 'ReplicaSet', 'name': 'sprout-768bcbd967', 'uid': '26335e3e-fb3e-433d-b982-b9dbc88774d2'}], 'resource_version': '1642884', 'self_link': '/api/v1/namespaces/ims-cgn86/pods/sprout-768bcbd967-mgkf9', 'uid': '1fd593ff-30a5-4c84-9ee1-e4334fd71997'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-sprout:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 ' '5054']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 3, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'sprout', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 ' '5054']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/sprout', 'mount_propagation': None, 'name': 'sproutlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-qcbq9', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/sprout/sprout_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/sprout', 'mount_propagation': None, 'name': 'sproutlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-qcbq9', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup ralf; do echo ' 'waiting for ralf to start ...; ' 'sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-ralf', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-qcbq9', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until nslookup chronos; do echo ' 'waiting for chronos to start ...; ' 'sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-chronos', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-qcbq9', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until nslookup homestead; do echo ' 'waiting for homestead to start ' '...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-homestead', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-qcbq9', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'node_name': 'kali-worker', 'node_selector': None, 'preemption_policy': 'PreemptLowerPriority', 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'sproutlogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}, {'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-qcbq9', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-qcbq9'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 55, 58, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 56, 10, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 56, 10, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://3af62a353aa34d535c2c1670882c7b660c7af6bf56d72c535d121a37dbc5797b', 'image': 'docker.io/ollivier/clearwater-sprout:hunter', 'image_id': 'docker.io/ollivier/clearwater-sprout@sha256:dbc1c137720484bc549a753db56cc8b652e44787827364609510b9a73879d104', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'sprout', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 9, 29, 12, 56, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}, {'container_id': 'containerd://92a6cf866e4a2e7d40794d4237b9e57c45205174b2134a2f87645fe032ceeea8', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'tailer', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 9, 29, 12, 56, 3, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.12', 'init_container_statuses': [{'container_id': 'containerd://22675d0e60cb9f331cafd9e366b03148acc33adabdb77ce8d7e8dfe226e9325d', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-ralf', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://22675d0e60cb9f331cafd9e366b03148acc33adabdb77ce8d7e8dfe226e9325d', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 9, 29, 12, 54, 56, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 9, 29, 12, 52, 15, tzinfo=tzlocal())}, 'waiting': None}}, {'container_id': 'containerd://d79ca835293fa66734e3713e4f4da7406003c5722e323cccb72710de291939dc', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-chronos', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://d79ca835293fa66734e3713e4f4da7406003c5722e323cccb72710de291939dc', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 9, 29, 12, 54, 59, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 9, 29, 12, 54, 59, tzinfo=tzlocal())}, 'waiting': None}}, {'container_id': 'containerd://e34c4aeb962e702ffb50a47817be98555f9f5366e281dad8d794b94899a4d0b5', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-homestead', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://e34c4aeb962e702ffb50a47817be98555f9f5366e281dad8d794b94899a4d0b5', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 9, 29, 12, 55, 56, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 9, 29, 12, 55, 4, tzinfo=tzlocal())}, 'waiting': None}}], 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.2.118', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal())}}, 'raw_object': {'kind': 'Pod', 'apiVersion': 'v1', 'metadata': {'name': 'sprout-768bcbd967-mgkf9', 'generateName': 'sprout-768bcbd967-', 'namespace': 'ims-cgn86', 'selfLink': '/api/v1/namespaces/ims-cgn86/pods/sprout-768bcbd967-mgkf9', 'uid': '1fd593ff-30a5-4c84-9ee1-e4334fd71997', 'resourceVersion': '1642884', 'creationTimestamp': '2020-09-29T12:52:06Z', 'labels': {'pod-template-hash': '768bcbd967', 'service': 'sprout', 'snmp': 'enabled'}, 'ownerReferences': [{'apiVersion': 'apps/v1', 'kind': 'ReplicaSet', 'name': 'sprout-768bcbd967', 'uid': '26335e3e-fb3e-433d-b982-b9dbc88774d2', 'controller': True, 'blockOwnerDeletion': True}]}, 'spec': {'volumes': [{'name': 'sproutlogs', 'emptyDir': {}}, {'name': 'default-token-qcbq9', 'secret': {'secretName': 'default-token-qcbq9', 'defaultMode': 420}}], 'initContainers': [{'name': 'wait-on-ralf', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup ralf; do echo waiting for ralf to start ...; sleep 2; done'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-qcbq9', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'wait-on-chronos', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup chronos; do echo waiting for chronos to start ...; sleep 2; done'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-qcbq9', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'wait-on-homestead', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup homestead; do echo waiting for homestead to start ...; sleep 2; done'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-qcbq9', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'sprout', 'image': 'ollivier/clearwater-sprout:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'sproutlogs', 'mountPath': '/var/log/sprout'}, {'name': 'default-token-qcbq9', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 5054']}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 3, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 5054']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'tailer', 'image': 'busybox:1.28', 'command': ['tail', '-F', '/var/log/sprout/sprout_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'sproutlogs', 'mountPath': '/var/log/sprout'}, {'name': 'default-token-qcbq9', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'serviceAccountName': 'default', 'serviceAccount': 'default', 'nodeName': 'kali-worker', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler', 'tolerations': [{'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}, {'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}], 'priority': 0, 'enableServiceLinks': True, 'preemptionPolicy': 'PreemptLowerPriority'}, 'status': {'phase': 'Running', 'conditions': [{'type': 'Initialized', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:55:58Z'}, {'type': 'Ready', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:56:10Z'}, {'type': 'ContainersReady', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:56:10Z'}, {'type': 'PodScheduled', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:52:07Z'}], 'hostIP': '172.18.0.12', 'podIP': '10.244.2.118', 'podIPs': [{'ip': '10.244.2.118'}], 'startTime': '2020-09-29T12:52:08Z', 'initContainerStatuses': [{'name': 'wait-on-ralf', 'state': {'terminated': {'exitCode': 0, 'reason': 'Completed', 'startedAt': '2020-09-29T12:52:15Z', 'finishedAt': '2020-09-29T12:54:56Z', 'containerID': 'containerd://22675d0e60cb9f331cafd9e366b03148acc33adabdb77ce8d7e8dfe226e9325d'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:1.28', 'imageID': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'containerID': 'containerd://22675d0e60cb9f331cafd9e366b03148acc33adabdb77ce8d7e8dfe226e9325d'}, {'name': 'wait-on-chronos', 'state': {'terminated': {'exitCode': 0, 'reason': 'Completed', 'startedAt': '2020-09-29T12:54:59Z', 'finishedAt': '2020-09-29T12:54:59Z', 'containerID': 'containerd://d79ca835293fa66734e3713e4f4da7406003c5722e323cccb72710de291939dc'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:1.28', 'imageID': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'containerID': 'containerd://d79ca835293fa66734e3713e4f4da7406003c5722e323cccb72710de291939dc'}, {'name': 'wait-on-homestead', 'state': {'terminated': {'exitCode': 0, 'reason': 'Completed', 'startedAt': '2020-09-29T12:55:04Z', 'finishedAt': '2020-09-29T12:55:56Z', 'containerID': 'containerd://e34c4aeb962e702ffb50a47817be98555f9f5366e281dad8d794b94899a4d0b5'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:1.28', 'imageID': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'containerID': 'containerd://e34c4aeb962e702ffb50a47817be98555f9f5366e281dad8d794b94899a4d0b5'}], 'containerStatuses': [{'name': 'sprout', 'state': {'running': {'startedAt': '2020-09-29T12:56:00Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/ollivier/clearwater-sprout:hunter', 'imageID': 'docker.io/ollivier/clearwater-sprout@sha256:dbc1c137720484bc549a753db56cc8b652e44787827364609510b9a73879d104', 'containerID': 'containerd://3af62a353aa34d535c2c1670882c7b660c7af6bf56d72c535d121a37dbc5797b', 'started': True}, {'name': 'tailer', 'state': {'running': {'startedAt': '2020-09-29T12:56:03Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:1.28', 'imageID': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'containerID': 'containerd://92a6cf866e4a2e7d40794d4237b9e57c45205174b2134a2f87645fe032ceeea8', 'started': True}], 'qosClass': 'BestEffort'}}} 2020-09-29 12:58:59,755 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 58, 59, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields': None, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2020, 9, 29, 12, 58, 59, tzinfo=tzlocal())}], 'name': 'live-test', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1643417', 'self_link': '/api/v1/namespaces/ims-cgn86/pods/live-test', 'uid': 'bebe78f5-b78c-4b02-8673-4230b3f7569f'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': ['rake', 'test[ims-cgn86.svc.cluster.local]', 'PROXY=bono.ims-cgn86.svc.cluster.local', 'ELLIS=ellis.ims-cgn86.svc.cluster.local', 'SIGNUP_CODE=secret', '--trace'], 'env': None, 'env_from': None, 'image': 'ollivier/clearwater-live-test:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'live-test', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-qcbq9', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': None, 'init_containers': None, 'node_name': 'kali-worker', 'node_selector': None, 'preemption_policy': 'PreemptLowerPriority', 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Never', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-qcbq9', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-qcbq9'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 58, 59, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': None, 'host_ip': None, 'init_container_statuses': None, 'message': None, 'nominated_node_name': None, 'phase': 'Pending', 'pod_ip': None, 'qos_class': 'BestEffort', 'reason': None, 'start_time': None}}, 'raw_object': {'kind': 'Pod', 'apiVersion': 'v1', 'metadata': {'name': 'live-test', 'namespace': 'ims-cgn86', 'selfLink': '/api/v1/namespaces/ims-cgn86/pods/live-test', 'uid': 'bebe78f5-b78c-4b02-8673-4230b3f7569f', 'resourceVersion': '1643417', 'creationTimestamp': '2020-09-29T12:58:59Z', 'managedFields': [{'manager': 'OpenAPI-Generator', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2020-09-29T12:58:59Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:spec': {'f:containers': {'k:{"name":"live-test"}': {'.': {}, 'f:command': {}, 'f:image': {}, 'f:imagePullPolicy': {}, 'f:name': {}, 'f:resources': {}, 'f:terminationMessagePath': {}, 'f:terminationMessagePolicy': {}}}, 'f:dnsPolicy': {}, 'f:enableServiceLinks': {}, 'f:restartPolicy': {}, 'f:schedulerName': {}, 'f:securityContext': {}, 'f:terminationGracePeriodSeconds': {}}}}]}, 'spec': {'volumes': [{'name': 'default-token-qcbq9', 'secret': {'secretName': 'default-token-qcbq9', 'defaultMode': 420}}], 'containers': [{'name': 'live-test', 'image': 'ollivier/clearwater-live-test:hunter', 'command': ['rake', 'test[ims-cgn86.svc.cluster.local]', 'PROXY=bono.ims-cgn86.svc.cluster.local', 'ELLIS=ellis.ims-cgn86.svc.cluster.local', 'SIGNUP_CODE=secret', '--trace'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-qcbq9', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Never', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'serviceAccountName': 'default', 'serviceAccount': 'default', 'nodeName': 'kali-worker', 'securityContext': {}, 'schedulerName': 'default-scheduler', 'tolerations': [{'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}, {'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}], 'priority': 0, 'enableServiceLinks': True, 'preemptionPolicy': 'PreemptLowerPriority'}, 'status': {'phase': 'Pending', 'conditions': [{'type': 'PodScheduled', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:58:59Z'}], 'qosClass': 'BestEffort'}}} 2020-09-29 12:58:59,762 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 6, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'homestead-prov-d67ff8f9b-', 'generation': None, 'initializers': None, 'labels': {'pod-template-hash': 'd67ff8f9b', 'service': 'homestead-prov', 'snmp': 'enabled'}, 'managed_fields': None, 'name': 'homestead-prov-d67ff8f9b-cjfxl', 'namespace': 'ims-cgn86', 'owner_references': [{'api_version': 'apps/v1', 'block_owner_deletion': True, 'controller': True, 'kind': 'ReplicaSet', 'name': 'homestead-prov-d67ff8f9b', 'uid': 'c5091130-93ef-47c9-bfc2-b116d04a8568'}], 'resource_version': '1642903', 'self_link': '/api/v1/namespaces/ims-cgn86/pods/homestead-prov-d67ff8f9b-cjfxl', 'uid': '7798f14c-ffea-4eb5-8f8e-5bbe90e05f7a'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-homestead-prov:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 60, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'homestead-prov', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 8889, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-qcbq9', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup homestead; do echo ' 'waiting for homestead to start ' '...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-homestead', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-qcbq9', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'node_name': 'kali-worker', 'node_selector': None, 'preemption_policy': 'PreemptLowerPriority', 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-qcbq9', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-qcbq9'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 55, 59, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 56, 15, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 56, 15, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://dbef4bf784747def59f34d10477e6b52d38e7b57ed8c5e8c052ce129e505fb94', 'image': 'docker.io/ollivier/clearwater-homestead-prov:hunter', 'image_id': 'docker.io/ollivier/clearwater-homestead-prov@sha256:6e369295193fa6e5326c41cd2d4f4bdda75c5cb829bd3f7ee899554a48a7b323', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'homestead-prov', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 9, 29, 12, 56, 3, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.12', 'init_container_statuses': [{'container_id': 'containerd://8e96a38e55b80832445846e8fd0b96c0aa4baad38911b52fbb8586648544a54b', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-homestead', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://8e96a38e55b80832445846e8fd0b96c0aa4baad38911b52fbb8586648544a54b', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 9, 29, 12, 55, 58, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 9, 29, 12, 52, 13, tzinfo=tzlocal())}, 'waiting': None}}], 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.2.117', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal())}}, 'raw_object': {'kind': 'Pod', 'apiVersion': 'v1', 'metadata': {'name': 'homestead-prov-d67ff8f9b-cjfxl', 'generateName': 'homestead-prov-d67ff8f9b-', 'namespace': 'ims-cgn86', 'selfLink': '/api/v1/namespaces/ims-cgn86/pods/homestead-prov-d67ff8f9b-cjfxl', 'uid': '7798f14c-ffea-4eb5-8f8e-5bbe90e05f7a', 'resourceVersion': '1642903', 'creationTimestamp': '2020-09-29T12:52:06Z', 'labels': {'pod-template-hash': 'd67ff8f9b', 'service': 'homestead-prov', 'snmp': 'enabled'}, 'ownerReferences': [{'apiVersion': 'apps/v1', 'kind': 'ReplicaSet', 'name': 'homestead-prov-d67ff8f9b', 'uid': 'c5091130-93ef-47c9-bfc2-b116d04a8568', 'controller': True, 'blockOwnerDeletion': True}]}, 'spec': {'volumes': [{'name': 'default-token-qcbq9', 'secret': {'secretName': 'default-token-qcbq9', 'defaultMode': 420}}], 'initContainers': [{'name': 'wait-on-homestead', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup homestead; do echo waiting for homestead to start ...; sleep 2; done'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-qcbq9', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'homestead-prov', 'image': 'ollivier/clearwater-homestead-prov:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 8889, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'default-token-qcbq9', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'initialDelaySeconds': 60, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'serviceAccountName': 'default', 'serviceAccount': 'default', 'nodeName': 'kali-worker', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler', 'tolerations': [{'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}, {'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}], 'priority': 0, 'enableServiceLinks': True, 'preemptionPolicy': 'PreemptLowerPriority'}, 'status': {'phase': 'Running', 'conditions': [{'type': 'Initialized', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:55:59Z'}, {'type': 'Ready', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:56:15Z'}, {'type': 'ContainersReady', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:56:15Z'}, {'type': 'PodScheduled', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:52:07Z'}], 'hostIP': '172.18.0.12', 'podIP': '10.244.2.117', 'podIPs': [{'ip': '10.244.2.117'}], 'startTime': '2020-09-29T12:52:08Z', 'initContainerStatuses': [{'name': 'wait-on-homestead', 'state': {'terminated': {'exitCode': 0, 'reason': 'Completed', 'startedAt': '2020-09-29T12:52:13Z', 'finishedAt': '2020-09-29T12:55:58Z', 'containerID': 'containerd://8e96a38e55b80832445846e8fd0b96c0aa4baad38911b52fbb8586648544a54b'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:1.28', 'imageID': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'containerID': 'containerd://8e96a38e55b80832445846e8fd0b96c0aa4baad38911b52fbb8586648544a54b'}], 'containerStatuses': [{'name': 'homestead-prov', 'state': {'running': {'startedAt': '2020-09-29T12:56:03Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/ollivier/clearwater-homestead-prov:hunter', 'imageID': 'docker.io/ollivier/clearwater-homestead-prov@sha256:6e369295193fa6e5326c41cd2d4f4bdda75c5cb829bd3f7ee899554a48a7b323', 'containerID': 'containerd://dbef4bf784747def59f34d10477e6b52d38e7b57ed8c5e8c052ce129e505fb94', 'started': True}], 'qosClass': 'BestEffort'}}} 2020-09-29 12:58:59,774 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 6, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'ellis-54c6475b79-', 'generation': None, 'initializers': None, 'labels': {'pod-template-hash': '54c6475b79', 'service': 'ellis'}, 'managed_fields': None, 'name': 'ellis-54c6475b79-l56jb', 'namespace': 'ims-cgn86', 'owner_references': [{'api_version': 'apps/v1', 'block_owner_deletion': True, 'controller': True, 'kind': 'ReplicaSet', 'name': 'ellis-54c6475b79', 'uid': '8f4dacbe-0cf0-4806-a105-08a56f763615'}], 'resource_version': '1643058', 'self_link': '/api/v1/namespaces/ims-cgn86/pods/ellis-54c6475b79-l56jb', 'uid': '09eab168-aecf-460c-bec2-097933d7e7d9'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-ellis:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 80}, 'timeout_seconds': 1}, 'name': 'ellis', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 80, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 80}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-qcbq9', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup homestead-prov; do ' 'echo waiting for homestead-prov to ' 'start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-homestead-prov', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-qcbq9', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until nslookup sprout; do echo ' 'waiting for sprout to start ...; ' 'sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-sprout', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-qcbq9', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'node_name': 'kali-worker2', 'node_selector': None, 'preemption_policy': 'PreemptLowerPriority', 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-qcbq9', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-qcbq9'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 56, 42, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 56, 59, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 56, 59, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://de55a9e0f1538f9562a484cff44f3ce96a9d876f6df93e3ec3415d532d13412e', 'image': 'docker.io/ollivier/clearwater-ellis:hunter', 'image_id': 'docker.io/ollivier/clearwater-ellis@sha256:177c949e01031a68e56315b8b70974e0fdd9bedece200dbcd68c9a0859bca477', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'ellis', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 9, 29, 12, 56, 44, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.13', 'init_container_statuses': [{'container_id': 'containerd://6ab748f12330c7c0e02ebdd5f9e27c3fd55881185e93585a877bf73e085e7a56', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-homestead-prov', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://6ab748f12330c7c0e02ebdd5f9e27c3fd55881185e93585a877bf73e085e7a56', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 9, 29, 12, 56, 38, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 9, 29, 12, 52, 19, tzinfo=tzlocal())}, 'waiting': None}}, {'container_id': 'containerd://20167568124a44e3c7ac53c759cf02e4025a33760923460daecc124a93aa54ad', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-sprout', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://20167568124a44e3c7ac53c759cf02e4025a33760923460daecc124a93aa54ad', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 9, 29, 12, 56, 42, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 9, 29, 12, 56, 42, tzinfo=tzlocal())}, 'waiting': None}}], 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.1.124', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal())}}, 'raw_object': {'kind': 'Pod', 'apiVersion': 'v1', 'metadata': {'name': 'ellis-54c6475b79-l56jb', 'generateName': 'ellis-54c6475b79-', 'namespace': 'ims-cgn86', 'selfLink': '/api/v1/namespaces/ims-cgn86/pods/ellis-54c6475b79-l56jb', 'uid': '09eab168-aecf-460c-bec2-097933d7e7d9', 'resourceVersion': '1643058', 'creationTimestamp': '2020-09-29T12:52:06Z', 'labels': {'pod-template-hash': '54c6475b79', 'service': 'ellis'}, 'ownerReferences': [{'apiVersion': 'apps/v1', 'kind': 'ReplicaSet', 'name': 'ellis-54c6475b79', 'uid': '8f4dacbe-0cf0-4806-a105-08a56f763615', 'controller': True, 'blockOwnerDeletion': True}]}, 'spec': {'volumes': [{'name': 'default-token-qcbq9', 'secret': {'secretName': 'default-token-qcbq9', 'defaultMode': 420}}], 'initContainers': [{'name': 'wait-on-homestead-prov', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup homestead-prov; do echo waiting for homestead-prov to start ...; sleep 2; done'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-qcbq9', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'wait-on-sprout', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup sprout; do echo waiting for sprout to start ...; sleep 2; done'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-qcbq9', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'ellis', 'image': 'ollivier/clearwater-ellis:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 80, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'default-token-qcbq9', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'livenessProbe': {'tcpSocket': {'port': 80}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'tcpSocket': {'port': 80}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'serviceAccountName': 'default', 'serviceAccount': 'default', 'nodeName': 'kali-worker2', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler', 'tolerations': [{'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}, {'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}], 'priority': 0, 'enableServiceLinks': True, 'preemptionPolicy': 'PreemptLowerPriority'}, 'status': {'phase': 'Running', 'conditions': [{'type': 'Initialized', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:56:42Z'}, {'type': 'Ready', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:56:59Z'}, {'type': 'ContainersReady', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:56:59Z'}, {'type': 'PodScheduled', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:52:07Z'}], 'hostIP': '172.18.0.13', 'podIP': '10.244.1.124', 'podIPs': [{'ip': '10.244.1.124'}], 'startTime': '2020-09-29T12:52:08Z', 'initContainerStatuses': [{'name': 'wait-on-homestead-prov', 'state': {'terminated': {'exitCode': 0, 'reason': 'Completed', 'startedAt': '2020-09-29T12:52:19Z', 'finishedAt': '2020-09-29T12:56:38Z', 'containerID': 'containerd://6ab748f12330c7c0e02ebdd5f9e27c3fd55881185e93585a877bf73e085e7a56'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:1.28', 'imageID': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'containerID': 'containerd://6ab748f12330c7c0e02ebdd5f9e27c3fd55881185e93585a877bf73e085e7a56'}, {'name': 'wait-on-sprout', 'state': {'terminated': {'exitCode': 0, 'reason': 'Completed', 'startedAt': '2020-09-29T12:56:42Z', 'finishedAt': '2020-09-29T12:56:42Z', 'containerID': 'containerd://20167568124a44e3c7ac53c759cf02e4025a33760923460daecc124a93aa54ad'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:1.28', 'imageID': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'containerID': 'containerd://20167568124a44e3c7ac53c759cf02e4025a33760923460daecc124a93aa54ad'}], 'containerStatuses': [{'name': 'ellis', 'state': {'running': {'startedAt': '2020-09-29T12:56:44Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/ollivier/clearwater-ellis:hunter', 'imageID': 'docker.io/ollivier/clearwater-ellis@sha256:177c949e01031a68e56315b8b70974e0fdd9bedece200dbcd68c9a0859bca477', 'containerID': 'containerd://de55a9e0f1538f9562a484cff44f3ce96a9d876f6df93e3ec3415d532d13412e', 'started': True}], 'qosClass': 'BestEffort'}}} 2020-09-29 12:58:59,788 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'chronos-6dff6b5ff8-', 'generation': None, 'initializers': None, 'labels': {'pod-template-hash': '6dff6b5ff8', 'service': 'chronos'}, 'managed_fields': None, 'name': 'chronos-6dff6b5ff8-bkzmh', 'namespace': 'ims-cgn86', 'owner_references': [{'api_version': 'apps/v1', 'block_owner_deletion': True, 'controller': True, 'kind': 'ReplicaSet', 'name': 'chronos-6dff6b5ff8', 'uid': '10d93816-7342-4f33-bf4a-b43508b8d756'}], 'resource_version': '1642452', 'self_link': '/api/v1/namespaces/ims-cgn86/pods/chronos-6dff6b5ff8-bkzmh', 'uid': '53e49d79-4a5e-44f3-829b-910ce7a37cc8'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-chronos:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': {'post_start': None, 'pre_stop': {'_exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}, 'http_get': None, 'tcp_socket': None}}, 'liveness_probe': {'_exec': None, 'failure_threshold': 9, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7253}, 'timeout_seconds': 1}, 'name': 'chronos', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 7253, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7253}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/chronos', 'mount_propagation': None, 'name': 'chronoslogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-qcbq9', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/chronos/chronos_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/chronos', 'mount_propagation': None, 'name': 'chronoslogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-qcbq9', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup cassandra; do echo ' 'waiting for cassandra to start ' '...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-qcbq9', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'node_name': 'kali-worker2', 'node_selector': None, 'preemption_policy': 'PreemptLowerPriority', 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 120, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'chronoslogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}, {'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-qcbq9', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-qcbq9'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 54, 20, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 54, 39, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 54, 39, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 52, 7, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://791947f9ce6d1a9c252da133e48c409c072a96b2e1ef420cf02ece1693fcf28d', 'image': 'docker.io/ollivier/clearwater-chronos:hunter', 'image_id': 'docker.io/ollivier/clearwater-chronos@sha256:b279a65dd77e19f7b33f1af41ab2938c35d3d05f232e525231dfce95183d589f', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'chronos', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 9, 29, 12, 54, 25, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}, {'container_id': 'containerd://fe67c24e8b363c402f911d5fd2c9a24d1da677f374ca58be2617ca49791392a6', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'tailer', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 9, 29, 12, 54, 29, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.13', 'init_container_statuses': [{'container_id': 'containerd://9ca45750f90762295131d65daebc2fa5a414bae9dc23c6ec4294b6edafafba1a', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-cassandra', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://9ca45750f90762295131d65daebc2fa5a414bae9dc23c6ec4294b6edafafba1a', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 9, 29, 12, 54, 17, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 9, 29, 12, 52, 21, tzinfo=tzlocal())}, 'waiting': None}}], 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.1.126', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 9, 29, 12, 52, 8, tzinfo=tzlocal())}}, 'raw_object': {'kind': 'Pod', 'apiVersion': 'v1', 'metadata': {'name': 'chronos-6dff6b5ff8-bkzmh', 'generateName': 'chronos-6dff6b5ff8-', 'namespace': 'ims-cgn86', 'selfLink': '/api/v1/namespaces/ims-cgn86/pods/chronos-6dff6b5ff8-bkzmh', 'uid': '53e49d79-4a5e-44f3-829b-910ce7a37cc8', 'resourceVersion': '1642452', 'creationTimestamp': '2020-09-29T12:52:07Z', 'labels': {'pod-template-hash': '6dff6b5ff8', 'service': 'chronos'}, 'ownerReferences': [{'apiVersion': 'apps/v1', 'kind': 'ReplicaSet', 'name': 'chronos-6dff6b5ff8', 'uid': '10d93816-7342-4f33-bf4a-b43508b8d756', 'controller': True, 'blockOwnerDeletion': True}]}, 'spec': {'volumes': [{'name': 'chronoslogs', 'emptyDir': {}}, {'name': 'default-token-qcbq9', 'secret': {'secretName': 'default-token-qcbq9', 'defaultMode': 420}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-qcbq9', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'chronos', 'image': 'ollivier/clearwater-chronos:hunter', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 7253, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'chronoslogs', 'mountPath': '/var/log/chronos'}, {'name': 'default-token-qcbq9', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'livenessProbe': {'tcpSocket': {'port': 7253}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 9}, 'readinessProbe': {'tcpSocket': {'port': 7253}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'lifecycle': {'preStop': {'exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}}}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'tailer', 'image': 'busybox:1.28', 'command': ['tail', '-F', '/var/log/chronos/chronos_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'chronoslogs', 'mountPath': '/var/log/chronos'}, {'name': 'default-token-qcbq9', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 120, 'dnsPolicy': 'ClusterFirst', 'serviceAccountName': 'default', 'serviceAccount': 'default', 'nodeName': 'kali-worker2', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler', 'tolerations': [{'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}, {'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}], 'priority': 0, 'enableServiceLinks': True, 'preemptionPolicy': 'PreemptLowerPriority'}, 'status': {'phase': 'Running', 'conditions': [{'type': 'Initialized', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:54:20Z'}, {'type': 'Ready', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:54:39Z'}, {'type': 'ContainersReady', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:54:39Z'}, {'type': 'PodScheduled', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:52:07Z'}], 'hostIP': '172.18.0.13', 'podIP': '10.244.1.126', 'podIPs': [{'ip': '10.244.1.126'}], 'startTime': '2020-09-29T12:52:08Z', 'initContainerStatuses': [{'name': 'wait-on-cassandra', 'state': {'terminated': {'exitCode': 0, 'reason': 'Completed', 'startedAt': '2020-09-29T12:52:21Z', 'finishedAt': '2020-09-29T12:54:17Z', 'containerID': 'containerd://9ca45750f90762295131d65daebc2fa5a414bae9dc23c6ec4294b6edafafba1a'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:1.28', 'imageID': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'containerID': 'containerd://9ca45750f90762295131d65daebc2fa5a414bae9dc23c6ec4294b6edafafba1a'}], 'containerStatuses': [{'name': 'chronos', 'state': {'running': {'startedAt': '2020-09-29T12:54:25Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/ollivier/clearwater-chronos:hunter', 'imageID': 'docker.io/ollivier/clearwater-chronos@sha256:b279a65dd77e19f7b33f1af41ab2938c35d3d05f232e525231dfce95183d589f', 'containerID': 'containerd://791947f9ce6d1a9c252da133e48c409c072a96b2e1ef420cf02ece1693fcf28d', 'started': True}, {'name': 'tailer', 'state': {'running': {'startedAt': '2020-09-29T12:54:29Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:1.28', 'imageID': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'containerID': 'containerd://fe67c24e8b363c402f911d5fd2c9a24d1da677f374ca58be2617ca49791392a6', 'started': True}], 'qosClass': 'BestEffort'}}} 2020-09-29 12:58:59,802 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 58, 59, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields': None, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2020, 9, 29, 12, 58, 59, tzinfo=tzlocal())}, {'api_version': 'v1', 'fields': None, 'manager': 'kubelet', 'operation': 'Update', 'time': datetime.datetime(2020, 9, 29, 12, 58, 59, tzinfo=tzlocal())}], 'name': 'live-test', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1643419', 'self_link': '/api/v1/namespaces/ims-cgn86/pods/live-test', 'uid': 'bebe78f5-b78c-4b02-8673-4230b3f7569f'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': ['rake', 'test[ims-cgn86.svc.cluster.local]', 'PROXY=bono.ims-cgn86.svc.cluster.local', 'ELLIS=ellis.ims-cgn86.svc.cluster.local', 'SIGNUP_CODE=secret', '--trace'], 'env': None, 'env_from': None, 'image': 'ollivier/clearwater-live-test:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'live-test', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-qcbq9', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': None, 'init_containers': None, 'node_name': 'kali-worker', 'node_selector': None, 'preemption_policy': 'PreemptLowerPriority', 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Never', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-qcbq9', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-qcbq9'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 58, 59, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 58, 59, tzinfo=tzlocal()), 'message': 'containers with unready status: ' '[live-test]', 'reason': 'ContainersNotReady', 'status': 'False', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 58, 59, tzinfo=tzlocal()), 'message': 'containers with unready status: ' '[live-test]', 'reason': 'ContainersNotReady', 'status': 'False', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 58, 59, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': None, 'image': 'ollivier/clearwater-live-test:hunter', 'image_id': '', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'live-test', 'ready': False, 'restart_count': 0, 'state': {'running': None, 'terminated': None, 'waiting': {'message': None, 'reason': 'ContainerCreating'}}}], 'host_ip': '172.18.0.12', 'init_container_statuses': None, 'message': None, 'nominated_node_name': None, 'phase': 'Pending', 'pod_ip': None, 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 9, 29, 12, 58, 59, tzinfo=tzlocal())}}, 'raw_object': {'kind': 'Pod', 'apiVersion': 'v1', 'metadata': {'name': 'live-test', 'namespace': 'ims-cgn86', 'selfLink': '/api/v1/namespaces/ims-cgn86/pods/live-test', 'uid': 'bebe78f5-b78c-4b02-8673-4230b3f7569f', 'resourceVersion': '1643419', 'creationTimestamp': '2020-09-29T12:58:59Z', 'managedFields': [{'manager': 'OpenAPI-Generator', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2020-09-29T12:58:59Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:spec': {'f:containers': {'k:{"name":"live-test"}': {'.': {}, 'f:command': {}, 'f:image': {}, 'f:imagePullPolicy': {}, 'f:name': {}, 'f:resources': {}, 'f:terminationMessagePath': {}, 'f:terminationMessagePolicy': {}}}, 'f:dnsPolicy': {}, 'f:enableServiceLinks': {}, 'f:restartPolicy': {}, 'f:schedulerName': {}, 'f:securityContext': {}, 'f:terminationGracePeriodSeconds': {}}}}, {'manager': 'kubelet', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2020-09-29T12:58:59Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:status': {'f:conditions': {'k:{"type":"ContainersReady"}': {'.': {}, 'f:lastProbeTime': {}, 'f:lastTransitionTime': {}, 'f:message': {}, 'f:reason': {}, 'f:status': {}, 'f:type': {}}, 'k:{"type":"Initialized"}': {'.': {}, 'f:lastProbeTime': {}, 'f:lastTransitionTime': {}, 'f:status': {}, 'f:type': {}}, 'k:{"type":"Ready"}': {'.': {}, 'f:lastProbeTime': {}, 'f:lastTransitionTime': {}, 'f:message': {}, 'f:reason': {}, 'f:status': {}, 'f:type': {}}}, 'f:containerStatuses': {}, 'f:hostIP': {}, 'f:startTime': {}}}}]}, 'spec': {'volumes': [{'name': 'default-token-qcbq9', 'secret': {'secretName': 'default-token-qcbq9', 'defaultMode': 420}}], 'containers': [{'name': 'live-test', 'image': 'ollivier/clearwater-live-test:hunter', 'command': ['rake', 'test[ims-cgn86.svc.cluster.local]', 'PROXY=bono.ims-cgn86.svc.cluster.local', 'ELLIS=ellis.ims-cgn86.svc.cluster.local', 'SIGNUP_CODE=secret', '--trace'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-qcbq9', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Never', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'serviceAccountName': 'default', 'serviceAccount': 'default', 'nodeName': 'kali-worker', 'securityContext': {}, 'schedulerName': 'default-scheduler', 'tolerations': [{'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}, {'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}], 'priority': 0, 'enableServiceLinks': True, 'preemptionPolicy': 'PreemptLowerPriority'}, 'status': {'phase': 'Pending', 'conditions': [{'type': 'Initialized', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:58:59Z'}, {'type': 'Ready', 'status': 'False', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:58:59Z', 'reason': 'ContainersNotReady', 'message': 'containers with unready status: [live-test]'}, {'type': 'ContainersReady', 'status': 'False', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:58:59Z', 'reason': 'ContainersNotReady', 'message': 'containers with unready status: [live-test]'}, {'type': 'PodScheduled', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:58:59Z'}], 'hostIP': '172.18.0.12', 'startTime': '2020-09-29T12:58:59Z', 'containerStatuses': [{'name': 'live-test', 'state': {'waiting': {'reason': 'ContainerCreating'}}, 'lastState': {}, 'ready': False, 'restartCount': 0, 'image': 'ollivier/clearwater-live-test:hunter', 'imageID': '', 'started': False}], 'qosClass': 'BestEffort'}}} 2020-09-29 12:59:03,894 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 58, 59, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields': None, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2020, 9, 29, 12, 58, 59, tzinfo=tzlocal())}, {'api_version': 'v1', 'fields': None, 'manager': 'kubelet', 'operation': 'Update', 'time': datetime.datetime(2020, 9, 29, 12, 59, 3, tzinfo=tzlocal())}], 'name': 'live-test', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1643436', 'self_link': '/api/v1/namespaces/ims-cgn86/pods/live-test', 'uid': 'bebe78f5-b78c-4b02-8673-4230b3f7569f'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': ['rake', 'test[ims-cgn86.svc.cluster.local]', 'PROXY=bono.ims-cgn86.svc.cluster.local', 'ELLIS=ellis.ims-cgn86.svc.cluster.local', 'SIGNUP_CODE=secret', '--trace'], 'env': None, 'env_from': None, 'image': 'ollivier/clearwater-live-test:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'live-test', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-qcbq9', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': None, 'init_containers': None, 'node_name': 'kali-worker', 'node_selector': None, 'preemption_policy': 'PreemptLowerPriority', 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Never', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-qcbq9', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-qcbq9'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 58, 59, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 59, 3, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 59, 3, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 58, 59, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://4d2f4e028ef755700372faade75a7e2a1b3aa601aaf8850bbd552fc9e509de1a', 'image': 'docker.io/ollivier/clearwater-live-test:hunter', 'image_id': 'docker.io/ollivier/clearwater-live-test@sha256:178bbf9b98c044c0d84af42f9adcd167d6b5e6658219ebfbd352215dbc27dc97', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'live-test', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 9, 29, 12, 59, 2, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.12', 'init_container_statuses': None, 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.2.127', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 9, 29, 12, 58, 59, tzinfo=tzlocal())}}, 'raw_object': {'kind': 'Pod', 'apiVersion': 'v1', 'metadata': {'name': 'live-test', 'namespace': 'ims-cgn86', 'selfLink': '/api/v1/namespaces/ims-cgn86/pods/live-test', 'uid': 'bebe78f5-b78c-4b02-8673-4230b3f7569f', 'resourceVersion': '1643436', 'creationTimestamp': '2020-09-29T12:58:59Z', 'managedFields': [{'manager': 'OpenAPI-Generator', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2020-09-29T12:58:59Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:spec': {'f:containers': {'k:{"name":"live-test"}': {'.': {}, 'f:command': {}, 'f:image': {}, 'f:imagePullPolicy': {}, 'f:name': {}, 'f:resources': {}, 'f:terminationMessagePath': {}, 'f:terminationMessagePolicy': {}}}, 'f:dnsPolicy': {}, 'f:enableServiceLinks': {}, 'f:restartPolicy': {}, 'f:schedulerName': {}, 'f:securityContext': {}, 'f:terminationGracePeriodSeconds': {}}}}, {'manager': 'kubelet', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2020-09-29T12:59:03Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:status': {'f:conditions': {'k:{"type":"ContainersReady"}': {'.': {}, 'f:lastProbeTime': {}, 'f:lastTransitionTime': {}, 'f:status': {}, 'f:type': {}}, 'k:{"type":"Initialized"}': {'.': {}, 'f:lastProbeTime': {}, 'f:lastTransitionTime': {}, 'f:status': {}, 'f:type': {}}, 'k:{"type":"Ready"}': {'.': {}, 'f:lastProbeTime': {}, 'f:lastTransitionTime': {}, 'f:status': {}, 'f:type': {}}}, 'f:containerStatuses': {}, 'f:hostIP': {}, 'f:phase': {}, 'f:podIP': {}, 'f:podIPs': {'.': {}, 'k:{"ip":"10.244.2.127"}': {'.': {}, 'f:ip': {}}}, 'f:startTime': {}}}}]}, 'spec': {'volumes': [{'name': 'default-token-qcbq9', 'secret': {'secretName': 'default-token-qcbq9', 'defaultMode': 420}}], 'containers': [{'name': 'live-test', 'image': 'ollivier/clearwater-live-test:hunter', 'command': ['rake', 'test[ims-cgn86.svc.cluster.local]', 'PROXY=bono.ims-cgn86.svc.cluster.local', 'ELLIS=ellis.ims-cgn86.svc.cluster.local', 'SIGNUP_CODE=secret', '--trace'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-qcbq9', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Never', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'serviceAccountName': 'default', 'serviceAccount': 'default', 'nodeName': 'kali-worker', 'securityContext': {}, 'schedulerName': 'default-scheduler', 'tolerations': [{'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}, {'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}], 'priority': 0, 'enableServiceLinks': True, 'preemptionPolicy': 'PreemptLowerPriority'}, 'status': {'phase': 'Running', 'conditions': [{'type': 'Initialized', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:58:59Z'}, {'type': 'Ready', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:59:03Z'}, {'type': 'ContainersReady', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:59:03Z'}, {'type': 'PodScheduled', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:58:59Z'}], 'hostIP': '172.18.0.12', 'podIP': '10.244.2.127', 'podIPs': [{'ip': '10.244.2.127'}], 'startTime': '2020-09-29T12:58:59Z', 'containerStatuses': [{'name': 'live-test', 'state': {'running': {'startedAt': '2020-09-29T12:59:02Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/ollivier/clearwater-live-test:hunter', 'imageID': 'docker.io/ollivier/clearwater-live-test@sha256:178bbf9b98c044c0d84af42f9adcd167d6b5e6658219ebfbd352215dbc27dc97', 'containerID': 'containerd://4d2f4e028ef755700372faade75a7e2a1b3aa601aaf8850bbd552fc9e509de1a', 'started': True}], 'qosClass': 'BestEffort'}}} 2020-09-29 13:01:38,383 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 9, 29, 12, 58, 59, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields': None, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2020, 9, 29, 12, 58, 59, tzinfo=tzlocal())}, {'api_version': 'v1', 'fields': None, 'manager': 'kubelet', 'operation': 'Update', 'time': datetime.datetime(2020, 9, 29, 13, 1, 38, tzinfo=tzlocal())}], 'name': 'live-test', 'namespace': 'ims-cgn86', 'owner_references': None, 'resource_version': '1644160', 'self_link': '/api/v1/namespaces/ims-cgn86/pods/live-test', 'uid': 'bebe78f5-b78c-4b02-8673-4230b3f7569f'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': ['rake', 'test[ims-cgn86.svc.cluster.local]', 'PROXY=bono.ims-cgn86.svc.cluster.local', 'ELLIS=ellis.ims-cgn86.svc.cluster.local', 'SIGNUP_CODE=secret', '--trace'], 'env': None, 'env_from': None, 'image': 'ollivier/clearwater-live-test:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'live-test', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-qcbq9', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': None, 'init_containers': None, 'node_name': 'kali-worker', 'node_selector': None, 'preemption_policy': 'PreemptLowerPriority', 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Never', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-qcbq9', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-qcbq9'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 58, 59, tzinfo=tzlocal()), 'message': None, 'reason': 'PodCompleted', 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 13, 1, 38, tzinfo=tzlocal()), 'message': None, 'reason': 'PodCompleted', 'status': 'False', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 13, 1, 38, tzinfo=tzlocal()), 'message': None, 'reason': 'PodCompleted', 'status': 'False', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 9, 29, 12, 58, 59, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://4d2f4e028ef755700372faade75a7e2a1b3aa601aaf8850bbd552fc9e509de1a', 'image': 'docker.io/ollivier/clearwater-live-test:hunter', 'image_id': 'docker.io/ollivier/clearwater-live-test@sha256:178bbf9b98c044c0d84af42f9adcd167d6b5e6658219ebfbd352215dbc27dc97', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'live-test', 'ready': False, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://4d2f4e028ef755700372faade75a7e2a1b3aa601aaf8850bbd552fc9e509de1a', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 9, 29, 13, 1, 37, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 9, 29, 12, 59, 2, tzinfo=tzlocal())}, 'waiting': None}}], 'host_ip': '172.18.0.12', 'init_container_statuses': None, 'message': None, 'nominated_node_name': None, 'phase': 'Succeeded', 'pod_ip': '10.244.2.127', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 9, 29, 12, 58, 59, tzinfo=tzlocal())}}, 'raw_object': {'kind': 'Pod', 'apiVersion': 'v1', 'metadata': {'name': 'live-test', 'namespace': 'ims-cgn86', 'selfLink': '/api/v1/namespaces/ims-cgn86/pods/live-test', 'uid': 'bebe78f5-b78c-4b02-8673-4230b3f7569f', 'resourceVersion': '1644160', 'creationTimestamp': '2020-09-29T12:58:59Z', 'managedFields': [{'manager': 'OpenAPI-Generator', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2020-09-29T12:58:59Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:spec': {'f:containers': {'k:{"name":"live-test"}': {'.': {}, 'f:command': {}, 'f:image': {}, 'f:imagePullPolicy': {}, 'f:name': {}, 'f:resources': {}, 'f:terminationMessagePath': {}, 'f:terminationMessagePolicy': {}}}, 'f:dnsPolicy': {}, 'f:enableServiceLinks': {}, 'f:restartPolicy': {}, 'f:schedulerName': {}, 'f:securityContext': {}, 'f:terminationGracePeriodSeconds': {}}}}, {'manager': 'kubelet', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2020-09-29T13:01:38Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:status': {'f:conditions': {'k:{"type":"ContainersReady"}': {'.': {}, 'f:lastProbeTime': {}, 'f:lastTransitionTime': {}, 'f:reason': {}, 'f:status': {}, 'f:type': {}}, 'k:{"type":"Initialized"}': {'.': {}, 'f:lastProbeTime': {}, 'f:lastTransitionTime': {}, 'f:reason': {}, 'f:status': {}, 'f:type': {}}, 'k:{"type":"Ready"}': {'.': {}, 'f:lastProbeTime': {}, 'f:lastTransitionTime': {}, 'f:reason': {}, 'f:status': {}, 'f:type': {}}}, 'f:containerStatuses': {}, 'f:hostIP': {}, 'f:phase': {}, 'f:podIP': {}, 'f:podIPs': {'.': {}, 'k:{"ip":"10.244.2.127"}': {'.': {}, 'f:ip': {}}}, 'f:startTime': {}}}}]}, 'spec': {'volumes': [{'name': 'default-token-qcbq9', 'secret': {'secretName': 'default-token-qcbq9', 'defaultMode': 420}}], 'containers': [{'name': 'live-test', 'image': 'ollivier/clearwater-live-test:hunter', 'command': ['rake', 'test[ims-cgn86.svc.cluster.local]', 'PROXY=bono.ims-cgn86.svc.cluster.local', 'ELLIS=ellis.ims-cgn86.svc.cluster.local', 'SIGNUP_CODE=secret', '--trace'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-qcbq9', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Never', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'serviceAccountName': 'default', 'serviceAccount': 'default', 'nodeName': 'kali-worker', 'securityContext': {}, 'schedulerName': 'default-scheduler', 'tolerations': [{'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}, {'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}], 'priority': 0, 'enableServiceLinks': True, 'preemptionPolicy': 'PreemptLowerPriority'}, 'status': {'phase': 'Succeeded', 'conditions': [{'type': 'Initialized', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:58:59Z', 'reason': 'PodCompleted'}, {'type': 'Ready', 'status': 'False', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T13:01:38Z', 'reason': 'PodCompleted'}, {'type': 'ContainersReady', 'status': 'False', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T13:01:38Z', 'reason': 'PodCompleted'}, {'type': 'PodScheduled', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-09-29T12:58:59Z'}], 'hostIP': '172.18.0.12', 'podIP': '10.244.2.127', 'podIPs': [{'ip': '10.244.2.127'}], 'startTime': '2020-09-29T12:58:59Z', 'containerStatuses': [{'name': 'live-test', 'state': {'terminated': {'exitCode': 0, 'reason': 'Completed', 'startedAt': '2020-09-29T12:59:02Z', 'finishedAt': '2020-09-29T13:01:37Z', 'containerID': 'containerd://4d2f4e028ef755700372faade75a7e2a1b3aa601aaf8850bbd552fc9e509de1a'}}, 'lastState': {}, 'ready': False, 'restartCount': 0, 'image': 'docker.io/ollivier/clearwater-live-test:hunter', 'imageID': 'docker.io/ollivier/clearwater-live-test@sha256:178bbf9b98c044c0d84af42f9adcd167d6b5e6658219ebfbd352215dbc27dc97', 'containerID': 'containerd://4d2f4e028ef755700372faade75a7e2a1b3aa601aaf8850bbd552fc9e509de1a', 'started': False}], 'qosClass': 'BestEffort'}}} 2020-09-29 13:01:38,424 - kubernetes.client.rest - DEBUG - response body: ** Invoke test (first_time) ** Execute test /usr/lib/ruby/gems/2.6.0/gems/httpi-2.4.0/lib/httpi/auth/ssl.rb:13: warning: constant OpenSSL::SSL::SSLContext::METHODS is deprecated # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' Basic Call - Mainline (TCP) - (6505550300, 6505550405) Passed Basic Call - SDP (TCP) - (6505550317, 6505550049) Passed Basic Call - Tel URIs (TCP) - (6505550183, 6505550720) Passed Basic Call - Unknown number (TCP) - (6505550456, 6505550585) Passed Basic Call - Rejected by remote endpoint (TCP) - (6505550756, 6505550440) Passed Basic Call - Messages - Pager model (TCP) - (6505550313, 6505550088) Passed Basic Call - Pracks (TCP) - (6505550292, 6505550827) Passed Basic Registration (TCP) - (6505550725) Passed Emergency Registration (TCP) - Skipped (Not running with emergency registrations) - Call with EMERGENCY_REG=Y to run test Multiple Identities (TCP) - (6505550163, 6505550260) Passed Call Barring - Outbound Rejection (TCP) - (6505550299, 6505550211) Passed Call Barring - Allow non-international call (TCP) - Skipped (No PSTN support) - Call with PSTN=true to run test Call Barring - Reject international call (TCP) - Skipped (No PSTN support) - Call with PSTN=true to run test Call Barring - Inbound Rejection (TCP) - (6505550890, 6505550434) Passed Call Diversion AS - Unconditional (TCP) - Skipped (No Call Diversion AS hostname provided) - Call with CDIV_AS= Call Diversion AS - Busy (TCP) - Skipped (No Call Diversion AS hostname provided) - Call with CDIV_AS= Call Diversion AS - No answer (TCP) - Skipped (No Call Diversion AS hostname provided) - Call with CDIV_AS= Call Diversion - Not registered (TCP) - (6505550450, 6505550344, 6505550022) Passed Call Diversion - Not reachable (not registered) (TCP) - (6505550433, 6505550137, 6505550551) Passed Call Diversion - Not reachable (408) (TCP) - (6505550059, 6505550274, 6505550978) Passed Call Diversion - Not reachable (503) (TCP) - (6505550371, 6505550069, 6505550527) Passed Call Diversion - Not reachable (500) (TCP) - (6505550535, 6505550165, 6505550710) Passed Call Diversion - Busy (TCP) - (6505550719, 6505550796, 6505550306) Passed Call Diversion - Unconditional (TCP) - (6505550913, 6505550471, 6505550935) Passed Call Diversion - No answer (TCP) - (6505550789, 6505550154, 6505550928) Passed Call Diversion - Bad target URI (TCP) - (6505550282, 6505550434) Passed Call Diversion - Audio-only call (TCP) - (6505550146, 6505550802, 6505550950, 6505550722) Passed Call Diversion - Audio-video call (TCP) - (6505550345, 6505550343, 6505550566, 6505550184) Passed Call Forking - Mainline (TCP) - (6505550712, 6505550924) Passed Call Forking - Endpoint offline (TCP) - (6505550869, 6505550495) Passed Call Forking - Endpoint offline while ringing (TCP) - (6505550006, 6505550274) Passed Call Waiting - Accepted (TCP) - (6505550425, 6505550726, 6505550614) Passed Call Waiting - Cancelled (TCP) - (6505550411, 6505550700, 6505550754) Passed CANCEL - Mainline (TCP) - (6505550783, 6505550805) Passed Filtering - Accept-Contact (TCP) - (6505550582, 6505550232) Passed Accept-Contact with response (SFR 511195) (TCP) - (6505550850, 6505550358) Passed Filtering - Accept-Contact no match (TCP) - (6505550932, 6505550947) Passed Filtering - Accept-Contact negated match (TCP) - (6505550854, 6505550775) Passed Filtering - RFC3841 example (TCP) - (6505550580, 6505550001) Leaked sip:6505550001@ims-cgn86.svc.cluster.local, DELETE returned 502 Passed Filtering - Reject-Contact no match (TCP) - (6505550154, 6505550645) Passed Filtering - Reject-Contact match (TCP) - (6505550426, 6505550263) Passed Gemini - INVITE - VoIP device answers (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Mobile device answers (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - VoIP device rejects (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Mobile device rejects (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Mobile device rejects with a 480 (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Both reject, choose mobile response (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Both reject, choose VoIP response (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Successful call with GR (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Failed call with GR (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Successful call with Accept-Contact (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Failed call with Accept-Contact (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - SUBSCRIBE - Mobile Notifies (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - SUBSCRIBE - Joint 408 (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= GRUU - REGISTER - two bindings with and without GRUU (TCP) - (6505550011) Passed GRUU - REGISTER - binding suggested GRUU (TCP) - (6505550679) Passed GRUU - REGISTER - instance ID requires escaping (TCP) - (6505550249) Passed GRUU - REGISTER - three bindings with GRUUs (TCP) - (6505550423) Passed GRUU - Call - first endpoint GRUU as target (TCP) - (6505550037, 6505550826) Passed GRUU - Call - second endpoint GRUU as target (TCP) - (6505550209, 6505550601) Passed GRUU - Call - only GRUU as target (TCP) - (6505550201, 6505550593) Passed GRUU - Call - AoR as target (TCP) - (6505550691, 6505550198) Passed GRUU - Call - unknown GRUU as target (TCP) - (6505550437, 6505550536) Passed GRUU - Call - unknown GRUU as target - no GRUUs assigned (TCP) - (6505550971, 6505550096) Passed GRUU - Call - Reject-Contact interop (TCP) - (6505550059, 6505550914) Passed GRUU - Call - Accept-Contact interop (TCP) - (6505550186, 6505550467) Passed GRUU - Call - AoR with other param as target (TCP) - (6505550627, 6505550708) Passed GRUU - Call - GRUU with other param as target (TCP) - (6505550823, 6505550227) Passed ISC Interface - Terminating (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - Terminating (UDP AS) (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - Terminating Failed (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - Third-party Registration (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - Third-party Registration - implicit registration (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - Redirect (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - B2BUA (TCP) - Skipped (No hostname given) - Call with HOST_NAME= Live Call - Dial out to a real number (TCP) - Skipped (No live number given) - Call with LIVENUMBER= Memento - Incorrect Password (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Wrong Call List (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Basic Call (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Unknown Number (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Rejected Call (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Cancelled Call (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= /usr/lib/ruby/gems/2.6.0/gems/nokogiri-1.6.6.2/lib/nokogiri/xml/document.rb:44: warning: constant ::Fixnum is deprecated # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' Memento - Privacy Call (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Barred Call (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Busy Call Forwarding (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Nonce-Count - Mainline (TCP) - Skipped (No nonce-count support) - Call with NONCE_COUNT=Y to run test Nonce-Count - Reject Re-Use (TCP) - Skipped (No nonce-count support) - Call with NONCE_COUNT=Y to run test Off-net calls - tel: URI (TCP) - Skipped (No off-net number given) - Call with OFF_NET_TEL= Off-net calls - sip: URI (TCP) - Skipped (No off-net number given) - Call with OFF_NET_TEL= SUBSCRIBE - reg-event (TCP) - (6505550018) Passed SUBSCRIBE - reg-event with a GRUU (TCP) - (6505550259) Passed SUBSCRIBE - Subscription timeout (TCP) - (6505550409) Passed SUBSCRIBE - Registration timeout (TCP) - Skipped (Not running with short registrations) - Call with SHORT_REG=Y to run test Multiple SUBSCRIBErs to one UE's reg-event (TCP) - (6505550729, 6505550576) Passed Message - URN URIs (TCP) - Skipped (No hostname given) - Call with HOST_NAME= Message - URN URIs to AS (TCP) - Skipped (No hostname given) - Call with HOST_NAME= Listing existing numbers failed with HTTP code 502 Detailed error output: {"Upstream error"=>"404", "Upstream URL"=>"http://homestead-prov.ims-cgn86.svc.cluster.local:8889/public/sip%3A6505550001%40ims-cgn86.svc.cluster.local/associated_private_ids"} 0 failures out of 96 tests run 43 tests skipped 2020-09-29 13:01:38,425 - functest_kubernetes.ims.ims - INFO - ** Invoke test (first_time) ** Execute test /usr/lib/ruby/gems/2.6.0/gems/httpi-2.4.0/lib/httpi/auth/ssl.rb:13: warning: constant OpenSSL::SSL::SSLContext::METHODS is deprecated # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' Basic Call - Mainline (TCP) - (6505550300, 6505550405) Passed Basic Call - SDP (TCP) - (6505550317, 6505550049) Passed Basic Call - Tel URIs (TCP) - (6505550183, 6505550720) Passed Basic Call - Unknown number (TCP) - (6505550456, 6505550585) Passed Basic Call - Rejected by remote endpoint (TCP) - (6505550756, 6505550440) Passed Basic Call - Messages - Pager model (TCP) - (6505550313, 6505550088) Passed Basic Call - Pracks (TCP) - (6505550292, 6505550827) Passed Basic Registration (TCP) - (6505550725) Passed Emergency Registration (TCP) - Skipped (Not running with emergency registrations) - Call with EMERGENCY_REG=Y to run test Multiple Identities (TCP) - (6505550163, 6505550260) Passed Call Barring - Outbound Rejection (TCP) - (6505550299, 6505550211) Passed Call Barring - Allow non-international call (TCP) - Skipped (No PSTN support) - Call with PSTN=true to run test Call Barring - Reject international call (TCP) - Skipped (No PSTN support) - Call with PSTN=true to run test Call Barring - Inbound Rejection (TCP) - (6505550890, 6505550434) Passed Call Diversion AS - Unconditional (TCP) - Skipped (No Call Diversion AS hostname provided) - Call with CDIV_AS= Call Diversion AS - Busy (TCP) - Skipped (No Call Diversion AS hostname provided) - Call with CDIV_AS= Call Diversion AS - No answer (TCP) - Skipped (No Call Diversion AS hostname provided) - Call with CDIV_AS= Call Diversion - Not registered (TCP) - (6505550450, 6505550344, 6505550022) Passed Call Diversion - Not reachable (not registered) (TCP) - (6505550433, 6505550137, 6505550551) Passed Call Diversion - Not reachable (408) (TCP) - (6505550059, 6505550274, 6505550978) Passed Call Diversion - Not reachable (503) (TCP) - (6505550371, 6505550069, 6505550527) Passed Call Diversion - Not reachable (500) (TCP) - (6505550535, 6505550165, 6505550710) Passed Call Diversion - Busy (TCP) - (6505550719, 6505550796, 6505550306) Passed Call Diversion - Unconditional (TCP) - (6505550913, 6505550471, 6505550935) Passed Call Diversion - No answer (TCP) - (6505550789, 6505550154, 6505550928) Passed Call Diversion - Bad target URI (TCP) - (6505550282, 6505550434) Passed Call Diversion - Audio-only call (TCP) - (6505550146, 6505550802, 6505550950, 6505550722) Passed Call Diversion - Audio-video call (TCP) - (6505550345, 6505550343, 6505550566, 6505550184) Passed Call Forking - Mainline (TCP) - (6505550712, 6505550924) Passed Call Forking - Endpoint offline (TCP) - (6505550869, 6505550495) Passed Call Forking - Endpoint offline while ringing (TCP) - (6505550006, 6505550274) Passed Call Waiting - Accepted (TCP) - (6505550425, 6505550726, 6505550614) Passed Call Waiting - Cancelled (TCP) - (6505550411, 6505550700, 6505550754) Passed CANCEL - Mainline (TCP) - (6505550783, 6505550805) Passed Filtering - Accept-Contact (TCP) - (6505550582, 6505550232) Passed Accept-Contact with response (SFR 511195) (TCP) - (6505550850, 6505550358) Passed Filtering - Accept-Contact no match (TCP) - (6505550932, 6505550947) Passed Filtering - Accept-Contact negated match (TCP) - (6505550854, 6505550775) Passed Filtering - RFC3841 example (TCP) - (6505550580, 6505550001) Leaked sip:6505550001@ims-cgn86.svc.cluster.local, DELETE returned 502 Passed Filtering - Reject-Contact no match (TCP) - (6505550154, 6505550645) Passed Filtering - Reject-Contact match (TCP) - (6505550426, 6505550263) Passed Gemini - INVITE - VoIP device answers (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Mobile device answers (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - VoIP device rejects (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Mobile device rejects (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Mobile device rejects with a 480 (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Both reject, choose mobile response (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Both reject, choose VoIP response (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Successful call with GR (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Failed call with GR (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Successful call with Accept-Contact (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Failed call with Accept-Contact (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - SUBSCRIBE - Mobile Notifies (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - SUBSCRIBE - Joint 408 (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= GRUU - REGISTER - two bindings with and without GRUU (TCP) - (6505550011) Passed GRUU - REGISTER - binding suggested GRUU (TCP) - (6505550679) Passed GRUU - REGISTER - instance ID requires escaping (TCP) - (6505550249) Passed GRUU - REGISTER - three bindings with GRUUs (TCP) - (6505550423) Passed GRUU - Call - first endpoint GRUU as target (TCP) - (6505550037, 6505550826) Passed GRUU - Call - second endpoint GRUU as target (TCP) - (6505550209, 6505550601) Passed GRUU - Call - only GRUU as target (TCP) - (6505550201, 6505550593) Passed GRUU - Call - AoR as target (TCP) - (6505550691, 6505550198) Passed GRUU - Call - unknown GRUU as target (TCP) - (6505550437, 6505550536) Passed GRUU - Call - unknown GRUU as target - no GRUUs assigned (TCP) - (6505550971, 6505550096) Passed GRUU - Call - Reject-Contact interop (TCP) - (6505550059, 6505550914) Passed GRUU - Call - Accept-Contact interop (TCP) - (6505550186, 6505550467) Passed GRUU - Call - AoR with other param as target (TCP) - (6505550627, 6505550708) Passed GRUU - Call - GRUU with other param as target (TCP) - (6505550823, 6505550227) Passed ISC Interface - Terminating (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - Terminating (UDP AS) (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - Terminating Failed (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - Third-party Registration (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - Third-party Registration - implicit registration (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - Redirect (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - B2BUA (TCP) - Skipped (No hostname given) - Call with HOST_NAME= Live Call - Dial out to a real number (TCP) - Skipped (No live number given) - Call with LIVENUMBER= Memento - Incorrect Password (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Wrong Call List (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Basic Call (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Unknown Number (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Rejected Call (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Cancelled Call (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= /usr/lib/ruby/gems/2.6.0/gems/nokogiri-1.6.6.2/lib/nokogiri/xml/document.rb:44: warning: constant ::Fixnum is deprecated # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' Memento - Privacy Call (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Barred Call (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Busy Call Forwarding (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Nonce-Count - Mainline (TCP) - Skipped (No nonce-count support) - Call with NONCE_COUNT=Y to run test Nonce-Count - Reject Re-Use (TCP) - Skipped (No nonce-count support) - Call with NONCE_COUNT=Y to run test Off-net calls - tel: URI (TCP) - Skipped (No off-net number given) - Call with OFF_NET_TEL= Off-net calls - sip: URI (TCP) - Skipped (No off-net number given) - Call with OFF_NET_TEL= SUBSCRIBE - reg-event (TCP) - (6505550018) Passed SUBSCRIBE - reg-event with a GRUU (TCP) - (6505550259) Passed SUBSCRIBE - Subscription timeout (TCP) - (6505550409) Passed SUBSCRIBE - Registration timeout (TCP) - Skipped (Not running with short registrations) - Call with SHORT_REG=Y to run test Multiple SUBSCRIBErs to one UE's reg-event (TCP) - (6505550729, 6505550576) Passed Message - URN URIs (TCP) - Skipped (No hostname given) - Call with HOST_NAME= Message - URN URIs to AS (TCP) - Skipped (No hostname given) - Call with HOST_NAME= Listing existing numbers failed with HTTP code 502 Detailed error output: {"Upstream error"=>"404", "Upstream URL"=>"http://homestead-prov.ims-cgn86.svc.cluster.local:8889/public/sip%3A6505550001%40ims-cgn86.svc.cluster.local/associated_private_ids"} 0 failures out of 96 tests run 43 tests skipped 2020-09-29 13:01:38,427 - xtesting.ci.run_tests - INFO - Test result: +-------------------+------------------+------------------+----------------+ | TEST CASE | PROJECT | DURATION | RESULT | +-------------------+------------------+------------------+----------------+ | helm_vims | functest | 09:45 | PASS | +-------------------+------------------+------------------+----------------+ 2020-09-29 13:01:41,974 - functest_kubernetes.ims.ims - DEBUG - release "clearwater" uninstalled 2020-09-29 13:01:42,031 - kubernetes.client.rest - DEBUG - response body: {"kind":"Pod","apiVersion":"v1","metadata":{"name":"live-test","namespace":"ims-cgn86","selfLink":"/api/v1/namespaces/ims-cgn86/pods/live-test","uid":"bebe78f5-b78c-4b02-8673-4230b3f7569f","resourceVersion":"1644201","creationTimestamp":"2020-09-29T12:58:59Z","deletionTimestamp":"2020-09-29T13:01:41Z","deletionGracePeriodSeconds":0,"managedFields":[{"manager":"OpenAPI-Generator","operation":"Update","apiVersion":"v1","time":"2020-09-29T12:58:59Z","fieldsType":"FieldsV1","fieldsV1":{"f:spec":{"f:containers":{"k:{\"name\":\"live-test\"}":{".":{},"f:command":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{},"f:terminationGracePeriodSeconds":{}}}},{"manager":"kubelet","operation":"Update","apiVersion":"v1","time":"2020-09-29T13:01:38Z","fieldsType":"FieldsV1","fieldsV1":{"f:status":{"f:conditions":{"k:{\"type\":\"ContainersReady\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Initialized\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:containerStatuses":{},"f:hostIP":{},"f:phase":{},"f:podIP":{},"f:podIPs":{".":{},"k:{\"ip\":\"10.244.2.127\"}":{".":{},"f:ip":{}}},"f:startTime":{}}}}]},"spec":{"volumes":[{"name":"default-token-qcbq9","secret":{"secretName":"default-token-qcbq9","defaultMode":420}}],"containers":[{"name":"live-test","image":"ollivier/clearwater-live-test:hunter","command":["rake","test[ims-cgn86.svc.cluster.local]","PROXY=bono.ims-cgn86.svc.cluster.local","ELLIS=ellis.ims-cgn86.svc.cluster.local","SIGNUP_CODE=secret","--trace"],"resources":{},"volumeMounts":[{"name":"default-token-qcbq9","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Never","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"kali-worker","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}],"priority":0,"enableServiceLinks":true,"preemptionPolicy":"PreemptLowerPriority"},"status":{"phase":"Succeeded","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2020-09-29T12:58:59Z","reason":"PodCompleted"},{"type":"Ready","status":"False","lastProbeTime":null,"lastTransitionTime":"2020-09-29T13:01:38Z","reason":"PodCompleted"},{"type":"ContainersReady","status":"False","lastProbeTime":null,"lastTransitionTime":"2020-09-29T13:01:38Z","reason":"PodCompleted"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2020-09-29T12:58:59Z"}],"hostIP":"172.18.0.12","podIP":"10.244.2.127","podIPs":[{"ip":"10.244.2.127"}],"startTime":"2020-09-29T12:58:59Z","containerStatuses":[{"name":"live-test","state":{"terminated":{"exitCode":0,"reason":"Completed","startedAt":"2020-09-29T12:59:02Z","finishedAt":"2020-09-29T13:01:37Z","containerID":"containerd://4d2f4e028ef755700372faade75a7e2a1b3aa601aaf8850bbd552fc9e509de1a"}},"lastState":{},"ready":false,"restartCount":0,"image":"docker.io/ollivier/clearwater-live-test:hunter","imageID":"docker.io/ollivier/clearwater-live-test@sha256:178bbf9b98c044c0d84af42f9adcd167d6b5e6658219ebfbd352215dbc27dc97","containerID":"containerd://4d2f4e028ef755700372faade75a7e2a1b3aa601aaf8850bbd552fc9e509de1a","started":false}],"qosClass":"BestEffort"}} 2020-09-29 13:01:42,032 - functest_kubernetes.ims.ims - DEBUG - delete_namespaced_pod: {'api_version': 'v1', 'code': None, 'details': None, 'kind': 'Pod', 'message': None, 'metadata': {'_continue': None, 'remaining_item_count': None, 'resource_version': '1644201', 'self_link': '/api/v1/namespaces/ims-cgn86/pods/live-test'}, 'reason': None, 'status': "{'phase': 'Succeeded', 'conditions': [{'type': 'Initialized', " "'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': " "'2020-09-29T12:58:59Z', 'reason': 'PodCompleted'}, {'type': " "'Ready', 'status': 'False', 'lastProbeTime': None, " "'lastTransitionTime': '2020-09-29T13:01:38Z', 'reason': " "'PodCompleted'}, {'type': 'ContainersReady', 'status': 'False', " "'lastProbeTime': None, 'lastTransitionTime': " "'2020-09-29T13:01:38Z', 'reason': 'PodCompleted'}, {'type': " "'PodScheduled', 'status': 'True', 'lastProbeTime': None, " "'lastTransitionTime': '2020-09-29T12:58:59Z'}], 'hostIP': " "'172.18.0.12', 'podIP': '10.244.2.127', 'podIPs': [{'ip': " "'10.244.2.127'}], 'startTime': '2020-09-29T12:58:59Z', " "'containerStatuses': [{'name': 'live-test', 'state': " "{'terminated': {'exitCode': 0, 'reason': 'Completed', 'startedAt': " "'2020-09-29T12:59:02Z', 'finishedAt': '2020-09-29T13:01:37Z', " "'containerID': " "'containerd://4d2f4e028ef755700372faade75a7e2a1b3aa601aaf8850bbd552fc9e509de1a'}}, " "'lastState': {}, 'ready': False, 'restartCount': 0, 'image': " "'docker.io/ollivier/clearwater-live-test:hunter', 'imageID': " "'docker.io/ollivier/clearwater-live-test@sha256:178bbf9b98c044c0d84af42f9adcd167d6b5e6658219ebfbd352215dbc27dc97', " "'containerID': " "'containerd://4d2f4e028ef755700372faade75a7e2a1b3aa601aaf8850bbd552fc9e509de1a', " "'started': False}], 'qosClass': 'BestEffort'}"} 2020-09-29 13:01:42,055 - kubernetes.client.rest - DEBUG - response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success","details":{"name":"env-vars","kind":"configmaps","uid":"4df669de-3fb7-4700-9467-d63c11040302"}} 2020-09-29 13:01:42,056 - functest_kubernetes.ims.ims - DEBUG - delete_namespaced_config_map: {'api_version': 'v1', 'code': None, 'details': {'causes': None, 'group': None, 'kind': 'configmaps', 'name': 'env-vars', 'retry_after_seconds': None, 'uid': '4df669de-3fb7-4700-9467-d63c11040302'}, 'kind': 'Status', 'message': None, 'metadata': {'_continue': None, 'remaining_item_count': None, 'resource_version': None, 'self_link': None}, 'reason': None, 'status': 'Success'} 2020-09-29 13:01:42,062 - kubernetes.client.rest - DEBUG - response body: {"kind":"Namespace","apiVersion":"v1","metadata":{"name":"ims-cgn86","generateName":"ims-","selfLink":"/api/v1/namespaces/ims-cgn86","uid":"8d2154e8-9081-4f92-8cc1-0a71dcef344e","resourceVersion":"1644204","creationTimestamp":"2020-09-29T12:51:53Z","deletionTimestamp":"2020-09-29T13:01:42Z","managedFields":[{"manager":"OpenAPI-Generator","operation":"Update","apiVersion":"v1","time":"2020-09-29T12:51:53Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:generateName":{}},"f:status":{"f:phase":{}}}}]},"spec":{"finalizers":["kubernetes"]},"status":{"phase":"Terminating"}} 2020-09-29 13:01:42,063 - functest_kubernetes.ims.ims - DEBUG - delete_namespace: ims-cgn86 2020-09-29 13:01:42,399 - xtesting.core.testcase - DEBUG - Publishing /var/lib/xtesting/results/functest-kubernetes.log ('text/plain', None) 2020-09-29 13:01:42,638 - xtesting.core.testcase - DEBUG - Publishing /var/lib/xtesting/results/functest-kubernetes.debug.log ('text/plain', None)