2020-11-27 11:03:20,033 - xtesting.ci.run_tests - INFO - Deployment description: +-------------------------+----------------------------------------------------------+ | ENV VAR | VALUE | +-------------------------+----------------------------------------------------------+ | TEST_DB_EXT_URL | http://testresults.opnfv.org/test/api/v1/results | | S3_DST_URL | s3://artifacts.opnfv.org/functest- | | | kubernetes/1VXXZ36663JI/functest-kubernetes-pi- | | | ollivier-functest-kubernetes-cnf-arm-hunter- | | | helm_vims-run-11 | | S3_ENDPOINT_URL | https://storage.googleapis.com | | DEPLOY_SCENARIO | k8-nosdn-nofeature-noha | | BUILD_TAG | 1VXXZ36663JI | | DEBUG | true | | INSTALLER_TYPE | unknown | | CI_LOOP | daily | | TEST_DB_URL | http://testresults.opnfv.org/test/api/v1/results | | HTTP_DST_URL | http://artifacts.opnfv.org/functest- | | | kubernetes/1VXXZ36663JI/functest-kubernetes-pi- | | | ollivier-functest-kubernetes-cnf-arm-hunter- | | | helm_vims-run-11 | | NODE_NAME | lf-virtual1-1 | +-------------------------+----------------------------------------------------------+ 2020-11-27 11:03:20,097 - xtesting.ci.run_tests - DEBUG - No env file /var/lib/xtesting/conf/env_file found 2020-11-27 11:03:20,099 - xtesting.ci.run_tests - DEBUG - Test args: helm_vims 2020-11-27 11:03:20,181 - xtesting.ci.run_tests - INFO - Loading test case 'helm_vims'... 2020-11-27 11:03:21,296 - xtesting.ci.run_tests - INFO - Running test case 'helm_vims'... 2020-11-27 11:03:21,464 - kubernetes.client.rest - DEBUG - response body: {"kind":"Namespace","apiVersion":"v1","metadata":{"name":"ims-w9fzp","generateName":"ims-","selfLink":"/api/v1/namespaces/ims-w9fzp","uid":"2985dc91-30a0-11eb-8302-0242ac120002","resourceVersion":"11211331","creationTimestamp":"2020-11-27T11:03:21Z"},"spec":{"finalizers":["kubernetes"]},"status":{"phase":"Active"}} 2020-11-27 11:03:21,474 - functest_kubernetes.ims.ims - DEBUG - create_namespace: {'api_version': 'v1', 'kind': 'Namespace', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 3, 21, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'ims-', 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': None, 'name': 'ims-w9fzp', 'namespace': None, 'owner_references': None, 'resource_version': '11211331', 'self_link': '/api/v1/namespaces/ims-w9fzp', 'uid': '2985dc91-30a0-11eb-8302-0242ac120002'}, 'spec': {'finalizers': ['kubernetes']}, 'status': {'phase': 'Active'}} 2020-11-27 11:03:21,505 - kubernetes.client.rest - DEBUG - response body: {"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"env-vars","namespace":"ims-w9fzp","selfLink":"/api/v1/namespaces/ims-w9fzp/configmaps/env-vars","uid":"298cef9c-30a0-11eb-8302-0242ac120002","resourceVersion":"11211336","creationTimestamp":"2020-11-27T11:03:21Z"},"data":{"ADDITIONAL_SHARED_CONFIG":"","ZONE":"ims-w9fzp.svc.cluster.local"}} 2020-11-27 11:03:21,513 - functest_kubernetes.ims.ims - DEBUG - create_namespaced_config_map: {'api_version': 'v1', 'binary_data': None, 'data': {u'ADDITIONAL_SHARED_CONFIG': '', u'ZONE': 'ims-w9fzp.svc.cluster.local'}, 'kind': 'ConfigMap', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 3, 21, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': None, 'name': 'env-vars', 'namespace': 'ims-w9fzp', 'owner_references': None, 'resource_version': '11211336', 'self_link': '/api/v1/namespaces/ims-w9fzp/configmaps/env-vars', 'uid': '298cef9c-30a0-11eb-8302-0242ac120002'}} 2020-11-27 11:04:26,635 - functest_kubernetes.ims.ims - DEBUG - time="2020-11-27T11:03:23Z" level=error msg="failure getting variant" error="getCPUInfo for pattern: Cpu architecture: not found" NAME: clearwater LAST DEPLOYED: Fri Nov 27 11:03:43 2020 NAMESPACE: ims-w9fzp STATUS: deployed REVISION: 1 TEST SUITE: None 2020-11-27 11:04:26,754 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'observedGeneration': 1, u'updatedReplicas': 1, u'unavailableReplicas': 1, u'conditions': [{u'status': u'False', u'lastUpdateTime': u'2020-11-27T11:04:22Z', u'lastTransitionTime': u'2020-11-27T11:04:22Z', u'reason': u'MinimumReplicasUnavailable', u'message': u'Deployment does not have minimum availability.', u'type': u'Available'}, {u'status': u'True', u'lastUpdateTime': u'2020-11-27T11:04:22Z', u'lastTransitionTime': u'2020-11-27T11:04:21Z', u'reason': u'ReplicaSetUpdated', u'message': u'ReplicaSet "etcd-c8fddbb95" is progressing.', u'type': u'Progressing'}], u'replicas': 1}, u'kind': u'Deployment', u'spec': {u'progressDeadlineSeconds': 600, u'replicas': 1, u'strategy': {u'rollingUpdate': {u'maxSurge': u'25%', u'maxUnavailable': u'25%'}, u'type': u'RollingUpdate'}, u'template': {u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'schedulerName': u'default-scheduler', u'terminationGracePeriodSeconds': 30, u'restartPolicy': u'Always', u'imagePullSecrets': [{}], u'containers': [{u'livenessProbe': {u'tcpSocket': {u'port': 4001}, u'timeoutSeconds': 1, u'initialDelaySeconds': 300, u'periodSeconds': 10, u'successThreshold': 1, u'failureThreshold': 3}, u'terminationMessagePath': u'/dev/termination-log', u'name': u'etcd', u'image': u'quay.io/coreos/etcd:v2.2.5', u'args': [u'--name', u'$(MY_POD_NAME)', u'--advertise-client-urls', u'http://$(MY_POD_IP):2379,http://$(MY_POD_IP):4001', u'--listen-client-urls', u'http://0.0.0.0:2379,http://0.0.0.0:4001', u'--initial-advertise-peer-urls', u'http://$(MY_POD_IP):2380', u'--listen-peer-urls', u'http://0.0.0.0:2380', u'--initial-cluster', u'$(MY_POD_NAME)=http://$(MY_POD_IP):2380', u'--initial-cluster-state', u'new'], u'terminationMessagePolicy': u'File', u'env': [{u'valueFrom': {u'fieldRef': {u'fieldPath': u'status.podIP', u'apiVersion': u'v1'}}, u'name': u'MY_POD_IP'}, {u'valueFrom': {u'fieldRef': {u'fieldPath': u'metadata.name', u'apiVersion': u'v1'}}, u'name': u'MY_POD_NAME'}], u'imagePullPolicy': u'IfNotPresent', u'readinessProbe': {u'successThreshold': 1, u'tcpSocket': {u'port': 4001}, u'failureThreshold': 3, u'timeoutSeconds': 1, u'periodSeconds': 10}, u'ports': [{u'protocol': u'TCP', u'containerPort': 2379}, {u'protocol': u'TCP', u'containerPort': 2380}, {u'protocol': u'TCP', u'containerPort': 4001}], u'resources': {}}]}, u'metadata': {u'labels': {u'instance-type': u'etcd-pod'}, u'creationTimestamp': None}}, u'revisionHistoryLimit': 10, u'selector': {u'matchLabels': {u'instance-type': u'etcd-pod'}}}, u'apiVersion': u'apps/v1', u'metadata': {u'name': u'etcd', u'generation': 1, u'labels': {u'app.kubernetes.io/managed-by': u'Helm'}, u'namespace': u'ims-w9fzp', u'resourceVersion': u'11211814', u'creationTimestamp': u'2020-11-27T11:04:21Z', u'annotations': {u'meta.helm.sh/release-name': u'clearwater', u'deployment.kubernetes.io/revision': u'1', u'meta.helm.sh/release-namespace': u'ims-w9fzp'}, u'selfLink': u'/apis/apps/v1/namespaces/ims-w9fzp/deployments/etcd', u'uid': u'4d5bf704-30a0-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {u'deployment.kubernetes.io/revision': '1', u'meta.helm.sh/release-name': 'clearwater', u'meta.helm.sh/release-namespace': 'ims-w9fzp'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {u'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'etcd', 'namespace': 'ims-w9fzp', 'owner_references': None, 'resource_version': '11211814', 'self_link': '/apis/apps/v1/namespaces/ims-w9fzp/deployments/etcd', 'uid': '4d5bf704-30a0-11eb-8302-0242ac120002'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {u'instance-type': 'etcd-pod'}}, 'strategy': {'rolling_update': {'max_surge': u'25%', 'max_unavailable': u'25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {u'instance-type': 'etcd-pod'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': ['--name', '$(MY_POD_NAME)', '--advertise-client-urls', 'http://$(MY_POD_IP):2379,http://$(MY_POD_IP):4001', '--listen-client-urls', 'http://0.0.0.0:2379,http://0.0.0.0:4001', '--initial-advertise-peer-urls', 'http://$(MY_POD_IP):2380', '--listen-peer-urls', 'http://0.0.0.0:2380', '--initial-cluster', '$(MY_POD_NAME)=http://$(MY_POD_IP):2380', '--initial-cluster-state', 'new'], 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}, {'name': 'MY_POD_NAME', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'metadata.name'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': None, 'image': 'quay.io/coreos/etcd:v2.2.5', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 300, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 4001}, 'timeout_seconds': 1}, 'name': 'etcd', 'ports': [{'container_port': 2379, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 2380, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 4001, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 4001}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': None, 'node_name': None, 'node_selector': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal()), 'message': 'ReplicaSet "etcd-c8fddbb95" is progressing.', 'reason': 'ReplicaSetUpdated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': 1, 'unavailable_replicas': 1, 'updated_replicas': 1}}, u'type': u'ADDED'} 2020-11-27 11:04:27,020 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'observedGeneration': 1, u'updatedReplicas': 1, u'unavailableReplicas': 1, u'conditions': [{u'status': u'False', u'lastUpdateTime': u'2020-11-27T11:04:21Z', u'lastTransitionTime': u'2020-11-27T11:04:21Z', u'reason': u'MinimumReplicasUnavailable', u'message': u'Deployment does not have minimum availability.', u'type': u'Available'}, {u'status': u'True', u'lastUpdateTime': u'2020-11-27T11:04:22Z', u'lastTransitionTime': u'2020-11-27T11:04:21Z', u'reason': u'ReplicaSetUpdated', u'message': u'ReplicaSet "homer-65d4f8455b" is progressing.', u'type': u'Progressing'}], u'replicas': 1}, u'kind': u'Deployment', u'spec': {u'progressDeadlineSeconds': 600, u'replicas': 1, u'strategy': {u'rollingUpdate': {u'maxSurge': u'25%', u'maxUnavailable': u'25%'}, u'type': u'RollingUpdate'}, u'template': {u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'schedulerName': u'default-scheduler', u'terminationGracePeriodSeconds': 30, u'restartPolicy': u'Always', u'initContainers': [{u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-cassandra', u'image': u'busybox:1.28', u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'imagePullSecrets': [{}], u'containers': [{u'livenessProbe': {u'tcpSocket': {u'port': 7888}, u'timeoutSeconds': 1, u'initialDelaySeconds': 30, u'periodSeconds': 10, u'successThreshold': 1, u'failureThreshold': 3}, u'terminationMessagePath': u'/dev/termination-log', u'name': u'homer', u'envFrom': [{u'configMapRef': {u'name': u'env-vars'}}], u'image': u'ollivier/clearwater-homer:hunter', u'terminationMessagePolicy': u'File', u'env': [{u'valueFrom': {u'fieldRef': {u'fieldPath': u'status.podIP', u'apiVersion': u'v1'}}, u'name': u'MY_POD_IP'}], u'imagePullPolicy': u'IfNotPresent', u'readinessProbe': {u'successThreshold': 1, u'tcpSocket': {u'port': 7888}, u'failureThreshold': 3, u'timeoutSeconds': 1, u'periodSeconds': 10}, u'ports': [{u'protocol': u'TCP', u'containerPort': 22}, {u'protocol': u'TCP', u'containerPort': 7888}], u'resources': {}}]}, u'metadata': {u'labels': {u'service': u'homer'}, u'creationTimestamp': None}}, u'revisionHistoryLimit': 10, u'selector': {u'matchLabels': {u'service': u'homer'}}}, u'apiVersion': u'apps/v1', u'metadata': {u'name': u'homer', u'generation': 1, u'labels': {u'app.kubernetes.io/managed-by': u'Helm'}, u'namespace': u'ims-w9fzp', u'resourceVersion': u'11211825', u'creationTimestamp': u'2020-11-27T11:04:21Z', u'annotations': {u'meta.helm.sh/release-name': u'clearwater', u'deployment.kubernetes.io/revision': u'1', u'meta.helm.sh/release-namespace': u'ims-w9fzp'}, u'selfLink': u'/apis/apps/v1/namespaces/ims-w9fzp/deployments/homer', u'uid': u'4d5d0a0f-30a0-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {u'deployment.kubernetes.io/revision': '1', u'meta.helm.sh/release-name': 'clearwater', u'meta.helm.sh/release-namespace': 'ims-w9fzp'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {u'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'homer', 'namespace': 'ims-w9fzp', 'owner_references': None, 'resource_version': '11211825', 'self_link': '/apis/apps/v1/namespaces/ims-w9fzp/deployments/homer', 'uid': '4d5d0a0f-30a0-11eb-8302-0242ac120002'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {u'service': 'homer'}}, 'strategy': {'rolling_update': {'max_surge': u'25%', 'max_unavailable': u'25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {u'service': 'homer'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-homer:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7888}, 'timeout_seconds': 1}, 'name': 'homer', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 7888, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7888}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal()), 'message': 'ReplicaSet "homer-65d4f8455b" is progressing.', 'reason': 'ReplicaSetUpdated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': 1, 'unavailable_replicas': 1, 'updated_replicas': 1}}, u'type': u'ADDED'} 2020-11-27 11:04:27,257 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'observedGeneration': 1, u'updatedReplicas': 1, u'unavailableReplicas': 1, u'conditions': [{u'status': u'False', u'lastUpdateTime': u'2020-11-27T11:04:22Z', u'lastTransitionTime': u'2020-11-27T11:04:22Z', u'reason': u'MinimumReplicasUnavailable', u'message': u'Deployment does not have minimum availability.', u'type': u'Available'}, {u'status': u'True', u'lastUpdateTime': u'2020-11-27T11:04:22Z', u'lastTransitionTime': u'2020-11-27T11:04:21Z', u'reason': u'ReplicaSetUpdated', u'message': u'ReplicaSet "ralf-797bfb6988" is progressing.', u'type': u'Progressing'}], u'replicas': 1}, u'kind': u'Deployment', u'spec': {u'progressDeadlineSeconds': 600, u'replicas': 1, u'strategy': {u'rollingUpdate': {u'maxSurge': u'25%', u'maxUnavailable': u'25%'}, u'type': u'RollingUpdate'}, u'template': {u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'schedulerName': u'default-scheduler', u'terminationGracePeriodSeconds': 30, u'restartPolicy': u'Always', u'volumes': [{u'emptyDir': {}, u'name': u'ralflogs'}], u'initContainers': [{u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-cassandra', u'image': u'busybox:1.28', u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'imagePullSecrets': [{}], u'containers': [{u'livenessProbe': {u'tcpSocket': {u'port': 10888}, u'timeoutSeconds': 1, u'initialDelaySeconds': 30, u'periodSeconds': 10, u'successThreshold': 1, u'failureThreshold': 3}, u'terminationMessagePath': u'/dev/termination-log', u'name': u'ralf', u'envFrom': [{u'configMapRef': {u'name': u'env-vars'}}], u'image': u'ollivier/clearwater-ralf:hunter', u'volumeMounts': [{u'mountPath': u'/var/log/ralf', u'name': u'ralflogs'}], u'terminationMessagePolicy': u'File', u'env': [{u'valueFrom': {u'fieldRef': {u'fieldPath': u'status.podIP', u'apiVersion': u'v1'}}, u'name': u'MY_POD_IP'}], u'imagePullPolicy': u'IfNotPresent', u'readinessProbe': {u'successThreshold': 1, u'tcpSocket': {u'port': 10888}, u'failureThreshold': 3, u'timeoutSeconds': 1, u'periodSeconds': 10}, u'ports': [{u'protocol': u'TCP', u'containerPort': 22}, {u'protocol': u'TCP', u'containerPort': 10888}], u'resources': {}}, {u'terminationMessagePath': u'/dev/termination-log', u'name': u'tailer', u'image': u'busybox:1.28', u'volumeMounts': [{u'mountPath': u'/var/log/ralf', u'name': u'ralflogs'}], u'terminationMessagePolicy': u'File', u'command': [u'tail', u'-F', u'/var/log/ralf/ralf_current.txt'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}]}, u'metadata': {u'labels': {u'snmp': u'enabled', u'service': u'ralf'}, u'creationTimestamp': None}}, u'revisionHistoryLimit': 10, u'selector': {u'matchLabels': {u'service': u'ralf'}}}, u'apiVersion': u'apps/v1', u'metadata': {u'name': u'ralf', u'generation': 1, u'labels': {u'app.kubernetes.io/managed-by': u'Helm'}, u'namespace': u'ims-w9fzp', u'resourceVersion': u'11211820', u'creationTimestamp': u'2020-11-27T11:04:21Z', u'annotations': {u'meta.helm.sh/release-name': u'clearwater', u'deployment.kubernetes.io/revision': u'1', u'meta.helm.sh/release-namespace': u'ims-w9fzp'}, u'selfLink': u'/apis/apps/v1/namespaces/ims-w9fzp/deployments/ralf', u'uid': u'4d62aa90-30a0-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {u'deployment.kubernetes.io/revision': '1', u'meta.helm.sh/release-name': 'clearwater', u'meta.helm.sh/release-namespace': 'ims-w9fzp'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {u'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'ralf', 'namespace': 'ims-w9fzp', 'owner_references': None, 'resource_version': '11211820', 'self_link': '/apis/apps/v1/namespaces/ims-w9fzp/deployments/ralf', 'uid': '4d62aa90-30a0-11eb-8302-0242ac120002'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {u'service': 'ralf'}}, 'strategy': {'rolling_update': {'max_surge': u'25%', 'max_unavailable': u'25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {u'service': 'ralf', u'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-ralf:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 10888}, 'timeout_seconds': 1}, 'name': 'ralf', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 10888, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 10888}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/ralf', 'mount_propagation': None, 'name': 'ralflogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/ralf/ralf_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/ralf', 'mount_propagation': None, 'name': 'ralflogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'ralflogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal()), 'message': 'ReplicaSet "ralf-797bfb6988" is progressing.', 'reason': 'ReplicaSetUpdated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': 1, 'unavailable_replicas': 1, 'updated_replicas': 1}}, u'type': u'ADDED'} 2020-11-27 11:04:27,743 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'observedGeneration': 1, u'updatedReplicas': 1, u'unavailableReplicas': 1, u'conditions': [{u'status': u'False', u'lastUpdateTime': u'2020-11-27T11:04:21Z', u'lastTransitionTime': u'2020-11-27T11:04:21Z', u'reason': u'MinimumReplicasUnavailable', u'message': u'Deployment does not have minimum availability.', u'type': u'Available'}, {u'status': u'True', u'lastUpdateTime': u'2020-11-27T11:04:22Z', u'lastTransitionTime': u'2020-11-27T11:04:21Z', u'reason': u'ReplicaSetUpdated', u'message': u'ReplicaSet "sprout-55bcf5566" is progressing.', u'type': u'Progressing'}], u'replicas': 1}, u'kind': u'Deployment', u'spec': {u'progressDeadlineSeconds': 600, u'replicas': 1, u'strategy': {u'rollingUpdate': {u'maxSurge': u'25%', u'maxUnavailable': u'25%'}, u'type': u'RollingUpdate'}, u'template': {u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'schedulerName': u'default-scheduler', u'terminationGracePeriodSeconds': 30, u'restartPolicy': u'Always', u'volumes': [{u'emptyDir': {}, u'name': u'sproutlogs'}], u'initContainers': [{u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-ralf', u'image': u'busybox:1.28', u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup ralf; do echo waiting for ralf to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}, {u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-chronos', u'image': u'busybox:1.28', u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup chronos; do echo waiting for chronos to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}, {u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-homestead', u'image': u'busybox:1.28', u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup homestead; do echo waiting for homestead to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'imagePullSecrets': [{}], u'containers': [{u'livenessProbe': {u'timeoutSeconds': 1, u'exec': {u'command': [u'/bin/bash', u'/usr/share/kubernetes/liveness.sh', u'5052 5054']}, u'initialDelaySeconds': 30, u'periodSeconds': 3, u'successThreshold': 1, u'failureThreshold': 3}, u'terminationMessagePath': u'/dev/termination-log', u'name': u'sprout', u'envFrom': [{u'configMapRef': {u'name': u'env-vars'}}], u'image': u'ollivier/clearwater-sprout:hunter', u'volumeMounts': [{u'mountPath': u'/var/log/sprout', u'name': u'sproutlogs'}], u'terminationMessagePolicy': u'File', u'env': [{u'valueFrom': {u'fieldRef': {u'fieldPath': u'status.podIP', u'apiVersion': u'v1'}}, u'name': u'MY_POD_IP'}], u'imagePullPolicy': u'IfNotPresent', u'readinessProbe': {u'successThreshold': 1, u'periodSeconds': 10, u'failureThreshold': 3, u'timeoutSeconds': 1, u'exec': {u'command': [u'/bin/bash', u'/usr/share/kubernetes/liveness.sh', u'5052 5054']}}, u'ports': [{u'protocol': u'TCP', u'containerPort': 22}], u'resources': {}}, {u'terminationMessagePath': u'/dev/termination-log', u'name': u'tailer', u'image': u'busybox:1.28', u'volumeMounts': [{u'mountPath': u'/var/log/sprout', u'name': u'sproutlogs'}], u'terminationMessagePolicy': u'File', u'command': [u'tail', u'-F', u'/var/log/sprout/sprout_current.txt'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}]}, u'metadata': {u'labels': {u'snmp': u'enabled', u'service': u'sprout'}, u'creationTimestamp': None}}, u'revisionHistoryLimit': 10, u'selector': {u'matchLabels': {u'service': u'sprout'}}}, u'apiVersion': u'apps/v1', u'metadata': {u'name': u'sprout', u'generation': 1, u'labels': {u'app.kubernetes.io/managed-by': u'Helm'}, u'namespace': u'ims-w9fzp', u'resourceVersion': u'11211823', u'creationTimestamp': u'2020-11-27T11:04:21Z', u'annotations': {u'meta.helm.sh/release-name': u'clearwater', u'deployment.kubernetes.io/revision': u'1', u'meta.helm.sh/release-namespace': u'ims-w9fzp'}, u'selfLink': u'/apis/apps/v1/namespaces/ims-w9fzp/deployments/sprout', u'uid': u'4d5b1b1d-30a0-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {u'deployment.kubernetes.io/revision': '1', u'meta.helm.sh/release-name': 'clearwater', u'meta.helm.sh/release-namespace': 'ims-w9fzp'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {u'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'sprout', 'namespace': 'ims-w9fzp', 'owner_references': None, 'resource_version': '11211823', 'self_link': '/apis/apps/v1/namespaces/ims-w9fzp/deployments/sprout', 'uid': '4d5b1b1d-30a0-11eb-8302-0242ac120002'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {u'service': 'sprout'}}, 'strategy': {'rolling_update': {'max_surge': u'25%', 'max_unavailable': u'25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {u'service': 'sprout', u'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-sprout:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 5054']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 3, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'sprout', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 5054']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/sprout', 'mount_propagation': None, 'name': 'sproutlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/sprout/sprout_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/sprout', 'mount_propagation': None, 'name': 'sproutlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup ralf; do echo waiting for ralf to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-ralf', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until nslookup chronos; do echo waiting for chronos to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-chronos', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until nslookup homestead; do echo waiting for homestead to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-homestead', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'sproutlogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal()), 'message': 'ReplicaSet "sprout-55bcf5566" is progressing.', 'reason': 'ReplicaSetUpdated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': 1, 'unavailable_replicas': 1, 'updated_replicas': 1}}, u'type': u'ADDED'} 2020-11-27 11:04:28,096 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'observedGeneration': 1, u'updatedReplicas': 1, u'unavailableReplicas': 1, u'conditions': [{u'status': u'False', u'lastUpdateTime': u'2020-11-27T11:04:21Z', u'lastTransitionTime': u'2020-11-27T11:04:21Z', u'reason': u'MinimumReplicasUnavailable', u'message': u'Deployment does not have minimum availability.', u'type': u'Available'}, {u'status': u'True', u'lastUpdateTime': u'2020-11-27T11:04:22Z', u'lastTransitionTime': u'2020-11-27T11:04:21Z', u'reason': u'ReplicaSetUpdated', u'message': u'ReplicaSet "cassandra-56f89b7f94" is progressing.', u'type': u'Progressing'}], u'replicas': 1}, u'kind': u'Deployment', u'spec': {u'progressDeadlineSeconds': 600, u'replicas': 1, u'strategy': {u'rollingUpdate': {u'maxSurge': u'25%', u'maxUnavailable': u'25%'}, u'type': u'RollingUpdate'}, u'template': {u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'schedulerName': u'default-scheduler', u'terminationGracePeriodSeconds': 30, u'restartPolicy': u'Always', u'initContainers': [{u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-etcd', u'image': u'busybox:1.28', u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup etcd; do echo waiting for etcd to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'imagePullSecrets': [{}], u'containers': [{u'livenessProbe': {u'timeoutSeconds': 1, u'exec': {u'command': [u'/bin/bash', u'/usr/share/kubernetes/liveness.sh', u'7000 9042 9160']}, u'initialDelaySeconds': 600, u'periodSeconds': 10, u'successThreshold': 1, u'failureThreshold': 3}, u'terminationMessagePath': u'/dev/termination-log', u'name': u'cassandra', u'envFrom': [{u'configMapRef': {u'name': u'env-vars'}}], u'image': u'ollivier/clearwater-cassandra:hunter', u'terminationMessagePolicy': u'File', u'env': [{u'valueFrom': {u'fieldRef': {u'fieldPath': u'status.podIP', u'apiVersion': u'v1'}}, u'name': u'MY_POD_IP'}], u'imagePullPolicy': u'IfNotPresent', u'readinessProbe': {u'successThreshold': 1, u'periodSeconds': 10, u'failureThreshold': 3, u'timeoutSeconds': 1, u'exec': {u'command': [u'/bin/bash', u'/usr/share/kubernetes/liveness.sh', u'7000 9042 9160']}}, u'ports': [{u'protocol': u'TCP', u'containerPort': 22}, {u'protocol': u'TCP', u'containerPort': 7001}, {u'protocol': u'TCP', u'containerPort': 9042}, {u'protocol': u'TCP', u'containerPort': 9160}], u'resources': {}}]}, u'metadata': {u'labels': {u'service': u'cassandra'}, u'creationTimestamp': None}}, u'revisionHistoryLimit': 10, u'selector': {u'matchLabels': {u'service': u'cassandra'}}}, u'apiVersion': u'apps/v1', u'metadata': {u'name': u'cassandra', u'generation': 1, u'labels': {u'app.kubernetes.io/managed-by': u'Helm'}, u'namespace': u'ims-w9fzp', u'resourceVersion': u'11211822', u'creationTimestamp': u'2020-11-27T11:04:21Z', u'annotations': {u'meta.helm.sh/release-name': u'clearwater', u'deployment.kubernetes.io/revision': u'1', u'meta.helm.sh/release-namespace': u'ims-w9fzp'}, u'selfLink': u'/apis/apps/v1/namespaces/ims-w9fzp/deployments/cassandra', u'uid': u'4d5a0df6-30a0-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {u'deployment.kubernetes.io/revision': '1', u'meta.helm.sh/release-name': 'clearwater', u'meta.helm.sh/release-namespace': 'ims-w9fzp'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {u'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'cassandra', 'namespace': 'ims-w9fzp', 'owner_references': None, 'resource_version': '11211822', 'self_link': '/apis/apps/v1/namespaces/ims-w9fzp/deployments/cassandra', 'uid': '4d5a0df6-30a0-11eb-8302-0242ac120002'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {u'service': 'cassandra'}}, 'strategy': {'rolling_update': {'max_surge': u'25%', 'max_unavailable': u'25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {u'service': 'cassandra'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-cassandra:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 9042 9160']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 600, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'cassandra', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 7001, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 9042, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 9160, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 9042 9160']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup etcd; do echo waiting for etcd to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-etcd', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal()), 'message': 'ReplicaSet "cassandra-56f89b7f94" is progressing.', 'reason': 'ReplicaSetUpdated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': 1, 'unavailable_replicas': 1, 'updated_replicas': 1}}, u'type': u'ADDED'} 2020-11-27 11:04:28,331 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'observedGeneration': 1, u'updatedReplicas': 1, u'unavailableReplicas': 1, u'conditions': [{u'status': u'False', u'lastUpdateTime': u'2020-11-27T11:04:22Z', u'lastTransitionTime': u'2020-11-27T11:04:22Z', u'reason': u'MinimumReplicasUnavailable', u'message': u'Deployment does not have minimum availability.', u'type': u'Available'}, {u'status': u'True', u'lastUpdateTime': u'2020-11-27T11:04:22Z', u'lastTransitionTime': u'2020-11-27T11:04:21Z', u'reason': u'ReplicaSetUpdated', u'message': u'ReplicaSet "astaire-5d44675d59" is progressing.', u'type': u'Progressing'}], u'replicas': 1}, u'kind': u'Deployment', u'spec': {u'progressDeadlineSeconds': 600, u'replicas': 1, u'strategy': {u'rollingUpdate': {u'maxSurge': u'25%', u'maxUnavailable': u'25%'}, u'type': u'RollingUpdate'}, u'template': {u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'schedulerName': u'default-scheduler', u'terminationGracePeriodSeconds': 120, u'restartPolicy': u'Always', u'volumes': [{u'emptyDir': {}, u'name': u'astairelogs'}], u'initContainers': [{u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-cassandra', u'image': u'busybox:1.28', u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'imagePullSecrets': [{}], u'containers': [{u'livenessProbe': {u'successThreshold': 1, u'tcpSocket': {u'port': 11311}, u'failureThreshold': 9, u'timeoutSeconds': 1, u'periodSeconds': 10}, u'terminationMessagePath': u'/dev/termination-log', u'name': u'astaire', u'envFrom': [{u'configMapRef': {u'name': u'env-vars'}}], u'image': u'ollivier/clearwater-astaire:hunter', u'volumeMounts': [{u'mountPath': u'/var/log/astaire', u'name': u'astairelogs'}], u'terminationMessagePolicy': u'File', u'lifecycle': {u'preStop': {u'exec': {u'command': [u'/bin/bash', u'-c', u'/usr/bin/pre-stop']}}}, u'env': [{u'valueFrom': {u'fieldRef': {u'fieldPath': u'status.podIP', u'apiVersion': u'v1'}}, u'name': u'MY_POD_IP'}], u'imagePullPolicy': u'IfNotPresent', u'readinessProbe': {u'successThreshold': 1, u'tcpSocket': {u'port': 11311}, u'failureThreshold': 3, u'timeoutSeconds': 1, u'periodSeconds': 10}, u'ports': [{u'protocol': u'TCP', u'containerPort': 22}, {u'protocol': u'TCP', u'containerPort': 11211}, {u'protocol': u'TCP', u'containerPort': 11311}], u'resources': {}}, {u'terminationMessagePath': u'/dev/termination-log', u'name': u'tailer', u'image': u'busybox:1.28', u'volumeMounts': [{u'mountPath': u'/var/log/astaire', u'name': u'astairelogs'}], u'terminationMessagePolicy': u'File', u'command': [u'tail', u'-F', u'/var/log/astaire/astaire_current.txt'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}]}, u'metadata': {u'labels': {u'service': u'astaire'}, u'creationTimestamp': None}}, u'revisionHistoryLimit': 10, u'selector': {u'matchLabels': {u'service': u'astaire'}}}, u'apiVersion': u'apps/v1', u'metadata': {u'name': u'astaire', u'generation': 1, u'labels': {u'app.kubernetes.io/managed-by': u'Helm'}, u'namespace': u'ims-w9fzp', u'resourceVersion': u'11211827', u'creationTimestamp': u'2020-11-27T11:04:21Z', u'annotations': {u'meta.helm.sh/release-name': u'clearwater', u'deployment.kubernetes.io/revision': u'1', u'meta.helm.sh/release-namespace': u'ims-w9fzp'}, u'selfLink': u'/apis/apps/v1/namespaces/ims-w9fzp/deployments/astaire', u'uid': u'4d633896-30a0-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {u'deployment.kubernetes.io/revision': '1', u'meta.helm.sh/release-name': 'clearwater', u'meta.helm.sh/release-namespace': 'ims-w9fzp'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {u'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'astaire', 'namespace': 'ims-w9fzp', 'owner_references': None, 'resource_version': '11211827', 'self_link': '/apis/apps/v1/namespaces/ims-w9fzp/deployments/astaire', 'uid': '4d633896-30a0-11eb-8302-0242ac120002'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {u'service': 'astaire'}}, 'strategy': {'rolling_update': {'max_surge': u'25%', 'max_unavailable': u'25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {u'service': 'astaire'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-astaire:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': {'post_start': None, 'pre_stop': {'_exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}, 'http_get': None, 'tcp_socket': None}}, 'liveness_probe': {'_exec': None, 'failure_threshold': 9, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 11311}, 'timeout_seconds': 1}, 'name': 'astaire', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 11211, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 11311, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 11311}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/astaire', 'mount_propagation': None, 'name': 'astairelogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/astaire/astaire_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/astaire', 'mount_propagation': None, 'name': 'astairelogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 120, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'astairelogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal()), 'message': 'ReplicaSet "astaire-5d44675d59" is progressing.', 'reason': 'ReplicaSetUpdated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': 1, 'unavailable_replicas': 1, 'updated_replicas': 1}}, u'type': u'ADDED'} 2020-11-27 11:04:28,639 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'observedGeneration': 1, u'updatedReplicas': 1, u'unavailableReplicas': 1, u'conditions': [{u'status': u'False', u'lastUpdateTime': u'2020-11-27T11:04:21Z', u'lastTransitionTime': u'2020-11-27T11:04:21Z', u'reason': u'MinimumReplicasUnavailable', u'message': u'Deployment does not have minimum availability.', u'type': u'Available'}, {u'status': u'True', u'lastUpdateTime': u'2020-11-27T11:04:22Z', u'lastTransitionTime': u'2020-11-27T11:04:21Z', u'reason': u'ReplicaSetUpdated', u'message': u'ReplicaSet "homestead-prov-6654f8cc8f" is progressing.', u'type': u'Progressing'}], u'replicas': 1}, u'kind': u'Deployment', u'spec': {u'progressDeadlineSeconds': 600, u'replicas': 1, u'strategy': {u'rollingUpdate': {u'maxSurge': u'25%', u'maxUnavailable': u'25%'}, u'type': u'RollingUpdate'}, u'template': {u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'schedulerName': u'default-scheduler', u'terminationGracePeriodSeconds': 30, u'restartPolicy': u'Always', u'initContainers': [{u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-homestead', u'image': u'busybox:1.28', u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup homestead; do echo waiting for homestead to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'imagePullSecrets': [{}], u'containers': [{u'livenessProbe': {u'timeoutSeconds': 1, u'exec': {u'command': [u'/bin/bash', u'/usr/share/clearwater/bin/poll_homestead-prov.sh']}, u'initialDelaySeconds': 60, u'periodSeconds': 10, u'successThreshold': 1, u'failureThreshold': 3}, u'terminationMessagePath': u'/dev/termination-log', u'name': u'homestead-prov', u'envFrom': [{u'configMapRef': {u'name': u'env-vars'}}], u'image': u'ollivier/clearwater-homestead-prov:hunter', u'terminationMessagePolicy': u'File', u'env': [{u'valueFrom': {u'fieldRef': {u'fieldPath': u'status.podIP', u'apiVersion': u'v1'}}, u'name': u'MY_POD_IP'}], u'imagePullPolicy': u'IfNotPresent', u'readinessProbe': {u'successThreshold': 1, u'periodSeconds': 10, u'failureThreshold': 3, u'timeoutSeconds': 1, u'exec': {u'command': [u'/bin/bash', u'/usr/share/clearwater/bin/poll_homestead-prov.sh']}}, u'ports': [{u'protocol': u'TCP', u'containerPort': 22}, {u'protocol': u'TCP', u'containerPort': 8889}], u'resources': {}}]}, u'metadata': {u'labels': {u'snmp': u'enabled', u'service': u'homestead-prov'}, u'creationTimestamp': None}}, u'revisionHistoryLimit': 10, u'selector': {u'matchLabels': {u'service': u'homestead-prov'}}}, u'apiVersion': u'apps/v1', u'metadata': {u'name': u'homestead-prov', u'generation': 1, u'labels': {u'app.kubernetes.io/managed-by': u'Helm'}, u'namespace': u'ims-w9fzp', u'resourceVersion': u'11211826', u'creationTimestamp': u'2020-11-27T11:04:21Z', u'annotations': {u'meta.helm.sh/release-name': u'clearwater', u'deployment.kubernetes.io/revision': u'1', u'meta.helm.sh/release-namespace': u'ims-w9fzp'}, u'selfLink': u'/apis/apps/v1/namespaces/ims-w9fzp/deployments/homestead-prov', u'uid': u'4d5c5a69-30a0-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {u'deployment.kubernetes.io/revision': '1', u'meta.helm.sh/release-name': 'clearwater', u'meta.helm.sh/release-namespace': 'ims-w9fzp'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {u'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'homestead-prov', 'namespace': 'ims-w9fzp', 'owner_references': None, 'resource_version': '11211826', 'self_link': '/apis/apps/v1/namespaces/ims-w9fzp/deployments/homestead-prov', 'uid': '4d5c5a69-30a0-11eb-8302-0242ac120002'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {u'service': 'homestead-prov'}}, 'strategy': {'rolling_update': {'max_surge': u'25%', 'max_unavailable': u'25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {u'service': 'homestead-prov', u'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-homestead-prov:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 60, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'homestead-prov', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 8889, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup homestead; do echo waiting for homestead to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-homestead', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal()), 'message': 'ReplicaSet "homestead-prov-6654f8cc8f" is progressing.', 'reason': 'ReplicaSetUpdated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': 1, 'unavailable_replicas': 1, 'updated_replicas': 1}}, u'type': u'ADDED'} 2020-11-27 11:04:28,872 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'observedGeneration': 1, u'updatedReplicas': 1, u'unavailableReplicas': 1, u'conditions': [{u'status': u'False', u'lastUpdateTime': u'2020-11-27T11:04:22Z', u'lastTransitionTime': u'2020-11-27T11:04:22Z', u'reason': u'MinimumReplicasUnavailable', u'message': u'Deployment does not have minimum availability.', u'type': u'Available'}, {u'status': u'True', u'lastUpdateTime': u'2020-11-27T11:04:22Z', u'lastTransitionTime': u'2020-11-27T11:04:21Z', u'reason': u'ReplicaSetUpdated', u'message': u'ReplicaSet "homestead-7f8f7d7cc9" is progressing.', u'type': u'Progressing'}], u'replicas': 1}, u'kind': u'Deployment', u'spec': {u'progressDeadlineSeconds': 600, u'replicas': 1, u'strategy': {u'rollingUpdate': {u'maxSurge': u'25%', u'maxUnavailable': u'25%'}, u'type': u'RollingUpdate'}, u'template': {u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'schedulerName': u'default-scheduler', u'terminationGracePeriodSeconds': 30, u'restartPolicy': u'Always', u'volumes': [{u'emptyDir': {}, u'name': u'homesteadlogs'}], u'initContainers': [{u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-cassandra', u'image': u'busybox:1.28', u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}, {u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-astaire', u'image': u'busybox:1.28', u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup astaire; do echo waiting for astaire to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'imagePullSecrets': [{}], u'containers': [{u'livenessProbe': {u'timeoutSeconds': 1, u'exec': {u'command': [u'/bin/bash', u'/usr/share/kubernetes/liveness.sh', u'8888']}, u'initialDelaySeconds': 60, u'periodSeconds': 10, u'successThreshold': 1, u'failureThreshold': 3}, u'terminationMessagePath': u'/dev/termination-log', u'name': u'homestead', u'envFrom': [{u'configMapRef': {u'name': u'env-vars'}}], u'image': u'ollivier/clearwater-homestead:hunter', u'volumeMounts': [{u'mountPath': u'/var/log/homestead', u'name': u'homesteadlogs'}], u'terminationMessagePolicy': u'File', u'env': [{u'valueFrom': {u'fieldRef': {u'fieldPath': u'status.podIP', u'apiVersion': u'v1'}}, u'name': u'MY_POD_IP'}], u'imagePullPolicy': u'IfNotPresent', u'readinessProbe': {u'successThreshold': 1, u'periodSeconds': 10, u'failureThreshold': 3, u'timeoutSeconds': 1, u'exec': {u'command': [u'/bin/bash', u'/usr/share/kubernetes/liveness.sh', u'8888']}}, u'ports': [{u'protocol': u'TCP', u'containerPort': 22}, {u'protocol': u'TCP', u'containerPort': 8888}], u'resources': {}}, {u'terminationMessagePath': u'/dev/termination-log', u'name': u'tailer', u'image': u'busybox:1.28', u'volumeMounts': [{u'mountPath': u'/var/log/homestead', u'name': u'homesteadlogs'}], u'terminationMessagePolicy': u'File', u'command': [u'tail', u'-F', u'/var/log/homestead/homestead_current.txt'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}]}, u'metadata': {u'labels': {u'snmp': u'enabled', u'service': u'homestead'}, u'creationTimestamp': None}}, u'revisionHistoryLimit': 10, u'selector': {u'matchLabels': {u'service': u'homestead'}}}, u'apiVersion': u'apps/v1', u'metadata': {u'name': u'homestead', u'generation': 1, u'labels': {u'app.kubernetes.io/managed-by': u'Helm'}, u'namespace': u'ims-w9fzp', u'resourceVersion': u'11211815', u'creationTimestamp': u'2020-11-27T11:04:21Z', u'annotations': {u'meta.helm.sh/release-name': u'clearwater', u'deployment.kubernetes.io/revision': u'1', u'meta.helm.sh/release-namespace': u'ims-w9fzp'}, u'selfLink': u'/apis/apps/v1/namespaces/ims-w9fzp/deployments/homestead', u'uid': u'4d5dedfa-30a0-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {u'deployment.kubernetes.io/revision': '1', u'meta.helm.sh/release-name': 'clearwater', u'meta.helm.sh/release-namespace': 'ims-w9fzp'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {u'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'homestead', 'namespace': 'ims-w9fzp', 'owner_references': None, 'resource_version': '11211815', 'self_link': '/apis/apps/v1/namespaces/ims-w9fzp/deployments/homestead', 'uid': '4d5dedfa-30a0-11eb-8302-0242ac120002'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {u'service': 'homestead'}}, 'strategy': {'rolling_update': {'max_surge': u'25%', 'max_unavailable': u'25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {u'service': 'homestead', u'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-homestead:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 60, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'homestead', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 8888, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/homestead', 'mount_propagation': None, 'name': 'homesteadlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/homestead/homestead_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/homestead', 'mount_propagation': None, 'name': 'homesteadlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until nslookup astaire; do echo waiting for astaire to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-astaire', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'homesteadlogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal()), 'message': 'ReplicaSet "homestead-7f8f7d7cc9" is progressing.', 'reason': 'ReplicaSetUpdated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': 1, 'unavailable_replicas': 1, 'updated_replicas': 1}}, u'type': u'ADDED'} 2020-11-27 11:04:29,224 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'observedGeneration': 1, u'updatedReplicas': 1, u'unavailableReplicas': 1, u'conditions': [{u'status': u'False', u'lastUpdateTime': u'2020-11-27T11:04:22Z', u'lastTransitionTime': u'2020-11-27T11:04:22Z', u'reason': u'MinimumReplicasUnavailable', u'message': u'Deployment does not have minimum availability.', u'type': u'Available'}, {u'status': u'True', u'lastUpdateTime': u'2020-11-27T11:04:22Z', u'lastTransitionTime': u'2020-11-27T11:04:21Z', u'reason': u'ReplicaSetUpdated', u'message': u'ReplicaSet "bono-658d668944" is progressing.', u'type': u'Progressing'}], u'replicas': 1}, u'kind': u'Deployment', u'spec': {u'progressDeadlineSeconds': 600, u'replicas': 1, u'strategy': {u'rollingUpdate': {u'maxSurge': u'25%', u'maxUnavailable': u'25%'}, u'type': u'RollingUpdate'}, u'template': {u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'schedulerName': u'default-scheduler', u'terminationGracePeriodSeconds': 30, u'restartPolicy': u'Always', u'volumes': [{u'emptyDir': {}, u'name': u'bonologs'}], u'initContainers': [{u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-sprout', u'image': u'busybox:1.28', u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup sprout; do echo waiting for sprout to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'imagePullSecrets': [{}], u'containers': [{u'livenessProbe': {u'timeoutSeconds': 1, u'exec': {u'command': [u'/bin/bash', u'/usr/share/kubernetes/liveness.sh', u'3478 5060 5062']}, u'initialDelaySeconds': 30, u'periodSeconds': 10, u'successThreshold': 1, u'failureThreshold': 3}, u'terminationMessagePath': u'/dev/termination-log', u'name': u'bono', u'envFrom': [{u'configMapRef': {u'name': u'env-vars'}}], u'image': u'ollivier/clearwater-bono:hunter', u'volumeMounts': [{u'mountPath': u'/var/log/bono', u'name': u'bonologs'}], u'terminationMessagePolicy': u'File', u'env': [{u'valueFrom': {u'fieldRef': {u'fieldPath': u'status.podIP', u'apiVersion': u'v1'}}, u'name': u'MY_POD_IP'}], u'imagePullPolicy': u'IfNotPresent', u'readinessProbe': {u'successThreshold': 1, u'periodSeconds': 10, u'failureThreshold': 3, u'timeoutSeconds': 1, u'exec': {u'command': [u'/bin/bash', u'/usr/share/kubernetes/liveness.sh', u'3478 5060 5062']}}, u'ports': [{u'protocol': u'TCP', u'containerPort': 22}, {u'protocol': u'TCP', u'containerPort': 3478}, {u'protocol': u'TCP', u'containerPort': 5060}, {u'protocol': u'TCP', u'containerPort': 5062}, {u'protocol': u'UDP', u'containerPort': 5060}, {u'protocol': u'UDP', u'containerPort': 5062}], u'resources': {}}, {u'terminationMessagePath': u'/dev/termination-log', u'name': u'tailer', u'image': u'busybox:1.28', u'volumeMounts': [{u'mountPath': u'/var/log/bono', u'name': u'bonologs'}], u'terminationMessagePolicy': u'File', u'command': [u'tail', u'-F', u'/var/log/bono/bono_current.txt'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}]}, u'metadata': {u'labels': {u'snmp': u'enabled', u'service': u'bono'}, u'creationTimestamp': None}}, u'revisionHistoryLimit': 10, u'selector': {u'matchLabels': {u'service': u'bono'}}}, u'apiVersion': u'apps/v1', u'metadata': {u'name': u'bono', u'generation': 1, u'labels': {u'app.kubernetes.io/managed-by': u'Helm'}, u'namespace': u'ims-w9fzp', u'resourceVersion': u'11211816', u'creationTimestamp': u'2020-11-27T11:04:21Z', u'annotations': {u'meta.helm.sh/release-name': u'clearwater', u'deployment.kubernetes.io/revision': u'1', u'meta.helm.sh/release-namespace': u'ims-w9fzp'}, u'selfLink': u'/apis/apps/v1/namespaces/ims-w9fzp/deployments/bono', u'uid': u'4d5f05a7-30a0-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {u'deployment.kubernetes.io/revision': '1', u'meta.helm.sh/release-name': 'clearwater', u'meta.helm.sh/release-namespace': 'ims-w9fzp'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {u'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'bono', 'namespace': 'ims-w9fzp', 'owner_references': None, 'resource_version': '11211816', 'self_link': '/apis/apps/v1/namespaces/ims-w9fzp/deployments/bono', 'uid': '4d5f05a7-30a0-11eb-8302-0242ac120002'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {u'service': 'bono'}}, 'strategy': {'rolling_update': {'max_surge': u'25%', 'max_unavailable': u'25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {u'service': 'bono', u'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-bono:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 5060 5062']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'bono', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 3478, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 5060, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 5062, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 5060, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'UDP'}, {'container_port': 5062, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'UDP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 5060 5062']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/bono', 'mount_propagation': None, 'name': 'bonologs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/bono/bono_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/bono', 'mount_propagation': None, 'name': 'bonologs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup sprout; do echo waiting for sprout to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-sprout', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'bonologs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal()), 'message': 'ReplicaSet "bono-658d668944" is progressing.', 'reason': 'ReplicaSetUpdated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': 1, 'unavailable_replicas': 1, 'updated_replicas': 1}}, u'type': u'ADDED'} 2020-11-27 11:04:29,565 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'observedGeneration': 1, u'updatedReplicas': 1, u'unavailableReplicas': 1, u'conditions': [{u'status': u'False', u'lastUpdateTime': u'2020-11-27T11:04:22Z', u'lastTransitionTime': u'2020-11-27T11:04:22Z', u'reason': u'MinimumReplicasUnavailable', u'message': u'Deployment does not have minimum availability.', u'type': u'Available'}, {u'status': u'True', u'lastUpdateTime': u'2020-11-27T11:04:22Z', u'lastTransitionTime': u'2020-11-27T11:04:21Z', u'reason': u'ReplicaSetUpdated', u'message': u'ReplicaSet "ellis-857689fbdd" is progressing.', u'type': u'Progressing'}], u'replicas': 1}, u'kind': u'Deployment', u'spec': {u'progressDeadlineSeconds': 600, u'replicas': 1, u'strategy': {u'rollingUpdate': {u'maxSurge': u'25%', u'maxUnavailable': u'25%'}, u'type': u'RollingUpdate'}, u'template': {u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'schedulerName': u'default-scheduler', u'terminationGracePeriodSeconds': 30, u'restartPolicy': u'Always', u'initContainers': [{u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-homestead-prov', u'image': u'busybox:1.28', u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup homestead-prov; do echo waiting for homestead-prov to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}, {u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-sprout', u'image': u'busybox:1.28', u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup sprout; do echo waiting for sprout to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'imagePullSecrets': [{}], u'containers': [{u'livenessProbe': {u'tcpSocket': {u'port': 80}, u'timeoutSeconds': 1, u'initialDelaySeconds': 30, u'periodSeconds': 10, u'successThreshold': 1, u'failureThreshold': 3}, u'terminationMessagePath': u'/dev/termination-log', u'name': u'ellis', u'envFrom': [{u'configMapRef': {u'name': u'env-vars'}}], u'image': u'ollivier/clearwater-ellis:hunter', u'terminationMessagePolicy': u'File', u'env': [{u'valueFrom': {u'fieldRef': {u'fieldPath': u'status.podIP', u'apiVersion': u'v1'}}, u'name': u'MY_POD_IP'}], u'imagePullPolicy': u'IfNotPresent', u'readinessProbe': {u'successThreshold': 1, u'tcpSocket': {u'port': 80}, u'failureThreshold': 3, u'timeoutSeconds': 1, u'periodSeconds': 10}, u'ports': [{u'protocol': u'TCP', u'containerPort': 22}, {u'protocol': u'TCP', u'containerPort': 80}], u'resources': {}}]}, u'metadata': {u'labels': {u'service': u'ellis'}, u'creationTimestamp': None}}, u'revisionHistoryLimit': 10, u'selector': {u'matchLabels': {u'service': u'ellis'}}}, u'apiVersion': u'apps/v1', u'metadata': {u'name': u'ellis', u'generation': 1, u'labels': {u'app.kubernetes.io/managed-by': u'Helm'}, u'namespace': u'ims-w9fzp', u'resourceVersion': u'11211821', u'creationTimestamp': u'2020-11-27T11:04:21Z', u'annotations': {u'meta.helm.sh/release-name': u'clearwater', u'deployment.kubernetes.io/revision': u'1', u'meta.helm.sh/release-namespace': u'ims-w9fzp'}, u'selfLink': u'/apis/apps/v1/namespaces/ims-w9fzp/deployments/ellis', u'uid': u'4d60c225-30a0-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {u'deployment.kubernetes.io/revision': '1', u'meta.helm.sh/release-name': 'clearwater', u'meta.helm.sh/release-namespace': 'ims-w9fzp'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {u'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'ellis', 'namespace': 'ims-w9fzp', 'owner_references': None, 'resource_version': '11211821', 'self_link': '/apis/apps/v1/namespaces/ims-w9fzp/deployments/ellis', 'uid': '4d60c225-30a0-11eb-8302-0242ac120002'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {u'service': 'ellis'}}, 'strategy': {'rolling_update': {'max_surge': u'25%', 'max_unavailable': u'25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {u'service': 'ellis'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-ellis:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 80}, 'timeout_seconds': 1}, 'name': 'ellis', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 80, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 80}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup homestead-prov; do echo waiting for homestead-prov to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-homestead-prov', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until nslookup sprout; do echo waiting for sprout to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-sprout', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal()), 'message': 'ReplicaSet "ellis-857689fbdd" is progressing.', 'reason': 'ReplicaSetUpdated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': 1, 'unavailable_replicas': 1, 'updated_replicas': 1}}, u'type': u'ADDED'} 2020-11-27 11:04:29,942 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'observedGeneration': 1, u'updatedReplicas': 1, u'unavailableReplicas': 1, u'conditions': [{u'status': u'False', u'lastUpdateTime': u'2020-11-27T11:04:22Z', u'lastTransitionTime': u'2020-11-27T11:04:22Z', u'reason': u'MinimumReplicasUnavailable', u'message': u'Deployment does not have minimum availability.', u'type': u'Available'}, {u'status': u'True', u'lastUpdateTime': u'2020-11-27T11:04:22Z', u'lastTransitionTime': u'2020-11-27T11:04:21Z', u'reason': u'ReplicaSetUpdated', u'message': u'ReplicaSet "chronos-b74775b9" is progressing.', u'type': u'Progressing'}], u'replicas': 1}, u'kind': u'Deployment', u'spec': {u'progressDeadlineSeconds': 600, u'replicas': 1, u'strategy': {u'rollingUpdate': {u'maxSurge': u'25%', u'maxUnavailable': u'25%'}, u'type': u'RollingUpdate'}, u'template': {u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'schedulerName': u'default-scheduler', u'terminationGracePeriodSeconds': 120, u'restartPolicy': u'Always', u'volumes': [{u'emptyDir': {}, u'name': u'chronoslogs'}], u'initContainers': [{u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-cassandra', u'image': u'busybox:1.28', u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'imagePullSecrets': [{}], u'containers': [{u'livenessProbe': {u'successThreshold': 1, u'tcpSocket': {u'port': 7253}, u'failureThreshold': 9, u'timeoutSeconds': 1, u'periodSeconds': 10}, u'terminationMessagePath': u'/dev/termination-log', u'name': u'chronos', u'envFrom': [{u'configMapRef': {u'name': u'env-vars'}}], u'image': u'ollivier/clearwater-chronos:hunter', u'volumeMounts': [{u'mountPath': u'/var/log/chronos', u'name': u'chronoslogs'}], u'terminationMessagePolicy': u'File', u'lifecycle': {u'preStop': {u'exec': {u'command': [u'/bin/bash', u'-c', u'/usr/bin/pre-stop']}}}, u'env': [{u'valueFrom': {u'fieldRef': {u'fieldPath': u'status.podIP', u'apiVersion': u'v1'}}, u'name': u'MY_POD_IP'}], u'imagePullPolicy': u'IfNotPresent', u'readinessProbe': {u'successThreshold': 1, u'tcpSocket': {u'port': 7253}, u'failureThreshold': 3, u'timeoutSeconds': 1, u'periodSeconds': 10}, u'ports': [{u'protocol': u'TCP', u'containerPort': 22}, {u'protocol': u'TCP', u'containerPort': 7253}], u'resources': {}}, {u'terminationMessagePath': u'/dev/termination-log', u'name': u'tailer', u'image': u'busybox:1.28', u'volumeMounts': [{u'mountPath': u'/var/log/chronos', u'name': u'chronoslogs'}], u'terminationMessagePolicy': u'File', u'command': [u'tail', u'-F', u'/var/log/chronos/chronos_current.txt'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}]}, u'metadata': {u'labels': {u'service': u'chronos'}, u'creationTimestamp': None}}, u'revisionHistoryLimit': 10, u'selector': {u'matchLabels': {u'service': u'chronos'}}}, u'apiVersion': u'apps/v1', u'metadata': {u'name': u'chronos', u'generation': 1, u'labels': {u'service': u'chronos', u'app.kubernetes.io/managed-by': u'Helm'}, u'namespace': u'ims-w9fzp', u'resourceVersion': u'11211817', u'creationTimestamp': u'2020-11-27T11:04:21Z', u'annotations': {u'meta.helm.sh/release-name': u'clearwater', u'deployment.kubernetes.io/revision': u'1', u'meta.helm.sh/release-namespace': u'ims-w9fzp'}, u'selfLink': u'/apis/apps/v1/namespaces/ims-w9fzp/deployments/chronos', u'uid': u'4d622a8c-30a0-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {u'deployment.kubernetes.io/revision': '1', u'meta.helm.sh/release-name': 'clearwater', u'meta.helm.sh/release-namespace': 'ims-w9fzp'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {u'app.kubernetes.io/managed-by': 'Helm', u'service': 'chronos'}, 'managed_fields': None, 'name': 'chronos', 'namespace': 'ims-w9fzp', 'owner_references': None, 'resource_version': '11211817', 'self_link': '/apis/apps/v1/namespaces/ims-w9fzp/deployments/chronos', 'uid': '4d622a8c-30a0-11eb-8302-0242ac120002'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {u'service': 'chronos'}}, 'strategy': {'rolling_update': {'max_surge': u'25%', 'max_unavailable': u'25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {u'service': 'chronos'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-chronos:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': {'post_start': None, 'pre_stop': {'_exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}, 'http_get': None, 'tcp_socket': None}}, 'liveness_probe': {'_exec': None, 'failure_threshold': 9, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7253}, 'timeout_seconds': 1}, 'name': 'chronos', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 7253, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7253}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/chronos', 'mount_propagation': None, 'name': 'chronoslogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/chronos/chronos_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/chronos', 'mount_propagation': None, 'name': 'chronoslogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 120, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'chronoslogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal()), 'message': 'ReplicaSet "chronos-b74775b9" is progressing.', 'reason': 'ReplicaSetUpdated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': 1, 'unavailable_replicas': 1, 'updated_replicas': 1}}, u'type': u'ADDED'} 2020-11-27 11:04:40,607 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'replicas': 1, u'observedGeneration': 1, u'updatedReplicas': 1, u'availableReplicas': 1, u'readyReplicas': 1, u'conditions': [{u'status': u'True', u'lastUpdateTime': u'2020-11-27T11:04:40Z', u'lastTransitionTime': u'2020-11-27T11:04:40Z', u'reason': u'MinimumReplicasAvailable', u'message': u'Deployment has minimum availability.', u'type': u'Available'}, {u'status': u'True', u'lastUpdateTime': u'2020-11-27T11:04:40Z', u'lastTransitionTime': u'2020-11-27T11:04:21Z', u'reason': u'NewReplicaSetAvailable', u'message': u'ReplicaSet "etcd-c8fddbb95" has successfully progressed.', u'type': u'Progressing'}]}, u'kind': u'Deployment', u'spec': {u'progressDeadlineSeconds': 600, u'replicas': 1, u'strategy': {u'rollingUpdate': {u'maxSurge': u'25%', u'maxUnavailable': u'25%'}, u'type': u'RollingUpdate'}, u'template': {u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'schedulerName': u'default-scheduler', u'terminationGracePeriodSeconds': 30, u'restartPolicy': u'Always', u'imagePullSecrets': [{}], u'containers': [{u'livenessProbe': {u'tcpSocket': {u'port': 4001}, u'timeoutSeconds': 1, u'initialDelaySeconds': 300, u'periodSeconds': 10, u'successThreshold': 1, u'failureThreshold': 3}, u'terminationMessagePath': u'/dev/termination-log', u'name': u'etcd', u'image': u'quay.io/coreos/etcd:v2.2.5', u'args': [u'--name', u'$(MY_POD_NAME)', u'--advertise-client-urls', u'http://$(MY_POD_IP):2379,http://$(MY_POD_IP):4001', u'--listen-client-urls', u'http://0.0.0.0:2379,http://0.0.0.0:4001', u'--initial-advertise-peer-urls', u'http://$(MY_POD_IP):2380', u'--listen-peer-urls', u'http://0.0.0.0:2380', u'--initial-cluster', u'$(MY_POD_NAME)=http://$(MY_POD_IP):2380', u'--initial-cluster-state', u'new'], u'terminationMessagePolicy': u'File', u'env': [{u'valueFrom': {u'fieldRef': {u'fieldPath': u'status.podIP', u'apiVersion': u'v1'}}, u'name': u'MY_POD_IP'}, {u'valueFrom': {u'fieldRef': {u'fieldPath': u'metadata.name', u'apiVersion': u'v1'}}, u'name': u'MY_POD_NAME'}], u'imagePullPolicy': u'IfNotPresent', u'readinessProbe': {u'successThreshold': 1, u'tcpSocket': {u'port': 4001}, u'failureThreshold': 3, u'timeoutSeconds': 1, u'periodSeconds': 10}, u'ports': [{u'protocol': u'TCP', u'containerPort': 2379}, {u'protocol': u'TCP', u'containerPort': 2380}, {u'protocol': u'TCP', u'containerPort': 4001}], u'resources': {}}]}, u'metadata': {u'labels': {u'instance-type': u'etcd-pod'}, u'creationTimestamp': None}}, u'revisionHistoryLimit': 10, u'selector': {u'matchLabels': {u'instance-type': u'etcd-pod'}}}, u'apiVersion': u'apps/v1', u'metadata': {u'name': u'etcd', u'generation': 1, u'labels': {u'app.kubernetes.io/managed-by': u'Helm'}, u'namespace': u'ims-w9fzp', u'resourceVersion': u'11211927', u'creationTimestamp': u'2020-11-27T11:04:21Z', u'annotations': {u'meta.helm.sh/release-name': u'clearwater', u'deployment.kubernetes.io/revision': u'1', u'meta.helm.sh/release-namespace': u'ims-w9fzp'}, u'selfLink': u'/apis/apps/v1/namespaces/ims-w9fzp/deployments/etcd', u'uid': u'4d5bf704-30a0-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {u'deployment.kubernetes.io/revision': '1', u'meta.helm.sh/release-name': 'clearwater', u'meta.helm.sh/release-namespace': 'ims-w9fzp'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {u'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'etcd', 'namespace': 'ims-w9fzp', 'owner_references': None, 'resource_version': '11211927', 'self_link': '/apis/apps/v1/namespaces/ims-w9fzp/deployments/etcd', 'uid': '4d5bf704-30a0-11eb-8302-0242ac120002'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {u'instance-type': 'etcd-pod'}}, 'strategy': {'rolling_update': {'max_surge': u'25%', 'max_unavailable': u'25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {u'instance-type': 'etcd-pod'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': ['--name', '$(MY_POD_NAME)', '--advertise-client-urls', 'http://$(MY_POD_IP):2379,http://$(MY_POD_IP):4001', '--listen-client-urls', 'http://0.0.0.0:2379,http://0.0.0.0:4001', '--initial-advertise-peer-urls', 'http://$(MY_POD_IP):2380', '--listen-peer-urls', 'http://0.0.0.0:2380', '--initial-cluster', '$(MY_POD_NAME)=http://$(MY_POD_IP):2380', '--initial-cluster-state', 'new'], 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}, {'name': 'MY_POD_NAME', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'metadata.name'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': None, 'image': 'quay.io/coreos/etcd:v2.2.5', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 300, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 4001}, 'timeout_seconds': 1}, 'name': 'etcd', 'ports': [{'container_port': 2379, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 2380, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 4001, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 4001}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': None, 'node_name': None, 'node_selector': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': 1, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 40, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 4, 40, tzinfo=tzlocal()), 'message': 'Deployment has minimum availability.', 'reason': 'MinimumReplicasAvailable', 'status': 'True', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 4, 40, tzinfo=tzlocal()), 'message': 'ReplicaSet "etcd-c8fddbb95" has successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': 1, 'replicas': 1, 'unavailable_replicas': None, 'updated_replicas': 1}}, u'type': u'MODIFIED'} 2020-11-27 11:04:40,829 - functest_kubernetes.ims.ims - INFO - etcd started in 79.53 sec 2020-11-27 11:06:10,557 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'replicas': 1, u'observedGeneration': 1, u'updatedReplicas': 1, u'availableReplicas': 1, u'readyReplicas': 1, u'conditions': [{u'status': u'True', u'lastUpdateTime': u'2020-11-27T11:06:10Z', u'lastTransitionTime': u'2020-11-27T11:06:10Z', u'reason': u'MinimumReplicasAvailable', u'message': u'Deployment has minimum availability.', u'type': u'Available'}, {u'status': u'True', u'lastUpdateTime': u'2020-11-27T11:06:10Z', u'lastTransitionTime': u'2020-11-27T11:04:21Z', u'reason': u'NewReplicaSetAvailable', u'message': u'ReplicaSet "cassandra-56f89b7f94" has successfully progressed.', u'type': u'Progressing'}]}, u'kind': u'Deployment', u'spec': {u'progressDeadlineSeconds': 600, u'replicas': 1, u'strategy': {u'rollingUpdate': {u'maxSurge': u'25%', u'maxUnavailable': u'25%'}, u'type': u'RollingUpdate'}, u'template': {u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'schedulerName': u'default-scheduler', u'terminationGracePeriodSeconds': 30, u'restartPolicy': u'Always', u'initContainers': [{u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-etcd', u'image': u'busybox:1.28', u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup etcd; do echo waiting for etcd to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'imagePullSecrets': [{}], u'containers': [{u'livenessProbe': {u'timeoutSeconds': 1, u'exec': {u'command': [u'/bin/bash', u'/usr/share/kubernetes/liveness.sh', u'7000 9042 9160']}, u'initialDelaySeconds': 600, u'periodSeconds': 10, u'successThreshold': 1, u'failureThreshold': 3}, u'terminationMessagePath': u'/dev/termination-log', u'name': u'cassandra', u'envFrom': [{u'configMapRef': {u'name': u'env-vars'}}], u'image': u'ollivier/clearwater-cassandra:hunter', u'terminationMessagePolicy': u'File', u'env': [{u'valueFrom': {u'fieldRef': {u'fieldPath': u'status.podIP', u'apiVersion': u'v1'}}, u'name': u'MY_POD_IP'}], u'imagePullPolicy': u'IfNotPresent', u'readinessProbe': {u'successThreshold': 1, u'periodSeconds': 10, u'failureThreshold': 3, u'timeoutSeconds': 1, u'exec': {u'command': [u'/bin/bash', u'/usr/share/kubernetes/liveness.sh', u'7000 9042 9160']}}, u'ports': [{u'protocol': u'TCP', u'containerPort': 22}, {u'protocol': u'TCP', u'containerPort': 7001}, {u'protocol': u'TCP', u'containerPort': 9042}, {u'protocol': u'TCP', u'containerPort': 9160}], u'resources': {}}]}, u'metadata': {u'labels': {u'service': u'cassandra'}, u'creationTimestamp': None}}, u'revisionHistoryLimit': 10, u'selector': {u'matchLabels': {u'service': u'cassandra'}}}, u'apiVersion': u'apps/v1', u'metadata': {u'name': u'cassandra', u'generation': 1, u'labels': {u'app.kubernetes.io/managed-by': u'Helm'}, u'namespace': u'ims-w9fzp', u'resourceVersion': u'11212170', u'creationTimestamp': u'2020-11-27T11:04:21Z', u'annotations': {u'meta.helm.sh/release-name': u'clearwater', u'deployment.kubernetes.io/revision': u'1', u'meta.helm.sh/release-namespace': u'ims-w9fzp'}, u'selfLink': u'/apis/apps/v1/namespaces/ims-w9fzp/deployments/cassandra', u'uid': u'4d5a0df6-30a0-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {u'deployment.kubernetes.io/revision': '1', u'meta.helm.sh/release-name': 'clearwater', u'meta.helm.sh/release-namespace': 'ims-w9fzp'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {u'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'cassandra', 'namespace': 'ims-w9fzp', 'owner_references': None, 'resource_version': '11212170', 'self_link': '/apis/apps/v1/namespaces/ims-w9fzp/deployments/cassandra', 'uid': '4d5a0df6-30a0-11eb-8302-0242ac120002'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {u'service': 'cassandra'}}, 'strategy': {'rolling_update': {'max_surge': u'25%', 'max_unavailable': u'25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {u'service': 'cassandra'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-cassandra:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 9042 9160']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 600, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'cassandra', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 7001, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 9042, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 9160, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 9042 9160']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup etcd; do echo waiting for etcd to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-etcd', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': 1, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 11, 27, 11, 6, 10, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 6, 10, tzinfo=tzlocal()), 'message': 'Deployment has minimum availability.', 'reason': 'MinimumReplicasAvailable', 'status': 'True', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 6, 10, tzinfo=tzlocal()), 'message': 'ReplicaSet "cassandra-56f89b7f94" has successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': 1, 'replicas': 1, 'unavailable_replicas': None, 'updated_replicas': 1}}, u'type': u'MODIFIED'} 2020-11-27 11:06:10,808 - functest_kubernetes.ims.ims - INFO - cassandra started in 169.51 sec 2020-11-27 11:06:49,547 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'replicas': 1, u'observedGeneration': 1, u'updatedReplicas': 1, u'availableReplicas': 1, u'readyReplicas': 1, u'conditions': [{u'status': u'True', u'lastUpdateTime': u'2020-11-27T11:06:49Z', u'lastTransitionTime': u'2020-11-27T11:06:49Z', u'reason': u'MinimumReplicasAvailable', u'message': u'Deployment has minimum availability.', u'type': u'Available'}, {u'status': u'True', u'lastUpdateTime': u'2020-11-27T11:06:49Z', u'lastTransitionTime': u'2020-11-27T11:04:21Z', u'reason': u'NewReplicaSetAvailable', u'message': u'ReplicaSet "ralf-797bfb6988" has successfully progressed.', u'type': u'Progressing'}]}, u'kind': u'Deployment', u'spec': {u'progressDeadlineSeconds': 600, u'replicas': 1, u'strategy': {u'rollingUpdate': {u'maxSurge': u'25%', u'maxUnavailable': u'25%'}, u'type': u'RollingUpdate'}, u'template': {u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'schedulerName': u'default-scheduler', u'terminationGracePeriodSeconds': 30, u'restartPolicy': u'Always', u'volumes': [{u'emptyDir': {}, u'name': u'ralflogs'}], u'initContainers': [{u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-cassandra', u'image': u'busybox:1.28', u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'imagePullSecrets': [{}], u'containers': [{u'livenessProbe': {u'tcpSocket': {u'port': 10888}, u'timeoutSeconds': 1, u'initialDelaySeconds': 30, u'periodSeconds': 10, u'successThreshold': 1, u'failureThreshold': 3}, u'terminationMessagePath': u'/dev/termination-log', u'name': u'ralf', u'envFrom': [{u'configMapRef': {u'name': u'env-vars'}}], u'image': u'ollivier/clearwater-ralf:hunter', u'volumeMounts': [{u'mountPath': u'/var/log/ralf', u'name': u'ralflogs'}], u'terminationMessagePolicy': u'File', u'env': [{u'valueFrom': {u'fieldRef': {u'fieldPath': u'status.podIP', u'apiVersion': u'v1'}}, u'name': u'MY_POD_IP'}], u'imagePullPolicy': u'IfNotPresent', u'readinessProbe': {u'successThreshold': 1, u'tcpSocket': {u'port': 10888}, u'failureThreshold': 3, u'timeoutSeconds': 1, u'periodSeconds': 10}, u'ports': [{u'protocol': u'TCP', u'containerPort': 22}, {u'protocol': u'TCP', u'containerPort': 10888}], u'resources': {}}, {u'terminationMessagePath': u'/dev/termination-log', u'name': u'tailer', u'image': u'busybox:1.28', u'volumeMounts': [{u'mountPath': u'/var/log/ralf', u'name': u'ralflogs'}], u'terminationMessagePolicy': u'File', u'command': [u'tail', u'-F', u'/var/log/ralf/ralf_current.txt'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}]}, u'metadata': {u'labels': {u'snmp': u'enabled', u'service': u'ralf'}, u'creationTimestamp': None}}, u'revisionHistoryLimit': 10, u'selector': {u'matchLabels': {u'service': u'ralf'}}}, u'apiVersion': u'apps/v1', u'metadata': {u'name': u'ralf', u'generation': 1, u'labels': {u'app.kubernetes.io/managed-by': u'Helm'}, u'namespace': u'ims-w9fzp', u'resourceVersion': u'11212301', u'creationTimestamp': u'2020-11-27T11:04:21Z', u'annotations': {u'meta.helm.sh/release-name': u'clearwater', u'deployment.kubernetes.io/revision': u'1', u'meta.helm.sh/release-namespace': u'ims-w9fzp'}, u'selfLink': u'/apis/apps/v1/namespaces/ims-w9fzp/deployments/ralf', u'uid': u'4d62aa90-30a0-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {u'deployment.kubernetes.io/revision': '1', u'meta.helm.sh/release-name': 'clearwater', u'meta.helm.sh/release-namespace': 'ims-w9fzp'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {u'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'ralf', 'namespace': 'ims-w9fzp', 'owner_references': None, 'resource_version': '11212301', 'self_link': '/apis/apps/v1/namespaces/ims-w9fzp/deployments/ralf', 'uid': '4d62aa90-30a0-11eb-8302-0242ac120002'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {u'service': 'ralf'}}, 'strategy': {'rolling_update': {'max_surge': u'25%', 'max_unavailable': u'25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {u'service': 'ralf', u'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-ralf:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 10888}, 'timeout_seconds': 1}, 'name': 'ralf', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 10888, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 10888}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/ralf', 'mount_propagation': None, 'name': 'ralflogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/ralf/ralf_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/ralf', 'mount_propagation': None, 'name': 'ralflogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'ralflogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': 1, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 11, 27, 11, 6, 49, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 6, 49, tzinfo=tzlocal()), 'message': 'Deployment has minimum availability.', 'reason': 'MinimumReplicasAvailable', 'status': 'True', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 6, 49, tzinfo=tzlocal()), 'message': 'ReplicaSet "ralf-797bfb6988" has successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': 1, 'replicas': 1, 'unavailable_replicas': None, 'updated_replicas': 1}}, u'type': u'MODIFIED'} 2020-11-27 11:06:49,854 - functest_kubernetes.ims.ims - INFO - ralf started in 208.56 sec 2020-11-27 11:06:57,395 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'replicas': 1, u'observedGeneration': 1, u'updatedReplicas': 1, u'availableReplicas': 1, u'readyReplicas': 1, u'conditions': [{u'status': u'True', u'lastUpdateTime': u'2020-11-27T11:06:57Z', u'lastTransitionTime': u'2020-11-27T11:06:57Z', u'reason': u'MinimumReplicasAvailable', u'message': u'Deployment has minimum availability.', u'type': u'Available'}, {u'status': u'True', u'lastUpdateTime': u'2020-11-27T11:06:57Z', u'lastTransitionTime': u'2020-11-27T11:04:21Z', u'reason': u'NewReplicaSetAvailable', u'message': u'ReplicaSet "homer-65d4f8455b" has successfully progressed.', u'type': u'Progressing'}]}, u'kind': u'Deployment', u'spec': {u'progressDeadlineSeconds': 600, u'replicas': 1, u'strategy': {u'rollingUpdate': {u'maxSurge': u'25%', u'maxUnavailable': u'25%'}, u'type': u'RollingUpdate'}, u'template': {u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'schedulerName': u'default-scheduler', u'terminationGracePeriodSeconds': 30, u'restartPolicy': u'Always', u'initContainers': [{u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-cassandra', u'image': u'busybox:1.28', u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'imagePullSecrets': [{}], u'containers': [{u'livenessProbe': {u'tcpSocket': {u'port': 7888}, u'timeoutSeconds': 1, u'initialDelaySeconds': 30, u'periodSeconds': 10, u'successThreshold': 1, u'failureThreshold': 3}, u'terminationMessagePath': u'/dev/termination-log', u'name': u'homer', u'envFrom': [{u'configMapRef': {u'name': u'env-vars'}}], u'image': u'ollivier/clearwater-homer:hunter', u'terminationMessagePolicy': u'File', u'env': [{u'valueFrom': {u'fieldRef': {u'fieldPath': u'status.podIP', u'apiVersion': u'v1'}}, u'name': u'MY_POD_IP'}], u'imagePullPolicy': u'IfNotPresent', u'readinessProbe': {u'successThreshold': 1, u'tcpSocket': {u'port': 7888}, u'failureThreshold': 3, u'timeoutSeconds': 1, u'periodSeconds': 10}, u'ports': [{u'protocol': u'TCP', u'containerPort': 22}, {u'protocol': u'TCP', u'containerPort': 7888}], u'resources': {}}]}, u'metadata': {u'labels': {u'service': u'homer'}, u'creationTimestamp': None}}, u'revisionHistoryLimit': 10, u'selector': {u'matchLabels': {u'service': u'homer'}}}, u'apiVersion': u'apps/v1', u'metadata': {u'name': u'homer', u'generation': 1, u'labels': {u'app.kubernetes.io/managed-by': u'Helm'}, u'namespace': u'ims-w9fzp', u'resourceVersion': u'11212331', u'creationTimestamp': u'2020-11-27T11:04:21Z', u'annotations': {u'meta.helm.sh/release-name': u'clearwater', u'deployment.kubernetes.io/revision': u'1', u'meta.helm.sh/release-namespace': u'ims-w9fzp'}, u'selfLink': u'/apis/apps/v1/namespaces/ims-w9fzp/deployments/homer', u'uid': u'4d5d0a0f-30a0-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {u'deployment.kubernetes.io/revision': '1', u'meta.helm.sh/release-name': 'clearwater', u'meta.helm.sh/release-namespace': 'ims-w9fzp'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {u'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'homer', 'namespace': 'ims-w9fzp', 'owner_references': None, 'resource_version': '11212331', 'self_link': '/apis/apps/v1/namespaces/ims-w9fzp/deployments/homer', 'uid': '4d5d0a0f-30a0-11eb-8302-0242ac120002'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {u'service': 'homer'}}, 'strategy': {'rolling_update': {'max_surge': u'25%', 'max_unavailable': u'25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {u'service': 'homer'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-homer:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7888}, 'timeout_seconds': 1}, 'name': 'homer', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 7888, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7888}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': 1, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 11, 27, 11, 6, 57, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 6, 57, tzinfo=tzlocal()), 'message': 'Deployment has minimum availability.', 'reason': 'MinimumReplicasAvailable', 'status': 'True', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 6, 57, tzinfo=tzlocal()), 'message': 'ReplicaSet "homer-65d4f8455b" has successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': 1, 'replicas': 1, 'unavailable_replicas': None, 'updated_replicas': 1}}, u'type': u'MODIFIED'} 2020-11-27 11:06:57,610 - functest_kubernetes.ims.ims - INFO - homer started in 216.31 sec 2020-11-27 11:07:00,718 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'replicas': 1, u'observedGeneration': 1, u'updatedReplicas': 1, u'availableReplicas': 1, u'readyReplicas': 1, u'conditions': [{u'status': u'True', u'lastUpdateTime': u'2020-11-27T11:07:00Z', u'lastTransitionTime': u'2020-11-27T11:07:00Z', u'reason': u'MinimumReplicasAvailable', u'message': u'Deployment has minimum availability.', u'type': u'Available'}, {u'status': u'True', u'lastUpdateTime': u'2020-11-27T11:07:00Z', u'lastTransitionTime': u'2020-11-27T11:04:21Z', u'reason': u'NewReplicaSetAvailable', u'message': u'ReplicaSet "chronos-b74775b9" has successfully progressed.', u'type': u'Progressing'}]}, u'kind': u'Deployment', u'spec': {u'progressDeadlineSeconds': 600, u'replicas': 1, u'strategy': {u'rollingUpdate': {u'maxSurge': u'25%', u'maxUnavailable': u'25%'}, u'type': u'RollingUpdate'}, u'template': {u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'schedulerName': u'default-scheduler', u'terminationGracePeriodSeconds': 120, u'restartPolicy': u'Always', u'volumes': [{u'emptyDir': {}, u'name': u'chronoslogs'}], u'initContainers': [{u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-cassandra', u'image': u'busybox:1.28', u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'imagePullSecrets': [{}], u'containers': [{u'livenessProbe': {u'successThreshold': 1, u'tcpSocket': {u'port': 7253}, u'failureThreshold': 9, u'timeoutSeconds': 1, u'periodSeconds': 10}, u'terminationMessagePath': u'/dev/termination-log', u'name': u'chronos', u'envFrom': [{u'configMapRef': {u'name': u'env-vars'}}], u'image': u'ollivier/clearwater-chronos:hunter', u'volumeMounts': [{u'mountPath': u'/var/log/chronos', u'name': u'chronoslogs'}], u'terminationMessagePolicy': u'File', u'lifecycle': {u'preStop': {u'exec': {u'command': [u'/bin/bash', u'-c', u'/usr/bin/pre-stop']}}}, u'env': [{u'valueFrom': {u'fieldRef': {u'fieldPath': u'status.podIP', u'apiVersion': u'v1'}}, u'name': u'MY_POD_IP'}], u'imagePullPolicy': u'IfNotPresent', u'readinessProbe': {u'successThreshold': 1, u'tcpSocket': {u'port': 7253}, u'failureThreshold': 3, u'timeoutSeconds': 1, u'periodSeconds': 10}, u'ports': [{u'protocol': u'TCP', u'containerPort': 22}, {u'protocol': u'TCP', u'containerPort': 7253}], u'resources': {}}, {u'terminationMessagePath': u'/dev/termination-log', u'name': u'tailer', u'image': u'busybox:1.28', u'volumeMounts': [{u'mountPath': u'/var/log/chronos', u'name': u'chronoslogs'}], u'terminationMessagePolicy': u'File', u'command': [u'tail', u'-F', u'/var/log/chronos/chronos_current.txt'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}]}, u'metadata': {u'labels': {u'service': u'chronos'}, u'creationTimestamp': None}}, u'revisionHistoryLimit': 10, u'selector': {u'matchLabels': {u'service': u'chronos'}}}, u'apiVersion': u'apps/v1', u'metadata': {u'name': u'chronos', u'generation': 1, u'labels': {u'service': u'chronos', u'app.kubernetes.io/managed-by': u'Helm'}, u'namespace': u'ims-w9fzp', u'resourceVersion': u'11212341', u'creationTimestamp': u'2020-11-27T11:04:21Z', u'annotations': {u'meta.helm.sh/release-name': u'clearwater', u'deployment.kubernetes.io/revision': u'1', u'meta.helm.sh/release-namespace': u'ims-w9fzp'}, u'selfLink': u'/apis/apps/v1/namespaces/ims-w9fzp/deployments/chronos', u'uid': u'4d622a8c-30a0-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {u'deployment.kubernetes.io/revision': '1', u'meta.helm.sh/release-name': 'clearwater', u'meta.helm.sh/release-namespace': 'ims-w9fzp'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {u'app.kubernetes.io/managed-by': 'Helm', u'service': 'chronos'}, 'managed_fields': None, 'name': 'chronos', 'namespace': 'ims-w9fzp', 'owner_references': None, 'resource_version': '11212341', 'self_link': '/apis/apps/v1/namespaces/ims-w9fzp/deployments/chronos', 'uid': '4d622a8c-30a0-11eb-8302-0242ac120002'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {u'service': 'chronos'}}, 'strategy': {'rolling_update': {'max_surge': u'25%', 'max_unavailable': u'25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {u'service': 'chronos'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-chronos:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': {'post_start': None, 'pre_stop': {'_exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}, 'http_get': None, 'tcp_socket': None}}, 'liveness_probe': {'_exec': None, 'failure_threshold': 9, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7253}, 'timeout_seconds': 1}, 'name': 'chronos', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 7253, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7253}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/chronos', 'mount_propagation': None, 'name': 'chronoslogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/chronos/chronos_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/chronos', 'mount_propagation': None, 'name': 'chronoslogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 120, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'chronoslogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': 1, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 11, 27, 11, 7, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 7, tzinfo=tzlocal()), 'message': 'Deployment has minimum availability.', 'reason': 'MinimumReplicasAvailable', 'status': 'True', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 7, tzinfo=tzlocal()), 'message': 'ReplicaSet "chronos-b74775b9" has successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': 1, 'replicas': 1, 'unavailable_replicas': None, 'updated_replicas': 1}}, u'type': u'MODIFIED'} 2020-11-27 11:07:01,038 - functest_kubernetes.ims.ims - INFO - chronos started in 219.74 sec 2020-11-27 11:07:33,841 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'replicas': 1, u'observedGeneration': 1, u'updatedReplicas': 1, u'availableReplicas': 1, u'readyReplicas': 1, u'conditions': [{u'status': u'True', u'lastUpdateTime': u'2020-11-27T11:07:33Z', u'lastTransitionTime': u'2020-11-27T11:07:33Z', u'reason': u'MinimumReplicasAvailable', u'message': u'Deployment has minimum availability.', u'type': u'Available'}, {u'status': u'True', u'lastUpdateTime': u'2020-11-27T11:07:33Z', u'lastTransitionTime': u'2020-11-27T11:04:21Z', u'reason': u'NewReplicaSetAvailable', u'message': u'ReplicaSet "astaire-5d44675d59" has successfully progressed.', u'type': u'Progressing'}]}, u'kind': u'Deployment', u'spec': {u'progressDeadlineSeconds': 600, u'replicas': 1, u'strategy': {u'rollingUpdate': {u'maxSurge': u'25%', u'maxUnavailable': u'25%'}, u'type': u'RollingUpdate'}, u'template': {u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'schedulerName': u'default-scheduler', u'terminationGracePeriodSeconds': 120, u'restartPolicy': u'Always', u'volumes': [{u'emptyDir': {}, u'name': u'astairelogs'}], u'initContainers': [{u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-cassandra', u'image': u'busybox:1.28', u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'imagePullSecrets': [{}], u'containers': [{u'livenessProbe': {u'successThreshold': 1, u'tcpSocket': {u'port': 11311}, u'failureThreshold': 9, u'timeoutSeconds': 1, u'periodSeconds': 10}, u'terminationMessagePath': u'/dev/termination-log', u'name': u'astaire', u'envFrom': [{u'configMapRef': {u'name': u'env-vars'}}], u'image': u'ollivier/clearwater-astaire:hunter', u'volumeMounts': [{u'mountPath': u'/var/log/astaire', u'name': u'astairelogs'}], u'terminationMessagePolicy': u'File', u'lifecycle': {u'preStop': {u'exec': {u'command': [u'/bin/bash', u'-c', u'/usr/bin/pre-stop']}}}, u'env': [{u'valueFrom': {u'fieldRef': {u'fieldPath': u'status.podIP', u'apiVersion': u'v1'}}, u'name': u'MY_POD_IP'}], u'imagePullPolicy': u'IfNotPresent', u'readinessProbe': {u'successThreshold': 1, u'tcpSocket': {u'port': 11311}, u'failureThreshold': 3, u'timeoutSeconds': 1, u'periodSeconds': 10}, u'ports': [{u'protocol': u'TCP', u'containerPort': 22}, {u'protocol': u'TCP', u'containerPort': 11211}, {u'protocol': u'TCP', u'containerPort': 11311}], u'resources': {}}, {u'terminationMessagePath': u'/dev/termination-log', u'name': u'tailer', u'image': u'busybox:1.28', u'volumeMounts': [{u'mountPath': u'/var/log/astaire', u'name': u'astairelogs'}], u'terminationMessagePolicy': u'File', u'command': [u'tail', u'-F', u'/var/log/astaire/astaire_current.txt'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}]}, u'metadata': {u'labels': {u'service': u'astaire'}, u'creationTimestamp': None}}, u'revisionHistoryLimit': 10, u'selector': {u'matchLabels': {u'service': u'astaire'}}}, u'apiVersion': u'apps/v1', u'metadata': {u'name': u'astaire', u'generation': 1, u'labels': {u'app.kubernetes.io/managed-by': u'Helm'}, u'namespace': u'ims-w9fzp', u'resourceVersion': u'11212466', u'creationTimestamp': u'2020-11-27T11:04:21Z', u'annotations': {u'meta.helm.sh/release-name': u'clearwater', u'deployment.kubernetes.io/revision': u'1', u'meta.helm.sh/release-namespace': u'ims-w9fzp'}, u'selfLink': u'/apis/apps/v1/namespaces/ims-w9fzp/deployments/astaire', u'uid': u'4d633896-30a0-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {u'deployment.kubernetes.io/revision': '1', u'meta.helm.sh/release-name': 'clearwater', u'meta.helm.sh/release-namespace': 'ims-w9fzp'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {u'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'astaire', 'namespace': 'ims-w9fzp', 'owner_references': None, 'resource_version': '11212466', 'self_link': '/apis/apps/v1/namespaces/ims-w9fzp/deployments/astaire', 'uid': '4d633896-30a0-11eb-8302-0242ac120002'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {u'service': 'astaire'}}, 'strategy': {'rolling_update': {'max_surge': u'25%', 'max_unavailable': u'25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {u'service': 'astaire'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-astaire:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': {'post_start': None, 'pre_stop': {'_exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}, 'http_get': None, 'tcp_socket': None}}, 'liveness_probe': {'_exec': None, 'failure_threshold': 9, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 11311}, 'timeout_seconds': 1}, 'name': 'astaire', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 11211, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 11311, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 11311}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/astaire', 'mount_propagation': None, 'name': 'astairelogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/astaire/astaire_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/astaire', 'mount_propagation': None, 'name': 'astairelogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 120, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'astairelogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': 1, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 11, 27, 11, 7, 33, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 7, 33, tzinfo=tzlocal()), 'message': 'Deployment has minimum availability.', 'reason': 'MinimumReplicasAvailable', 'status': 'True', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 7, 33, tzinfo=tzlocal()), 'message': 'ReplicaSet "astaire-5d44675d59" has successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': 1, 'replicas': 1, 'unavailable_replicas': None, 'updated_replicas': 1}}, u'type': u'MODIFIED'} 2020-11-27 11:07:34,179 - functest_kubernetes.ims.ims - INFO - astaire started in 252.88 sec 2020-11-27 11:08:04,071 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'replicas': 1, u'observedGeneration': 1, u'updatedReplicas': 1, u'availableReplicas': 1, u'readyReplicas': 1, u'conditions': [{u'status': u'True', u'lastUpdateTime': u'2020-11-27T11:08:03Z', u'lastTransitionTime': u'2020-11-27T11:08:03Z', u'reason': u'MinimumReplicasAvailable', u'message': u'Deployment has minimum availability.', u'type': u'Available'}, {u'status': u'True', u'lastUpdateTime': u'2020-11-27T11:08:03Z', u'lastTransitionTime': u'2020-11-27T11:04:21Z', u'reason': u'NewReplicaSetAvailable', u'message': u'ReplicaSet "homestead-7f8f7d7cc9" has successfully progressed.', u'type': u'Progressing'}]}, u'kind': u'Deployment', u'spec': {u'progressDeadlineSeconds': 600, u'replicas': 1, u'strategy': {u'rollingUpdate': {u'maxSurge': u'25%', u'maxUnavailable': u'25%'}, u'type': u'RollingUpdate'}, u'template': {u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'schedulerName': u'default-scheduler', u'terminationGracePeriodSeconds': 30, u'restartPolicy': u'Always', u'volumes': [{u'emptyDir': {}, u'name': u'homesteadlogs'}], u'initContainers': [{u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-cassandra', u'image': u'busybox:1.28', u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}, {u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-astaire', u'image': u'busybox:1.28', u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup astaire; do echo waiting for astaire to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'imagePullSecrets': [{}], u'containers': [{u'livenessProbe': {u'timeoutSeconds': 1, u'exec': {u'command': [u'/bin/bash', u'/usr/share/kubernetes/liveness.sh', u'8888']}, u'initialDelaySeconds': 60, u'periodSeconds': 10, u'successThreshold': 1, u'failureThreshold': 3}, u'terminationMessagePath': u'/dev/termination-log', u'name': u'homestead', u'envFrom': [{u'configMapRef': {u'name': u'env-vars'}}], u'image': u'ollivier/clearwater-homestead:hunter', u'volumeMounts': [{u'mountPath': u'/var/log/homestead', u'name': u'homesteadlogs'}], u'terminationMessagePolicy': u'File', u'env': [{u'valueFrom': {u'fieldRef': {u'fieldPath': u'status.podIP', u'apiVersion': u'v1'}}, u'name': u'MY_POD_IP'}], u'imagePullPolicy': u'IfNotPresent', u'readinessProbe': {u'successThreshold': 1, u'periodSeconds': 10, u'failureThreshold': 3, u'timeoutSeconds': 1, u'exec': {u'command': [u'/bin/bash', u'/usr/share/kubernetes/liveness.sh', u'8888']}}, u'ports': [{u'protocol': u'TCP', u'containerPort': 22}, {u'protocol': u'TCP', u'containerPort': 8888}], u'resources': {}}, {u'terminationMessagePath': u'/dev/termination-log', u'name': u'tailer', u'image': u'busybox:1.28', u'volumeMounts': [{u'mountPath': u'/var/log/homestead', u'name': u'homesteadlogs'}], u'terminationMessagePolicy': u'File', u'command': [u'tail', u'-F', u'/var/log/homestead/homestead_current.txt'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}]}, u'metadata': {u'labels': {u'snmp': u'enabled', u'service': u'homestead'}, u'creationTimestamp': None}}, u'revisionHistoryLimit': 10, u'selector': {u'matchLabels': {u'service': u'homestead'}}}, u'apiVersion': u'apps/v1', u'metadata': {u'name': u'homestead', u'generation': 1, u'labels': {u'app.kubernetes.io/managed-by': u'Helm'}, u'namespace': u'ims-w9fzp', u'resourceVersion': u'11212540', u'creationTimestamp': u'2020-11-27T11:04:21Z', u'annotations': {u'meta.helm.sh/release-name': u'clearwater', u'deployment.kubernetes.io/revision': u'1', u'meta.helm.sh/release-namespace': u'ims-w9fzp'}, u'selfLink': u'/apis/apps/v1/namespaces/ims-w9fzp/deployments/homestead', u'uid': u'4d5dedfa-30a0-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {u'deployment.kubernetes.io/revision': '1', u'meta.helm.sh/release-name': 'clearwater', u'meta.helm.sh/release-namespace': 'ims-w9fzp'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {u'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'homestead', 'namespace': 'ims-w9fzp', 'owner_references': None, 'resource_version': '11212540', 'self_link': '/apis/apps/v1/namespaces/ims-w9fzp/deployments/homestead', 'uid': '4d5dedfa-30a0-11eb-8302-0242ac120002'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {u'service': 'homestead'}}, 'strategy': {'rolling_update': {'max_surge': u'25%', 'max_unavailable': u'25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {u'service': 'homestead', u'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-homestead:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 60, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'homestead', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 8888, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/homestead', 'mount_propagation': None, 'name': 'homesteadlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/homestead/homestead_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/homestead', 'mount_propagation': None, 'name': 'homesteadlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until nslookup astaire; do echo waiting for astaire to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-astaire', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'homesteadlogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': 1, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 11, 27, 11, 8, 3, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 8, 3, tzinfo=tzlocal()), 'message': 'Deployment has minimum availability.', 'reason': 'MinimumReplicasAvailable', 'status': 'True', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 8, 3, tzinfo=tzlocal()), 'message': 'ReplicaSet "homestead-7f8f7d7cc9" has successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': 1, 'replicas': 1, 'unavailable_replicas': None, 'updated_replicas': 1}}, u'type': u'MODIFIED'} 2020-11-27 11:08:04,402 - functest_kubernetes.ims.ims - INFO - homestead started in 283.10 sec 2020-11-27 11:08:19,393 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'replicas': 1, u'observedGeneration': 1, u'updatedReplicas': 1, u'availableReplicas': 1, u'readyReplicas': 1, u'conditions': [{u'status': u'True', u'lastUpdateTime': u'2020-11-27T11:08:19Z', u'lastTransitionTime': u'2020-11-27T11:08:19Z', u'reason': u'MinimumReplicasAvailable', u'message': u'Deployment has minimum availability.', u'type': u'Available'}, {u'status': u'True', u'lastUpdateTime': u'2020-11-27T11:08:19Z', u'lastTransitionTime': u'2020-11-27T11:04:21Z', u'reason': u'NewReplicaSetAvailable', u'message': u'ReplicaSet "sprout-55bcf5566" has successfully progressed.', u'type': u'Progressing'}]}, u'kind': u'Deployment', u'spec': {u'progressDeadlineSeconds': 600, u'replicas': 1, u'strategy': {u'rollingUpdate': {u'maxSurge': u'25%', u'maxUnavailable': u'25%'}, u'type': u'RollingUpdate'}, u'template': {u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'schedulerName': u'default-scheduler', u'terminationGracePeriodSeconds': 30, u'restartPolicy': u'Always', u'volumes': [{u'emptyDir': {}, u'name': u'sproutlogs'}], u'initContainers': [{u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-ralf', u'image': u'busybox:1.28', u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup ralf; do echo waiting for ralf to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}, {u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-chronos', u'image': u'busybox:1.28', u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup chronos; do echo waiting for chronos to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}, {u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-homestead', u'image': u'busybox:1.28', u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup homestead; do echo waiting for homestead to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'imagePullSecrets': [{}], u'containers': [{u'livenessProbe': {u'timeoutSeconds': 1, u'exec': {u'command': [u'/bin/bash', u'/usr/share/kubernetes/liveness.sh', u'5052 5054']}, u'initialDelaySeconds': 30, u'periodSeconds': 3, u'successThreshold': 1, u'failureThreshold': 3}, u'terminationMessagePath': u'/dev/termination-log', u'name': u'sprout', u'envFrom': [{u'configMapRef': {u'name': u'env-vars'}}], u'image': u'ollivier/clearwater-sprout:hunter', u'volumeMounts': [{u'mountPath': u'/var/log/sprout', u'name': u'sproutlogs'}], u'terminationMessagePolicy': u'File', u'env': [{u'valueFrom': {u'fieldRef': {u'fieldPath': u'status.podIP', u'apiVersion': u'v1'}}, u'name': u'MY_POD_IP'}], u'imagePullPolicy': u'IfNotPresent', u'readinessProbe': {u'successThreshold': 1, u'periodSeconds': 10, u'failureThreshold': 3, u'timeoutSeconds': 1, u'exec': {u'command': [u'/bin/bash', u'/usr/share/kubernetes/liveness.sh', u'5052 5054']}}, u'ports': [{u'protocol': u'TCP', u'containerPort': 22}], u'resources': {}}, {u'terminationMessagePath': u'/dev/termination-log', u'name': u'tailer', u'image': u'busybox:1.28', u'volumeMounts': [{u'mountPath': u'/var/log/sprout', u'name': u'sproutlogs'}], u'terminationMessagePolicy': u'File', u'command': [u'tail', u'-F', u'/var/log/sprout/sprout_current.txt'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}]}, u'metadata': {u'labels': {u'snmp': u'enabled', u'service': u'sprout'}, u'creationTimestamp': None}}, u'revisionHistoryLimit': 10, u'selector': {u'matchLabels': {u'service': u'sprout'}}}, u'apiVersion': u'apps/v1', u'metadata': {u'name': u'sprout', u'generation': 1, u'labels': {u'app.kubernetes.io/managed-by': u'Helm'}, u'namespace': u'ims-w9fzp', u'resourceVersion': u'11212586', u'creationTimestamp': u'2020-11-27T11:04:21Z', u'annotations': {u'meta.helm.sh/release-name': u'clearwater', u'deployment.kubernetes.io/revision': u'1', u'meta.helm.sh/release-namespace': u'ims-w9fzp'}, u'selfLink': u'/apis/apps/v1/namespaces/ims-w9fzp/deployments/sprout', u'uid': u'4d5b1b1d-30a0-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {u'deployment.kubernetes.io/revision': '1', u'meta.helm.sh/release-name': 'clearwater', u'meta.helm.sh/release-namespace': 'ims-w9fzp'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {u'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'sprout', 'namespace': 'ims-w9fzp', 'owner_references': None, 'resource_version': '11212586', 'self_link': '/apis/apps/v1/namespaces/ims-w9fzp/deployments/sprout', 'uid': '4d5b1b1d-30a0-11eb-8302-0242ac120002'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {u'service': 'sprout'}}, 'strategy': {'rolling_update': {'max_surge': u'25%', 'max_unavailable': u'25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {u'service': 'sprout', u'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-sprout:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 5054']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 3, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'sprout', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 5054']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/sprout', 'mount_propagation': None, 'name': 'sproutlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/sprout/sprout_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/sprout', 'mount_propagation': None, 'name': 'sproutlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup ralf; do echo waiting for ralf to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-ralf', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until nslookup chronos; do echo waiting for chronos to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-chronos', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until nslookup homestead; do echo waiting for homestead to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-homestead', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'sproutlogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': 1, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 11, 27, 11, 8, 19, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 8, 19, tzinfo=tzlocal()), 'message': 'Deployment has minimum availability.', 'reason': 'MinimumReplicasAvailable', 'status': 'True', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 8, 19, tzinfo=tzlocal()), 'message': 'ReplicaSet "sprout-55bcf5566" has successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': 1, 'replicas': 1, 'unavailable_replicas': None, 'updated_replicas': 1}}, u'type': u'MODIFIED'} 2020-11-27 11:08:19,761 - functest_kubernetes.ims.ims - INFO - sprout started in 298.46 sec 2020-11-27 11:08:28,393 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'replicas': 1, u'observedGeneration': 1, u'updatedReplicas': 1, u'availableReplicas': 1, u'readyReplicas': 1, u'conditions': [{u'status': u'True', u'lastUpdateTime': u'2020-11-27T11:08:28Z', u'lastTransitionTime': u'2020-11-27T11:08:28Z', u'reason': u'MinimumReplicasAvailable', u'message': u'Deployment has minimum availability.', u'type': u'Available'}, {u'status': u'True', u'lastUpdateTime': u'2020-11-27T11:08:28Z', u'lastTransitionTime': u'2020-11-27T11:04:21Z', u'reason': u'NewReplicaSetAvailable', u'message': u'ReplicaSet "homestead-prov-6654f8cc8f" has successfully progressed.', u'type': u'Progressing'}]}, u'kind': u'Deployment', u'spec': {u'progressDeadlineSeconds': 600, u'replicas': 1, u'strategy': {u'rollingUpdate': {u'maxSurge': u'25%', u'maxUnavailable': u'25%'}, u'type': u'RollingUpdate'}, u'template': {u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'schedulerName': u'default-scheduler', u'terminationGracePeriodSeconds': 30, u'restartPolicy': u'Always', u'initContainers': [{u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-homestead', u'image': u'busybox:1.28', u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup homestead; do echo waiting for homestead to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'imagePullSecrets': [{}], u'containers': [{u'livenessProbe': {u'timeoutSeconds': 1, u'exec': {u'command': [u'/bin/bash', u'/usr/share/clearwater/bin/poll_homestead-prov.sh']}, u'initialDelaySeconds': 60, u'periodSeconds': 10, u'successThreshold': 1, u'failureThreshold': 3}, u'terminationMessagePath': u'/dev/termination-log', u'name': u'homestead-prov', u'envFrom': [{u'configMapRef': {u'name': u'env-vars'}}], u'image': u'ollivier/clearwater-homestead-prov:hunter', u'terminationMessagePolicy': u'File', u'env': [{u'valueFrom': {u'fieldRef': {u'fieldPath': u'status.podIP', u'apiVersion': u'v1'}}, u'name': u'MY_POD_IP'}], u'imagePullPolicy': u'IfNotPresent', u'readinessProbe': {u'successThreshold': 1, u'periodSeconds': 10, u'failureThreshold': 3, u'timeoutSeconds': 1, u'exec': {u'command': [u'/bin/bash', u'/usr/share/clearwater/bin/poll_homestead-prov.sh']}}, u'ports': [{u'protocol': u'TCP', u'containerPort': 22}, {u'protocol': u'TCP', u'containerPort': 8889}], u'resources': {}}]}, u'metadata': {u'labels': {u'snmp': u'enabled', u'service': u'homestead-prov'}, u'creationTimestamp': None}}, u'revisionHistoryLimit': 10, u'selector': {u'matchLabels': {u'service': u'homestead-prov'}}}, u'apiVersion': u'apps/v1', u'metadata': {u'name': u'homestead-prov', u'generation': 1, u'labels': {u'app.kubernetes.io/managed-by': u'Helm'}, u'namespace': u'ims-w9fzp', u'resourceVersion': u'11212614', u'creationTimestamp': u'2020-11-27T11:04:21Z', u'annotations': {u'meta.helm.sh/release-name': u'clearwater', u'deployment.kubernetes.io/revision': u'1', u'meta.helm.sh/release-namespace': u'ims-w9fzp'}, u'selfLink': u'/apis/apps/v1/namespaces/ims-w9fzp/deployments/homestead-prov', u'uid': u'4d5c5a69-30a0-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {u'deployment.kubernetes.io/revision': '1', u'meta.helm.sh/release-name': 'clearwater', u'meta.helm.sh/release-namespace': 'ims-w9fzp'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {u'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'homestead-prov', 'namespace': 'ims-w9fzp', 'owner_references': None, 'resource_version': '11212614', 'self_link': '/apis/apps/v1/namespaces/ims-w9fzp/deployments/homestead-prov', 'uid': '4d5c5a69-30a0-11eb-8302-0242ac120002'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {u'service': 'homestead-prov'}}, 'strategy': {'rolling_update': {'max_surge': u'25%', 'max_unavailable': u'25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {u'service': 'homestead-prov', u'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-homestead-prov:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 60, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'homestead-prov', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 8889, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup homestead; do echo waiting for homestead to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-homestead', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': 1, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 11, 27, 11, 8, 28, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 8, 28, tzinfo=tzlocal()), 'message': 'Deployment has minimum availability.', 'reason': 'MinimumReplicasAvailable', 'status': 'True', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 8, 28, tzinfo=tzlocal()), 'message': 'ReplicaSet "homestead-prov-6654f8cc8f" has successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': 1, 'replicas': 1, 'unavailable_replicas': None, 'updated_replicas': 1}}, u'type': u'MODIFIED'} 2020-11-27 11:08:28,635 - functest_kubernetes.ims.ims - INFO - homestead-prov started in 307.34 sec 2020-11-27 11:08:32,961 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'replicas': 1, u'observedGeneration': 1, u'updatedReplicas': 1, u'availableReplicas': 1, u'readyReplicas': 1, u'conditions': [{u'status': u'True', u'lastUpdateTime': u'2020-11-27T11:08:32Z', u'lastTransitionTime': u'2020-11-27T11:08:32Z', u'reason': u'MinimumReplicasAvailable', u'message': u'Deployment has minimum availability.', u'type': u'Available'}, {u'status': u'True', u'lastUpdateTime': u'2020-11-27T11:08:32Z', u'lastTransitionTime': u'2020-11-27T11:04:21Z', u'reason': u'NewReplicaSetAvailable', u'message': u'ReplicaSet "bono-658d668944" has successfully progressed.', u'type': u'Progressing'}]}, u'kind': u'Deployment', u'spec': {u'progressDeadlineSeconds': 600, u'replicas': 1, u'strategy': {u'rollingUpdate': {u'maxSurge': u'25%', u'maxUnavailable': u'25%'}, u'type': u'RollingUpdate'}, u'template': {u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'schedulerName': u'default-scheduler', u'terminationGracePeriodSeconds': 30, u'restartPolicy': u'Always', u'volumes': [{u'emptyDir': {}, u'name': u'bonologs'}], u'initContainers': [{u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-sprout', u'image': u'busybox:1.28', u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup sprout; do echo waiting for sprout to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'imagePullSecrets': [{}], u'containers': [{u'livenessProbe': {u'timeoutSeconds': 1, u'exec': {u'command': [u'/bin/bash', u'/usr/share/kubernetes/liveness.sh', u'3478 5060 5062']}, u'initialDelaySeconds': 30, u'periodSeconds': 10, u'successThreshold': 1, u'failureThreshold': 3}, u'terminationMessagePath': u'/dev/termination-log', u'name': u'bono', u'envFrom': [{u'configMapRef': {u'name': u'env-vars'}}], u'image': u'ollivier/clearwater-bono:hunter', u'volumeMounts': [{u'mountPath': u'/var/log/bono', u'name': u'bonologs'}], u'terminationMessagePolicy': u'File', u'env': [{u'valueFrom': {u'fieldRef': {u'fieldPath': u'status.podIP', u'apiVersion': u'v1'}}, u'name': u'MY_POD_IP'}], u'imagePullPolicy': u'IfNotPresent', u'readinessProbe': {u'successThreshold': 1, u'periodSeconds': 10, u'failureThreshold': 3, u'timeoutSeconds': 1, u'exec': {u'command': [u'/bin/bash', u'/usr/share/kubernetes/liveness.sh', u'3478 5060 5062']}}, u'ports': [{u'protocol': u'TCP', u'containerPort': 22}, {u'protocol': u'TCP', u'containerPort': 3478}, {u'protocol': u'TCP', u'containerPort': 5060}, {u'protocol': u'TCP', u'containerPort': 5062}, {u'protocol': u'UDP', u'containerPort': 5060}, {u'protocol': u'UDP', u'containerPort': 5062}], u'resources': {}}, {u'terminationMessagePath': u'/dev/termination-log', u'name': u'tailer', u'image': u'busybox:1.28', u'volumeMounts': [{u'mountPath': u'/var/log/bono', u'name': u'bonologs'}], u'terminationMessagePolicy': u'File', u'command': [u'tail', u'-F', u'/var/log/bono/bono_current.txt'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}]}, u'metadata': {u'labels': {u'snmp': u'enabled', u'service': u'bono'}, u'creationTimestamp': None}}, u'revisionHistoryLimit': 10, u'selector': {u'matchLabels': {u'service': u'bono'}}}, u'apiVersion': u'apps/v1', u'metadata': {u'name': u'bono', u'generation': 1, u'labels': {u'app.kubernetes.io/managed-by': u'Helm'}, u'namespace': u'ims-w9fzp', u'resourceVersion': u'11212628', u'creationTimestamp': u'2020-11-27T11:04:21Z', u'annotations': {u'meta.helm.sh/release-name': u'clearwater', u'deployment.kubernetes.io/revision': u'1', u'meta.helm.sh/release-namespace': u'ims-w9fzp'}, u'selfLink': u'/apis/apps/v1/namespaces/ims-w9fzp/deployments/bono', u'uid': u'4d5f05a7-30a0-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {u'deployment.kubernetes.io/revision': '1', u'meta.helm.sh/release-name': 'clearwater', u'meta.helm.sh/release-namespace': 'ims-w9fzp'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {u'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'bono', 'namespace': 'ims-w9fzp', 'owner_references': None, 'resource_version': '11212628', 'self_link': '/apis/apps/v1/namespaces/ims-w9fzp/deployments/bono', 'uid': '4d5f05a7-30a0-11eb-8302-0242ac120002'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {u'service': 'bono'}}, 'strategy': {'rolling_update': {'max_surge': u'25%', 'max_unavailable': u'25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {u'service': 'bono', u'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-bono:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 5060 5062']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'bono', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 3478, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 5060, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 5062, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 5060, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'UDP'}, {'container_port': 5062, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'UDP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 5060 5062']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/bono', 'mount_propagation': None, 'name': 'bonologs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/bono/bono_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/bono', 'mount_propagation': None, 'name': 'bonologs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup sprout; do echo waiting for sprout to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-sprout', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'bonologs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': 1, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 11, 27, 11, 8, 32, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 8, 32, tzinfo=tzlocal()), 'message': 'Deployment has minimum availability.', 'reason': 'MinimumReplicasAvailable', 'status': 'True', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 8, 32, tzinfo=tzlocal()), 'message': 'ReplicaSet "bono-658d668944" has successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': 1, 'replicas': 1, 'unavailable_replicas': None, 'updated_replicas': 1}}, u'type': u'MODIFIED'} 2020-11-27 11:08:33,279 - functest_kubernetes.ims.ims - INFO - bono started in 311.98 sec 2020-11-27 11:08:47,795 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'replicas': 1, u'observedGeneration': 1, u'updatedReplicas': 1, u'availableReplicas': 1, u'readyReplicas': 1, u'conditions': [{u'status': u'True', u'lastUpdateTime': u'2020-11-27T11:08:47Z', u'lastTransitionTime': u'2020-11-27T11:08:47Z', u'reason': u'MinimumReplicasAvailable', u'message': u'Deployment has minimum availability.', u'type': u'Available'}, {u'status': u'True', u'lastUpdateTime': u'2020-11-27T11:08:47Z', u'lastTransitionTime': u'2020-11-27T11:04:21Z', u'reason': u'NewReplicaSetAvailable', u'message': u'ReplicaSet "ellis-857689fbdd" has successfully progressed.', u'type': u'Progressing'}]}, u'kind': u'Deployment', u'spec': {u'progressDeadlineSeconds': 600, u'replicas': 1, u'strategy': {u'rollingUpdate': {u'maxSurge': u'25%', u'maxUnavailable': u'25%'}, u'type': u'RollingUpdate'}, u'template': {u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'schedulerName': u'default-scheduler', u'terminationGracePeriodSeconds': 30, u'restartPolicy': u'Always', u'initContainers': [{u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-homestead-prov', u'image': u'busybox:1.28', u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup homestead-prov; do echo waiting for homestead-prov to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}, {u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-sprout', u'image': u'busybox:1.28', u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup sprout; do echo waiting for sprout to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'imagePullSecrets': [{}], u'containers': [{u'livenessProbe': {u'tcpSocket': {u'port': 80}, u'timeoutSeconds': 1, u'initialDelaySeconds': 30, u'periodSeconds': 10, u'successThreshold': 1, u'failureThreshold': 3}, u'terminationMessagePath': u'/dev/termination-log', u'name': u'ellis', u'envFrom': [{u'configMapRef': {u'name': u'env-vars'}}], u'image': u'ollivier/clearwater-ellis:hunter', u'terminationMessagePolicy': u'File', u'env': [{u'valueFrom': {u'fieldRef': {u'fieldPath': u'status.podIP', u'apiVersion': u'v1'}}, u'name': u'MY_POD_IP'}], u'imagePullPolicy': u'IfNotPresent', u'readinessProbe': {u'successThreshold': 1, u'tcpSocket': {u'port': 80}, u'failureThreshold': 3, u'timeoutSeconds': 1, u'periodSeconds': 10}, u'ports': [{u'protocol': u'TCP', u'containerPort': 22}, {u'protocol': u'TCP', u'containerPort': 80}], u'resources': {}}]}, u'metadata': {u'labels': {u'service': u'ellis'}, u'creationTimestamp': None}}, u'revisionHistoryLimit': 10, u'selector': {u'matchLabels': {u'service': u'ellis'}}}, u'apiVersion': u'apps/v1', u'metadata': {u'name': u'ellis', u'generation': 1, u'labels': {u'app.kubernetes.io/managed-by': u'Helm'}, u'namespace': u'ims-w9fzp', u'resourceVersion': u'11212669', u'creationTimestamp': u'2020-11-27T11:04:21Z', u'annotations': {u'meta.helm.sh/release-name': u'clearwater', u'deployment.kubernetes.io/revision': u'1', u'meta.helm.sh/release-namespace': u'ims-w9fzp'}, u'selfLink': u'/apis/apps/v1/namespaces/ims-w9fzp/deployments/ellis', u'uid': u'4d60c225-30a0-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {u'deployment.kubernetes.io/revision': '1', u'meta.helm.sh/release-name': 'clearwater', u'meta.helm.sh/release-namespace': 'ims-w9fzp'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {u'app.kubernetes.io/managed-by': 'Helm'}, 'managed_fields': None, 'name': 'ellis', 'namespace': 'ims-w9fzp', 'owner_references': None, 'resource_version': '11212669', 'self_link': '/apis/apps/v1/namespaces/ims-w9fzp/deployments/ellis', 'uid': '4d60c225-30a0-11eb-8302-0242ac120002'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {u'service': 'ellis'}}, 'strategy': {'rolling_update': {'max_surge': u'25%', 'max_unavailable': u'25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {u'service': 'ellis'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-ellis:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 80}, 'timeout_seconds': 1}, 'name': 'ellis', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 80, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 80}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup homestead-prov; do echo waiting for homestead-prov to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-homestead-prov', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until nslookup sprout; do echo waiting for sprout to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-sprout', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': 1, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 11, 27, 11, 8, 47, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 8, 47, tzinfo=tzlocal()), 'message': 'Deployment has minimum availability.', 'reason': 'MinimumReplicasAvailable', 'status': 'True', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 11, 27, 11, 8, 47, tzinfo=tzlocal()), 'message': 'ReplicaSet "ellis-857689fbdd" has successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': 1, 'replicas': 1, 'unavailable_replicas': None, 'updated_replicas': 1}}, u'type': u'MODIFIED'} 2020-11-27 11:08:48,041 - functest_kubernetes.ims.ims - INFO - ellis started in 326.74 sec 2020-11-27 11:10:48,164 - kubernetes.client.rest - DEBUG - response body: {"kind":"Pod","apiVersion":"v1","metadata":{"name":"live-test","namespace":"ims-w9fzp","selfLink":"/api/v1/namespaces/ims-w9fzp/pods/live-test","uid":"33c848de-30a1-11eb-8302-0242ac120002","resourceVersion":"11212905","creationTimestamp":"2020-11-27T11:10:48Z"},"spec":{"volumes":[{"name":"default-token-c2jnd","secret":{"secretName":"default-token-c2jnd","defaultMode":420}}],"containers":[{"name":"live-test","image":"ollivier/clearwater-live-test:hunter","command":["rake","test[ims-w9fzp.svc.cluster.local]","PROXY=bono.ims-w9fzp.svc.cluster.local","ELLIS=ellis.ims-w9fzp.svc.cluster.local","SIGNUP_CODE=secret","--trace"],"resources":{},"volumeMounts":[{"name":"default-token-c2jnd","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Never","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}],"priority":0,"enableServiceLinks":true},"status":{"phase":"Pending","qosClass":"BestEffort"}} 2020-11-27 11:10:48,231 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'initContainerStatuses': [{u'restartCount': 0, u'name': u'wait-on-homestead', u'image': u'docker.io/library/busybox:1.28', u'imageID': u'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', u'state': {u'terminated': {u'startedAt': u'2020-11-27T11:04:26Z', u'reason': u'Completed', u'finishedAt': u'2020-11-27T11:08:13Z', u'containerID': u'containerd://d35e4f916c4f129bb0ce82e0f9d9701c2edcebe654b7723e7a92109d41617d59', u'exitCode': 0}}, u'ready': True, u'lastState': {}, u'containerID': u'containerd://d35e4f916c4f129bb0ce82e0f9d9701c2edcebe654b7723e7a92109d41617d59'}], u'qosClass': u'BestEffort', u'containerStatuses': [{u'restartCount': 0, u'name': u'homestead-prov', u'image': u'docker.io/ollivier/clearwater-homestead-prov:hunter', u'imageID': u'docker.io/ollivier/clearwater-homestead-prov@sha256:6e369295193fa6e5326c41cd2d4f4bdda75c5cb829bd3f7ee899554a48a7b323', u'state': {u'running': {u'startedAt': u'2020-11-27T11:08:15Z'}}, u'ready': True, u'lastState': {}, u'containerID': u'containerd://6d8f8b5a323f6b6113a4747ae4f699dc964477011bd7767cc337c6e0f5d2b69d'}], u'podIP': u'10.244.1.117', u'startTime': u'2020-11-27T11:04:21Z', u'hostIP': u'172.18.0.4', u'phase': u'Running', u'conditions': [{u'status': u'True', u'lastProbeTime': None, u'type': u'Initialized', u'lastTransitionTime': u'2020-11-27T11:08:14Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'Ready', u'lastTransitionTime': u'2020-11-27T11:08:28Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'ContainersReady', u'lastTransitionTime': u'2020-11-27T11:08:28Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'PodScheduled', u'lastTransitionTime': u'2020-11-27T11:04:21Z'}]}, u'kind': u'Pod', u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'serviceAccountName': u'default', u'schedulerName': u'default-scheduler', u'enableServiceLinks': True, u'serviceAccount': u'default', u'priority': 0, u'terminationGracePeriodSeconds': 30, u'restartPolicy': u'Always', u'volumes': [{u'secret': {u'defaultMode': 420, u'secretName': u'default-token-c2jnd'}, u'name': u'default-token-c2jnd'}], u'tolerations': [{u'operator': u'Exists', u'tolerationSeconds': 300, u'effect': u'NoExecute', u'key': u'node.kubernetes.io/not-ready'}, {u'operator': u'Exists', u'tolerationSeconds': 300, u'effect': u'NoExecute', u'key': u'node.kubernetes.io/unreachable'}], u'initContainers': [{u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-homestead', u'image': u'busybox:1.28', u'volumeMounts': [{u'readOnly': True, u'mountPath': u'/var/run/secrets/kubernetes.io/serviceaccount', u'name': u'default-token-c2jnd'}], u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup homestead; do echo waiting for homestead to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'imagePullSecrets': [{}], u'containers': [{u'livenessProbe': {u'timeoutSeconds': 1, u'exec': {u'command': [u'/bin/bash', u'/usr/share/clearwater/bin/poll_homestead-prov.sh']}, u'initialDelaySeconds': 60, u'periodSeconds': 10, u'successThreshold': 1, u'failureThreshold': 3}, u'terminationMessagePath': u'/dev/termination-log', u'name': u'homestead-prov', u'envFrom': [{u'configMapRef': {u'name': u'env-vars'}}], u'image': u'ollivier/clearwater-homestead-prov:hunter', u'volumeMounts': [{u'readOnly': True, u'mountPath': u'/var/run/secrets/kubernetes.io/serviceaccount', u'name': u'default-token-c2jnd'}], u'terminationMessagePolicy': u'File', u'env': [{u'valueFrom': {u'fieldRef': {u'fieldPath': u'status.podIP', u'apiVersion': u'v1'}}, u'name': u'MY_POD_IP'}], u'imagePullPolicy': u'IfNotPresent', u'readinessProbe': {u'successThreshold': 1, u'periodSeconds': 10, u'failureThreshold': 3, u'timeoutSeconds': 1, u'exec': {u'command': [u'/bin/bash', u'/usr/share/clearwater/bin/poll_homestead-prov.sh']}}, u'ports': [{u'protocol': u'TCP', u'containerPort': 22}, {u'protocol': u'TCP', u'containerPort': 8889}], u'resources': {}}], u'nodeName': u'hunter-worker'}, u'apiVersion': u'v1', u'metadata': {u'name': u'homestead-prov-6654f8cc8f-xc65n', u'labels': {u'pod-template-hash': u'6654f8cc8f', u'snmp': u'enabled', u'service': u'homestead-prov'}, u'namespace': u'ims-w9fzp', u'ownerReferences': [{u'kind': u'ReplicaSet', u'name': u'homestead-prov-6654f8cc8f', u'apiVersion': u'apps/v1', u'controller': True, u'blockOwnerDeletion': True, u'uid': u'4d5e38c9-30a0-11eb-8302-0242ac120002'}], u'resourceVersion': u'11212611', u'generateName': u'homestead-prov-6654f8cc8f-', u'creationTimestamp': u'2020-11-27T11:04:21Z', u'selfLink': u'/api/v1/namespaces/ims-w9fzp/pods/homestead-prov-6654f8cc8f-xc65n', u'uid': u'4d7cb597-30a0-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'homestead-prov-6654f8cc8f-', 'generation': None, 'initializers': None, 'labels': {u'pod-template-hash': '6654f8cc8f', u'service': 'homestead-prov', u'snmp': 'enabled'}, 'managed_fields': None, 'name': 'homestead-prov-6654f8cc8f-xc65n', 'namespace': 'ims-w9fzp', 'owner_references': [{'api_version': 'apps/v1', 'block_owner_deletion': True, 'controller': True, 'kind': 'ReplicaSet', 'name': 'homestead-prov-6654f8cc8f', 'uid': '4d5e38c9-30a0-11eb-8302-0242ac120002'}], 'resource_version': '11212611', 'self_link': '/api/v1/namespaces/ims-w9fzp/pods/homestead-prov-6654f8cc8f-xc65n', 'uid': '4d7cb597-30a0-11eb-8302-0242ac120002'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-homestead-prov:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 60, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'homestead-prov', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 8889, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-c2jnd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup homestead; do echo waiting for homestead to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-homestead', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-c2jnd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'node_name': 'hunter-worker', 'node_selector': None, 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-c2jnd', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-c2jnd'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 8, 14, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 8, 28, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 8, 28, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://6d8f8b5a323f6b6113a4747ae4f699dc964477011bd7767cc337c6e0f5d2b69d', 'image': 'docker.io/ollivier/clearwater-homestead-prov:hunter', 'image_id': 'docker.io/ollivier/clearwater-homestead-prov@sha256:6e369295193fa6e5326c41cd2d4f4bdda75c5cb829bd3f7ee899554a48a7b323', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'homestead-prov', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 11, 27, 11, 8, 15, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.4', 'init_container_statuses': [{'container_id': 'containerd://d35e4f916c4f129bb0ce82e0f9d9701c2edcebe654b7723e7a92109d41617d59', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-homestead', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://d35e4f916c4f129bb0ce82e0f9d9701c2edcebe654b7723e7a92109d41617d59', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 11, 27, 11, 8, 13, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 11, 27, 11, 4, 26, tzinfo=tzlocal())}, 'waiting': None}}], 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.1.117', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal())}}, u'type': u'ADDED'} 2020-11-27 11:10:48,545 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'initContainerStatuses': [{u'restartCount': 0, u'name': u'wait-on-cassandra', u'image': u'docker.io/library/busybox:1.28', u'imageID': u'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', u'state': {u'terminated': {u'startedAt': u'2020-11-27T11:04:33Z', u'reason': u'Completed', u'finishedAt': u'2020-11-27T11:06:37Z', u'containerID': u'containerd://866cc9193a82e8d25a94a52215b602b0798933bec3939766f7d5d9b7cd6ddd7a', u'exitCode': 0}}, u'ready': True, u'lastState': {}, u'containerID': u'containerd://866cc9193a82e8d25a94a52215b602b0798933bec3939766f7d5d9b7cd6ddd7a'}], u'qosClass': u'BestEffort', u'containerStatuses': [{u'restartCount': 0, u'name': u'homer', u'image': u'docker.io/ollivier/clearwater-homer:hunter', u'imageID': u'docker.io/ollivier/clearwater-homer@sha256:63a6cb127f0e57ecd4f7d066ae194be6d8f021e96f4b6022bf9daacee0f1ff5a', u'state': {u'running': {u'startedAt': u'2020-11-27T11:06:44Z'}}, u'ready': True, u'lastState': {}, u'containerID': u'containerd://d204be479edf73615cb3cf9f4955ab198b1e3d4d8fc57c0fdb9ab3849d3e1a33'}], u'podIP': u'10.244.2.54', u'startTime': u'2020-11-27T11:04:22Z', u'hostIP': u'172.18.0.3', u'phase': u'Running', u'conditions': [{u'status': u'True', u'lastProbeTime': None, u'type': u'Initialized', u'lastTransitionTime': u'2020-11-27T11:06:38Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'Ready', u'lastTransitionTime': u'2020-11-27T11:06:57Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'ContainersReady', u'lastTransitionTime': u'2020-11-27T11:06:57Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'PodScheduled', u'lastTransitionTime': u'2020-11-27T11:04:21Z'}]}, u'kind': u'Pod', u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'serviceAccountName': u'default', u'schedulerName': u'default-scheduler', u'enableServiceLinks': True, u'serviceAccount': u'default', u'priority': 0, u'terminationGracePeriodSeconds': 30, u'restartPolicy': u'Always', u'volumes': [{u'secret': {u'defaultMode': 420, u'secretName': u'default-token-c2jnd'}, u'name': u'default-token-c2jnd'}], u'tolerations': [{u'operator': u'Exists', u'tolerationSeconds': 300, u'effect': u'NoExecute', u'key': u'node.kubernetes.io/not-ready'}, {u'operator': u'Exists', u'tolerationSeconds': 300, u'effect': u'NoExecute', u'key': u'node.kubernetes.io/unreachable'}], u'initContainers': [{u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-cassandra', u'image': u'busybox:1.28', u'volumeMounts': [{u'readOnly': True, u'mountPath': u'/var/run/secrets/kubernetes.io/serviceaccount', u'name': u'default-token-c2jnd'}], u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'imagePullSecrets': [{}], u'containers': [{u'livenessProbe': {u'tcpSocket': {u'port': 7888}, u'timeoutSeconds': 1, u'initialDelaySeconds': 30, u'periodSeconds': 10, u'successThreshold': 1, u'failureThreshold': 3}, u'terminationMessagePath': u'/dev/termination-log', u'name': u'homer', u'envFrom': [{u'configMapRef': {u'name': u'env-vars'}}], u'image': u'ollivier/clearwater-homer:hunter', u'volumeMounts': [{u'readOnly': True, u'mountPath': u'/var/run/secrets/kubernetes.io/serviceaccount', u'name': u'default-token-c2jnd'}], u'terminationMessagePolicy': u'File', u'env': [{u'valueFrom': {u'fieldRef': {u'fieldPath': u'status.podIP', u'apiVersion': u'v1'}}, u'name': u'MY_POD_IP'}], u'imagePullPolicy': u'IfNotPresent', u'readinessProbe': {u'successThreshold': 1, u'tcpSocket': {u'port': 7888}, u'failureThreshold': 3, u'timeoutSeconds': 1, u'periodSeconds': 10}, u'ports': [{u'protocol': u'TCP', u'containerPort': 22}, {u'protocol': u'TCP', u'containerPort': 7888}], u'resources': {}}], u'nodeName': u'hunter-worker2'}, u'apiVersion': u'v1', u'metadata': {u'name': u'homer-65d4f8455b-9rsrw', u'labels': {u'pod-template-hash': u'65d4f8455b', u'service': u'homer'}, u'namespace': u'ims-w9fzp', u'ownerReferences': [{u'kind': u'ReplicaSet', u'name': u'homer-65d4f8455b', u'apiVersion': u'apps/v1', u'controller': True, u'blockOwnerDeletion': True, u'uid': u'4d5e4b96-30a0-11eb-8302-0242ac120002'}], u'resourceVersion': u'11212328', u'generateName': u'homer-65d4f8455b-', u'creationTimestamp': u'2020-11-27T11:04:21Z', u'selfLink': u'/api/v1/namespaces/ims-w9fzp/pods/homer-65d4f8455b-9rsrw', u'uid': u'4d7ccbae-30a0-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'homer-65d4f8455b-', 'generation': None, 'initializers': None, 'labels': {u'pod-template-hash': '65d4f8455b', u'service': 'homer'}, 'managed_fields': None, 'name': 'homer-65d4f8455b-9rsrw', 'namespace': 'ims-w9fzp', 'owner_references': [{'api_version': 'apps/v1', 'block_owner_deletion': True, 'controller': True, 'kind': 'ReplicaSet', 'name': 'homer-65d4f8455b', 'uid': '4d5e4b96-30a0-11eb-8302-0242ac120002'}], 'resource_version': '11212328', 'self_link': '/api/v1/namespaces/ims-w9fzp/pods/homer-65d4f8455b-9rsrw', 'uid': '4d7ccbae-30a0-11eb-8302-0242ac120002'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-homer:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7888}, 'timeout_seconds': 1}, 'name': 'homer', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 7888, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7888}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-c2jnd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-c2jnd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'node_name': 'hunter-worker2', 'node_selector': None, 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-c2jnd', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-c2jnd'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 6, 38, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 6, 57, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 6, 57, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://d204be479edf73615cb3cf9f4955ab198b1e3d4d8fc57c0fdb9ab3849d3e1a33', 'image': 'docker.io/ollivier/clearwater-homer:hunter', 'image_id': 'docker.io/ollivier/clearwater-homer@sha256:63a6cb127f0e57ecd4f7d066ae194be6d8f021e96f4b6022bf9daacee0f1ff5a', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'homer', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 11, 27, 11, 6, 44, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.3', 'init_container_statuses': [{'container_id': 'containerd://866cc9193a82e8d25a94a52215b602b0798933bec3939766f7d5d9b7cd6ddd7a', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-cassandra', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://866cc9193a82e8d25a94a52215b602b0798933bec3939766f7d5d9b7cd6ddd7a', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 11, 27, 11, 6, 37, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 11, 27, 11, 4, 33, tzinfo=tzlocal())}, 'waiting': None}}], 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.2.54', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal())}}, u'type': u'ADDED'} 2020-11-27 11:10:48,862 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'initContainerStatuses': [{u'restartCount': 0, u'name': u'wait-on-sprout', u'image': u'docker.io/library/busybox:1.28', u'imageID': u'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', u'state': {u'terminated': {u'startedAt': u'2020-11-27T11:04:29Z', u'reason': u'Completed', u'finishedAt': u'2020-11-27T11:08:24Z', u'containerID': u'containerd://88b97317191b998c9e5e163db0422e8ec4b0c65ec43ce1cd2d9bacbe49500564', u'exitCode': 0}}, u'ready': True, u'lastState': {}, u'containerID': u'containerd://88b97317191b998c9e5e163db0422e8ec4b0c65ec43ce1cd2d9bacbe49500564'}], u'qosClass': u'BestEffort', u'containerStatuses': [{u'restartCount': 0, u'name': u'bono', u'image': u'docker.io/ollivier/clearwater-bono:hunter', u'imageID': u'docker.io/ollivier/clearwater-bono@sha256:0aa1f9ff35f9d5682b1b48a3ceb95acce0166fbf17401af2f86efd61ea91e760', u'state': {u'running': {u'startedAt': u'2020-11-27T11:08:26Z'}}, u'ready': True, u'lastState': {}, u'containerID': u'containerd://2ee4e042b4bab53aaf610589684e04f038ec27fe4b9d7ffa46188e16193ddfb0'}, {u'restartCount': 0, u'name': u'tailer', u'image': u'docker.io/library/busybox:1.28', u'imageID': u'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', u'state': {u'running': {u'startedAt': u'2020-11-27T11:08:28Z'}}, u'ready': True, u'lastState': {}, u'containerID': u'containerd://66dc1b16f97ce4f7f4546df73c3226fd3d212c9bd79b49e4b60f29958c885f3b'}], u'podIP': u'10.244.1.119', u'startTime': u'2020-11-27T11:04:22Z', u'hostIP': u'172.18.0.4', u'phase': u'Running', u'conditions': [{u'status': u'True', u'lastProbeTime': None, u'type': u'Initialized', u'lastTransitionTime': u'2020-11-27T11:08:25Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'Ready', u'lastTransitionTime': u'2020-11-27T11:08:32Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'ContainersReady', u'lastTransitionTime': u'2020-11-27T11:08:32Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'PodScheduled', u'lastTransitionTime': u'2020-11-27T11:04:21Z'}]}, u'kind': u'Pod', u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'serviceAccountName': u'default', u'schedulerName': u'default-scheduler', u'enableServiceLinks': True, u'serviceAccount': u'default', u'priority': 0, u'terminationGracePeriodSeconds': 30, u'restartPolicy': u'Always', u'volumes': [{u'emptyDir': {}, u'name': u'bonologs'}, {u'secret': {u'defaultMode': 420, u'secretName': u'default-token-c2jnd'}, u'name': u'default-token-c2jnd'}], u'tolerations': [{u'operator': u'Exists', u'tolerationSeconds': 300, u'effect': u'NoExecute', u'key': u'node.kubernetes.io/not-ready'}, {u'operator': u'Exists', u'tolerationSeconds': 300, u'effect': u'NoExecute', u'key': u'node.kubernetes.io/unreachable'}], u'initContainers': [{u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-sprout', u'image': u'busybox:1.28', u'volumeMounts': [{u'readOnly': True, u'mountPath': u'/var/run/secrets/kubernetes.io/serviceaccount', u'name': u'default-token-c2jnd'}], u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup sprout; do echo waiting for sprout to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'imagePullSecrets': [{}], u'containers': [{u'livenessProbe': {u'timeoutSeconds': 1, u'exec': {u'command': [u'/bin/bash', u'/usr/share/kubernetes/liveness.sh', u'3478 5060 5062']}, u'initialDelaySeconds': 30, u'periodSeconds': 10, u'successThreshold': 1, u'failureThreshold': 3}, u'terminationMessagePath': u'/dev/termination-log', u'name': u'bono', u'envFrom': [{u'configMapRef': {u'name': u'env-vars'}}], u'image': u'ollivier/clearwater-bono:hunter', u'volumeMounts': [{u'mountPath': u'/var/log/bono', u'name': u'bonologs'}, {u'readOnly': True, u'mountPath': u'/var/run/secrets/kubernetes.io/serviceaccount', u'name': u'default-token-c2jnd'}], u'terminationMessagePolicy': u'File', u'env': [{u'valueFrom': {u'fieldRef': {u'fieldPath': u'status.podIP', u'apiVersion': u'v1'}}, u'name': u'MY_POD_IP'}], u'imagePullPolicy': u'IfNotPresent', u'readinessProbe': {u'successThreshold': 1, u'periodSeconds': 10, u'failureThreshold': 3, u'timeoutSeconds': 1, u'exec': {u'command': [u'/bin/bash', u'/usr/share/kubernetes/liveness.sh', u'3478 5060 5062']}}, u'ports': [{u'protocol': u'TCP', u'containerPort': 22}, {u'protocol': u'TCP', u'containerPort': 3478}, {u'protocol': u'TCP', u'containerPort': 5060}, {u'protocol': u'TCP', u'containerPort': 5062}, {u'protocol': u'UDP', u'containerPort': 5060}, {u'protocol': u'UDP', u'containerPort': 5062}], u'resources': {}}, {u'terminationMessagePath': u'/dev/termination-log', u'name': u'tailer', u'image': u'busybox:1.28', u'volumeMounts': [{u'mountPath': u'/var/log/bono', u'name': u'bonologs'}, {u'readOnly': True, u'mountPath': u'/var/run/secrets/kubernetes.io/serviceaccount', u'name': u'default-token-c2jnd'}], u'terminationMessagePolicy': u'File', u'command': [u'tail', u'-F', u'/var/log/bono/bono_current.txt'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'nodeName': u'hunter-worker'}, u'apiVersion': u'v1', u'metadata': {u'name': u'bono-658d668944-fxdvs', u'labels': {u'pod-template-hash': u'658d668944', u'snmp': u'enabled', u'service': u'bono'}, u'namespace': u'ims-w9fzp', u'ownerReferences': [{u'kind': u'ReplicaSet', u'name': u'bono-658d668944', u'apiVersion': u'apps/v1', u'controller': True, u'blockOwnerDeletion': True, u'uid': u'4d6b2b06-30a0-11eb-8302-0242ac120002'}], u'resourceVersion': u'11212625', u'generateName': u'bono-658d668944-', u'creationTimestamp': u'2020-11-27T11:04:21Z', u'selfLink': u'/api/v1/namespaces/ims-w9fzp/pods/bono-658d668944-fxdvs', u'uid': u'4d8b2236-30a0-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'bono-658d668944-', 'generation': None, 'initializers': None, 'labels': {u'pod-template-hash': '658d668944', u'service': 'bono', u'snmp': 'enabled'}, 'managed_fields': None, 'name': 'bono-658d668944-fxdvs', 'namespace': 'ims-w9fzp', 'owner_references': [{'api_version': 'apps/v1', 'block_owner_deletion': True, 'controller': True, 'kind': 'ReplicaSet', 'name': 'bono-658d668944', 'uid': '4d6b2b06-30a0-11eb-8302-0242ac120002'}], 'resource_version': '11212625', 'self_link': '/api/v1/namespaces/ims-w9fzp/pods/bono-658d668944-fxdvs', 'uid': '4d8b2236-30a0-11eb-8302-0242ac120002'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-bono:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 5060 5062']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'bono', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 3478, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 5060, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 5062, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 5060, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'UDP'}, {'container_port': 5062, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'UDP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 5060 5062']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/bono', 'mount_propagation': None, 'name': 'bonologs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-c2jnd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/bono/bono_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/bono', 'mount_propagation': None, 'name': 'bonologs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-c2jnd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup sprout; do echo waiting for sprout to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-sprout', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-c2jnd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'node_name': 'hunter-worker', 'node_selector': None, 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'bonologs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}, {'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-c2jnd', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-c2jnd'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 8, 25, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 8, 32, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 8, 32, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://2ee4e042b4bab53aaf610589684e04f038ec27fe4b9d7ffa46188e16193ddfb0', 'image': 'docker.io/ollivier/clearwater-bono:hunter', 'image_id': 'docker.io/ollivier/clearwater-bono@sha256:0aa1f9ff35f9d5682b1b48a3ceb95acce0166fbf17401af2f86efd61ea91e760', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'bono', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 11, 27, 11, 8, 26, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}, {'container_id': 'containerd://66dc1b16f97ce4f7f4546df73c3226fd3d212c9bd79b49e4b60f29958c885f3b', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'tailer', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 11, 27, 11, 8, 28, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.4', 'init_container_statuses': [{'container_id': 'containerd://88b97317191b998c9e5e163db0422e8ec4b0c65ec43ce1cd2d9bacbe49500564', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-sprout', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://88b97317191b998c9e5e163db0422e8ec4b0c65ec43ce1cd2d9bacbe49500564', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 11, 27, 11, 8, 24, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 11, 27, 11, 4, 29, tzinfo=tzlocal())}, 'waiting': None}}], 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.1.119', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal())}}, u'type': u'ADDED'} 2020-11-27 11:10:49,253 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'initContainerStatuses': [{u'restartCount': 0, u'name': u'wait-on-cassandra', u'image': u'docker.io/library/busybox:1.28', u'imageID': u'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', u'state': {u'terminated': {u'startedAt': u'2020-11-27T11:04:33Z', u'reason': u'Completed', u'finishedAt': u'2020-11-27T11:06:38Z', u'containerID': u'containerd://219ab0d88f9997e1ec805d98fd25804d0964869d4fed7284db08d94efbc22a0d', u'exitCode': 0}}, u'ready': True, u'lastState': {}, u'containerID': u'containerd://219ab0d88f9997e1ec805d98fd25804d0964869d4fed7284db08d94efbc22a0d'}], u'qosClass': u'BestEffort', u'containerStatuses': [{u'restartCount': 0, u'name': u'astaire', u'image': u'docker.io/ollivier/clearwater-astaire:hunter', u'imageID': u'docker.io/ollivier/clearwater-astaire@sha256:952f8fa55e26181c90aad327c5a9ec98661ef933decbc3f547836c03447a4c3c', u'state': {u'running': {u'startedAt': u'2020-11-27T11:06:46Z'}}, u'ready': True, u'lastState': {}, u'containerID': u'containerd://40af2cd0a3eb1e710045728a4e8d4b7a00fa3fa844039fbeb61f8b213068c956'}, {u'restartCount': 0, u'name': u'tailer', u'image': u'docker.io/library/busybox:1.28', u'imageID': u'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', u'state': {u'running': {u'startedAt': u'2020-11-27T11:06:49Z'}}, u'ready': True, u'lastState': {}, u'containerID': u'containerd://444a7380071ff8756206c4085edd886a64a96a132251640ede95e596aced5343'}], u'podIP': u'10.244.2.60', u'startTime': u'2020-11-27T11:04:22Z', u'hostIP': u'172.18.0.3', u'phase': u'Running', u'conditions': [{u'status': u'True', u'lastProbeTime': None, u'type': u'Initialized', u'lastTransitionTime': u'2020-11-27T11:06:40Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'Ready', u'lastTransitionTime': u'2020-11-27T11:07:33Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'ContainersReady', u'lastTransitionTime': u'2020-11-27T11:07:33Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'PodScheduled', u'lastTransitionTime': u'2020-11-27T11:04:22Z'}]}, u'kind': u'Pod', u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'serviceAccountName': u'default', u'schedulerName': u'default-scheduler', u'enableServiceLinks': True, u'serviceAccount': u'default', u'priority': 0, u'terminationGracePeriodSeconds': 120, u'restartPolicy': u'Always', u'volumes': [{u'emptyDir': {}, u'name': u'astairelogs'}, {u'secret': {u'defaultMode': 420, u'secretName': u'default-token-c2jnd'}, u'name': u'default-token-c2jnd'}], u'tolerations': [{u'operator': u'Exists', u'tolerationSeconds': 300, u'effect': u'NoExecute', u'key': u'node.kubernetes.io/not-ready'}, {u'operator': u'Exists', u'tolerationSeconds': 300, u'effect': u'NoExecute', u'key': u'node.kubernetes.io/unreachable'}], u'initContainers': [{u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-cassandra', u'image': u'busybox:1.28', u'volumeMounts': [{u'readOnly': True, u'mountPath': u'/var/run/secrets/kubernetes.io/serviceaccount', u'name': u'default-token-c2jnd'}], u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'imagePullSecrets': [{}], u'containers': [{u'livenessProbe': {u'successThreshold': 1, u'tcpSocket': {u'port': 11311}, u'failureThreshold': 9, u'timeoutSeconds': 1, u'periodSeconds': 10}, u'terminationMessagePath': u'/dev/termination-log', u'name': u'astaire', u'envFrom': [{u'configMapRef': {u'name': u'env-vars'}}], u'image': u'ollivier/clearwater-astaire:hunter', u'volumeMounts': [{u'mountPath': u'/var/log/astaire', u'name': u'astairelogs'}, {u'readOnly': True, u'mountPath': u'/var/run/secrets/kubernetes.io/serviceaccount', u'name': u'default-token-c2jnd'}], u'terminationMessagePolicy': u'File', u'lifecycle': {u'preStop': {u'exec': {u'command': [u'/bin/bash', u'-c', u'/usr/bin/pre-stop']}}}, u'env': [{u'valueFrom': {u'fieldRef': {u'fieldPath': u'status.podIP', u'apiVersion': u'v1'}}, u'name': u'MY_POD_IP'}], u'imagePullPolicy': u'IfNotPresent', u'readinessProbe': {u'successThreshold': 1, u'tcpSocket': {u'port': 11311}, u'failureThreshold': 3, u'timeoutSeconds': 1, u'periodSeconds': 10}, u'ports': [{u'protocol': u'TCP', u'containerPort': 22}, {u'protocol': u'TCP', u'containerPort': 11211}, {u'protocol': u'TCP', u'containerPort': 11311}], u'resources': {}}, {u'terminationMessagePath': u'/dev/termination-log', u'name': u'tailer', u'image': u'busybox:1.28', u'volumeMounts': [{u'mountPath': u'/var/log/astaire', u'name': u'astairelogs'}, {u'readOnly': True, u'mountPath': u'/var/run/secrets/kubernetes.io/serviceaccount', u'name': u'default-token-c2jnd'}], u'terminationMessagePolicy': u'File', u'command': [u'tail', u'-F', u'/var/log/astaire/astaire_current.txt'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'nodeName': u'hunter-worker2'}, u'apiVersion': u'v1', u'metadata': {u'name': u'astaire-5d44675d59-6lz9v', u'labels': {u'pod-template-hash': u'5d44675d59', u'service': u'astaire'}, u'namespace': u'ims-w9fzp', u'ownerReferences': [{u'kind': u'ReplicaSet', u'name': u'astaire-5d44675d59', u'apiVersion': u'apps/v1', u'controller': True, u'blockOwnerDeletion': True, u'uid': u'4d97c7c4-30a0-11eb-8302-0242ac120002'}], u'resourceVersion': u'11212463', u'generateName': u'astaire-5d44675d59-', u'creationTimestamp': u'2020-11-27T11:04:22Z', u'selfLink': u'/api/v1/namespaces/ims-w9fzp/pods/astaire-5d44675d59-6lz9v', u'uid': u'4da6d6bb-30a0-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'astaire-5d44675d59-', 'generation': None, 'initializers': None, 'labels': {u'pod-template-hash': '5d44675d59', u'service': 'astaire'}, 'managed_fields': None, 'name': 'astaire-5d44675d59-6lz9v', 'namespace': 'ims-w9fzp', 'owner_references': [{'api_version': 'apps/v1', 'block_owner_deletion': True, 'controller': True, 'kind': 'ReplicaSet', 'name': 'astaire-5d44675d59', 'uid': '4d97c7c4-30a0-11eb-8302-0242ac120002'}], 'resource_version': '11212463', 'self_link': '/api/v1/namespaces/ims-w9fzp/pods/astaire-5d44675d59-6lz9v', 'uid': '4da6d6bb-30a0-11eb-8302-0242ac120002'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-astaire:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': {'post_start': None, 'pre_stop': {'_exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}, 'http_get': None, 'tcp_socket': None}}, 'liveness_probe': {'_exec': None, 'failure_threshold': 9, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 11311}, 'timeout_seconds': 1}, 'name': 'astaire', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 11211, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 11311, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 11311}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/astaire', 'mount_propagation': None, 'name': 'astairelogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-c2jnd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/astaire/astaire_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/astaire', 'mount_propagation': None, 'name': 'astairelogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-c2jnd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-c2jnd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'node_name': 'hunter-worker2', 'node_selector': None, 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 120, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'astairelogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}, {'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-c2jnd', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-c2jnd'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 6, 40, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 7, 33, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 7, 33, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://40af2cd0a3eb1e710045728a4e8d4b7a00fa3fa844039fbeb61f8b213068c956', 'image': 'docker.io/ollivier/clearwater-astaire:hunter', 'image_id': 'docker.io/ollivier/clearwater-astaire@sha256:952f8fa55e26181c90aad327c5a9ec98661ef933decbc3f547836c03447a4c3c', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'astaire', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 11, 27, 11, 6, 46, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}, {'container_id': 'containerd://444a7380071ff8756206c4085edd886a64a96a132251640ede95e596aced5343', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'tailer', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 11, 27, 11, 6, 49, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.3', 'init_container_statuses': [{'container_id': 'containerd://219ab0d88f9997e1ec805d98fd25804d0964869d4fed7284db08d94efbc22a0d', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-cassandra', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://219ab0d88f9997e1ec805d98fd25804d0964869d4fed7284db08d94efbc22a0d', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 11, 27, 11, 6, 38, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 11, 27, 11, 4, 33, tzinfo=tzlocal())}, 'waiting': None}}], 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.2.60', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal())}}, u'type': u'ADDED'} 2020-11-27 11:10:49,662 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'phase': u'Pending', u'conditions': [{u'status': u'True', u'lastProbeTime': None, u'type': u'PodScheduled', u'lastTransitionTime': u'2020-11-27T11:10:48Z'}], u'qosClass': u'BestEffort'}, u'kind': u'Pod', u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'serviceAccountName': u'default', u'schedulerName': u'default-scheduler', u'enableServiceLinks': True, u'serviceAccount': u'default', u'priority': 0, u'terminationGracePeriodSeconds': 30, u'restartPolicy': u'Never', u'volumes': [{u'secret': {u'defaultMode': 420, u'secretName': u'default-token-c2jnd'}, u'name': u'default-token-c2jnd'}], u'tolerations': [{u'operator': u'Exists', u'tolerationSeconds': 300, u'effect': u'NoExecute', u'key': u'node.kubernetes.io/not-ready'}, {u'operator': u'Exists', u'tolerationSeconds': 300, u'effect': u'NoExecute', u'key': u'node.kubernetes.io/unreachable'}], u'containers': [{u'terminationMessagePath': u'/dev/termination-log', u'name': u'live-test', u'image': u'ollivier/clearwater-live-test:hunter', u'volumeMounts': [{u'readOnly': True, u'mountPath': u'/var/run/secrets/kubernetes.io/serviceaccount', u'name': u'default-token-c2jnd'}], u'terminationMessagePolicy': u'File', u'command': [u'rake', u'test[ims-w9fzp.svc.cluster.local]', u'PROXY=bono.ims-w9fzp.svc.cluster.local', u'ELLIS=ellis.ims-w9fzp.svc.cluster.local', u'SIGNUP_CODE=secret', u'--trace'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'nodeName': u'hunter-worker2'}, u'apiVersion': u'v1', u'metadata': {u'name': u'live-test', u'namespace': u'ims-w9fzp', u'resourceVersion': u'11212906', u'creationTimestamp': u'2020-11-27T11:10:48Z', u'selfLink': u'/api/v1/namespaces/ims-w9fzp/pods/live-test', u'uid': u'33c848de-30a1-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 10, 48, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': None, 'name': 'live-test', 'namespace': 'ims-w9fzp', 'owner_references': None, 'resource_version': '11212906', 'self_link': '/api/v1/namespaces/ims-w9fzp/pods/live-test', 'uid': '33c848de-30a1-11eb-8302-0242ac120002'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': ['rake', 'test[ims-w9fzp.svc.cluster.local]', 'PROXY=bono.ims-w9fzp.svc.cluster.local', 'ELLIS=ellis.ims-w9fzp.svc.cluster.local', 'SIGNUP_CODE=secret', '--trace'], 'env': None, 'env_from': None, 'image': 'ollivier/clearwater-live-test:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'live-test', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-c2jnd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': None, 'init_containers': None, 'node_name': 'hunter-worker2', 'node_selector': None, 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Never', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-c2jnd', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-c2jnd'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 10, 48, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': None, 'host_ip': None, 'init_container_statuses': None, 'message': None, 'nominated_node_name': None, 'phase': 'Pending', 'pod_ip': None, 'qos_class': 'BestEffort', 'reason': None, 'start_time': None}}, u'type': u'ADDED'} 2020-11-27 11:10:49,830 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'initContainerStatuses': [{u'restartCount': 0, u'name': u'wait-on-ralf', u'image': u'docker.io/library/busybox:1.28', u'imageID': u'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', u'state': {u'terminated': {u'startedAt': u'2020-11-27T11:04:29Z', u'reason': u'Completed', u'finishedAt': u'2020-11-27T11:07:11Z', u'containerID': u'containerd://897459c4b07c4ced30d56a47b8e5cd17c08549afc728dcd8530731653069f499', u'exitCode': 0}}, u'ready': True, u'lastState': {}, u'containerID': u'containerd://897459c4b07c4ced30d56a47b8e5cd17c08549afc728dcd8530731653069f499'}, {u'restartCount': 0, u'name': u'wait-on-chronos', u'image': u'docker.io/library/busybox:1.28', u'imageID': u'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', u'state': {u'terminated': {u'startedAt': u'2020-11-27T11:07:15Z', u'reason': u'Completed', u'finishedAt': u'2020-11-27T11:07:15Z', u'containerID': u'containerd://536e625b0261d1ca52f695849d29b46fdee44fcb8d8be7c2c511e919d3f2aa3a', u'exitCode': 0}}, u'ready': True, u'lastState': {}, u'containerID': u'containerd://536e625b0261d1ca52f695849d29b46fdee44fcb8d8be7c2c511e919d3f2aa3a'}, {u'restartCount': 0, u'name': u'wait-on-homestead', u'image': u'docker.io/library/busybox:1.28', u'imageID': u'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', u'state': {u'terminated': {u'startedAt': u'2020-11-27T11:07:19Z', u'reason': u'Completed', u'finishedAt': u'2020-11-27T11:08:08Z', u'containerID': u'containerd://5bf35fef5cc571b7ddbb1981b92dbc6056aebd7c25aab5d2910f950c938f911e', u'exitCode': 0}}, u'ready': True, u'lastState': {}, u'containerID': u'containerd://5bf35fef5cc571b7ddbb1981b92dbc6056aebd7c25aab5d2910f950c938f911e'}], u'qosClass': u'BestEffort', u'containerStatuses': [{u'restartCount': 0, u'name': u'sprout', u'image': u'docker.io/ollivier/clearwater-sprout:hunter', u'imageID': u'docker.io/ollivier/clearwater-sprout@sha256:dbc1c137720484bc549a753db56cc8b652e44787827364609510b9a73879d104', u'state': {u'running': {u'startedAt': u'2020-11-27T11:08:10Z'}}, u'ready': True, u'lastState': {}, u'containerID': u'containerd://fd4910e03bd71c888523ded39e0eefc4769b4124fb7cd00fd46850e5bb21d7ef'}, {u'restartCount': 0, u'name': u'tailer', u'image': u'docker.io/library/busybox:1.28', u'imageID': u'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', u'state': {u'running': {u'startedAt': u'2020-11-27T11:08:11Z'}}, u'ready': True, u'lastState': {}, u'containerID': u'containerd://3917636245805e663066996d5b8ec89a9304f6ac6dc43f9d3c33656d6cbfe108'}], u'podIP': u'10.244.1.118', u'startTime': u'2020-11-27T11:04:22Z', u'hostIP': u'172.18.0.4', u'phase': u'Running', u'conditions': [{u'status': u'True', u'lastProbeTime': None, u'type': u'Initialized', u'lastTransitionTime': u'2020-11-27T11:08:09Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'Ready', u'lastTransitionTime': u'2020-11-27T11:08:19Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'ContainersReady', u'lastTransitionTime': u'2020-11-27T11:08:19Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'PodScheduled', u'lastTransitionTime': u'2020-11-27T11:04:21Z'}]}, u'kind': u'Pod', u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'serviceAccountName': u'default', u'schedulerName': u'default-scheduler', u'enableServiceLinks': True, u'serviceAccount': u'default', u'priority': 0, u'terminationGracePeriodSeconds': 30, u'restartPolicy': u'Always', u'volumes': [{u'emptyDir': {}, u'name': u'sproutlogs'}, {u'secret': {u'defaultMode': 420, u'secretName': u'default-token-c2jnd'}, u'name': u'default-token-c2jnd'}], u'tolerations': [{u'operator': u'Exists', u'tolerationSeconds': 300, u'effect': u'NoExecute', u'key': u'node.kubernetes.io/not-ready'}, {u'operator': u'Exists', u'tolerationSeconds': 300, u'effect': u'NoExecute', u'key': u'node.kubernetes.io/unreachable'}], u'initContainers': [{u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-ralf', u'image': u'busybox:1.28', u'volumeMounts': [{u'readOnly': True, u'mountPath': u'/var/run/secrets/kubernetes.io/serviceaccount', u'name': u'default-token-c2jnd'}], u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup ralf; do echo waiting for ralf to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}, {u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-chronos', u'image': u'busybox:1.28', u'volumeMounts': [{u'readOnly': True, u'mountPath': u'/var/run/secrets/kubernetes.io/serviceaccount', u'name': u'default-token-c2jnd'}], u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup chronos; do echo waiting for chronos to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}, {u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-homestead', u'image': u'busybox:1.28', u'volumeMounts': [{u'readOnly': True, u'mountPath': u'/var/run/secrets/kubernetes.io/serviceaccount', u'name': u'default-token-c2jnd'}], u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup homestead; do echo waiting for homestead to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'imagePullSecrets': [{}], u'containers': [{u'livenessProbe': {u'timeoutSeconds': 1, u'exec': {u'command': [u'/bin/bash', u'/usr/share/kubernetes/liveness.sh', u'5052 5054']}, u'initialDelaySeconds': 30, u'periodSeconds': 3, u'successThreshold': 1, u'failureThreshold': 3}, u'terminationMessagePath': u'/dev/termination-log', u'name': u'sprout', u'envFrom': [{u'configMapRef': {u'name': u'env-vars'}}], u'image': u'ollivier/clearwater-sprout:hunter', u'volumeMounts': [{u'mountPath': u'/var/log/sprout', u'name': u'sproutlogs'}, {u'readOnly': True, u'mountPath': u'/var/run/secrets/kubernetes.io/serviceaccount', u'name': u'default-token-c2jnd'}], u'terminationMessagePolicy': u'File', u'env': [{u'valueFrom': {u'fieldRef': {u'fieldPath': u'status.podIP', u'apiVersion': u'v1'}}, u'name': u'MY_POD_IP'}], u'imagePullPolicy': u'IfNotPresent', u'readinessProbe': {u'successThreshold': 1, u'periodSeconds': 10, u'failureThreshold': 3, u'timeoutSeconds': 1, u'exec': {u'command': [u'/bin/bash', u'/usr/share/kubernetes/liveness.sh', u'5052 5054']}}, u'ports': [{u'protocol': u'TCP', u'containerPort': 22}], u'resources': {}}, {u'terminationMessagePath': u'/dev/termination-log', u'name': u'tailer', u'image': u'busybox:1.28', u'volumeMounts': [{u'mountPath': u'/var/log/sprout', u'name': u'sproutlogs'}, {u'readOnly': True, u'mountPath': u'/var/run/secrets/kubernetes.io/serviceaccount', u'name': u'default-token-c2jnd'}], u'terminationMessagePolicy': u'File', u'command': [u'tail', u'-F', u'/var/log/sprout/sprout_current.txt'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'nodeName': u'hunter-worker'}, u'apiVersion': u'v1', u'metadata': {u'name': u'sprout-55bcf5566-dgrt2', u'labels': {u'pod-template-hash': u'55bcf5566', u'snmp': u'enabled', u'service': u'sprout'}, u'namespace': u'ims-w9fzp', u'ownerReferences': [{u'kind': u'ReplicaSet', u'name': u'sprout-55bcf5566', u'apiVersion': u'apps/v1', u'controller': True, u'blockOwnerDeletion': True, u'uid': u'4d5b8f5d-30a0-11eb-8302-0242ac120002'}], u'resourceVersion': u'11212583', u'generateName': u'sprout-55bcf5566-', u'creationTimestamp': u'2020-11-27T11:04:21Z', u'selfLink': u'/api/v1/namespaces/ims-w9fzp/pods/sprout-55bcf5566-dgrt2', u'uid': u'4d8459c9-30a0-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'sprout-55bcf5566-', 'generation': None, 'initializers': None, 'labels': {u'pod-template-hash': '55bcf5566', u'service': 'sprout', u'snmp': 'enabled'}, 'managed_fields': None, 'name': 'sprout-55bcf5566-dgrt2', 'namespace': 'ims-w9fzp', 'owner_references': [{'api_version': 'apps/v1', 'block_owner_deletion': True, 'controller': True, 'kind': 'ReplicaSet', 'name': 'sprout-55bcf5566', 'uid': '4d5b8f5d-30a0-11eb-8302-0242ac120002'}], 'resource_version': '11212583', 'self_link': '/api/v1/namespaces/ims-w9fzp/pods/sprout-55bcf5566-dgrt2', 'uid': '4d8459c9-30a0-11eb-8302-0242ac120002'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-sprout:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 5054']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 3, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'sprout', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 5054']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/sprout', 'mount_propagation': None, 'name': 'sproutlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-c2jnd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/sprout/sprout_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/sprout', 'mount_propagation': None, 'name': 'sproutlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-c2jnd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup ralf; do echo waiting for ralf to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-ralf', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-c2jnd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until nslookup chronos; do echo waiting for chronos to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-chronos', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-c2jnd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until nslookup homestead; do echo waiting for homestead to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-homestead', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-c2jnd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'node_name': 'hunter-worker', 'node_selector': None, 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'sproutlogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}, {'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-c2jnd', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-c2jnd'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 8, 9, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 8, 19, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 8, 19, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://fd4910e03bd71c888523ded39e0eefc4769b4124fb7cd00fd46850e5bb21d7ef', 'image': 'docker.io/ollivier/clearwater-sprout:hunter', 'image_id': 'docker.io/ollivier/clearwater-sprout@sha256:dbc1c137720484bc549a753db56cc8b652e44787827364609510b9a73879d104', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'sprout', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 11, 27, 11, 8, 10, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}, {'container_id': 'containerd://3917636245805e663066996d5b8ec89a9304f6ac6dc43f9d3c33656d6cbfe108', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'tailer', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 11, 27, 11, 8, 11, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.4', 'init_container_statuses': [{'container_id': 'containerd://897459c4b07c4ced30d56a47b8e5cd17c08549afc728dcd8530731653069f499', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-ralf', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://897459c4b07c4ced30d56a47b8e5cd17c08549afc728dcd8530731653069f499', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 11, 27, 11, 7, 11, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 11, 27, 11, 4, 29, tzinfo=tzlocal())}, 'waiting': None}}, {'container_id': 'containerd://536e625b0261d1ca52f695849d29b46fdee44fcb8d8be7c2c511e919d3f2aa3a', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-chronos', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://536e625b0261d1ca52f695849d29b46fdee44fcb8d8be7c2c511e919d3f2aa3a', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 11, 27, 11, 7, 15, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 11, 27, 11, 7, 15, tzinfo=tzlocal())}, 'waiting': None}}, {'container_id': 'containerd://5bf35fef5cc571b7ddbb1981b92dbc6056aebd7c25aab5d2910f950c938f911e', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-homestead', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://5bf35fef5cc571b7ddbb1981b92dbc6056aebd7c25aab5d2910f950c938f911e', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 11, 27, 11, 8, 8, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 11, 27, 11, 7, 19, tzinfo=tzlocal())}, 'waiting': None}}], 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.1.118', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal())}}, u'type': u'ADDED'} 2020-11-27 11:10:50,370 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'initContainerStatuses': [{u'restartCount': 0, u'name': u'wait-on-cassandra', u'image': u'docker.io/library/busybox:1.28', u'imageID': u'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', u'state': {u'terminated': {u'startedAt': u'2020-11-27T11:04:34Z', u'reason': u'Completed', u'finishedAt': u'2020-11-27T11:06:36Z', u'containerID': u'containerd://935383e09363832d4158079c919835be4ad15057e00df73e24d83805562f35c5', u'exitCode': 0}}, u'ready': True, u'lastState': {}, u'containerID': u'containerd://935383e09363832d4158079c919835be4ad15057e00df73e24d83805562f35c5'}], u'qosClass': u'BestEffort', u'containerStatuses': [{u'restartCount': 0, u'name': u'ralf', u'image': u'docker.io/ollivier/clearwater-ralf:hunter', u'imageID': u'docker.io/ollivier/clearwater-ralf@sha256:5dfc81029d63fae2fba5c896a0ba94abd85c745e66e6efa4fc3b0bfec8c820d7', u'state': {u'running': {u'startedAt': u'2020-11-27T11:06:39Z'}}, u'ready': True, u'lastState': {}, u'containerID': u'containerd://2c47bc7dd17391d82bcaf11ab3437da6a126c9b61872d301ec78594e8c1f0c99'}, {u'restartCount': 0, u'name': u'tailer', u'image': u'docker.io/library/busybox:1.28', u'imageID': u'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', u'state': {u'running': {u'startedAt': u'2020-11-27T11:06:46Z'}}, u'ready': True, u'lastState': {}, u'containerID': u'containerd://d0d9b70148d7f1058a289995cce4b591e8ce003a7cdc5ce08f0f5904f70f0ffa'}], u'podIP': u'10.244.2.57', u'startTime': u'2020-11-27T11:04:22Z', u'hostIP': u'172.18.0.3', u'phase': u'Running', u'conditions': [{u'status': u'True', u'lastProbeTime': None, u'type': u'Initialized', u'lastTransitionTime': u'2020-11-27T11:06:37Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'Ready', u'lastTransitionTime': u'2020-11-27T11:06:49Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'ContainersReady', u'lastTransitionTime': u'2020-11-27T11:06:49Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'PodScheduled', u'lastTransitionTime': u'2020-11-27T11:04:21Z'}]}, u'kind': u'Pod', u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'serviceAccountName': u'default', u'schedulerName': u'default-scheduler', u'enableServiceLinks': True, u'serviceAccount': u'default', u'priority': 0, u'terminationGracePeriodSeconds': 30, u'restartPolicy': u'Always', u'volumes': [{u'emptyDir': {}, u'name': u'ralflogs'}, {u'secret': {u'defaultMode': 420, u'secretName': u'default-token-c2jnd'}, u'name': u'default-token-c2jnd'}], u'tolerations': [{u'operator': u'Exists', u'tolerationSeconds': 300, u'effect': u'NoExecute', u'key': u'node.kubernetes.io/not-ready'}, {u'operator': u'Exists', u'tolerationSeconds': 300, u'effect': u'NoExecute', u'key': u'node.kubernetes.io/unreachable'}], u'initContainers': [{u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-cassandra', u'image': u'busybox:1.28', u'volumeMounts': [{u'readOnly': True, u'mountPath': u'/var/run/secrets/kubernetes.io/serviceaccount', u'name': u'default-token-c2jnd'}], u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'imagePullSecrets': [{}], u'containers': [{u'livenessProbe': {u'tcpSocket': {u'port': 10888}, u'timeoutSeconds': 1, u'initialDelaySeconds': 30, u'periodSeconds': 10, u'successThreshold': 1, u'failureThreshold': 3}, u'terminationMessagePath': u'/dev/termination-log', u'name': u'ralf', u'envFrom': [{u'configMapRef': {u'name': u'env-vars'}}], u'image': u'ollivier/clearwater-ralf:hunter', u'volumeMounts': [{u'mountPath': u'/var/log/ralf', u'name': u'ralflogs'}, {u'readOnly': True, u'mountPath': u'/var/run/secrets/kubernetes.io/serviceaccount', u'name': u'default-token-c2jnd'}], u'terminationMessagePolicy': u'File', u'env': [{u'valueFrom': {u'fieldRef': {u'fieldPath': u'status.podIP', u'apiVersion': u'v1'}}, u'name': u'MY_POD_IP'}], u'imagePullPolicy': u'IfNotPresent', u'readinessProbe': {u'successThreshold': 1, u'tcpSocket': {u'port': 10888}, u'failureThreshold': 3, u'timeoutSeconds': 1, u'periodSeconds': 10}, u'ports': [{u'protocol': u'TCP', u'containerPort': 22}, {u'protocol': u'TCP', u'containerPort': 10888}], u'resources': {}}, {u'terminationMessagePath': u'/dev/termination-log', u'name': u'tailer', u'image': u'busybox:1.28', u'volumeMounts': [{u'mountPath': u'/var/log/ralf', u'name': u'ralflogs'}, {u'readOnly': True, u'mountPath': u'/var/run/secrets/kubernetes.io/serviceaccount', u'name': u'default-token-c2jnd'}], u'terminationMessagePolicy': u'File', u'command': [u'tail', u'-F', u'/var/log/ralf/ralf_current.txt'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'nodeName': u'hunter-worker2'}, u'apiVersion': u'v1', u'metadata': {u'name': u'ralf-797bfb6988-w8zn8', u'labels': {u'pod-template-hash': u'797bfb6988', u'snmp': u'enabled', u'service': u'ralf'}, u'namespace': u'ims-w9fzp', u'ownerReferences': [{u'kind': u'ReplicaSet', u'name': u'ralf-797bfb6988', u'apiVersion': u'apps/v1', u'controller': True, u'blockOwnerDeletion': True, u'uid': u'4d7183d4-30a0-11eb-8302-0242ac120002'}], u'resourceVersion': u'11212296', u'generateName': u'ralf-797bfb6988-', u'creationTimestamp': u'2020-11-27T11:04:21Z', u'selfLink': u'/api/v1/namespaces/ims-w9fzp/pods/ralf-797bfb6988-w8zn8', u'uid': u'4d8b391f-30a0-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'ralf-797bfb6988-', 'generation': None, 'initializers': None, 'labels': {u'pod-template-hash': '797bfb6988', u'service': 'ralf', u'snmp': 'enabled'}, 'managed_fields': None, 'name': 'ralf-797bfb6988-w8zn8', 'namespace': 'ims-w9fzp', 'owner_references': [{'api_version': 'apps/v1', 'block_owner_deletion': True, 'controller': True, 'kind': 'ReplicaSet', 'name': 'ralf-797bfb6988', 'uid': '4d7183d4-30a0-11eb-8302-0242ac120002'}], 'resource_version': '11212296', 'self_link': '/api/v1/namespaces/ims-w9fzp/pods/ralf-797bfb6988-w8zn8', 'uid': '4d8b391f-30a0-11eb-8302-0242ac120002'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-ralf:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 10888}, 'timeout_seconds': 1}, 'name': 'ralf', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 10888, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 10888}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/ralf', 'mount_propagation': None, 'name': 'ralflogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-c2jnd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/ralf/ralf_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/ralf', 'mount_propagation': None, 'name': 'ralflogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-c2jnd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-c2jnd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'node_name': 'hunter-worker2', 'node_selector': None, 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'ralflogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}, {'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-c2jnd', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-c2jnd'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 6, 37, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 6, 49, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 6, 49, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://2c47bc7dd17391d82bcaf11ab3437da6a126c9b61872d301ec78594e8c1f0c99', 'image': 'docker.io/ollivier/clearwater-ralf:hunter', 'image_id': 'docker.io/ollivier/clearwater-ralf@sha256:5dfc81029d63fae2fba5c896a0ba94abd85c745e66e6efa4fc3b0bfec8c820d7', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'ralf', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 11, 27, 11, 6, 39, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}, {'container_id': 'containerd://d0d9b70148d7f1058a289995cce4b591e8ce003a7cdc5ce08f0f5904f70f0ffa', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'tailer', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 11, 27, 11, 6, 46, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.3', 'init_container_statuses': [{'container_id': 'containerd://935383e09363832d4158079c919835be4ad15057e00df73e24d83805562f35c5', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-cassandra', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://935383e09363832d4158079c919835be4ad15057e00df73e24d83805562f35c5', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 11, 27, 11, 6, 36, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 11, 27, 11, 4, 34, tzinfo=tzlocal())}, 'waiting': None}}], 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.2.57', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal())}}, u'type': u'ADDED'} 2020-11-27 11:10:50,748 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'initContainerStatuses': [{u'restartCount': 0, u'name': u'wait-on-etcd', u'image': u'docker.io/library/busybox:1.28', u'imageID': u'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', u'state': {u'terminated': {u'startedAt': u'2020-11-27T11:04:29Z', u'reason': u'Completed', u'finishedAt': u'2020-11-27T11:05:01Z', u'containerID': u'containerd://2d6785db49b26f158a285174332cb4c82aa3220044cfbc50f993ec38968d7628', u'exitCode': 0}}, u'ready': True, u'lastState': {}, u'containerID': u'containerd://2d6785db49b26f158a285174332cb4c82aa3220044cfbc50f993ec38968d7628'}], u'qosClass': u'BestEffort', u'containerStatuses': [{u'restartCount': 0, u'name': u'cassandra', u'image': u'docker.io/ollivier/clearwater-cassandra:hunter', u'imageID': u'docker.io/ollivier/clearwater-cassandra@sha256:e0a6ca94460ea7a08848fca34ac5379e12427b63083a6cd409f4b4b94ccddb57', u'state': {u'running': {u'startedAt': u'2020-11-27T11:05:04Z'}}, u'ready': True, u'lastState': {}, u'containerID': u'containerd://bb772d2c572f98d62eaa751ca8e2f6b8d5f53dfc11fb73bace678c68a13a8387'}], u'podIP': u'10.244.2.53', u'startTime': u'2020-11-27T11:04:21Z', u'hostIP': u'172.18.0.3', u'phase': u'Running', u'conditions': [{u'status': u'True', u'lastProbeTime': None, u'type': u'Initialized', u'lastTransitionTime': u'2020-11-27T11:05:02Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'Ready', u'lastTransitionTime': u'2020-11-27T11:06:10Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'ContainersReady', u'lastTransitionTime': u'2020-11-27T11:06:10Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'PodScheduled', u'lastTransitionTime': u'2020-11-27T11:04:21Z'}]}, u'kind': u'Pod', u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'serviceAccountName': u'default', u'schedulerName': u'default-scheduler', u'enableServiceLinks': True, u'serviceAccount': u'default', u'priority': 0, u'terminationGracePeriodSeconds': 30, u'restartPolicy': u'Always', u'volumes': [{u'secret': {u'defaultMode': 420, u'secretName': u'default-token-c2jnd'}, u'name': u'default-token-c2jnd'}], u'tolerations': [{u'operator': u'Exists', u'tolerationSeconds': 300, u'effect': u'NoExecute', u'key': u'node.kubernetes.io/not-ready'}, {u'operator': u'Exists', u'tolerationSeconds': 300, u'effect': u'NoExecute', u'key': u'node.kubernetes.io/unreachable'}], u'initContainers': [{u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-etcd', u'image': u'busybox:1.28', u'volumeMounts': [{u'readOnly': True, u'mountPath': u'/var/run/secrets/kubernetes.io/serviceaccount', u'name': u'default-token-c2jnd'}], u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup etcd; do echo waiting for etcd to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'imagePullSecrets': [{}], u'containers': [{u'livenessProbe': {u'timeoutSeconds': 1, u'exec': {u'command': [u'/bin/bash', u'/usr/share/kubernetes/liveness.sh', u'7000 9042 9160']}, u'initialDelaySeconds': 600, u'periodSeconds': 10, u'successThreshold': 1, u'failureThreshold': 3}, u'terminationMessagePath': u'/dev/termination-log', u'name': u'cassandra', u'envFrom': [{u'configMapRef': {u'name': u'env-vars'}}], u'image': u'ollivier/clearwater-cassandra:hunter', u'volumeMounts': [{u'readOnly': True, u'mountPath': u'/var/run/secrets/kubernetes.io/serviceaccount', u'name': u'default-token-c2jnd'}], u'terminationMessagePolicy': u'File', u'env': [{u'valueFrom': {u'fieldRef': {u'fieldPath': u'status.podIP', u'apiVersion': u'v1'}}, u'name': u'MY_POD_IP'}], u'imagePullPolicy': u'IfNotPresent', u'readinessProbe': {u'successThreshold': 1, u'periodSeconds': 10, u'failureThreshold': 3, u'timeoutSeconds': 1, u'exec': {u'command': [u'/bin/bash', u'/usr/share/kubernetes/liveness.sh', u'7000 9042 9160']}}, u'ports': [{u'protocol': u'TCP', u'containerPort': 22}, {u'protocol': u'TCP', u'containerPort': 7001}, {u'protocol': u'TCP', u'containerPort': 9042}, {u'protocol': u'TCP', u'containerPort': 9160}], u'resources': {}}], u'nodeName': u'hunter-worker2'}, u'apiVersion': u'v1', u'metadata': {u'name': u'cassandra-56f89b7f94-htkfb', u'labels': {u'pod-template-hash': u'56f89b7f94', u'service': u'cassandra'}, u'namespace': u'ims-w9fzp', u'ownerReferences': [{u'kind': u'ReplicaSet', u'name': u'cassandra-56f89b7f94', u'apiVersion': u'apps/v1', u'controller': True, u'blockOwnerDeletion': True, u'uid': u'4d5b660b-30a0-11eb-8302-0242ac120002'}], u'resourceVersion': u'11212167', u'generateName': u'cassandra-56f89b7f94-', u'creationTimestamp': u'2020-11-27T11:04:21Z', u'selfLink': u'/api/v1/namespaces/ims-w9fzp/pods/cassandra-56f89b7f94-htkfb', u'uid': u'4d7ca52d-30a0-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'cassandra-56f89b7f94-', 'generation': None, 'initializers': None, 'labels': {u'pod-template-hash': '56f89b7f94', u'service': 'cassandra'}, 'managed_fields': None, 'name': 'cassandra-56f89b7f94-htkfb', 'namespace': 'ims-w9fzp', 'owner_references': [{'api_version': 'apps/v1', 'block_owner_deletion': True, 'controller': True, 'kind': 'ReplicaSet', 'name': 'cassandra-56f89b7f94', 'uid': '4d5b660b-30a0-11eb-8302-0242ac120002'}], 'resource_version': '11212167', 'self_link': '/api/v1/namespaces/ims-w9fzp/pods/cassandra-56f89b7f94-htkfb', 'uid': '4d7ca52d-30a0-11eb-8302-0242ac120002'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-cassandra:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 9042 9160']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 600, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'cassandra', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 7001, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 9042, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 9160, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 9042 9160']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-c2jnd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup etcd; do echo waiting for etcd to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-etcd', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-c2jnd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'node_name': 'hunter-worker2', 'node_selector': None, 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-c2jnd', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-c2jnd'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 5, 2, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 6, 10, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 6, 10, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://bb772d2c572f98d62eaa751ca8e2f6b8d5f53dfc11fb73bace678c68a13a8387', 'image': 'docker.io/ollivier/clearwater-cassandra:hunter', 'image_id': 'docker.io/ollivier/clearwater-cassandra@sha256:e0a6ca94460ea7a08848fca34ac5379e12427b63083a6cd409f4b4b94ccddb57', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'cassandra', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 11, 27, 11, 5, 4, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.3', 'init_container_statuses': [{'container_id': 'containerd://2d6785db49b26f158a285174332cb4c82aa3220044cfbc50f993ec38968d7628', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-etcd', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://2d6785db49b26f158a285174332cb4c82aa3220044cfbc50f993ec38968d7628', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 11, 27, 11, 5, 1, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 11, 27, 11, 4, 29, tzinfo=tzlocal())}, 'waiting': None}}], 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.2.53', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal())}}, u'type': u'ADDED'} 2020-11-27 11:10:51,053 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'initContainerStatuses': [{u'restartCount': 0, u'name': u'wait-on-cassandra', u'image': u'docker.io/library/busybox:1.28', u'imageID': u'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', u'state': {u'terminated': {u'startedAt': u'2020-11-27T11:04:34Z', u'reason': u'Completed', u'finishedAt': u'2020-11-27T11:06:38Z', u'containerID': u'containerd://cadeae1754a388805f7414e7484ba81837743d9a4cdfca33c5b3a24706bb0085', u'exitCode': 0}}, u'ready': True, u'lastState': {}, u'containerID': u'containerd://cadeae1754a388805f7414e7484ba81837743d9a4cdfca33c5b3a24706bb0085'}], u'qosClass': u'BestEffort', u'containerStatuses': [{u'restartCount': 0, u'name': u'chronos', u'image': u'docker.io/ollivier/clearwater-chronos:hunter', u'imageID': u'docker.io/ollivier/clearwater-chronos@sha256:b279a65dd77e19f7b33f1af41ab2938c35d3d05f232e525231dfce95183d589f', u'state': {u'running': {u'startedAt': u'2020-11-27T11:06:47Z'}}, u'ready': True, u'lastState': {}, u'containerID': u'containerd://ead58c11de97f7ded5e0dae8f5111535cdfb6d51a4877cc050501e478c6fe34f'}, {u'restartCount': 0, u'name': u'tailer', u'image': u'docker.io/library/busybox:1.28', u'imageID': u'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', u'state': {u'running': {u'startedAt': u'2020-11-27T11:06:49Z'}}, u'ready': True, u'lastState': {}, u'containerID': u'containerd://63a379426cab1530af0121aa7152635741abbecae53c540dc358729cf73988fd'}], u'podIP': u'10.244.2.59', u'startTime': u'2020-11-27T11:04:22Z', u'hostIP': u'172.18.0.3', u'phase': u'Running', u'conditions': [{u'status': u'True', u'lastProbeTime': None, u'type': u'Initialized', u'lastTransitionTime': u'2020-11-27T11:06:41Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'Ready', u'lastTransitionTime': u'2020-11-27T11:07:00Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'ContainersReady', u'lastTransitionTime': u'2020-11-27T11:07:00Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'PodScheduled', u'lastTransitionTime': u'2020-11-27T11:04:21Z'}]}, u'kind': u'Pod', u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'serviceAccountName': u'default', u'schedulerName': u'default-scheduler', u'enableServiceLinks': True, u'serviceAccount': u'default', u'priority': 0, u'terminationGracePeriodSeconds': 120, u'restartPolicy': u'Always', u'volumes': [{u'emptyDir': {}, u'name': u'chronoslogs'}, {u'secret': {u'defaultMode': 420, u'secretName': u'default-token-c2jnd'}, u'name': u'default-token-c2jnd'}], u'tolerations': [{u'operator': u'Exists', u'tolerationSeconds': 300, u'effect': u'NoExecute', u'key': u'node.kubernetes.io/not-ready'}, {u'operator': u'Exists', u'tolerationSeconds': 300, u'effect': u'NoExecute', u'key': u'node.kubernetes.io/unreachable'}], u'initContainers': [{u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-cassandra', u'image': u'busybox:1.28', u'volumeMounts': [{u'readOnly': True, u'mountPath': u'/var/run/secrets/kubernetes.io/serviceaccount', u'name': u'default-token-c2jnd'}], u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'imagePullSecrets': [{}], u'containers': [{u'livenessProbe': {u'successThreshold': 1, u'tcpSocket': {u'port': 7253}, u'failureThreshold': 9, u'timeoutSeconds': 1, u'periodSeconds': 10}, u'terminationMessagePath': u'/dev/termination-log', u'name': u'chronos', u'envFrom': [{u'configMapRef': {u'name': u'env-vars'}}], u'image': u'ollivier/clearwater-chronos:hunter', u'volumeMounts': [{u'mountPath': u'/var/log/chronos', u'name': u'chronoslogs'}, {u'readOnly': True, u'mountPath': u'/var/run/secrets/kubernetes.io/serviceaccount', u'name': u'default-token-c2jnd'}], u'terminationMessagePolicy': u'File', u'lifecycle': {u'preStop': {u'exec': {u'command': [u'/bin/bash', u'-c', u'/usr/bin/pre-stop']}}}, u'env': [{u'valueFrom': {u'fieldRef': {u'fieldPath': u'status.podIP', u'apiVersion': u'v1'}}, u'name': u'MY_POD_IP'}], u'imagePullPolicy': u'IfNotPresent', u'readinessProbe': {u'successThreshold': 1, u'tcpSocket': {u'port': 7253}, u'failureThreshold': 3, u'timeoutSeconds': 1, u'periodSeconds': 10}, u'ports': [{u'protocol': u'TCP', u'containerPort': 22}, {u'protocol': u'TCP', u'containerPort': 7253}], u'resources': {}}, {u'terminationMessagePath': u'/dev/termination-log', u'name': u'tailer', u'image': u'busybox:1.28', u'volumeMounts': [{u'mountPath': u'/var/log/chronos', u'name': u'chronoslogs'}, {u'readOnly': True, u'mountPath': u'/var/run/secrets/kubernetes.io/serviceaccount', u'name': u'default-token-c2jnd'}], u'terminationMessagePolicy': u'File', u'command': [u'tail', u'-F', u'/var/log/chronos/chronos_current.txt'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'nodeName': u'hunter-worker2'}, u'apiVersion': u'v1', u'metadata': {u'name': u'chronos-b74775b9-sb672', u'labels': {u'pod-template-hash': u'b74775b9', u'service': u'chronos'}, u'namespace': u'ims-w9fzp', u'ownerReferences': [{u'kind': u'ReplicaSet', u'name': u'chronos-b74775b9', u'apiVersion': u'apps/v1', u'controller': True, u'blockOwnerDeletion': True, u'uid': u'4d7194a6-30a0-11eb-8302-0242ac120002'}], u'resourceVersion': u'11212338', u'generateName': u'chronos-b74775b9-', u'creationTimestamp': u'2020-11-27T11:04:21Z', u'selfLink': u'/api/v1/namespaces/ims-w9fzp/pods/chronos-b74775b9-sb672', u'uid': u'4d932cb8-30a0-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'chronos-b74775b9-', 'generation': None, 'initializers': None, 'labels': {u'pod-template-hash': 'b74775b9', u'service': 'chronos'}, 'managed_fields': None, 'name': 'chronos-b74775b9-sb672', 'namespace': 'ims-w9fzp', 'owner_references': [{'api_version': 'apps/v1', 'block_owner_deletion': True, 'controller': True, 'kind': 'ReplicaSet', 'name': 'chronos-b74775b9', 'uid': '4d7194a6-30a0-11eb-8302-0242ac120002'}], 'resource_version': '11212338', 'self_link': '/api/v1/namespaces/ims-w9fzp/pods/chronos-b74775b9-sb672', 'uid': '4d932cb8-30a0-11eb-8302-0242ac120002'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-chronos:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': {'post_start': None, 'pre_stop': {'_exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}, 'http_get': None, 'tcp_socket': None}}, 'liveness_probe': {'_exec': None, 'failure_threshold': 9, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7253}, 'timeout_seconds': 1}, 'name': 'chronos', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 7253, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7253}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/chronos', 'mount_propagation': None, 'name': 'chronoslogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-c2jnd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/chronos/chronos_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/chronos', 'mount_propagation': None, 'name': 'chronoslogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-c2jnd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-c2jnd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'node_name': 'hunter-worker2', 'node_selector': None, 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 120, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'chronoslogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}, {'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-c2jnd', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-c2jnd'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 6, 41, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 7, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 7, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://ead58c11de97f7ded5e0dae8f5111535cdfb6d51a4877cc050501e478c6fe34f', 'image': 'docker.io/ollivier/clearwater-chronos:hunter', 'image_id': 'docker.io/ollivier/clearwater-chronos@sha256:b279a65dd77e19f7b33f1af41ab2938c35d3d05f232e525231dfce95183d589f', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'chronos', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 11, 27, 11, 6, 47, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}, {'container_id': 'containerd://63a379426cab1530af0121aa7152635741abbecae53c540dc358729cf73988fd', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'tailer', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 11, 27, 11, 6, 49, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.3', 'init_container_statuses': [{'container_id': 'containerd://cadeae1754a388805f7414e7484ba81837743d9a4cdfca33c5b3a24706bb0085', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-cassandra', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://cadeae1754a388805f7414e7484ba81837743d9a4cdfca33c5b3a24706bb0085', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 11, 27, 11, 6, 38, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 11, 27, 11, 4, 34, tzinfo=tzlocal())}, 'waiting': None}}], 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.2.59', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal())}}, u'type': u'ADDED'} 2020-11-27 11:10:51,442 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'initContainerStatuses': [{u'restartCount': 0, u'name': u'wait-on-homestead-prov', u'image': u'docker.io/library/busybox:1.28', u'imageID': u'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', u'state': {u'terminated': {u'startedAt': u'2020-11-27T11:04:34Z', u'reason': u'Completed', u'finishedAt': u'2020-11-27T11:08:35Z', u'containerID': u'containerd://b80ab8443027f662e5482d2d8dd53992422666f91e2358bac783e4cacedd7227', u'exitCode': 0}}, u'ready': True, u'lastState': {}, u'containerID': u'containerd://b80ab8443027f662e5482d2d8dd53992422666f91e2358bac783e4cacedd7227'}, {u'restartCount': 0, u'name': u'wait-on-sprout', u'image': u'docker.io/library/busybox:1.28', u'imageID': u'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', u'state': {u'terminated': {u'startedAt': u'2020-11-27T11:08:37Z', u'reason': u'Completed', u'finishedAt': u'2020-11-27T11:08:37Z', u'containerID': u'containerd://589af7404effbaf993c6c02469f4e78636174a9041ee5046440d99aa30f83e09', u'exitCode': 0}}, u'ready': True, u'lastState': {}, u'containerID': u'containerd://589af7404effbaf993c6c02469f4e78636174a9041ee5046440d99aa30f83e09'}], u'qosClass': u'BestEffort', u'containerStatuses': [{u'restartCount': 0, u'name': u'ellis', u'image': u'docker.io/ollivier/clearwater-ellis:hunter', u'imageID': u'docker.io/ollivier/clearwater-ellis@sha256:177c949e01031a68e56315b8b70974e0fdd9bedece200dbcd68c9a0859bca477', u'state': {u'running': {u'startedAt': u'2020-11-27T11:08:39Z'}}, u'ready': True, u'lastState': {}, u'containerID': u'containerd://ae271c3be103aacff931ecda94ccdfb183b08e9c510b50c71ff531b89446ae53'}], u'podIP': u'10.244.2.58', u'startTime': u'2020-11-27T11:04:22Z', u'hostIP': u'172.18.0.3', u'phase': u'Running', u'conditions': [{u'status': u'True', u'lastProbeTime': None, u'type': u'Initialized', u'lastTransitionTime': u'2020-11-27T11:08:38Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'Ready', u'lastTransitionTime': u'2020-11-27T11:08:47Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'ContainersReady', u'lastTransitionTime': u'2020-11-27T11:08:47Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'PodScheduled', u'lastTransitionTime': u'2020-11-27T11:04:21Z'}]}, u'kind': u'Pod', u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'serviceAccountName': u'default', u'schedulerName': u'default-scheduler', u'enableServiceLinks': True, u'serviceAccount': u'default', u'priority': 0, u'terminationGracePeriodSeconds': 30, u'restartPolicy': u'Always', u'volumes': [{u'secret': {u'defaultMode': 420, u'secretName': u'default-token-c2jnd'}, u'name': u'default-token-c2jnd'}], u'tolerations': [{u'operator': u'Exists', u'tolerationSeconds': 300, u'effect': u'NoExecute', u'key': u'node.kubernetes.io/not-ready'}, {u'operator': u'Exists', u'tolerationSeconds': 300, u'effect': u'NoExecute', u'key': u'node.kubernetes.io/unreachable'}], u'initContainers': [{u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-homestead-prov', u'image': u'busybox:1.28', u'volumeMounts': [{u'readOnly': True, u'mountPath': u'/var/run/secrets/kubernetes.io/serviceaccount', u'name': u'default-token-c2jnd'}], u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup homestead-prov; do echo waiting for homestead-prov to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}, {u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-sprout', u'image': u'busybox:1.28', u'volumeMounts': [{u'readOnly': True, u'mountPath': u'/var/run/secrets/kubernetes.io/serviceaccount', u'name': u'default-token-c2jnd'}], u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup sprout; do echo waiting for sprout to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'imagePullSecrets': [{}], u'containers': [{u'livenessProbe': {u'tcpSocket': {u'port': 80}, u'timeoutSeconds': 1, u'initialDelaySeconds': 30, u'periodSeconds': 10, u'successThreshold': 1, u'failureThreshold': 3}, u'terminationMessagePath': u'/dev/termination-log', u'name': u'ellis', u'envFrom': [{u'configMapRef': {u'name': u'env-vars'}}], u'image': u'ollivier/clearwater-ellis:hunter', u'volumeMounts': [{u'readOnly': True, u'mountPath': u'/var/run/secrets/kubernetes.io/serviceaccount', u'name': u'default-token-c2jnd'}], u'terminationMessagePolicy': u'File', u'env': [{u'valueFrom': {u'fieldRef': {u'fieldPath': u'status.podIP', u'apiVersion': u'v1'}}, u'name': u'MY_POD_IP'}], u'imagePullPolicy': u'IfNotPresent', u'readinessProbe': {u'successThreshold': 1, u'tcpSocket': {u'port': 80}, u'failureThreshold': 3, u'timeoutSeconds': 1, u'periodSeconds': 10}, u'ports': [{u'protocol': u'TCP', u'containerPort': 22}, {u'protocol': u'TCP', u'containerPort': 80}], u'resources': {}}], u'nodeName': u'hunter-worker2'}, u'apiVersion': u'v1', u'metadata': {u'name': u'ellis-857689fbdd-t8fjx', u'labels': {u'pod-template-hash': u'857689fbdd', u'service': u'ellis'}, u'namespace': u'ims-w9fzp', u'ownerReferences': [{u'kind': u'ReplicaSet', u'name': u'ellis-857689fbdd', u'apiVersion': u'apps/v1', u'controller': True, u'blockOwnerDeletion': True, u'uid': u'4d71774f-30a0-11eb-8302-0242ac120002'}], u'resourceVersion': u'11212665', u'generateName': u'ellis-857689fbdd-', u'creationTimestamp': u'2020-11-27T11:04:21Z', u'selfLink': u'/api/v1/namespaces/ims-w9fzp/pods/ellis-857689fbdd-t8fjx', u'uid': u'4d8b3c28-30a0-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'ellis-857689fbdd-', 'generation': None, 'initializers': None, 'labels': {u'pod-template-hash': '857689fbdd', u'service': 'ellis'}, 'managed_fields': None, 'name': 'ellis-857689fbdd-t8fjx', 'namespace': 'ims-w9fzp', 'owner_references': [{'api_version': 'apps/v1', 'block_owner_deletion': True, 'controller': True, 'kind': 'ReplicaSet', 'name': 'ellis-857689fbdd', 'uid': '4d71774f-30a0-11eb-8302-0242ac120002'}], 'resource_version': '11212665', 'self_link': '/api/v1/namespaces/ims-w9fzp/pods/ellis-857689fbdd-t8fjx', 'uid': '4d8b3c28-30a0-11eb-8302-0242ac120002'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-ellis:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 80}, 'timeout_seconds': 1}, 'name': 'ellis', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 80, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 80}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-c2jnd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup homestead-prov; do echo waiting for homestead-prov to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-homestead-prov', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-c2jnd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until nslookup sprout; do echo waiting for sprout to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-sprout', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-c2jnd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'node_name': 'hunter-worker2', 'node_selector': None, 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-c2jnd', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-c2jnd'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 8, 38, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 8, 47, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 8, 47, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://ae271c3be103aacff931ecda94ccdfb183b08e9c510b50c71ff531b89446ae53', 'image': 'docker.io/ollivier/clearwater-ellis:hunter', 'image_id': 'docker.io/ollivier/clearwater-ellis@sha256:177c949e01031a68e56315b8b70974e0fdd9bedece200dbcd68c9a0859bca477', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'ellis', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 11, 27, 11, 8, 39, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.3', 'init_container_statuses': [{'container_id': 'containerd://b80ab8443027f662e5482d2d8dd53992422666f91e2358bac783e4cacedd7227', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-homestead-prov', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://b80ab8443027f662e5482d2d8dd53992422666f91e2358bac783e4cacedd7227', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 11, 27, 11, 8, 35, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 11, 27, 11, 4, 34, tzinfo=tzlocal())}, 'waiting': None}}, {'container_id': 'containerd://589af7404effbaf993c6c02469f4e78636174a9041ee5046440d99aa30f83e09', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-sprout', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://589af7404effbaf993c6c02469f4e78636174a9041ee5046440d99aa30f83e09', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 11, 27, 11, 8, 37, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 11, 27, 11, 8, 37, tzinfo=tzlocal())}, 'waiting': None}}], 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.2.58', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal())}}, u'type': u'ADDED'} 2020-11-27 11:10:51,775 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'qosClass': u'BestEffort', u'containerStatuses': [{u'restartCount': 0, u'name': u'etcd', u'image': u'quay.io/coreos/etcd:v2.2.5', u'imageID': u'sha256:66590e2dd05422771324d8ffe9ad35165ba35db0842ac5e0f1668c2d56733860', u'state': {u'running': {u'startedAt': u'2020-11-27T11:04:33Z'}}, u'ready': True, u'lastState': {}, u'containerID': u'containerd://71ba7581d4bb6fc51a8a35845a89bda56be0599e6aa2f4860850ada2d1e48acf'}], u'podIP': u'10.244.2.55', u'startTime': u'2020-11-27T11:04:22Z', u'hostIP': u'172.18.0.3', u'phase': u'Running', u'conditions': [{u'status': u'True', u'lastProbeTime': None, u'type': u'Initialized', u'lastTransitionTime': u'2020-11-27T11:04:22Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'Ready', u'lastTransitionTime': u'2020-11-27T11:04:38Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'ContainersReady', u'lastTransitionTime': u'2020-11-27T11:04:38Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'PodScheduled', u'lastTransitionTime': u'2020-11-27T11:04:21Z'}]}, u'kind': u'Pod', u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'serviceAccountName': u'default', u'schedulerName': u'default-scheduler', u'enableServiceLinks': True, u'serviceAccount': u'default', u'priority': 0, u'terminationGracePeriodSeconds': 30, u'restartPolicy': u'Always', u'volumes': [{u'secret': {u'defaultMode': 420, u'secretName': u'default-token-c2jnd'}, u'name': u'default-token-c2jnd'}], u'tolerations': [{u'operator': u'Exists', u'tolerationSeconds': 300, u'effect': u'NoExecute', u'key': u'node.kubernetes.io/not-ready'}, {u'operator': u'Exists', u'tolerationSeconds': 300, u'effect': u'NoExecute', u'key': u'node.kubernetes.io/unreachable'}], u'imagePullSecrets': [{}], u'containers': [{u'livenessProbe': {u'tcpSocket': {u'port': 4001}, u'timeoutSeconds': 1, u'initialDelaySeconds': 300, u'periodSeconds': 10, u'successThreshold': 1, u'failureThreshold': 3}, u'terminationMessagePath': u'/dev/termination-log', u'name': u'etcd', u'image': u'quay.io/coreos/etcd:v2.2.5', u'args': [u'--name', u'$(MY_POD_NAME)', u'--advertise-client-urls', u'http://$(MY_POD_IP):2379,http://$(MY_POD_IP):4001', u'--listen-client-urls', u'http://0.0.0.0:2379,http://0.0.0.0:4001', u'--initial-advertise-peer-urls', u'http://$(MY_POD_IP):2380', u'--listen-peer-urls', u'http://0.0.0.0:2380', u'--initial-cluster', u'$(MY_POD_NAME)=http://$(MY_POD_IP):2380', u'--initial-cluster-state', u'new'], u'volumeMounts': [{u'readOnly': True, u'mountPath': u'/var/run/secrets/kubernetes.io/serviceaccount', u'name': u'default-token-c2jnd'}], u'terminationMessagePolicy': u'File', u'env': [{u'valueFrom': {u'fieldRef': {u'fieldPath': u'status.podIP', u'apiVersion': u'v1'}}, u'name': u'MY_POD_IP'}, {u'valueFrom': {u'fieldRef': {u'fieldPath': u'metadata.name', u'apiVersion': u'v1'}}, u'name': u'MY_POD_NAME'}], u'imagePullPolicy': u'IfNotPresent', u'readinessProbe': {u'successThreshold': 1, u'tcpSocket': {u'port': 4001}, u'failureThreshold': 3, u'timeoutSeconds': 1, u'periodSeconds': 10}, u'ports': [{u'protocol': u'TCP', u'containerPort': 2379}, {u'protocol': u'TCP', u'containerPort': 2380}, {u'protocol': u'TCP', u'containerPort': 4001}], u'resources': {}}], u'nodeName': u'hunter-worker2'}, u'apiVersion': u'v1', u'metadata': {u'name': u'etcd-c8fddbb95-jc89j', u'labels': {u'pod-template-hash': u'c8fddbb95', u'instance-type': u'etcd-pod'}, u'namespace': u'ims-w9fzp', u'ownerReferences': [{u'kind': u'ReplicaSet', u'name': u'etcd-c8fddbb95', u'apiVersion': u'apps/v1', u'controller': True, u'blockOwnerDeletion': True, u'uid': u'4d5e39c2-30a0-11eb-8302-0242ac120002'}], u'resourceVersion': u'11211923', u'generateName': u'etcd-c8fddbb95-', u'creationTimestamp': u'2020-11-27T11:04:21Z', u'selfLink': u'/api/v1/namespaces/ims-w9fzp/pods/etcd-c8fddbb95-jc89j', u'uid': u'4d7cc2f8-30a0-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'etcd-c8fddbb95-', 'generation': None, 'initializers': None, 'labels': {u'instance-type': 'etcd-pod', u'pod-template-hash': 'c8fddbb95'}, 'managed_fields': None, 'name': 'etcd-c8fddbb95-jc89j', 'namespace': 'ims-w9fzp', 'owner_references': [{'api_version': 'apps/v1', 'block_owner_deletion': True, 'controller': True, 'kind': 'ReplicaSet', 'name': 'etcd-c8fddbb95', 'uid': '4d5e39c2-30a0-11eb-8302-0242ac120002'}], 'resource_version': '11211923', 'self_link': '/api/v1/namespaces/ims-w9fzp/pods/etcd-c8fddbb95-jc89j', 'uid': '4d7cc2f8-30a0-11eb-8302-0242ac120002'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': ['--name', '$(MY_POD_NAME)', '--advertise-client-urls', 'http://$(MY_POD_IP):2379,http://$(MY_POD_IP):4001', '--listen-client-urls', 'http://0.0.0.0:2379,http://0.0.0.0:4001', '--initial-advertise-peer-urls', 'http://$(MY_POD_IP):2380', '--listen-peer-urls', 'http://0.0.0.0:2380', '--initial-cluster', '$(MY_POD_NAME)=http://$(MY_POD_IP):2380', '--initial-cluster-state', 'new'], 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}, {'name': 'MY_POD_NAME', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'metadata.name'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': None, 'image': 'quay.io/coreos/etcd:v2.2.5', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 300, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 4001}, 'timeout_seconds': 1}, 'name': 'etcd', 'ports': [{'container_port': 2379, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 2380, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 4001, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 4001}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-c2jnd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': None, 'node_name': 'hunter-worker2', 'node_selector': None, 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-c2jnd', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-c2jnd'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 38, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 38, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://71ba7581d4bb6fc51a8a35845a89bda56be0599e6aa2f4860850ada2d1e48acf', 'image': 'quay.io/coreos/etcd:v2.2.5', 'image_id': 'sha256:66590e2dd05422771324d8ffe9ad35165ba35db0842ac5e0f1668c2d56733860', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'etcd', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 11, 27, 11, 4, 33, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.3', 'init_container_statuses': None, 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.2.55', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal())}}, u'type': u'ADDED'} 2020-11-27 11:10:52,046 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'initContainerStatuses': [{u'restartCount': 0, u'name': u'wait-on-cassandra', u'image': u'docker.io/library/busybox:1.28', u'imageID': u'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', u'state': {u'terminated': {u'startedAt': u'2020-11-27T11:04:33Z', u'reason': u'Completed', u'finishedAt': u'2020-11-27T11:06:38Z', u'containerID': u'containerd://59f07095e5fbb94c2c35dc3ffc272b9a53fe3ccefd4d693c611d8cc803c51599', u'exitCode': 0}}, u'ready': True, u'lastState': {}, u'containerID': u'containerd://59f07095e5fbb94c2c35dc3ffc272b9a53fe3ccefd4d693c611d8cc803c51599'}, {u'restartCount': 0, u'name': u'wait-on-astaire', u'image': u'docker.io/library/busybox:1.28', u'imageID': u'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', u'state': {u'terminated': {u'startedAt': u'2020-11-27T11:06:46Z', u'reason': u'Completed', u'finishedAt': u'2020-11-27T11:07:46Z', u'containerID': u'containerd://69c4bd48b39f7127ed7805947701a48a9672dd7281ecf8d94ffd7219fbd0999a', u'exitCode': 0}}, u'ready': True, u'lastState': {}, u'containerID': u'containerd://69c4bd48b39f7127ed7805947701a48a9672dd7281ecf8d94ffd7219fbd0999a'}], u'qosClass': u'BestEffort', u'containerStatuses': [{u'restartCount': 0, u'name': u'homestead', u'image': u'docker.io/ollivier/clearwater-homestead:hunter', u'imageID': u'docker.io/ollivier/clearwater-homestead@sha256:709052509ff00074676a8d53840c92e45004d4bc0c1615ff4ede6bfccf90437d', u'state': {u'running': {u'startedAt': u'2020-11-27T11:07:49Z'}}, u'ready': True, u'lastState': {}, u'containerID': u'containerd://22fee5a10c268613715318fc28d7cbd1a3f5a1a77827eeaa4e032986327a328c'}, {u'restartCount': 0, u'name': u'tailer', u'image': u'docker.io/library/busybox:1.28', u'imageID': u'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', u'state': {u'running': {u'startedAt': u'2020-11-27T11:07:50Z'}}, u'ready': True, u'lastState': {}, u'containerID': u'containerd://1a74a9fd611152a8a6772d03d1776825c4b6852d1f78d684b0da95277977af74'}], u'podIP': u'10.244.2.56', u'startTime': u'2020-11-27T11:04:22Z', u'hostIP': u'172.18.0.3', u'phase': u'Running', u'conditions': [{u'status': u'True', u'lastProbeTime': None, u'type': u'Initialized', u'lastTransitionTime': u'2020-11-27T11:07:48Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'Ready', u'lastTransitionTime': u'2020-11-27T11:08:03Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'ContainersReady', u'lastTransitionTime': u'2020-11-27T11:08:03Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'PodScheduled', u'lastTransitionTime': u'2020-11-27T11:04:21Z'}]}, u'kind': u'Pod', u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'serviceAccountName': u'default', u'schedulerName': u'default-scheduler', u'enableServiceLinks': True, u'serviceAccount': u'default', u'priority': 0, u'terminationGracePeriodSeconds': 30, u'restartPolicy': u'Always', u'volumes': [{u'emptyDir': {}, u'name': u'homesteadlogs'}, {u'secret': {u'defaultMode': 420, u'secretName': u'default-token-c2jnd'}, u'name': u'default-token-c2jnd'}], u'tolerations': [{u'operator': u'Exists', u'tolerationSeconds': 300, u'effect': u'NoExecute', u'key': u'node.kubernetes.io/not-ready'}, {u'operator': u'Exists', u'tolerationSeconds': 300, u'effect': u'NoExecute', u'key': u'node.kubernetes.io/unreachable'}], u'initContainers': [{u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-cassandra', u'image': u'busybox:1.28', u'volumeMounts': [{u'readOnly': True, u'mountPath': u'/var/run/secrets/kubernetes.io/serviceaccount', u'name': u'default-token-c2jnd'}], u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}, {u'terminationMessagePath': u'/dev/termination-log', u'name': u'wait-on-astaire', u'image': u'busybox:1.28', u'volumeMounts': [{u'readOnly': True, u'mountPath': u'/var/run/secrets/kubernetes.io/serviceaccount', u'name': u'default-token-c2jnd'}], u'terminationMessagePolicy': u'File', u'command': [u'sh', u'-c', u'until nslookup astaire; do echo waiting for astaire to start ...; sleep 2; done'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'imagePullSecrets': [{}], u'containers': [{u'livenessProbe': {u'timeoutSeconds': 1, u'exec': {u'command': [u'/bin/bash', u'/usr/share/kubernetes/liveness.sh', u'8888']}, u'initialDelaySeconds': 60, u'periodSeconds': 10, u'successThreshold': 1, u'failureThreshold': 3}, u'terminationMessagePath': u'/dev/termination-log', u'name': u'homestead', u'envFrom': [{u'configMapRef': {u'name': u'env-vars'}}], u'image': u'ollivier/clearwater-homestead:hunter', u'volumeMounts': [{u'mountPath': u'/var/log/homestead', u'name': u'homesteadlogs'}, {u'readOnly': True, u'mountPath': u'/var/run/secrets/kubernetes.io/serviceaccount', u'name': u'default-token-c2jnd'}], u'terminationMessagePolicy': u'File', u'env': [{u'valueFrom': {u'fieldRef': {u'fieldPath': u'status.podIP', u'apiVersion': u'v1'}}, u'name': u'MY_POD_IP'}], u'imagePullPolicy': u'IfNotPresent', u'readinessProbe': {u'successThreshold': 1, u'periodSeconds': 10, u'failureThreshold': 3, u'timeoutSeconds': 1, u'exec': {u'command': [u'/bin/bash', u'/usr/share/kubernetes/liveness.sh', u'8888']}}, u'ports': [{u'protocol': u'TCP', u'containerPort': 22}, {u'protocol': u'TCP', u'containerPort': 8888}], u'resources': {}}, {u'terminationMessagePath': u'/dev/termination-log', u'name': u'tailer', u'image': u'busybox:1.28', u'volumeMounts': [{u'mountPath': u'/var/log/homestead', u'name': u'homesteadlogs'}, {u'readOnly': True, u'mountPath': u'/var/run/secrets/kubernetes.io/serviceaccount', u'name': u'default-token-c2jnd'}], u'terminationMessagePolicy': u'File', u'command': [u'tail', u'-F', u'/var/log/homestead/homestead_current.txt'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'nodeName': u'hunter-worker2'}, u'apiVersion': u'v1', u'metadata': {u'name': u'homestead-7f8f7d7cc9-wfjbg', u'labels': {u'pod-template-hash': u'7f8f7d7cc9', u'snmp': u'enabled', u'service': u'homestead'}, u'namespace': u'ims-w9fzp', u'ownerReferences': [{u'kind': u'ReplicaSet', u'name': u'homestead-7f8f7d7cc9', u'apiVersion': u'apps/v1', u'controller': True, u'blockOwnerDeletion': True, u'uid': u'4d6b8d26-30a0-11eb-8302-0242ac120002'}], u'resourceVersion': u'11212537', u'generateName': u'homestead-7f8f7d7cc9-', u'creationTimestamp': u'2020-11-27T11:04:21Z', u'selfLink': u'/api/v1/namespaces/ims-w9fzp/pods/homestead-7f8f7d7cc9-wfjbg', u'uid': u'4d8b3004-30a0-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'homestead-7f8f7d7cc9-', 'generation': None, 'initializers': None, 'labels': {u'pod-template-hash': '7f8f7d7cc9', u'service': 'homestead', u'snmp': 'enabled'}, 'managed_fields': None, 'name': 'homestead-7f8f7d7cc9-wfjbg', 'namespace': 'ims-w9fzp', 'owner_references': [{'api_version': 'apps/v1', 'block_owner_deletion': True, 'controller': True, 'kind': 'ReplicaSet', 'name': 'homestead-7f8f7d7cc9', 'uid': '4d6b8d26-30a0-11eb-8302-0242ac120002'}], 'resource_version': '11212537', 'self_link': '/api/v1/namespaces/ims-w9fzp/pods/homestead-7f8f7d7cc9-wfjbg', 'uid': '4d8b3004-30a0-11eb-8302-0242ac120002'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-homestead:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 60, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'homestead', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 8888, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/homestead', 'mount_propagation': None, 'name': 'homesteadlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-c2jnd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/homestead/homestead_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/homestead', 'mount_propagation': None, 'name': 'homesteadlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-c2jnd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-c2jnd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until nslookup astaire; do echo waiting for astaire to start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-astaire', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-c2jnd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'node_name': 'hunter-worker2', 'node_selector': None, 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'homesteadlogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}, {'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-c2jnd', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-c2jnd'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 7, 48, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 8, 3, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 8, 3, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 4, 21, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://22fee5a10c268613715318fc28d7cbd1a3f5a1a77827eeaa4e032986327a328c', 'image': 'docker.io/ollivier/clearwater-homestead:hunter', 'image_id': 'docker.io/ollivier/clearwater-homestead@sha256:709052509ff00074676a8d53840c92e45004d4bc0c1615ff4ede6bfccf90437d', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'homestead', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 11, 27, 11, 7, 49, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}, {'container_id': 'containerd://1a74a9fd611152a8a6772d03d1776825c4b6852d1f78d684b0da95277977af74', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'tailer', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 11, 27, 11, 7, 50, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.3', 'init_container_statuses': [{'container_id': 'containerd://59f07095e5fbb94c2c35dc3ffc272b9a53fe3ccefd4d693c611d8cc803c51599', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-cassandra', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://59f07095e5fbb94c2c35dc3ffc272b9a53fe3ccefd4d693c611d8cc803c51599', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 11, 27, 11, 6, 38, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 11, 27, 11, 4, 33, tzinfo=tzlocal())}, 'waiting': None}}, {'container_id': 'containerd://69c4bd48b39f7127ed7805947701a48a9672dd7281ecf8d94ffd7219fbd0999a', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-astaire', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://69c4bd48b39f7127ed7805947701a48a9672dd7281ecf8d94ffd7219fbd0999a', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 11, 27, 11, 7, 46, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 11, 27, 11, 6, 46, tzinfo=tzlocal())}, 'waiting': None}}], 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.2.56', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 11, 27, 11, 4, 22, tzinfo=tzlocal())}}, u'type': u'ADDED'} 2020-11-27 11:10:52,463 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'qosClass': u'BestEffort', u'containerStatuses': [{u'restartCount': 0, u'name': u'live-test', u'image': u'ollivier/clearwater-live-test:hunter', u'imageID': u'', u'state': {u'waiting': {u'reason': u'ContainerCreating'}}, u'ready': False, u'lastState': {}}], u'startTime': u'2020-11-27T11:10:48Z', u'hostIP': u'172.18.0.3', u'phase': u'Pending', u'conditions': [{u'status': u'True', u'lastProbeTime': None, u'type': u'Initialized', u'lastTransitionTime': u'2020-11-27T11:10:48Z'}, {u'status': u'False', u'lastTransitionTime': u'2020-11-27T11:10:48Z', u'reason': u'ContainersNotReady', u'lastProbeTime': None, u'message': u'containers with unready status: [live-test]', u'type': u'Ready'}, {u'status': u'False', u'lastTransitionTime': u'2020-11-27T11:10:48Z', u'reason': u'ContainersNotReady', u'lastProbeTime': None, u'message': u'containers with unready status: [live-test]', u'type': u'ContainersReady'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'PodScheduled', u'lastTransitionTime': u'2020-11-27T11:10:48Z'}]}, u'kind': u'Pod', u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'serviceAccountName': u'default', u'schedulerName': u'default-scheduler', u'enableServiceLinks': True, u'serviceAccount': u'default', u'priority': 0, u'terminationGracePeriodSeconds': 30, u'restartPolicy': u'Never', u'volumes': [{u'secret': {u'defaultMode': 420, u'secretName': u'default-token-c2jnd'}, u'name': u'default-token-c2jnd'}], u'tolerations': [{u'operator': u'Exists', u'tolerationSeconds': 300, u'effect': u'NoExecute', u'key': u'node.kubernetes.io/not-ready'}, {u'operator': u'Exists', u'tolerationSeconds': 300, u'effect': u'NoExecute', u'key': u'node.kubernetes.io/unreachable'}], u'containers': [{u'terminationMessagePath': u'/dev/termination-log', u'name': u'live-test', u'image': u'ollivier/clearwater-live-test:hunter', u'volumeMounts': [{u'readOnly': True, u'mountPath': u'/var/run/secrets/kubernetes.io/serviceaccount', u'name': u'default-token-c2jnd'}], u'terminationMessagePolicy': u'File', u'command': [u'rake', u'test[ims-w9fzp.svc.cluster.local]', u'PROXY=bono.ims-w9fzp.svc.cluster.local', u'ELLIS=ellis.ims-w9fzp.svc.cluster.local', u'SIGNUP_CODE=secret', u'--trace'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'nodeName': u'hunter-worker2'}, u'apiVersion': u'v1', u'metadata': {u'name': u'live-test', u'namespace': u'ims-w9fzp', u'resourceVersion': u'11212908', u'creationTimestamp': u'2020-11-27T11:10:48Z', u'selfLink': u'/api/v1/namespaces/ims-w9fzp/pods/live-test', u'uid': u'33c848de-30a1-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 10, 48, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': None, 'name': 'live-test', 'namespace': 'ims-w9fzp', 'owner_references': None, 'resource_version': '11212908', 'self_link': '/api/v1/namespaces/ims-w9fzp/pods/live-test', 'uid': '33c848de-30a1-11eb-8302-0242ac120002'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': ['rake', 'test[ims-w9fzp.svc.cluster.local]', 'PROXY=bono.ims-w9fzp.svc.cluster.local', 'ELLIS=ellis.ims-w9fzp.svc.cluster.local', 'SIGNUP_CODE=secret', '--trace'], 'env': None, 'env_from': None, 'image': 'ollivier/clearwater-live-test:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'live-test', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-c2jnd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': None, 'init_containers': None, 'node_name': 'hunter-worker2', 'node_selector': None, 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Never', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-c2jnd', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-c2jnd'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 10, 48, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 10, 48, tzinfo=tzlocal()), 'message': 'containers with unready status: [live-test]', 'reason': 'ContainersNotReady', 'status': 'False', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 10, 48, tzinfo=tzlocal()), 'message': 'containers with unready status: [live-test]', 'reason': 'ContainersNotReady', 'status': 'False', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 10, 48, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': None, 'image': 'ollivier/clearwater-live-test:hunter', 'image_id': '', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'live-test', 'ready': False, 'restart_count': 0, 'state': {'running': None, 'terminated': None, 'waiting': {'message': None, 'reason': 'ContainerCreating'}}}], 'host_ip': '172.18.0.3', 'init_container_statuses': None, 'message': None, 'nominated_node_name': None, 'phase': 'Pending', 'pod_ip': None, 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 11, 27, 11, 10, 48, tzinfo=tzlocal())}}, u'type': u'MODIFIED'} 2020-11-27 11:10:52,635 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'qosClass': u'BestEffort', u'containerStatuses': [{u'restartCount': 0, u'name': u'live-test', u'image': u'docker.io/ollivier/clearwater-live-test:hunter', u'imageID': u'docker.io/ollivier/clearwater-live-test@sha256:178bbf9b98c044c0d84af42f9adcd167d6b5e6658219ebfbd352215dbc27dc97', u'state': {u'running': {u'startedAt': u'2020-11-27T11:10:51Z'}}, u'ready': True, u'lastState': {}, u'containerID': u'containerd://87b09b470ea3672dc62f2f505179fea689cfdecf0ed478154c0105ebd1a445a9'}], u'podIP': u'10.244.2.62', u'startTime': u'2020-11-27T11:10:48Z', u'hostIP': u'172.18.0.3', u'phase': u'Running', u'conditions': [{u'status': u'True', u'lastProbeTime': None, u'type': u'Initialized', u'lastTransitionTime': u'2020-11-27T11:10:48Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'Ready', u'lastTransitionTime': u'2020-11-27T11:10:51Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'ContainersReady', u'lastTransitionTime': u'2020-11-27T11:10:51Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'PodScheduled', u'lastTransitionTime': u'2020-11-27T11:10:48Z'}]}, u'kind': u'Pod', u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'serviceAccountName': u'default', u'schedulerName': u'default-scheduler', u'enableServiceLinks': True, u'serviceAccount': u'default', u'priority': 0, u'terminationGracePeriodSeconds': 30, u'restartPolicy': u'Never', u'volumes': [{u'secret': {u'defaultMode': 420, u'secretName': u'default-token-c2jnd'}, u'name': u'default-token-c2jnd'}], u'tolerations': [{u'operator': u'Exists', u'tolerationSeconds': 300, u'effect': u'NoExecute', u'key': u'node.kubernetes.io/not-ready'}, {u'operator': u'Exists', u'tolerationSeconds': 300, u'effect': u'NoExecute', u'key': u'node.kubernetes.io/unreachable'}], u'containers': [{u'terminationMessagePath': u'/dev/termination-log', u'name': u'live-test', u'image': u'ollivier/clearwater-live-test:hunter', u'volumeMounts': [{u'readOnly': True, u'mountPath': u'/var/run/secrets/kubernetes.io/serviceaccount', u'name': u'default-token-c2jnd'}], u'terminationMessagePolicy': u'File', u'command': [u'rake', u'test[ims-w9fzp.svc.cluster.local]', u'PROXY=bono.ims-w9fzp.svc.cluster.local', u'ELLIS=ellis.ims-w9fzp.svc.cluster.local', u'SIGNUP_CODE=secret', u'--trace'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'nodeName': u'hunter-worker2'}, u'apiVersion': u'v1', u'metadata': {u'name': u'live-test', u'namespace': u'ims-w9fzp', u'resourceVersion': u'11212920', u'creationTimestamp': u'2020-11-27T11:10:48Z', u'selfLink': u'/api/v1/namespaces/ims-w9fzp/pods/live-test', u'uid': u'33c848de-30a1-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 10, 48, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': None, 'name': 'live-test', 'namespace': 'ims-w9fzp', 'owner_references': None, 'resource_version': '11212920', 'self_link': '/api/v1/namespaces/ims-w9fzp/pods/live-test', 'uid': '33c848de-30a1-11eb-8302-0242ac120002'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': ['rake', 'test[ims-w9fzp.svc.cluster.local]', 'PROXY=bono.ims-w9fzp.svc.cluster.local', 'ELLIS=ellis.ims-w9fzp.svc.cluster.local', 'SIGNUP_CODE=secret', '--trace'], 'env': None, 'env_from': None, 'image': 'ollivier/clearwater-live-test:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'live-test', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-c2jnd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': None, 'init_containers': None, 'node_name': 'hunter-worker2', 'node_selector': None, 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Never', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-c2jnd', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-c2jnd'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 10, 48, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 10, 51, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 10, 51, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 10, 48, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://87b09b470ea3672dc62f2f505179fea689cfdecf0ed478154c0105ebd1a445a9', 'image': 'docker.io/ollivier/clearwater-live-test:hunter', 'image_id': 'docker.io/ollivier/clearwater-live-test@sha256:178bbf9b98c044c0d84af42f9adcd167d6b5e6658219ebfbd352215dbc27dc97', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'live-test', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 11, 27, 11, 10, 51, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.3', 'init_container_statuses': None, 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.2.62', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 11, 27, 11, 10, 48, tzinfo=tzlocal())}}, u'type': u'MODIFIED'} 2020-11-27 11:13:30,147 - functest_kubernetes.ims.ims - DEBUG - {'raw_object': {u'status': {u'qosClass': u'BestEffort', u'containerStatuses': [{u'restartCount': 0, u'name': u'live-test', u'image': u'docker.io/ollivier/clearwater-live-test:hunter', u'imageID': u'docker.io/ollivier/clearwater-live-test@sha256:178bbf9b98c044c0d84af42f9adcd167d6b5e6658219ebfbd352215dbc27dc97', u'state': {u'terminated': {u'startedAt': u'2020-11-27T11:10:51Z', u'reason': u'Completed', u'finishedAt': u'2020-11-27T11:13:28Z', u'containerID': u'containerd://87b09b470ea3672dc62f2f505179fea689cfdecf0ed478154c0105ebd1a445a9', u'exitCode': 0}}, u'ready': False, u'lastState': {}, u'containerID': u'containerd://87b09b470ea3672dc62f2f505179fea689cfdecf0ed478154c0105ebd1a445a9'}], u'podIP': u'10.244.2.62', u'startTime': u'2020-11-27T11:10:48Z', u'hostIP': u'172.18.0.3', u'phase': u'Succeeded', u'conditions': [{u'status': u'True', u'lastProbeTime': None, u'reason': u'PodCompleted', u'type': u'Initialized', u'lastTransitionTime': u'2020-11-27T11:10:48Z'}, {u'status': u'False', u'lastProbeTime': None, u'reason': u'PodCompleted', u'type': u'Ready', u'lastTransitionTime': u'2020-11-27T11:13:30Z'}, {u'status': u'False', u'lastProbeTime': None, u'reason': u'PodCompleted', u'type': u'ContainersReady', u'lastTransitionTime': u'2020-11-27T11:13:30Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'PodScheduled', u'lastTransitionTime': u'2020-11-27T11:10:48Z'}]}, u'kind': u'Pod', u'spec': {u'dnsPolicy': u'ClusterFirst', u'securityContext': {}, u'serviceAccountName': u'default', u'schedulerName': u'default-scheduler', u'enableServiceLinks': True, u'serviceAccount': u'default', u'priority': 0, u'terminationGracePeriodSeconds': 30, u'restartPolicy': u'Never', u'volumes': [{u'secret': {u'defaultMode': 420, u'secretName': u'default-token-c2jnd'}, u'name': u'default-token-c2jnd'}], u'tolerations': [{u'operator': u'Exists', u'tolerationSeconds': 300, u'effect': u'NoExecute', u'key': u'node.kubernetes.io/not-ready'}, {u'operator': u'Exists', u'tolerationSeconds': 300, u'effect': u'NoExecute', u'key': u'node.kubernetes.io/unreachable'}], u'containers': [{u'terminationMessagePath': u'/dev/termination-log', u'name': u'live-test', u'image': u'ollivier/clearwater-live-test:hunter', u'volumeMounts': [{u'readOnly': True, u'mountPath': u'/var/run/secrets/kubernetes.io/serviceaccount', u'name': u'default-token-c2jnd'}], u'terminationMessagePolicy': u'File', u'command': [u'rake', u'test[ims-w9fzp.svc.cluster.local]', u'PROXY=bono.ims-w9fzp.svc.cluster.local', u'ELLIS=ellis.ims-w9fzp.svc.cluster.local', u'SIGNUP_CODE=secret', u'--trace'], u'imagePullPolicy': u'IfNotPresent', u'resources': {}}], u'nodeName': u'hunter-worker2'}, u'apiVersion': u'v1', u'metadata': {u'name': u'live-test', u'namespace': u'ims-w9fzp', u'resourceVersion': u'11213467', u'creationTimestamp': u'2020-11-27T11:10:48Z', u'selfLink': u'/api/v1/namespaces/ims-w9fzp/pods/live-test', u'uid': u'33c848de-30a1-11eb-8302-0242ac120002'}}, u'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 11, 27, 11, 10, 48, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': None, 'name': 'live-test', 'namespace': 'ims-w9fzp', 'owner_references': None, 'resource_version': '11213467', 'self_link': '/api/v1/namespaces/ims-w9fzp/pods/live-test', 'uid': '33c848de-30a1-11eb-8302-0242ac120002'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': ['rake', 'test[ims-w9fzp.svc.cluster.local]', 'PROXY=bono.ims-w9fzp.svc.cluster.local', 'ELLIS=ellis.ims-w9fzp.svc.cluster.local', 'SIGNUP_CODE=secret', '--trace'], 'env': None, 'env_from': None, 'image': 'ollivier/clearwater-live-test:hunter', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'live-test', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-c2jnd', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': None, 'init_containers': None, 'node_name': 'hunter-worker2', 'node_selector': None, 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Never', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-c2jnd', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-c2jnd'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 10, 48, tzinfo=tzlocal()), 'message': None, 'reason': 'PodCompleted', 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 13, 30, tzinfo=tzlocal()), 'message': None, 'reason': 'PodCompleted', 'status': 'False', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 13, 30, tzinfo=tzlocal()), 'message': None, 'reason': 'PodCompleted', 'status': 'False', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 11, 27, 11, 10, 48, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://87b09b470ea3672dc62f2f505179fea689cfdecf0ed478154c0105ebd1a445a9', 'image': 'docker.io/ollivier/clearwater-live-test:hunter', 'image_id': 'docker.io/ollivier/clearwater-live-test@sha256:178bbf9b98c044c0d84af42f9adcd167d6b5e6658219ebfbd352215dbc27dc97', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'live-test', 'ready': False, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://87b09b470ea3672dc62f2f505179fea689cfdecf0ed478154c0105ebd1a445a9', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 11, 27, 11, 13, 28, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 11, 27, 11, 10, 51, tzinfo=tzlocal())}, 'waiting': None}}], 'host_ip': '172.18.0.3', 'init_container_statuses': None, 'message': None, 'nominated_node_name': None, 'phase': 'Succeeded', 'pod_ip': '10.244.2.62', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 11, 27, 11, 10, 48, tzinfo=tzlocal())}}, u'type': u'MODIFIED'} 2020-11-27 11:13:30,515 - kubernetes.client.rest - DEBUG - response body: ** Invoke test (first_time) ** Execute test /usr/lib/ruby/gems/2.6.0/gems/httpi-2.4.0/lib/httpi/auth/ssl.rb:13: warning: constant OpenSSL::SSL::SSLContext::METHODS is deprecated # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' Basic Call - Mainline (TCP) - (6505550699, 6505550865) Passed Basic Call - SDP (TCP) - (6505550664, 6505550391) Passed Basic Call - Tel URIs (TCP) - (6505550019, 6505550118) Passed Basic Call - Unknown number (TCP) - (6505550742, 6505550659) Passed Basic Call - Rejected by remote endpoint (TCP) - (6505550580, 6505550979) Passed Basic Call - Messages - Pager model (TCP) - (6505550702, 6505550979) Passed Basic Call - Pracks (TCP) - (6505550484, 6505550067) Passed Basic Registration (TCP) - (6505550051) Passed Emergency Registration (TCP) - Skipped (Not running with emergency registrations) - Call with EMERGENCY_REG=Y to run test Multiple Identities (TCP) - (6505550517, 6505550495) Passed Call Barring - Outbound Rejection (TCP) - (6505550649, 6505550129) Passed Call Barring - Allow non-international call (TCP) - Skipped (No PSTN support) - Call with PSTN=true to run test Call Barring - Reject international call (TCP) - Skipped (No PSTN support) - Call with PSTN=true to run test Call Barring - Inbound Rejection (TCP) - (6505550654, 6505550544) Passed Call Diversion AS - Unconditional (TCP) - Skipped (No Call Diversion AS hostname provided) - Call with CDIV_AS= Call Diversion AS - Busy (TCP) - Skipped (No Call Diversion AS hostname provided) - Call with CDIV_AS= Call Diversion AS - No answer (TCP) - Skipped (No Call Diversion AS hostname provided) - Call with CDIV_AS= Call Diversion - Not registered (TCP) - (6505550139, 6505550112, 6505550853) Passed Call Diversion - Not reachable (not registered) (TCP) - (6505550143, 6505550982, 6505550811) Passed Call Diversion - Not reachable (408) (TCP) - (6505550872, 6505550845, 6505550894) Passed Call Diversion - Not reachable (503) (TCP) - (6505550954, 6505550313, 6505550870) Passed Call Diversion - Not reachable (500) (TCP) - (6505550785, 6505550278, 6505550841) Passed Call Diversion - Busy (TCP) - (6505550222, 6505550218, 6505550266) Passed Call Diversion - Unconditional (TCP) - (6505550204, 6505550811, 6505550532) Passed Call Diversion - No answer (TCP) - (6505550806, 6505550687, 6505550770) Passed Call Diversion - Bad target URI (TCP) - (6505550613, 6505550311) Passed Call Diversion - Audio-only call (TCP) - (6505550942, 6505550922, 6505550518, 6505550918) Passed Call Diversion - Audio-video call (TCP) - (6505550618, 6505550203, 6505550266, 6505550561) Passed Call Forking - Mainline (TCP) - (6505550736, 6505550120) Passed Call Forking - Endpoint offline (TCP) - (6505550959, 6505550055) Passed Call Forking - Endpoint offline while ringing (TCP) - (6505550998, 6505550835) Passed Call Waiting - Accepted (TCP) - (6505550161, 6505550189, 6505550946) Passed Call Waiting - Cancelled (TCP) - (6505550925, 6505550783, 6505550049) Passed CANCEL - Mainline (TCP) - (6505550609, 6505550852) Passed Filtering - Accept-Contact (TCP) - (6505550065, 6505550456) Passed Accept-Contact with response (SFR 511195) (TCP) - (6505550768, 6505550952) Passed Filtering - Accept-Contact no match (TCP) - (6505550042, 6505550869) Passed Filtering - Accept-Contact negated match (TCP) - (6505550429, 6505550495) Passed Filtering - RFC3841 example (TCP) - (6505550988, 6505550979) Passed Filtering - Reject-Contact no match (TCP) - (6505550525, 6505550213) Passed Filtering - Reject-Contact match (TCP) - (6505550356, 6505550285) Passed Gemini - INVITE - VoIP device answers (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Mobile device answers (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - VoIP device rejects (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Mobile device rejects (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Mobile device rejects with a 480 (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Both reject, choose mobile response (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Both reject, choose VoIP response (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Successful call with GR (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Failed call with GR (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Successful call with Accept-Contact (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Failed call with Accept-Contact (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - SUBSCRIBE - Mobile Notifies (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - SUBSCRIBE - Joint 408 (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= GRUU - REGISTER - two bindings with and without GRUU (TCP) - (6505550727) Passed GRUU - REGISTER - binding suggested GRUU (TCP) - (6505550797) Passed GRUU - REGISTER - instance ID requires escaping (TCP) - (6505550652) Passed GRUU - REGISTER - three bindings with GRUUs (TCP) - (6505550588) Passed GRUU - Call - first endpoint GRUU as target (TCP) - (6505550659, 6505550790) Passed GRUU - Call - second endpoint GRUU as target (TCP) - (6505550634, 6505550543) Passed GRUU - Call - only GRUU as target (TCP) - (6505550461, 6505550972) Passed GRUU - Call - AoR as target (TCP) - (6505550004, 6505550523) Passed GRUU - Call - unknown GRUU as target (TCP) - (6505550754, 6505550958) Passed GRUU - Call - unknown GRUU as target - no GRUUs assigned (TCP) - (6505550378, 6505550620) Passed GRUU - Call - Reject-Contact interop (TCP) - (6505550718, 6505550392) Passed GRUU - Call - Accept-Contact interop (TCP) - (6505550787, 6505550310) Passed GRUU - Call - AoR with other param as target (TCP) - (6505550847, 6505550384) Passed GRUU - Call - GRUU with other param as target (TCP) - (6505550215, 6505550110) Passed ISC Interface - Terminating (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - Terminating (UDP AS) (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - Terminating Failed (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - Third-party Registration (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - Third-party Registration - implicit registration (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - Redirect (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - B2BUA (TCP) - Skipped (No hostname given) - Call with HOST_NAME= Live Call - Dial out to a real number (TCP) - Skipped (No live number given) - Call with LIVENUMBER= Memento - Incorrect Password (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Wrong Call List (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Basic Call (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Unknown Number (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Rejected Call (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Cancelled Call (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Privacy Call (TCP) - Skipped (No memento hostnames provided) /usr/lib/ruby/gems/2.6.0/gems/nokogiri-1.6.6.2/lib/nokogiri/xml/document.rb:44: warning: constant ::Fixnum is deprecated # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Barred Call (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Busy Call Forwarding (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Nonce-Count - Mainline (TCP) - Skipped (No nonce-count support) - Call with NONCE_COUNT=Y to run test Nonce-Count - Reject Re-Use (TCP) - Skipped (No nonce-count support) - Call with NONCE_COUNT=Y to run test Off-net calls - tel: URI (TCP) - Skipped (No off-net number given) - Call with OFF_NET_TEL= Off-net calls - sip: URI (TCP) - Skipped (No off-net number given) - Call with OFF_NET_TEL= SUBSCRIBE - reg-event (TCP) - (6505550810) Passed SUBSCRIBE - reg-event with a GRUU (TCP) - (6505550184) Passed SUBSCRIBE - Subscription timeout (TCP) - (6505550207) Passed SUBSCRIBE - Registration timeout (TCP) - Skipped (Not running with short registrations) - Call with SHORT_REG=Y to run test Multiple SUBSCRIBErs to one UE's reg-event (TCP) - (6505550005, 6505550170) Passed Message - URN URIs (TCP) - Skipped (No hostname given) - Call with HOST_NAME= Message - URN URIs to AS (TCP) - Skipped (No hostname given) - Call with HOST_NAME= 0 failures out of 96 tests run 43 tests skipped 2020-11-27 11:13:30,541 - functest_kubernetes.ims.ims - INFO - ** Invoke test (first_time) ** Execute test /usr/lib/ruby/gems/2.6.0/gems/httpi-2.4.0/lib/httpi/auth/ssl.rb:13: warning: constant OpenSSL::SSL::SSLContext::METHODS is deprecated # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' Basic Call - Mainline (TCP) - (6505550699, 6505550865) Passed Basic Call - SDP (TCP) - (6505550664, 6505550391) Passed Basic Call - Tel URIs (TCP) - (6505550019, 6505550118) Passed Basic Call - Unknown number (TCP) - (6505550742, 6505550659) Passed Basic Call - Rejected by remote endpoint (TCP) - (6505550580, 6505550979) Passed Basic Call - Messages - Pager model (TCP) - (6505550702, 6505550979) Passed Basic Call - Pracks (TCP) - (6505550484, 6505550067) Passed Basic Registration (TCP) - (6505550051) Passed Emergency Registration (TCP) - Skipped (Not running with emergency registrations) - Call with EMERGENCY_REG=Y to run test Multiple Identities (TCP) - (6505550517, 6505550495) Passed Call Barring - Outbound Rejection (TCP) - (6505550649, 6505550129) Passed Call Barring - Allow non-international call (TCP) - Skipped (No PSTN support) - Call with PSTN=true to run test Call Barring - Reject international call (TCP) - Skipped (No PSTN support) - Call with PSTN=true to run test Call Barring - Inbound Rejection (TCP) - (6505550654, 6505550544) Passed Call Diversion AS - Unconditional (TCP) - Skipped (No Call Diversion AS hostname provided) - Call with CDIV_AS= Call Diversion AS - Busy (TCP) - Skipped (No Call Diversion AS hostname provided) - Call with CDIV_AS= Call Diversion AS - No answer (TCP) - Skipped (No Call Diversion AS hostname provided) - Call with CDIV_AS= Call Diversion - Not registered (TCP) - (6505550139, 6505550112, 6505550853) Passed Call Diversion - Not reachable (not registered) (TCP) - (6505550143, 6505550982, 6505550811) Passed Call Diversion - Not reachable (408) (TCP) - (6505550872, 6505550845, 6505550894) Passed Call Diversion - Not reachable (503) (TCP) - (6505550954, 6505550313, 6505550870) Passed Call Diversion - Not reachable (500) (TCP) - (6505550785, 6505550278, 6505550841) Passed Call Diversion - Busy (TCP) - (6505550222, 6505550218, 6505550266) Passed Call Diversion - Unconditional (TCP) - (6505550204, 6505550811, 6505550532) Passed Call Diversion - No answer (TCP) - (6505550806, 6505550687, 6505550770) Passed Call Diversion - Bad target URI (TCP) - (6505550613, 6505550311) Passed Call Diversion - Audio-only call (TCP) - (6505550942, 6505550922, 6505550518, 6505550918) Passed Call Diversion - Audio-video call (TCP) - (6505550618, 6505550203, 6505550266, 6505550561) Passed Call Forking - Mainline (TCP) - (6505550736, 6505550120) Passed Call Forking - Endpoint offline (TCP) - (6505550959, 6505550055) Passed Call Forking - Endpoint offline while ringing (TCP) - (6505550998, 6505550835) Passed Call Waiting - Accepted (TCP) - (6505550161, 6505550189, 6505550946) Passed Call Waiting - Cancelled (TCP) - (6505550925, 6505550783, 6505550049) Passed CANCEL - Mainline (TCP) - (6505550609, 6505550852) Passed Filtering - Accept-Contact (TCP) - (6505550065, 6505550456) Passed Accept-Contact with response (SFR 511195) (TCP) - (6505550768, 6505550952) Passed Filtering - Accept-Contact no match (TCP) - (6505550042, 6505550869) Passed Filtering - Accept-Contact negated match (TCP) - (6505550429, 6505550495) Passed Filtering - RFC3841 example (TCP) - (6505550988, 6505550979) Passed Filtering - Reject-Contact no match (TCP) - (6505550525, 6505550213) Passed Filtering - Reject-Contact match (TCP) - (6505550356, 6505550285) Passed Gemini - INVITE - VoIP device answers (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Mobile device answers (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - VoIP device rejects (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Mobile device rejects (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Mobile device rejects with a 480 (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Both reject, choose mobile response (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Both reject, choose VoIP response (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Successful call with GR (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Failed call with GR (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Successful call with Accept-Contact (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Failed call with Accept-Contact (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - SUBSCRIBE - Mobile Notifies (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - SUBSCRIBE - Joint 408 (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= GRUU - REGISTER - two bindings with and without GRUU (TCP) - (6505550727) Passed GRUU - REGISTER - binding suggested GRUU (TCP) - (6505550797) Passed GRUU - REGISTER - instance ID requires escaping (TCP) - (6505550652) Passed GRUU - REGISTER - three bindings with GRUUs (TCP) - (6505550588) Passed GRUU - Call - first endpoint GRUU as target (TCP) - (6505550659, 6505550790) Passed GRUU - Call - second endpoint GRUU as target (TCP) - (6505550634, 6505550543) Passed GRUU - Call - only GRUU as target (TCP) - (6505550461, 6505550972) Passed GRUU - Call - AoR as target (TCP) - (6505550004, 6505550523) Passed GRUU - Call - unknown GRUU as target (TCP) - (6505550754, 6505550958) Passed GRUU - Call - unknown GRUU as target - no GRUUs assigned (TCP) - (6505550378, 6505550620) Passed GRUU - Call - Reject-Contact interop (TCP) - (6505550718, 6505550392) Passed GRUU - Call - Accept-Contact interop (TCP) - (6505550787, 6505550310) Passed GRUU - Call - AoR with other param as target (TCP) - (6505550847, 6505550384) Passed GRUU - Call - GRUU with other param as target (TCP) - (6505550215, 6505550110) Passed ISC Interface - Terminating (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - Terminating (UDP AS) (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - Terminating Failed (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - Third-party Registration (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - Third-party Registration - implicit registration (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - Redirect (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - B2BUA (TCP) - Skipped (No hostname given) - Call with HOST_NAME= Live Call - Dial out to a real number (TCP) - Skipped (No live number given) - Call with LIVENUMBER= Memento - Incorrect Password (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Wrong Call List (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Basic Call (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Unknown Number (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Rejected Call (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Cancelled Call (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Privacy Call (TCP) - Skipped (No memento hostnames provided) /usr/lib/ruby/gems/2.6.0/gems/nokogiri-1.6.6.2/lib/nokogiri/xml/document.rb:44: warning: constant ::Fixnum is deprecated # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Barred Call (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Busy Call Forwarding (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Nonce-Count - Mainline (TCP) - Skipped (No nonce-count support) - Call with NONCE_COUNT=Y to run test Nonce-Count - Reject Re-Use (TCP) - Skipped (No nonce-count support) - Call with NONCE_COUNT=Y to run test Off-net calls - tel: URI (TCP) - Skipped (No off-net number given) - Call with OFF_NET_TEL= Off-net calls - sip: URI (TCP) - Skipped (No off-net number given) - Call with OFF_NET_TEL= SUBSCRIBE - reg-event (TCP) - (6505550810) Passed SUBSCRIBE - reg-event with a GRUU (TCP) - (6505550184) Passed SUBSCRIBE - Subscription timeout (TCP) - (6505550207) Passed SUBSCRIBE - Registration timeout (TCP) - Skipped (Not running with short registrations) - Call with SHORT_REG=Y to run test Multiple SUBSCRIBErs to one UE's reg-event (TCP) - (6505550005, 6505550170) Passed Message - URN URIs (TCP) - Skipped (No hostname given) - Call with HOST_NAME= Message - URN URIs to AS (TCP) - Skipped (No hostname given) - Call with HOST_NAME= 0 failures out of 96 tests run 43 tests skipped 2020-11-27 11:13:30,556 - xtesting.ci.run_tests - INFO - Test result: +-------------------+------------------+------------------+----------------+ | TEST CASE | PROJECT | DURATION | RESULT | +-------------------+------------------+------------------+----------------+ | helm_vims | functest | 10:09 | PASS | +-------------------+------------------+------------------+----------------+ 2020-11-27 11:14:05,669 - functest_kubernetes.ims.ims - DEBUG - time="2020-11-27T11:13:32Z" level=error msg="failure getting variant" error="getCPUInfo for pattern: Cpu architecture: not found" I1127 11:13:44.265662 33 request.go:621] Throttling request took 1.373299424s, request: GET:https://172.30.12.66:43795/apis/authorization.k8s.io/v1beta1?timeout=32s release "clearwater" uninstalled 2020-11-27 11:14:05,690 - kubernetes.client.rest - DEBUG - response body: {"kind":"Pod","apiVersion":"v1","metadata":{"name":"live-test","namespace":"ims-w9fzp","selfLink":"/api/v1/namespaces/ims-w9fzp/pods/live-test","uid":"33c848de-30a1-11eb-8302-0242ac120002","resourceVersion":"11213588","creationTimestamp":"2020-11-27T11:10:48Z","deletionTimestamp":"2020-11-27T11:14:05Z","deletionGracePeriodSeconds":0},"spec":{"volumes":[{"name":"default-token-c2jnd","secret":{"secretName":"default-token-c2jnd","defaultMode":420}}],"containers":[{"name":"live-test","image":"ollivier/clearwater-live-test:hunter","command":["rake","test[ims-w9fzp.svc.cluster.local]","PROXY=bono.ims-w9fzp.svc.cluster.local","ELLIS=ellis.ims-w9fzp.svc.cluster.local","SIGNUP_CODE=secret","--trace"],"resources":{},"volumeMounts":[{"name":"default-token-c2jnd","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Never","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"hunter-worker2","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}],"priority":0,"enableServiceLinks":true},"status":{"phase":"Succeeded","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2020-11-27T11:10:48Z","reason":"PodCompleted"},{"type":"Ready","status":"False","lastProbeTime":null,"lastTransitionTime":"2020-11-27T11:13:30Z","reason":"PodCompleted"},{"type":"ContainersReady","status":"False","lastProbeTime":null,"lastTransitionTime":"2020-11-27T11:13:30Z","reason":"PodCompleted"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2020-11-27T11:10:48Z"}],"hostIP":"172.18.0.3","podIP":"10.244.2.62","startTime":"2020-11-27T11:10:48Z","containerStatuses":[{"name":"live-test","state":{"terminated":{"exitCode":0,"reason":"Completed","startedAt":"2020-11-27T11:10:51Z","finishedAt":"2020-11-27T11:13:28Z","containerID":"containerd://87b09b470ea3672dc62f2f505179fea689cfdecf0ed478154c0105ebd1a445a9"}},"lastState":{},"ready":false,"restartCount":0,"image":"docker.io/ollivier/clearwater-live-test:hunter","imageID":"docker.io/ollivier/clearwater-live-test@sha256:178bbf9b98c044c0d84af42f9adcd167d6b5e6658219ebfbd352215dbc27dc97","containerID":"containerd://87b09b470ea3672dc62f2f505179fea689cfdecf0ed478154c0105ebd1a445a9"}],"qosClass":"BestEffort"}} 2020-11-27 11:14:05,695 - functest_kubernetes.ims.ims - DEBUG - delete_namespaced_pod: {'api_version': 'v1', 'code': None, 'details': None, 'kind': 'Pod', 'message': None, 'metadata': {'_continue': None, 'resource_version': '11213588', 'self_link': '/api/v1/namespaces/ims-w9fzp/pods/live-test'}, 'reason': None, 'status': "{u'qosClass': u'BestEffort', u'containerStatuses': [{u'restartCount': 0, u'name': u'live-test', u'image': u'docker.io/ollivier/clearwater-live-test:hunter', u'imageID': u'docker.io/ollivier/clearwater-live-test@sha256:178bbf9b98c044c0d84af42f9adcd167d6b5e6658219ebfbd352215dbc27dc97', u'state': {u'terminated': {u'startedAt': u'2020-11-27T11:10:51Z', u'reason': u'Completed', u'finishedAt': u'2020-11-27T11:13:28Z', u'containerID': u'containerd://87b09b470ea3672dc62f2f505179fea689cfdecf0ed478154c0105ebd1a445a9', u'exitCode': 0}}, u'ready': False, u'lastState': {}, u'containerID': u'containerd://87b09b470ea3672dc62f2f505179fea689cfdecf0ed478154c0105ebd1a445a9'}], u'podIP': u'10.244.2.62', u'startTime': u'2020-11-27T11:10:48Z', u'hostIP': u'172.18.0.3', u'phase': u'Succeeded', u'conditions': [{u'status': u'True', u'lastProbeTime': None, u'reason': u'PodCompleted', u'type': u'Initialized', u'lastTransitionTime': u'2020-11-27T11:10:48Z'}, {u'status': u'False', u'lastProbeTime': None, u'reason': u'PodCompleted', u'type': u'Ready', u'lastTransitionTime': u'2020-11-27T11:13:30Z'}, {u'status': u'False', u'lastProbeTime': None, u'reason': u'PodCompleted', u'type': u'ContainersReady', u'lastTransitionTime': u'2020-11-27T11:13:30Z'}, {u'status': u'True', u'lastProbeTime': None, u'type': u'PodScheduled', u'lastTransitionTime': u'2020-11-27T11:10:48Z'}]}"} 2020-11-27 11:14:05,750 - kubernetes.client.rest - DEBUG - response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success","details":{"name":"env-vars","kind":"configmaps","uid":"298cef9c-30a0-11eb-8302-0242ac120002"}} 2020-11-27 11:14:05,752 - functest_kubernetes.ims.ims - DEBUG - delete_namespaced_config_map: {'api_version': 'v1', 'code': None, 'details': {'causes': None, 'group': None, 'kind': 'configmaps', 'name': 'env-vars', 'retry_after_seconds': None, 'uid': '298cef9c-30a0-11eb-8302-0242ac120002'}, 'kind': 'Status', 'message': None, 'metadata': {'_continue': None, 'resource_version': None, 'self_link': None}, 'reason': None, 'status': 'Success'} 2020-11-27 11:14:05,772 - kubernetes.client.rest - DEBUG - response body: {"kind":"Namespace","apiVersion":"v1","metadata":{"name":"ims-w9fzp","generateName":"ims-","selfLink":"/api/v1/namespaces/ims-w9fzp","uid":"2985dc91-30a0-11eb-8302-0242ac120002","resourceVersion":"11213591","creationTimestamp":"2020-11-27T11:03:21Z","deletionTimestamp":"2020-11-27T11:14:05Z"},"spec":{"finalizers":["kubernetes"]},"status":{"phase":"Terminating"}} 2020-11-27 11:14:05,774 - functest_kubernetes.ims.ims - DEBUG - delete_namespace: ims-w9fzp 2020-11-27 11:14:07,287 - xtesting.core.testcase - DEBUG - Publishing /var/lib/xtesting/results/functest-kubernetes.log ('text/plain', None) 2020-11-27 11:14:07,724 - xtesting.core.testcase - DEBUG - Publishing /var/lib/xtesting/results/functest-kubernetes.debug.log ('text/plain', None)