2020-08-28 15:56:45,809 - xtesting.ci.run_tests - INFO - Deployment description: +-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+ | ENV VAR | VALUE | +-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+ | CI_LOOP | daily | | DEBUG | true | | DEPLOY_SCENARIO | k8-nosdn-nofeature-noha | | INSTALLER_TYPE | unknown | | BUILD_TAG | 25KAPYS59QPL | | NODE_NAME | lf-virtual1-4 | | TEST_DB_URL | http://testresults.opnfv.org/test/api/v1/results | | TEST_DB_EXT_URL | http://testresults.opnfv.org/test/api/v1/results | | S3_ENDPOINT_URL | https://storage.googleapis.com | | S3_DST_URL | s3://artifacts.opnfv.org/functest-kubernetes/25KAPYS59QPL/functest-kubernetes-opnfv-functest-kubernetes-cnf-arm64-kali-k8s_vims-run-16 | | HTTP_DST_URL | http://artifacts.opnfv.org/functest-kubernetes/25KAPYS59QPL/functest-kubernetes-opnfv-functest-kubernetes-cnf-arm64-kali-k8s_vims-run-16 | +-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+ 2020-08-28 15:56:45,861 - xtesting.ci.run_tests - DEBUG - No env file /var/lib/xtesting/conf/env_file found 2020-08-28 15:56:45,863 - xtesting.ci.run_tests - DEBUG - Test args: k8s_vims 2020-08-28 15:56:45,912 - xtesting.ci.run_tests - INFO - Loading test case 'k8s_vims'... 2020-08-28 15:56:48,139 - xtesting.ci.run_tests - INFO - Running test case 'k8s_vims'... 2020-08-28 15:56:48,315 - kubernetes.client.rest - DEBUG - response body: {"kind":"Namespace","apiVersion":"v1","metadata":{"name":"ims-gc7hd","generateName":"ims-","selfLink":"/api/v1/namespaces/ims-gc7hd","uid":"b9ee84d8-4810-418e-9a54-b43d163a1a17","resourceVersion":"1801073","creationTimestamp":"2020-08-28T15:56:48Z","managedFields":[{"manager":"OpenAPI-Generator","operation":"Update","apiVersion":"v1","time":"2020-08-28T15:56:48Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:generateName":{}},"f:status":{"f:phase":{}}}}]},"spec":{"finalizers":["kubernetes"]},"status":{"phase":"Active"}} 2020-08-28 15:56:48,428 - functest_kubernetes.ims.ims - DEBUG - create_namespace: {'api_version': 'v1', 'kind': 'Namespace', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'ims-', 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields': None, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal())}], 'name': 'ims-gc7hd', 'namespace': None, 'owner_references': None, 'resource_version': '1801073', 'self_link': '/api/v1/namespaces/ims-gc7hd', 'uid': 'b9ee84d8-4810-418e-9a54-b43d163a1a17'}, 'spec': {'finalizers': ['kubernetes']}, 'status': {'phase': 'Active'}} 2020-08-28 15:56:48,581 - kubernetes.client.rest - DEBUG - response body: {"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"env-vars","namespace":"ims-gc7hd","selfLink":"/api/v1/namespaces/ims-gc7hd/configmaps/env-vars","uid":"b2decaa3-a673-4992-9eb4-2cb8cacbe86a","resourceVersion":"1801075","creationTimestamp":"2020-08-28T15:56:48Z","managedFields":[{"manager":"OpenAPI-Generator","operation":"Update","apiVersion":"v1","time":"2020-08-28T15:56:48Z","fieldsType":"FieldsV1","fieldsV1":{"f:data":{".":{},"f:ADDITIONAL_SHARED_CONFIG":{},"f:ZONE":{}}}}]},"data":{"ADDITIONAL_SHARED_CONFIG":"","ZONE":"ims-gc7hd.svc.cluster.local"}} 2020-08-28 15:56:48,589 - functest_kubernetes.ims.ims - DEBUG - create_namespaced_config_map: {'api_version': 'v1', 'binary_data': None, 'data': {'ADDITIONAL_SHARED_CONFIG': '', 'ZONE': 'ims-gc7hd.svc.cluster.local'}, 'kind': 'ConfigMap', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields': None, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal())}], 'name': 'env-vars', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801075', 'self_link': '/api/v1/namespaces/ims-gc7hd/configmaps/env-vars', 'uid': 'b2decaa3-a673-4992-9eb4-2cb8cacbe86a'}} 2020-08-28 15:56:48,969 - kubernetes.client.rest - DEBUG - response body: {"kind":"Deployment","apiVersion":"apps/v1","metadata":{"name":"astaire","namespace":"ims-gc7hd","selfLink":"/apis/apps/v1/namespaces/ims-gc7hd/deployments/astaire","uid":"a586a36b-c53d-4586-be2a-1620364a644b","resourceVersion":"1801080","generation":1,"creationTimestamp":"2020-08-28T15:56:48Z"},"spec":{"replicas":1,"selector":{"matchLabels":{"service":"astaire"}},"template":{"metadata":{"creationTimestamp":null,"labels":{"service":"astaire"}},"spec":{"volumes":[{"name":"astairelogs","emptyDir":{}}],"initContainers":[{"name":"wait-on-cassandra","image":"busybox:1.28","command":["sh","-c","until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done"],"resources":{},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"containers":[{"name":"astaire","image":"ollivier/clearwater-astaire:latest","ports":[{"containerPort":22,"protocol":"TCP"},{"containerPort":11211,"protocol":"TCP"},{"containerPort":11311,"protocol":"TCP"}],"envFrom":[{"configMapRef":{"name":"env-vars"}}],"env":[{"name":"MY_POD_IP","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"status.podIP"}}}],"resources":{},"volumeMounts":[{"name":"astairelogs","mountPath":"/var/log/astaire"}],"livenessProbe":{"tcpSocket":{"port":11311},"timeoutSeconds":1,"periodSeconds":10,"successThreshold":1,"failureThreshold":9},"readinessProbe":{"tcpSocket":{"port":11311},"timeoutSeconds":1,"periodSeconds":10,"successThreshold":1,"failureThreshold":3},"lifecycle":{"preStop":{"exec":{"command":["/bin/bash","-c","/usr/bin/pre-stop"]}}},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always"},{"name":"tailer","image":"busybox","command":["tail","-F","/var/log/astaire/astaire_current.txt"],"resources":{},"volumeMounts":[{"name":"astairelogs","mountPath":"/var/log/astaire"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always"}],"restartPolicy":"Always","terminationGracePeriodSeconds":120,"dnsPolicy":"ClusterFirst","securityContext":{},"imagePullSecrets":[{}],"schedulerName":"default-scheduler"}},"strategy":{"type":"RollingUpdate","rollingUpdate":{"maxUnavailable":"25%","maxSurge":"25%"}},"revisionHistoryLimit":10,"progressDeadlineSeconds":600},"status":{}} 2020-08-28 15:56:48,983 - functest_kubernetes.ims.ims - INFO - Deployment astaire created 2020-08-28 15:56:48,984 - functest_kubernetes.ims.ims - DEBUG - create_namespaced_deployment: {'api_version': 'v1', 'binary_data': None, 'data': {'ADDITIONAL_SHARED_CONFIG': '', 'ZONE': 'ims-gc7hd.svc.cluster.local'}, 'kind': 'ConfigMap', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields': None, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal())}], 'name': 'env-vars', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801075', 'self_link': '/api/v1/namespaces/ims-gc7hd/configmaps/env-vars', 'uid': 'b2decaa3-a673-4992-9eb4-2cb8cacbe86a'}} 2020-08-28 15:56:49,700 - kubernetes.client.rest - DEBUG - response body: {"kind":"Deployment","apiVersion":"apps/v1","metadata":{"name":"bono","namespace":"ims-gc7hd","selfLink":"/apis/apps/v1/namespaces/ims-gc7hd/deployments/bono","uid":"d6bc69e0-8471-4b52-ae66-fd1f8e8bf787","resourceVersion":"1801088","generation":1,"creationTimestamp":"2020-08-28T15:56:49Z"},"spec":{"replicas":1,"selector":{"matchLabels":{"service":"bono"}},"template":{"metadata":{"creationTimestamp":null,"labels":{"service":"bono","snmp":"enabled"}},"spec":{"volumes":[{"name":"bonologs","emptyDir":{}}],"initContainers":[{"name":"wait-on-sprout","image":"busybox:1.28","command":["sh","-c","until nslookup sprout; do echo waiting for sprout to start ...; sleep 2; done"],"resources":{},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"containers":[{"name":"bono","image":"ollivier/clearwater-bono:latest","ports":[{"containerPort":22,"protocol":"TCP"},{"containerPort":3478,"protocol":"TCP"},{"containerPort":5060,"protocol":"TCP"},{"containerPort":5062,"protocol":"TCP"},{"containerPort":5060,"protocol":"UDP"},{"containerPort":5062,"protocol":"UDP"}],"envFrom":[{"configMapRef":{"name":"env-vars"}}],"env":[{"name":"MY_POD_IP","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"status.podIP"}}}],"resources":{},"volumeMounts":[{"name":"bonologs","mountPath":"/var/log/bono"}],"livenessProbe":{"exec":{"command":["/bin/bash","/usr/share/kubernetes/liveness.sh","3478 5060 5062"]},"initialDelaySeconds":30,"timeoutSeconds":1,"periodSeconds":10,"successThreshold":1,"failureThreshold":3},"readinessProbe":{"exec":{"command":["/bin/bash","/usr/share/kubernetes/liveness.sh","3478 5060 5062"]},"timeoutSeconds":1,"periodSeconds":10,"successThreshold":1,"failureThreshold":3},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always"},{"name":"tailer","image":"busybox","command":["tail","-F","/var/log/bono/bono_current.txt"],"resources":{},"volumeMounts":[{"name":"bonologs","mountPath":"/var/log/bono"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","securityContext":{},"imagePullSecrets":[{}],"schedulerName":"default-scheduler"}},"strategy":{"type":"RollingUpdate","rollingUpdate":{"maxUnavailable":"25%","maxSurge":"25%"}},"revisionHistoryLimit":10,"progressDeadlineSeconds":600},"status":{}} 2020-08-28 15:56:49,714 - functest_kubernetes.ims.ims - INFO - Deployment bono created 2020-08-28 15:56:49,715 - functest_kubernetes.ims.ims - DEBUG - create_namespaced_deployment: {'api_version': 'v1', 'binary_data': None, 'data': {'ADDITIONAL_SHARED_CONFIG': '', 'ZONE': 'ims-gc7hd.svc.cluster.local'}, 'kind': 'ConfigMap', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields': None, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal())}], 'name': 'env-vars', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801075', 'self_link': '/api/v1/namespaces/ims-gc7hd/configmaps/env-vars', 'uid': 'b2decaa3-a673-4992-9eb4-2cb8cacbe86a'}} 2020-08-28 15:56:50,073 - kubernetes.client.rest - DEBUG - response body: {"kind":"Deployment","apiVersion":"apps/v1","metadata":{"name":"cassandra","namespace":"ims-gc7hd","selfLink":"/apis/apps/v1/namespaces/ims-gc7hd/deployments/cassandra","uid":"dc3fbaac-7655-45e2-b0da-c94598a92d8b","resourceVersion":"1801097","generation":1,"creationTimestamp":"2020-08-28T15:56:49Z"},"spec":{"replicas":1,"selector":{"matchLabels":{"service":"cassandra"}},"template":{"metadata":{"creationTimestamp":null,"labels":{"service":"cassandra"}},"spec":{"initContainers":[{"name":"wait-on-etcd","image":"busybox:1.28","command":["sh","-c","until nslookup etcd; do echo waiting for etcd to start ...; sleep 2; done"],"resources":{},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"containers":[{"name":"cassandra","image":"ollivier/clearwater-cassandra:latest","ports":[{"containerPort":22,"protocol":"TCP"},{"containerPort":7001,"protocol":"TCP"},{"containerPort":9042,"protocol":"TCP"},{"containerPort":9160,"protocol":"TCP"}],"envFrom":[{"configMapRef":{"name":"env-vars"}}],"env":[{"name":"MY_POD_IP","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"status.podIP"}}}],"resources":{},"livenessProbe":{"exec":{"command":["/bin/bash","/usr/share/kubernetes/liveness.sh","7000 9042 9160"]},"initialDelaySeconds":600,"timeoutSeconds":1,"periodSeconds":10,"successThreshold":1,"failureThreshold":3},"readinessProbe":{"exec":{"command":["/bin/bash","/usr/share/kubernetes/liveness.sh","7000 9042 9160"]},"timeoutSeconds":1,"periodSeconds":10,"successThreshold":1,"failureThreshold":3},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","securityContext":{},"imagePullSecrets":[{}],"schedulerName":"default-scheduler"}},"strategy":{"type":"RollingUpdate","rollingUpdate":{"maxUnavailable":"25%","maxSurge":"25%"}},"revisionHistoryLimit":10,"progressDeadlineSeconds":600},"status":{}} 2020-08-28 15:56:50,085 - functest_kubernetes.ims.ims - INFO - Deployment cassandra created 2020-08-28 15:56:50,086 - functest_kubernetes.ims.ims - DEBUG - create_namespaced_deployment: {'api_version': 'v1', 'binary_data': None, 'data': {'ADDITIONAL_SHARED_CONFIG': '', 'ZONE': 'ims-gc7hd.svc.cluster.local'}, 'kind': 'ConfigMap', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields': None, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal())}], 'name': 'env-vars', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801075', 'self_link': '/api/v1/namespaces/ims-gc7hd/configmaps/env-vars', 'uid': 'b2decaa3-a673-4992-9eb4-2cb8cacbe86a'}} 2020-08-28 15:56:50,461 - kubernetes.client.rest - DEBUG - response body: {"kind":"Deployment","apiVersion":"apps/v1","metadata":{"name":"chronos","namespace":"ims-gc7hd","selfLink":"/apis/apps/v1/namespaces/ims-gc7hd/deployments/chronos","uid":"8911bd1e-c40e-4013-af3e-2e070733afb4","resourceVersion":"1801112","generation":1,"creationTimestamp":"2020-08-28T15:56:50Z","labels":{"service":"chronos"}},"spec":{"replicas":1,"selector":{"matchLabels":{"service":"chronos"}},"template":{"metadata":{"creationTimestamp":null,"labels":{"service":"chronos"}},"spec":{"volumes":[{"name":"chronoslogs","emptyDir":{}}],"initContainers":[{"name":"wait-on-cassandra","image":"busybox:1.28","command":["sh","-c","until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done"],"resources":{},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"containers":[{"name":"chronos","image":"ollivier/clearwater-chronos:latest","ports":[{"containerPort":22,"protocol":"TCP"},{"containerPort":7253,"protocol":"TCP"}],"envFrom":[{"configMapRef":{"name":"env-vars"}}],"env":[{"name":"MY_POD_IP","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"status.podIP"}}}],"resources":{},"volumeMounts":[{"name":"chronoslogs","mountPath":"/var/log/chronos"}],"livenessProbe":{"tcpSocket":{"port":7253},"timeoutSeconds":1,"periodSeconds":10,"successThreshold":1,"failureThreshold":9},"readinessProbe":{"tcpSocket":{"port":7253},"timeoutSeconds":1,"periodSeconds":10,"successThreshold":1,"failureThreshold":3},"lifecycle":{"preStop":{"exec":{"command":["/bin/bash","-c","/usr/bin/pre-stop"]}}},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always"},{"name":"tailer","image":"busybox","command":["tail","-F","/var/log/chronos/chronos_current.txt"],"resources":{},"volumeMounts":[{"name":"chronoslogs","mountPath":"/var/log/chronos"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always"}],"restartPolicy":"Always","terminationGracePeriodSeconds":120,"dnsPolicy":"ClusterFirst","securityContext":{},"imagePullSecrets":[{}],"schedulerName":"default-scheduler"}},"strategy":{"type":"RollingUpdate","rollingUpdate":{"maxUnavailable":"25%","maxSurge":"25%"}},"revisionHistoryLimit":10,"progressDeadlineSeconds":600},"status":{}} 2020-08-28 15:56:50,474 - functest_kubernetes.ims.ims - INFO - Deployment chronos created 2020-08-28 15:56:50,475 - functest_kubernetes.ims.ims - DEBUG - create_namespaced_deployment: {'api_version': 'v1', 'binary_data': None, 'data': {'ADDITIONAL_SHARED_CONFIG': '', 'ZONE': 'ims-gc7hd.svc.cluster.local'}, 'kind': 'ConfigMap', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields': None, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal())}], 'name': 'env-vars', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801075', 'self_link': '/api/v1/namespaces/ims-gc7hd/configmaps/env-vars', 'uid': 'b2decaa3-a673-4992-9eb4-2cb8cacbe86a'}} 2020-08-28 15:56:50,683 - kubernetes.client.rest - DEBUG - response body: {"kind":"Deployment","apiVersion":"apps/v1","metadata":{"name":"ellis","namespace":"ims-gc7hd","selfLink":"/apis/apps/v1/namespaces/ims-gc7hd/deployments/ellis","uid":"9d4959be-6eec-4d20-a139-c33010481c17","resourceVersion":"1801131","generation":1,"creationTimestamp":"2020-08-28T15:56:50Z"},"spec":{"replicas":1,"selector":{"matchLabels":{"service":"ellis"}},"template":{"metadata":{"creationTimestamp":null,"labels":{"service":"ellis"}},"spec":{"initContainers":[{"name":"wait-on-homestead-prov","image":"busybox:1.28","command":["sh","-c","until nslookup homestead-prov; do echo waiting for homestead-prov to start ...; sleep 2; done"],"resources":{},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"},{"name":"wait-on-sprout","image":"busybox:1.28","command":["sh","-c","until nslookup sprout; do echo waiting for sprout to start ...; sleep 2; done"],"resources":{},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"containers":[{"name":"ellis","image":"ollivier/clearwater-ellis:latest","ports":[{"containerPort":22,"protocol":"TCP"},{"containerPort":80,"protocol":"TCP"}],"envFrom":[{"configMapRef":{"name":"env-vars"}}],"env":[{"name":"MY_POD_IP","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"status.podIP"}}}],"resources":{},"livenessProbe":{"tcpSocket":{"port":80},"initialDelaySeconds":30,"timeoutSeconds":1,"periodSeconds":10,"successThreshold":1,"failureThreshold":3},"readinessProbe":{"tcpSocket":{"port":80},"timeoutSeconds":1,"periodSeconds":10,"successThreshold":1,"failureThreshold":3},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","securityContext":{},"imagePullSecrets":[{}],"schedulerName":"default-scheduler"}},"strategy":{"type":"RollingUpdate","rollingUpdate":{"maxUnavailable":"25%","maxSurge":"25%"}},"revisionHistoryLimit":10,"progressDeadlineSeconds":600},"status":{}} 2020-08-28 15:56:50,697 - functest_kubernetes.ims.ims - INFO - Deployment ellis created 2020-08-28 15:56:50,699 - functest_kubernetes.ims.ims - DEBUG - create_namespaced_deployment: {'api_version': 'v1', 'binary_data': None, 'data': {'ADDITIONAL_SHARED_CONFIG': '', 'ZONE': 'ims-gc7hd.svc.cluster.local'}, 'kind': 'ConfigMap', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields': None, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal())}], 'name': 'env-vars', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801075', 'self_link': '/api/v1/namespaces/ims-gc7hd/configmaps/env-vars', 'uid': 'b2decaa3-a673-4992-9eb4-2cb8cacbe86a'}} 2020-08-28 15:56:50,997 - kubernetes.client.rest - DEBUG - response body: {"kind":"Deployment","apiVersion":"apps/v1","metadata":{"name":"etcd","namespace":"ims-gc7hd","selfLink":"/apis/apps/v1/namespaces/ims-gc7hd/deployments/etcd","uid":"38f69988-86b5-423a-8e0b-b9f3382efc35","resourceVersion":"1801145","generation":1,"creationTimestamp":"2020-08-28T15:56:50Z"},"spec":{"replicas":1,"selector":{"matchLabels":{"instance-type":"etcd-pod"}},"template":{"metadata":{"creationTimestamp":null,"labels":{"instance-type":"etcd-pod"}},"spec":{"containers":[{"name":"etcd","image":"quay.io/coreos/etcd:v2.2.5","args":["--name","$(MY_POD_NAME)","--advertise-client-urls","http://$(MY_POD_IP):2379,http://$(MY_POD_IP):4001","--listen-client-urls","http://0.0.0.0:2379,http://0.0.0.0:4001","--initial-advertise-peer-urls","http://$(MY_POD_IP):2380","--listen-peer-urls","http://0.0.0.0:2380","--initial-cluster","$(MY_POD_NAME)=http://$(MY_POD_IP):2380","--initial-cluster-state","new"],"ports":[{"containerPort":2379,"protocol":"TCP"},{"containerPort":2380,"protocol":"TCP"},{"containerPort":4001,"protocol":"TCP"}],"env":[{"name":"MY_POD_IP","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"status.podIP"}}},{"name":"MY_POD_NAME","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"metadata.name"}}}],"resources":{},"livenessProbe":{"tcpSocket":{"port":4001},"initialDelaySeconds":300,"timeoutSeconds":1,"periodSeconds":10,"successThreshold":1,"failureThreshold":3},"readinessProbe":{"tcpSocket":{"port":4001},"timeoutSeconds":1,"periodSeconds":10,"successThreshold":1,"failureThreshold":3},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","securityContext":{},"imagePullSecrets":[{}],"schedulerName":"default-scheduler"}},"strategy":{"type":"RollingUpdate","rollingUpdate":{"maxUnavailable":"25%","maxSurge":"25%"}},"revisionHistoryLimit":10,"progressDeadlineSeconds":600},"status":{}} 2020-08-28 15:56:51,107 - functest_kubernetes.ims.ims - INFO - Deployment etcd created 2020-08-28 15:56:51,108 - functest_kubernetes.ims.ims - DEBUG - create_namespaced_deployment: {'api_version': 'v1', 'binary_data': None, 'data': {'ADDITIONAL_SHARED_CONFIG': '', 'ZONE': 'ims-gc7hd.svc.cluster.local'}, 'kind': 'ConfigMap', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields': None, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal())}], 'name': 'env-vars', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801075', 'self_link': '/api/v1/namespaces/ims-gc7hd/configmaps/env-vars', 'uid': 'b2decaa3-a673-4992-9eb4-2cb8cacbe86a'}} 2020-08-28 15:56:51,557 - kubernetes.client.rest - DEBUG - response body: {"kind":"Deployment","apiVersion":"apps/v1","metadata":{"name":"homer","namespace":"ims-gc7hd","selfLink":"/apis/apps/v1/namespaces/ims-gc7hd/deployments/homer","uid":"fef4c095-1313-4b44-b9d6-df772baa5d24","resourceVersion":"1801149","generation":1,"creationTimestamp":"2020-08-28T15:56:51Z"},"spec":{"replicas":1,"selector":{"matchLabels":{"service":"homer"}},"template":{"metadata":{"creationTimestamp":null,"labels":{"service":"homer"}},"spec":{"initContainers":[{"name":"wait-on-cassandra","image":"busybox:1.28","command":["sh","-c","until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done"],"resources":{},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"containers":[{"name":"homer","image":"ollivier/clearwater-homer:latest","ports":[{"containerPort":22,"protocol":"TCP"},{"containerPort":7888,"protocol":"TCP"}],"envFrom":[{"configMapRef":{"name":"env-vars"}}],"env":[{"name":"MY_POD_IP","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"status.podIP"}}}],"resources":{},"livenessProbe":{"tcpSocket":{"port":7888},"initialDelaySeconds":30,"timeoutSeconds":1,"periodSeconds":10,"successThreshold":1,"failureThreshold":3},"readinessProbe":{"tcpSocket":{"port":7888},"timeoutSeconds":1,"periodSeconds":10,"successThreshold":1,"failureThreshold":3},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","securityContext":{},"imagePullSecrets":[{}],"schedulerName":"default-scheduler"}},"strategy":{"type":"RollingUpdate","rollingUpdate":{"maxUnavailable":"25%","maxSurge":"25%"}},"revisionHistoryLimit":10,"progressDeadlineSeconds":600},"status":{}} 2020-08-28 15:56:51,570 - functest_kubernetes.ims.ims - INFO - Deployment homer created 2020-08-28 15:56:51,571 - functest_kubernetes.ims.ims - DEBUG - create_namespaced_deployment: {'api_version': 'v1', 'binary_data': None, 'data': {'ADDITIONAL_SHARED_CONFIG': '', 'ZONE': 'ims-gc7hd.svc.cluster.local'}, 'kind': 'ConfigMap', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields': None, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal())}], 'name': 'env-vars', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801075', 'self_link': '/api/v1/namespaces/ims-gc7hd/configmaps/env-vars', 'uid': 'b2decaa3-a673-4992-9eb4-2cb8cacbe86a'}} 2020-08-28 15:56:52,327 - kubernetes.client.rest - DEBUG - response body: {"kind":"Deployment","apiVersion":"apps/v1","metadata":{"name":"homestead","namespace":"ims-gc7hd","selfLink":"/apis/apps/v1/namespaces/ims-gc7hd/deployments/homestead","uid":"84ae3654-a60b-43f4-9ba3-8550a7a2c87f","resourceVersion":"1801166","generation":1,"creationTimestamp":"2020-08-28T15:56:51Z"},"spec":{"replicas":1,"selector":{"matchLabels":{"service":"homestead"}},"template":{"metadata":{"creationTimestamp":null,"labels":{"service":"homestead","snmp":"enabled"}},"spec":{"volumes":[{"name":"homesteadlogs","emptyDir":{}}],"initContainers":[{"name":"wait-on-cassandra","image":"busybox:1.28","command":["sh","-c","until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done"],"resources":{},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"},{"name":"wait-on-astaire","image":"busybox:1.28","command":["sh","-c","until nslookup astaire; do echo waiting for astaire to start ...; sleep 2; done"],"resources":{},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"containers":[{"name":"homestead","image":"ollivier/clearwater-homestead:latest","ports":[{"containerPort":22,"protocol":"TCP"},{"containerPort":8888,"protocol":"TCP"}],"envFrom":[{"configMapRef":{"name":"env-vars"}}],"env":[{"name":"MY_POD_IP","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"status.podIP"}}}],"resources":{},"volumeMounts":[{"name":"homesteadlogs","mountPath":"/var/log/homestead"}],"livenessProbe":{"exec":{"command":["/bin/bash","/usr/share/kubernetes/liveness.sh","8888"]},"initialDelaySeconds":60,"timeoutSeconds":1,"periodSeconds":10,"successThreshold":1,"failureThreshold":3},"readinessProbe":{"exec":{"command":["/bin/bash","/usr/share/kubernetes/liveness.sh","8888"]},"timeoutSeconds":1,"periodSeconds":10,"successThreshold":1,"failureThreshold":3},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always"},{"name":"tailer","image":"busybox","command":["tail","-F","/var/log/homestead/homestead_current.txt"],"resources":{},"volumeMounts":[{"name":"homesteadlogs","mountPath":"/var/log/homestead"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","securityContext":{},"imagePullSecrets":[{}],"schedulerName":"default-scheduler"}},"strategy":{"type":"RollingUpdate","rollingUpdate":{"maxUnavailable":"25%","maxSurge":"25%"}},"revisionHistoryLimit":10,"progressDeadlineSeconds":600},"status":{}} 2020-08-28 15:56:52,342 - functest_kubernetes.ims.ims - INFO - Deployment homestead created 2020-08-28 15:56:52,343 - functest_kubernetes.ims.ims - DEBUG - create_namespaced_deployment: {'api_version': 'v1', 'binary_data': None, 'data': {'ADDITIONAL_SHARED_CONFIG': '', 'ZONE': 'ims-gc7hd.svc.cluster.local'}, 'kind': 'ConfigMap', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields': None, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal())}], 'name': 'env-vars', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801075', 'self_link': '/api/v1/namespaces/ims-gc7hd/configmaps/env-vars', 'uid': 'b2decaa3-a673-4992-9eb4-2cb8cacbe86a'}} 2020-08-28 15:56:52,772 - kubernetes.client.rest - DEBUG - response body: {"kind":"Deployment","apiVersion":"apps/v1","metadata":{"name":"homestead-prov","namespace":"ims-gc7hd","selfLink":"/apis/apps/v1/namespaces/ims-gc7hd/deployments/homestead-prov","uid":"e8b84542-2695-4e30-a7a1-42b0e16de3fe","resourceVersion":"1801178","generation":1,"creationTimestamp":"2020-08-28T15:56:52Z"},"spec":{"replicas":1,"selector":{"matchLabels":{"service":"homestead-prov"}},"template":{"metadata":{"creationTimestamp":null,"labels":{"service":"homestead-prov","snmp":"enabled"}},"spec":{"initContainers":[{"name":"wait-on-homestead","image":"busybox:1.28","command":["sh","-c","until nslookup homestead; do echo waiting for homestead to start ...; sleep 2; done"],"resources":{},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"containers":[{"name":"homestead-prov","image":"ollivier/clearwater-homestead-prov:latest","ports":[{"containerPort":22,"protocol":"TCP"},{"containerPort":8889,"protocol":"TCP"}],"envFrom":[{"configMapRef":{"name":"env-vars"}}],"env":[{"name":"MY_POD_IP","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"status.podIP"}}}],"resources":{},"livenessProbe":{"exec":{"command":["/bin/bash","/usr/share/clearwater/bin/poll_homestead-prov.sh"]},"initialDelaySeconds":60,"timeoutSeconds":1,"periodSeconds":10,"successThreshold":1,"failureThreshold":3},"readinessProbe":{"exec":{"command":["/bin/bash","/usr/share/clearwater/bin/poll_homestead-prov.sh"]},"timeoutSeconds":1,"periodSeconds":10,"successThreshold":1,"failureThreshold":3},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","securityContext":{},"imagePullSecrets":[{}],"schedulerName":"default-scheduler"}},"strategy":{"type":"RollingUpdate","rollingUpdate":{"maxUnavailable":"25%","maxSurge":"25%"}},"revisionHistoryLimit":10,"progressDeadlineSeconds":600},"status":{}} 2020-08-28 15:56:52,783 - functest_kubernetes.ims.ims - INFO - Deployment homestead-prov created 2020-08-28 15:56:52,784 - functest_kubernetes.ims.ims - DEBUG - create_namespaced_deployment: {'api_version': 'v1', 'binary_data': None, 'data': {'ADDITIONAL_SHARED_CONFIG': '', 'ZONE': 'ims-gc7hd.svc.cluster.local'}, 'kind': 'ConfigMap', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields': None, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal())}], 'name': 'env-vars', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801075', 'self_link': '/api/v1/namespaces/ims-gc7hd/configmaps/env-vars', 'uid': 'b2decaa3-a673-4992-9eb4-2cb8cacbe86a'}} 2020-08-28 15:56:53,020 - kubernetes.client.rest - DEBUG - response body: {"kind":"Deployment","apiVersion":"apps/v1","metadata":{"name":"ralf","namespace":"ims-gc7hd","selfLink":"/apis/apps/v1/namespaces/ims-gc7hd/deployments/ralf","uid":"c0685c0a-f472-46a2-9b78-32d4cbb5720b","resourceVersion":"1801193","generation":1,"creationTimestamp":"2020-08-28T15:56:52Z"},"spec":{"replicas":1,"selector":{"matchLabels":{"service":"ralf"}},"template":{"metadata":{"creationTimestamp":null,"labels":{"service":"ralf","snmp":"enabled"}},"spec":{"volumes":[{"name":"ralflogs","emptyDir":{}}],"initContainers":[{"name":"wait-on-cassandra","image":"busybox:1.28","command":["sh","-c","until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done"],"resources":{},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"containers":[{"name":"ralf","image":"ollivier/clearwater-ralf:latest","ports":[{"containerPort":22,"protocol":"TCP"},{"containerPort":10888,"protocol":"TCP"}],"envFrom":[{"configMapRef":{"name":"env-vars"}}],"env":[{"name":"MY_POD_IP","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"status.podIP"}}}],"resources":{},"volumeMounts":[{"name":"ralflogs","mountPath":"/var/log/ralf"}],"livenessProbe":{"tcpSocket":{"port":10888},"initialDelaySeconds":30,"timeoutSeconds":1,"periodSeconds":10,"successThreshold":1,"failureThreshold":3},"readinessProbe":{"tcpSocket":{"port":10888},"timeoutSeconds":1,"periodSeconds":10,"successThreshold":1,"failureThreshold":3},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always"},{"name":"tailer","image":"busybox","command":["tail","-F","/var/log/ralf/ralf_current.txt"],"resources":{},"volumeMounts":[{"name":"ralflogs","mountPath":"/var/log/ralf"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","securityContext":{},"imagePullSecrets":[{}],"schedulerName":"default-scheduler"}},"strategy":{"type":"RollingUpdate","rollingUpdate":{"maxUnavailable":"25%","maxSurge":"25%"}},"revisionHistoryLimit":10,"progressDeadlineSeconds":600},"status":{}} 2020-08-28 15:56:53,034 - functest_kubernetes.ims.ims - INFO - Deployment ralf created 2020-08-28 15:56:53,035 - functest_kubernetes.ims.ims - DEBUG - create_namespaced_deployment: {'api_version': 'v1', 'binary_data': None, 'data': {'ADDITIONAL_SHARED_CONFIG': '', 'ZONE': 'ims-gc7hd.svc.cluster.local'}, 'kind': 'ConfigMap', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields': None, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal())}], 'name': 'env-vars', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801075', 'self_link': '/api/v1/namespaces/ims-gc7hd/configmaps/env-vars', 'uid': 'b2decaa3-a673-4992-9eb4-2cb8cacbe86a'}} 2020-08-28 15:56:53,494 - kubernetes.client.rest - DEBUG - response body: {"kind":"Deployment","apiVersion":"apps/v1","metadata":{"name":"sprout","namespace":"ims-gc7hd","selfLink":"/apis/apps/v1/namespaces/ims-gc7hd/deployments/sprout","uid":"b3390d47-04d1-4afc-9a3e-fc9de582b4fc","resourceVersion":"1801207","generation":1,"creationTimestamp":"2020-08-28T15:56:53Z"},"spec":{"replicas":1,"selector":{"matchLabels":{"service":"sprout"}},"template":{"metadata":{"creationTimestamp":null,"labels":{"service":"sprout","snmp":"enabled"}},"spec":{"volumes":[{"name":"sproutlogs","emptyDir":{}}],"initContainers":[{"name":"wait-on-ralf","image":"busybox:1.28","command":["sh","-c","until nslookup ralf; do echo waiting for ralf to start ...; sleep 2; done"],"resources":{},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"},{"name":"wait-on-chronos","image":"busybox:1.28","command":["sh","-c","until nslookup chronos; do echo waiting for chronos to start ...; sleep 2; done"],"resources":{},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"},{"name":"wait-on-homestead","image":"busybox:1.28","command":["sh","-c","until nslookup homestead; do echo waiting for homestead to start ...; sleep 2; done"],"resources":{},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"containers":[{"name":"sprout","image":"ollivier/clearwater-sprout:latest","ports":[{"containerPort":22,"protocol":"TCP"}],"envFrom":[{"configMapRef":{"name":"env-vars"}}],"env":[{"name":"MY_POD_IP","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"status.podIP"}}}],"resources":{},"volumeMounts":[{"name":"sproutlogs","mountPath":"/var/log/sprout"}],"livenessProbe":{"exec":{"command":["/bin/bash","/usr/share/kubernetes/liveness.sh","5052 5054"]},"initialDelaySeconds":30,"timeoutSeconds":1,"periodSeconds":3,"successThreshold":1,"failureThreshold":3},"readinessProbe":{"exec":{"command":["/bin/bash","/usr/share/kubernetes/liveness.sh","5052 5054"]},"timeoutSeconds":1,"periodSeconds":10,"successThreshold":1,"failureThreshold":3},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always"},{"name":"tailer","image":"busybox","command":["tail","-F","/var/log/sprout/sprout_current.txt"],"resources":{},"volumeMounts":[{"name":"sproutlogs","mountPath":"/var/log/sprout"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","securityContext":{},"imagePullSecrets":[{}],"schedulerName":"default-scheduler"}},"strategy":{"type":"RollingUpdate","rollingUpdate":{"maxUnavailable":"25%","maxSurge":"25%"}},"revisionHistoryLimit":10,"progressDeadlineSeconds":600},"status":{}} 2020-08-28 15:56:53,508 - functest_kubernetes.ims.ims - INFO - Deployment sprout created 2020-08-28 15:56:53,509 - functest_kubernetes.ims.ims - DEBUG - create_namespaced_deployment: {'api_version': 'v1', 'binary_data': None, 'data': {'ADDITIONAL_SHARED_CONFIG': '', 'ZONE': 'ims-gc7hd.svc.cluster.local'}, 'kind': 'ConfigMap', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields': None, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal())}], 'name': 'env-vars', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801075', 'self_link': '/api/v1/namespaces/ims-gc7hd/configmaps/env-vars', 'uid': 'b2decaa3-a673-4992-9eb4-2cb8cacbe86a'}} 2020-08-28 15:56:53,701 - kubernetes.client.rest - DEBUG - response body: {"kind":"Service","apiVersion":"v1","metadata":{"name":"astaire","namespace":"ims-gc7hd","selfLink":"/api/v1/namespaces/ims-gc7hd/services/astaire","uid":"d1b5ab72-d74f-4f5d-b2ad-a76ea9b870cf","resourceVersion":"1801221","creationTimestamp":"2020-08-28T15:56:53Z","managedFields":[{"manager":"OpenAPI-Generator","operation":"Update","apiVersion":"v1","time":"2020-08-28T15:56:53Z","fieldsType":"FieldsV1","fieldsV1":{"f:spec":{"f:clusterIP":{},"f:ports":{".":{},"k:{\"port\":11311,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}}},"f:selector":{".":{},"f:service":{}},"f:sessionAffinity":{},"f:type":{}}}}]},"spec":{"ports":[{"name":"11311","protocol":"TCP","port":11311,"targetPort":11311}],"selector":{"service":"astaire"},"clusterIP":"None","type":"ClusterIP","sessionAffinity":"None"},"status":{"loadBalancer":{}}} 2020-08-28 15:56:53,710 - functest_kubernetes.ims.ims - INFO - Service astaire created 2020-08-28 15:56:53,751 - functest_kubernetes.ims.ims - DEBUG - create_namespaced_service: {'api_version': 'v1', 'binary_data': None, 'data': {'ADDITIONAL_SHARED_CONFIG': '', 'ZONE': 'ims-gc7hd.svc.cluster.local'}, 'kind': 'ConfigMap', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields': None, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal())}], 'name': 'env-vars', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801075', 'self_link': '/api/v1/namespaces/ims-gc7hd/configmaps/env-vars', 'uid': 'b2decaa3-a673-4992-9eb4-2cb8cacbe86a'}} 2020-08-28 15:56:54,821 - kubernetes.client.rest - DEBUG - response body: {"kind":"Service","apiVersion":"v1","metadata":{"name":"bono","namespace":"ims-gc7hd","selfLink":"/api/v1/namespaces/ims-gc7hd/services/bono","uid":"20bb1f4d-2640-4e68-9db9-fbf06db037ec","resourceVersion":"1801228","creationTimestamp":"2020-08-28T15:56:53Z","managedFields":[{"manager":"OpenAPI-Generator","operation":"Update","apiVersion":"v1","time":"2020-08-28T15:56:53Z","fieldsType":"FieldsV1","fieldsV1":{"f:spec":{"f:clusterIP":{},"f:ports":{".":{},"k:{\"port\":3478,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}},"k:{\"port\":5060,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}},"k:{\"port\":5062,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}}},"f:selector":{".":{},"f:service":{}},"f:sessionAffinity":{},"f:type":{}}}}]},"spec":{"ports":[{"name":"3478","protocol":"TCP","port":3478,"targetPort":3478},{"name":"5060","protocol":"TCP","port":5060,"targetPort":5060},{"name":"5062","protocol":"TCP","port":5062,"targetPort":5062}],"selector":{"service":"bono"},"clusterIP":"None","type":"ClusterIP","sessionAffinity":"None"},"status":{"loadBalancer":{}}} 2020-08-28 15:56:54,856 - functest_kubernetes.ims.ims - INFO - Service bono created 2020-08-28 15:56:54,858 - functest_kubernetes.ims.ims - DEBUG - create_namespaced_service: {'api_version': 'v1', 'binary_data': None, 'data': {'ADDITIONAL_SHARED_CONFIG': '', 'ZONE': 'ims-gc7hd.svc.cluster.local'}, 'kind': 'ConfigMap', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields': None, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal())}], 'name': 'env-vars', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801075', 'self_link': '/api/v1/namespaces/ims-gc7hd/configmaps/env-vars', 'uid': 'b2decaa3-a673-4992-9eb4-2cb8cacbe86a'}} 2020-08-28 15:56:55,354 - kubernetes.client.rest - DEBUG - response body: {"kind":"Service","apiVersion":"v1","metadata":{"name":"cassandra","namespace":"ims-gc7hd","selfLink":"/api/v1/namespaces/ims-gc7hd/services/cassandra","uid":"3e0bf5b6-d35e-455c-a8e7-201a08d58707","resourceVersion":"1801239","creationTimestamp":"2020-08-28T15:56:54Z","managedFields":[{"manager":"OpenAPI-Generator","operation":"Update","apiVersion":"v1","time":"2020-08-28T15:56:54Z","fieldsType":"FieldsV1","fieldsV1":{"f:spec":{"f:clusterIP":{},"f:ports":{".":{},"k:{\"port\":7000,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}},"k:{\"port\":7001,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}},"k:{\"port\":9042,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}},"k:{\"port\":9160,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}}},"f:selector":{".":{},"f:service":{}},"f:sessionAffinity":{},"f:type":{}}}}]},"spec":{"ports":[{"name":"7001","protocol":"TCP","port":7001,"targetPort":7001},{"name":"7000","protocol":"TCP","port":7000,"targetPort":7000},{"name":"9042","protocol":"TCP","port":9042,"targetPort":9042},{"name":"9160","protocol":"TCP","port":9160,"targetPort":9160}],"selector":{"service":"cassandra"},"clusterIP":"None","type":"ClusterIP","sessionAffinity":"None"},"status":{"loadBalancer":{}}} 2020-08-28 15:56:55,497 - functest_kubernetes.ims.ims - INFO - Service cassandra created 2020-08-28 15:56:55,498 - functest_kubernetes.ims.ims - DEBUG - create_namespaced_service: {'api_version': 'v1', 'binary_data': None, 'data': {'ADDITIONAL_SHARED_CONFIG': '', 'ZONE': 'ims-gc7hd.svc.cluster.local'}, 'kind': 'ConfigMap', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields': None, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal())}], 'name': 'env-vars', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801075', 'self_link': '/api/v1/namespaces/ims-gc7hd/configmaps/env-vars', 'uid': 'b2decaa3-a673-4992-9eb4-2cb8cacbe86a'}} 2020-08-28 15:56:55,701 - kubernetes.client.rest - DEBUG - response body: {"kind":"Service","apiVersion":"v1","metadata":{"name":"chronos","namespace":"ims-gc7hd","selfLink":"/api/v1/namespaces/ims-gc7hd/services/chronos","uid":"baa01d7d-9efa-4652-b7a6-dc6c2fa21e30","resourceVersion":"1801244","creationTimestamp":"2020-08-28T15:56:55Z","managedFields":[{"manager":"OpenAPI-Generator","operation":"Update","apiVersion":"v1","time":"2020-08-28T15:56:55Z","fieldsType":"FieldsV1","fieldsV1":{"f:spec":{"f:clusterIP":{},"f:ports":{".":{},"k:{\"port\":7253,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}}},"f:selector":{".":{},"f:service":{}},"f:sessionAffinity":{},"f:type":{}}}}]},"spec":{"ports":[{"name":"7253","protocol":"TCP","port":7253,"targetPort":7253}],"selector":{"service":"chronos"},"clusterIP":"None","type":"ClusterIP","sessionAffinity":"None"},"status":{"loadBalancer":{}}} 2020-08-28 15:56:55,711 - functest_kubernetes.ims.ims - INFO - Service chronos created 2020-08-28 15:56:55,712 - functest_kubernetes.ims.ims - DEBUG - create_namespaced_service: {'api_version': 'v1', 'binary_data': None, 'data': {'ADDITIONAL_SHARED_CONFIG': '', 'ZONE': 'ims-gc7hd.svc.cluster.local'}, 'kind': 'ConfigMap', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields': None, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal())}], 'name': 'env-vars', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801075', 'self_link': '/api/v1/namespaces/ims-gc7hd/configmaps/env-vars', 'uid': 'b2decaa3-a673-4992-9eb4-2cb8cacbe86a'}} 2020-08-28 15:56:55,905 - kubernetes.client.rest - DEBUG - response body: {"kind":"Service","apiVersion":"v1","metadata":{"name":"ellis","namespace":"ims-gc7hd","selfLink":"/api/v1/namespaces/ims-gc7hd/services/ellis","uid":"a09acecf-b0a9-4287-9f96-2e82e68f0d5f","resourceVersion":"1801249","creationTimestamp":"2020-08-28T15:56:55Z","managedFields":[{"manager":"OpenAPI-Generator","operation":"Update","apiVersion":"v1","time":"2020-08-28T15:56:55Z","fieldsType":"FieldsV1","fieldsV1":{"f:spec":{"f:clusterIP":{},"f:ports":{".":{},"k:{\"port\":80,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}}},"f:selector":{".":{},"f:service":{}},"f:sessionAffinity":{},"f:type":{}}}}]},"spec":{"ports":[{"name":"http","protocol":"TCP","port":80,"targetPort":80}],"selector":{"service":"ellis"},"clusterIP":"None","type":"ClusterIP","sessionAffinity":"None"},"status":{"loadBalancer":{}}} 2020-08-28 15:56:55,926 - functest_kubernetes.ims.ims - INFO - Service ellis created 2020-08-28 15:56:55,927 - functest_kubernetes.ims.ims - DEBUG - create_namespaced_service: {'api_version': 'v1', 'binary_data': None, 'data': {'ADDITIONAL_SHARED_CONFIG': '', 'ZONE': 'ims-gc7hd.svc.cluster.local'}, 'kind': 'ConfigMap', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields': None, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal())}], 'name': 'env-vars', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801075', 'self_link': '/api/v1/namespaces/ims-gc7hd/configmaps/env-vars', 'uid': 'b2decaa3-a673-4992-9eb4-2cb8cacbe86a'}} 2020-08-28 15:56:56,571 - kubernetes.client.rest - DEBUG - response body: {"kind":"Service","apiVersion":"v1","metadata":{"name":"etcd","namespace":"ims-gc7hd","selfLink":"/api/v1/namespaces/ims-gc7hd/services/etcd","uid":"3d35f292-6126-40fa-a013-acca8671c34b","resourceVersion":"1801253","creationTimestamp":"2020-08-28T15:56:56Z","labels":{"instance-type":"etcd-pod"},"managedFields":[{"manager":"OpenAPI-Generator","operation":"Update","apiVersion":"v1","time":"2020-08-28T15:56:56Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:instance-type":{}}},"f:spec":{"f:clusterIP":{},"f:ports":{".":{},"k:{\"port\":2379,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}},"k:{\"port\":2380,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}},"k:{\"port\":4001,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}}},"f:selector":{".":{},"f:instance-type":{}},"f:sessionAffinity":{},"f:type":{}}}}]},"spec":{"ports":[{"name":"etcd-client","protocol":"TCP","port":2379,"targetPort":2379},{"name":"etcd-server","protocol":"TCP","port":2380,"targetPort":2380},{"name":"4001","protocol":"TCP","port":4001,"targetPort":4001}],"selector":{"instance-type":"etcd-pod"},"clusterIP":"None","type":"ClusterIP","sessionAffinity":"None"},"status":{"loadBalancer":{}}} 2020-08-28 15:56:56,881 - functest_kubernetes.ims.ims - INFO - Service etcd created 2020-08-28 15:56:56,882 - functest_kubernetes.ims.ims - DEBUG - create_namespaced_service: {'api_version': 'v1', 'binary_data': None, 'data': {'ADDITIONAL_SHARED_CONFIG': '', 'ZONE': 'ims-gc7hd.svc.cluster.local'}, 'kind': 'ConfigMap', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields': None, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal())}], 'name': 'env-vars', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801075', 'self_link': '/api/v1/namespaces/ims-gc7hd/configmaps/env-vars', 'uid': 'b2decaa3-a673-4992-9eb4-2cb8cacbe86a'}} 2020-08-28 15:56:57,015 - kubernetes.client.rest - DEBUG - response body: {"kind":"Service","apiVersion":"v1","metadata":{"name":"homer","namespace":"ims-gc7hd","selfLink":"/api/v1/namespaces/ims-gc7hd/services/homer","uid":"fe38849d-12a1-41de-851c-21559f9f3e64","resourceVersion":"1801260","creationTimestamp":"2020-08-28T15:56:56Z","managedFields":[{"manager":"OpenAPI-Generator","operation":"Update","apiVersion":"v1","time":"2020-08-28T15:56:56Z","fieldsType":"FieldsV1","fieldsV1":{"f:spec":{"f:clusterIP":{},"f:ports":{".":{},"k:{\"port\":7888,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}}},"f:selector":{".":{},"f:service":{}},"f:sessionAffinity":{},"f:type":{}}}}]},"spec":{"ports":[{"name":"7888","protocol":"TCP","port":7888,"targetPort":7888}],"selector":{"service":"homer"},"clusterIP":"None","type":"ClusterIP","sessionAffinity":"None"},"status":{"loadBalancer":{}}} 2020-08-28 15:56:57,025 - functest_kubernetes.ims.ims - INFO - Service homer created 2020-08-28 15:56:57,026 - functest_kubernetes.ims.ims - DEBUG - create_namespaced_service: {'api_version': 'v1', 'binary_data': None, 'data': {'ADDITIONAL_SHARED_CONFIG': '', 'ZONE': 'ims-gc7hd.svc.cluster.local'}, 'kind': 'ConfigMap', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields': None, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal())}], 'name': 'env-vars', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801075', 'self_link': '/api/v1/namespaces/ims-gc7hd/configmaps/env-vars', 'uid': 'b2decaa3-a673-4992-9eb4-2cb8cacbe86a'}} 2020-08-28 15:56:57,193 - kubernetes.client.rest - DEBUG - response body: {"kind":"Service","apiVersion":"v1","metadata":{"name":"homestead","namespace":"ims-gc7hd","selfLink":"/api/v1/namespaces/ims-gc7hd/services/homestead","uid":"172963ed-c537-495b-9f68-db72fee85565","resourceVersion":"1801267","creationTimestamp":"2020-08-28T15:56:57Z","managedFields":[{"manager":"OpenAPI-Generator","operation":"Update","apiVersion":"v1","time":"2020-08-28T15:56:57Z","fieldsType":"FieldsV1","fieldsV1":{"f:spec":{"f:clusterIP":{},"f:ports":{".":{},"k:{\"port\":8888,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}}},"f:selector":{".":{},"f:service":{}},"f:sessionAffinity":{},"f:type":{}}}}]},"spec":{"ports":[{"name":"8888","protocol":"TCP","port":8888,"targetPort":8888}],"selector":{"service":"homestead"},"clusterIP":"None","type":"ClusterIP","sessionAffinity":"None"},"status":{"loadBalancer":{}}} 2020-08-28 15:56:57,203 - functest_kubernetes.ims.ims - INFO - Service homestead created 2020-08-28 15:56:57,205 - functest_kubernetes.ims.ims - DEBUG - create_namespaced_service: {'api_version': 'v1', 'binary_data': None, 'data': {'ADDITIONAL_SHARED_CONFIG': '', 'ZONE': 'ims-gc7hd.svc.cluster.local'}, 'kind': 'ConfigMap', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields': None, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal())}], 'name': 'env-vars', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801075', 'self_link': '/api/v1/namespaces/ims-gc7hd/configmaps/env-vars', 'uid': 'b2decaa3-a673-4992-9eb4-2cb8cacbe86a'}} 2020-08-28 15:56:57,442 - kubernetes.client.rest - DEBUG - response body: {"kind":"Service","apiVersion":"v1","metadata":{"name":"homestead-prov","namespace":"ims-gc7hd","selfLink":"/api/v1/namespaces/ims-gc7hd/services/homestead-prov","uid":"d58cea9f-1061-46dd-8d10-749e83430b52","resourceVersion":"1801272","creationTimestamp":"2020-08-28T15:56:57Z","managedFields":[{"manager":"OpenAPI-Generator","operation":"Update","apiVersion":"v1","time":"2020-08-28T15:56:57Z","fieldsType":"FieldsV1","fieldsV1":{"f:spec":{"f:clusterIP":{},"f:ports":{".":{},"k:{\"port\":8889,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}}},"f:selector":{".":{},"f:service":{}},"f:sessionAffinity":{},"f:type":{}}}}]},"spec":{"ports":[{"name":"8889","protocol":"TCP","port":8889,"targetPort":8889}],"selector":{"service":"homestead-prov"},"clusterIP":"None","type":"ClusterIP","sessionAffinity":"None"},"status":{"loadBalancer":{}}} 2020-08-28 15:56:57,451 - functest_kubernetes.ims.ims - INFO - Service homestead-prov created 2020-08-28 15:56:57,452 - functest_kubernetes.ims.ims - DEBUG - create_namespaced_service: {'api_version': 'v1', 'binary_data': None, 'data': {'ADDITIONAL_SHARED_CONFIG': '', 'ZONE': 'ims-gc7hd.svc.cluster.local'}, 'kind': 'ConfigMap', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields': None, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal())}], 'name': 'env-vars', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801075', 'self_link': '/api/v1/namespaces/ims-gc7hd/configmaps/env-vars', 'uid': 'b2decaa3-a673-4992-9eb4-2cb8cacbe86a'}} 2020-08-28 15:56:57,974 - kubernetes.client.rest - DEBUG - response body: {"kind":"Service","apiVersion":"v1","metadata":{"name":"ralf","namespace":"ims-gc7hd","selfLink":"/api/v1/namespaces/ims-gc7hd/services/ralf","uid":"6e07aaf1-485b-4d3a-9fdd-fe293afede7c","resourceVersion":"1801276","creationTimestamp":"2020-08-28T15:56:57Z","managedFields":[{"manager":"OpenAPI-Generator","operation":"Update","apiVersion":"v1","time":"2020-08-28T15:56:57Z","fieldsType":"FieldsV1","fieldsV1":{"f:spec":{"f:clusterIP":{},"f:ports":{".":{},"k:{\"port\":10888,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}}},"f:selector":{".":{},"f:service":{}},"f:sessionAffinity":{},"f:type":{}}}}]},"spec":{"ports":[{"name":"10888","protocol":"TCP","port":10888,"targetPort":10888}],"selector":{"service":"ralf"},"clusterIP":"None","type":"ClusterIP","sessionAffinity":"None"},"status":{"loadBalancer":{}}} 2020-08-28 15:56:58,225 - functest_kubernetes.ims.ims - INFO - Service ralf created 2020-08-28 15:56:58,227 - functest_kubernetes.ims.ims - DEBUG - create_namespaced_service: {'api_version': 'v1', 'binary_data': None, 'data': {'ADDITIONAL_SHARED_CONFIG': '', 'ZONE': 'ims-gc7hd.svc.cluster.local'}, 'kind': 'ConfigMap', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields': None, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal())}], 'name': 'env-vars', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801075', 'self_link': '/api/v1/namespaces/ims-gc7hd/configmaps/env-vars', 'uid': 'b2decaa3-a673-4992-9eb4-2cb8cacbe86a'}} 2020-08-28 15:56:58,686 - kubernetes.client.rest - DEBUG - response body: {"kind":"Service","apiVersion":"v1","metadata":{"name":"sprout","namespace":"ims-gc7hd","selfLink":"/api/v1/namespaces/ims-gc7hd/services/sprout","uid":"8e492691-50b8-4b8c-8197-28da9d1dbf50","resourceVersion":"1801283","creationTimestamp":"2020-08-28T15:56:58Z","managedFields":[{"manager":"OpenAPI-Generator","operation":"Update","apiVersion":"v1","time":"2020-08-28T15:56:58Z","fieldsType":"FieldsV1","fieldsV1":{"f:spec":{"f:clusterIP":{},"f:ports":{".":{},"k:{\"port\":5052,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}},"k:{\"port\":5054,\"protocol\":\"TCP\"}":{".":{},"f:name":{},"f:port":{},"f:protocol":{},"f:targetPort":{}}},"f:selector":{".":{},"f:service":{}},"f:sessionAffinity":{},"f:type":{}}}}]},"spec":{"ports":[{"name":"5052","protocol":"TCP","port":5052,"targetPort":5052},{"name":"5054","protocol":"TCP","port":5054,"targetPort":5054}],"selector":{"service":"sprout"},"clusterIP":"None","type":"ClusterIP","sessionAffinity":"None"},"status":{"loadBalancer":{}}} 2020-08-28 15:56:58,697 - functest_kubernetes.ims.ims - INFO - Service sprout created 2020-08-28 15:56:58,698 - functest_kubernetes.ims.ims - DEBUG - create_namespaced_service: {'api_version': 'v1', 'binary_data': None, 'data': {'ADDITIONAL_SHARED_CONFIG': '', 'ZONE': 'ims-gc7hd.svc.cluster.local'}, 'kind': 'ConfigMap', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields': None, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal())}], 'name': 'env-vars', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801075', 'self_link': '/api/v1/namespaces/ims-gc7hd/configmaps/env-vars', 'uid': 'b2decaa3-a673-4992-9eb4-2cb8cacbe86a'}} 2020-08-28 15:56:58,978 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': None, 'managed_fields': None, 'name': 'astaire', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801105', 'self_link': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/astaire', 'uid': 'a586a36b-c53d-4586-be2a-1620364a644b'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'astaire'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'astaire'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-astaire:latest', 'image_pull_policy': 'Always', 'lifecycle': {'post_start': None, 'pre_stop': {'_exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}, 'http_get': None, 'tcp_socket': None}}, 'liveness_probe': {'_exec': None, 'failure_threshold': 9, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 11311}, 'timeout_seconds': 1}, 'name': 'astaire', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 11211, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 11311, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 11311}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/astaire', 'mount_propagation': None, 'name': 'astairelogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/astaire/astaire_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/astaire', 'mount_propagation': None, 'name': 'astairelogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'cassandra; ' 'do echo ' 'waiting for ' 'cassandra to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 120, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'astairelogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 49, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 15, 56, 49, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum ' 'availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 49, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 15, 56, 50, tzinfo=tzlocal()), 'message': 'ReplicaSet "astaire-7666ff5885" is ' 'progressing.', 'reason': 'ReplicaSetUpdated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': 1, 'unavailable_replicas': 1, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'astaire', 'namespace': 'ims-gc7hd', 'selfLink': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/astaire', 'uid': 'a586a36b-c53d-4586-be2a-1620364a644b', 'resourceVersion': '1801105', 'generation': 1, 'creationTimestamp': '2020-08-28T15:56:48Z', 'annotations': {'deployment.kubernetes.io/revision': '1'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'astaire'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'astaire'}}, 'spec': {'volumes': [{'name': 'astairelogs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'astaire', 'image': 'ollivier/clearwater-astaire:latest', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 11211, 'protocol': 'TCP'}, {'containerPort': 11311, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'astairelogs', 'mountPath': '/var/log/astaire'}], 'livenessProbe': {'tcpSocket': {'port': 11311}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 9}, 'readinessProbe': {'tcpSocket': {'port': 11311}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'lifecycle': {'preStop': {'exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}}}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}, {'name': 'tailer', 'image': 'busybox', 'command': ['tail', '-F', '/var/log/astaire/astaire_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'astairelogs', 'mountPath': '/var/log/astaire'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 120, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'unavailableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'False', 'lastUpdateTime': '2020-08-28T15:56:49Z', 'lastTransitionTime': '2020-08-28T15:56:49Z', 'reason': 'MinimumReplicasUnavailable', 'message': 'Deployment does not have minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-08-28T15:56:50Z', 'lastTransitionTime': '2020-08-28T15:56:49Z', 'reason': 'ReplicaSetUpdated', 'message': 'ReplicaSet "astaire-7666ff5885" is progressing.'}]}}} 2020-08-28 15:56:59,266 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 49, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': None, 'managed_fields': None, 'name': 'bono', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801123', 'self_link': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/bono', 'uid': 'd6bc69e0-8471-4b52-ae66-fd1f8e8bf787'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'bono'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'bono', 'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-bono:latest', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 ' '5060 ' '5062']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'bono', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 3478, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 5060, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 5062, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 5060, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'UDP'}, {'container_port': 5062, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'UDP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 ' '5060 ' '5062']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/bono', 'mount_propagation': None, 'name': 'bonologs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/bono/bono_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/bono', 'mount_propagation': None, 'name': 'bonologs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'sprout; do ' 'echo waiting ' 'for sprout ' 'to start ' '...; sleep ' '2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-sprout', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'bonologs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 50, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 15, 56, 50, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum ' 'availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 49, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 15, 56, 50, tzinfo=tzlocal()), 'message': 'ReplicaSet "bono-69fbbd8646" is ' 'progressing.', 'reason': 'ReplicaSetUpdated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': 1, 'unavailable_replicas': 1, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'bono', 'namespace': 'ims-gc7hd', 'selfLink': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/bono', 'uid': 'd6bc69e0-8471-4b52-ae66-fd1f8e8bf787', 'resourceVersion': '1801123', 'generation': 1, 'creationTimestamp': '2020-08-28T15:56:49Z', 'annotations': {'deployment.kubernetes.io/revision': '1'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'bono'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'bono', 'snmp': 'enabled'}}, 'spec': {'volumes': [{'name': 'bonologs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-sprout', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup sprout; do echo waiting for sprout to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'bono', 'image': 'ollivier/clearwater-bono:latest', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 3478, 'protocol': 'TCP'}, {'containerPort': 5060, 'protocol': 'TCP'}, {'containerPort': 5062, 'protocol': 'TCP'}, {'containerPort': 5060, 'protocol': 'UDP'}, {'containerPort': 5062, 'protocol': 'UDP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'bonologs', 'mountPath': '/var/log/bono'}], 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 5060 5062']}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 5060 5062']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}, {'name': 'tailer', 'image': 'busybox', 'command': ['tail', '-F', '/var/log/bono/bono_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'bonologs', 'mountPath': '/var/log/bono'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'unavailableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'False', 'lastUpdateTime': '2020-08-28T15:56:50Z', 'lastTransitionTime': '2020-08-28T15:56:50Z', 'reason': 'MinimumReplicasUnavailable', 'message': 'Deployment does not have minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-08-28T15:56:50Z', 'lastTransitionTime': '2020-08-28T15:56:49Z', 'reason': 'ReplicaSetUpdated', 'message': 'ReplicaSet "bono-69fbbd8646" is progressing.'}]}}} 2020-08-28 15:56:59,778 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 50, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'service': 'chronos'}, 'managed_fields': None, 'name': 'chronos', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801132', 'self_link': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/chronos', 'uid': '8911bd1e-c40e-4013-af3e-2e070733afb4'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'chronos'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'chronos'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-chronos:latest', 'image_pull_policy': 'Always', 'lifecycle': {'post_start': None, 'pre_stop': {'_exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}, 'http_get': None, 'tcp_socket': None}}, 'liveness_probe': {'_exec': None, 'failure_threshold': 9, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7253}, 'timeout_seconds': 1}, 'name': 'chronos', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 7253, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7253}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/chronos', 'mount_propagation': None, 'name': 'chronoslogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/chronos/chronos_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/chronos', 'mount_propagation': None, 'name': 'chronoslogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'cassandra; ' 'do echo ' 'waiting for ' 'cassandra to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 120, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'chronoslogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 50, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 15, 56, 50, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum ' 'availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 50, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 15, 56, 50, tzinfo=tzlocal()), 'message': 'ReplicaSet "chronos-c55b4b9ff" is ' 'progressing.', 'reason': 'ReplicaSetUpdated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': 1, 'unavailable_replicas': 1, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'chronos', 'namespace': 'ims-gc7hd', 'selfLink': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/chronos', 'uid': '8911bd1e-c40e-4013-af3e-2e070733afb4', 'resourceVersion': '1801132', 'generation': 1, 'creationTimestamp': '2020-08-28T15:56:50Z', 'labels': {'service': 'chronos'}, 'annotations': {'deployment.kubernetes.io/revision': '1'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'chronos'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'chronos'}}, 'spec': {'volumes': [{'name': 'chronoslogs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'chronos', 'image': 'ollivier/clearwater-chronos:latest', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 7253, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'chronoslogs', 'mountPath': '/var/log/chronos'}], 'livenessProbe': {'tcpSocket': {'port': 7253}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 9}, 'readinessProbe': {'tcpSocket': {'port': 7253}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'lifecycle': {'preStop': {'exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}}}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}, {'name': 'tailer', 'image': 'busybox', 'command': ['tail', '-F', '/var/log/chronos/chronos_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'chronoslogs', 'mountPath': '/var/log/chronos'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 120, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'unavailableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'False', 'lastUpdateTime': '2020-08-28T15:56:50Z', 'lastTransitionTime': '2020-08-28T15:56:50Z', 'reason': 'MinimumReplicasUnavailable', 'message': 'Deployment does not have minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-08-28T15:56:50Z', 'lastTransitionTime': '2020-08-28T15:56:50Z', 'reason': 'ReplicaSetUpdated', 'message': 'ReplicaSet "chronos-c55b4b9ff" is progressing.'}]}}} 2020-08-28 15:57:00,187 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 50, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': None, 'managed_fields': None, 'name': 'etcd', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801168', 'self_link': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/etcd', 'uid': '38f69988-86b5-423a-8e0b-b9f3382efc35'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'instance-type': 'etcd-pod'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'instance-type': 'etcd-pod'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': ['--name', '$(MY_POD_NAME)', '--advertise-client-urls', 'http://$(MY_POD_IP):2379,http://$(MY_POD_IP):4001', '--listen-client-urls', 'http://0.0.0.0:2379,http://0.0.0.0:4001', '--initial-advertise-peer-urls', 'http://$(MY_POD_IP):2380', '--listen-peer-urls', 'http://0.0.0.0:2380', '--initial-cluster', '$(MY_POD_NAME)=http://$(MY_POD_IP):2380', '--initial-cluster-state', 'new'], 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}, {'name': 'MY_POD_NAME', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'metadata.name'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': None, 'image': 'quay.io/coreos/etcd:v2.2.5', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 300, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 4001}, 'timeout_seconds': 1}, 'name': 'etcd', 'ports': [{'container_port': 2379, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 2380, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 4001, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 4001}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': None, 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 51, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 15, 56, 51, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum ' 'availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 51, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 15, 56, 52, tzinfo=tzlocal()), 'message': 'ReplicaSet "etcd-596fb59b77" is ' 'progressing.', 'reason': 'ReplicaSetUpdated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': 1, 'unavailable_replicas': 1, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'etcd', 'namespace': 'ims-gc7hd', 'selfLink': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/etcd', 'uid': '38f69988-86b5-423a-8e0b-b9f3382efc35', 'resourceVersion': '1801168', 'generation': 1, 'creationTimestamp': '2020-08-28T15:56:50Z', 'annotations': {'deployment.kubernetes.io/revision': '1'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'instance-type': 'etcd-pod'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'instance-type': 'etcd-pod'}}, 'spec': {'containers': [{'name': 'etcd', 'image': 'quay.io/coreos/etcd:v2.2.5', 'args': ['--name', '$(MY_POD_NAME)', '--advertise-client-urls', 'http://$(MY_POD_IP):2379,http://$(MY_POD_IP):4001', '--listen-client-urls', 'http://0.0.0.0:2379,http://0.0.0.0:4001', '--initial-advertise-peer-urls', 'http://$(MY_POD_IP):2380', '--listen-peer-urls', 'http://0.0.0.0:2380', '--initial-cluster', '$(MY_POD_NAME)=http://$(MY_POD_IP):2380', '--initial-cluster-state', 'new'], 'ports': [{'containerPort': 2379, 'protocol': 'TCP'}, {'containerPort': 2380, 'protocol': 'TCP'}, {'containerPort': 4001, 'protocol': 'TCP'}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}, {'name': 'MY_POD_NAME', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'metadata.name'}}}], 'resources': {}, 'livenessProbe': {'tcpSocket': {'port': 4001}, 'initialDelaySeconds': 300, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'tcpSocket': {'port': 4001}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'unavailableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'False', 'lastUpdateTime': '2020-08-28T15:56:51Z', 'lastTransitionTime': '2020-08-28T15:56:51Z', 'reason': 'MinimumReplicasUnavailable', 'message': 'Deployment does not have minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-08-28T15:56:52Z', 'lastTransitionTime': '2020-08-28T15:56:51Z', 'reason': 'ReplicaSetUpdated', 'message': 'ReplicaSet "etcd-596fb59b77" is progressing.'}]}}} 2020-08-28 15:57:00,382 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 51, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': None, 'managed_fields': None, 'name': 'homer', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801183', 'self_link': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/homer', 'uid': 'fef4c095-1313-4b44-b9d6-df772baa5d24'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'homer'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'homer'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-homer:latest', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7888}, 'timeout_seconds': 1}, 'name': 'homer', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 7888, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7888}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'cassandra; ' 'do echo ' 'waiting for ' 'cassandra to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 52, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 15, 56, 52, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum ' 'availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 51, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 15, 56, 52, tzinfo=tzlocal()), 'message': 'ReplicaSet "homer-5fc8774db7" is ' 'progressing.', 'reason': 'ReplicaSetUpdated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': 1, 'unavailable_replicas': 1, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'homer', 'namespace': 'ims-gc7hd', 'selfLink': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/homer', 'uid': 'fef4c095-1313-4b44-b9d6-df772baa5d24', 'resourceVersion': '1801183', 'generation': 1, 'creationTimestamp': '2020-08-28T15:56:51Z', 'annotations': {'deployment.kubernetes.io/revision': '1'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'homer'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'homer'}}, 'spec': {'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'homer', 'image': 'ollivier/clearwater-homer:latest', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 7888, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'livenessProbe': {'tcpSocket': {'port': 7888}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'tcpSocket': {'port': 7888}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'unavailableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'False', 'lastUpdateTime': '2020-08-28T15:56:52Z', 'lastTransitionTime': '2020-08-28T15:56:52Z', 'reason': 'MinimumReplicasUnavailable', 'message': 'Deployment does not have minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-08-28T15:56:52Z', 'lastTransitionTime': '2020-08-28T15:56:51Z', 'reason': 'ReplicaSetUpdated', 'message': 'ReplicaSet "homer-5fc8774db7" is progressing.'}]}}} 2020-08-28 15:57:00,573 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 49, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': None, 'managed_fields': None, 'name': 'cassandra', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801125', 'self_link': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/cassandra', 'uid': 'dc3fbaac-7655-45e2-b0da-c94598a92d8b'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'cassandra'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'cassandra'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-cassandra:latest', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 ' '9042 ' '9160']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 600, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'cassandra', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 7001, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 9042, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 9160, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 ' '9042 ' '9160']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'etcd; do ' 'echo waiting ' 'for etcd to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-etcd', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 50, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 15, 56, 50, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum ' 'availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 50, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 15, 56, 50, tzinfo=tzlocal()), 'message': 'ReplicaSet "cassandra-7fbf8bb9cf" is ' 'progressing.', 'reason': 'ReplicaSetUpdated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': 1, 'unavailable_replicas': 1, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'cassandra', 'namespace': 'ims-gc7hd', 'selfLink': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/cassandra', 'uid': 'dc3fbaac-7655-45e2-b0da-c94598a92d8b', 'resourceVersion': '1801125', 'generation': 1, 'creationTimestamp': '2020-08-28T15:56:49Z', 'annotations': {'deployment.kubernetes.io/revision': '1'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'cassandra'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'cassandra'}}, 'spec': {'initContainers': [{'name': 'wait-on-etcd', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup etcd; do echo waiting for etcd to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'cassandra', 'image': 'ollivier/clearwater-cassandra:latest', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 7001, 'protocol': 'TCP'}, {'containerPort': 9042, 'protocol': 'TCP'}, {'containerPort': 9160, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 9042 9160']}, 'initialDelaySeconds': 600, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 9042 9160']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'unavailableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'False', 'lastUpdateTime': '2020-08-28T15:56:50Z', 'lastTransitionTime': '2020-08-28T15:56:50Z', 'reason': 'MinimumReplicasUnavailable', 'message': 'Deployment does not have minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-08-28T15:56:50Z', 'lastTransitionTime': '2020-08-28T15:56:50Z', 'reason': 'ReplicaSetUpdated', 'message': 'ReplicaSet "cassandra-7fbf8bb9cf" is progressing.'}]}}} 2020-08-28 15:57:00,764 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 50, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': None, 'managed_fields': None, 'name': 'ellis', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801158', 'self_link': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/ellis', 'uid': '9d4959be-6eec-4d20-a139-c33010481c17'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'ellis'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'ellis'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-ellis:latest', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 80}, 'timeout_seconds': 1}, 'name': 'ellis', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 80, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 80}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'homestead-prov; ' 'do echo ' 'waiting for ' 'homestead-prov ' 'to start ' '...; sleep ' '2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-homestead-prov', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'sprout; do ' 'echo waiting ' 'for sprout ' 'to start ' '...; sleep ' '2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-sprout', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 50, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 15, 56, 50, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum ' 'availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 50, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 15, 56, 51, tzinfo=tzlocal()), 'message': 'ReplicaSet "ellis-5dfd856d6c" is ' 'progressing.', 'reason': 'ReplicaSetUpdated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': 1, 'unavailable_replicas': 1, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'ellis', 'namespace': 'ims-gc7hd', 'selfLink': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/ellis', 'uid': '9d4959be-6eec-4d20-a139-c33010481c17', 'resourceVersion': '1801158', 'generation': 1, 'creationTimestamp': '2020-08-28T15:56:50Z', 'annotations': {'deployment.kubernetes.io/revision': '1'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'ellis'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'ellis'}}, 'spec': {'initContainers': [{'name': 'wait-on-homestead-prov', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup homestead-prov; do echo waiting for homestead-prov to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'wait-on-sprout', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup sprout; do echo waiting for sprout to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'ellis', 'image': 'ollivier/clearwater-ellis:latest', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 80, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'livenessProbe': {'tcpSocket': {'port': 80}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'tcpSocket': {'port': 80}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'unavailableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'False', 'lastUpdateTime': '2020-08-28T15:56:50Z', 'lastTransitionTime': '2020-08-28T15:56:50Z', 'reason': 'MinimumReplicasUnavailable', 'message': 'Deployment does not have minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-08-28T15:56:51Z', 'lastTransitionTime': '2020-08-28T15:56:50Z', 'reason': 'ReplicaSetUpdated', 'message': 'ReplicaSet "ellis-5dfd856d6c" is progressing.'}]}}} 2020-08-28 15:57:01,061 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 51, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': None, 'managed_fields': None, 'name': 'homestead', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801208', 'self_link': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/homestead', 'uid': '84ae3654-a60b-43f4-9ba3-8550a7a2c87f'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'homestead'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'homestead', 'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-homestead:latest', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 60, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'homestead', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 8888, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/homestead', 'mount_propagation': None, 'name': 'homesteadlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/homestead/homestead_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/homestead', 'mount_propagation': None, 'name': 'homesteadlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'cassandra; ' 'do echo ' 'waiting for ' 'cassandra to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'astaire; do ' 'echo waiting ' 'for astaire ' 'to start ' '...; sleep ' '2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-astaire', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'homesteadlogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 52, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 15, 56, 52, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum ' 'availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 52, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 15, 56, 53, tzinfo=tzlocal()), 'message': 'ReplicaSet "homestead-7dbc69fbf9" is ' 'progressing.', 'reason': 'ReplicaSetUpdated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': 1, 'unavailable_replicas': 1, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'homestead', 'namespace': 'ims-gc7hd', 'selfLink': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/homestead', 'uid': '84ae3654-a60b-43f4-9ba3-8550a7a2c87f', 'resourceVersion': '1801208', 'generation': 1, 'creationTimestamp': '2020-08-28T15:56:51Z', 'annotations': {'deployment.kubernetes.io/revision': '1'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'homestead'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'homestead', 'snmp': 'enabled'}}, 'spec': {'volumes': [{'name': 'homesteadlogs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'wait-on-astaire', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup astaire; do echo waiting for astaire to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'homestead', 'image': 'ollivier/clearwater-homestead:latest', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 8888, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'homesteadlogs', 'mountPath': '/var/log/homestead'}], 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'initialDelaySeconds': 60, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}, {'name': 'tailer', 'image': 'busybox', 'command': ['tail', '-F', '/var/log/homestead/homestead_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'homesteadlogs', 'mountPath': '/var/log/homestead'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'unavailableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'False', 'lastUpdateTime': '2020-08-28T15:56:52Z', 'lastTransitionTime': '2020-08-28T15:56:52Z', 'reason': 'MinimumReplicasUnavailable', 'message': 'Deployment does not have minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-08-28T15:56:53Z', 'lastTransitionTime': '2020-08-28T15:56:52Z', 'reason': 'ReplicaSetUpdated', 'message': 'ReplicaSet "homestead-7dbc69fbf9" is progressing.'}]}}} 2020-08-28 15:57:01,484 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 52, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': None, 'managed_fields': None, 'name': 'homestead-prov', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801224', 'self_link': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/homestead-prov', 'uid': 'e8b84542-2695-4e30-a7a1-42b0e16de3fe'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'homestead-prov'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'homestead-prov', 'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-homestead-prov:latest', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 60, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'homestead-prov', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 8889, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'homestead; ' 'do echo ' 'waiting for ' 'homestead to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-homestead', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 53, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 15, 56, 53, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum ' 'availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 52, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 15, 56, 53, tzinfo=tzlocal()), 'message': 'ReplicaSet "homestead-prov-555fbf67b" ' 'is progressing.', 'reason': 'ReplicaSetUpdated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': 1, 'unavailable_replicas': 1, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'homestead-prov', 'namespace': 'ims-gc7hd', 'selfLink': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/homestead-prov', 'uid': 'e8b84542-2695-4e30-a7a1-42b0e16de3fe', 'resourceVersion': '1801224', 'generation': 1, 'creationTimestamp': '2020-08-28T15:56:52Z', 'annotations': {'deployment.kubernetes.io/revision': '1'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'homestead-prov'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'homestead-prov', 'snmp': 'enabled'}}, 'spec': {'initContainers': [{'name': 'wait-on-homestead', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup homestead; do echo waiting for homestead to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'homestead-prov', 'image': 'ollivier/clearwater-homestead-prov:latest', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 8889, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'initialDelaySeconds': 60, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'unavailableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'False', 'lastUpdateTime': '2020-08-28T15:56:53Z', 'lastTransitionTime': '2020-08-28T15:56:53Z', 'reason': 'MinimumReplicasUnavailable', 'message': 'Deployment does not have minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-08-28T15:56:53Z', 'lastTransitionTime': '2020-08-28T15:56:52Z', 'reason': 'ReplicaSetUpdated', 'message': 'ReplicaSet "homestead-prov-555fbf67b" is progressing.'}]}}} 2020-08-28 15:57:01,720 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 52, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': None, 'managed_fields': None, 'name': 'ralf', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801236', 'self_link': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/ralf', 'uid': 'c0685c0a-f472-46a2-9b78-32d4cbb5720b'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'ralf'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'ralf', 'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-ralf:latest', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 10888}, 'timeout_seconds': 1}, 'name': 'ralf', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 10888, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 10888}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/ralf', 'mount_propagation': None, 'name': 'ralflogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/ralf/ralf_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/ralf', 'mount_propagation': None, 'name': 'ralflogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'cassandra; ' 'do echo ' 'waiting for ' 'cassandra to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'ralflogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 53, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 15, 56, 53, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum ' 'availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 53, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 15, 56, 54, tzinfo=tzlocal()), 'message': 'ReplicaSet "ralf-545844cc6d" is ' 'progressing.', 'reason': 'ReplicaSetUpdated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': 1, 'unavailable_replicas': 1, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'ralf', 'namespace': 'ims-gc7hd', 'selfLink': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/ralf', 'uid': 'c0685c0a-f472-46a2-9b78-32d4cbb5720b', 'resourceVersion': '1801236', 'generation': 1, 'creationTimestamp': '2020-08-28T15:56:52Z', 'annotations': {'deployment.kubernetes.io/revision': '1'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'ralf'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'ralf', 'snmp': 'enabled'}}, 'spec': {'volumes': [{'name': 'ralflogs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'ralf', 'image': 'ollivier/clearwater-ralf:latest', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 10888, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'ralflogs', 'mountPath': '/var/log/ralf'}], 'livenessProbe': {'tcpSocket': {'port': 10888}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'tcpSocket': {'port': 10888}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}, {'name': 'tailer', 'image': 'busybox', 'command': ['tail', '-F', '/var/log/ralf/ralf_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'ralflogs', 'mountPath': '/var/log/ralf'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'unavailableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'False', 'lastUpdateTime': '2020-08-28T15:56:53Z', 'lastTransitionTime': '2020-08-28T15:56:53Z', 'reason': 'MinimumReplicasUnavailable', 'message': 'Deployment does not have minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-08-28T15:56:54Z', 'lastTransitionTime': '2020-08-28T15:56:53Z', 'reason': 'ReplicaSetUpdated', 'message': 'ReplicaSet "ralf-545844cc6d" is progressing.'}]}}} 2020-08-28 15:57:01,983 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 53, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': None, 'managed_fields': None, 'name': 'sprout', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801240', 'self_link': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/sprout', 'uid': 'b3390d47-04d1-4afc-9a3e-fc9de582b4fc'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'sprout'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'sprout', 'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-sprout:latest', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 ' '5054']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 3, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'sprout', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 ' '5054']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/sprout', 'mount_propagation': None, 'name': 'sproutlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/sprout/sprout_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/sprout', 'mount_propagation': None, 'name': 'sproutlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'ralf; do ' 'echo waiting ' 'for ralf to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-ralf', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'chronos; do ' 'echo waiting ' 'for chronos ' 'to start ' '...; sleep ' '2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-chronos', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'homestead; ' 'do echo ' 'waiting for ' 'homestead to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-homestead', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'sproutlogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': None, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 54, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 15, 56, 54, tzinfo=tzlocal()), 'message': 'Deployment does not have minimum ' 'availability.', 'reason': 'MinimumReplicasUnavailable', 'status': 'False', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 53, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 15, 56, 55, tzinfo=tzlocal()), 'message': 'ReplicaSet "sprout-669689d654" is ' 'progressing.', 'reason': 'ReplicaSetUpdated', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': None, 'replicas': 1, 'unavailable_replicas': 1, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'sprout', 'namespace': 'ims-gc7hd', 'selfLink': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/sprout', 'uid': 'b3390d47-04d1-4afc-9a3e-fc9de582b4fc', 'resourceVersion': '1801240', 'generation': 1, 'creationTimestamp': '2020-08-28T15:56:53Z', 'annotations': {'deployment.kubernetes.io/revision': '1'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'sprout'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'sprout', 'snmp': 'enabled'}}, 'spec': {'volumes': [{'name': 'sproutlogs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-ralf', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup ralf; do echo waiting for ralf to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'wait-on-chronos', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup chronos; do echo waiting for chronos to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'wait-on-homestead', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup homestead; do echo waiting for homestead to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'sprout', 'image': 'ollivier/clearwater-sprout:latest', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'sproutlogs', 'mountPath': '/var/log/sprout'}], 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 5054']}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 3, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 5054']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}, {'name': 'tailer', 'image': 'busybox', 'command': ['tail', '-F', '/var/log/sprout/sprout_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'sproutlogs', 'mountPath': '/var/log/sprout'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'unavailableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'False', 'lastUpdateTime': '2020-08-28T15:56:54Z', 'lastTransitionTime': '2020-08-28T15:56:54Z', 'reason': 'MinimumReplicasUnavailable', 'message': 'Deployment does not have minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-08-28T15:56:55Z', 'lastTransitionTime': '2020-08-28T15:56:53Z', 'reason': 'ReplicaSetUpdated', 'message': 'ReplicaSet "sprout-669689d654" is progressing.'}]}}} 2020-08-28 15:57:15,143 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 50, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': None, 'managed_fields': None, 'name': 'etcd', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801389', 'self_link': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/etcd', 'uid': '38f69988-86b5-423a-8e0b-b9f3382efc35'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'instance-type': 'etcd-pod'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'instance-type': 'etcd-pod'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': ['--name', '$(MY_POD_NAME)', '--advertise-client-urls', 'http://$(MY_POD_IP):2379,http://$(MY_POD_IP):4001', '--listen-client-urls', 'http://0.0.0.0:2379,http://0.0.0.0:4001', '--initial-advertise-peer-urls', 'http://$(MY_POD_IP):2380', '--listen-peer-urls', 'http://0.0.0.0:2380', '--initial-cluster', '$(MY_POD_NAME)=http://$(MY_POD_IP):2380', '--initial-cluster-state', 'new'], 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}, {'name': 'MY_POD_NAME', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'metadata.name'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': None, 'image': 'quay.io/coreos/etcd:v2.2.5', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 300, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 4001}, 'timeout_seconds': 1}, 'name': 'etcd', 'ports': [{'container_port': 2379, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 2380, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 4001, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 4001}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': None, 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': 1, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 8, 28, 15, 57, 15, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 15, 57, 15, tzinfo=tzlocal()), 'message': 'Deployment has minimum availability.', 'reason': 'MinimumReplicasAvailable', 'status': 'True', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 51, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 15, 57, 15, tzinfo=tzlocal()), 'message': 'ReplicaSet "etcd-596fb59b77" has ' 'successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': 1, 'replicas': 1, 'unavailable_replicas': None, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'etcd', 'namespace': 'ims-gc7hd', 'selfLink': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/etcd', 'uid': '38f69988-86b5-423a-8e0b-b9f3382efc35', 'resourceVersion': '1801389', 'generation': 1, 'creationTimestamp': '2020-08-28T15:56:50Z', 'annotations': {'deployment.kubernetes.io/revision': '1'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'instance-type': 'etcd-pod'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'instance-type': 'etcd-pod'}}, 'spec': {'containers': [{'name': 'etcd', 'image': 'quay.io/coreos/etcd:v2.2.5', 'args': ['--name', '$(MY_POD_NAME)', '--advertise-client-urls', 'http://$(MY_POD_IP):2379,http://$(MY_POD_IP):4001', '--listen-client-urls', 'http://0.0.0.0:2379,http://0.0.0.0:4001', '--initial-advertise-peer-urls', 'http://$(MY_POD_IP):2380', '--listen-peer-urls', 'http://0.0.0.0:2380', '--initial-cluster', '$(MY_POD_NAME)=http://$(MY_POD_IP):2380', '--initial-cluster-state', 'new'], 'ports': [{'containerPort': 2379, 'protocol': 'TCP'}, {'containerPort': 2380, 'protocol': 'TCP'}, {'containerPort': 4001, 'protocol': 'TCP'}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}, {'name': 'MY_POD_NAME', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'metadata.name'}}}], 'resources': {}, 'livenessProbe': {'tcpSocket': {'port': 4001}, 'initialDelaySeconds': 300, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'tcpSocket': {'port': 4001}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'readyReplicas': 1, 'availableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'True', 'lastUpdateTime': '2020-08-28T15:57:15Z', 'lastTransitionTime': '2020-08-28T15:57:15Z', 'reason': 'MinimumReplicasAvailable', 'message': 'Deployment has minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-08-28T15:57:15Z', 'lastTransitionTime': '2020-08-28T15:56:51Z', 'reason': 'NewReplicaSetAvailable', 'message': 'ReplicaSet "etcd-596fb59b77" has successfully progressed.'}]}}} 2020-08-28 15:57:15,319 - functest_kubernetes.ims.ims - INFO - etcd started in 27.18 sec 2020-08-28 15:59:02,043 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 49, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': None, 'managed_fields': None, 'name': 'cassandra', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801710', 'self_link': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/cassandra', 'uid': 'dc3fbaac-7655-45e2-b0da-c94598a92d8b'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'cassandra'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'cassandra'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-cassandra:latest', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 ' '9042 ' '9160']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 600, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'cassandra', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 7001, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 9042, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 9160, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 ' '9042 ' '9160']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'etcd; do ' 'echo waiting ' 'for etcd to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-etcd', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': 1, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 8, 28, 15, 59, 1, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 15, 59, 1, tzinfo=tzlocal()), 'message': 'Deployment has minimum availability.', 'reason': 'MinimumReplicasAvailable', 'status': 'True', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 50, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 15, 59, 1, tzinfo=tzlocal()), 'message': 'ReplicaSet "cassandra-7fbf8bb9cf" has ' 'successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': 1, 'replicas': 1, 'unavailable_replicas': None, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'cassandra', 'namespace': 'ims-gc7hd', 'selfLink': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/cassandra', 'uid': 'dc3fbaac-7655-45e2-b0da-c94598a92d8b', 'resourceVersion': '1801710', 'generation': 1, 'creationTimestamp': '2020-08-28T15:56:49Z', 'annotations': {'deployment.kubernetes.io/revision': '1'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'cassandra'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'cassandra'}}, 'spec': {'initContainers': [{'name': 'wait-on-etcd', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup etcd; do echo waiting for etcd to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'cassandra', 'image': 'ollivier/clearwater-cassandra:latest', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 7001, 'protocol': 'TCP'}, {'containerPort': 9042, 'protocol': 'TCP'}, {'containerPort': 9160, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 9042 9160']}, 'initialDelaySeconds': 600, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 9042 9160']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'readyReplicas': 1, 'availableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'True', 'lastUpdateTime': '2020-08-28T15:59:01Z', 'lastTransitionTime': '2020-08-28T15:59:01Z', 'reason': 'MinimumReplicasAvailable', 'message': 'Deployment has minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-08-28T15:59:01Z', 'lastTransitionTime': '2020-08-28T15:56:50Z', 'reason': 'NewReplicaSetAvailable', 'message': 'ReplicaSet "cassandra-7fbf8bb9cf" has successfully progressed.'}]}}} 2020-08-28 15:59:02,247 - functest_kubernetes.ims.ims - INFO - cassandra started in 134.11 sec 2020-08-28 15:59:26,864 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 51, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': None, 'managed_fields': None, 'name': 'homer', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801820', 'self_link': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/homer', 'uid': 'fef4c095-1313-4b44-b9d6-df772baa5d24'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'homer'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'homer'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-homer:latest', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7888}, 'timeout_seconds': 1}, 'name': 'homer', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 7888, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7888}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'cassandra; ' 'do echo ' 'waiting for ' 'cassandra to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': 1, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 8, 28, 15, 59, 26, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 15, 59, 26, tzinfo=tzlocal()), 'message': 'Deployment has minimum availability.', 'reason': 'MinimumReplicasAvailable', 'status': 'True', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 51, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 15, 59, 26, tzinfo=tzlocal()), 'message': 'ReplicaSet "homer-5fc8774db7" has ' 'successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': 1, 'replicas': 1, 'unavailable_replicas': None, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'homer', 'namespace': 'ims-gc7hd', 'selfLink': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/homer', 'uid': 'fef4c095-1313-4b44-b9d6-df772baa5d24', 'resourceVersion': '1801820', 'generation': 1, 'creationTimestamp': '2020-08-28T15:56:51Z', 'annotations': {'deployment.kubernetes.io/revision': '1'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'homer'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'homer'}}, 'spec': {'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'homer', 'image': 'ollivier/clearwater-homer:latest', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 7888, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'livenessProbe': {'tcpSocket': {'port': 7888}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'tcpSocket': {'port': 7888}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'readyReplicas': 1, 'availableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'True', 'lastUpdateTime': '2020-08-28T15:59:26Z', 'lastTransitionTime': '2020-08-28T15:59:26Z', 'reason': 'MinimumReplicasAvailable', 'message': 'Deployment has minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-08-28T15:59:26Z', 'lastTransitionTime': '2020-08-28T15:56:51Z', 'reason': 'NewReplicaSetAvailable', 'message': 'ReplicaSet "homer-5fc8774db7" has successfully progressed.'}]}}} 2020-08-28 15:59:27,046 - functest_kubernetes.ims.ims - INFO - homer started in 158.91 sec 2020-08-28 15:59:28,901 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 52, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': None, 'managed_fields': None, 'name': 'ralf', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801831', 'self_link': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/ralf', 'uid': 'c0685c0a-f472-46a2-9b78-32d4cbb5720b'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'ralf'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'ralf', 'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-ralf:latest', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 10888}, 'timeout_seconds': 1}, 'name': 'ralf', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 10888, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 10888}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/ralf', 'mount_propagation': None, 'name': 'ralflogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/ralf/ralf_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/ralf', 'mount_propagation': None, 'name': 'ralflogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'cassandra; ' 'do echo ' 'waiting for ' 'cassandra to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'ralflogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': 1, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 8, 28, 15, 59, 28, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 15, 59, 28, tzinfo=tzlocal()), 'message': 'Deployment has minimum availability.', 'reason': 'MinimumReplicasAvailable', 'status': 'True', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 53, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 15, 59, 28, tzinfo=tzlocal()), 'message': 'ReplicaSet "ralf-545844cc6d" has ' 'successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': 1, 'replicas': 1, 'unavailable_replicas': None, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'ralf', 'namespace': 'ims-gc7hd', 'selfLink': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/ralf', 'uid': 'c0685c0a-f472-46a2-9b78-32d4cbb5720b', 'resourceVersion': '1801831', 'generation': 1, 'creationTimestamp': '2020-08-28T15:56:52Z', 'annotations': {'deployment.kubernetes.io/revision': '1'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'ralf'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'ralf', 'snmp': 'enabled'}}, 'spec': {'volumes': [{'name': 'ralflogs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'ralf', 'image': 'ollivier/clearwater-ralf:latest', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 10888, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'ralflogs', 'mountPath': '/var/log/ralf'}], 'livenessProbe': {'tcpSocket': {'port': 10888}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'tcpSocket': {'port': 10888}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}, {'name': 'tailer', 'image': 'busybox', 'command': ['tail', '-F', '/var/log/ralf/ralf_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'ralflogs', 'mountPath': '/var/log/ralf'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'readyReplicas': 1, 'availableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'True', 'lastUpdateTime': '2020-08-28T15:59:28Z', 'lastTransitionTime': '2020-08-28T15:59:28Z', 'reason': 'MinimumReplicasAvailable', 'message': 'Deployment has minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-08-28T15:59:28Z', 'lastTransitionTime': '2020-08-28T15:56:53Z', 'reason': 'NewReplicaSetAvailable', 'message': 'ReplicaSet "ralf-545844cc6d" has successfully progressed.'}]}}} 2020-08-28 15:59:29,143 - functest_kubernetes.ims.ims - INFO - ralf started in 161.00 sec 2020-08-28 15:59:31,347 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 50, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': {'service': 'chronos'}, 'managed_fields': None, 'name': 'chronos', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801845', 'self_link': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/chronos', 'uid': '8911bd1e-c40e-4013-af3e-2e070733afb4'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'chronos'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'chronos'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-chronos:latest', 'image_pull_policy': 'Always', 'lifecycle': {'post_start': None, 'pre_stop': {'_exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}, 'http_get': None, 'tcp_socket': None}}, 'liveness_probe': {'_exec': None, 'failure_threshold': 9, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7253}, 'timeout_seconds': 1}, 'name': 'chronos', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 7253, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7253}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/chronos', 'mount_propagation': None, 'name': 'chronoslogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/chronos/chronos_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/chronos', 'mount_propagation': None, 'name': 'chronoslogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'cassandra; ' 'do echo ' 'waiting for ' 'cassandra to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 120, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'chronoslogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': 1, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 8, 28, 15, 59, 31, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 15, 59, 31, tzinfo=tzlocal()), 'message': 'Deployment has minimum availability.', 'reason': 'MinimumReplicasAvailable', 'status': 'True', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 50, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 15, 59, 31, tzinfo=tzlocal()), 'message': 'ReplicaSet "chronos-c55b4b9ff" has ' 'successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': 1, 'replicas': 1, 'unavailable_replicas': None, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'chronos', 'namespace': 'ims-gc7hd', 'selfLink': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/chronos', 'uid': '8911bd1e-c40e-4013-af3e-2e070733afb4', 'resourceVersion': '1801845', 'generation': 1, 'creationTimestamp': '2020-08-28T15:56:50Z', 'labels': {'service': 'chronos'}, 'annotations': {'deployment.kubernetes.io/revision': '1'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'chronos'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'chronos'}}, 'spec': {'volumes': [{'name': 'chronoslogs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'chronos', 'image': 'ollivier/clearwater-chronos:latest', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 7253, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'chronoslogs', 'mountPath': '/var/log/chronos'}], 'livenessProbe': {'tcpSocket': {'port': 7253}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 9}, 'readinessProbe': {'tcpSocket': {'port': 7253}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'lifecycle': {'preStop': {'exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}}}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}, {'name': 'tailer', 'image': 'busybox', 'command': ['tail', '-F', '/var/log/chronos/chronos_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'chronoslogs', 'mountPath': '/var/log/chronos'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 120, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'readyReplicas': 1, 'availableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'True', 'lastUpdateTime': '2020-08-28T15:59:31Z', 'lastTransitionTime': '2020-08-28T15:59:31Z', 'reason': 'MinimumReplicasAvailable', 'message': 'Deployment has minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-08-28T15:59:31Z', 'lastTransitionTime': '2020-08-28T15:56:50Z', 'reason': 'NewReplicaSetAvailable', 'message': 'ReplicaSet "chronos-c55b4b9ff" has successfully progressed.'}]}}} 2020-08-28 15:59:31,597 - functest_kubernetes.ims.ims - INFO - chronos started in 163.46 sec 2020-08-28 16:00:09,103 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 48, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': None, 'managed_fields': None, 'name': 'astaire', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1801973', 'self_link': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/astaire', 'uid': 'a586a36b-c53d-4586-be2a-1620364a644b'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'astaire'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'astaire'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-astaire:latest', 'image_pull_policy': 'Always', 'lifecycle': {'post_start': None, 'pre_stop': {'_exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}, 'http_get': None, 'tcp_socket': None}}, 'liveness_probe': {'_exec': None, 'failure_threshold': 9, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 11311}, 'timeout_seconds': 1}, 'name': 'astaire', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 11211, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 11311, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 11311}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/astaire', 'mount_propagation': None, 'name': 'astairelogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/astaire/astaire_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/astaire', 'mount_propagation': None, 'name': 'astairelogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'cassandra; ' 'do echo ' 'waiting for ' 'cassandra to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 120, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'astairelogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': 1, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 8, 28, 16, 0, 9, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 16, 0, 9, tzinfo=tzlocal()), 'message': 'Deployment has minimum availability.', 'reason': 'MinimumReplicasAvailable', 'status': 'True', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 49, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 16, 0, 9, tzinfo=tzlocal()), 'message': 'ReplicaSet "astaire-7666ff5885" has ' 'successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': 1, 'replicas': 1, 'unavailable_replicas': None, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'astaire', 'namespace': 'ims-gc7hd', 'selfLink': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/astaire', 'uid': 'a586a36b-c53d-4586-be2a-1620364a644b', 'resourceVersion': '1801973', 'generation': 1, 'creationTimestamp': '2020-08-28T15:56:48Z', 'annotations': {'deployment.kubernetes.io/revision': '1'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'astaire'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'astaire'}}, 'spec': {'volumes': [{'name': 'astairelogs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'astaire', 'image': 'ollivier/clearwater-astaire:latest', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 11211, 'protocol': 'TCP'}, {'containerPort': 11311, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'astairelogs', 'mountPath': '/var/log/astaire'}], 'livenessProbe': {'tcpSocket': {'port': 11311}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 9}, 'readinessProbe': {'tcpSocket': {'port': 11311}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'lifecycle': {'preStop': {'exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}}}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}, {'name': 'tailer', 'image': 'busybox', 'command': ['tail', '-F', '/var/log/astaire/astaire_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'astairelogs', 'mountPath': '/var/log/astaire'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 120, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'readyReplicas': 1, 'availableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'True', 'lastUpdateTime': '2020-08-28T16:00:09Z', 'lastTransitionTime': '2020-08-28T16:00:09Z', 'reason': 'MinimumReplicasAvailable', 'message': 'Deployment has minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-08-28T16:00:09Z', 'lastTransitionTime': '2020-08-28T15:56:49Z', 'reason': 'NewReplicaSetAvailable', 'message': 'ReplicaSet "astaire-7666ff5885" has successfully progressed.'}]}}} 2020-08-28 16:00:09,358 - functest_kubernetes.ims.ims - INFO - astaire started in 201.22 sec 2020-08-28 16:00:34,922 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 51, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': None, 'managed_fields': None, 'name': 'homestead', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1802058', 'self_link': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/homestead', 'uid': '84ae3654-a60b-43f4-9ba3-8550a7a2c87f'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'homestead'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'homestead', 'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-homestead:latest', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 60, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'homestead', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 8888, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/homestead', 'mount_propagation': None, 'name': 'homesteadlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/homestead/homestead_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/homestead', 'mount_propagation': None, 'name': 'homesteadlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'cassandra; ' 'do echo ' 'waiting for ' 'cassandra to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'astaire; do ' 'echo waiting ' 'for astaire ' 'to start ' '...; sleep ' '2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-astaire', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'homesteadlogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': 1, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 8, 28, 16, 0, 34, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 16, 0, 34, tzinfo=tzlocal()), 'message': 'Deployment has minimum availability.', 'reason': 'MinimumReplicasAvailable', 'status': 'True', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 52, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 16, 0, 34, tzinfo=tzlocal()), 'message': 'ReplicaSet "homestead-7dbc69fbf9" has ' 'successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': 1, 'replicas': 1, 'unavailable_replicas': None, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'homestead', 'namespace': 'ims-gc7hd', 'selfLink': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/homestead', 'uid': '84ae3654-a60b-43f4-9ba3-8550a7a2c87f', 'resourceVersion': '1802058', 'generation': 1, 'creationTimestamp': '2020-08-28T15:56:51Z', 'annotations': {'deployment.kubernetes.io/revision': '1'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'homestead'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'homestead', 'snmp': 'enabled'}}, 'spec': {'volumes': [{'name': 'homesteadlogs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'wait-on-astaire', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup astaire; do echo waiting for astaire to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'homestead', 'image': 'ollivier/clearwater-homestead:latest', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 8888, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'homesteadlogs', 'mountPath': '/var/log/homestead'}], 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'initialDelaySeconds': 60, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}, {'name': 'tailer', 'image': 'busybox', 'command': ['tail', '-F', '/var/log/homestead/homestead_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'homesteadlogs', 'mountPath': '/var/log/homestead'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'readyReplicas': 1, 'availableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'True', 'lastUpdateTime': '2020-08-28T16:00:34Z', 'lastTransitionTime': '2020-08-28T16:00:34Z', 'reason': 'MinimumReplicasAvailable', 'message': 'Deployment has minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-08-28T16:00:34Z', 'lastTransitionTime': '2020-08-28T15:56:52Z', 'reason': 'NewReplicaSetAvailable', 'message': 'ReplicaSet "homestead-7dbc69fbf9" has successfully progressed.'}]}}} 2020-08-28 16:00:35,196 - functest_kubernetes.ims.ims - INFO - homestead started in 227.06 sec 2020-08-28 16:00:52,301 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 53, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': None, 'managed_fields': None, 'name': 'sprout', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1802127', 'self_link': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/sprout', 'uid': 'b3390d47-04d1-4afc-9a3e-fc9de582b4fc'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'sprout'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'sprout', 'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-sprout:latest', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 ' '5054']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 3, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'sprout', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 ' '5054']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/sprout', 'mount_propagation': None, 'name': 'sproutlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/sprout/sprout_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/sprout', 'mount_propagation': None, 'name': 'sproutlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'ralf; do ' 'echo waiting ' 'for ralf to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-ralf', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'chronos; do ' 'echo waiting ' 'for chronos ' 'to start ' '...; sleep ' '2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-chronos', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'homestead; ' 'do echo ' 'waiting for ' 'homestead to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-homestead', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'sproutlogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': 1, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 8, 28, 16, 0, 52, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 16, 0, 52, tzinfo=tzlocal()), 'message': 'Deployment has minimum availability.', 'reason': 'MinimumReplicasAvailable', 'status': 'True', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 53, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 16, 0, 52, tzinfo=tzlocal()), 'message': 'ReplicaSet "sprout-669689d654" has ' 'successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': 1, 'replicas': 1, 'unavailable_replicas': None, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'sprout', 'namespace': 'ims-gc7hd', 'selfLink': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/sprout', 'uid': 'b3390d47-04d1-4afc-9a3e-fc9de582b4fc', 'resourceVersion': '1802127', 'generation': 1, 'creationTimestamp': '2020-08-28T15:56:53Z', 'annotations': {'deployment.kubernetes.io/revision': '1'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'sprout'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'sprout', 'snmp': 'enabled'}}, 'spec': {'volumes': [{'name': 'sproutlogs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-ralf', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup ralf; do echo waiting for ralf to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'wait-on-chronos', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup chronos; do echo waiting for chronos to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'wait-on-homestead', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup homestead; do echo waiting for homestead to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'sprout', 'image': 'ollivier/clearwater-sprout:latest', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'sproutlogs', 'mountPath': '/var/log/sprout'}], 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 5054']}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 3, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 5054']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}, {'name': 'tailer', 'image': 'busybox', 'command': ['tail', '-F', '/var/log/sprout/sprout_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'sproutlogs', 'mountPath': '/var/log/sprout'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'readyReplicas': 1, 'availableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'True', 'lastUpdateTime': '2020-08-28T16:00:52Z', 'lastTransitionTime': '2020-08-28T16:00:52Z', 'reason': 'MinimumReplicasAvailable', 'message': 'Deployment has minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-08-28T16:00:52Z', 'lastTransitionTime': '2020-08-28T15:56:53Z', 'reason': 'NewReplicaSetAvailable', 'message': 'ReplicaSet "sprout-669689d654" has successfully progressed.'}]}}} 2020-08-28 16:00:52,585 - functest_kubernetes.ims.ims - INFO - sprout started in 244.44 sec 2020-08-28 16:00:59,780 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 52, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': None, 'managed_fields': None, 'name': 'homestead-prov', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1802155', 'self_link': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/homestead-prov', 'uid': 'e8b84542-2695-4e30-a7a1-42b0e16de3fe'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'homestead-prov'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'homestead-prov', 'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-homestead-prov:latest', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 60, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'homestead-prov', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 8889, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'homestead; ' 'do echo ' 'waiting for ' 'homestead to ' 'start ...; ' 'sleep 2; ' 'done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-homestead', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': 1, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 8, 28, 16, 0, 59, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 16, 0, 59, tzinfo=tzlocal()), 'message': 'Deployment has minimum availability.', 'reason': 'MinimumReplicasAvailable', 'status': 'True', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 52, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 16, 0, 59, tzinfo=tzlocal()), 'message': 'ReplicaSet "homestead-prov-555fbf67b" ' 'has successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': 1, 'replicas': 1, 'unavailable_replicas': None, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'homestead-prov', 'namespace': 'ims-gc7hd', 'selfLink': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/homestead-prov', 'uid': 'e8b84542-2695-4e30-a7a1-42b0e16de3fe', 'resourceVersion': '1802155', 'generation': 1, 'creationTimestamp': '2020-08-28T15:56:52Z', 'annotations': {'deployment.kubernetes.io/revision': '1'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'homestead-prov'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'homestead-prov', 'snmp': 'enabled'}}, 'spec': {'initContainers': [{'name': 'wait-on-homestead', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup homestead; do echo waiting for homestead to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'homestead-prov', 'image': 'ollivier/clearwater-homestead-prov:latest', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 8889, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'initialDelaySeconds': 60, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'readyReplicas': 1, 'availableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'True', 'lastUpdateTime': '2020-08-28T16:00:59Z', 'lastTransitionTime': '2020-08-28T16:00:59Z', 'reason': 'MinimumReplicasAvailable', 'message': 'Deployment has minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-08-28T16:00:59Z', 'lastTransitionTime': '2020-08-28T15:56:52Z', 'reason': 'NewReplicaSetAvailable', 'message': 'ReplicaSet "homestead-prov-555fbf67b" has successfully progressed.'}]}}} 2020-08-28 16:00:59,955 - functest_kubernetes.ims.ims - INFO - homestead-prov started in 251.81 sec 2020-08-28 16:01:36,631 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 49, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': None, 'managed_fields': None, 'name': 'bono', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1802272', 'self_link': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/bono', 'uid': 'd6bc69e0-8471-4b52-ae66-fd1f8e8bf787'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'bono'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'bono', 'snmp': 'enabled'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-bono:latest', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 ' '5060 ' '5062']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'bono', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 3478, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 5060, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 5062, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 5060, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'UDP'}, {'container_port': 5062, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'UDP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 ' '5060 ' '5062']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/bono', 'mount_propagation': None, 'name': 'bonologs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/bono/bono_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/bono', 'mount_propagation': None, 'name': 'bonologs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'sprout; do ' 'echo waiting ' 'for sprout ' 'to start ' '...; sleep ' '2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-sprout', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'bonologs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}]}}}, 'status': {'available_replicas': 1, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 8, 28, 16, 1, 36, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 16, 1, 36, tzinfo=tzlocal()), 'message': 'Deployment has minimum availability.', 'reason': 'MinimumReplicasAvailable', 'status': 'True', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 49, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 16, 1, 36, tzinfo=tzlocal()), 'message': 'ReplicaSet "bono-69fbbd8646" has ' 'successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': 1, 'replicas': 1, 'unavailable_replicas': None, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'bono', 'namespace': 'ims-gc7hd', 'selfLink': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/bono', 'uid': 'd6bc69e0-8471-4b52-ae66-fd1f8e8bf787', 'resourceVersion': '1802272', 'generation': 1, 'creationTimestamp': '2020-08-28T15:56:49Z', 'annotations': {'deployment.kubernetes.io/revision': '1'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'bono'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'bono', 'snmp': 'enabled'}}, 'spec': {'volumes': [{'name': 'bonologs', 'emptyDir': {}}], 'initContainers': [{'name': 'wait-on-sprout', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup sprout; do echo waiting for sprout to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'bono', 'image': 'ollivier/clearwater-bono:latest', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 3478, 'protocol': 'TCP'}, {'containerPort': 5060, 'protocol': 'TCP'}, {'containerPort': 5062, 'protocol': 'TCP'}, {'containerPort': 5060, 'protocol': 'UDP'}, {'containerPort': 5062, 'protocol': 'UDP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'bonologs', 'mountPath': '/var/log/bono'}], 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 5060 5062']}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 5060 5062']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}, {'name': 'tailer', 'image': 'busybox', 'command': ['tail', '-F', '/var/log/bono/bono_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'bonologs', 'mountPath': '/var/log/bono'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'readyReplicas': 1, 'availableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'True', 'lastUpdateTime': '2020-08-28T16:01:36Z', 'lastTransitionTime': '2020-08-28T16:01:36Z', 'reason': 'MinimumReplicasAvailable', 'message': 'Deployment has minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-08-28T16:01:36Z', 'lastTransitionTime': '2020-08-28T15:56:49Z', 'reason': 'NewReplicaSetAvailable', 'message': 'ReplicaSet "bono-69fbbd8646" has successfully progressed.'}]}}} 2020-08-28 16:01:36,907 - functest_kubernetes.ims.ims - INFO - bono started in 288.77 sec 2020-08-28 16:01:42,709 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'deployment.kubernetes.io/revision': '1'}, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 50, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': 1, 'initializers': None, 'labels': None, 'managed_fields': None, 'name': 'ellis', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1802295', 'self_link': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/ellis', 'uid': '9d4959be-6eec-4d20-a139-c33010481c17'}, 'spec': {'min_ready_seconds': None, 'paused': None, 'progress_deadline_seconds': 600, 'replicas': 1, 'revision_history_limit': 10, 'selector': {'match_expressions': None, 'match_labels': {'service': 'ellis'}}, 'strategy': {'rolling_update': {'max_surge': '25%', 'max_unavailable': '25%'}, 'type': 'RollingUpdate'}, 'template': {'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': None, 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': {'service': 'ellis'}, 'managed_fields': None, 'name': None, 'namespace': None, 'owner_references': None, 'resource_version': None, 'self_link': None, 'uid': None}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-ellis:latest', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 80}, 'timeout_seconds': 1}, 'name': 'ellis', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 80, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 80}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': None, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'homestead-prov; ' 'do echo ' 'waiting for ' 'homestead-prov ' 'to start ' '...; sleep ' '2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-homestead-prov', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until ' 'nslookup ' 'sprout; do ' 'echo waiting ' 'for sprout ' 'to start ' '...; sleep ' '2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-sprout', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': None, 'working_dir': None}], 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': None, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': None, 'service_account_name': None, 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': None, 'volumes': None}}}, 'status': {'available_replicas': 1, 'collision_count': None, 'conditions': [{'last_transition_time': datetime.datetime(2020, 8, 28, 16, 1, 42, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 16, 1, 42, tzinfo=tzlocal()), 'message': 'Deployment has minimum availability.', 'reason': 'MinimumReplicasAvailable', 'status': 'True', 'type': 'Available'}, {'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 50, tzinfo=tzlocal()), 'last_update_time': datetime.datetime(2020, 8, 28, 16, 1, 42, tzinfo=tzlocal()), 'message': 'ReplicaSet "ellis-5dfd856d6c" has ' 'successfully progressed.', 'reason': 'NewReplicaSetAvailable', 'status': 'True', 'type': 'Progressing'}], 'observed_generation': 1, 'ready_replicas': 1, 'replicas': 1, 'unavailable_replicas': None, 'updated_replicas': 1}}, 'raw_object': {'kind': 'Deployment', 'apiVersion': 'apps/v1', 'metadata': {'name': 'ellis', 'namespace': 'ims-gc7hd', 'selfLink': '/apis/apps/v1/namespaces/ims-gc7hd/deployments/ellis', 'uid': '9d4959be-6eec-4d20-a139-c33010481c17', 'resourceVersion': '1802295', 'generation': 1, 'creationTimestamp': '2020-08-28T15:56:50Z', 'annotations': {'deployment.kubernetes.io/revision': '1'}}, 'spec': {'replicas': 1, 'selector': {'matchLabels': {'service': 'ellis'}}, 'template': {'metadata': {'creationTimestamp': None, 'labels': {'service': 'ellis'}}, 'spec': {'initContainers': [{'name': 'wait-on-homestead-prov', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup homestead-prov; do echo waiting for homestead-prov to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'wait-on-sprout', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup sprout; do echo waiting for sprout to start ...; sleep 2; done'], 'resources': {}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'ellis', 'image': 'ollivier/clearwater-ellis:latest', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 80, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'livenessProbe': {'tcpSocket': {'port': 80}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'tcpSocket': {'port': 80}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler'}}, 'strategy': {'type': 'RollingUpdate', 'rollingUpdate': {'maxUnavailable': '25%', 'maxSurge': '25%'}}, 'revisionHistoryLimit': 10, 'progressDeadlineSeconds': 600}, 'status': {'observedGeneration': 1, 'replicas': 1, 'updatedReplicas': 1, 'readyReplicas': 1, 'availableReplicas': 1, 'conditions': [{'type': 'Available', 'status': 'True', 'lastUpdateTime': '2020-08-28T16:01:42Z', 'lastTransitionTime': '2020-08-28T16:01:42Z', 'reason': 'MinimumReplicasAvailable', 'message': 'Deployment has minimum availability.'}, {'type': 'Progressing', 'status': 'True', 'lastUpdateTime': '2020-08-28T16:01:42Z', 'lastTransitionTime': '2020-08-28T15:56:50Z', 'reason': 'NewReplicaSetAvailable', 'message': 'ReplicaSet "ellis-5dfd856d6c" has successfully progressed.'}]}}} 2020-08-28 16:01:42,904 - functest_kubernetes.ims.ims - INFO - ellis started in 294.76 sec 2020-08-28 16:02:42,985 - kubernetes.client.rest - DEBUG - response body: {"kind":"Pod","apiVersion":"v1","metadata":{"name":"live-test","namespace":"ims-gc7hd","selfLink":"/api/v1/namespaces/ims-gc7hd/pods/live-test","uid":"bebc59b7-ad08-4316-b33c-15b0a8b7cf0b","resourceVersion":"1802468","creationTimestamp":"2020-08-28T16:02:42Z","managedFields":[{"manager":"OpenAPI-Generator","operation":"Update","apiVersion":"v1","time":"2020-08-28T16:02:42Z","fieldsType":"FieldsV1","fieldsV1":{"f:spec":{"f:containers":{"k:{\"name\":\"live-test\"}":{".":{},"f:command":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{},"f:terminationGracePeriodSeconds":{}}}}]},"spec":{"volumes":[{"name":"default-token-52qls","secret":{"secretName":"default-token-52qls","defaultMode":420}}],"containers":[{"name":"live-test","image":"ollivier/clearwater-live-test:latest","command":["rake","test[ims-gc7hd.svc.cluster.local]","PROXY=bono.ims-gc7hd.svc.cluster.local","ELLIS=ellis.ims-gc7hd.svc.cluster.local","SIGNUP_CODE=secret","--trace"],"resources":{},"volumeMounts":[{"name":"default-token-52qls","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always"}],"restartPolicy":"Never","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}],"priority":0,"enableServiceLinks":true},"status":{"phase":"Pending","qosClass":"BestEffort"}} 2020-08-28 16:02:43,040 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 52, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'homestead-prov-555fbf67b-', 'generation': None, 'initializers': None, 'labels': {'pod-template-hash': '555fbf67b', 'service': 'homestead-prov', 'snmp': 'enabled'}, 'managed_fields': None, 'name': 'homestead-prov-555fbf67b-lf6z7', 'namespace': 'ims-gc7hd', 'owner_references': [{'api_version': 'apps/v1', 'block_owner_deletion': True, 'controller': True, 'kind': 'ReplicaSet', 'name': 'homestead-prov-555fbf67b', 'uid': '7da2db74-0dbf-4c55-9aec-2b4446fae69c'}], 'resource_version': '1802151', 'self_link': '/api/v1/namespaces/ims-gc7hd/pods/homestead-prov-555fbf67b-lf6z7', 'uid': 'c4faf57a-668d-4740-b21c-b1d35d6f5f6e'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-homestead-prov:latest', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 60, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'homestead-prov', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 8889, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup homestead; do echo ' 'waiting for homestead to start ' '...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-homestead', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'node_name': 'kali-worker', 'node_selector': None, 'preemption_policy': None, 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-52qls', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-52qls'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 16, 0, 47, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 16, 0, 59, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 16, 0, 59, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 52, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://e5f4e6f6b7bd7ff7da78cdf33069025ff5bd7eb048f82baf5938e752a294da42', 'image': 'docker.io/ollivier/clearwater-homestead-prov:latest', 'image_id': 'docker.io/ollivier/clearwater-homestead-prov@sha256:0b4d47a5161ecb6b44f6a479a27522b802096a2deea049cd6f3c01a62b585318', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'homestead-prov', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 8, 28, 16, 0, 51, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.15', 'init_container_statuses': [{'container_id': 'containerd://ffcfe52a72f011db92fff5e4b8e827997be41b25430eb2a3cfa1c507cf9d4bdd', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-homestead', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://ffcfe52a72f011db92fff5e4b8e827997be41b25430eb2a3cfa1c507cf9d4bdd', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 8, 28, 16, 0, 46, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 8, 28, 15, 57, 5, tzinfo=tzlocal())}, 'waiting': None}}], 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.1.30', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 8, 28, 15, 56, 53, tzinfo=tzlocal())}}, 'raw_object': {'kind': 'Pod', 'apiVersion': 'v1', 'metadata': {'name': 'homestead-prov-555fbf67b-lf6z7', 'generateName': 'homestead-prov-555fbf67b-', 'namespace': 'ims-gc7hd', 'selfLink': '/api/v1/namespaces/ims-gc7hd/pods/homestead-prov-555fbf67b-lf6z7', 'uid': 'c4faf57a-668d-4740-b21c-b1d35d6f5f6e', 'resourceVersion': '1802151', 'creationTimestamp': '2020-08-28T15:56:52Z', 'labels': {'pod-template-hash': '555fbf67b', 'service': 'homestead-prov', 'snmp': 'enabled'}, 'ownerReferences': [{'apiVersion': 'apps/v1', 'kind': 'ReplicaSet', 'name': 'homestead-prov-555fbf67b', 'uid': '7da2db74-0dbf-4c55-9aec-2b4446fae69c', 'controller': True, 'blockOwnerDeletion': True}]}, 'spec': {'volumes': [{'name': 'default-token-52qls', 'secret': {'secretName': 'default-token-52qls', 'defaultMode': 420}}], 'initContainers': [{'name': 'wait-on-homestead', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup homestead; do echo waiting for homestead to start ...; sleep 2; done'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'homestead-prov', 'image': 'ollivier/clearwater-homestead-prov:latest', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 8889, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'initialDelaySeconds': 60, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/clearwater/bin/poll_homestead-prov.sh']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'serviceAccountName': 'default', 'serviceAccount': 'default', 'nodeName': 'kali-worker', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler', 'tolerations': [{'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}, {'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}], 'priority': 0, 'enableServiceLinks': True}, 'status': {'phase': 'Running', 'conditions': [{'type': 'Initialized', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T16:00:47Z'}, {'type': 'Ready', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T16:00:59Z'}, {'type': 'ContainersReady', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T16:00:59Z'}, {'type': 'PodScheduled', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T15:56:52Z'}], 'hostIP': '172.18.0.15', 'podIP': '10.244.1.30', 'podIPs': [{'ip': '10.244.1.30'}], 'startTime': '2020-08-28T15:56:53Z', 'initContainerStatuses': [{'name': 'wait-on-homestead', 'state': {'terminated': {'exitCode': 0, 'reason': 'Completed', 'startedAt': '2020-08-28T15:57:05Z', 'finishedAt': '2020-08-28T16:00:46Z', 'containerID': 'containerd://ffcfe52a72f011db92fff5e4b8e827997be41b25430eb2a3cfa1c507cf9d4bdd'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:1.28', 'imageID': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'containerID': 'containerd://ffcfe52a72f011db92fff5e4b8e827997be41b25430eb2a3cfa1c507cf9d4bdd'}], 'containerStatuses': [{'name': 'homestead-prov', 'state': {'running': {'startedAt': '2020-08-28T16:00:51Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/ollivier/clearwater-homestead-prov:latest', 'imageID': 'docker.io/ollivier/clearwater-homestead-prov@sha256:0b4d47a5161ecb6b44f6a479a27522b802096a2deea049cd6f3c01a62b585318', 'containerID': 'containerd://e5f4e6f6b7bd7ff7da78cdf33069025ff5bd7eb048f82baf5938e752a294da42', 'started': True}], 'qosClass': 'BestEffort'}}} 2020-08-28 16:02:43,290 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 53, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'ralf-545844cc6d-', 'generation': None, 'initializers': None, 'labels': {'pod-template-hash': '545844cc6d', 'service': 'ralf', 'snmp': 'enabled'}, 'managed_fields': None, 'name': 'ralf-545844cc6d-hgddd', 'namespace': 'ims-gc7hd', 'owner_references': [{'api_version': 'apps/v1', 'block_owner_deletion': True, 'controller': True, 'kind': 'ReplicaSet', 'name': 'ralf-545844cc6d', 'uid': '054abc50-bd8f-4492-a3c3-abb66df74715'}], 'resource_version': '1801827', 'self_link': '/api/v1/namespaces/ims-gc7hd/pods/ralf-545844cc6d-hgddd', 'uid': '6311a53f-b09f-4936-9fb8-ba411a92a4f3'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-ralf:latest', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 10888}, 'timeout_seconds': 1}, 'name': 'ralf', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 10888, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 10888}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/ralf', 'mount_propagation': None, 'name': 'ralflogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/ralf/ralf_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/ralf', 'mount_propagation': None, 'name': 'ralflogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup cassandra; do echo ' 'waiting for cassandra to start ' '...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'node_name': 'kali-worker2', 'node_selector': None, 'preemption_policy': None, 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'ralflogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}, {'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-52qls', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-52qls'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 15, 59, 3, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 15, 59, 28, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 15, 59, 28, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 53, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://df806f9d75b8b560c442b089b36213a2b547095152275bfc041382543ae90a43', 'image': 'docker.io/ollivier/clearwater-ralf:latest', 'image_id': 'docker.io/ollivier/clearwater-ralf@sha256:adfa3978f2c94734010c014a2be7db9bc328419e0a205904543a86cd0719bd1a', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'ralf', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 8, 28, 15, 59, 9, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}, {'container_id': 'containerd://9224b0a871e064b75f348f08aeba7cfc04e7009100b5b83c2d8f783a0dc799d2', 'image': 'docker.io/library/busybox:latest', 'image_id': 'docker.io/library/busybox@sha256:4f47c01fa91355af2865ac10fef5bf6ec9c7f42ad2321377c21e844427972977', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'tailer', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 8, 28, 15, 59, 17, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.13', 'init_container_statuses': [{'container_id': 'containerd://b419f79a245e19abb654212cca14d3f11c976bcf19f4b8557a953efcc79af5e2', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-cassandra', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://b419f79a245e19abb654212cca14d3f11c976bcf19f4b8557a953efcc79af5e2', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 8, 28, 15, 59, 2, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 8, 28, 15, 57, 6, tzinfo=tzlocal())}, 'waiting': None}}], 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.2.7', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 8, 28, 15, 56, 54, tzinfo=tzlocal())}}, 'raw_object': {'kind': 'Pod', 'apiVersion': 'v1', 'metadata': {'name': 'ralf-545844cc6d-hgddd', 'generateName': 'ralf-545844cc6d-', 'namespace': 'ims-gc7hd', 'selfLink': '/api/v1/namespaces/ims-gc7hd/pods/ralf-545844cc6d-hgddd', 'uid': '6311a53f-b09f-4936-9fb8-ba411a92a4f3', 'resourceVersion': '1801827', 'creationTimestamp': '2020-08-28T15:56:53Z', 'labels': {'pod-template-hash': '545844cc6d', 'service': 'ralf', 'snmp': 'enabled'}, 'ownerReferences': [{'apiVersion': 'apps/v1', 'kind': 'ReplicaSet', 'name': 'ralf-545844cc6d', 'uid': '054abc50-bd8f-4492-a3c3-abb66df74715', 'controller': True, 'blockOwnerDeletion': True}]}, 'spec': {'volumes': [{'name': 'ralflogs', 'emptyDir': {}}, {'name': 'default-token-52qls', 'secret': {'secretName': 'default-token-52qls', 'defaultMode': 420}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'ralf', 'image': 'ollivier/clearwater-ralf:latest', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 10888, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'ralflogs', 'mountPath': '/var/log/ralf'}, {'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'livenessProbe': {'tcpSocket': {'port': 10888}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'tcpSocket': {'port': 10888}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}, {'name': 'tailer', 'image': 'busybox', 'command': ['tail', '-F', '/var/log/ralf/ralf_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'ralflogs', 'mountPath': '/var/log/ralf'}, {'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'serviceAccountName': 'default', 'serviceAccount': 'default', 'nodeName': 'kali-worker2', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler', 'tolerations': [{'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}, {'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}], 'priority': 0, 'enableServiceLinks': True}, 'status': {'phase': 'Running', 'conditions': [{'type': 'Initialized', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T15:59:03Z'}, {'type': 'Ready', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T15:59:28Z'}, {'type': 'ContainersReady', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T15:59:28Z'}, {'type': 'PodScheduled', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T15:56:53Z'}], 'hostIP': '172.18.0.13', 'podIP': '10.244.2.7', 'podIPs': [{'ip': '10.244.2.7'}], 'startTime': '2020-08-28T15:56:54Z', 'initContainerStatuses': [{'name': 'wait-on-cassandra', 'state': {'terminated': {'exitCode': 0, 'reason': 'Completed', 'startedAt': '2020-08-28T15:57:06Z', 'finishedAt': '2020-08-28T15:59:02Z', 'containerID': 'containerd://b419f79a245e19abb654212cca14d3f11c976bcf19f4b8557a953efcc79af5e2'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:1.28', 'imageID': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'containerID': 'containerd://b419f79a245e19abb654212cca14d3f11c976bcf19f4b8557a953efcc79af5e2'}], 'containerStatuses': [{'name': 'ralf', 'state': {'running': {'startedAt': '2020-08-28T15:59:09Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/ollivier/clearwater-ralf:latest', 'imageID': 'docker.io/ollivier/clearwater-ralf@sha256:adfa3978f2c94734010c014a2be7db9bc328419e0a205904543a86cd0719bd1a', 'containerID': 'containerd://df806f9d75b8b560c442b089b36213a2b547095152275bfc041382543ae90a43', 'started': True}, {'name': 'tailer', 'state': {'running': {'startedAt': '2020-08-28T15:59:17Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:latest', 'imageID': 'docker.io/library/busybox@sha256:4f47c01fa91355af2865ac10fef5bf6ec9c7f42ad2321377c21e844427972977', 'containerID': 'containerd://9224b0a871e064b75f348f08aeba7cfc04e7009100b5b83c2d8f783a0dc799d2', 'started': True}], 'qosClass': 'BestEffort'}}} 2020-08-28 16:02:43,615 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 50, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'ellis-5dfd856d6c-', 'generation': None, 'initializers': None, 'labels': {'pod-template-hash': '5dfd856d6c', 'service': 'ellis'}, 'managed_fields': None, 'name': 'ellis-5dfd856d6c-ppd88', 'namespace': 'ims-gc7hd', 'owner_references': [{'api_version': 'apps/v1', 'block_owner_deletion': True, 'controller': True, 'kind': 'ReplicaSet', 'name': 'ellis-5dfd856d6c', 'uid': '77d1d546-eff9-4c47-981c-0bbce21ee31b'}], 'resource_version': '1802291', 'self_link': '/api/v1/namespaces/ims-gc7hd/pods/ellis-5dfd856d6c-ppd88', 'uid': 'e0ca59ca-71af-4de8-bc7d-d6ed42d10462'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-ellis:latest', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 80}, 'timeout_seconds': 1}, 'name': 'ellis', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 80, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 80}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup homestead-prov; do ' 'echo waiting for homestead-prov to ' 'start ...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-homestead-prov', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until nslookup sprout; do echo ' 'waiting for sprout to start ...; ' 'sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-sprout', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'node_name': 'kali-worker2', 'node_selector': None, 'preemption_policy': None, 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-52qls', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-52qls'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 16, 1, 30, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 16, 1, 42, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 16, 1, 42, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 50, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://b9ac2a35151b57ed02ee3b39296d22749bd9e7a1289ff6e8a968b541a220445e', 'image': 'docker.io/ollivier/clearwater-ellis:latest', 'image_id': 'docker.io/ollivier/clearwater-ellis@sha256:28557b896e190c72f02121314ca7c9abaca30f91a733b566b2c44b761e5a252c', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'ellis', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 8, 28, 16, 1, 35, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.13', 'init_container_statuses': [{'container_id': 'containerd://340405c750fdeb28444bfd0cb789c5f795aafc87bf294769f549de9dd0238978', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-homestead-prov', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://340405c750fdeb28444bfd0cb789c5f795aafc87bf294769f549de9dd0238978', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 8, 28, 16, 1, 21, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 8, 28, 15, 57, 3, tzinfo=tzlocal())}, 'waiting': None}}, {'container_id': 'containerd://7d4ce54f81cb6b77a7610e932d8d35c46a398732d54ce1ff12ef17c3f341a148', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-sprout', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://7d4ce54f81cb6b77a7610e932d8d35c46a398732d54ce1ff12ef17c3f341a148', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 8, 28, 16, 1, 29, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 8, 28, 16, 1, 29, tzinfo=tzlocal())}, 'waiting': None}}], 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.2.4', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 8, 28, 15, 56, 51, tzinfo=tzlocal())}}, 'raw_object': {'kind': 'Pod', 'apiVersion': 'v1', 'metadata': {'name': 'ellis-5dfd856d6c-ppd88', 'generateName': 'ellis-5dfd856d6c-', 'namespace': 'ims-gc7hd', 'selfLink': '/api/v1/namespaces/ims-gc7hd/pods/ellis-5dfd856d6c-ppd88', 'uid': 'e0ca59ca-71af-4de8-bc7d-d6ed42d10462', 'resourceVersion': '1802291', 'creationTimestamp': '2020-08-28T15:56:50Z', 'labels': {'pod-template-hash': '5dfd856d6c', 'service': 'ellis'}, 'ownerReferences': [{'apiVersion': 'apps/v1', 'kind': 'ReplicaSet', 'name': 'ellis-5dfd856d6c', 'uid': '77d1d546-eff9-4c47-981c-0bbce21ee31b', 'controller': True, 'blockOwnerDeletion': True}]}, 'spec': {'volumes': [{'name': 'default-token-52qls', 'secret': {'secretName': 'default-token-52qls', 'defaultMode': 420}}], 'initContainers': [{'name': 'wait-on-homestead-prov', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup homestead-prov; do echo waiting for homestead-prov to start ...; sleep 2; done'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'wait-on-sprout', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup sprout; do echo waiting for sprout to start ...; sleep 2; done'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'ellis', 'image': 'ollivier/clearwater-ellis:latest', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 80, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'livenessProbe': {'tcpSocket': {'port': 80}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'tcpSocket': {'port': 80}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'serviceAccountName': 'default', 'serviceAccount': 'default', 'nodeName': 'kali-worker2', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler', 'tolerations': [{'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}, {'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}], 'priority': 0, 'enableServiceLinks': True}, 'status': {'phase': 'Running', 'conditions': [{'type': 'Initialized', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T16:01:30Z'}, {'type': 'Ready', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T16:01:42Z'}, {'type': 'ContainersReady', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T16:01:42Z'}, {'type': 'PodScheduled', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T15:56:50Z'}], 'hostIP': '172.18.0.13', 'podIP': '10.244.2.4', 'podIPs': [{'ip': '10.244.2.4'}], 'startTime': '2020-08-28T15:56:51Z', 'initContainerStatuses': [{'name': 'wait-on-homestead-prov', 'state': {'terminated': {'exitCode': 0, 'reason': 'Completed', 'startedAt': '2020-08-28T15:57:03Z', 'finishedAt': '2020-08-28T16:01:21Z', 'containerID': 'containerd://340405c750fdeb28444bfd0cb789c5f795aafc87bf294769f549de9dd0238978'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:1.28', 'imageID': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'containerID': 'containerd://340405c750fdeb28444bfd0cb789c5f795aafc87bf294769f549de9dd0238978'}, {'name': 'wait-on-sprout', 'state': {'terminated': {'exitCode': 0, 'reason': 'Completed', 'startedAt': '2020-08-28T16:01:29Z', 'finishedAt': '2020-08-28T16:01:29Z', 'containerID': 'containerd://7d4ce54f81cb6b77a7610e932d8d35c46a398732d54ce1ff12ef17c3f341a148'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:1.28', 'imageID': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'containerID': 'containerd://7d4ce54f81cb6b77a7610e932d8d35c46a398732d54ce1ff12ef17c3f341a148'}], 'containerStatuses': [{'name': 'ellis', 'state': {'running': {'startedAt': '2020-08-28T16:01:35Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/ollivier/clearwater-ellis:latest', 'imageID': 'docker.io/ollivier/clearwater-ellis@sha256:28557b896e190c72f02121314ca7c9abaca30f91a733b566b2c44b761e5a252c', 'containerID': 'containerd://b9ac2a35151b57ed02ee3b39296d22749bd9e7a1289ff6e8a968b541a220445e', 'started': True}], 'qosClass': 'BestEffort'}}} 2020-08-28 16:02:43,913 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 52, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'homestead-7dbc69fbf9-', 'generation': None, 'initializers': None, 'labels': {'pod-template-hash': '7dbc69fbf9', 'service': 'homestead', 'snmp': 'enabled'}, 'managed_fields': None, 'name': 'homestead-7dbc69fbf9-2dp4g', 'namespace': 'ims-gc7hd', 'owner_references': [{'api_version': 'apps/v1', 'block_owner_deletion': True, 'controller': True, 'kind': 'ReplicaSet', 'name': 'homestead-7dbc69fbf9', 'uid': 'dca9ff04-dc2f-4929-a7c6-8c60247ec239'}], 'resource_version': '1802054', 'self_link': '/api/v1/namespaces/ims-gc7hd/pods/homestead-7dbc69fbf9-2dp4g', 'uid': '96c10ac3-990e-462c-8da3-4c84407aa3a4'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-homestead:latest', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 60, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'homestead', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 8888, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/homestead', 'mount_propagation': None, 'name': 'homesteadlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/homestead/homestead_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/homestead', 'mount_propagation': None, 'name': 'homesteadlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup cassandra; do echo ' 'waiting for cassandra to start ' '...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until nslookup astaire; do echo ' 'waiting for astaire to start ...; ' 'sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-astaire', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'node_name': 'kali-worker2', 'node_selector': None, 'preemption_policy': None, 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'homesteadlogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}, {'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-52qls', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-52qls'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 16, 0, 22, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 16, 0, 34, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 16, 0, 34, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 52, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://07d7c1d01bec05db37ff383ff40b8672e4804a2111a136c94f9529e552f1fe60', 'image': 'docker.io/ollivier/clearwater-homestead:latest', 'image_id': 'docker.io/ollivier/clearwater-homestead@sha256:98347f9bf0eaf79649590e3fa0ea8d1938ae50d7703e8f9c171f0d74520ac7fb', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'homestead', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 8, 28, 16, 0, 27, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}, {'container_id': 'containerd://bb8c46863176c1235b1e9a57c5b1486cdca5141c325ae69809db43c421357fed', 'image': 'docker.io/library/busybox:latest', 'image_id': 'docker.io/library/busybox@sha256:4f47c01fa91355af2865ac10fef5bf6ec9c7f42ad2321377c21e844427972977', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'tailer', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 8, 28, 16, 0, 30, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.13', 'init_container_statuses': [{'container_id': 'containerd://2bb8a096682bea66b7a2e55946c1ccd2844037e867d88dab68f4a59bfc6b1651', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-cassandra', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://2bb8a096682bea66b7a2e55946c1ccd2844037e867d88dab68f4a59bfc6b1651', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 8, 28, 15, 59, 2, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 8, 28, 15, 57, 6, tzinfo=tzlocal())}, 'waiting': None}}, {'container_id': 'containerd://40425f819e62b8b27c701aa68bef483db308d5df8423583b3680d468706acab9', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-astaire', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://40425f819e62b8b27c701aa68bef483db308d5df8423583b3680d468706acab9', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 8, 28, 16, 0, 21, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 8, 28, 15, 59, 9, tzinfo=tzlocal())}, 'waiting': None}}], 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.2.6', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 8, 28, 15, 56, 53, tzinfo=tzlocal())}}, 'raw_object': {'kind': 'Pod', 'apiVersion': 'v1', 'metadata': {'name': 'homestead-7dbc69fbf9-2dp4g', 'generateName': 'homestead-7dbc69fbf9-', 'namespace': 'ims-gc7hd', 'selfLink': '/api/v1/namespaces/ims-gc7hd/pods/homestead-7dbc69fbf9-2dp4g', 'uid': '96c10ac3-990e-462c-8da3-4c84407aa3a4', 'resourceVersion': '1802054', 'creationTimestamp': '2020-08-28T15:56:52Z', 'labels': {'pod-template-hash': '7dbc69fbf9', 'service': 'homestead', 'snmp': 'enabled'}, 'ownerReferences': [{'apiVersion': 'apps/v1', 'kind': 'ReplicaSet', 'name': 'homestead-7dbc69fbf9', 'uid': 'dca9ff04-dc2f-4929-a7c6-8c60247ec239', 'controller': True, 'blockOwnerDeletion': True}]}, 'spec': {'volumes': [{'name': 'homesteadlogs', 'emptyDir': {}}, {'name': 'default-token-52qls', 'secret': {'secretName': 'default-token-52qls', 'defaultMode': 420}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'wait-on-astaire', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup astaire; do echo waiting for astaire to start ...; sleep 2; done'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'homestead', 'image': 'ollivier/clearwater-homestead:latest', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 8888, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'homesteadlogs', 'mountPath': '/var/log/homestead'}, {'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'initialDelaySeconds': 60, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '8888']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}, {'name': 'tailer', 'image': 'busybox', 'command': ['tail', '-F', '/var/log/homestead/homestead_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'homesteadlogs', 'mountPath': '/var/log/homestead'}, {'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'serviceAccountName': 'default', 'serviceAccount': 'default', 'nodeName': 'kali-worker2', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler', 'tolerations': [{'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}, {'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}], 'priority': 0, 'enableServiceLinks': True}, 'status': {'phase': 'Running', 'conditions': [{'type': 'Initialized', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T16:00:22Z'}, {'type': 'Ready', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T16:00:34Z'}, {'type': 'ContainersReady', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T16:00:34Z'}, {'type': 'PodScheduled', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T15:56:52Z'}], 'hostIP': '172.18.0.13', 'podIP': '10.244.2.6', 'podIPs': [{'ip': '10.244.2.6'}], 'startTime': '2020-08-28T15:56:53Z', 'initContainerStatuses': [{'name': 'wait-on-cassandra', 'state': {'terminated': {'exitCode': 0, 'reason': 'Completed', 'startedAt': '2020-08-28T15:57:06Z', 'finishedAt': '2020-08-28T15:59:02Z', 'containerID': 'containerd://2bb8a096682bea66b7a2e55946c1ccd2844037e867d88dab68f4a59bfc6b1651'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:1.28', 'imageID': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'containerID': 'containerd://2bb8a096682bea66b7a2e55946c1ccd2844037e867d88dab68f4a59bfc6b1651'}, {'name': 'wait-on-astaire', 'state': {'terminated': {'exitCode': 0, 'reason': 'Completed', 'startedAt': '2020-08-28T15:59:09Z', 'finishedAt': '2020-08-28T16:00:21Z', 'containerID': 'containerd://40425f819e62b8b27c701aa68bef483db308d5df8423583b3680d468706acab9'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:1.28', 'imageID': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'containerID': 'containerd://40425f819e62b8b27c701aa68bef483db308d5df8423583b3680d468706acab9'}], 'containerStatuses': [{'name': 'homestead', 'state': {'running': {'startedAt': '2020-08-28T16:00:27Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/ollivier/clearwater-homestead:latest', 'imageID': 'docker.io/ollivier/clearwater-homestead@sha256:98347f9bf0eaf79649590e3fa0ea8d1938ae50d7703e8f9c171f0d74520ac7fb', 'containerID': 'containerd://07d7c1d01bec05db37ff383ff40b8672e4804a2111a136c94f9529e552f1fe60', 'started': True}, {'name': 'tailer', 'state': {'running': {'startedAt': '2020-08-28T16:00:30Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:latest', 'imageID': 'docker.io/library/busybox@sha256:4f47c01fa91355af2865ac10fef5bf6ec9c7f42ad2321377c21e844427972977', 'containerID': 'containerd://bb8c46863176c1235b1e9a57c5b1486cdca5141c325ae69809db43c421357fed', 'started': True}], 'qosClass': 'BestEffort'}}} 2020-08-28 16:02:44,269 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 53, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'sprout-669689d654-', 'generation': None, 'initializers': None, 'labels': {'pod-template-hash': '669689d654', 'service': 'sprout', 'snmp': 'enabled'}, 'managed_fields': None, 'name': 'sprout-669689d654-46vtm', 'namespace': 'ims-gc7hd', 'owner_references': [{'api_version': 'apps/v1', 'block_owner_deletion': True, 'controller': True, 'kind': 'ReplicaSet', 'name': 'sprout-669689d654', 'uid': '15418e1c-0c46-4357-957a-3f671989d24d'}], 'resource_version': '1802123', 'self_link': '/api/v1/namespaces/ims-gc7hd/pods/sprout-669689d654-46vtm', 'uid': 'b99e79ee-7c38-4c31-9ddd-537252f70d65'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-sprout:latest', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 ' '5054']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 3, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'sprout', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 ' '5054']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/sprout', 'mount_propagation': None, 'name': 'sproutlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/sprout/sprout_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/sprout', 'mount_propagation': None, 'name': 'sproutlogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup ralf; do echo ' 'waiting for ralf to start ...; ' 'sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-ralf', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until nslookup chronos; do echo ' 'waiting for chronos to start ...; ' 'sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-chronos', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['sh', '-c', 'until nslookup homestead; do echo ' 'waiting for homestead to start ' '...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-homestead', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'node_name': 'kali-worker', 'node_selector': None, 'preemption_policy': None, 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'sproutlogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}, {'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-52qls', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-52qls'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 16, 0, 41, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 16, 0, 52, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 16, 0, 52, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 53, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://4f666aac0f41918a38db45dfb1705369dfe4dba601516b008359ddd1e33f1667', 'image': 'docker.io/ollivier/clearwater-sprout:latest', 'image_id': 'docker.io/ollivier/clearwater-sprout@sha256:861863a8f603b8851858fcb66492d5fa8af26e14ec84a26da5d75fe762b144b2', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'sprout', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 8, 28, 16, 0, 44, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}, {'container_id': 'containerd://15392395a1d83329e0b245c2dd7ccbfd4758183eef6811c7c8bca8f6ad9b0141', 'image': 'docker.io/library/busybox:latest', 'image_id': 'docker.io/library/busybox@sha256:4f47c01fa91355af2865ac10fef5bf6ec9c7f42ad2321377c21e844427972977', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'tailer', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 8, 28, 16, 0, 47, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.15', 'init_container_statuses': [{'container_id': 'containerd://7c6fc421dbee115dbd3df7c18ca2d2ef29fcae023650421bf64630a2495e9aaa', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-ralf', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://7c6fc421dbee115dbd3df7c18ca2d2ef29fcae023650421bf64630a2495e9aaa', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 8, 28, 15, 59, 42, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 8, 28, 15, 57, 7, tzinfo=tzlocal())}, 'waiting': None}}, {'container_id': 'containerd://1bbac6dcdee9da31790f3a19de9327661db81f1364996eeff4a6c503d4fd805e', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-chronos', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://1bbac6dcdee9da31790f3a19de9327661db81f1364996eeff4a6c503d4fd805e', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 8, 28, 15, 59, 46, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 8, 28, 15, 59, 46, tzinfo=tzlocal())}, 'waiting': None}}, {'container_id': 'containerd://10b0704a6b7a249401dd02b34bfd11923f14b9b62f4378d1381acbd130faa453', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-homestead', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://10b0704a6b7a249401dd02b34bfd11923f14b9b62f4378d1381acbd130faa453', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 8, 28, 16, 0, 40, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 8, 28, 15, 59, 50, tzinfo=tzlocal())}, 'waiting': None}}], 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.1.31', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 8, 28, 15, 56, 57, tzinfo=tzlocal())}}, 'raw_object': {'kind': 'Pod', 'apiVersion': 'v1', 'metadata': {'name': 'sprout-669689d654-46vtm', 'generateName': 'sprout-669689d654-', 'namespace': 'ims-gc7hd', 'selfLink': '/api/v1/namespaces/ims-gc7hd/pods/sprout-669689d654-46vtm', 'uid': 'b99e79ee-7c38-4c31-9ddd-537252f70d65', 'resourceVersion': '1802123', 'creationTimestamp': '2020-08-28T15:56:53Z', 'labels': {'pod-template-hash': '669689d654', 'service': 'sprout', 'snmp': 'enabled'}, 'ownerReferences': [{'apiVersion': 'apps/v1', 'kind': 'ReplicaSet', 'name': 'sprout-669689d654', 'uid': '15418e1c-0c46-4357-957a-3f671989d24d', 'controller': True, 'blockOwnerDeletion': True}]}, 'spec': {'volumes': [{'name': 'sproutlogs', 'emptyDir': {}}, {'name': 'default-token-52qls', 'secret': {'secretName': 'default-token-52qls', 'defaultMode': 420}}], 'initContainers': [{'name': 'wait-on-ralf', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup ralf; do echo waiting for ralf to start ...; sleep 2; done'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'wait-on-chronos', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup chronos; do echo waiting for chronos to start ...; sleep 2; done'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}, {'name': 'wait-on-homestead', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup homestead; do echo waiting for homestead to start ...; sleep 2; done'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'sprout', 'image': 'ollivier/clearwater-sprout:latest', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'sproutlogs', 'mountPath': '/var/log/sprout'}, {'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 5054']}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 3, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '5052 5054']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}, {'name': 'tailer', 'image': 'busybox', 'command': ['tail', '-F', '/var/log/sprout/sprout_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'sproutlogs', 'mountPath': '/var/log/sprout'}, {'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'serviceAccountName': 'default', 'serviceAccount': 'default', 'nodeName': 'kali-worker', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler', 'tolerations': [{'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}, {'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}], 'priority': 0, 'enableServiceLinks': True}, 'status': {'phase': 'Running', 'conditions': [{'type': 'Initialized', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T16:00:41Z'}, {'type': 'Ready', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T16:00:52Z'}, {'type': 'ContainersReady', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T16:00:52Z'}, {'type': 'PodScheduled', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T15:56:53Z'}], 'hostIP': '172.18.0.15', 'podIP': '10.244.1.31', 'podIPs': [{'ip': '10.244.1.31'}], 'startTime': '2020-08-28T15:56:57Z', 'initContainerStatuses': [{'name': 'wait-on-ralf', 'state': {'terminated': {'exitCode': 0, 'reason': 'Completed', 'startedAt': '2020-08-28T15:57:07Z', 'finishedAt': '2020-08-28T15:59:42Z', 'containerID': 'containerd://7c6fc421dbee115dbd3df7c18ca2d2ef29fcae023650421bf64630a2495e9aaa'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:1.28', 'imageID': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'containerID': 'containerd://7c6fc421dbee115dbd3df7c18ca2d2ef29fcae023650421bf64630a2495e9aaa'}, {'name': 'wait-on-chronos', 'state': {'terminated': {'exitCode': 0, 'reason': 'Completed', 'startedAt': '2020-08-28T15:59:46Z', 'finishedAt': '2020-08-28T15:59:46Z', 'containerID': 'containerd://1bbac6dcdee9da31790f3a19de9327661db81f1364996eeff4a6c503d4fd805e'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:1.28', 'imageID': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'containerID': 'containerd://1bbac6dcdee9da31790f3a19de9327661db81f1364996eeff4a6c503d4fd805e'}, {'name': 'wait-on-homestead', 'state': {'terminated': {'exitCode': 0, 'reason': 'Completed', 'startedAt': '2020-08-28T15:59:50Z', 'finishedAt': '2020-08-28T16:00:40Z', 'containerID': 'containerd://10b0704a6b7a249401dd02b34bfd11923f14b9b62f4378d1381acbd130faa453'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:1.28', 'imageID': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'containerID': 'containerd://10b0704a6b7a249401dd02b34bfd11923f14b9b62f4378d1381acbd130faa453'}], 'containerStatuses': [{'name': 'sprout', 'state': {'running': {'startedAt': '2020-08-28T16:00:44Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/ollivier/clearwater-sprout:latest', 'imageID': 'docker.io/ollivier/clearwater-sprout@sha256:861863a8f603b8851858fcb66492d5fa8af26e14ec84a26da5d75fe762b144b2', 'containerID': 'containerd://4f666aac0f41918a38db45dfb1705369dfe4dba601516b008359ddd1e33f1667', 'started': True}, {'name': 'tailer', 'state': {'running': {'startedAt': '2020-08-28T16:00:47Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:latest', 'imageID': 'docker.io/library/busybox@sha256:4f47c01fa91355af2865ac10fef5bf6ec9c7f42ad2321377c21e844427972977', 'containerID': 'containerd://15392395a1d83329e0b245c2dd7ccbfd4758183eef6811c7c8bca8f6ad9b0141', 'started': True}], 'qosClass': 'BestEffort'}}} 2020-08-28 16:02:44,668 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 50, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'cassandra-7fbf8bb9cf-', 'generation': None, 'initializers': None, 'labels': {'pod-template-hash': '7fbf8bb9cf', 'service': 'cassandra'}, 'managed_fields': None, 'name': 'cassandra-7fbf8bb9cf-lvbfp', 'namespace': 'ims-gc7hd', 'owner_references': [{'api_version': 'apps/v1', 'block_owner_deletion': True, 'controller': True, 'kind': 'ReplicaSet', 'name': 'cassandra-7fbf8bb9cf', 'uid': 'df53d0a9-b34f-4cc0-9e8c-9de9376b193d'}], 'resource_version': '1801706', 'self_link': '/api/v1/namespaces/ims-gc7hd/pods/cassandra-7fbf8bb9cf-lvbfp', 'uid': 'f1a106ad-73e0-44af-b2c5-59623cfdbd5a'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-cassandra:latest', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 9042 ' '9160']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 600, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'cassandra', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 7001, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 9042, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 9160, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 9042 ' '9160']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup etcd; do echo ' 'waiting for etcd to start ...; ' 'sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-etcd', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'node_name': 'kali-worker2', 'node_selector': None, 'preemption_policy': None, 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-52qls', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-52qls'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 15, 57, 37, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 15, 59, 1, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 15, 59, 1, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 50, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://d5df623bfdc299157636a48c9286bedd1237c93b2e04837f71c53db223dc50f0', 'image': 'docker.io/ollivier/clearwater-cassandra:latest', 'image_id': 'docker.io/ollivier/clearwater-cassandra@sha256:232be9c5a4400e4c5e0932fde50af8f379e3e9ddd4d3f28da6ec78c86f6ed9f6', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'cassandra', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 8, 28, 15, 57, 41, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.13', 'init_container_statuses': [{'container_id': 'containerd://d6a6faed41ffa7c2ad42928d7218a55ed4225c54ffaf549634ab9460929619d3', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-etcd', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://d6a6faed41ffa7c2ad42928d7218a55ed4225c54ffaf549634ab9460929619d3', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 8, 28, 15, 57, 36, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 8, 28, 15, 57, 4, tzinfo=tzlocal())}, 'waiting': None}}], 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.2.3', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 8, 28, 15, 56, 50, tzinfo=tzlocal())}}, 'raw_object': {'kind': 'Pod', 'apiVersion': 'v1', 'metadata': {'name': 'cassandra-7fbf8bb9cf-lvbfp', 'generateName': 'cassandra-7fbf8bb9cf-', 'namespace': 'ims-gc7hd', 'selfLink': '/api/v1/namespaces/ims-gc7hd/pods/cassandra-7fbf8bb9cf-lvbfp', 'uid': 'f1a106ad-73e0-44af-b2c5-59623cfdbd5a', 'resourceVersion': '1801706', 'creationTimestamp': '2020-08-28T15:56:50Z', 'labels': {'pod-template-hash': '7fbf8bb9cf', 'service': 'cassandra'}, 'ownerReferences': [{'apiVersion': 'apps/v1', 'kind': 'ReplicaSet', 'name': 'cassandra-7fbf8bb9cf', 'uid': 'df53d0a9-b34f-4cc0-9e8c-9de9376b193d', 'controller': True, 'blockOwnerDeletion': True}]}, 'spec': {'volumes': [{'name': 'default-token-52qls', 'secret': {'secretName': 'default-token-52qls', 'defaultMode': 420}}], 'initContainers': [{'name': 'wait-on-etcd', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup etcd; do echo waiting for etcd to start ...; sleep 2; done'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'cassandra', 'image': 'ollivier/clearwater-cassandra:latest', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 7001, 'protocol': 'TCP'}, {'containerPort': 9042, 'protocol': 'TCP'}, {'containerPort': 9160, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 9042 9160']}, 'initialDelaySeconds': 600, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '7000 9042 9160']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'serviceAccountName': 'default', 'serviceAccount': 'default', 'nodeName': 'kali-worker2', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler', 'tolerations': [{'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}, {'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}], 'priority': 0, 'enableServiceLinks': True}, 'status': {'phase': 'Running', 'conditions': [{'type': 'Initialized', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T15:57:37Z'}, {'type': 'Ready', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T15:59:01Z'}, {'type': 'ContainersReady', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T15:59:01Z'}, {'type': 'PodScheduled', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T15:56:50Z'}], 'hostIP': '172.18.0.13', 'podIP': '10.244.2.3', 'podIPs': [{'ip': '10.244.2.3'}], 'startTime': '2020-08-28T15:56:50Z', 'initContainerStatuses': [{'name': 'wait-on-etcd', 'state': {'terminated': {'exitCode': 0, 'reason': 'Completed', 'startedAt': '2020-08-28T15:57:04Z', 'finishedAt': '2020-08-28T15:57:36Z', 'containerID': 'containerd://d6a6faed41ffa7c2ad42928d7218a55ed4225c54ffaf549634ab9460929619d3'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:1.28', 'imageID': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'containerID': 'containerd://d6a6faed41ffa7c2ad42928d7218a55ed4225c54ffaf549634ab9460929619d3'}], 'containerStatuses': [{'name': 'cassandra', 'state': {'running': {'startedAt': '2020-08-28T15:57:41Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/ollivier/clearwater-cassandra:latest', 'imageID': 'docker.io/ollivier/clearwater-cassandra@sha256:232be9c5a4400e4c5e0932fde50af8f379e3e9ddd4d3f28da6ec78c86f6ed9f6', 'containerID': 'containerd://d5df623bfdc299157636a48c9286bedd1237c93b2e04837f71c53db223dc50f0', 'started': True}], 'qosClass': 'BestEffort'}}} 2020-08-28 16:02:44,925 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 51, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'homer-5fc8774db7-', 'generation': None, 'initializers': None, 'labels': {'pod-template-hash': '5fc8774db7', 'service': 'homer'}, 'managed_fields': None, 'name': 'homer-5fc8774db7-2l6n8', 'namespace': 'ims-gc7hd', 'owner_references': [{'api_version': 'apps/v1', 'block_owner_deletion': True, 'controller': True, 'kind': 'ReplicaSet', 'name': 'homer-5fc8774db7', 'uid': 'a7bbdb55-7eec-4689-9e7b-790902cdc98d'}], 'resource_version': '1801816', 'self_link': '/api/v1/namespaces/ims-gc7hd/pods/homer-5fc8774db7-2l6n8', 'uid': '61b10561-b107-48d0-996e-56762ad41e9a'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-homer:latest', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7888}, 'timeout_seconds': 1}, 'name': 'homer', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 7888, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7888}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup cassandra; do echo ' 'waiting for cassandra to start ' '...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'node_name': 'kali-worker2', 'node_selector': None, 'preemption_policy': None, 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-52qls', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-52qls'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 15, 59, 4, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 15, 59, 26, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 15, 59, 26, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 52, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://9f744d7658fe49fd65516fdef6b173cfde038e2628a36ea58175c1a2f3951749', 'image': 'docker.io/ollivier/clearwater-homer:latest', 'image_id': 'docker.io/ollivier/clearwater-homer@sha256:257ef9011d4ff30771c0c48ef7e3b16926dce88c17d4435953f433fa9e0d731a', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'homer', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 8, 28, 15, 59, 14, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.13', 'init_container_statuses': [{'container_id': 'containerd://dd3a1d405d641c3347ac823fee2f81a9b5475a54bb3b67bca3cafa468b4438fe', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-cassandra', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://dd3a1d405d641c3347ac823fee2f81a9b5475a54bb3b67bca3cafa468b4438fe', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 8, 28, 15, 59, 4, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 8, 28, 15, 57, 5, tzinfo=tzlocal())}, 'waiting': None}}], 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.2.5', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 8, 28, 15, 56, 52, tzinfo=tzlocal())}}, 'raw_object': {'kind': 'Pod', 'apiVersion': 'v1', 'metadata': {'name': 'homer-5fc8774db7-2l6n8', 'generateName': 'homer-5fc8774db7-', 'namespace': 'ims-gc7hd', 'selfLink': '/api/v1/namespaces/ims-gc7hd/pods/homer-5fc8774db7-2l6n8', 'uid': '61b10561-b107-48d0-996e-56762ad41e9a', 'resourceVersion': '1801816', 'creationTimestamp': '2020-08-28T15:56:51Z', 'labels': {'pod-template-hash': '5fc8774db7', 'service': 'homer'}, 'ownerReferences': [{'apiVersion': 'apps/v1', 'kind': 'ReplicaSet', 'name': 'homer-5fc8774db7', 'uid': 'a7bbdb55-7eec-4689-9e7b-790902cdc98d', 'controller': True, 'blockOwnerDeletion': True}]}, 'spec': {'volumes': [{'name': 'default-token-52qls', 'secret': {'secretName': 'default-token-52qls', 'defaultMode': 420}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'homer', 'image': 'ollivier/clearwater-homer:latest', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 7888, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'livenessProbe': {'tcpSocket': {'port': 7888}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'tcpSocket': {'port': 7888}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'serviceAccountName': 'default', 'serviceAccount': 'default', 'nodeName': 'kali-worker2', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler', 'tolerations': [{'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}, {'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}], 'priority': 0, 'enableServiceLinks': True}, 'status': {'phase': 'Running', 'conditions': [{'type': 'Initialized', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T15:59:04Z'}, {'type': 'Ready', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T15:59:26Z'}, {'type': 'ContainersReady', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T15:59:26Z'}, {'type': 'PodScheduled', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T15:56:52Z'}], 'hostIP': '172.18.0.13', 'podIP': '10.244.2.5', 'podIPs': [{'ip': '10.244.2.5'}], 'startTime': '2020-08-28T15:56:52Z', 'initContainerStatuses': [{'name': 'wait-on-cassandra', 'state': {'terminated': {'exitCode': 0, 'reason': 'Completed', 'startedAt': '2020-08-28T15:57:05Z', 'finishedAt': '2020-08-28T15:59:04Z', 'containerID': 'containerd://dd3a1d405d641c3347ac823fee2f81a9b5475a54bb3b67bca3cafa468b4438fe'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:1.28', 'imageID': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'containerID': 'containerd://dd3a1d405d641c3347ac823fee2f81a9b5475a54bb3b67bca3cafa468b4438fe'}], 'containerStatuses': [{'name': 'homer', 'state': {'running': {'startedAt': '2020-08-28T15:59:14Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/ollivier/clearwater-homer:latest', 'imageID': 'docker.io/ollivier/clearwater-homer@sha256:257ef9011d4ff30771c0c48ef7e3b16926dce88c17d4435953f433fa9e0d731a', 'containerID': 'containerd://9f744d7658fe49fd65516fdef6b173cfde038e2628a36ea58175c1a2f3951749', 'started': True}], 'qosClass': 'BestEffort'}}} 2020-08-28 16:02:45,129 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 16, 2, 42, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields': None, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2020, 8, 28, 16, 2, 42, tzinfo=tzlocal())}], 'name': 'live-test', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1802468', 'self_link': '/api/v1/namespaces/ims-gc7hd/pods/live-test', 'uid': 'bebc59b7-ad08-4316-b33c-15b0a8b7cf0b'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': ['rake', 'test[ims-gc7hd.svc.cluster.local]', 'PROXY=bono.ims-gc7hd.svc.cluster.local', 'ELLIS=ellis.ims-gc7hd.svc.cluster.local', 'SIGNUP_CODE=secret', '--trace'], 'env': None, 'env_from': None, 'image': 'ollivier/clearwater-live-test:latest', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': None, 'name': 'live-test', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': None, 'init_containers': None, 'node_name': None, 'node_selector': None, 'preemption_policy': None, 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Never', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-52qls', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-52qls'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': None, 'container_statuses': None, 'host_ip': None, 'init_container_statuses': None, 'message': None, 'nominated_node_name': None, 'phase': 'Pending', 'pod_ip': None, 'qos_class': 'BestEffort', 'reason': None, 'start_time': None}}, 'raw_object': {'kind': 'Pod', 'apiVersion': 'v1', 'metadata': {'name': 'live-test', 'namespace': 'ims-gc7hd', 'selfLink': '/api/v1/namespaces/ims-gc7hd/pods/live-test', 'uid': 'bebc59b7-ad08-4316-b33c-15b0a8b7cf0b', 'resourceVersion': '1802468', 'creationTimestamp': '2020-08-28T16:02:42Z', 'managedFields': [{'manager': 'OpenAPI-Generator', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2020-08-28T16:02:42Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:spec': {'f:containers': {'k:{"name":"live-test"}': {'.': {}, 'f:command': {}, 'f:image': {}, 'f:imagePullPolicy': {}, 'f:name': {}, 'f:resources': {}, 'f:terminationMessagePath': {}, 'f:terminationMessagePolicy': {}}}, 'f:dnsPolicy': {}, 'f:enableServiceLinks': {}, 'f:restartPolicy': {}, 'f:schedulerName': {}, 'f:securityContext': {}, 'f:terminationGracePeriodSeconds': {}}}}]}, 'spec': {'volumes': [{'name': 'default-token-52qls', 'secret': {'secretName': 'default-token-52qls', 'defaultMode': 420}}], 'containers': [{'name': 'live-test', 'image': 'ollivier/clearwater-live-test:latest', 'command': ['rake', 'test[ims-gc7hd.svc.cluster.local]', 'PROXY=bono.ims-gc7hd.svc.cluster.local', 'ELLIS=ellis.ims-gc7hd.svc.cluster.local', 'SIGNUP_CODE=secret', '--trace'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}], 'restartPolicy': 'Never', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'serviceAccountName': 'default', 'serviceAccount': 'default', 'securityContext': {}, 'schedulerName': 'default-scheduler', 'tolerations': [{'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}, {'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}], 'priority': 0, 'enableServiceLinks': True}, 'status': {'phase': 'Pending', 'qosClass': 'BestEffort'}}} 2020-08-28 16:02:45,243 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 49, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'bono-69fbbd8646-', 'generation': None, 'initializers': None, 'labels': {'pod-template-hash': '69fbbd8646', 'service': 'bono', 'snmp': 'enabled'}, 'managed_fields': None, 'name': 'bono-69fbbd8646-5x5jx', 'namespace': 'ims-gc7hd', 'owner_references': [{'api_version': 'apps/v1', 'block_owner_deletion': True, 'controller': True, 'kind': 'ReplicaSet', 'name': 'bono-69fbbd8646', 'uid': '004b7c68-9bb5-47fc-9e94-19389d3524b6'}], 'resource_version': '1802268', 'self_link': '/api/v1/namespaces/ims-gc7hd/pods/bono-69fbbd8646-5x5jx', 'uid': '1ce66071-06d5-464a-b359-9d7b36ebb3e6'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-bono:latest', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 5060 ' '5062']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 30, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'name': 'bono', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 3478, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 5060, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 5062, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 5060, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'UDP'}, {'container_port': 5062, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'UDP'}], 'readiness_probe': {'_exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 5060 ' '5062']}, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': None, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/bono', 'mount_propagation': None, 'name': 'bonologs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/bono/bono_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/bono', 'mount_propagation': None, 'name': 'bonologs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup sprout; do echo ' 'waiting for sprout to start ...; ' 'sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-sprout', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'node_name': 'kali-worker', 'node_selector': None, 'preemption_policy': None, 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'bonologs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}, {'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-52qls', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-52qls'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 16, 1, 17, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 16, 1, 36, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 16, 1, 36, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 50, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://2a38c82e0ef487016888663735ede12bed1ec76ce087733fdd182df90a45c892', 'image': 'docker.io/ollivier/clearwater-bono:latest', 'image_id': 'docker.io/ollivier/clearwater-bono@sha256:95d9d53fc68c24deb2095b7b91aa7e53090f99e9c1d5c43dcf5d9a6fb8a8cdc2', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'bono', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 8, 28, 16, 1, 27, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}, {'container_id': 'containerd://275bfac293bf01850854c88fd9efa1900fa8439cc42e7cfde4ede48d9078d637', 'image': 'docker.io/library/busybox:latest', 'image_id': 'docker.io/library/busybox@sha256:4f47c01fa91355af2865ac10fef5bf6ec9c7f42ad2321377c21e844427972977', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'tailer', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 8, 28, 16, 1, 31, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.15', 'init_container_statuses': [{'container_id': 'containerd://417d55cb3190b0288a4d479ebaecc4194a944ca6551195e4601acfc9fe270955', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-sprout', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://417d55cb3190b0288a4d479ebaecc4194a944ca6551195e4601acfc9fe270955', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 8, 28, 16, 1, 14, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 8, 28, 15, 56, 58, tzinfo=tzlocal())}, 'waiting': None}}], 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.1.27', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 8, 28, 15, 56, 50, tzinfo=tzlocal())}}, 'raw_object': {'kind': 'Pod', 'apiVersion': 'v1', 'metadata': {'name': 'bono-69fbbd8646-5x5jx', 'generateName': 'bono-69fbbd8646-', 'namespace': 'ims-gc7hd', 'selfLink': '/api/v1/namespaces/ims-gc7hd/pods/bono-69fbbd8646-5x5jx', 'uid': '1ce66071-06d5-464a-b359-9d7b36ebb3e6', 'resourceVersion': '1802268', 'creationTimestamp': '2020-08-28T15:56:49Z', 'labels': {'pod-template-hash': '69fbbd8646', 'service': 'bono', 'snmp': 'enabled'}, 'ownerReferences': [{'apiVersion': 'apps/v1', 'kind': 'ReplicaSet', 'name': 'bono-69fbbd8646', 'uid': '004b7c68-9bb5-47fc-9e94-19389d3524b6', 'controller': True, 'blockOwnerDeletion': True}]}, 'spec': {'volumes': [{'name': 'bonologs', 'emptyDir': {}}, {'name': 'default-token-52qls', 'secret': {'secretName': 'default-token-52qls', 'defaultMode': 420}}], 'initContainers': [{'name': 'wait-on-sprout', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup sprout; do echo waiting for sprout to start ...; sleep 2; done'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'bono', 'image': 'ollivier/clearwater-bono:latest', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 3478, 'protocol': 'TCP'}, {'containerPort': 5060, 'protocol': 'TCP'}, {'containerPort': 5062, 'protocol': 'TCP'}, {'containerPort': 5060, 'protocol': 'UDP'}, {'containerPort': 5062, 'protocol': 'UDP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'bonologs', 'mountPath': '/var/log/bono'}, {'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'livenessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 5060 5062']}, 'initialDelaySeconds': 30, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'exec': {'command': ['/bin/bash', '/usr/share/kubernetes/liveness.sh', '3478 5060 5062']}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}, {'name': 'tailer', 'image': 'busybox', 'command': ['tail', '-F', '/var/log/bono/bono_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'bonologs', 'mountPath': '/var/log/bono'}, {'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'serviceAccountName': 'default', 'serviceAccount': 'default', 'nodeName': 'kali-worker', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler', 'tolerations': [{'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}, {'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}], 'priority': 0, 'enableServiceLinks': True}, 'status': {'phase': 'Running', 'conditions': [{'type': 'Initialized', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T16:01:17Z'}, {'type': 'Ready', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T16:01:36Z'}, {'type': 'ContainersReady', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T16:01:36Z'}, {'type': 'PodScheduled', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T15:56:50Z'}], 'hostIP': '172.18.0.15', 'podIP': '10.244.1.27', 'podIPs': [{'ip': '10.244.1.27'}], 'startTime': '2020-08-28T15:56:50Z', 'initContainerStatuses': [{'name': 'wait-on-sprout', 'state': {'terminated': {'exitCode': 0, 'reason': 'Completed', 'startedAt': '2020-08-28T15:56:58Z', 'finishedAt': '2020-08-28T16:01:14Z', 'containerID': 'containerd://417d55cb3190b0288a4d479ebaecc4194a944ca6551195e4601acfc9fe270955'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:1.28', 'imageID': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'containerID': 'containerd://417d55cb3190b0288a4d479ebaecc4194a944ca6551195e4601acfc9fe270955'}], 'containerStatuses': [{'name': 'bono', 'state': {'running': {'startedAt': '2020-08-28T16:01:27Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/ollivier/clearwater-bono:latest', 'imageID': 'docker.io/ollivier/clearwater-bono@sha256:95d9d53fc68c24deb2095b7b91aa7e53090f99e9c1d5c43dcf5d9a6fb8a8cdc2', 'containerID': 'containerd://2a38c82e0ef487016888663735ede12bed1ec76ce087733fdd182df90a45c892', 'started': True}, {'name': 'tailer', 'state': {'running': {'startedAt': '2020-08-28T16:01:31Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:latest', 'imageID': 'docker.io/library/busybox@sha256:4f47c01fa91355af2865ac10fef5bf6ec9c7f42ad2321377c21e844427972977', 'containerID': 'containerd://275bfac293bf01850854c88fd9efa1900fa8439cc42e7cfde4ede48d9078d637', 'started': True}], 'qosClass': 'BestEffort'}}} 2020-08-28 16:02:45,550 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 50, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'chronos-c55b4b9ff-', 'generation': None, 'initializers': None, 'labels': {'pod-template-hash': 'c55b4b9ff', 'service': 'chronos'}, 'managed_fields': None, 'name': 'chronos-c55b4b9ff-8gqcm', 'namespace': 'ims-gc7hd', 'owner_references': [{'api_version': 'apps/v1', 'block_owner_deletion': True, 'controller': True, 'kind': 'ReplicaSet', 'name': 'chronos-c55b4b9ff', 'uid': '18065a32-be9b-40c0-9d0a-e9e0eacfb16d'}], 'resource_version': '1801841', 'self_link': '/api/v1/namespaces/ims-gc7hd/pods/chronos-c55b4b9ff-8gqcm', 'uid': '84e95bc4-e899-4ef9-a1d3-3214899fc8ae'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-chronos:latest', 'image_pull_policy': 'Always', 'lifecycle': {'post_start': None, 'pre_stop': {'_exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}, 'http_get': None, 'tcp_socket': None}}, 'liveness_probe': {'_exec': None, 'failure_threshold': 9, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7253}, 'timeout_seconds': 1}, 'name': 'chronos', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 7253, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 7253}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/chronos', 'mount_propagation': None, 'name': 'chronoslogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/chronos/chronos_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/chronos', 'mount_propagation': None, 'name': 'chronoslogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup cassandra; do echo ' 'waiting for cassandra to start ' '...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'node_name': 'kali-worker', 'node_selector': None, 'preemption_policy': None, 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 120, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'chronoslogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}, {'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-52qls', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-52qls'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 15, 59, 6, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 15, 59, 31, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 15, 59, 31, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 50, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://e18ec4cfce53e611378bbc443b5e81a3207045d183423d9e7bd87d2b2cc237be', 'image': 'docker.io/ollivier/clearwater-chronos:latest', 'image_id': 'docker.io/ollivier/clearwater-chronos@sha256:3e838bae03946022eae06e3d343167d4b28507909e9c17e1bf574a23b423f83d', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'chronos', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 8, 28, 15, 59, 13, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}, {'container_id': 'containerd://703a4d0fd0b8037538fe6dab8b456c01c6ba205216da3fb4d7371166e54a79b3', 'image': 'docker.io/library/busybox:latest', 'image_id': 'docker.io/library/busybox@sha256:4f47c01fa91355af2865ac10fef5bf6ec9c7f42ad2321377c21e844427972977', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'tailer', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 8, 28, 15, 59, 20, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.15', 'init_container_statuses': [{'container_id': 'containerd://fda0e7bbf7829e64b2fc831487357d0e6aea81c4f853e7c8a1a83199cccb9fb9', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-cassandra', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://fda0e7bbf7829e64b2fc831487357d0e6aea81c4f853e7c8a1a83199cccb9fb9', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 8, 28, 15, 59, 5, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 8, 28, 15, 57, 3, tzinfo=tzlocal())}, 'waiting': None}}], 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.1.28', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 8, 28, 15, 56, 50, tzinfo=tzlocal())}}, 'raw_object': {'kind': 'Pod', 'apiVersion': 'v1', 'metadata': {'name': 'chronos-c55b4b9ff-8gqcm', 'generateName': 'chronos-c55b4b9ff-', 'namespace': 'ims-gc7hd', 'selfLink': '/api/v1/namespaces/ims-gc7hd/pods/chronos-c55b4b9ff-8gqcm', 'uid': '84e95bc4-e899-4ef9-a1d3-3214899fc8ae', 'resourceVersion': '1801841', 'creationTimestamp': '2020-08-28T15:56:50Z', 'labels': {'pod-template-hash': 'c55b4b9ff', 'service': 'chronos'}, 'ownerReferences': [{'apiVersion': 'apps/v1', 'kind': 'ReplicaSet', 'name': 'chronos-c55b4b9ff', 'uid': '18065a32-be9b-40c0-9d0a-e9e0eacfb16d', 'controller': True, 'blockOwnerDeletion': True}]}, 'spec': {'volumes': [{'name': 'chronoslogs', 'emptyDir': {}}, {'name': 'default-token-52qls', 'secret': {'secretName': 'default-token-52qls', 'defaultMode': 420}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'chronos', 'image': 'ollivier/clearwater-chronos:latest', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 7253, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'chronoslogs', 'mountPath': '/var/log/chronos'}, {'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'livenessProbe': {'tcpSocket': {'port': 7253}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 9}, 'readinessProbe': {'tcpSocket': {'port': 7253}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'lifecycle': {'preStop': {'exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}}}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}, {'name': 'tailer', 'image': 'busybox', 'command': ['tail', '-F', '/var/log/chronos/chronos_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'chronoslogs', 'mountPath': '/var/log/chronos'}, {'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 120, 'dnsPolicy': 'ClusterFirst', 'serviceAccountName': 'default', 'serviceAccount': 'default', 'nodeName': 'kali-worker', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler', 'tolerations': [{'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}, {'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}], 'priority': 0, 'enableServiceLinks': True}, 'status': {'phase': 'Running', 'conditions': [{'type': 'Initialized', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T15:59:06Z'}, {'type': 'Ready', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T15:59:31Z'}, {'type': 'ContainersReady', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T15:59:31Z'}, {'type': 'PodScheduled', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T15:56:50Z'}], 'hostIP': '172.18.0.15', 'podIP': '10.244.1.28', 'podIPs': [{'ip': '10.244.1.28'}], 'startTime': '2020-08-28T15:56:50Z', 'initContainerStatuses': [{'name': 'wait-on-cassandra', 'state': {'terminated': {'exitCode': 0, 'reason': 'Completed', 'startedAt': '2020-08-28T15:57:03Z', 'finishedAt': '2020-08-28T15:59:05Z', 'containerID': 'containerd://fda0e7bbf7829e64b2fc831487357d0e6aea81c4f853e7c8a1a83199cccb9fb9'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:1.28', 'imageID': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'containerID': 'containerd://fda0e7bbf7829e64b2fc831487357d0e6aea81c4f853e7c8a1a83199cccb9fb9'}], 'containerStatuses': [{'name': 'chronos', 'state': {'running': {'startedAt': '2020-08-28T15:59:13Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/ollivier/clearwater-chronos:latest', 'imageID': 'docker.io/ollivier/clearwater-chronos@sha256:3e838bae03946022eae06e3d343167d4b28507909e9c17e1bf574a23b423f83d', 'containerID': 'containerd://e18ec4cfce53e611378bbc443b5e81a3207045d183423d9e7bd87d2b2cc237be', 'started': True}, {'name': 'tailer', 'state': {'running': {'startedAt': '2020-08-28T15:59:20Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:latest', 'imageID': 'docker.io/library/busybox@sha256:4f47c01fa91355af2865ac10fef5bf6ec9c7f42ad2321377c21e844427972977', 'containerID': 'containerd://703a4d0fd0b8037538fe6dab8b456c01c6ba205216da3fb4d7371166e54a79b3', 'started': True}], 'qosClass': 'BestEffort'}}} 2020-08-28 16:02:45,849 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 49, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'astaire-7666ff5885-', 'generation': None, 'initializers': None, 'labels': {'pod-template-hash': '7666ff5885', 'service': 'astaire'}, 'managed_fields': None, 'name': 'astaire-7666ff5885-bgmth', 'namespace': 'ims-gc7hd', 'owner_references': [{'api_version': 'apps/v1', 'block_owner_deletion': True, 'controller': True, 'kind': 'ReplicaSet', 'name': 'astaire-7666ff5885', 'uid': '445fdcd5-a7d2-47c1-a664-a68d3ad1383a'}], 'resource_version': '1801968', 'self_link': '/api/v1/namespaces/ims-gc7hd/pods/astaire-7666ff5885-bgmth', 'uid': '928f72b0-089f-4c24-8994-a2c7b5e75cc6'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': [{'config_map_ref': {'name': 'env-vars', 'optional': None}, 'prefix': None, 'secret_ref': None}], 'image': 'ollivier/clearwater-astaire:latest', 'image_pull_policy': 'Always', 'lifecycle': {'post_start': None, 'pre_stop': {'_exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}, 'http_get': None, 'tcp_socket': None}}, 'liveness_probe': {'_exec': None, 'failure_threshold': 9, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 11311}, 'timeout_seconds': 1}, 'name': 'astaire', 'ports': [{'container_port': 22, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 11211, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 11311, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 11311}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/astaire', 'mount_propagation': None, 'name': 'astairelogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}, {'args': None, 'command': ['tail', '-F', '/var/log/astaire/astaire_current.txt'], 'env': None, 'env_from': None, 'image': 'busybox', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': None, 'name': 'tailer', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/log/astaire', 'mount_propagation': None, 'name': 'astairelogs', 'read_only': None, 'sub_path': None, 'sub_path_expr': None}, {'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': [{'args': None, 'command': ['sh', '-c', 'until nslookup cassandra; do echo ' 'waiting for cassandra to start ' '...; sleep 2; done'], 'env': None, 'env_from': None, 'image': 'busybox:1.28', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': None, 'name': 'wait-on-cassandra', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'node_name': 'kali-worker2', 'node_selector': None, 'preemption_policy': None, 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 120, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': {'medium': None, 'size_limit': None}, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'astairelogs', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': None, 'storageos': None, 'vsphere_volume': None}, {'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-52qls', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-52qls'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 15, 59, 4, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 16, 0, 8, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 16, 0, 8, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 49, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://9e96b3c93d0e16cc4424cfe3169e38b4bf7a7fcb60f3c5f138eb330745bcc1b7', 'image': 'docker.io/ollivier/clearwater-astaire:latest', 'image_id': 'docker.io/ollivier/clearwater-astaire@sha256:eb85c150a60609d7b22b70b99d6a1a7a1c035fd64e30cca203a8b8d167bb7938', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'astaire', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 8, 28, 15, 59, 17, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}, {'container_id': 'containerd://cd581913fc390ba79a7d51e199a3ef3b287360ca8e74ea02ba406e5a19d5bb3e', 'image': 'docker.io/library/busybox:latest', 'image_id': 'docker.io/library/busybox@sha256:4f47c01fa91355af2865ac10fef5bf6ec9c7f42ad2321377c21e844427972977', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'tailer', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 8, 28, 15, 59, 22, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.13', 'init_container_statuses': [{'container_id': 'containerd://67fcbf9ac48b443b57b01bb1d0b2e7c8d80cded20434affbb04939d18ab2c9f4', 'image': 'docker.io/library/busybox:1.28', 'image_id': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'wait-on-cassandra', 'ready': True, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://67fcbf9ac48b443b57b01bb1d0b2e7c8d80cded20434affbb04939d18ab2c9f4', 'exit_code': 0, 'finished_at': datetime.datetime(2020, 8, 28, 15, 59, 3, tzinfo=tzlocal()), 'message': None, 'reason': 'Completed', 'signal': None, 'started_at': datetime.datetime(2020, 8, 28, 15, 56, 58, tzinfo=tzlocal())}, 'waiting': None}}], 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.2.2', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 8, 28, 15, 56, 50, tzinfo=tzlocal())}}, 'raw_object': {'kind': 'Pod', 'apiVersion': 'v1', 'metadata': {'name': 'astaire-7666ff5885-bgmth', 'generateName': 'astaire-7666ff5885-', 'namespace': 'ims-gc7hd', 'selfLink': '/api/v1/namespaces/ims-gc7hd/pods/astaire-7666ff5885-bgmth', 'uid': '928f72b0-089f-4c24-8994-a2c7b5e75cc6', 'resourceVersion': '1801968', 'creationTimestamp': '2020-08-28T15:56:49Z', 'labels': {'pod-template-hash': '7666ff5885', 'service': 'astaire'}, 'ownerReferences': [{'apiVersion': 'apps/v1', 'kind': 'ReplicaSet', 'name': 'astaire-7666ff5885', 'uid': '445fdcd5-a7d2-47c1-a664-a68d3ad1383a', 'controller': True, 'blockOwnerDeletion': True}]}, 'spec': {'volumes': [{'name': 'astairelogs', 'emptyDir': {}}, {'name': 'default-token-52qls', 'secret': {'secretName': 'default-token-52qls', 'defaultMode': 420}}], 'initContainers': [{'name': 'wait-on-cassandra', 'image': 'busybox:1.28', 'command': ['sh', '-c', 'until nslookup cassandra; do echo waiting for cassandra to start ...; sleep 2; done'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'containers': [{'name': 'astaire', 'image': 'ollivier/clearwater-astaire:latest', 'ports': [{'containerPort': 22, 'protocol': 'TCP'}, {'containerPort': 11211, 'protocol': 'TCP'}, {'containerPort': 11311, 'protocol': 'TCP'}], 'envFrom': [{'configMapRef': {'name': 'env-vars'}}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}], 'resources': {}, 'volumeMounts': [{'name': 'astairelogs', 'mountPath': '/var/log/astaire'}, {'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'livenessProbe': {'tcpSocket': {'port': 11311}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 9}, 'readinessProbe': {'tcpSocket': {'port': 11311}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'lifecycle': {'preStop': {'exec': {'command': ['/bin/bash', '-c', '/usr/bin/pre-stop']}}}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}, {'name': 'tailer', 'image': 'busybox', 'command': ['tail', '-F', '/var/log/astaire/astaire_current.txt'], 'resources': {}, 'volumeMounts': [{'name': 'astairelogs', 'mountPath': '/var/log/astaire'}, {'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 120, 'dnsPolicy': 'ClusterFirst', 'serviceAccountName': 'default', 'serviceAccount': 'default', 'nodeName': 'kali-worker2', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler', 'tolerations': [{'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}, {'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}], 'priority': 0, 'enableServiceLinks': True}, 'status': {'phase': 'Running', 'conditions': [{'type': 'Initialized', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T15:59:04Z'}, {'type': 'Ready', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T16:00:08Z'}, {'type': 'ContainersReady', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T16:00:08Z'}, {'type': 'PodScheduled', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T15:56:49Z'}], 'hostIP': '172.18.0.13', 'podIP': '10.244.2.2', 'podIPs': [{'ip': '10.244.2.2'}], 'startTime': '2020-08-28T15:56:50Z', 'initContainerStatuses': [{'name': 'wait-on-cassandra', 'state': {'terminated': {'exitCode': 0, 'reason': 'Completed', 'startedAt': '2020-08-28T15:56:58Z', 'finishedAt': '2020-08-28T15:59:03Z', 'containerID': 'containerd://67fcbf9ac48b443b57b01bb1d0b2e7c8d80cded20434affbb04939d18ab2c9f4'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:1.28', 'imageID': 'docker.io/library/busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47', 'containerID': 'containerd://67fcbf9ac48b443b57b01bb1d0b2e7c8d80cded20434affbb04939d18ab2c9f4'}], 'containerStatuses': [{'name': 'astaire', 'state': {'running': {'startedAt': '2020-08-28T15:59:17Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/ollivier/clearwater-astaire:latest', 'imageID': 'docker.io/ollivier/clearwater-astaire@sha256:eb85c150a60609d7b22b70b99d6a1a7a1c035fd64e30cca203a8b8d167bb7938', 'containerID': 'containerd://9e96b3c93d0e16cc4424cfe3169e38b4bf7a7fcb60f3c5f138eb330745bcc1b7', 'started': True}, {'name': 'tailer', 'state': {'running': {'startedAt': '2020-08-28T15:59:22Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/library/busybox:latest', 'imageID': 'docker.io/library/busybox@sha256:4f47c01fa91355af2865ac10fef5bf6ec9c7f42ad2321377c21e844427972977', 'containerID': 'containerd://cd581913fc390ba79a7d51e199a3ef3b287360ca8e74ea02ba406e5a19d5bb3e', 'started': True}], 'qosClass': 'BestEffort'}}} 2020-08-28 16:02:46,551 - functest_kubernetes.ims.ims - DEBUG - {'type': 'ADDED', 'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 15, 56, 51, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': 'etcd-596fb59b77-', 'generation': None, 'initializers': None, 'labels': {'instance-type': 'etcd-pod', 'pod-template-hash': '596fb59b77'}, 'managed_fields': None, 'name': 'etcd-596fb59b77-2f565', 'namespace': 'ims-gc7hd', 'owner_references': [{'api_version': 'apps/v1', 'block_owner_deletion': True, 'controller': True, 'kind': 'ReplicaSet', 'name': 'etcd-596fb59b77', 'uid': 'badc387a-aba8-4964-9ce2-da8f037a1fba'}], 'resource_version': '1801384', 'self_link': '/api/v1/namespaces/ims-gc7hd/pods/etcd-596fb59b77-2f565', 'uid': '9376a207-d244-40fc-8275-0cca8a9476ee'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': ['--name', '$(MY_POD_NAME)', '--advertise-client-urls', 'http://$(MY_POD_IP):2379,http://$(MY_POD_IP):4001', '--listen-client-urls', 'http://0.0.0.0:2379,http://0.0.0.0:4001', '--initial-advertise-peer-urls', 'http://$(MY_POD_IP):2380', '--listen-peer-urls', 'http://0.0.0.0:2380', '--initial-cluster', '$(MY_POD_NAME)=http://$(MY_POD_IP):2380', '--initial-cluster-state', 'new'], 'command': None, 'env': [{'name': 'MY_POD_IP', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'status.podIP'}, 'resource_field_ref': None, 'secret_key_ref': None}}, {'name': 'MY_POD_NAME', 'value': None, 'value_from': {'config_map_key_ref': None, 'field_ref': {'api_version': 'v1', 'field_path': 'metadata.name'}, 'resource_field_ref': None, 'secret_key_ref': None}}], 'env_from': None, 'image': 'quay.io/coreos/etcd:v2.2.5', 'image_pull_policy': 'IfNotPresent', 'lifecycle': None, 'liveness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': 300, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 4001}, 'timeout_seconds': 1}, 'name': 'etcd', 'ports': [{'container_port': 2379, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 2380, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}, {'container_port': 4001, 'host_ip': None, 'host_port': None, 'name': None, 'protocol': 'TCP'}], 'readiness_probe': {'_exec': None, 'failure_threshold': 3, 'http_get': None, 'initial_delay_seconds': None, 'period_seconds': 10, 'success_threshold': 1, 'tcp_socket': {'host': None, 'port': 4001}, 'timeout_seconds': 1}, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': [{'name': None}], 'init_containers': None, 'node_name': 'kali-worker', 'node_selector': None, 'preemption_policy': None, 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Always', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-52qls', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-52qls'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 52, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 15, 57, 15, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 15, 57, 15, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 15, 56, 51, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://ea9617a47ea57fa3e7f73a95c742800c0fe90099cb2f9021bc070e23d1f2ccf5', 'image': 'quay.io/coreos/etcd:v2.2.5', 'image_id': 'sha256:66590e2dd05422771324d8ffe9ad35165ba35db0842ac5e0f1668c2d56733860', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'etcd', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 8, 28, 15, 57, 3, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.15', 'init_container_statuses': None, 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.1.29', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 8, 28, 15, 56, 52, tzinfo=tzlocal())}}, 'raw_object': {'kind': 'Pod', 'apiVersion': 'v1', 'metadata': {'name': 'etcd-596fb59b77-2f565', 'generateName': 'etcd-596fb59b77-', 'namespace': 'ims-gc7hd', 'selfLink': '/api/v1/namespaces/ims-gc7hd/pods/etcd-596fb59b77-2f565', 'uid': '9376a207-d244-40fc-8275-0cca8a9476ee', 'resourceVersion': '1801384', 'creationTimestamp': '2020-08-28T15:56:51Z', 'labels': {'instance-type': 'etcd-pod', 'pod-template-hash': '596fb59b77'}, 'ownerReferences': [{'apiVersion': 'apps/v1', 'kind': 'ReplicaSet', 'name': 'etcd-596fb59b77', 'uid': 'badc387a-aba8-4964-9ce2-da8f037a1fba', 'controller': True, 'blockOwnerDeletion': True}]}, 'spec': {'volumes': [{'name': 'default-token-52qls', 'secret': {'secretName': 'default-token-52qls', 'defaultMode': 420}}], 'containers': [{'name': 'etcd', 'image': 'quay.io/coreos/etcd:v2.2.5', 'args': ['--name', '$(MY_POD_NAME)', '--advertise-client-urls', 'http://$(MY_POD_IP):2379,http://$(MY_POD_IP):4001', '--listen-client-urls', 'http://0.0.0.0:2379,http://0.0.0.0:4001', '--initial-advertise-peer-urls', 'http://$(MY_POD_IP):2380', '--listen-peer-urls', 'http://0.0.0.0:2380', '--initial-cluster', '$(MY_POD_NAME)=http://$(MY_POD_IP):2380', '--initial-cluster-state', 'new'], 'ports': [{'containerPort': 2379, 'protocol': 'TCP'}, {'containerPort': 2380, 'protocol': 'TCP'}, {'containerPort': 4001, 'protocol': 'TCP'}], 'env': [{'name': 'MY_POD_IP', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'status.podIP'}}}, {'name': 'MY_POD_NAME', 'valueFrom': {'fieldRef': {'apiVersion': 'v1', 'fieldPath': 'metadata.name'}}}], 'resources': {}, 'volumeMounts': [{'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'livenessProbe': {'tcpSocket': {'port': 4001}, 'initialDelaySeconds': 300, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'readinessProbe': {'tcpSocket': {'port': 4001}, 'timeoutSeconds': 1, 'periodSeconds': 10, 'successThreshold': 1, 'failureThreshold': 3}, 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'IfNotPresent'}], 'restartPolicy': 'Always', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'serviceAccountName': 'default', 'serviceAccount': 'default', 'nodeName': 'kali-worker', 'securityContext': {}, 'imagePullSecrets': [{}], 'schedulerName': 'default-scheduler', 'tolerations': [{'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}, {'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}], 'priority': 0, 'enableServiceLinks': True}, 'status': {'phase': 'Running', 'conditions': [{'type': 'Initialized', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T15:56:52Z'}, {'type': 'Ready', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T15:57:15Z'}, {'type': 'ContainersReady', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T15:57:15Z'}, {'type': 'PodScheduled', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T15:56:51Z'}], 'hostIP': '172.18.0.15', 'podIP': '10.244.1.29', 'podIPs': [{'ip': '10.244.1.29'}], 'startTime': '2020-08-28T15:56:52Z', 'containerStatuses': [{'name': 'etcd', 'state': {'running': {'startedAt': '2020-08-28T15:57:03Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'quay.io/coreos/etcd:v2.2.5', 'imageID': 'sha256:66590e2dd05422771324d8ffe9ad35165ba35db0842ac5e0f1668c2d56733860', 'containerID': 'containerd://ea9617a47ea57fa3e7f73a95c742800c0fe90099cb2f9021bc070e23d1f2ccf5', 'started': True}], 'qosClass': 'BestEffort'}}} 2020-08-28 16:02:46,775 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 16, 2, 42, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields': None, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2020, 8, 28, 16, 2, 42, tzinfo=tzlocal())}], 'name': 'live-test', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1802469', 'self_link': '/api/v1/namespaces/ims-gc7hd/pods/live-test', 'uid': 'bebc59b7-ad08-4316-b33c-15b0a8b7cf0b'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': ['rake', 'test[ims-gc7hd.svc.cluster.local]', 'PROXY=bono.ims-gc7hd.svc.cluster.local', 'ELLIS=ellis.ims-gc7hd.svc.cluster.local', 'SIGNUP_CODE=secret', '--trace'], 'env': None, 'env_from': None, 'image': 'ollivier/clearwater-live-test:latest', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': None, 'name': 'live-test', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': None, 'init_containers': None, 'node_name': 'kali-worker', 'node_selector': None, 'preemption_policy': None, 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Never', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-52qls', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-52qls'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 16, 2, 42, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': None, 'host_ip': None, 'init_container_statuses': None, 'message': None, 'nominated_node_name': None, 'phase': 'Pending', 'pod_ip': None, 'qos_class': 'BestEffort', 'reason': None, 'start_time': None}}, 'raw_object': {'kind': 'Pod', 'apiVersion': 'v1', 'metadata': {'name': 'live-test', 'namespace': 'ims-gc7hd', 'selfLink': '/api/v1/namespaces/ims-gc7hd/pods/live-test', 'uid': 'bebc59b7-ad08-4316-b33c-15b0a8b7cf0b', 'resourceVersion': '1802469', 'creationTimestamp': '2020-08-28T16:02:42Z', 'managedFields': [{'manager': 'OpenAPI-Generator', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2020-08-28T16:02:42Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:spec': {'f:containers': {'k:{"name":"live-test"}': {'.': {}, 'f:command': {}, 'f:image': {}, 'f:imagePullPolicy': {}, 'f:name': {}, 'f:resources': {}, 'f:terminationMessagePath': {}, 'f:terminationMessagePolicy': {}}}, 'f:dnsPolicy': {}, 'f:enableServiceLinks': {}, 'f:restartPolicy': {}, 'f:schedulerName': {}, 'f:securityContext': {}, 'f:terminationGracePeriodSeconds': {}}}}]}, 'spec': {'volumes': [{'name': 'default-token-52qls', 'secret': {'secretName': 'default-token-52qls', 'defaultMode': 420}}], 'containers': [{'name': 'live-test', 'image': 'ollivier/clearwater-live-test:latest', 'command': ['rake', 'test[ims-gc7hd.svc.cluster.local]', 'PROXY=bono.ims-gc7hd.svc.cluster.local', 'ELLIS=ellis.ims-gc7hd.svc.cluster.local', 'SIGNUP_CODE=secret', '--trace'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}], 'restartPolicy': 'Never', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'serviceAccountName': 'default', 'serviceAccount': 'default', 'nodeName': 'kali-worker', 'securityContext': {}, 'schedulerName': 'default-scheduler', 'tolerations': [{'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}, {'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}], 'priority': 0, 'enableServiceLinks': True}, 'status': {'phase': 'Pending', 'conditions': [{'type': 'PodScheduled', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T16:02:42Z'}], 'qosClass': 'BestEffort'}}} 2020-08-28 16:02:46,903 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 16, 2, 42, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields': None, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2020, 8, 28, 16, 2, 42, tzinfo=tzlocal())}, {'api_version': 'v1', 'fields': None, 'manager': 'kubelet', 'operation': 'Update', 'time': datetime.datetime(2020, 8, 28, 16, 2, 43, tzinfo=tzlocal())}], 'name': 'live-test', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1802472', 'self_link': '/api/v1/namespaces/ims-gc7hd/pods/live-test', 'uid': 'bebc59b7-ad08-4316-b33c-15b0a8b7cf0b'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': ['rake', 'test[ims-gc7hd.svc.cluster.local]', 'PROXY=bono.ims-gc7hd.svc.cluster.local', 'ELLIS=ellis.ims-gc7hd.svc.cluster.local', 'SIGNUP_CODE=secret', '--trace'], 'env': None, 'env_from': None, 'image': 'ollivier/clearwater-live-test:latest', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': None, 'name': 'live-test', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': None, 'init_containers': None, 'node_name': 'kali-worker', 'node_selector': None, 'preemption_policy': None, 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Never', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-52qls', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-52qls'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 16, 2, 43, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 16, 2, 43, tzinfo=tzlocal()), 'message': 'containers with unready status: ' '[live-test]', 'reason': 'ContainersNotReady', 'status': 'False', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 16, 2, 43, tzinfo=tzlocal()), 'message': 'containers with unready status: ' '[live-test]', 'reason': 'ContainersNotReady', 'status': 'False', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 16, 2, 42, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': None, 'image': 'ollivier/clearwater-live-test:latest', 'image_id': '', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'live-test', 'ready': False, 'restart_count': 0, 'state': {'running': None, 'terminated': None, 'waiting': {'message': None, 'reason': 'ContainerCreating'}}}], 'host_ip': '172.18.0.15', 'init_container_statuses': None, 'message': None, 'nominated_node_name': None, 'phase': 'Pending', 'pod_ip': None, 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 8, 28, 16, 2, 43, tzinfo=tzlocal())}}, 'raw_object': {'kind': 'Pod', 'apiVersion': 'v1', 'metadata': {'name': 'live-test', 'namespace': 'ims-gc7hd', 'selfLink': '/api/v1/namespaces/ims-gc7hd/pods/live-test', 'uid': 'bebc59b7-ad08-4316-b33c-15b0a8b7cf0b', 'resourceVersion': '1802472', 'creationTimestamp': '2020-08-28T16:02:42Z', 'managedFields': [{'manager': 'OpenAPI-Generator', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2020-08-28T16:02:42Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:spec': {'f:containers': {'k:{"name":"live-test"}': {'.': {}, 'f:command': {}, 'f:image': {}, 'f:imagePullPolicy': {}, 'f:name': {}, 'f:resources': {}, 'f:terminationMessagePath': {}, 'f:terminationMessagePolicy': {}}}, 'f:dnsPolicy': {}, 'f:enableServiceLinks': {}, 'f:restartPolicy': {}, 'f:schedulerName': {}, 'f:securityContext': {}, 'f:terminationGracePeriodSeconds': {}}}}, {'manager': 'kubelet', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2020-08-28T16:02:43Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:status': {'f:conditions': {'k:{"type":"ContainersReady"}': {'.': {}, 'f:lastProbeTime': {}, 'f:lastTransitionTime': {}, 'f:message': {}, 'f:reason': {}, 'f:status': {}, 'f:type': {}}, 'k:{"type":"Initialized"}': {'.': {}, 'f:lastProbeTime': {}, 'f:lastTransitionTime': {}, 'f:status': {}, 'f:type': {}}, 'k:{"type":"Ready"}': {'.': {}, 'f:lastProbeTime': {}, 'f:lastTransitionTime': {}, 'f:message': {}, 'f:reason': {}, 'f:status': {}, 'f:type': {}}}, 'f:containerStatuses': {}, 'f:hostIP': {}, 'f:startTime': {}}}}]}, 'spec': {'volumes': [{'name': 'default-token-52qls', 'secret': {'secretName': 'default-token-52qls', 'defaultMode': 420}}], 'containers': [{'name': 'live-test', 'image': 'ollivier/clearwater-live-test:latest', 'command': ['rake', 'test[ims-gc7hd.svc.cluster.local]', 'PROXY=bono.ims-gc7hd.svc.cluster.local', 'ELLIS=ellis.ims-gc7hd.svc.cluster.local', 'SIGNUP_CODE=secret', '--trace'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}], 'restartPolicy': 'Never', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'serviceAccountName': 'default', 'serviceAccount': 'default', 'nodeName': 'kali-worker', 'securityContext': {}, 'schedulerName': 'default-scheduler', 'tolerations': [{'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}, {'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}], 'priority': 0, 'enableServiceLinks': True}, 'status': {'phase': 'Pending', 'conditions': [{'type': 'Initialized', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T16:02:43Z'}, {'type': 'Ready', 'status': 'False', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T16:02:43Z', 'reason': 'ContainersNotReady', 'message': 'containers with unready status: [live-test]'}, {'type': 'ContainersReady', 'status': 'False', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T16:02:43Z', 'reason': 'ContainersNotReady', 'message': 'containers with unready status: [live-test]'}, {'type': 'PodScheduled', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T16:02:42Z'}], 'hostIP': '172.18.0.15', 'startTime': '2020-08-28T16:02:43Z', 'containerStatuses': [{'name': 'live-test', 'state': {'waiting': {'reason': 'ContainerCreating'}}, 'lastState': {}, 'ready': False, 'restartCount': 0, 'image': 'ollivier/clearwater-live-test:latest', 'imageID': '', 'started': False}], 'qosClass': 'BestEffort'}}} 2020-08-28 16:02:48,262 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 16, 2, 42, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields': None, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2020, 8, 28, 16, 2, 42, tzinfo=tzlocal())}, {'api_version': 'v1', 'fields': None, 'manager': 'kubelet', 'operation': 'Update', 'time': datetime.datetime(2020, 8, 28, 16, 2, 48, tzinfo=tzlocal())}], 'name': 'live-test', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1802490', 'self_link': '/api/v1/namespaces/ims-gc7hd/pods/live-test', 'uid': 'bebc59b7-ad08-4316-b33c-15b0a8b7cf0b'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': ['rake', 'test[ims-gc7hd.svc.cluster.local]', 'PROXY=bono.ims-gc7hd.svc.cluster.local', 'ELLIS=ellis.ims-gc7hd.svc.cluster.local', 'SIGNUP_CODE=secret', '--trace'], 'env': None, 'env_from': None, 'image': 'ollivier/clearwater-live-test:latest', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': None, 'name': 'live-test', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': None, 'init_containers': None, 'node_name': 'kali-worker', 'node_selector': None, 'preemption_policy': None, 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Never', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-52qls', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-52qls'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 16, 2, 43, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 16, 2, 48, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 16, 2, 48, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 16, 2, 42, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://d1167020ffa3f30cb07378fbd8da57e453fb249608265acd785095a708d80ba3', 'image': 'docker.io/ollivier/clearwater-live-test:latest', 'image_id': 'docker.io/ollivier/clearwater-live-test@sha256:77e928c23a5942aa681646be96dfb5897efe17b1e8676e8e94003ad08891b881', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'live-test', 'ready': True, 'restart_count': 0, 'state': {'running': {'started_at': datetime.datetime(2020, 8, 28, 16, 2, 47, tzinfo=tzlocal())}, 'terminated': None, 'waiting': None}}], 'host_ip': '172.18.0.15', 'init_container_statuses': None, 'message': None, 'nominated_node_name': None, 'phase': 'Running', 'pod_ip': '10.244.1.32', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 8, 28, 16, 2, 43, tzinfo=tzlocal())}}, 'raw_object': {'kind': 'Pod', 'apiVersion': 'v1', 'metadata': {'name': 'live-test', 'namespace': 'ims-gc7hd', 'selfLink': '/api/v1/namespaces/ims-gc7hd/pods/live-test', 'uid': 'bebc59b7-ad08-4316-b33c-15b0a8b7cf0b', 'resourceVersion': '1802490', 'creationTimestamp': '2020-08-28T16:02:42Z', 'managedFields': [{'manager': 'OpenAPI-Generator', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2020-08-28T16:02:42Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:spec': {'f:containers': {'k:{"name":"live-test"}': {'.': {}, 'f:command': {}, 'f:image': {}, 'f:imagePullPolicy': {}, 'f:name': {}, 'f:resources': {}, 'f:terminationMessagePath': {}, 'f:terminationMessagePolicy': {}}}, 'f:dnsPolicy': {}, 'f:enableServiceLinks': {}, 'f:restartPolicy': {}, 'f:schedulerName': {}, 'f:securityContext': {}, 'f:terminationGracePeriodSeconds': {}}}}, {'manager': 'kubelet', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2020-08-28T16:02:48Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:status': {'f:conditions': {'k:{"type":"ContainersReady"}': {'.': {}, 'f:lastProbeTime': {}, 'f:lastTransitionTime': {}, 'f:status': {}, 'f:type': {}}, 'k:{"type":"Initialized"}': {'.': {}, 'f:lastProbeTime': {}, 'f:lastTransitionTime': {}, 'f:status': {}, 'f:type': {}}, 'k:{"type":"Ready"}': {'.': {}, 'f:lastProbeTime': {}, 'f:lastTransitionTime': {}, 'f:status': {}, 'f:type': {}}}, 'f:containerStatuses': {}, 'f:hostIP': {}, 'f:phase': {}, 'f:podIP': {}, 'f:podIPs': {'.': {}, 'k:{"ip":"10.244.1.32"}': {'.': {}, 'f:ip': {}}}, 'f:startTime': {}}}}]}, 'spec': {'volumes': [{'name': 'default-token-52qls', 'secret': {'secretName': 'default-token-52qls', 'defaultMode': 420}}], 'containers': [{'name': 'live-test', 'image': 'ollivier/clearwater-live-test:latest', 'command': ['rake', 'test[ims-gc7hd.svc.cluster.local]', 'PROXY=bono.ims-gc7hd.svc.cluster.local', 'ELLIS=ellis.ims-gc7hd.svc.cluster.local', 'SIGNUP_CODE=secret', '--trace'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}], 'restartPolicy': 'Never', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'serviceAccountName': 'default', 'serviceAccount': 'default', 'nodeName': 'kali-worker', 'securityContext': {}, 'schedulerName': 'default-scheduler', 'tolerations': [{'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}, {'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}], 'priority': 0, 'enableServiceLinks': True}, 'status': {'phase': 'Running', 'conditions': [{'type': 'Initialized', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T16:02:43Z'}, {'type': 'Ready', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T16:02:48Z'}, {'type': 'ContainersReady', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T16:02:48Z'}, {'type': 'PodScheduled', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T16:02:42Z'}], 'hostIP': '172.18.0.15', 'podIP': '10.244.1.32', 'podIPs': [{'ip': '10.244.1.32'}], 'startTime': '2020-08-28T16:02:43Z', 'containerStatuses': [{'name': 'live-test', 'state': {'running': {'startedAt': '2020-08-28T16:02:47Z'}}, 'lastState': {}, 'ready': True, 'restartCount': 0, 'image': 'docker.io/ollivier/clearwater-live-test:latest', 'imageID': 'docker.io/ollivier/clearwater-live-test@sha256:77e928c23a5942aa681646be96dfb5897efe17b1e8676e8e94003ad08891b881', 'containerID': 'containerd://d1167020ffa3f30cb07378fbd8da57e453fb249608265acd785095a708d80ba3', 'started': True}], 'qosClass': 'BestEffort'}}} 2020-08-28 16:06:07,232 - functest_kubernetes.ims.ims - DEBUG - {'type': 'MODIFIED', 'object': {'api_version': 'v1', 'kind': 'Pod', 'metadata': {'annotations': None, 'cluster_name': None, 'creation_timestamp': datetime.datetime(2020, 8, 28, 16, 2, 42, tzinfo=tzlocal()), 'deletion_grace_period_seconds': None, 'deletion_timestamp': None, 'finalizers': None, 'generate_name': None, 'generation': None, 'initializers': None, 'labels': None, 'managed_fields': [{'api_version': 'v1', 'fields': None, 'manager': 'OpenAPI-Generator', 'operation': 'Update', 'time': datetime.datetime(2020, 8, 28, 16, 2, 42, tzinfo=tzlocal())}, {'api_version': 'v1', 'fields': None, 'manager': 'kubelet', 'operation': 'Update', 'time': datetime.datetime(2020, 8, 28, 16, 6, 6, tzinfo=tzlocal())}], 'name': 'live-test', 'namespace': 'ims-gc7hd', 'owner_references': None, 'resource_version': '1803021', 'self_link': '/api/v1/namespaces/ims-gc7hd/pods/live-test', 'uid': 'bebc59b7-ad08-4316-b33c-15b0a8b7cf0b'}, 'spec': {'active_deadline_seconds': None, 'affinity': None, 'automount_service_account_token': None, 'containers': [{'args': None, 'command': ['rake', 'test[ims-gc7hd.svc.cluster.local]', 'PROXY=bono.ims-gc7hd.svc.cluster.local', 'ELLIS=ellis.ims-gc7hd.svc.cluster.local', 'SIGNUP_CODE=secret', '--trace'], 'env': None, 'env_from': None, 'image': 'ollivier/clearwater-live-test:latest', 'image_pull_policy': 'Always', 'lifecycle': None, 'liveness_probe': None, 'name': 'live-test', 'ports': None, 'readiness_probe': None, 'resources': {'limits': None, 'requests': None}, 'security_context': None, 'stdin': None, 'stdin_once': None, 'termination_message_path': '/dev/termination-log', 'termination_message_policy': 'File', 'tty': None, 'volume_devices': None, 'volume_mounts': [{'mount_path': '/var/run/secrets/kubernetes.io/serviceaccount', 'mount_propagation': None, 'name': 'default-token-52qls', 'read_only': True, 'sub_path': None, 'sub_path_expr': None}], 'working_dir': None}], 'dns_config': None, 'dns_policy': 'ClusterFirst', 'enable_service_links': True, 'host_aliases': None, 'host_ipc': None, 'host_network': None, 'host_pid': None, 'hostname': None, 'image_pull_secrets': None, 'init_containers': None, 'node_name': 'kali-worker', 'node_selector': None, 'preemption_policy': None, 'priority': 0, 'priority_class_name': None, 'readiness_gates': None, 'restart_policy': 'Never', 'runtime_class_name': None, 'scheduler_name': 'default-scheduler', 'security_context': {'fs_group': None, 'run_as_group': None, 'run_as_non_root': None, 'run_as_user': None, 'se_linux_options': None, 'supplemental_groups': None, 'sysctls': None, 'windows_options': None}, 'service_account': 'default', 'service_account_name': 'default', 'share_process_namespace': None, 'subdomain': None, 'termination_grace_period_seconds': 30, 'tolerations': [{'effect': 'NoExecute', 'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}, {'effect': 'NoExecute', 'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'toleration_seconds': 300, 'value': None}], 'volumes': [{'aws_elastic_block_store': None, 'azure_disk': None, 'azure_file': None, 'cephfs': None, 'cinder': None, 'config_map': None, 'csi': None, 'downward_api': None, 'empty_dir': None, 'fc': None, 'flex_volume': None, 'flocker': None, 'gce_persistent_disk': None, 'git_repo': None, 'glusterfs': None, 'host_path': None, 'iscsi': None, 'name': 'default-token-52qls', 'nfs': None, 'persistent_volume_claim': None, 'photon_persistent_disk': None, 'portworx_volume': None, 'projected': None, 'quobyte': None, 'rbd': None, 'scale_io': None, 'secret': {'default_mode': 420, 'items': None, 'optional': None, 'secret_name': 'default-token-52qls'}, 'storageos': None, 'vsphere_volume': None}]}, 'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 16, 2, 43, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'Initialized'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 16, 6, 6, tzinfo=tzlocal()), 'message': 'containers with unready status: ' '[live-test]', 'reason': 'ContainersNotReady', 'status': 'False', 'type': 'Ready'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 16, 6, 6, tzinfo=tzlocal()), 'message': 'containers with unready status: ' '[live-test]', 'reason': 'ContainersNotReady', 'status': 'False', 'type': 'ContainersReady'}, {'last_probe_time': None, 'last_transition_time': datetime.datetime(2020, 8, 28, 16, 2, 42, tzinfo=tzlocal()), 'message': None, 'reason': None, 'status': 'True', 'type': 'PodScheduled'}], 'container_statuses': [{'container_id': 'containerd://d1167020ffa3f30cb07378fbd8da57e453fb249608265acd785095a708d80ba3', 'image': 'docker.io/ollivier/clearwater-live-test:latest', 'image_id': 'docker.io/ollivier/clearwater-live-test@sha256:77e928c23a5942aa681646be96dfb5897efe17b1e8676e8e94003ad08891b881', 'last_state': {'running': None, 'terminated': None, 'waiting': None}, 'name': 'live-test', 'ready': False, 'restart_count': 0, 'state': {'running': None, 'terminated': {'container_id': 'containerd://d1167020ffa3f30cb07378fbd8da57e453fb249608265acd785095a708d80ba3', 'exit_code': 1, 'finished_at': datetime.datetime(2020, 8, 28, 16, 6, 5, tzinfo=tzlocal()), 'message': None, 'reason': 'Error', 'signal': None, 'started_at': datetime.datetime(2020, 8, 28, 16, 2, 47, tzinfo=tzlocal())}, 'waiting': None}}], 'host_ip': '172.18.0.15', 'init_container_statuses': None, 'message': None, 'nominated_node_name': None, 'phase': 'Failed', 'pod_ip': '10.244.1.32', 'qos_class': 'BestEffort', 'reason': None, 'start_time': datetime.datetime(2020, 8, 28, 16, 2, 43, tzinfo=tzlocal())}}, 'raw_object': {'kind': 'Pod', 'apiVersion': 'v1', 'metadata': {'name': 'live-test', 'namespace': 'ims-gc7hd', 'selfLink': '/api/v1/namespaces/ims-gc7hd/pods/live-test', 'uid': 'bebc59b7-ad08-4316-b33c-15b0a8b7cf0b', 'resourceVersion': '1803021', 'creationTimestamp': '2020-08-28T16:02:42Z', 'managedFields': [{'manager': 'OpenAPI-Generator', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2020-08-28T16:02:42Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:spec': {'f:containers': {'k:{"name":"live-test"}': {'.': {}, 'f:command': {}, 'f:image': {}, 'f:imagePullPolicy': {}, 'f:name': {}, 'f:resources': {}, 'f:terminationMessagePath': {}, 'f:terminationMessagePolicy': {}}}, 'f:dnsPolicy': {}, 'f:enableServiceLinks': {}, 'f:restartPolicy': {}, 'f:schedulerName': {}, 'f:securityContext': {}, 'f:terminationGracePeriodSeconds': {}}}}, {'manager': 'kubelet', 'operation': 'Update', 'apiVersion': 'v1', 'time': '2020-08-28T16:06:06Z', 'fieldsType': 'FieldsV1', 'fieldsV1': {'f:status': {'f:conditions': {'k:{"type":"ContainersReady"}': {'.': {}, 'f:lastProbeTime': {}, 'f:lastTransitionTime': {}, 'f:message': {}, 'f:reason': {}, 'f:status': {}, 'f:type': {}}, 'k:{"type":"Initialized"}': {'.': {}, 'f:lastProbeTime': {}, 'f:lastTransitionTime': {}, 'f:status': {}, 'f:type': {}}, 'k:{"type":"Ready"}': {'.': {}, 'f:lastProbeTime': {}, 'f:lastTransitionTime': {}, 'f:message': {}, 'f:reason': {}, 'f:status': {}, 'f:type': {}}}, 'f:containerStatuses': {}, 'f:hostIP': {}, 'f:phase': {}, 'f:podIP': {}, 'f:podIPs': {'.': {}, 'k:{"ip":"10.244.1.32"}': {'.': {}, 'f:ip': {}}}, 'f:startTime': {}}}}]}, 'spec': {'volumes': [{'name': 'default-token-52qls', 'secret': {'secretName': 'default-token-52qls', 'defaultMode': 420}}], 'containers': [{'name': 'live-test', 'image': 'ollivier/clearwater-live-test:latest', 'command': ['rake', 'test[ims-gc7hd.svc.cluster.local]', 'PROXY=bono.ims-gc7hd.svc.cluster.local', 'ELLIS=ellis.ims-gc7hd.svc.cluster.local', 'SIGNUP_CODE=secret', '--trace'], 'resources': {}, 'volumeMounts': [{'name': 'default-token-52qls', 'readOnly': True, 'mountPath': '/var/run/secrets/kubernetes.io/serviceaccount'}], 'terminationMessagePath': '/dev/termination-log', 'terminationMessagePolicy': 'File', 'imagePullPolicy': 'Always'}], 'restartPolicy': 'Never', 'terminationGracePeriodSeconds': 30, 'dnsPolicy': 'ClusterFirst', 'serviceAccountName': 'default', 'serviceAccount': 'default', 'nodeName': 'kali-worker', 'securityContext': {}, 'schedulerName': 'default-scheduler', 'tolerations': [{'key': 'node.kubernetes.io/not-ready', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}, {'key': 'node.kubernetes.io/unreachable', 'operator': 'Exists', 'effect': 'NoExecute', 'tolerationSeconds': 300}], 'priority': 0, 'enableServiceLinks': True}, 'status': {'phase': 'Failed', 'conditions': [{'type': 'Initialized', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T16:02:43Z'}, {'type': 'Ready', 'status': 'False', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T16:06:06Z', 'reason': 'ContainersNotReady', 'message': 'containers with unready status: [live-test]'}, {'type': 'ContainersReady', 'status': 'False', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T16:06:06Z', 'reason': 'ContainersNotReady', 'message': 'containers with unready status: [live-test]'}, {'type': 'PodScheduled', 'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': '2020-08-28T16:02:42Z'}], 'hostIP': '172.18.0.15', 'podIP': '10.244.1.32', 'podIPs': [{'ip': '10.244.1.32'}], 'startTime': '2020-08-28T16:02:43Z', 'containerStatuses': [{'name': 'live-test', 'state': {'terminated': {'exitCode': 1, 'reason': 'Error', 'startedAt': '2020-08-28T16:02:47Z', 'finishedAt': '2020-08-28T16:06:05Z', 'containerID': 'containerd://d1167020ffa3f30cb07378fbd8da57e453fb249608265acd785095a708d80ba3'}}, 'lastState': {}, 'ready': False, 'restartCount': 0, 'image': 'docker.io/ollivier/clearwater-live-test:latest', 'imageID': 'docker.io/ollivier/clearwater-live-test@sha256:77e928c23a5942aa681646be96dfb5897efe17b1e8676e8e94003ad08891b881', 'containerID': 'containerd://d1167020ffa3f30cb07378fbd8da57e453fb249608265acd785095a708d80ba3', 'started': False}], 'qosClass': 'BestEffort'}}} 2020-08-28 16:06:08,278 - kubernetes.client.rest - DEBUG - response body: ** Invoke test (first_time) ** Execute test /usr/lib/ruby/gems/2.6.0/gems/httpi-2.4.0/lib/httpi/auth/ssl.rb:13: warning: constant OpenSSL::SSL::SSLContext::METHODS is deprecated # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' Basic Call - Mainline (TCP) - (6505550201, 6505550487) Passed Basic Call - SDP (TCP) - (6505550882, 6505550684) Passed Basic Call - Tel URIs (TCP) - (6505550505, 6505550847) Passed Basic Call - Unknown number (TCP) - (6505550669, 6505550655) Passed Basic Call - Rejected by remote endpoint (TCP) - (6505550289, 6505550893) Passed Basic Call - Messages - Pager model (TCP) - (6505550287, 6505550781) Passed Basic Call - Pracks (TCP) - (6505550960, 6505550261) Passed Basic Registration (TCP) - (6505550621) Passed Emergency Registration (TCP) - Skipped (Not running with emergency registrations) - Call with EMERGENCY_REG=Y to run test Multiple Identities (TCP) - (6505550943, 6505550160) Passed Call Barring - Outbound Rejection (TCP) - (6505550726, 6505550328) Passed Call Barring - Allow non-international call (TCP) - Skipped (No PSTN support) - Call with PSTN=true to run test Call Barring - Reject international call (TCP) - Skipped (No PSTN support) - Call with PSTN=true to run test Call Barring - Inbound Rejection (TCP) - (6505550046, 6505550075) Passed Call Diversion AS - Unconditional (TCP) - Skipped (No Call Diversion AS hostname provided) - Call with CDIV_AS= Call Diversion AS - Busy (TCP) - Skipped (No Call Diversion AS hostname provided) - Call with CDIV_AS= Call Diversion AS - No answer (TCP) - Skipped (No Call Diversion AS hostname provided) - Call with CDIV_AS= Call Diversion - Not registered (TCP) - (6505550343, 6505550696, 6505550836) Passed Call Diversion - Not reachable (not registered) (TCP) - Failed RuntimeError thrown: - Account creation failed with HTTP code 502, body {"status": 502, "message": "Bad Gateway", "reason": "Upstream request failed", "detail": {"Upstream error": "503", "Upstream URL": "http://homestead-prov.ims-gc7hd.svc.cluster.local:8889/irs/f8115a69-5bb3-4e3c-9024-9edfc6bde8a6/service_profiles/384b926c-08c9-4bd3-a9df-3c6371b361b9/public_ids/sip%3A6505550125%40ims-gc7hd.svc.cluster.local"}, "error": true} - /src/vims-test/lib/ellis.rb:193:in `rescue in get_number' - /src/vims-test/lib/ellis.rb:187:in `get_number' - /src/vims-test/lib/ellis.rb:71:in `initialize' - /src/vims-test/lib/test-definition.rb:354:in `new' - /src/vims-test/lib/test-definition.rb:354:in `provision_line' - /src/vims-test/lib/test-definition.rb:181:in `add_endpoint' - /src/vims-test/lib/tests/call-diversion.rb:101:in `block in ' - /src/vims-test/lib/test-definition.rb:256:in `run' - /src/vims-test/lib/test-definition.rb:126:in `block (2 levels) in run_all' - /src/vims-test/lib/test-definition.rb:112:in `collect' - /src/vims-test/lib/test-definition.rb:112:in `block in run_all' - /src/vims-test/lib/test-definition.rb:111:in `each' - /src/vims-test/lib/test-definition.rb:111:in `run_all' - /src/vims-test/lib/live-test.rb:23:in `run_tests' - /src/vims-test/Rakefile:18:in `block in ' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/task.rb:240:in `block in execute' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/task.rb:235:in `each' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/task.rb:235:in `execute' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/task.rb:179:in `block in invoke_with_call_chain' - /usr/lib/ruby/2.6.0/monitor.rb:235:in `mon_synchronize' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/task.rb:172:in `invoke_with_call_chain' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/task.rb:165:in `invoke' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:150:in `invoke_task' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:106:in `block (2 levels) in top_level' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:106:in `each' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:106:in `block in top_level' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:115:in `run_with_threads' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:100:in `top_level' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:78:in `block in run' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:176:in `standard_exception_handling' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:75:in `run' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/bin/rake:33:in `' - /usr/bin/rake:23:in `load' - /usr/bin/rake:23:in `
' Call Diversion - Not reachable (408) (TCP) - (6505550883, 6505550797, 6505550805) Passed Call Diversion - Not reachable (503) (TCP) - (6505550085, 6505550893, 6505550830) Passed Call Diversion - Not reachable (500) (TCP) - (6505550140, 6505550715, 6505550812) Passed Call Diversion - Busy (TCP) - (6505550283, 6505550191, 6505550890) Passed Call Diversion - Unconditional (TCP) - (6505550506, 6505550283, 6505550549) Passed Call Diversion - No answer (TCP) - (6505550465, 6505550566, 6505550552) Passed Call Diversion - Bad target URI (TCP) - (6505550944, 6505550943) Passed Call Diversion - Audio-only call (TCP) - (6505550189, 6505550672, 6505550898, 6505550918) Passed Call Diversion - Audio-video call (TCP) - (6505550053, 6505550300, 6505550336, 6505550595) Passed Call Forking - Mainline (TCP) - (6505550576, 6505550720) Passed Call Forking - Endpoint offline (TCP) - (6505550606, 6505550232) Passed Call Forking - Endpoint offline while ringing (TCP) - (6505550806, 6505550516) Passed Call Waiting - Accepted (TCP) - (6505550262, 6505550701, 6505550208) Passed Call Waiting - Cancelled (TCP) - (6505550353, 6505550868, 6505550946) Passed CANCEL - Mainline (TCP) - (6505550347, 6505550342) Passed Filtering - Accept-Contact (TCP) - (6505550735, 6505550853) Passed Accept-Contact with response (SFR 511195) (TCP) - (6505550141, 6505550685) Passed Filtering - Accept-Contact no match (TCP) - (6505550770, 6505550006) Passed Filtering - Accept-Contact negated match (TCP) - (6505550392, 6505550077) Passed Filtering - RFC3841 example (TCP) - (6505550210, 6505550247) Passed Filtering - Reject-Contact no match (TCP) - (6505550441, 6505550510) Passed Filtering - Reject-Contact match (TCP) - Failed RuntimeError thrown: - Account creation failed with HTTP code 502, body {"status": 502, "message": "Bad Gateway", "reason": "Upstream request failed", "detail": {"Upstream error": "503", "Upstream URL": "http://homestead-prov.ims-gc7hd.svc.cluster.local:8889/irs/1c0d002d-7862-4677-8bb1-2094e1833f46/service_profiles/4a1f208a-bc9d-4365-b35a-2064cbc9ca98/public_ids/sip%3A6505550962%40ims-gc7hd.svc.cluster.local"}, "error": true} - /src/vims-test/lib/ellis.rb:193:in `rescue in get_number' - /src/vims-test/lib/ellis.rb:187:in `get_number' - /src/vims-test/lib/ellis.rb:71:in `initialize' - /src/vims-test/lib/test-definition.rb:354:in `new' - /src/vims-test/lib/test-definition.rb:354:in `provision_line' - /src/vims-test/lib/test-definition.rb:181:in `add_endpoint' - /src/vims-test/lib/tests/contact-filtering.rb:270:in `block in ' - /src/vims-test/lib/test-definition.rb:256:in `run' - /src/vims-test/lib/test-definition.rb:126:in `block (2 levels) in run_all' - /src/vims-test/lib/test-definition.rb:112:in `collect' - /src/vims-test/lib/test-definition.rb:112:in `block in run_all' - /src/vims-test/lib/test-definition.rb:111:in `each' - /src/vims-test/lib/test-definition.rb:111:in `run_all' - /src/vims-test/lib/live-test.rb:23:in `run_tests' - /src/vims-test/Rakefile:18:in `block in ' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/task.rb:240:in `block in execute' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/task.rb:235:in `each' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/task.rb:235:in `execute' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/task.rb:179:in `block in invoke_with_call_chain' - /usr/lib/ruby/2.6.0/monitor.rb:235:in `mon_synchronize' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/task.rb:172:in `invoke_with_call_chain' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/task.rb:165:in `invoke' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:150:in `invoke_task' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:106:in `block (2 levels) in top_level' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:106:in `each' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:106:in `block in top_level' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:115:in `run_with_threads' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:100:in `top_level' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:78:in `block in run' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:176:in `standard_exception_handling' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:75:in `run' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/bin/rake:33:in `' - /usr/bin/rake:23:in `load' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' /usr/lib/ruby/gems/2.6.0/gems/nokogiri-1.6.6.2/lib/nokogiri/xml/document.rb:44: warning: constant ::Fixnum is deprecated # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' - /usr/bin/rake:23:in `
' Gemini - INVITE - VoIP device answers (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Mobile device answers (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - VoIP device rejects (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Mobile device rejects (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Mobile device rejects with a 480 (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Both reject, choose mobile response (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Both reject, choose VoIP response (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Successful call with GR (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Failed call with GR (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Successful call with Accept-Contact (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Failed call with Accept-Contact (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - SUBSCRIBE - Mobile Notifies (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - SUBSCRIBE - Joint 408 (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= GRUU - REGISTER - two bindings with and without GRUU (TCP) - (6505550364) Passed GRUU - REGISTER - binding suggested GRUU (TCP) - (6505550076) Passed GRUU - REGISTER - instance ID requires escaping (TCP) - (6505550373) Passed GRUU - REGISTER - three bindings with GRUUs (TCP) - (6505550986) Passed GRUU - Call - first endpoint GRUU as target (TCP) - (6505550590, 6505550762) Passed GRUU - Call - second endpoint GRUU as target (TCP) - (6505550790, 6505550416) Passed GRUU - Call - only GRUU as target (TCP) - (6505550458, 6505550281) Passed GRUU - Call - AoR as target (TCP) - (6505550159, 6505550494) Passed GRUU - Call - unknown GRUU as target (TCP) - (6505550959, 6505550400) Passed GRUU - Call - unknown GRUU as target - no GRUUs assigned (TCP) - (6505550366, 6505550570) Passed GRUU - Call - Reject-Contact interop (TCP) - (6505550271, 6505550032) Passed GRUU - Call - Accept-Contact interop (TCP) - (6505550428, 6505550740) Passed GRUU - Call - AoR with other param as target (TCP) - (6505550434, 6505550180) Passed GRUU - Call - GRUU with other param as target (TCP) - (6505550916, 6505550822) Passed ISC Interface - Terminating (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - Terminating (UDP AS) (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - Terminating Failed (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - Third-party Registration (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - Third-party Registration - implicit registration (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - Redirect (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - B2BUA (TCP) - Skipped (No hostname given) - Call with HOST_NAME= Live Call - Dial out to a real number (TCP) - Skipped (No live number given) - Call with LIVENUMBER= Memento - Incorrect Password (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Wrong Call List (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Basic Call (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Unknown Number (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Rejected Call (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Cancelled Call (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Privacy Call (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Barred Call (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Busy Call Forwarding (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Nonce-Count - Mainline (TCP) - Skipped (No nonce-count support) - Call with NONCE_COUNT=Y to run test Nonce-Count - Reject Re-Use (TCP) - Skipped (No nonce-count support) - Call with NONCE_COUNT=Y to run test Off-net calls - tel: URI (TCP) - Skipped (No off-net number given) - Call with OFF_NET_TEL= Off-net calls - sip: URI (TCP) - Skipped (No off-net number given) - Call with OFF_NET_TEL= SUBSCRIBE - reg-event (TCP) - (6505550429) Passed SUBSCRIBE - reg-event with a GRUU (TCP) - (6505550558) Passed SUBSCRIBE - Subscription timeout (TCP) - (6505550887) Passed SUBSCRIBE - Registration timeout (TCP) - Skipped (Not running with short registrations) - Call with SHORT_REG=Y to run test Multiple SUBSCRIBErs to one UE's reg-event (TCP) - (6505550250, 6505550246) Passed Message - URN URIs (TCP) - Skipped (No hostname given) - Call with HOST_NAME= Message - URN URIs to AS (TCP) - Skipped (No hostname given) - Call with HOST_NAME= 2 failures out of 96 tests run Call Diversion - Not reachable (not registered) (TCP) at 2020-08-28 16:03:25 +0000 Filtering - Reject-Contact match (TCP) at 2020-08-28 16:05:01 +0000 43 tests skipped Error logs, including Call-IDs of failed calls, are in the 'logfiles' directory 2020-08-28 16:06:08,280 - functest_kubernetes.ims.ims - INFO - ** Invoke test (first_time) ** Execute test /usr/lib/ruby/gems/2.6.0/gems/httpi-2.4.0/lib/httpi/auth/ssl.rb:13: warning: constant OpenSSL::SSL::SSLContext::METHODS is deprecated # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' Basic Call - Mainline (TCP) - (6505550201, 6505550487) Passed Basic Call - SDP (TCP) - (6505550882, 6505550684) Passed Basic Call - Tel URIs (TCP) - (6505550505, 6505550847) Passed Basic Call - Unknown number (TCP) - (6505550669, 6505550655) Passed Basic Call - Rejected by remote endpoint (TCP) - (6505550289, 6505550893) Passed Basic Call - Messages - Pager model (TCP) - (6505550287, 6505550781) Passed Basic Call - Pracks (TCP) - (6505550960, 6505550261) Passed Basic Registration (TCP) - (6505550621) Passed Emergency Registration (TCP) - Skipped (Not running with emergency registrations) - Call with EMERGENCY_REG=Y to run test Multiple Identities (TCP) - (6505550943, 6505550160) Passed Call Barring - Outbound Rejection (TCP) - (6505550726, 6505550328) Passed Call Barring - Allow non-international call (TCP) - Skipped (No PSTN support) - Call with PSTN=true to run test Call Barring - Reject international call (TCP) - Skipped (No PSTN support) - Call with PSTN=true to run test Call Barring - Inbound Rejection (TCP) - (6505550046, 6505550075) Passed Call Diversion AS - Unconditional (TCP) - Skipped (No Call Diversion AS hostname provided) - Call with CDIV_AS= Call Diversion AS - Busy (TCP) - Skipped (No Call Diversion AS hostname provided) - Call with CDIV_AS= Call Diversion AS - No answer (TCP) - Skipped (No Call Diversion AS hostname provided) - Call with CDIV_AS= Call Diversion - Not registered (TCP) - (6505550343, 6505550696, 6505550836) Passed Call Diversion - Not reachable (not registered) (TCP) - Failed RuntimeError thrown: - Account creation failed with HTTP code 502, body {"status": 502, "message": "Bad Gateway", "reason": "Upstream request failed", "detail": {"Upstream error": "503", "Upstream URL": "http://homestead-prov.ims-gc7hd.svc.cluster.local:8889/irs/f8115a69-5bb3-4e3c-9024-9edfc6bde8a6/service_profiles/384b926c-08c9-4bd3-a9df-3c6371b361b9/public_ids/sip%3A6505550125%40ims-gc7hd.svc.cluster.local"}, "error": true} - /src/vims-test/lib/ellis.rb:193:in `rescue in get_number' - /src/vims-test/lib/ellis.rb:187:in `get_number' - /src/vims-test/lib/ellis.rb:71:in `initialize' - /src/vims-test/lib/test-definition.rb:354:in `new' - /src/vims-test/lib/test-definition.rb:354:in `provision_line' - /src/vims-test/lib/test-definition.rb:181:in `add_endpoint' - /src/vims-test/lib/tests/call-diversion.rb:101:in `block in ' - /src/vims-test/lib/test-definition.rb:256:in `run' - /src/vims-test/lib/test-definition.rb:126:in `block (2 levels) in run_all' - /src/vims-test/lib/test-definition.rb:112:in `collect' - /src/vims-test/lib/test-definition.rb:112:in `block in run_all' - /src/vims-test/lib/test-definition.rb:111:in `each' - /src/vims-test/lib/test-definition.rb:111:in `run_all' - /src/vims-test/lib/live-test.rb:23:in `run_tests' - /src/vims-test/Rakefile:18:in `block in ' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/task.rb:240:in `block in execute' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/task.rb:235:in `each' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/task.rb:235:in `execute' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/task.rb:179:in `block in invoke_with_call_chain' - /usr/lib/ruby/2.6.0/monitor.rb:235:in `mon_synchronize' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/task.rb:172:in `invoke_with_call_chain' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/task.rb:165:in `invoke' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:150:in `invoke_task' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:106:in `block (2 levels) in top_level' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:106:in `each' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:106:in `block in top_level' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:115:in `run_with_threads' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:100:in `top_level' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:78:in `block in run' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:176:in `standard_exception_handling' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:75:in `run' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/bin/rake:33:in `' - /usr/bin/rake:23:in `load' - /usr/bin/rake:23:in `
' Call Diversion - Not reachable (408) (TCP) - (6505550883, 6505550797, 6505550805) Passed Call Diversion - Not reachable (503) (TCP) - (6505550085, 6505550893, 6505550830) Passed Call Diversion - Not reachable (500) (TCP) - (6505550140, 6505550715, 6505550812) Passed Call Diversion - Busy (TCP) - (6505550283, 6505550191, 6505550890) Passed Call Diversion - Unconditional (TCP) - (6505550506, 6505550283, 6505550549) Passed Call Diversion - No answer (TCP) - (6505550465, 6505550566, 6505550552) Passed Call Diversion - Bad target URI (TCP) - (6505550944, 6505550943) Passed Call Diversion - Audio-only call (TCP) - (6505550189, 6505550672, 6505550898, 6505550918) Passed Call Diversion - Audio-video call (TCP) - (6505550053, 6505550300, 6505550336, 6505550595) Passed Call Forking - Mainline (TCP) - (6505550576, 6505550720) Passed Call Forking - Endpoint offline (TCP) - (6505550606, 6505550232) Passed Call Forking - Endpoint offline while ringing (TCP) - (6505550806, 6505550516) Passed Call Waiting - Accepted (TCP) - (6505550262, 6505550701, 6505550208) Passed Call Waiting - Cancelled (TCP) - (6505550353, 6505550868, 6505550946) Passed CANCEL - Mainline (TCP) - (6505550347, 6505550342) Passed Filtering - Accept-Contact (TCP) - (6505550735, 6505550853) Passed Accept-Contact with response (SFR 511195) (TCP) - (6505550141, 6505550685) Passed Filtering - Accept-Contact no match (TCP) - (6505550770, 6505550006) Passed Filtering - Accept-Contact negated match (TCP) - (6505550392, 6505550077) Passed Filtering - RFC3841 example (TCP) - (6505550210, 6505550247) Passed Filtering - Reject-Contact no match (TCP) - (6505550441, 6505550510) Passed Filtering - Reject-Contact match (TCP) - Failed RuntimeError thrown: - Account creation failed with HTTP code 502, body {"status": 502, "message": "Bad Gateway", "reason": "Upstream request failed", "detail": {"Upstream error": "503", "Upstream URL": "http://homestead-prov.ims-gc7hd.svc.cluster.local:8889/irs/1c0d002d-7862-4677-8bb1-2094e1833f46/service_profiles/4a1f208a-bc9d-4365-b35a-2064cbc9ca98/public_ids/sip%3A6505550962%40ims-gc7hd.svc.cluster.local"}, "error": true} - /src/vims-test/lib/ellis.rb:193:in `rescue in get_number' - /src/vims-test/lib/ellis.rb:187:in `get_number' - /src/vims-test/lib/ellis.rb:71:in `initialize' - /src/vims-test/lib/test-definition.rb:354:in `new' - /src/vims-test/lib/test-definition.rb:354:in `provision_line' - /src/vims-test/lib/test-definition.rb:181:in `add_endpoint' - /src/vims-test/lib/tests/contact-filtering.rb:270:in `block in ' - /src/vims-test/lib/test-definition.rb:256:in `run' - /src/vims-test/lib/test-definition.rb:126:in `block (2 levels) in run_all' - /src/vims-test/lib/test-definition.rb:112:in `collect' - /src/vims-test/lib/test-definition.rb:112:in `block in run_all' - /src/vims-test/lib/test-definition.rb:111:in `each' - /src/vims-test/lib/test-definition.rb:111:in `run_all' - /src/vims-test/lib/live-test.rb:23:in `run_tests' - /src/vims-test/Rakefile:18:in `block in ' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/task.rb:240:in `block in execute' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/task.rb:235:in `each' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/task.rb:235:in `execute' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/task.rb:179:in `block in invoke_with_call_chain' - /usr/lib/ruby/2.6.0/monitor.rb:235:in `mon_synchronize' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/task.rb:172:in `invoke_with_call_chain' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/task.rb:165:in `invoke' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:150:in `invoke_task' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:106:in `block (2 levels) in top_level' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:106:in `each' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:106:in `block in top_level' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:115:in `run_with_threads' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:100:in `top_level' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:78:in `block in run' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:176:in `standard_exception_handling' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/lib/rake/application.rb:75:in `run' - /usr/lib/ruby/gems/2.6.0/gems/rake-10.4.2/bin/rake:33:in `' - /usr/bin/rake:23:in `load' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' /usr/lib/ruby/gems/2.6.0/gems/nokogiri-1.6.6.2/lib/nokogiri/xml/document.rb:44: warning: constant ::Fixnum is deprecated # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' # terminated with exception (report_on_exception is true): /src/vims-test/quaff/lib/sip_parser.rb:23:in `gets': stream closed in another thread (IOError) from /src/vims-test/quaff/lib/sip_parser.rb:23:in `parse_from_io' from /src/vims-test/quaff/lib/endpoint.rb:343:in `recv_msg_from_sock' from /src/vims-test/quaff/lib/endpoint.rb:338:in `block in recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:337:in `each' from /src/vims-test/quaff/lib/endpoint.rb:337:in `recv_msg' from /src/vims-test/quaff/lib/endpoint.rb:246:in `block in start' - /usr/bin/rake:23:in `
' Gemini - INVITE - VoIP device answers (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Mobile device answers (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - VoIP device rejects (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Mobile device rejects (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Mobile device rejects with a 480 (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Both reject, choose mobile response (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Both reject, choose VoIP response (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Successful call with GR (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Failed call with GR (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Successful call with Accept-Contact (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - INVITE - Failed call with Accept-Contact (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - SUBSCRIBE - Mobile Notifies (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= Gemini - SUBSCRIBE - Joint 408 (TCP) - Skipped (No gemini hostname provided) - Call with GEMINI= GRUU - REGISTER - two bindings with and without GRUU (TCP) - (6505550364) Passed GRUU - REGISTER - binding suggested GRUU (TCP) - (6505550076) Passed GRUU - REGISTER - instance ID requires escaping (TCP) - (6505550373) Passed GRUU - REGISTER - three bindings with GRUUs (TCP) - (6505550986) Passed GRUU - Call - first endpoint GRUU as target (TCP) - (6505550590, 6505550762) Passed GRUU - Call - second endpoint GRUU as target (TCP) - (6505550790, 6505550416) Passed GRUU - Call - only GRUU as target (TCP) - (6505550458, 6505550281) Passed GRUU - Call - AoR as target (TCP) - (6505550159, 6505550494) Passed GRUU - Call - unknown GRUU as target (TCP) - (6505550959, 6505550400) Passed GRUU - Call - unknown GRUU as target - no GRUUs assigned (TCP) - (6505550366, 6505550570) Passed GRUU - Call - Reject-Contact interop (TCP) - (6505550271, 6505550032) Passed GRUU - Call - Accept-Contact interop (TCP) - (6505550428, 6505550740) Passed GRUU - Call - AoR with other param as target (TCP) - (6505550434, 6505550180) Passed GRUU - Call - GRUU with other param as target (TCP) - (6505550916, 6505550822) Passed ISC Interface - Terminating (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - Terminating (UDP AS) (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - Terminating Failed (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - Third-party Registration (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - Third-party Registration - implicit registration (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - Redirect (TCP) - Skipped (No hostname given) - Call with HOST_NAME= ISC Interface - B2BUA (TCP) - Skipped (No hostname given) - Call with HOST_NAME= Live Call - Dial out to a real number (TCP) - Skipped (No live number given) - Call with LIVENUMBER= Memento - Incorrect Password (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Wrong Call List (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Basic Call (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Unknown Number (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Rejected Call (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Cancelled Call (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Privacy Call (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Barred Call (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Memento - Busy Call Forwarding (TCP) - Skipped (No memento hostnames provided) - Call with MEMENTO_SIP= and MEMENTO_HTTP= Nonce-Count - Mainline (TCP) - Skipped (No nonce-count support) - Call with NONCE_COUNT=Y to run test Nonce-Count - Reject Re-Use (TCP) - Skipped (No nonce-count support) - Call with NONCE_COUNT=Y to run test Off-net calls - tel: URI (TCP) - Skipped (No off-net number given) - Call with OFF_NET_TEL= Off-net calls - sip: URI (TCP) - Skipped (No off-net number given) - Call with OFF_NET_TEL= SUBSCRIBE - reg-event (TCP) - (6505550429) Passed SUBSCRIBE - reg-event with a GRUU (TCP) - (6505550558) Passed SUBSCRIBE - Subscription timeout (TCP) - (6505550887) Passed SUBSCRIBE - Registration timeout (TCP) - Skipped (Not running with short registrations) - Call with SHORT_REG=Y to run test Multiple SUBSCRIBErs to one UE's reg-event (TCP) - (6505550250, 6505550246) Passed Message - URN URIs (TCP) - Skipped (No hostname given) - Call with HOST_NAME= Message - URN URIs to AS (TCP) - Skipped (No hostname given) - Call with HOST_NAME= 2 failures out of 96 tests run Call Diversion - Not reachable (not registered) (TCP) at 2020-08-28 16:03:25 +0000 Filtering - Reject-Contact match (TCP) at 2020-08-28 16:05:01 +0000 43 tests skipped Error logs, including Call-IDs of failed calls, are in the 'logfiles' directory 2020-08-28 16:06:08,293 - xtesting.ci.run_tests - INFO - Test result: +-------------------+------------------+------------------+----------------+ | TEST CASE | PROJECT | DURATION | RESULT | +-------------------+------------------+------------------+----------------+ | k8s_vims | functest | 09:20 | FAIL | +-------------------+------------------+------------------+----------------+ 2020-08-28 16:06:08,320 - kubernetes.client.rest - DEBUG - response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success","details":{"name":"env-vars","kind":"configmaps","uid":"b2decaa3-a673-4992-9eb4-2cb8cacbe86a"}} 2020-08-28 16:06:08,321 - functest_kubernetes.ims.ims - DEBUG - delete_namespaced_config_map: {'api_version': 'v1', 'code': None, 'details': {'causes': None, 'group': None, 'kind': 'configmaps', 'name': 'env-vars', 'retry_after_seconds': None, 'uid': 'b2decaa3-a673-4992-9eb4-2cb8cacbe86a'}, 'kind': 'Status', 'message': None, 'metadata': {'_continue': None, 'remaining_item_count': None, 'resource_version': None, 'self_link': None}, 'reason': None, 'status': 'Success'} 2020-08-28 16:06:08,688 - kubernetes.client.rest - DEBUG - response body: {"kind":"Pod","apiVersion":"v1","metadata":{"name":"live-test","namespace":"ims-gc7hd","selfLink":"/api/v1/namespaces/ims-gc7hd/pods/live-test","uid":"bebc59b7-ad08-4316-b33c-15b0a8b7cf0b","resourceVersion":"1803027","creationTimestamp":"2020-08-28T16:02:42Z","deletionTimestamp":"2020-08-28T16:06:08Z","deletionGracePeriodSeconds":0,"managedFields":[{"manager":"OpenAPI-Generator","operation":"Update","apiVersion":"v1","time":"2020-08-28T16:02:42Z","fieldsType":"FieldsV1","fieldsV1":{"f:spec":{"f:containers":{"k:{\"name\":\"live-test\"}":{".":{},"f:command":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{},"f:terminationGracePeriodSeconds":{}}}},{"manager":"kubelet","operation":"Update","apiVersion":"v1","time":"2020-08-28T16:06:06Z","fieldsType":"FieldsV1","fieldsV1":{"f:status":{"f:conditions":{"k:{\"type\":\"ContainersReady\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Initialized\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:containerStatuses":{},"f:hostIP":{},"f:phase":{},"f:podIP":{},"f:podIPs":{".":{},"k:{\"ip\":\"10.244.1.32\"}":{".":{},"f:ip":{}}},"f:startTime":{}}}}]},"spec":{"volumes":[{"name":"default-token-52qls","secret":{"secretName":"default-token-52qls","defaultMode":420}}],"containers":[{"name":"live-test","image":"ollivier/clearwater-live-test:latest","command":["rake","test[ims-gc7hd.svc.cluster.local]","PROXY=bono.ims-gc7hd.svc.cluster.local","ELLIS=ellis.ims-gc7hd.svc.cluster.local","SIGNUP_CODE=secret","--trace"],"resources":{},"volumeMounts":[{"name":"default-token-52qls","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"Always"}],"restartPolicy":"Never","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"kali-worker","securityContext":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}],"priority":0,"enableServiceLinks":true},"status":{"phase":"Failed","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2020-08-28T16:02:43Z"},{"type":"Ready","status":"False","lastProbeTime":null,"lastTransitionTime":"2020-08-28T16:06:06Z","reason":"ContainersNotReady","message":"containers with unready status: [live-test]"},{"type":"ContainersReady","status":"False","lastProbeTime":null,"lastTransitionTime":"2020-08-28T16:06:06Z","reason":"ContainersNotReady","message":"containers with unready status: [live-test]"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2020-08-28T16:02:42Z"}],"hostIP":"172.18.0.15","podIP":"10.244.1.32","podIPs":[{"ip":"10.244.1.32"}],"startTime":"2020-08-28T16:02:43Z","containerStatuses":[{"name":"live-test","state":{"terminated":{"exitCode":1,"reason":"Error","startedAt":"2020-08-28T16:02:47Z","finishedAt":"2020-08-28T16:06:05Z","containerID":"containerd://d1167020ffa3f30cb07378fbd8da57e453fb249608265acd785095a708d80ba3"}},"lastState":{},"ready":false,"restartCount":0,"image":"docker.io/ollivier/clearwater-live-test:latest","imageID":"docker.io/ollivier/clearwater-live-test@sha256:77e928c23a5942aa681646be96dfb5897efe17b1e8676e8e94003ad08891b881","containerID":"containerd://d1167020ffa3f30cb07378fbd8da57e453fb249608265acd785095a708d80ba3","started":false}],"qosClass":"BestEffort"}} 2020-08-28 16:06:08,691 - functest_kubernetes.ims.ims - DEBUG - delete_namespaced_pod: {'api_version': 'v1', 'code': None, 'details': None, 'kind': 'Pod', 'message': None, 'metadata': {'_continue': None, 'remaining_item_count': None, 'resource_version': '1803027', 'self_link': '/api/v1/namespaces/ims-gc7hd/pods/live-test'}, 'reason': None, 'status': "{'phase': 'Failed', 'conditions': [{'type': 'Initialized', " "'status': 'True', 'lastProbeTime': None, 'lastTransitionTime': " "'2020-08-28T16:02:43Z'}, {'type': 'Ready', 'status': 'False', " "'lastProbeTime': None, 'lastTransitionTime': " "'2020-08-28T16:06:06Z', 'reason': 'ContainersNotReady', 'message': " "'containers with unready status: [live-test]'}, {'type': " "'ContainersReady', 'status': 'False', 'lastProbeTime': None, " "'lastTransitionTime': '2020-08-28T16:06:06Z', 'reason': " "'ContainersNotReady', 'message': 'containers with unready status: " "[live-test]'}, {'type': 'PodScheduled', 'status': 'True', " "'lastProbeTime': None, 'lastTransitionTime': " "'2020-08-28T16:02:42Z'}], 'hostIP': '172.18.0.15', 'podIP': " "'10.244.1.32', 'podIPs': [{'ip': '10.244.1.32'}], 'startTime': " "'2020-08-28T16:02:43Z', 'containerStatuses': [{'name': " "'live-test', 'state': {'terminated': {'exitCode': 1, 'reason': " "'Error', 'startedAt': '2020-08-28T16:02:47Z', 'finishedAt': " "'2020-08-28T16:06:05Z', 'containerID': " "'containerd://d1167020ffa3f30cb07378fbd8da57e453fb249608265acd785095a708d80ba3'}}, " "'lastState': {}, 'ready': False, 'restartCount': 0, 'image': " "'docker.io/ollivier/clearwater-live-test:latest', 'imageID': " "'docker.io/ollivier/clearwater-live-test@sha256:77e928c23a5942aa681646be96dfb5897efe17b1e8676e8e94003ad08891b881', " "'containerID': " "'containerd://d1167020ffa3f30cb07378fbd8da57e453fb249608265acd785095a708d80ba3', " "'started': False}], 'qosClass': 'BestEffort'}"} 2020-08-28 16:06:08,768 - kubernetes.client.rest - DEBUG - response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success","details":{"name":"astaire","group":"apps","kind":"deployments","uid":"a586a36b-c53d-4586-be2a-1620364a644b"}} 2020-08-28 16:06:08,771 - functest_kubernetes.ims.ims - DEBUG - delete_namespaced_deployment: {'api_version': 'v1', 'code': None, 'details': {'causes': None, 'group': 'apps', 'kind': 'deployments', 'name': 'astaire', 'retry_after_seconds': None, 'uid': 'a586a36b-c53d-4586-be2a-1620364a644b'}, 'kind': 'Status', 'message': None, 'metadata': {'_continue': None, 'remaining_item_count': None, 'resource_version': None, 'self_link': None}, 'reason': None, 'status': 'Success'} 2020-08-28 16:06:08,935 - kubernetes.client.rest - DEBUG - response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success","details":{"name":"astaire","kind":"services","uid":"d1b5ab72-d74f-4f5d-b2ad-a76ea9b870cf"}} 2020-08-28 16:06:08,937 - functest_kubernetes.ims.ims - DEBUG - delete_namespaced_service: {'api_version': 'v1', 'code': None, 'details': {'causes': None, 'group': None, 'kind': 'services', 'name': 'astaire', 'retry_after_seconds': None, 'uid': 'd1b5ab72-d74f-4f5d-b2ad-a76ea9b870cf'}, 'kind': 'Status', 'message': None, 'metadata': {'_continue': None, 'remaining_item_count': None, 'resource_version': None, 'self_link': None}, 'reason': None, 'status': 'Success'} 2020-08-28 16:06:09,066 - kubernetes.client.rest - DEBUG - response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success","details":{"name":"bono","group":"apps","kind":"deployments","uid":"d6bc69e0-8471-4b52-ae66-fd1f8e8bf787"}} 2020-08-28 16:06:09,068 - functest_kubernetes.ims.ims - DEBUG - delete_namespaced_deployment: {'api_version': 'v1', 'code': None, 'details': {'causes': None, 'group': 'apps', 'kind': 'deployments', 'name': 'bono', 'retry_after_seconds': None, 'uid': 'd6bc69e0-8471-4b52-ae66-fd1f8e8bf787'}, 'kind': 'Status', 'message': None, 'metadata': {'_continue': None, 'remaining_item_count': None, 'resource_version': None, 'self_link': None}, 'reason': None, 'status': 'Success'} 2020-08-28 16:06:09,318 - kubernetes.client.rest - DEBUG - response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success","details":{"name":"bono","kind":"services","uid":"20bb1f4d-2640-4e68-9db9-fbf06db037ec"}} 2020-08-28 16:06:09,320 - functest_kubernetes.ims.ims - DEBUG - delete_namespaced_service: {'api_version': 'v1', 'code': None, 'details': {'causes': None, 'group': None, 'kind': 'services', 'name': 'bono', 'retry_after_seconds': None, 'uid': '20bb1f4d-2640-4e68-9db9-fbf06db037ec'}, 'kind': 'Status', 'message': None, 'metadata': {'_continue': None, 'remaining_item_count': None, 'resource_version': None, 'self_link': None}, 'reason': None, 'status': 'Success'} 2020-08-28 16:06:09,885 - kubernetes.client.rest - DEBUG - response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success","details":{"name":"cassandra","group":"apps","kind":"deployments","uid":"dc3fbaac-7655-45e2-b0da-c94598a92d8b"}} 2020-08-28 16:06:09,887 - functest_kubernetes.ims.ims - DEBUG - delete_namespaced_deployment: {'api_version': 'v1', 'code': None, 'details': {'causes': None, 'group': 'apps', 'kind': 'deployments', 'name': 'cassandra', 'retry_after_seconds': None, 'uid': 'dc3fbaac-7655-45e2-b0da-c94598a92d8b'}, 'kind': 'Status', 'message': None, 'metadata': {'_continue': None, 'remaining_item_count': None, 'resource_version': None, 'self_link': None}, 'reason': None, 'status': 'Success'} 2020-08-28 16:06:12,886 - kubernetes.client.rest - DEBUG - response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success","details":{"name":"cassandra","kind":"services","uid":"3e0bf5b6-d35e-455c-a8e7-201a08d58707"}} 2020-08-28 16:06:12,889 - functest_kubernetes.ims.ims - DEBUG - delete_namespaced_service: {'api_version': 'v1', 'code': None, 'details': {'causes': None, 'group': None, 'kind': 'services', 'name': 'cassandra', 'retry_after_seconds': None, 'uid': '3e0bf5b6-d35e-455c-a8e7-201a08d58707'}, 'kind': 'Status', 'message': None, 'metadata': {'_continue': None, 'remaining_item_count': None, 'resource_version': None, 'self_link': None}, 'reason': None, 'status': 'Success'} 2020-08-28 16:06:13,402 - kubernetes.client.rest - DEBUG - response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success","details":{"name":"chronos","group":"apps","kind":"deployments","uid":"8911bd1e-c40e-4013-af3e-2e070733afb4"}} 2020-08-28 16:06:13,404 - functest_kubernetes.ims.ims - DEBUG - delete_namespaced_deployment: {'api_version': 'v1', 'code': None, 'details': {'causes': None, 'group': 'apps', 'kind': 'deployments', 'name': 'chronos', 'retry_after_seconds': None, 'uid': '8911bd1e-c40e-4013-af3e-2e070733afb4'}, 'kind': 'Status', 'message': None, 'metadata': {'_continue': None, 'remaining_item_count': None, 'resource_version': None, 'self_link': None}, 'reason': None, 'status': 'Success'} 2020-08-28 16:06:13,552 - kubernetes.client.rest - DEBUG - response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success","details":{"name":"chronos","kind":"services","uid":"baa01d7d-9efa-4652-b7a6-dc6c2fa21e30"}} 2020-08-28 16:06:13,554 - functest_kubernetes.ims.ims - DEBUG - delete_namespaced_service: {'api_version': 'v1', 'code': None, 'details': {'causes': None, 'group': None, 'kind': 'services', 'name': 'chronos', 'retry_after_seconds': None, 'uid': 'baa01d7d-9efa-4652-b7a6-dc6c2fa21e30'}, 'kind': 'Status', 'message': None, 'metadata': {'_continue': None, 'remaining_item_count': None, 'resource_version': None, 'self_link': None}, 'reason': None, 'status': 'Success'} 2020-08-28 16:06:13,662 - kubernetes.client.rest - DEBUG - response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success","details":{"name":"ellis","group":"apps","kind":"deployments","uid":"9d4959be-6eec-4d20-a139-c33010481c17"}} 2020-08-28 16:06:13,665 - functest_kubernetes.ims.ims - DEBUG - delete_namespaced_deployment: {'api_version': 'v1', 'code': None, 'details': {'causes': None, 'group': 'apps', 'kind': 'deployments', 'name': 'ellis', 'retry_after_seconds': None, 'uid': '9d4959be-6eec-4d20-a139-c33010481c17'}, 'kind': 'Status', 'message': None, 'metadata': {'_continue': None, 'remaining_item_count': None, 'resource_version': None, 'self_link': None}, 'reason': None, 'status': 'Success'} 2020-08-28 16:06:13,834 - kubernetes.client.rest - DEBUG - response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success","details":{"name":"ellis","kind":"services","uid":"a09acecf-b0a9-4287-9f96-2e82e68f0d5f"}} 2020-08-28 16:06:13,836 - functest_kubernetes.ims.ims - DEBUG - delete_namespaced_service: {'api_version': 'v1', 'code': None, 'details': {'causes': None, 'group': None, 'kind': 'services', 'name': 'ellis', 'retry_after_seconds': None, 'uid': 'a09acecf-b0a9-4287-9f96-2e82e68f0d5f'}, 'kind': 'Status', 'message': None, 'metadata': {'_continue': None, 'remaining_item_count': None, 'resource_version': None, 'self_link': None}, 'reason': None, 'status': 'Success'} 2020-08-28 16:06:14,040 - kubernetes.client.rest - DEBUG - response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success","details":{"name":"etcd","group":"apps","kind":"deployments","uid":"38f69988-86b5-423a-8e0b-b9f3382efc35"}} 2020-08-28 16:06:14,398 - functest_kubernetes.ims.ims - DEBUG - delete_namespaced_deployment: {'api_version': 'v1', 'code': None, 'details': {'causes': None, 'group': 'apps', 'kind': 'deployments', 'name': 'etcd', 'retry_after_seconds': None, 'uid': '38f69988-86b5-423a-8e0b-b9f3382efc35'}, 'kind': 'Status', 'message': None, 'metadata': {'_continue': None, 'remaining_item_count': None, 'resource_version': None, 'self_link': None}, 'reason': None, 'status': 'Success'} 2020-08-28 16:06:15,611 - kubernetes.client.rest - DEBUG - response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success","details":{"name":"etcd","kind":"services","uid":"3d35f292-6126-40fa-a013-acca8671c34b"}} 2020-08-28 16:06:15,710 - functest_kubernetes.ims.ims - DEBUG - delete_namespaced_service: {'api_version': 'v1', 'code': None, 'details': {'causes': None, 'group': None, 'kind': 'services', 'name': 'etcd', 'retry_after_seconds': None, 'uid': '3d35f292-6126-40fa-a013-acca8671c34b'}, 'kind': 'Status', 'message': None, 'metadata': {'_continue': None, 'remaining_item_count': None, 'resource_version': None, 'self_link': None}, 'reason': None, 'status': 'Success'} 2020-08-28 16:06:15,816 - kubernetes.client.rest - DEBUG - response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success","details":{"name":"homer","group":"apps","kind":"deployments","uid":"fef4c095-1313-4b44-b9d6-df772baa5d24"}} 2020-08-28 16:06:15,819 - functest_kubernetes.ims.ims - DEBUG - delete_namespaced_deployment: {'api_version': 'v1', 'code': None, 'details': {'causes': None, 'group': 'apps', 'kind': 'deployments', 'name': 'homer', 'retry_after_seconds': None, 'uid': 'fef4c095-1313-4b44-b9d6-df772baa5d24'}, 'kind': 'Status', 'message': None, 'metadata': {'_continue': None, 'remaining_item_count': None, 'resource_version': None, 'self_link': None}, 'reason': None, 'status': 'Success'} 2020-08-28 16:06:17,165 - kubernetes.client.rest - DEBUG - response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success","details":{"name":"homer","kind":"services","uid":"fe38849d-12a1-41de-851c-21559f9f3e64"}} 2020-08-28 16:06:17,167 - functest_kubernetes.ims.ims - DEBUG - delete_namespaced_service: {'api_version': 'v1', 'code': None, 'details': {'causes': None, 'group': None, 'kind': 'services', 'name': 'homer', 'retry_after_seconds': None, 'uid': 'fe38849d-12a1-41de-851c-21559f9f3e64'}, 'kind': 'Status', 'message': None, 'metadata': {'_continue': None, 'remaining_item_count': None, 'resource_version': None, 'self_link': None}, 'reason': None, 'status': 'Success'} 2020-08-28 16:06:17,645 - kubernetes.client.rest - DEBUG - response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success","details":{"name":"homestead","group":"apps","kind":"deployments","uid":"84ae3654-a60b-43f4-9ba3-8550a7a2c87f"}} 2020-08-28 16:06:17,648 - functest_kubernetes.ims.ims - DEBUG - delete_namespaced_deployment: {'api_version': 'v1', 'code': None, 'details': {'causes': None, 'group': 'apps', 'kind': 'deployments', 'name': 'homestead', 'retry_after_seconds': None, 'uid': '84ae3654-a60b-43f4-9ba3-8550a7a2c87f'}, 'kind': 'Status', 'message': None, 'metadata': {'_continue': None, 'remaining_item_count': None, 'resource_version': None, 'self_link': None}, 'reason': None, 'status': 'Success'} 2020-08-28 16:06:18,381 - kubernetes.client.rest - DEBUG - response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success","details":{"name":"homestead","kind":"services","uid":"172963ed-c537-495b-9f68-db72fee85565"}} 2020-08-28 16:06:18,383 - functest_kubernetes.ims.ims - DEBUG - delete_namespaced_service: {'api_version': 'v1', 'code': None, 'details': {'causes': None, 'group': None, 'kind': 'services', 'name': 'homestead', 'retry_after_seconds': None, 'uid': '172963ed-c537-495b-9f68-db72fee85565'}, 'kind': 'Status', 'message': None, 'metadata': {'_continue': None, 'remaining_item_count': None, 'resource_version': None, 'self_link': None}, 'reason': None, 'status': 'Success'} 2020-08-28 16:06:18,464 - kubernetes.client.rest - DEBUG - response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success","details":{"name":"homestead-prov","group":"apps","kind":"deployments","uid":"e8b84542-2695-4e30-a7a1-42b0e16de3fe"}} 2020-08-28 16:06:18,601 - functest_kubernetes.ims.ims - DEBUG - delete_namespaced_deployment: {'api_version': 'v1', 'code': None, 'details': {'causes': None, 'group': 'apps', 'kind': 'deployments', 'name': 'homestead-prov', 'retry_after_seconds': None, 'uid': 'e8b84542-2695-4e30-a7a1-42b0e16de3fe'}, 'kind': 'Status', 'message': None, 'metadata': {'_continue': None, 'remaining_item_count': None, 'resource_version': None, 'self_link': None}, 'reason': None, 'status': 'Success'} 2020-08-28 16:06:19,372 - kubernetes.client.rest - DEBUG - response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success","details":{"name":"homestead-prov","kind":"services","uid":"d58cea9f-1061-46dd-8d10-749e83430b52"}} 2020-08-28 16:06:19,374 - functest_kubernetes.ims.ims - DEBUG - delete_namespaced_service: {'api_version': 'v1', 'code': None, 'details': {'causes': None, 'group': None, 'kind': 'services', 'name': 'homestead-prov', 'retry_after_seconds': None, 'uid': 'd58cea9f-1061-46dd-8d10-749e83430b52'}, 'kind': 'Status', 'message': None, 'metadata': {'_continue': None, 'remaining_item_count': None, 'resource_version': None, 'self_link': None}, 'reason': None, 'status': 'Success'} 2020-08-28 16:06:20,287 - kubernetes.client.rest - DEBUG - response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success","details":{"name":"ralf","group":"apps","kind":"deployments","uid":"c0685c0a-f472-46a2-9b78-32d4cbb5720b"}} 2020-08-28 16:06:20,289 - functest_kubernetes.ims.ims - DEBUG - delete_namespaced_deployment: {'api_version': 'v1', 'code': None, 'details': {'causes': None, 'group': 'apps', 'kind': 'deployments', 'name': 'ralf', 'retry_after_seconds': None, 'uid': 'c0685c0a-f472-46a2-9b78-32d4cbb5720b'}, 'kind': 'Status', 'message': None, 'metadata': {'_continue': None, 'remaining_item_count': None, 'resource_version': None, 'self_link': None}, 'reason': None, 'status': 'Success'} 2020-08-28 16:06:20,732 - kubernetes.client.rest - DEBUG - response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success","details":{"name":"ralf","kind":"services","uid":"6e07aaf1-485b-4d3a-9fdd-fe293afede7c"}} 2020-08-28 16:06:20,970 - functest_kubernetes.ims.ims - DEBUG - delete_namespaced_service: {'api_version': 'v1', 'code': None, 'details': {'causes': None, 'group': None, 'kind': 'services', 'name': 'ralf', 'retry_after_seconds': None, 'uid': '6e07aaf1-485b-4d3a-9fdd-fe293afede7c'}, 'kind': 'Status', 'message': None, 'metadata': {'_continue': None, 'remaining_item_count': None, 'resource_version': None, 'self_link': None}, 'reason': None, 'status': 'Success'} 2020-08-28 16:06:21,296 - kubernetes.client.rest - DEBUG - response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success","details":{"name":"sprout","group":"apps","kind":"deployments","uid":"b3390d47-04d1-4afc-9a3e-fc9de582b4fc"}} 2020-08-28 16:06:21,299 - functest_kubernetes.ims.ims - DEBUG - delete_namespaced_deployment: {'api_version': 'v1', 'code': None, 'details': {'causes': None, 'group': 'apps', 'kind': 'deployments', 'name': 'sprout', 'retry_after_seconds': None, 'uid': 'b3390d47-04d1-4afc-9a3e-fc9de582b4fc'}, 'kind': 'Status', 'message': None, 'metadata': {'_continue': None, 'remaining_item_count': None, 'resource_version': None, 'self_link': None}, 'reason': None, 'status': 'Success'} 2020-08-28 16:06:22,069 - kubernetes.client.rest - DEBUG - response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success","details":{"name":"sprout","kind":"services","uid":"8e492691-50b8-4b8c-8197-28da9d1dbf50"}} 2020-08-28 16:06:22,072 - functest_kubernetes.ims.ims - DEBUG - delete_namespaced_service: {'api_version': 'v1', 'code': None, 'details': {'causes': None, 'group': None, 'kind': 'services', 'name': 'sprout', 'retry_after_seconds': None, 'uid': '8e492691-50b8-4b8c-8197-28da9d1dbf50'}, 'kind': 'Status', 'message': None, 'metadata': {'_continue': None, 'remaining_item_count': None, 'resource_version': None, 'self_link': None}, 'reason': None, 'status': 'Success'} 2020-08-28 16:06:22,144 - kubernetes.client.rest - DEBUG - response body: {"kind":"Namespace","apiVersion":"v1","metadata":{"name":"ims-gc7hd","generateName":"ims-","selfLink":"/api/v1/namespaces/ims-gc7hd","uid":"b9ee84d8-4810-418e-9a54-b43d163a1a17","resourceVersion":"1803145","creationTimestamp":"2020-08-28T15:56:48Z","deletionTimestamp":"2020-08-28T16:06:22Z","managedFields":[{"manager":"OpenAPI-Generator","operation":"Update","apiVersion":"v1","time":"2020-08-28T15:56:48Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:generateName":{}},"f:status":{"f:phase":{}}}}]},"spec":{"finalizers":["kubernetes"]},"status":{"phase":"Terminating"}} 2020-08-28 16:06:22,146 - functest_kubernetes.ims.ims - DEBUG - delete_namespace: ims-gc7hd 2020-08-28 16:06:23,309 - xtesting.core.testcase - DEBUG - Publishing /var/lib/xtesting/results/functest-kubernetes.log ('text/plain', None) 2020-08-28 16:06:23,837 - xtesting.core.testcase - DEBUG - Publishing /var/lib/xtesting/results/functest-kubernetes.debug.log ('text/plain', None)