/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 May 28 21:20:36.688: Unexpected error: <*errors.errorString | 0xc002d32040>: { s: "service is not reachable within 2m0s timeout on endpoint 10.10.190.207:30970 over TCP protocol", } service is not reachable within 2m0s timeout on endpoint 10.10.190.207:30970 over TCP protocol occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/network/service.go:1760 [BeforeEach] [sig-network] Services /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 28 21:18:17.768: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename services May 28 21:18:17.785: INFO: Found PodSecurityPolicies; testing pod creation to see if PodSecurityPolicy is enabled May 28 21:18:17.787: INFO: Error creating dryrun pod; assuming PodSecurityPolicy is disabled: admission webhook "cmk.intel.com" does not support dry run STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-network] Services /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/network/service.go:782 [It] should be able to change the type from ExternalName to NodePort [Conformance] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 STEP: creating a service externalname-service with the type=ExternalName in namespace services-7309 STEP: changing the ExternalName service to type=NodePort STEP: creating replication controller externalname-service in namespace services-7309 I0528 21:18:17.808210 37 runners.go:190] Created replication controller with name: externalname-service, namespace: services-7309, replica count: 2 I0528 21:18:20.858759 37 runners.go:190] externalname-service Pods: 2 out of 2 created, 0 running, 2 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0528 21:18:23.858965 37 runners.go:190] externalname-service Pods: 2 out of 2 created, 1 running, 1 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0528 21:18:26.859340 37 runners.go:190] externalname-service Pods: 2 out of 2 created, 2 running, 0 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady May 28 21:18:26.859: INFO: Creating new exec pod May 28 21:18:33.882: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 externalname-service 80' May 28 21:18:34.448: INFO: stderr: "+ nc -zv -t -w 2 externalname-service 80\nConnection to externalname-service 80 port [tcp/http] succeeded!\n" May 28 21:18:34.448: INFO: stdout: "" May 28 21:18:34.450: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.233.9.87 80' May 28 21:18:35.496: INFO: stderr: "+ nc -zv -t -w 2 10.233.9.87 80\nConnection to 10.233.9.87 80 port [tcp/http] succeeded!\n" May 28 21:18:35.496: INFO: stdout: "" May 28 21:18:35.496: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:18:36.161: INFO: rc: 1 May 28 21:18:36.161: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:18:37.164: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:18:37.430: INFO: rc: 1 May 28 21:18:37.430: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:18:38.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:18:38.435: INFO: rc: 1 May 28 21:18:38.436: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:18:39.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:18:39.445: INFO: rc: 1 May 28 21:18:39.445: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:18:40.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:18:40.418: INFO: rc: 1 May 28 21:18:40.418: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:18:41.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:18:41.436: INFO: rc: 1 May 28 21:18:41.436: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:18:42.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:18:42.537: INFO: rc: 1 May 28 21:18:42.537: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:18:43.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:18:43.837: INFO: rc: 1 May 28 21:18:43.837: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:18:44.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:18:44.569: INFO: rc: 1 May 28 21:18:44.569: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:18:45.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:18:45.961: INFO: rc: 1 May 28 21:18:45.961: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:18:46.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:18:46.682: INFO: rc: 1 May 28 21:18:46.682: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:18:47.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:18:47.694: INFO: rc: 1 May 28 21:18:47.694: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:18:48.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:18:48.451: INFO: rc: 1 May 28 21:18:48.451: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:18:49.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:18:49.417: INFO: rc: 1 May 28 21:18:49.417: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:18:50.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:18:50.507: INFO: rc: 1 May 28 21:18:50.507: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:18:51.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:18:51.540: INFO: rc: 1 May 28 21:18:51.540: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:18:52.163: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:18:52.527: INFO: rc: 1 May 28 21:18:52.528: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:18:53.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:18:53.444: INFO: rc: 1 May 28 21:18:53.444: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:18:54.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:18:54.633: INFO: rc: 1 May 28 21:18:54.633: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:18:55.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:18:55.532: INFO: rc: 1 May 28 21:18:55.532: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:18:56.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:18:56.586: INFO: rc: 1 May 28 21:18:56.586: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:18:57.163: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:18:57.823: INFO: rc: 1 May 28 21:18:57.823: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:18:58.163: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:18:58.675: INFO: rc: 1 May 28 21:18:58.675: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:18:59.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:18:59.465: INFO: rc: 1 May 28 21:18:59.465: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:00.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:00.631: INFO: rc: 1 May 28 21:19:00.631: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:01.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:01.417: INFO: rc: 1 May 28 21:19:01.417: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:02.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:02.499: INFO: rc: 1 May 28 21:19:02.499: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:03.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:03.496: INFO: rc: 1 May 28 21:19:03.496: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:04.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:04.452: INFO: rc: 1 May 28 21:19:04.452: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:05.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:05.810: INFO: rc: 1 May 28 21:19:05.810: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:06.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:06.557: INFO: rc: 1 May 28 21:19:06.557: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:07.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:07.474: INFO: rc: 1 May 28 21:19:07.475: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:08.163: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:08.761: INFO: rc: 1 May 28 21:19:08.761: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:09.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:09.458: INFO: rc: 1 May 28 21:19:09.458: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:10.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:10.417: INFO: rc: 1 May 28 21:19:10.417: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:11.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:11.791: INFO: rc: 1 May 28 21:19:11.791: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:12.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:12.432: INFO: rc: 1 May 28 21:19:12.432: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:13.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:13.435: INFO: rc: 1 May 28 21:19:13.435: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:14.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:14.446: INFO: rc: 1 May 28 21:19:14.446: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:15.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:15.421: INFO: rc: 1 May 28 21:19:15.421: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:16.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:16.436: INFO: rc: 1 May 28 21:19:16.436: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:17.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:17.817: INFO: rc: 1 May 28 21:19:17.817: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:18.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:18.417: INFO: rc: 1 May 28 21:19:18.417: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:19.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:19.531: INFO: rc: 1 May 28 21:19:19.531: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:20.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:20.577: INFO: rc: 1 May 28 21:19:20.577: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:21.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:21.421: INFO: rc: 1 May 28 21:19:21.421: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:22.164: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:22.390: INFO: rc: 1 May 28 21:19:22.390: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:23.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:23.415: INFO: rc: 1 May 28 21:19:23.415: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:24.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:24.418: INFO: rc: 1 May 28 21:19:24.418: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:25.163: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:25.406: INFO: rc: 1 May 28 21:19:25.406: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:26.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:26.437: INFO: rc: 1 May 28 21:19:26.437: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:27.163: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:27.430: INFO: rc: 1 May 28 21:19:27.430: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:28.163: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:28.443: INFO: rc: 1 May 28 21:19:28.443: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:29.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:29.423: INFO: rc: 1 May 28 21:19:29.424: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:30.163: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:30.420: INFO: rc: 1 May 28 21:19:30.420: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:31.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:31.443: INFO: rc: 1 May 28 21:19:31.443: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:32.163: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:32.430: INFO: rc: 1 May 28 21:19:32.430: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:33.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:33.400: INFO: rc: 1 May 28 21:19:33.400: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:34.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:34.420: INFO: rc: 1 May 28 21:19:34.420: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:35.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:35.937: INFO: rc: 1 May 28 21:19:35.937: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:36.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:36.422: INFO: rc: 1 May 28 21:19:36.422: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:37.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:37.421: INFO: rc: 1 May 28 21:19:37.421: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:38.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:38.430: INFO: rc: 1 May 28 21:19:38.430: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:39.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:39.411: INFO: rc: 1 May 28 21:19:39.411: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:40.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:40.452: INFO: rc: 1 May 28 21:19:40.452: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:41.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:41.423: INFO: rc: 1 May 28 21:19:41.423: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:42.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:42.408: INFO: rc: 1 May 28 21:19:42.408: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:43.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:43.422: INFO: rc: 1 May 28 21:19:43.423: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:44.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:44.437: INFO: rc: 1 May 28 21:19:44.437: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:45.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:45.600: INFO: rc: 1 May 28 21:19:45.600: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:46.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:46.409: INFO: rc: 1 May 28 21:19:46.409: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:47.163: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:47.438: INFO: rc: 1 May 28 21:19:47.438: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:48.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:48.417: INFO: rc: 1 May 28 21:19:48.417: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:49.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:49.463: INFO: rc: 1 May 28 21:19:49.463: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:50.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:50.431: INFO: rc: 1 May 28 21:19:50.431: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:51.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:51.576: INFO: rc: 1 May 28 21:19:51.576: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:52.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:52.419: INFO: rc: 1 May 28 21:19:52.419: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:53.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:53.573: INFO: rc: 1 May 28 21:19:53.573: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:54.163: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:54.425: INFO: rc: 1 May 28 21:19:54.425: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:55.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:55.426: INFO: rc: 1 May 28 21:19:55.426: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:56.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:56.553: INFO: rc: 1 May 28 21:19:56.553: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:57.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:57.437: INFO: rc: 1 May 28 21:19:57.437: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:58.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:58.425: INFO: rc: 1 May 28 21:19:58.425: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:19:59.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:19:59.416: INFO: rc: 1 May 28 21:19:59.416: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:00.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:00.599: INFO: rc: 1 May 28 21:20:00.599: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:01.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:01.725: INFO: rc: 1 May 28 21:20:01.726: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:02.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:02.752: INFO: rc: 1 May 28 21:20:02.752: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:03.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:03.704: INFO: rc: 1 May 28 21:20:03.704: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:04.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:04.569: INFO: rc: 1 May 28 21:20:04.569: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:05.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:05.789: INFO: rc: 1 May 28 21:20:05.789: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:06.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:06.442: INFO: rc: 1 May 28 21:20:06.442: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:07.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:07.612: INFO: rc: 1 May 28 21:20:07.612: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:08.163: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:08.465: INFO: rc: 1 May 28 21:20:08.465: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:09.163: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:09.406: INFO: rc: 1 May 28 21:20:09.406: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:10.163: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:10.884: INFO: rc: 1 May 28 21:20:10.884: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:11.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:11.478: INFO: rc: 1 May 28 21:20:11.478: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:12.163: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:12.440: INFO: rc: 1 May 28 21:20:12.441: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:13.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:13.502: INFO: rc: 1 May 28 21:20:13.502: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:14.163: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:14.791: INFO: rc: 1 May 28 21:20:14.791: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:15.163: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:15.415: INFO: rc: 1 May 28 21:20:15.415: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:16.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:16.439: INFO: rc: 1 May 28 21:20:16.439: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:17.163: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:17.395: INFO: rc: 1 May 28 21:20:17.395: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:18.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:18.411: INFO: rc: 1 May 28 21:20:18.411: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:19.163: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:19.410: INFO: rc: 1 May 28 21:20:19.410: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:20.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:20.424: INFO: rc: 1 May 28 21:20:20.424: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:21.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:21.411: INFO: rc: 1 May 28 21:20:21.411: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:22.163: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:22.434: INFO: rc: 1 May 28 21:20:22.434: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:23.163: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:23.435: INFO: rc: 1 May 28 21:20:23.435: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:24.163: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:24.428: INFO: rc: 1 May 28 21:20:24.429: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:25.163: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:25.426: INFO: rc: 1 May 28 21:20:25.426: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:26.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:26.423: INFO: rc: 1 May 28 21:20:26.423: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:27.163: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:27.415: INFO: rc: 1 May 28 21:20:27.415: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:28.163: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:28.418: INFO: rc: 1 May 28 21:20:28.418: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:29.163: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:29.414: INFO: rc: 1 May 28 21:20:29.414: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:30.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:30.419: INFO: rc: 1 May 28 21:20:30.419: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:31.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:31.407: INFO: rc: 1 May 28 21:20:31.407: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:32.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:32.426: INFO: rc: 1 May 28 21:20:32.426: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:33.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:33.421: INFO: rc: 1 May 28 21:20:33.421: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:34.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:34.415: INFO: rc: 1 May 28 21:20:34.415: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:35.163: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:35.698: INFO: rc: 1 May 28 21:20:35.698: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:36.162: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:36.409: INFO: rc: 1 May 28 21:20:36.409: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:36.410: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970' May 28 21:20:36.687: INFO: rc: 1 May 28 21:20:36.687: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-7309 exec execpod54gcz -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 30970: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 30970 nc: connect to 10.10.190.207 port 30970 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 28 21:20:36.688: FAIL: Unexpected error: <*errors.errorString | 0xc002d32040>: { s: "service is not reachable within 2m0s timeout on endpoint 10.10.190.207:30970 over TCP protocol", } service is not reachable within 2m0s timeout on endpoint 10.10.190.207:30970 over TCP protocol occurred Full Stack Trace k8s.io/kubernetes/test/e2e/network.glob..func24.15() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/network/service.go:1760 +0x358 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc00254f980) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc00254f980) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc00254f980, 0x4de5140) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 May 28 21:20:36.689: INFO: Cleaning up the ExternalName to NodePort test service [AfterEach] [sig-network] Services /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "services-7309". STEP: Found 20 events. May 28 21:20:36.718: INFO: At 2021-05-28 21:18:17 +0000 UTC - event for externalname-service: {replication-controller } SuccessfulCreate: Created pod: externalname-service-xtqr4 May 28 21:20:36.718: INFO: At 2021-05-28 21:18:17 +0000 UTC - event for externalname-service: {replication-controller } SuccessfulCreate: Created pod: externalname-service-zxqv7 May 28 21:20:36.718: INFO: At 2021-05-28 21:18:17 +0000 UTC - event for externalname-service-xtqr4: {default-scheduler } Scheduled: Successfully assigned services-7309/externalname-service-xtqr4 to node1 May 28 21:20:36.718: INFO: At 2021-05-28 21:18:17 +0000 UTC - event for externalname-service-zxqv7: {default-scheduler } Scheduled: Successfully assigned services-7309/externalname-service-zxqv7 to node2 May 28 21:20:36.718: INFO: At 2021-05-28 21:18:20 +0000 UTC - event for externalname-service-zxqv7: {multus } AddedInterface: Add eth0 [10.244.3.41/24] May 28 21:20:36.718: INFO: At 2021-05-28 21:18:20 +0000 UTC - event for externalname-service-zxqv7: {kubelet node2} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 28 21:20:36.718: INFO: At 2021-05-28 21:18:21 +0000 UTC - event for externalname-service-xtqr4: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 28 21:20:36.718: INFO: At 2021-05-28 21:18:21 +0000 UTC - event for externalname-service-xtqr4: {multus } AddedInterface: Add eth0 [10.244.4.42/24] May 28 21:20:36.718: INFO: At 2021-05-28 21:18:21 +0000 UTC - event for externalname-service-zxqv7: {kubelet node2} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 570.328576ms May 28 21:20:36.718: INFO: At 2021-05-28 21:18:21 +0000 UTC - event for externalname-service-zxqv7: {kubelet node2} Created: Created container externalname-service May 28 21:20:36.718: INFO: At 2021-05-28 21:18:22 +0000 UTC - event for externalname-service-xtqr4: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 1.082603294s May 28 21:20:36.718: INFO: At 2021-05-28 21:18:22 +0000 UTC - event for externalname-service-xtqr4: {kubelet node1} Created: Created container externalname-service May 28 21:20:36.718: INFO: At 2021-05-28 21:18:23 +0000 UTC - event for externalname-service-xtqr4: {kubelet node1} Started: Started container externalname-service May 28 21:20:36.718: INFO: At 2021-05-28 21:18:23 +0000 UTC - event for externalname-service-zxqv7: {kubelet node2} Started: Started container externalname-service May 28 21:20:36.718: INFO: At 2021-05-28 21:18:26 +0000 UTC - event for execpod54gcz: {default-scheduler } Scheduled: Successfully assigned services-7309/execpod54gcz to node2 May 28 21:20:36.718: INFO: At 2021-05-28 21:18:28 +0000 UTC - event for execpod54gcz: {kubelet node2} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 28 21:20:36.718: INFO: At 2021-05-28 21:18:28 +0000 UTC - event for execpod54gcz: {multus } AddedInterface: Add eth0 [10.244.3.48/24] May 28 21:20:36.718: INFO: At 2021-05-28 21:18:31 +0000 UTC - event for execpod54gcz: {kubelet node2} Created: Created container agnhost-container May 28 21:20:36.718: INFO: At 2021-05-28 21:18:31 +0000 UTC - event for execpod54gcz: {kubelet node2} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 2.636323323s May 28 21:20:36.718: INFO: At 2021-05-28 21:18:31 +0000 UTC - event for execpod54gcz: {kubelet node2} Started: Started container agnhost-container May 28 21:20:36.720: INFO: POD NODE PHASE GRACE CONDITIONS May 28 21:20:36.720: INFO: execpod54gcz node2 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-28 21:18:26 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-28 21:18:32 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-28 21:18:32 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-28 21:18:26 +0000 UTC }] May 28 21:20:36.721: INFO: externalname-service-xtqr4 node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-28 21:18:17 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-28 21:18:23 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-28 21:18:23 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-28 21:18:17 +0000 UTC }] May 28 21:20:36.721: INFO: externalname-service-zxqv7 node2 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-28 21:18:17 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-28 21:18:24 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-28 21:18:24 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-28 21:18:17 +0000 UTC }] May 28 21:20:36.721: INFO: May 28 21:20:36.724: INFO: Logging node info for node master1 May 28 21:20:36.727: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 0aa78934-442a-44a3-8c5c-f827e18dd3d7 26365 0 2021-05-28 19:56:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:41:0b:9d:15:5a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-28 19:56:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-28 19:56:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-28 19:59:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-28 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-28 20:02:03 +0000 UTC,LastTransitionTime:2021-05-28 20:02:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-28 21:20:36 +0000 UTC,LastTransitionTime:2021-05-28 19:56:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-28 21:20:36 +0000 UTC,LastTransitionTime:2021-05-28 19:56:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-28 21:20:36 +0000 UTC,LastTransitionTime:2021-05-28 19:56:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-28 21:20:36 +0000 UTC,LastTransitionTime:2021-05-28 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f7fb2c462cae4b9c990ab2e5c72f7816,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:24c06694-15ae-4da4-9143-144d98afdd8d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:7f3d9945acdf5d86edd89b2b16fe1f6d63ba8bdb4cab50e66f9bce162df9e388 tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:9af6075c93013910787a4e97973da6e0739a86dee1186d7965a5d00b1ac35636 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 28 21:20:36.728: INFO: Logging kubelet events for node master1 May 28 21:20:36.730: INFO: Logging pods the kubelet thinks is on node master1 May 28 21:20:36.757: INFO: kube-proxy-994p2 started at 2021-05-28 19:58:24 +0000 UTC (0+1 container statuses recorded) May 28 21:20:36.757: INFO: Container kube-proxy ready: true, restart count 1 May 28 21:20:36.757: INFO: kube-flannel-d54gm started at 2021-05-28 19:59:00 +0000 UTC (1+1 container statuses recorded) May 28 21:20:36.757: INFO: Init container install-cni ready: true, restart count 0 May 28 21:20:36.757: INFO: Container kube-flannel ready: true, restart count 2 May 28 21:20:36.757: INFO: coredns-7677f9bb54-zb7h8 started at 2021-05-28 19:59:33 +0000 UTC (0+1 container statuses recorded) May 28 21:20:36.757: INFO: Container coredns ready: true, restart count 1 May 28 21:20:36.757: INFO: node-exporter-9b7pq started at 2021-05-28 20:10:09 +0000 UTC (0+2 container statuses recorded) May 28 21:20:36.757: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:20:36.758: INFO: Container node-exporter ready: true, restart count 0 May 28 21:20:36.758: INFO: kube-apiserver-master1 started at 2021-05-28 20:05:21 +0000 UTC (0+1 container statuses recorded) May 28 21:20:36.758: INFO: Container kube-apiserver ready: true, restart count 0 May 28 21:20:36.758: INFO: kube-controller-manager-master1 started at 2021-05-28 19:57:39 +0000 UTC (0+1 container statuses recorded) May 28 21:20:36.758: INFO: Container kube-controller-manager ready: true, restart count 2 May 28 21:20:36.758: INFO: kube-multus-ds-amd64-n9j8k started at 2021-05-28 19:59:08 +0000 UTC (0+1 container statuses recorded) May 28 21:20:36.758: INFO: Container kube-multus ready: true, restart count 1 May 28 21:20:36.758: INFO: docker-registry-docker-registry-56cbc7bc58-rbghz started at 2021-05-28 20:02:55 +0000 UTC (0+2 container statuses recorded) May 28 21:20:36.758: INFO: Container docker-registry ready: true, restart count 0 May 28 21:20:36.758: INFO: Container nginx ready: true, restart count 0 May 28 21:20:36.758: INFO: prometheus-operator-5bb8cb9d8f-7wdtq started at 2021-05-28 20:10:02 +0000 UTC (0+2 container statuses recorded) May 28 21:20:36.758: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:20:36.758: INFO: Container prometheus-operator ready: true, restart count 0 May 28 21:20:36.758: INFO: kube-scheduler-master1 started at 2021-05-28 19:57:39 +0000 UTC (0+1 container statuses recorded) May 28 21:20:36.758: INFO: Container kube-scheduler ready: true, restart count 0 W0528 21:20:36.769241 37 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 28 21:20:36.793: INFO: Latency metrics for node master1 May 28 21:20:36.793: INFO: Logging node info for node master2 May 28 21:20:36.795: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 b80f32b6-a396-4f09-a110-345a08d762ee 26339 0 2021-05-28 19:57:04 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"b2:be:c9:d8:cf:bb"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-28 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-28 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-28 19:59:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-28 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-28 20:06:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-28 20:00:49 +0000 UTC,LastTransitionTime:2021-05-28 20:00:49 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-28 21:20:32 +0000 UTC,LastTransitionTime:2021-05-28 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-28 21:20:32 +0000 UTC,LastTransitionTime:2021-05-28 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-28 21:20:32 +0000 UTC,LastTransitionTime:2021-05-28 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-28 21:20:32 +0000 UTC,LastTransitionTime:2021-05-28 20:00:43 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:2746caf91c53460599f165aa716150cd,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:b63b522f-706f-4e28-a104-c73edcd04319,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 28 21:20:36.795: INFO: Logging kubelet events for node master2 May 28 21:20:36.797: INFO: Logging pods the kubelet thinks is on node master2 May 28 21:20:36.811: INFO: node-exporter-frch9 started at 2021-05-28 20:10:09 +0000 UTC (0+2 container statuses recorded) May 28 21:20:36.811: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:20:36.811: INFO: Container node-exporter ready: true, restart count 0 May 28 21:20:36.811: INFO: kube-scheduler-master2 started at 2021-05-28 20:05:21 +0000 UTC (0+1 container statuses recorded) May 28 21:20:36.811: INFO: Container kube-scheduler ready: true, restart count 3 May 28 21:20:36.811: INFO: kube-proxy-jkbl8 started at 2021-05-28 19:58:24 +0000 UTC (0+1 container statuses recorded) May 28 21:20:36.811: INFO: Container kube-proxy ready: true, restart count 1 May 28 21:20:36.811: INFO: dns-autoscaler-5b7b5c9b6f-r797x started at 2021-05-28 19:59:31 +0000 UTC (0+1 container statuses recorded) May 28 21:20:36.811: INFO: Container autoscaler ready: true, restart count 1 May 28 21:20:36.811: INFO: kube-multus-ds-amd64-qjwcz started at 2021-05-28 19:59:08 +0000 UTC (0+1 container statuses recorded) May 28 21:20:36.811: INFO: Container kube-multus ready: true, restart count 1 May 28 21:20:36.811: INFO: node-feature-discovery-controller-5bf5c49849-n9ncl started at 2021-05-28 20:05:52 +0000 UTC (0+1 container statuses recorded) May 28 21:20:36.811: INFO: Container nfd-controller ready: true, restart count 0 May 28 21:20:36.811: INFO: kube-apiserver-master2 started at 2021-05-28 20:05:31 +0000 UTC (0+1 container statuses recorded) May 28 21:20:36.811: INFO: Container kube-apiserver ready: true, restart count 0 May 28 21:20:36.811: INFO: kube-controller-manager-master2 started at 2021-05-28 20:05:41 +0000 UTC (0+1 container statuses recorded) May 28 21:20:36.811: INFO: Container kube-controller-manager ready: true, restart count 3 May 28 21:20:36.811: INFO: kube-flannel-xvtkj started at 2021-05-28 19:59:00 +0000 UTC (1+1 container statuses recorded) May 28 21:20:36.811: INFO: Init container install-cni ready: true, restart count 0 May 28 21:20:36.811: INFO: Container kube-flannel ready: true, restart count 2 W0528 21:20:36.824866 37 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 28 21:20:36.852: INFO: Latency metrics for node master2 May 28 21:20:36.852: INFO: Logging node info for node master3 May 28 21:20:36.854: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 301b0b5b-fc42-4c78-adb7-75baf6e0cc7e 26363 0 2021-05-28 19:57:14 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"52:fa:ab:49:88:02"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-28 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-28 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-28 19:59:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-28 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-28 20:02:12 +0000 UTC,LastTransitionTime:2021-05-28 20:02:12 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-28 21:20:35 +0000 UTC,LastTransitionTime:2021-05-28 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-28 21:20:35 +0000 UTC,LastTransitionTime:2021-05-28 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-28 21:20:35 +0000 UTC,LastTransitionTime:2021-05-28 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-28 21:20:35 +0000 UTC,LastTransitionTime:2021-05-28 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a0cb6c0eb1d842469076fff344213c13,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:c6adcff4-8bf7-40d7-9d14-54b1c6a87bc8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 28 21:20:36.855: INFO: Logging kubelet events for node master3 May 28 21:20:36.857: INFO: Logging pods the kubelet thinks is on node master3 May 28 21:20:36.870: INFO: kube-apiserver-master3 started at 2021-05-28 20:05:21 +0000 UTC (0+1 container statuses recorded) May 28 21:20:36.870: INFO: Container kube-apiserver ready: true, restart count 0 May 28 21:20:36.870: INFO: kube-controller-manager-master3 started at 2021-05-28 20:06:02 +0000 UTC (0+1 container statuses recorded) May 28 21:20:36.870: INFO: Container kube-controller-manager ready: true, restart count 1 May 28 21:20:36.870: INFO: kube-scheduler-master3 started at 2021-05-28 20:01:23 +0000 UTC (0+1 container statuses recorded) May 28 21:20:36.870: INFO: Container kube-scheduler ready: true, restart count 1 May 28 21:20:36.870: INFO: kube-proxy-t5bh6 started at 2021-05-28 19:58:24 +0000 UTC (0+1 container statuses recorded) May 28 21:20:36.870: INFO: Container kube-proxy ready: true, restart count 1 May 28 21:20:36.870: INFO: kube-flannel-zrskq started at 2021-05-28 19:59:00 +0000 UTC (1+1 container statuses recorded) May 28 21:20:36.870: INFO: Init container install-cni ready: true, restart count 0 May 28 21:20:36.870: INFO: Container kube-flannel ready: true, restart count 1 May 28 21:20:36.870: INFO: kube-multus-ds-amd64-wqgf7 started at 2021-05-28 19:59:08 +0000 UTC (0+1 container statuses recorded) May 28 21:20:36.870: INFO: Container kube-multus ready: true, restart count 1 May 28 21:20:36.870: INFO: coredns-7677f9bb54-8v554 started at 2021-05-28 19:59:28 +0000 UTC (0+1 container statuses recorded) May 28 21:20:36.870: INFO: Container coredns ready: true, restart count 1 May 28 21:20:36.870: INFO: node-exporter-w42s5 started at 2021-05-28 20:10:09 +0000 UTC (0+2 container statuses recorded) May 28 21:20:36.870: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:20:36.870: INFO: Container node-exporter ready: true, restart count 0 W0528 21:20:36.882999 37 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 28 21:20:36.910: INFO: Latency metrics for node master3 May 28 21:20:36.910: INFO: Logging node info for node node1 May 28 21:20:36.913: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 43e51cb4-5acb-42b5-8f26-cd5e977f3829 26358 0 2021-05-28 19:58:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"d2:9d:b7:73:58:07"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-05-28 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubeadm Update v1 2021-05-28 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-28 19:59:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-28 20:06:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-28 20:08:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-05-28 20:08:43 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269628928 0} {<nil>} 196552372Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884628480 0} {<nil>} 174692020Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-28 20:01:58 +0000 UTC,LastTransitionTime:2021-05-28 20:01:58 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-28 21:20:35 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-28 21:20:35 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-28 21:20:35 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-28 21:20:35 +0000 UTC,LastTransitionTime:2021-05-28 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:abe6e95dbfa24a9abd34d8fa2abe7655,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:17719d1f-7df5-4d95-81f3-7d3ac5110ba2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:d731a0fc49b9ad6125b8d5dcb29da2b60bc940b48eacb6f5a9eb2a55c10598db localhost:30500/barometer-collectd:stable],SizeBytes:1464395058,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:1002495332,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:97953d03767e4c2eb5d156394aeaf4bb0b74f3fd1ad08c303cb7561e272a00ff cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:aa24a0a337084e0747e7c8e97e1131270ae38150e691314f1fa19f4b2f9093c0 golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:2bec7a43da8efe70cb7cb14020a6b10aecd02c87e020d394de84e6807e2cf620 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392623,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 28 21:20:36.914: INFO: Logging kubelet events for node node1 May 28 21:20:36.916: INFO: Logging pods the kubelet thinks is on node node1 May 28 21:20:36.932: INFO: kubernetes-metrics-scraper-678c97765c-wblkm started at 2021-05-28 19:59:33 +0000 UTC (0+1 container statuses recorded) May 28 21:20:36.932: INFO: Container kubernetes-metrics-scraper ready: true, restart count 1 May 28 21:20:36.932: INFO: kubernetes-dashboard-86c6f9df5b-c5sbq started at 2021-05-28 19:59:33 +0000 UTC (0+1 container statuses recorded) May 28 21:20:36.932: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 28 21:20:36.932: INFO: node-feature-discovery-worker-5x4qg started at 2021-05-28 20:05:52 +0000 UTC (0+1 container statuses recorded) May 28 21:20:36.932: INFO: Container nfd-worker ready: true, restart count 0 May 28 21:20:36.932: INFO: liveness-9fcb63ae-42dd-4f00-9330-2fbd99c319a5 started at 2021-05-28 21:19:02 +0000 UTC (0+1 container statuses recorded) May 28 21:20:36.932: INFO: Container liveness ready: true, restart count 0 May 28 21:20:36.932: INFO: simpletest-rc-to-be-deleted-2gb6g started at 2021-05-28 21:19:59 +0000 UTC (0+1 container statuses recorded) May 28 21:20:36.932: INFO: Container nginx ready: false, restart count 0 May 28 21:20:36.932: INFO: ss2-1 started at 2021-05-28 21:18:56 +0000 UTC (0+1 container statuses recorded) May 28 21:20:36.932: INFO: Container webserver ready: true, restart count 0 May 28 21:20:36.932: INFO: nginx-proxy-node1 started at 2021-05-28 20:05:21 +0000 UTC (0+1 container statuses recorded) May 28 21:20:36.932: INFO: Container nginx-proxy ready: true, restart count 1 May 28 21:20:36.932: INFO: cmk-init-discover-node1-rvqxm started at 2021-05-28 20:08:32 +0000 UTC (0+3 container statuses recorded) May 28 21:20:36.932: INFO: Container discover ready: false, restart count 0 May 28 21:20:36.932: INFO: Container init ready: false, restart count 0 May 28 21:20:36.932: INFO: Container install ready: false, restart count 0 May 28 21:20:36.932: INFO: cmk-jhzjr started at 2021-05-28 20:09:15 +0000 UTC (0+2 container statuses recorded) May 28 21:20:36.932: INFO: Container nodereport ready: true, restart count 0 May 28 21:20:36.932: INFO: Container reconcile ready: true, restart count 0 May 28 21:20:36.932: INFO: prometheus-k8s-0 started at 2021-05-28 20:10:26 +0000 UTC (0+5 container statuses recorded) May 28 21:20:36.932: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 28 21:20:36.932: INFO: Container grafana ready: true, restart count 0 May 28 21:20:36.932: INFO: Container prometheus ready: true, restart count 1 May 28 21:20:36.932: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 28 21:20:36.932: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 28 21:20:36.932: INFO: ss-0 started at 2021-05-28 21:19:57 +0000 UTC (0+1 container statuses recorded) May 28 21:20:36.932: INFO: Container webserver ready: false, restart count 0 May 28 21:20:36.932: INFO: externalname-service-xtqr4 started at 2021-05-28 21:18:17 +0000 UTC (0+1 container statuses recorded) May 28 21:20:36.932: INFO: Container externalname-service ready: true, restart count 0 May 28 21:20:36.932: INFO: simpletest-rc-to-be-deleted-p84q6 started at 2021-05-28 21:19:59 +0000 UTC (0+1 container statuses recorded) May 28 21:20:36.932: INFO: Container nginx ready: false, restart count 0 May 28 21:20:36.932: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-zk2pt started at 2021-05-28 20:06:47 +0000 UTC (0+1 container statuses recorded) May 28 21:20:36.932: INFO: Container kube-sriovdp ready: true, restart count 0 May 28 21:20:36.932: INFO: test-cleanup-controller-p2bgj started at 2021-05-28 21:19:02 +0000 UTC (0+1 container statuses recorded) May 28 21:20:36.932: INFO: Container httpd ready: false, restart count 0 May 28 21:20:36.932: INFO: node-exporter-khdpg started at 2021-05-28 20:10:09 +0000 UTC (0+2 container statuses recorded) May 28 21:20:36.932: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:20:36.932: INFO: Container node-exporter ready: true, restart count 0 May 28 21:20:36.932: INFO: ss2-2 started at 2021-05-28 21:19:02 +0000 UTC (0+1 container statuses recorded) May 28 21:20:36.932: INFO: Container webserver ready: false, restart count 0 May 28 21:20:36.932: INFO: fail-once-local-sgxcs started at 2021-05-28 21:19:10 +0000 UTC (0+1 container statuses recorded) May 28 21:20:36.932: INFO: Container c ready: false, restart count 0 May 28 21:20:36.932: INFO: kube-proxy-lsngv started at 2021-05-28 19:58:24 +0000 UTC (0+1 container statuses recorded) May 28 21:20:36.932: INFO: Container kube-proxy ready: true, restart count 2 May 28 21:20:36.932: INFO: kube-multus-ds-amd64-x7826 started at 2021-05-28 19:59:08 +0000 UTC (0+1 container statuses recorded) May 28 21:20:36.932: INFO: Container kube-multus ready: true, restart count 1 May 28 21:20:36.932: INFO: collectd-qw9nd started at 2021-05-28 20:16:29 +0000 UTC (0+3 container statuses recorded) May 28 21:20:36.932: INFO: Container collectd ready: true, restart count 0 May 28 21:20:36.932: INFO: Container collectd-exporter ready: true, restart count 0 May 28 21:20:36.932: INFO: Container rbac-proxy ready: true, restart count 0 May 28 21:20:36.932: INFO: kube-flannel-2tjjt started at 2021-05-28 19:59:00 +0000 UTC (1+1 container statuses recorded) May 28 21:20:36.932: INFO: Init container install-cni ready: true, restart count 0 May 28 21:20:36.932: INFO: Container kube-flannel ready: true, restart count 2 W0528 21:20:36.942418 37 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 28 21:20:36.987: INFO: Latency metrics for node node1 May 28 21:20:36.987: INFO: Logging node info for node node2 May 28 21:20:36.989: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 3cc89580-b568-4c82-bd1f-200d0823da3b 26341 0 2021-05-28 19:58:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"62:22:2c:ae:14:ae"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-05-28 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubeadm Update v1 2021-05-28 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-28 19:59:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-28 20:06:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-28 20:08:58 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-05-28 20:09:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269633024 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884632576 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-28 20:01:05 +0000 UTC,LastTransitionTime:2021-05-28 20:01:05 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-28 21:20:33 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-28 21:20:33 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-28 21:20:33 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-28 21:20:33 +0000 UTC,LastTransitionTime:2021-05-28 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b2730c4b09814ab9a78e7bc62c820fbb,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:f1459072-d21d-46de-a5d9-46ec9349aae0,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:d731a0fc49b9ad6125b8d5dcb29da2b60bc940b48eacb6f5a9eb2a55c10598db localhost:30500/barometer-collectd:stable],SizeBytes:1464395058,},ContainerImage{Names:[localhost:30500/cmk@sha256:97953d03767e4c2eb5d156394aeaf4bb0b74f3fd1ad08c303cb7561e272a00ff localhost:30500/cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:2bec7a43da8efe70cb7cb14020a6b10aecd02c87e020d394de84e6807e2cf620 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392623,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:7f3d9945acdf5d86edd89b2b16fe1f6d63ba8bdb4cab50e66f9bce162df9e388 localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:9af6075c93013910787a4e97973da6e0739a86dee1186d7965a5d00b1ac35636 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 28 21:20:36.990: INFO: Logging kubelet events for node node2 May 28 21:20:36.993: INFO: Logging pods the kubelet thinks is on node node2 May 28 21:20:37.006: INFO: busybox-55bfce72-bbef-49b4-9a47-fcc9e5958219 started at 2021-05-28 21:18:40 +0000 UTC (0+1 container statuses recorded) May 28 21:20:37.006: INFO: Container busybox ready: false, restart count 0 May 28 21:20:37.006: INFO: node-feature-discovery-worker-j2wnf started at 2021-05-28 20:05:52 +0000 UTC (0+1 container statuses recorded) May 28 21:20:37.006: INFO: Container nfd-worker ready: true, restart count 0 May 28 21:20:37.006: INFO: cmk-5vxwj started at 2021-05-28 20:09:16 +0000 UTC (0+2 container statuses recorded) May 28 21:20:37.006: INFO: Container nodereport ready: true, restart count 0 May 28 21:20:37.006: INFO: Container reconcile ready: true, restart count 0 May 28 21:20:37.006: INFO: fail-once-local-64bh4 started at 2021-05-28 21:19:10 +0000 UTC (0+1 container statuses recorded) May 28 21:20:37.006: INFO: Container c ready: false, restart count 0 May 28 21:20:37.006: INFO: ss-0 started at 2021-05-28 21:19:15 +0000 UTC (0+1 container statuses recorded) May 28 21:20:37.006: INFO: Container webserver ready: false, restart count 0 May 28 21:20:37.006: INFO: execpod54gcz started at 2021-05-28 21:18:26 +0000 UTC (0+1 container statuses recorded) May 28 21:20:37.006: INFO: Container agnhost-container ready: true, restart count 0 May 28 21:20:37.006: INFO: kube-flannel-d9wsg started at 2021-05-28 19:59:00 +0000 UTC (1+1 container statuses recorded) May 28 21:20:37.006: INFO: Init container install-cni ready: true, restart count 2 May 28 21:20:37.006: INFO: Container kube-flannel ready: true, restart count 2 May 28 21:20:37.006: INFO: kube-multus-ds-amd64-c9cj2 started at 2021-05-28 19:59:08 +0000 UTC (0+1 container statuses recorded) May 28 21:20:37.006: INFO: Container kube-multus ready: true, restart count 1 May 28 21:20:37.006: INFO: cmk-webhook-6c9d5f8578-6cpp6 started at 2021-05-28 20:09:16 +0000 UTC (0+1 container statuses recorded) May 28 21:20:37.006: INFO: Container cmk-webhook ready: true, restart count 0 May 28 21:20:37.006: INFO: node-exporter-sjsht started at 2021-05-28 20:10:09 +0000 UTC (0+2 container statuses recorded) May 28 21:20:37.006: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:20:37.006: INFO: Container node-exporter ready: true, restart count 0 May 28 21:20:37.006: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-csd8v started at 2021-05-28 20:13:00 +0000 UTC (0+2 container statuses recorded) May 28 21:20:37.006: INFO: Container tas-controller ready: true, restart count 0 May 28 21:20:37.006: INFO: Container tas-extender ready: true, restart count 0 May 28 21:20:37.007: INFO: nginx-proxy-node2 started at 2021-05-28 20:05:21 +0000 UTC (0+1 container statuses recorded) May 28 21:20:37.007: INFO: Container nginx-proxy ready: true, restart count 2 May 28 21:20:37.007: INFO: collectd-b5dgp started at 2021-05-28 20:16:29 +0000 UTC (0+3 container statuses recorded) May 28 21:20:37.007: INFO: Container collectd ready: true, restart count 0 May 28 21:20:37.007: INFO: Container collectd-exporter ready: true, restart count 0 May 28 21:20:37.007: INFO: Container rbac-proxy ready: true, restart count 0 May 28 21:20:37.007: INFO: simpletest-rc-to-be-deleted-r2pkc started at 2021-05-28 21:19:59 +0000 UTC (0+1 container statuses recorded) May 28 21:20:37.007: INFO: Container nginx ready: false, restart count 0 May 28 21:20:37.007: INFO: simpletest-rc-to-be-deleted-pvcfq started at 2021-05-28 21:19:59 +0000 UTC (0+1 container statuses recorded) May 28 21:20:37.007: INFO: Container nginx ready: false, restart count 0 May 28 21:20:37.007: INFO: kube-proxy-z5czn started at 2021-05-28 19:58:24 +0000 UTC (0+1 container statuses recorded) May 28 21:20:37.007: INFO: Container kube-proxy ready: true, restart count 2 May 28 21:20:37.007: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-29vc6 started at 2021-05-28 20:06:47 +0000 UTC (0+1 container statuses recorded) May 28 21:20:37.007: INFO: Container kube-sriovdp ready: true, restart count 0 May 28 21:20:37.007: INFO: cmk-init-discover-node2-95cbr started at 2021-05-28 20:08:52 +0000 UTC (0+3 container statuses recorded) May 28 21:20:37.007: INFO: Container discover ready: false, restart count 0 May 28 21:20:37.007: INFO: Container init ready: false, restart count 0 May 28 21:20:37.007: INFO: Container install ready: false, restart count 0 May 28 21:20:37.007: INFO: externalname-service-zxqv7 started at 2021-05-28 21:18:17 +0000 UTC (0+1 container statuses recorded) May 28 21:20:37.007: INFO: Container externalname-service ready: true, restart count 0 May 28 21:20:37.007: INFO: ss2-0 started at 2021-05-28 21:18:50 +0000 UTC (0+1 container statuses recorded) May 28 21:20:37.007: INFO: Container webserver ready: true, restart count 0 May 28 21:20:37.007: INFO: simpletest-rc-to-be-deleted-jg8jl started at 2021-05-28 21:19:59 +0000 UTC (0+1 container statuses recorded) May 28 21:20:37.007: INFO: Container nginx ready: false, restart count 0 May 28 21:20:37.007: INFO: pod-secrets-850685b8-dd93-4056-aa7b-ab314efee6a2 started at 2021-05-28 21:19:04 +0000 UTC (0+3 container statuses recorded) May 28 21:20:37.007: INFO: Container creates-volume-test ready: true, restart count 0 May 28 21:20:37.007: INFO: Container dels-volume-test ready: true, restart count 0 May 28 21:20:37.007: INFO: Container upds-volume-test ready: true, restart count 0 W0528 21:20:37.019550 37 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 28 21:20:37.072: INFO: Latency metrics for node node2 May 28 21:20:37.072: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "services-7309" for this suite. [AfterEach] [sig-network] Services /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/network/service.go:786 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 May 28 21:25:41.854: Unexpected error: <*errors.errorString | 0xc0002c6210>: { s: "timed out waiting for the condition", } timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/pods.go:103 [BeforeEach] [sig-storage] EmptyDir volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 28 21:20:41.813: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename emptydir STEP: Waiting for a default service account to be provisioned in namespace [It] pod should support shared volumes between containers [Conformance] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 STEP: Creating Pod May 28 21:25:41.854: FAIL: Unexpected error: <*errors.errorString | 0xc0002c6210>: { s: "timed out waiting for the condition", } timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/framework.(*PodClient).CreateSync(0xc003df9fa0, 0xc002fef800, 0x0) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/pods.go:103 +0xfe k8s.io/kubernetes/test/e2e/common.glob..func8.16() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/empty_dir.go:283 +0xa58 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc00254f980) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc00254f980) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc00254f980, 0x4de5140) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [sig-storage] EmptyDir volumes /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "emptydir-4072". STEP: Found 9 events. May 28 21:25:41.858: INFO: At 2021-05-28 21:20:41 +0000 UTC - event for pod-sharedvolume-14e76001-660d-437e-a211-4723afde06a6: {default-scheduler } Scheduled: Successfully assigned emptydir-4072/pod-sharedvolume-14e76001-660d-437e-a211-4723afde06a6 to node2 May 28 21:25:41.858: INFO: At 2021-05-28 21:20:43 +0000 UTC - event for pod-sharedvolume-14e76001-660d-437e-a211-4723afde06a6: {multus } AddedInterface: Add eth0 [10.244.3.84/24] May 28 21:25:41.858: INFO: At 2021-05-28 21:20:43 +0000 UTC - event for pod-sharedvolume-14e76001-660d-437e-a211-4723afde06a6: {kubelet node2} Pulling: Pulling image "docker.io/library/busybox:1.29" May 28 21:25:41.858: INFO: At 2021-05-28 21:20:45 +0000 UTC - event for pod-sharedvolume-14e76001-660d-437e-a211-4723afde06a6: {kubelet node2} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 28 21:25:41.858: INFO: At 2021-05-28 21:20:45 +0000 UTC - event for pod-sharedvolume-14e76001-660d-437e-a211-4723afde06a6: {kubelet node2} Failed: Error: ErrImagePull May 28 21:25:41.858: INFO: At 2021-05-28 21:20:45 +0000 UTC - event for pod-sharedvolume-14e76001-660d-437e-a211-4723afde06a6: {kubelet node2} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 28 21:25:41.858: INFO: At 2021-05-28 21:20:45 +0000 UTC - event for pod-sharedvolume-14e76001-660d-437e-a211-4723afde06a6: {kubelet node2} Failed: Error: ImagePullBackOff May 28 21:25:41.858: INFO: At 2021-05-28 21:20:46 +0000 UTC - event for pod-sharedvolume-14e76001-660d-437e-a211-4723afde06a6: {kubelet node2} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 28 21:25:41.858: INFO: At 2021-05-28 21:20:46 +0000 UTC - event for pod-sharedvolume-14e76001-660d-437e-a211-4723afde06a6: {kubelet node2} Failed: Error: ImagePullBackOff May 28 21:25:41.860: INFO: POD NODE PHASE GRACE CONDITIONS May 28 21:25:41.860: INFO: pod-sharedvolume-14e76001-660d-437e-a211-4723afde06a6 node2 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-28 21:20:41 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-28 21:20:41 +0000 UTC ContainersNotReady containers with unready status: [busybox-main-container busybox-sub-container]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-28 21:20:41 +0000 UTC ContainersNotReady containers with unready status: [busybox-main-container busybox-sub-container]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-28 21:20:41 +0000 UTC }] May 28 21:25:41.860: INFO: May 28 21:25:41.865: INFO: Logging node info for node master1 May 28 21:25:41.867: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 0aa78934-442a-44a3-8c5c-f827e18dd3d7 28945 0 2021-05-28 19:56:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:41:0b:9d:15:5a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-28 19:56:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-28 19:56:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-28 19:59:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-28 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-28 20:02:03 +0000 UTC,LastTransitionTime:2021-05-28 20:02:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-28 21:25:37 +0000 UTC,LastTransitionTime:2021-05-28 19:56:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-28 21:25:37 +0000 UTC,LastTransitionTime:2021-05-28 19:56:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-28 21:25:37 +0000 UTC,LastTransitionTime:2021-05-28 19:56:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-28 21:25:37 +0000 UTC,LastTransitionTime:2021-05-28 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f7fb2c462cae4b9c990ab2e5c72f7816,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:24c06694-15ae-4da4-9143-144d98afdd8d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:7f3d9945acdf5d86edd89b2b16fe1f6d63ba8bdb4cab50e66f9bce162df9e388 tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:9af6075c93013910787a4e97973da6e0739a86dee1186d7965a5d00b1ac35636 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 28 21:25:41.868: INFO: Logging kubelet events for node master1 May 28 21:25:41.869: INFO: Logging pods the kubelet thinks is on node master1 May 28 21:25:41.884: INFO: kube-apiserver-master1 started at 2021-05-28 20:05:21 +0000 UTC (0+1 container statuses recorded) May 28 21:25:41.884: INFO: Container kube-apiserver ready: true, restart count 0 May 28 21:25:41.884: INFO: kube-controller-manager-master1 started at 2021-05-28 19:57:39 +0000 UTC (0+1 container statuses recorded) May 28 21:25:41.884: INFO: Container kube-controller-manager ready: true, restart count 2 May 28 21:25:41.884: INFO: kube-multus-ds-amd64-n9j8k started at 2021-05-28 19:59:08 +0000 UTC (0+1 container statuses recorded) May 28 21:25:41.884: INFO: Container kube-multus ready: true, restart count 1 May 28 21:25:41.884: INFO: docker-registry-docker-registry-56cbc7bc58-rbghz started at 2021-05-28 20:02:55 +0000 UTC (0+2 container statuses recorded) May 28 21:25:41.884: INFO: Container docker-registry ready: true, restart count 0 May 28 21:25:41.884: INFO: Container nginx ready: true, restart count 0 May 28 21:25:41.884: INFO: prometheus-operator-5bb8cb9d8f-7wdtq started at 2021-05-28 20:10:02 +0000 UTC (0+2 container statuses recorded) May 28 21:25:41.884: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:25:41.884: INFO: Container prometheus-operator ready: true, restart count 0 May 28 21:25:41.884: INFO: kube-scheduler-master1 started at 2021-05-28 19:57:39 +0000 UTC (0+1 container statuses recorded) May 28 21:25:41.884: INFO: Container kube-scheduler ready: true, restart count 0 May 28 21:25:41.884: INFO: kube-proxy-994p2 started at 2021-05-28 19:58:24 +0000 UTC (0+1 container statuses recorded) May 28 21:25:41.884: INFO: Container kube-proxy ready: true, restart count 1 May 28 21:25:41.884: INFO: kube-flannel-d54gm started at 2021-05-28 19:59:00 +0000 UTC (1+1 container statuses recorded) May 28 21:25:41.884: INFO: Init container install-cni ready: true, restart count 0 May 28 21:25:41.884: INFO: Container kube-flannel ready: true, restart count 2 May 28 21:25:41.884: INFO: coredns-7677f9bb54-zb7h8 started at 2021-05-28 19:59:33 +0000 UTC (0+1 container statuses recorded) May 28 21:25:41.884: INFO: Container coredns ready: true, restart count 1 May 28 21:25:41.884: INFO: node-exporter-9b7pq started at 2021-05-28 20:10:09 +0000 UTC (0+2 container statuses recorded) May 28 21:25:41.884: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:25:41.884: INFO: Container node-exporter ready: true, restart count 0 W0528 21:25:41.899105 37 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 28 21:25:41.930: INFO: Latency metrics for node master1 May 28 21:25:41.930: INFO: Logging node info for node master2 May 28 21:25:41.934: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 b80f32b6-a396-4f09-a110-345a08d762ee 28930 0 2021-05-28 19:57:04 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"b2:be:c9:d8:cf:bb"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-28 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-28 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-28 19:59:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-28 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-28 20:06:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-28 20:00:49 +0000 UTC,LastTransitionTime:2021-05-28 20:00:49 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-28 21:25:33 +0000 UTC,LastTransitionTime:2021-05-28 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-28 21:25:33 +0000 UTC,LastTransitionTime:2021-05-28 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-28 21:25:33 +0000 UTC,LastTransitionTime:2021-05-28 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-28 21:25:33 +0000 UTC,LastTransitionTime:2021-05-28 20:00:43 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:2746caf91c53460599f165aa716150cd,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:b63b522f-706f-4e28-a104-c73edcd04319,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 28 21:25:41.934: INFO: Logging kubelet events for node master2 May 28 21:25:41.938: INFO: Logging pods the kubelet thinks is on node master2 May 28 21:25:41.953: INFO: kube-proxy-jkbl8 started at 2021-05-28 19:58:24 +0000 UTC (0+1 container statuses recorded) May 28 21:25:41.953: INFO: Container kube-proxy ready: true, restart count 1 May 28 21:25:41.953: INFO: dns-autoscaler-5b7b5c9b6f-r797x started at 2021-05-28 19:59:31 +0000 UTC (0+1 container statuses recorded) May 28 21:25:41.953: INFO: Container autoscaler ready: true, restart count 1 May 28 21:25:41.953: INFO: node-exporter-frch9 started at 2021-05-28 20:10:09 +0000 UTC (0+2 container statuses recorded) May 28 21:25:41.953: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:25:41.953: INFO: Container node-exporter ready: true, restart count 0 May 28 21:25:41.953: INFO: kube-scheduler-master2 started at 2021-05-28 20:05:21 +0000 UTC (0+1 container statuses recorded) May 28 21:25:41.953: INFO: Container kube-scheduler ready: true, restart count 3 May 28 21:25:41.953: INFO: kube-controller-manager-master2 started at 2021-05-28 20:05:41 +0000 UTC (0+1 container statuses recorded) May 28 21:25:41.953: INFO: Container kube-controller-manager ready: true, restart count 3 May 28 21:25:41.953: INFO: kube-flannel-xvtkj started at 2021-05-28 19:59:00 +0000 UTC (1+1 container statuses recorded) May 28 21:25:41.953: INFO: Init container install-cni ready: true, restart count 0 May 28 21:25:41.953: INFO: Container kube-flannel ready: true, restart count 2 May 28 21:25:41.953: INFO: kube-multus-ds-amd64-qjwcz started at 2021-05-28 19:59:08 +0000 UTC (0+1 container statuses recorded) May 28 21:25:41.953: INFO: Container kube-multus ready: true, restart count 1 May 28 21:25:41.953: INFO: node-feature-discovery-controller-5bf5c49849-n9ncl started at 2021-05-28 20:05:52 +0000 UTC (0+1 container statuses recorded) May 28 21:25:41.953: INFO: Container nfd-controller ready: true, restart count 0 May 28 21:25:41.953: INFO: kube-apiserver-master2 started at 2021-05-28 20:05:31 +0000 UTC (0+1 container statuses recorded) May 28 21:25:41.953: INFO: Container kube-apiserver ready: true, restart count 0 W0528 21:25:41.967383 37 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 28 21:25:41.991: INFO: Latency metrics for node master2 May 28 21:25:41.991: INFO: Logging node info for node master3 May 28 21:25:41.994: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 301b0b5b-fc42-4c78-adb7-75baf6e0cc7e 28944 0 2021-05-28 19:57:14 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"52:fa:ab:49:88:02"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-28 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-28 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-28 19:59:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-28 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-28 20:02:12 +0000 UTC,LastTransitionTime:2021-05-28 20:02:12 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-28 21:25:36 +0000 UTC,LastTransitionTime:2021-05-28 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-28 21:25:36 +0000 UTC,LastTransitionTime:2021-05-28 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-28 21:25:36 +0000 UTC,LastTransitionTime:2021-05-28 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-28 21:25:36 +0000 UTC,LastTransitionTime:2021-05-28 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a0cb6c0eb1d842469076fff344213c13,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:c6adcff4-8bf7-40d7-9d14-54b1c6a87bc8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 28 21:25:41.995: INFO: Logging kubelet events for node master3 May 28 21:25:41.996: INFO: Logging pods the kubelet thinks is on node master3 May 28 21:25:42.012: INFO: kube-proxy-t5bh6 started at 2021-05-28 19:58:24 +0000 UTC (0+1 container statuses recorded) May 28 21:25:42.012: INFO: Container kube-proxy ready: true, restart count 1 May 28 21:25:42.012: INFO: kube-flannel-zrskq started at 2021-05-28 19:59:00 +0000 UTC (1+1 container statuses recorded) May 28 21:25:42.012: INFO: Init container install-cni ready: true, restart count 0 May 28 21:25:42.012: INFO: Container kube-flannel ready: true, restart count 1 May 28 21:25:42.012: INFO: kube-multus-ds-amd64-wqgf7 started at 2021-05-28 19:59:08 +0000 UTC (0+1 container statuses recorded) May 28 21:25:42.012: INFO: Container kube-multus ready: true, restart count 1 May 28 21:25:42.012: INFO: coredns-7677f9bb54-8v554 started at 2021-05-28 19:59:28 +0000 UTC (0+1 container statuses recorded) May 28 21:25:42.012: INFO: Container coredns ready: true, restart count 1 May 28 21:25:42.012: INFO: node-exporter-w42s5 started at 2021-05-28 20:10:09 +0000 UTC (0+2 container statuses recorded) May 28 21:25:42.012: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:25:42.012: INFO: Container node-exporter ready: true, restart count 0 May 28 21:25:42.012: INFO: kube-apiserver-master3 started at 2021-05-28 20:05:21 +0000 UTC (0+1 container statuses recorded) May 28 21:25:42.012: INFO: Container kube-apiserver ready: true, restart count 0 May 28 21:25:42.012: INFO: kube-controller-manager-master3 started at 2021-05-28 20:06:02 +0000 UTC (0+1 container statuses recorded) May 28 21:25:42.012: INFO: Container kube-controller-manager ready: true, restart count 1 May 28 21:25:42.012: INFO: kube-scheduler-master3 started at 2021-05-28 20:01:23 +0000 UTC (0+1 container statuses recorded) May 28 21:25:42.012: INFO: Container kube-scheduler ready: true, restart count 1 W0528 21:25:42.024045 37 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 28 21:25:42.053: INFO: Latency metrics for node master3 May 28 21:25:42.053: INFO: Logging node info for node node1 May 28 21:25:42.055: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 43e51cb4-5acb-42b5-8f26-cd5e977f3829 28939 0 2021-05-28 19:58:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"d2:9d:b7:73:58:07"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-05-28 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubeadm Update v1 2021-05-28 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-28 19:59:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-28 20:06:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-28 20:08:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-05-28 20:08:43 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269628928 0} {<nil>} 196552372Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884628480 0} {<nil>} 174692020Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-28 20:01:58 +0000 UTC,LastTransitionTime:2021-05-28 20:01:58 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-28 21:25:36 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-28 21:25:36 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-28 21:25:36 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-28 21:25:36 +0000 UTC,LastTransitionTime:2021-05-28 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:abe6e95dbfa24a9abd34d8fa2abe7655,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:17719d1f-7df5-4d95-81f3-7d3ac5110ba2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:d731a0fc49b9ad6125b8d5dcb29da2b60bc940b48eacb6f5a9eb2a55c10598db localhost:30500/barometer-collectd:stable],SizeBytes:1464395058,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:1002495332,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:97953d03767e4c2eb5d156394aeaf4bb0b74f3fd1ad08c303cb7561e272a00ff cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:aa24a0a337084e0747e7c8e97e1131270ae38150e691314f1fa19f4b2f9093c0 golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:2bec7a43da8efe70cb7cb14020a6b10aecd02c87e020d394de84e6807e2cf620 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392623,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 28 21:25:42.056: INFO: Logging kubelet events for node node1 May 28 21:25:42.058: INFO: Logging pods the kubelet thinks is on node node1 May 28 21:25:42.073: INFO: collectd-qw9nd started at 2021-05-28 20:16:29 +0000 UTC (0+3 container statuses recorded) May 28 21:25:42.074: INFO: Container collectd ready: true, restart count 0 May 28 21:25:42.074: INFO: Container collectd-exporter ready: true, restart count 0 May 28 21:25:42.074: INFO: Container rbac-proxy ready: true, restart count 0 May 28 21:25:42.074: INFO: var-expansion-80b96b32-b368-4182-baf8-c58a6686db38 started at 2021-05-28 21:20:40 +0000 UTC (0+1 container statuses recorded) May 28 21:25:42.074: INFO: Container dapi-container ready: false, restart count 0 May 28 21:25:42.074: INFO: kube-flannel-2tjjt started at 2021-05-28 19:59:00 +0000 UTC (1+1 container statuses recorded) May 28 21:25:42.074: INFO: Init container install-cni ready: true, restart count 0 May 28 21:25:42.074: INFO: Container kube-flannel ready: true, restart count 2 May 28 21:25:42.074: INFO: kube-multus-ds-amd64-x7826 started at 2021-05-28 19:59:08 +0000 UTC (0+1 container statuses recorded) May 28 21:25:42.074: INFO: Container kube-multus ready: true, restart count 1 May 28 21:25:42.074: INFO: kubernetes-dashboard-86c6f9df5b-c5sbq started at 2021-05-28 19:59:33 +0000 UTC (0+1 container statuses recorded) May 28 21:25:42.074: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 28 21:25:42.074: INFO: node-feature-discovery-worker-5x4qg started at 2021-05-28 20:05:52 +0000 UTC (0+1 container statuses recorded) May 28 21:25:42.074: INFO: Container nfd-worker ready: true, restart count 0 May 28 21:25:42.074: INFO: downward-api-46a996fd-082e-4db5-8ca5-146a6a0c45ff started at 2021-05-28 21:23:19 +0000 UTC (0+1 container statuses recorded) May 28 21:25:42.074: INFO: Container dapi-container ready: false, restart count 0 May 28 21:25:42.074: INFO: nodeport-test-4sdfv started at 2021-05-28 21:24:40 +0000 UTC (0+1 container statuses recorded) May 28 21:25:42.074: INFO: Container nodeport-test ready: true, restart count 0 May 28 21:25:42.074: INFO: ss2-1 started at 2021-05-28 21:18:56 +0000 UTC (0+1 container statuses recorded) May 28 21:25:42.074: INFO: Container webserver ready: true, restart count 0 May 28 21:25:42.074: INFO: nginx-proxy-node1 started at 2021-05-28 20:05:21 +0000 UTC (0+1 container statuses recorded) May 28 21:25:42.074: INFO: Container nginx-proxy ready: true, restart count 1 May 28 21:25:42.074: INFO: kubernetes-metrics-scraper-678c97765c-wblkm started at 2021-05-28 19:59:33 +0000 UTC (0+1 container statuses recorded) May 28 21:25:42.074: INFO: Container kubernetes-metrics-scraper ready: true, restart count 1 May 28 21:25:42.074: INFO: cmk-jhzjr started at 2021-05-28 20:09:15 +0000 UTC (0+2 container statuses recorded) May 28 21:25:42.074: INFO: Container nodereport ready: true, restart count 0 May 28 21:25:42.074: INFO: Container reconcile ready: true, restart count 0 May 28 21:25:42.074: INFO: prometheus-k8s-0 started at 2021-05-28 20:10:26 +0000 UTC (0+5 container statuses recorded) May 28 21:25:42.074: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 28 21:25:42.074: INFO: Container grafana ready: true, restart count 0 May 28 21:25:42.074: INFO: Container prometheus ready: true, restart count 1 May 28 21:25:42.074: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 28 21:25:42.074: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 28 21:25:42.074: INFO: ss-0 started at 2021-05-28 21:19:57 +0000 UTC (0+1 container statuses recorded) May 28 21:25:42.074: INFO: Container webserver ready: false, restart count 0 May 28 21:25:42.074: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-zk2pt started at 2021-05-28 20:06:47 +0000 UTC (0+1 container statuses recorded) May 28 21:25:42.074: INFO: Container kube-sriovdp ready: true, restart count 0 May 28 21:25:42.074: INFO: cmk-init-discover-node1-rvqxm started at 2021-05-28 20:08:32 +0000 UTC (0+3 container statuses recorded) May 28 21:25:42.074: INFO: Container discover ready: false, restart count 0 May 28 21:25:42.074: INFO: Container init ready: false, restart count 0 May 28 21:25:42.074: INFO: Container install ready: false, restart count 0 May 28 21:25:42.074: INFO: ss2-2 started at 2021-05-28 21:19:02 +0000 UTC (0+1 container statuses recorded) May 28 21:25:42.074: INFO: Container webserver ready: false, restart count 0 May 28 21:25:42.074: INFO: fail-once-local-sgxcs started at 2021-05-28 21:19:10 +0000 UTC (0+1 container statuses recorded) May 28 21:25:42.074: INFO: Container c ready: false, restart count 0 May 28 21:25:42.074: INFO: kube-proxy-lsngv started at 2021-05-28 19:58:24 +0000 UTC (0+1 container statuses recorded) May 28 21:25:42.074: INFO: Container kube-proxy ready: true, restart count 2 May 28 21:25:42.074: INFO: node-exporter-khdpg started at 2021-05-28 20:10:09 +0000 UTC (0+2 container statuses recorded) May 28 21:25:42.074: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:25:42.074: INFO: Container node-exporter ready: true, restart count 0 W0528 21:25:42.087269 37 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 28 21:25:42.120: INFO: Latency metrics for node node1 May 28 21:25:42.120: INFO: Logging node info for node node2 May 28 21:25:42.123: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 3cc89580-b568-4c82-bd1f-200d0823da3b 28938 0 2021-05-28 19:58:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"62:22:2c:ae:14:ae"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-05-28 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubeadm Update v1 2021-05-28 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-28 19:59:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-28 20:06:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-28 20:08:58 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-05-28 20:09:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269633024 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884632576 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-28 20:01:05 +0000 UTC,LastTransitionTime:2021-05-28 20:01:05 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-28 21:25:36 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-28 21:25:36 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-28 21:25:36 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-28 21:25:36 +0000 UTC,LastTransitionTime:2021-05-28 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b2730c4b09814ab9a78e7bc62c820fbb,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:f1459072-d21d-46de-a5d9-46ec9349aae0,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:d731a0fc49b9ad6125b8d5dcb29da2b60bc940b48eacb6f5a9eb2a55c10598db localhost:30500/barometer-collectd:stable],SizeBytes:1464395058,},ContainerImage{Names:[localhost:30500/cmk@sha256:97953d03767e4c2eb5d156394aeaf4bb0b74f3fd1ad08c303cb7561e272a00ff localhost:30500/cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:2bec7a43da8efe70cb7cb14020a6b10aecd02c87e020d394de84e6807e2cf620 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392623,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:7f3d9945acdf5d86edd89b2b16fe1f6d63ba8bdb4cab50e66f9bce162df9e388 localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:9af6075c93013910787a4e97973da6e0739a86dee1186d7965a5d00b1ac35636 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 28 21:25:42.124: INFO: Logging kubelet events for node node2 May 28 21:25:42.127: INFO: Logging pods the kubelet thinks is on node node2 May 28 21:25:42.145: INFO: kube-multus-ds-amd64-c9cj2 started at 2021-05-28 19:59:08 +0000 UTC (0+1 container statuses recorded) May 28 21:25:42.145: INFO: Container kube-multus ready: true, restart count 1 May 28 21:25:42.145: INFO: cmk-webhook-6c9d5f8578-6cpp6 started at 2021-05-28 20:09:16 +0000 UTC (0+1 container statuses recorded) May 28 21:25:42.145: INFO: Container cmk-webhook ready: true, restart count 0 May 28 21:25:42.145: INFO: node-exporter-sjsht started at 2021-05-28 20:10:09 +0000 UTC (0+2 container statuses recorded) May 28 21:25:42.145: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:25:42.145: INFO: Container node-exporter ready: true, restart count 0 May 28 21:25:42.145: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-csd8v started at 2021-05-28 20:13:00 +0000 UTC (0+2 container statuses recorded) May 28 21:25:42.145: INFO: Container tas-controller ready: true, restart count 0 May 28 21:25:42.145: INFO: Container tas-extender ready: true, restart count 0 May 28 21:25:42.145: INFO: kube-flannel-d9wsg started at 2021-05-28 19:59:00 +0000 UTC (1+1 container statuses recorded) May 28 21:25:42.145: INFO: Init container install-cni ready: true, restart count 2 May 28 21:25:42.145: INFO: Container kube-flannel ready: true, restart count 2 May 28 21:25:42.145: INFO: collectd-b5dgp started at 2021-05-28 20:16:29 +0000 UTC (0+3 container statuses recorded) May 28 21:25:42.145: INFO: Container collectd ready: true, restart count 0 May 28 21:25:42.145: INFO: Container collectd-exporter ready: true, restart count 0 May 28 21:25:42.145: INFO: Container rbac-proxy ready: true, restart count 0 May 28 21:25:42.145: INFO: nodeport-test-sg5tf started at 2021-05-28 21:24:40 +0000 UTC (0+1 container statuses recorded) May 28 21:25:42.145: INFO: Container nodeport-test ready: true, restart count 0 May 28 21:25:42.145: INFO: pod-update-activedeadlineseconds-ee15b420-9ba9-4d77-8f6f-06ede70f595a started at 2021-05-28 21:23:20 +0000 UTC (0+1 container statuses recorded) May 28 21:25:42.145: INFO: Container nginx ready: false, restart count 0 May 28 21:25:42.145: INFO: e2e-test-httpd-pod started at 2021-05-28 21:22:56 +0000 UTC (0+1 container statuses recorded) May 28 21:25:42.145: INFO: Container e2e-test-httpd-pod ready: false, restart count 0 May 28 21:25:42.145: INFO: execpod8nn4h started at 2021-05-28 21:24:46 +0000 UTC (0+1 container statuses recorded) May 28 21:25:42.145: INFO: Container agnhost-container ready: true, restart count 0 May 28 21:25:42.145: INFO: nginx-proxy-node2 started at 2021-05-28 20:05:21 +0000 UTC (0+1 container statuses recorded) May 28 21:25:42.145: INFO: Container nginx-proxy ready: true, restart count 2 May 28 21:25:42.145: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-29vc6 started at 2021-05-28 20:06:47 +0000 UTC (0+1 container statuses recorded) May 28 21:25:42.145: INFO: Container kube-sriovdp ready: true, restart count 0 May 28 21:25:42.145: INFO: cmk-init-discover-node2-95cbr started at 2021-05-28 20:08:52 +0000 UTC (0+3 container statuses recorded) May 28 21:25:42.145: INFO: Container discover ready: false, restart count 0 May 28 21:25:42.145: INFO: Container init ready: false, restart count 0 May 28 21:25:42.145: INFO: Container install ready: false, restart count 0 May 28 21:25:42.145: INFO: ss2-0 started at 2021-05-28 21:18:50 +0000 UTC (0+1 container statuses recorded) May 28 21:25:42.145: INFO: Container webserver ready: true, restart count 0 May 28 21:25:42.145: INFO: kube-proxy-z5czn started at 2021-05-28 19:58:24 +0000 UTC (0+1 container statuses recorded) May 28 21:25:42.145: INFO: Container kube-proxy ready: true, restart count 2 May 28 21:25:42.145: INFO: cmk-5vxwj started at 2021-05-28 20:09:16 +0000 UTC (0+2 container statuses recorded) May 28 21:25:42.145: INFO: Container nodereport ready: true, restart count 0 May 28 21:25:42.145: INFO: Container reconcile ready: true, restart count 0 May 28 21:25:42.145: INFO: fail-once-local-64bh4 started at 2021-05-28 21:19:10 +0000 UTC (0+1 container statuses recorded) May 28 21:25:42.145: INFO: Container c ready: false, restart count 0 May 28 21:25:42.145: INFO: ss-0 started at 2021-05-28 21:19:15 +0000 UTC (0+1 container statuses recorded) May 28 21:25:42.145: INFO: Container webserver ready: false, restart count 0 May 28 21:25:42.145: INFO: pod-sharedvolume-14e76001-660d-437e-a211-4723afde06a6 started at 2021-05-28 21:20:41 +0000 UTC (0+2 container statuses recorded) May 28 21:25:42.145: INFO: Container busybox-main-container ready: false, restart count 0 May 28 21:25:42.145: INFO: Container busybox-sub-container ready: false, restart count 0 May 28 21:25:42.145: INFO: node-feature-discovery-worker-j2wnf started at 2021-05-28 20:05:52 +0000 UTC (0+1 container statuses recorded) May 28 21:25:42.145: INFO: Container nfd-worker ready: true, restart count 0 W0528 21:25:42.157649 37 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 28 21:25:42.190: INFO: Latency metrics for node node2 May 28 21:25:42.190: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "emptydir-4072" for this suite. /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 May 28 21:30:42.242: Expected <*errors.errorString | 0xc0002c6210>: { s: "timed out waiting for the condition", } to be nil /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/init_container.go:302 {"msg":"FAILED [sig-storage] EmptyDir volumes pod should support shared volumes between containers [Conformance]","total":-1,"completed":2,"skipped":64,"failed":2,"failures":["[sig-network] Services should be able to change the type from ExternalName to NodePort [Conformance]","[sig-storage] EmptyDir volumes pod should support shared volumes between containers [Conformance]"]} [BeforeEach] [k8s.io] InitContainer [NodeConformance] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 28 21:25:42.205: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename init-container STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [k8s.io] InitContainer [NodeConformance] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/init_container.go:162 [It] should invoke init containers on a RestartAlways pod [Conformance] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 STEP: creating the pod May 28 21:25:42.225: INFO: PodSpec: initContainers in spec.initContainers May 28 21:30:42.242: FAIL: Expected <*errors.errorString | 0xc0002c6210>: { s: "timed out waiting for the condition", } to be nil Full Stack Trace k8s.io/kubernetes/test/e2e/common.glob..func11.3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/init_container.go:302 +0xcde k8s.io/kubernetes/test/e2e.RunE2ETests(0xc00254f980) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc00254f980) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc00254f980, 0x4de5140) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [k8s.io] InitContainer [NodeConformance] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "init-container-559". STEP: Found 7 events. May 28 21:30:42.247: INFO: At 2021-05-28 21:25:42 +0000 UTC - event for pod-init-ff5fe8e0-d2d6-4fa2-bccb-878785589a0b: {default-scheduler } Scheduled: Successfully assigned init-container-559/pod-init-ff5fe8e0-d2d6-4fa2-bccb-878785589a0b to node1 May 28 21:30:42.247: INFO: At 2021-05-28 21:25:45 +0000 UTC - event for pod-init-ff5fe8e0-d2d6-4fa2-bccb-878785589a0b: {multus } AddedInterface: Add eth0 [10.244.4.94/24] May 28 21:30:42.247: INFO: At 2021-05-28 21:25:45 +0000 UTC - event for pod-init-ff5fe8e0-d2d6-4fa2-bccb-878785589a0b: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 28 21:30:42.247: INFO: At 2021-05-28 21:25:46 +0000 UTC - event for pod-init-ff5fe8e0-d2d6-4fa2-bccb-878785589a0b: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 28 21:30:42.247: INFO: At 2021-05-28 21:25:46 +0000 UTC - event for pod-init-ff5fe8e0-d2d6-4fa2-bccb-878785589a0b: {kubelet node1} Failed: Error: ErrImagePull May 28 21:30:42.247: INFO: At 2021-05-28 21:25:47 +0000 UTC - event for pod-init-ff5fe8e0-d2d6-4fa2-bccb-878785589a0b: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 28 21:30:42.247: INFO: At 2021-05-28 21:25:47 +0000 UTC - event for pod-init-ff5fe8e0-d2d6-4fa2-bccb-878785589a0b: {kubelet node1} Failed: Error: ImagePullBackOff May 28 21:30:42.250: INFO: POD NODE PHASE GRACE CONDITIONS May 28 21:30:42.250: INFO: pod-init-ff5fe8e0-d2d6-4fa2-bccb-878785589a0b node1 Pending [{Initialized False 0001-01-01 00:00:00 +0000 UTC 2021-05-28 21:25:42 +0000 UTC ContainersNotInitialized containers with incomplete status: [init1 init2]} {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-28 21:25:42 +0000 UTC ContainersNotReady containers with unready status: [run1]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-28 21:25:42 +0000 UTC ContainersNotReady containers with unready status: [run1]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-28 21:25:42 +0000 UTC }] May 28 21:30:42.250: INFO: May 28 21:30:42.254: INFO: Logging node info for node master1 May 28 21:30:42.256: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 0aa78934-442a-44a3-8c5c-f827e18dd3d7 32012 0 2021-05-28 19:56:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:41:0b:9d:15:5a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-28 19:56:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-28 19:56:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-28 19:59:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-28 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-28 20:02:03 +0000 UTC,LastTransitionTime:2021-05-28 20:02:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-28 21:30:38 +0000 UTC,LastTransitionTime:2021-05-28 19:56:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-28 21:30:38 +0000 UTC,LastTransitionTime:2021-05-28 19:56:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-28 21:30:38 +0000 UTC,LastTransitionTime:2021-05-28 19:56:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-28 21:30:38 +0000 UTC,LastTransitionTime:2021-05-28 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f7fb2c462cae4b9c990ab2e5c72f7816,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:24c06694-15ae-4da4-9143-144d98afdd8d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:7f3d9945acdf5d86edd89b2b16fe1f6d63ba8bdb4cab50e66f9bce162df9e388 tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:9af6075c93013910787a4e97973da6e0739a86dee1186d7965a5d00b1ac35636 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 28 21:30:42.257: INFO: Logging kubelet events for node master1 May 28 21:30:42.259: INFO: Logging pods the kubelet thinks is on node master1 May 28 21:30:42.271: INFO: node-exporter-9b7pq started at 2021-05-28 20:10:09 +0000 UTC (0+2 container statuses recorded) May 28 21:30:42.271: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:30:42.271: INFO: Container node-exporter ready: true, restart count 0 May 28 21:30:42.271: INFO: kube-proxy-994p2 started at 2021-05-28 19:58:24 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.271: INFO: Container kube-proxy ready: true, restart count 1 May 28 21:30:42.271: INFO: kube-flannel-d54gm started at 2021-05-28 19:59:00 +0000 UTC (1+1 container statuses recorded) May 28 21:30:42.271: INFO: Init container install-cni ready: true, restart count 0 May 28 21:30:42.271: INFO: Container kube-flannel ready: true, restart count 2 May 28 21:30:42.271: INFO: coredns-7677f9bb54-zb7h8 started at 2021-05-28 19:59:33 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.271: INFO: Container coredns ready: true, restart count 1 May 28 21:30:42.271: INFO: prometheus-operator-5bb8cb9d8f-7wdtq started at 2021-05-28 20:10:02 +0000 UTC (0+2 container statuses recorded) May 28 21:30:42.271: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:30:42.271: INFO: Container prometheus-operator ready: true, restart count 0 May 28 21:30:42.271: INFO: kube-scheduler-master1 started at 2021-05-28 19:57:39 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.271: INFO: Container kube-scheduler ready: true, restart count 0 May 28 21:30:42.271: INFO: kube-apiserver-master1 started at 2021-05-28 20:05:21 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.271: INFO: Container kube-apiserver ready: true, restart count 0 May 28 21:30:42.271: INFO: kube-controller-manager-master1 started at 2021-05-28 19:57:39 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.271: INFO: Container kube-controller-manager ready: true, restart count 2 May 28 21:30:42.271: INFO: kube-multus-ds-amd64-n9j8k started at 2021-05-28 19:59:08 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.271: INFO: Container kube-multus ready: true, restart count 1 May 28 21:30:42.271: INFO: docker-registry-docker-registry-56cbc7bc58-rbghz started at 2021-05-28 20:02:55 +0000 UTC (0+2 container statuses recorded) May 28 21:30:42.271: INFO: Container docker-registry ready: true, restart count 0 May 28 21:30:42.271: INFO: Container nginx ready: true, restart count 0 W0528 21:30:42.284080 37 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 28 21:30:42.317: INFO: Latency metrics for node master1 May 28 21:30:42.317: INFO: Logging node info for node master2 May 28 21:30:42.319: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 b80f32b6-a396-4f09-a110-345a08d762ee 31980 0 2021-05-28 19:57:04 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"b2:be:c9:d8:cf:bb"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-28 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-28 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-28 19:59:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-28 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-28 20:06:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-28 20:00:49 +0000 UTC,LastTransitionTime:2021-05-28 20:00:49 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-28 21:30:34 +0000 UTC,LastTransitionTime:2021-05-28 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-28 21:30:34 +0000 UTC,LastTransitionTime:2021-05-28 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-28 21:30:34 +0000 UTC,LastTransitionTime:2021-05-28 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-28 21:30:34 +0000 UTC,LastTransitionTime:2021-05-28 20:00:43 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:2746caf91c53460599f165aa716150cd,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:b63b522f-706f-4e28-a104-c73edcd04319,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 28 21:30:42.320: INFO: Logging kubelet events for node master2 May 28 21:30:42.322: INFO: Logging pods the kubelet thinks is on node master2 May 28 21:30:42.330: INFO: kube-scheduler-master2 started at 2021-05-28 20:05:21 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.330: INFO: Container kube-scheduler ready: true, restart count 3 May 28 21:30:42.330: INFO: kube-proxy-jkbl8 started at 2021-05-28 19:58:24 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.330: INFO: Container kube-proxy ready: true, restart count 1 May 28 21:30:42.330: INFO: dns-autoscaler-5b7b5c9b6f-r797x started at 2021-05-28 19:59:31 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.330: INFO: Container autoscaler ready: true, restart count 1 May 28 21:30:42.330: INFO: node-exporter-frch9 started at 2021-05-28 20:10:09 +0000 UTC (0+2 container statuses recorded) May 28 21:30:42.330: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:30:42.330: INFO: Container node-exporter ready: true, restart count 0 May 28 21:30:42.330: INFO: node-feature-discovery-controller-5bf5c49849-n9ncl started at 2021-05-28 20:05:52 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.330: INFO: Container nfd-controller ready: true, restart count 0 May 28 21:30:42.330: INFO: kube-apiserver-master2 started at 2021-05-28 20:05:31 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.330: INFO: Container kube-apiserver ready: true, restart count 0 May 28 21:30:42.330: INFO: kube-controller-manager-master2 started at 2021-05-28 20:05:41 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.330: INFO: Container kube-controller-manager ready: true, restart count 3 May 28 21:30:42.330: INFO: kube-flannel-xvtkj started at 2021-05-28 19:59:00 +0000 UTC (1+1 container statuses recorded) May 28 21:30:42.331: INFO: Init container install-cni ready: true, restart count 0 May 28 21:30:42.331: INFO: Container kube-flannel ready: true, restart count 2 May 28 21:30:42.331: INFO: kube-multus-ds-amd64-qjwcz started at 2021-05-28 19:59:08 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.331: INFO: Container kube-multus ready: true, restart count 1 W0528 21:30:42.342289 37 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 28 21:30:42.370: INFO: Latency metrics for node master2 May 28 21:30:42.370: INFO: Logging node info for node master3 May 28 21:30:42.373: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 301b0b5b-fc42-4c78-adb7-75baf6e0cc7e 32010 0 2021-05-28 19:57:14 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"52:fa:ab:49:88:02"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-28 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-28 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-28 19:59:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-28 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-28 20:02:12 +0000 UTC,LastTransitionTime:2021-05-28 20:02:12 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-28 21:30:38 +0000 UTC,LastTransitionTime:2021-05-28 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-28 21:30:38 +0000 UTC,LastTransitionTime:2021-05-28 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-28 21:30:38 +0000 UTC,LastTransitionTime:2021-05-28 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-28 21:30:38 +0000 UTC,LastTransitionTime:2021-05-28 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a0cb6c0eb1d842469076fff344213c13,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:c6adcff4-8bf7-40d7-9d14-54b1c6a87bc8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 28 21:30:42.373: INFO: Logging kubelet events for node master3 May 28 21:30:42.376: INFO: Logging pods the kubelet thinks is on node master3 May 28 21:30:42.385: INFO: kube-proxy-t5bh6 started at 2021-05-28 19:58:24 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.385: INFO: Container kube-proxy ready: true, restart count 1 May 28 21:30:42.385: INFO: kube-flannel-zrskq started at 2021-05-28 19:59:00 +0000 UTC (1+1 container statuses recorded) May 28 21:30:42.385: INFO: Init container install-cni ready: true, restart count 0 May 28 21:30:42.385: INFO: Container kube-flannel ready: true, restart count 1 May 28 21:30:42.385: INFO: kube-multus-ds-amd64-wqgf7 started at 2021-05-28 19:59:08 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.385: INFO: Container kube-multus ready: true, restart count 1 May 28 21:30:42.385: INFO: coredns-7677f9bb54-8v554 started at 2021-05-28 19:59:28 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.385: INFO: Container coredns ready: true, restart count 1 May 28 21:30:42.385: INFO: node-exporter-w42s5 started at 2021-05-28 20:10:09 +0000 UTC (0+2 container statuses recorded) May 28 21:30:42.385: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:30:42.385: INFO: Container node-exporter ready: true, restart count 0 May 28 21:30:42.385: INFO: kube-apiserver-master3 started at 2021-05-28 20:05:21 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.385: INFO: Container kube-apiserver ready: true, restart count 0 May 28 21:30:42.385: INFO: kube-controller-manager-master3 started at 2021-05-28 20:06:02 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.385: INFO: Container kube-controller-manager ready: true, restart count 1 May 28 21:30:42.385: INFO: kube-scheduler-master3 started at 2021-05-28 20:01:23 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.385: INFO: Container kube-scheduler ready: true, restart count 1 W0528 21:30:42.399012 37 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 28 21:30:42.428: INFO: Latency metrics for node master3 May 28 21:30:42.428: INFO: Logging node info for node node1 May 28 21:30:42.430: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 43e51cb4-5acb-42b5-8f26-cd5e977f3829 32008 0 2021-05-28 19:58:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"d2:9d:b7:73:58:07"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-05-28 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubeadm Update v1 2021-05-28 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-28 19:59:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-28 20:06:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-28 20:08:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-05-28 20:08:43 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269628928 0} {<nil>} 196552372Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884628480 0} {<nil>} 174692020Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-28 20:01:58 +0000 UTC,LastTransitionTime:2021-05-28 20:01:58 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-28 21:30:37 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-28 21:30:37 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-28 21:30:37 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-28 21:30:37 +0000 UTC,LastTransitionTime:2021-05-28 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:abe6e95dbfa24a9abd34d8fa2abe7655,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:17719d1f-7df5-4d95-81f3-7d3ac5110ba2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:d731a0fc49b9ad6125b8d5dcb29da2b60bc940b48eacb6f5a9eb2a55c10598db localhost:30500/barometer-collectd:stable],SizeBytes:1464395058,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:1002495332,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:97953d03767e4c2eb5d156394aeaf4bb0b74f3fd1ad08c303cb7561e272a00ff cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:aa24a0a337084e0747e7c8e97e1131270ae38150e691314f1fa19f4b2f9093c0 golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:2bec7a43da8efe70cb7cb14020a6b10aecd02c87e020d394de84e6807e2cf620 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392623,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 28 21:30:42.431: INFO: Logging kubelet events for node node1 May 28 21:30:42.433: INFO: Logging pods the kubelet thinks is on node node1 May 28 21:30:42.451: INFO: kubernetes-metrics-scraper-678c97765c-wblkm started at 2021-05-28 19:59:33 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.451: INFO: Container kubernetes-metrics-scraper ready: true, restart count 1 May 28 21:30:42.451: INFO: kubernetes-dashboard-86c6f9df5b-c5sbq started at 2021-05-28 19:59:33 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.451: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 28 21:30:42.451: INFO: node-feature-discovery-worker-5x4qg started at 2021-05-28 20:05:52 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.451: INFO: Container nfd-worker ready: true, restart count 0 May 28 21:30:42.451: INFO: webserver-deployment-dd94f59b7-5zxws started at 2021-05-28 21:29:31 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.451: INFO: Container httpd ready: false, restart count 0 May 28 21:30:42.451: INFO: pod-init-ff5fe8e0-d2d6-4fa2-bccb-878785589a0b started at 2021-05-28 21:25:42 +0000 UTC (2+1 container statuses recorded) May 28 21:30:42.451: INFO: Init container init1 ready: false, restart count 0 May 28 21:30:42.451: INFO: Init container init2 ready: false, restart count 0 May 28 21:30:42.451: INFO: Container run1 ready: false, restart count 0 May 28 21:30:42.451: INFO: affinity-nodeport-transition-k92kz started at 2021-05-28 21:29:38 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.451: INFO: Container affinity-nodeport-transition ready: true, restart count 0 May 28 21:30:42.451: INFO: nginx-proxy-node1 started at 2021-05-28 20:05:21 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.451: INFO: Container nginx-proxy ready: true, restart count 1 May 28 21:30:42.451: INFO: cmk-init-discover-node1-rvqxm started at 2021-05-28 20:08:32 +0000 UTC (0+3 container statuses recorded) May 28 21:30:42.451: INFO: Container discover ready: false, restart count 0 May 28 21:30:42.451: INFO: Container init ready: false, restart count 0 May 28 21:30:42.451: INFO: Container install ready: false, restart count 0 May 28 21:30:42.451: INFO: cmk-jhzjr started at 2021-05-28 20:09:15 +0000 UTC (0+2 container statuses recorded) May 28 21:30:42.451: INFO: Container nodereport ready: true, restart count 0 May 28 21:30:42.451: INFO: Container reconcile ready: true, restart count 0 May 28 21:30:42.451: INFO: prometheus-k8s-0 started at 2021-05-28 20:10:26 +0000 UTC (0+5 container statuses recorded) May 28 21:30:42.451: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 28 21:30:42.451: INFO: Container grafana ready: true, restart count 0 May 28 21:30:42.451: INFO: Container prometheus ready: true, restart count 1 May 28 21:30:42.451: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 28 21:30:42.451: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 28 21:30:42.451: INFO: webserver-deployment-dd94f59b7-dblrg started at 2021-05-28 21:29:31 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.451: INFO: Container httpd ready: false, restart count 0 May 28 21:30:42.451: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-zk2pt started at 2021-05-28 20:06:47 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.451: INFO: Container kube-sriovdp ready: true, restart count 0 May 28 21:30:42.451: INFO: node-exporter-khdpg started at 2021-05-28 20:10:09 +0000 UTC (0+2 container statuses recorded) May 28 21:30:42.451: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:30:42.451: INFO: Container node-exporter ready: true, restart count 0 May 28 21:30:42.451: INFO: webserver-deployment-dd94f59b7-dffjh started at 2021-05-28 21:29:31 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.451: INFO: Container httpd ready: false, restart count 0 May 28 21:30:42.451: INFO: fail-once-local-sgxcs started at 2021-05-28 21:19:10 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.451: INFO: Container c ready: false, restart count 0 May 28 21:30:42.451: INFO: webserver-deployment-dd94f59b7-m8r8b started at 2021-05-28 21:29:31 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.451: INFO: Container httpd ready: false, restart count 0 May 28 21:30:42.451: INFO: kube-proxy-lsngv started at 2021-05-28 19:58:24 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.451: INFO: Container kube-proxy ready: true, restart count 2 May 28 21:30:42.451: INFO: kube-multus-ds-amd64-x7826 started at 2021-05-28 19:59:08 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.451: INFO: Container kube-multus ready: true, restart count 1 May 28 21:30:42.451: INFO: collectd-qw9nd started at 2021-05-28 20:16:29 +0000 UTC (0+3 container statuses recorded) May 28 21:30:42.451: INFO: Container collectd ready: true, restart count 0 May 28 21:30:42.451: INFO: Container collectd-exporter ready: true, restart count 0 May 28 21:30:42.451: INFO: Container rbac-proxy ready: true, restart count 0 May 28 21:30:42.451: INFO: webserver-deployment-dd94f59b7-z9xcm started at 2021-05-28 21:29:31 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.451: INFO: Container httpd ready: false, restart count 0 May 28 21:30:42.451: INFO: sample-webhook-deployment-cbccbf6bb-4nljk started at 2021-05-28 21:30:40 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.451: INFO: Container sample-webhook ready: false, restart count 0 May 28 21:30:42.451: INFO: kube-flannel-2tjjt started at 2021-05-28 19:59:00 +0000 UTC (1+1 container statuses recorded) May 28 21:30:42.451: INFO: Init container install-cni ready: true, restart count 0 May 28 21:30:42.451: INFO: Container kube-flannel ready: true, restart count 2 W0528 21:30:42.464981 37 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 28 21:30:42.522: INFO: Latency metrics for node node1 May 28 21:30:42.522: INFO: Logging node info for node node2 May 28 21:30:42.525: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 3cc89580-b568-4c82-bd1f-200d0823da3b 32011 0 2021-05-28 19:58:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"62:22:2c:ae:14:ae"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-05-28 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubeadm Update v1 2021-05-28 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-28 19:59:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-28 20:06:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-28 20:08:58 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-05-28 20:09:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269633024 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884632576 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-28 20:01:05 +0000 UTC,LastTransitionTime:2021-05-28 20:01:05 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-28 21:30:38 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-28 21:30:38 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-28 21:30:38 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-28 21:30:38 +0000 UTC,LastTransitionTime:2021-05-28 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b2730c4b09814ab9a78e7bc62c820fbb,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:f1459072-d21d-46de-a5d9-46ec9349aae0,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:d731a0fc49b9ad6125b8d5dcb29da2b60bc940b48eacb6f5a9eb2a55c10598db localhost:30500/barometer-collectd:stable],SizeBytes:1464395058,},ContainerImage{Names:[localhost:30500/cmk@sha256:97953d03767e4c2eb5d156394aeaf4bb0b74f3fd1ad08c303cb7561e272a00ff localhost:30500/cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:2bec7a43da8efe70cb7cb14020a6b10aecd02c87e020d394de84e6807e2cf620 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392623,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:7f3d9945acdf5d86edd89b2b16fe1f6d63ba8bdb4cab50e66f9bce162df9e388 localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:9af6075c93013910787a4e97973da6e0739a86dee1186d7965a5d00b1ac35636 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 28 21:30:42.525: INFO: Logging kubelet events for node node2 May 28 21:30:42.528: INFO: Logging pods the kubelet thinks is on node node2 May 28 21:30:42.543: INFO: webserver-deployment-dd94f59b7-fctjg started at 2021-05-28 21:29:31 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.543: INFO: Container httpd ready: false, restart count 0 May 28 21:30:42.543: INFO: nginx-proxy-node2 started at 2021-05-28 20:05:21 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.543: INFO: Container nginx-proxy ready: true, restart count 2 May 28 21:30:42.543: INFO: collectd-b5dgp started at 2021-05-28 20:16:29 +0000 UTC (0+3 container statuses recorded) May 28 21:30:42.543: INFO: Container collectd ready: true, restart count 0 May 28 21:30:42.543: INFO: Container collectd-exporter ready: true, restart count 0 May 28 21:30:42.543: INFO: Container rbac-proxy ready: true, restart count 0 May 28 21:30:42.543: INFO: var-expansion-0aa9202a-ff7e-468c-a79e-ace324a064c7 started at 2021-05-28 21:28:22 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.543: INFO: Container dapi-container ready: false, restart count 0 May 28 21:30:42.543: INFO: termination-message-container86e5580e-c21e-4f60-bfa7-ed53bbdb01dd started at 2021-05-28 21:29:08 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.543: INFO: Container termination-message-container ready: false, restart count 0 May 28 21:30:42.543: INFO: e2e-test-httpd-pod started at 2021-05-28 21:22:56 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.543: INFO: Container e2e-test-httpd-pod ready: false, restart count 0 May 28 21:30:42.543: INFO: webserver-deployment-dd94f59b7-6vj8h started at 2021-05-28 21:29:31 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.543: INFO: Container httpd ready: false, restart count 0 May 28 21:30:42.543: INFO: affinity-nodeport-transition-xzp5t started at 2021-05-28 21:29:38 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.543: INFO: Container affinity-nodeport-transition ready: true, restart count 0 May 28 21:30:42.543: INFO: affinity-nodeport-transition-q28f2 started at 2021-05-28 21:29:38 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.543: INFO: Container affinity-nodeport-transition ready: true, restart count 0 May 28 21:30:42.543: INFO: execpod-affinityv5qbd started at 2021-05-28 21:29:47 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.543: INFO: Container agnhost-container ready: true, restart count 0 May 28 21:30:42.543: INFO: kube-proxy-z5czn started at 2021-05-28 19:58:24 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.543: INFO: Container kube-proxy ready: true, restart count 2 May 28 21:30:42.543: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-29vc6 started at 2021-05-28 20:06:47 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.543: INFO: Container kube-sriovdp ready: true, restart count 0 May 28 21:30:42.543: INFO: cmk-init-discover-node2-95cbr started at 2021-05-28 20:08:52 +0000 UTC (0+3 container statuses recorded) May 28 21:30:42.543: INFO: Container discover ready: false, restart count 0 May 28 21:30:42.543: INFO: Container init ready: false, restart count 0 May 28 21:30:42.543: INFO: Container install ready: false, restart count 0 May 28 21:30:42.543: INFO: node-feature-discovery-worker-j2wnf started at 2021-05-28 20:05:52 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.543: INFO: Container nfd-worker ready: true, restart count 0 May 28 21:30:42.543: INFO: cmk-5vxwj started at 2021-05-28 20:09:16 +0000 UTC (0+2 container statuses recorded) May 28 21:30:42.543: INFO: Container nodereport ready: true, restart count 0 May 28 21:30:42.543: INFO: Container reconcile ready: true, restart count 0 May 28 21:30:42.543: INFO: fail-once-local-64bh4 started at 2021-05-28 21:19:10 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.543: INFO: Container c ready: false, restart count 0 May 28 21:30:42.543: INFO: webserver-deployment-dd94f59b7-9lmgn started at 2021-05-28 21:29:31 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.543: INFO: Container httpd ready: false, restart count 0 May 28 21:30:42.543: INFO: pod-configmaps-7e4a87b3-903e-484b-8ff5-b9519024e72c started at 2021-05-28 21:28:20 +0000 UTC (0+2 container statuses recorded) May 28 21:30:42.543: INFO: Container configmap-volume-binary-test ready: false, restart count 0 May 28 21:30:42.543: INFO: Container configmap-volume-data-test ready: true, restart count 0 May 28 21:30:42.543: INFO: webserver-deployment-dd94f59b7-2b9cz started at 2021-05-28 21:29:31 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.543: INFO: Container httpd ready: false, restart count 0 May 28 21:30:42.543: INFO: kube-flannel-d9wsg started at 2021-05-28 19:59:00 +0000 UTC (1+1 container statuses recorded) May 28 21:30:42.544: INFO: Init container install-cni ready: true, restart count 2 May 28 21:30:42.544: INFO: Container kube-flannel ready: true, restart count 2 May 28 21:30:42.544: INFO: kube-multus-ds-amd64-c9cj2 started at 2021-05-28 19:59:08 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.544: INFO: Container kube-multus ready: true, restart count 1 May 28 21:30:42.544: INFO: cmk-webhook-6c9d5f8578-6cpp6 started at 2021-05-28 20:09:16 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.544: INFO: Container cmk-webhook ready: true, restart count 0 May 28 21:30:42.544: INFO: node-exporter-sjsht started at 2021-05-28 20:10:09 +0000 UTC (0+2 container statuses recorded) May 28 21:30:42.544: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:30:42.544: INFO: Container node-exporter ready: true, restart count 0 May 28 21:30:42.544: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-csd8v started at 2021-05-28 20:13:00 +0000 UTC (0+2 container statuses recorded) May 28 21:30:42.544: INFO: Container tas-controller ready: true, restart count 0 May 28 21:30:42.544: INFO: Container tas-extender ready: true, restart count 0 May 28 21:30:42.544: INFO: to-be-attached-pod started at 2021-05-28 21:30:35 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.544: INFO: Container container1 ready: true, restart count 0 May 28 21:30:42.544: INFO: webserver-deployment-dd94f59b7-txdtn started at 2021-05-28 21:29:31 +0000 UTC (0+1 container statuses recorded) May 28 21:30:42.544: INFO: Container httpd ready: false, restart count 0 W0528 21:30:42.557055 37 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 28 21:30:42.602: INFO: Latency metrics for node node2 May 28 21:30:42.602: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "init-container-559" for this suite. /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 May 28 21:35:42.684: Timed out after 300.001s. Expected <int32>: 0 to equal <int32>: 2 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/runtime.go:111 [BeforeEach] [k8s.io] Container Runtime /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 28 21:30:42.651: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename container-runtime STEP: Waiting for a default service account to be provisioned in namespace [It] should run with the expected status [NodeConformance] [Conformance] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 STEP: Container 'terminate-cmd-rpa': should get the expected 'RestartCount' May 28 21:35:42.684: FAIL: Timed out after 300.001s. Expected <int32>: 0 to equal <int32>: 2 Full Stack Trace k8s.io/kubernetes/test/e2e/common.glob..func25.1.1.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/runtime.go:111 +0x8ce k8s.io/kubernetes/test/e2e.RunE2ETests(0xc00254f980) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc00254f980) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc00254f980, 0x4de5140) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [k8s.io] Container Runtime /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "container-runtime-7021". STEP: Found 7 events. May 28 21:35:42.697: INFO: At 2021-05-28 21:30:42 +0000 UTC - event for terminate-cmd-rpaf16149aa-2596-49fd-8941-e59a29b66e43: {default-scheduler } Scheduled: Successfully assigned container-runtime-7021/terminate-cmd-rpaf16149aa-2596-49fd-8941-e59a29b66e43 to node1 May 28 21:35:42.697: INFO: At 2021-05-28 21:30:44 +0000 UTC - event for terminate-cmd-rpaf16149aa-2596-49fd-8941-e59a29b66e43: {multus } AddedInterface: Add eth0 [10.244.4.122/24] May 28 21:35:42.697: INFO: At 2021-05-28 21:30:44 +0000 UTC - event for terminate-cmd-rpaf16149aa-2596-49fd-8941-e59a29b66e43: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 28 21:35:42.697: INFO: At 2021-05-28 21:30:45 +0000 UTC - event for terminate-cmd-rpaf16149aa-2596-49fd-8941-e59a29b66e43: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 28 21:35:42.697: INFO: At 2021-05-28 21:30:45 +0000 UTC - event for terminate-cmd-rpaf16149aa-2596-49fd-8941-e59a29b66e43: {kubelet node1} Failed: Error: ErrImagePull May 28 21:35:42.697: INFO: At 2021-05-28 21:30:45 +0000 UTC - event for terminate-cmd-rpaf16149aa-2596-49fd-8941-e59a29b66e43: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 28 21:35:42.697: INFO: At 2021-05-28 21:30:45 +0000 UTC - event for terminate-cmd-rpaf16149aa-2596-49fd-8941-e59a29b66e43: {kubelet node1} Failed: Error: ImagePullBackOff May 28 21:35:42.699: INFO: POD NODE PHASE GRACE CONDITIONS May 28 21:35:42.699: INFO: May 28 21:35:42.704: INFO: Logging node info for node master1 May 28 21:35:42.706: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 0aa78934-442a-44a3-8c5c-f827e18dd3d7 35538 0 2021-05-28 19:56:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:41:0b:9d:15:5a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-28 19:56:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-28 19:56:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-28 19:59:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-28 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-28 20:02:03 +0000 UTC,LastTransitionTime:2021-05-28 20:02:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-28 21:35:39 +0000 UTC,LastTransitionTime:2021-05-28 19:56:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-28 21:35:39 +0000 UTC,LastTransitionTime:2021-05-28 19:56:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-28 21:35:39 +0000 UTC,LastTransitionTime:2021-05-28 19:56:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-28 21:35:39 +0000 UTC,LastTransitionTime:2021-05-28 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f7fb2c462cae4b9c990ab2e5c72f7816,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:24c06694-15ae-4da4-9143-144d98afdd8d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:7f3d9945acdf5d86edd89b2b16fe1f6d63ba8bdb4cab50e66f9bce162df9e388 tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:9af6075c93013910787a4e97973da6e0739a86dee1186d7965a5d00b1ac35636 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 28 21:35:42.706: INFO: Logging kubelet events for node master1 May 28 21:35:42.709: INFO: Logging pods the kubelet thinks is on node master1 May 28 21:35:42.719: INFO: prometheus-operator-5bb8cb9d8f-7wdtq started at 2021-05-28 20:10:02 +0000 UTC (0+2 container statuses recorded) May 28 21:35:42.719: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:35:42.719: INFO: Container prometheus-operator ready: true, restart count 0 May 28 21:35:42.719: INFO: kube-scheduler-master1 started at 2021-05-28 19:57:39 +0000 UTC (0+1 container statuses recorded) May 28 21:35:42.719: INFO: Container kube-scheduler ready: true, restart count 0 May 28 21:35:42.719: INFO: kube-apiserver-master1 started at 2021-05-28 20:05:21 +0000 UTC (0+1 container statuses recorded) May 28 21:35:42.719: INFO: Container kube-apiserver ready: true, restart count 0 May 28 21:35:42.719: INFO: kube-controller-manager-master1 started at 2021-05-28 19:57:39 +0000 UTC (0+1 container statuses recorded) May 28 21:35:42.719: INFO: Container kube-controller-manager ready: true, restart count 2 May 28 21:35:42.719: INFO: kube-multus-ds-amd64-n9j8k started at 2021-05-28 19:59:08 +0000 UTC (0+1 container statuses recorded) May 28 21:35:42.719: INFO: Container kube-multus ready: true, restart count 1 May 28 21:35:42.720: INFO: docker-registry-docker-registry-56cbc7bc58-rbghz started at 2021-05-28 20:02:55 +0000 UTC (0+2 container statuses recorded) May 28 21:35:42.720: INFO: Container docker-registry ready: true, restart count 0 May 28 21:35:42.720: INFO: Container nginx ready: true, restart count 0 May 28 21:35:42.720: INFO: node-exporter-9b7pq started at 2021-05-28 20:10:09 +0000 UTC (0+2 container statuses recorded) May 28 21:35:42.720: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:35:42.720: INFO: Container node-exporter ready: true, restart count 0 May 28 21:35:42.720: INFO: kube-proxy-994p2 started at 2021-05-28 19:58:24 +0000 UTC (0+1 container statuses recorded) May 28 21:35:42.720: INFO: Container kube-proxy ready: true, restart count 1 May 28 21:35:42.720: INFO: kube-flannel-d54gm started at 2021-05-28 19:59:00 +0000 UTC (1+1 container statuses recorded) May 28 21:35:42.720: INFO: Init container install-cni ready: true, restart count 0 May 28 21:35:42.720: INFO: Container kube-flannel ready: true, restart count 2 May 28 21:35:42.720: INFO: coredns-7677f9bb54-zb7h8 started at 2021-05-28 19:59:33 +0000 UTC (0+1 container statuses recorded) May 28 21:35:42.720: INFO: Container coredns ready: true, restart count 1 W0528 21:35:42.733189 37 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 28 21:35:42.763: INFO: Latency metrics for node master1 May 28 21:35:42.763: INFO: Logging node info for node master2 May 28 21:35:42.766: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 b80f32b6-a396-4f09-a110-345a08d762ee 35524 0 2021-05-28 19:57:04 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"b2:be:c9:d8:cf:bb"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-28 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-28 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-28 19:59:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-28 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-28 20:06:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-28 20:00:49 +0000 UTC,LastTransitionTime:2021-05-28 20:00:49 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-28 21:35:36 +0000 UTC,LastTransitionTime:2021-05-28 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-28 21:35:36 +0000 UTC,LastTransitionTime:2021-05-28 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-28 21:35:36 +0000 UTC,LastTransitionTime:2021-05-28 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-28 21:35:36 +0000 UTC,LastTransitionTime:2021-05-28 20:00:43 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:2746caf91c53460599f165aa716150cd,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:b63b522f-706f-4e28-a104-c73edcd04319,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 28 21:35:42.767: INFO: Logging kubelet events for node master2 May 28 21:35:42.770: INFO: Logging pods the kubelet thinks is on node master2 May 28 21:35:42.779: INFO: kube-apiserver-master2 started at 2021-05-28 20:05:31 +0000 UTC (0+1 container statuses recorded) May 28 21:35:42.779: INFO: Container kube-apiserver ready: true, restart count 0 May 28 21:35:42.779: INFO: kube-controller-manager-master2 started at 2021-05-28 20:05:41 +0000 UTC (0+1 container statuses recorded) May 28 21:35:42.779: INFO: Container kube-controller-manager ready: true, restart count 3 May 28 21:35:42.779: INFO: kube-flannel-xvtkj started at 2021-05-28 19:59:00 +0000 UTC (1+1 container statuses recorded) May 28 21:35:42.779: INFO: Init container install-cni ready: true, restart count 0 May 28 21:35:42.779: INFO: Container kube-flannel ready: true, restart count 2 May 28 21:35:42.779: INFO: kube-multus-ds-amd64-qjwcz started at 2021-05-28 19:59:08 +0000 UTC (0+1 container statuses recorded) May 28 21:35:42.779: INFO: Container kube-multus ready: true, restart count 1 May 28 21:35:42.779: INFO: node-feature-discovery-controller-5bf5c49849-n9ncl started at 2021-05-28 20:05:52 +0000 UTC (0+1 container statuses recorded) May 28 21:35:42.779: INFO: Container nfd-controller ready: true, restart count 0 May 28 21:35:42.779: INFO: kube-scheduler-master2 started at 2021-05-28 20:05:21 +0000 UTC (0+1 container statuses recorded) May 28 21:35:42.779: INFO: Container kube-scheduler ready: true, restart count 3 May 28 21:35:42.779: INFO: kube-proxy-jkbl8 started at 2021-05-28 19:58:24 +0000 UTC (0+1 container statuses recorded) May 28 21:35:42.779: INFO: Container kube-proxy ready: true, restart count 1 May 28 21:35:42.779: INFO: dns-autoscaler-5b7b5c9b6f-r797x started at 2021-05-28 19:59:31 +0000 UTC (0+1 container statuses recorded) May 28 21:35:42.779: INFO: Container autoscaler ready: true, restart count 1 May 28 21:35:42.779: INFO: node-exporter-frch9 started at 2021-05-28 20:10:09 +0000 UTC (0+2 container statuses recorded) May 28 21:35:42.779: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:35:42.779: INFO: Container node-exporter ready: true, restart count 0 W0528 21:35:42.791759 37 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 28 21:35:42.820: INFO: Latency metrics for node master2 May 28 21:35:42.820: INFO: Logging node info for node master3 May 28 21:35:42.823: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 301b0b5b-fc42-4c78-adb7-75baf6e0cc7e 35537 0 2021-05-28 19:57:14 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"52:fa:ab:49:88:02"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-28 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-28 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-28 19:59:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-28 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-28 20:02:12 +0000 UTC,LastTransitionTime:2021-05-28 20:02:12 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-28 21:35:39 +0000 UTC,LastTransitionTime:2021-05-28 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-28 21:35:39 +0000 UTC,LastTransitionTime:2021-05-28 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-28 21:35:39 +0000 UTC,LastTransitionTime:2021-05-28 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-28 21:35:39 +0000 UTC,LastTransitionTime:2021-05-28 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a0cb6c0eb1d842469076fff344213c13,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:c6adcff4-8bf7-40d7-9d14-54b1c6a87bc8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 28 21:35:42.824: INFO: Logging kubelet events for node master3 May 28 21:35:42.825: INFO: Logging pods the kubelet thinks is on node master3 May 28 21:35:42.834: INFO: kube-scheduler-master3 started at 2021-05-28 20:01:23 +0000 UTC (0+1 container statuses recorded) May 28 21:35:42.834: INFO: Container kube-scheduler ready: true, restart count 1 May 28 21:35:42.834: INFO: kube-proxy-t5bh6 started at 2021-05-28 19:58:24 +0000 UTC (0+1 container statuses recorded) May 28 21:35:42.834: INFO: Container kube-proxy ready: true, restart count 1 May 28 21:35:42.834: INFO: kube-flannel-zrskq started at 2021-05-28 19:59:00 +0000 UTC (1+1 container statuses recorded) May 28 21:35:42.834: INFO: Init container install-cni ready: true, restart count 0 May 28 21:35:42.834: INFO: Container kube-flannel ready: true, restart count 1 May 28 21:35:42.834: INFO: kube-multus-ds-amd64-wqgf7 started at 2021-05-28 19:59:08 +0000 UTC (0+1 container statuses recorded) May 28 21:35:42.834: INFO: Container kube-multus ready: true, restart count 1 May 28 21:35:42.834: INFO: coredns-7677f9bb54-8v554 started at 2021-05-28 19:59:28 +0000 UTC (0+1 container statuses recorded) May 28 21:35:42.834: INFO: Container coredns ready: true, restart count 1 May 28 21:35:42.834: INFO: node-exporter-w42s5 started at 2021-05-28 20:10:09 +0000 UTC (0+2 container statuses recorded) May 28 21:35:42.834: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:35:42.834: INFO: Container node-exporter ready: true, restart count 0 May 28 21:35:42.834: INFO: kube-apiserver-master3 started at 2021-05-28 20:05:21 +0000 UTC (0+1 container statuses recorded) May 28 21:35:42.834: INFO: Container kube-apiserver ready: true, restart count 0 May 28 21:35:42.834: INFO: kube-controller-manager-master3 started at 2021-05-28 20:06:02 +0000 UTC (0+1 container statuses recorded) May 28 21:35:42.834: INFO: Container kube-controller-manager ready: true, restart count 1 W0528 21:35:42.848315 37 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 28 21:35:42.878: INFO: Latency metrics for node master3 May 28 21:35:42.878: INFO: Logging node info for node node1 May 28 21:35:42.882: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 43e51cb4-5acb-42b5-8f26-cd5e977f3829 35540 0 2021-05-28 19:58:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"d2:9d:b7:73:58:07"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-05-28 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubeadm Update v1 2021-05-28 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-28 19:59:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-28 20:06:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-28 20:08:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-05-28 20:08:43 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269628928 0} {<nil>} 196552372Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884628480 0} {<nil>} 174692020Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-28 20:01:58 +0000 UTC,LastTransitionTime:2021-05-28 20:01:58 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-28 21:35:39 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-28 21:35:39 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-28 21:35:39 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-28 21:35:39 +0000 UTC,LastTransitionTime:2021-05-28 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:abe6e95dbfa24a9abd34d8fa2abe7655,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:17719d1f-7df5-4d95-81f3-7d3ac5110ba2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:d731a0fc49b9ad6125b8d5dcb29da2b60bc940b48eacb6f5a9eb2a55c10598db localhost:30500/barometer-collectd:stable],SizeBytes:1464395058,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:1002495332,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:97953d03767e4c2eb5d156394aeaf4bb0b74f3fd1ad08c303cb7561e272a00ff cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:aa24a0a337084e0747e7c8e97e1131270ae38150e691314f1fa19f4b2f9093c0 golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:2bec7a43da8efe70cb7cb14020a6b10aecd02c87e020d394de84e6807e2cf620 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392623,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 28 21:35:42.882: INFO: Logging kubelet events for node node1 May 28 21:35:42.885: INFO: Logging pods the kubelet thinks is on node node1 May 28 21:35:42.900: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-zk2pt started at 2021-05-28 20:06:47 +0000 UTC (0+1 container statuses recorded) May 28 21:35:42.900: INFO: Container kube-sriovdp ready: true, restart count 0 May 28 21:35:42.900: INFO: cmk-init-discover-node1-rvqxm started at 2021-05-28 20:08:32 +0000 UTC (0+3 container statuses recorded) May 28 21:35:42.900: INFO: Container discover ready: false, restart count 0 May 28 21:35:42.900: INFO: Container init ready: false, restart count 0 May 28 21:35:42.900: INFO: Container install ready: false, restart count 0 May 28 21:35:42.900: INFO: cmk-jhzjr started at 2021-05-28 20:09:15 +0000 UTC (0+2 container statuses recorded) May 28 21:35:42.900: INFO: Container nodereport ready: true, restart count 0 May 28 21:35:42.900: INFO: Container reconcile ready: true, restart count 0 May 28 21:35:42.900: INFO: prometheus-k8s-0 started at 2021-05-28 20:10:26 +0000 UTC (0+5 container statuses recorded) May 28 21:35:42.900: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 28 21:35:42.900: INFO: Container grafana ready: true, restart count 0 May 28 21:35:42.900: INFO: Container prometheus ready: true, restart count 1 May 28 21:35:42.900: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 28 21:35:42.900: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 28 21:35:42.900: INFO: kube-proxy-lsngv started at 2021-05-28 19:58:24 +0000 UTC (0+1 container statuses recorded) May 28 21:35:42.900: INFO: Container kube-proxy ready: true, restart count 2 May 28 21:35:42.900: INFO: node-exporter-khdpg started at 2021-05-28 20:10:09 +0000 UTC (0+2 container statuses recorded) May 28 21:35:42.900: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:35:42.900: INFO: Container node-exporter ready: true, restart count 0 May 28 21:35:42.900: INFO: pod-submit-remove-803c0a05-665d-419b-a3b2-58be13e6c6d4 started at 2021-05-28 21:34:46 +0000 UTC (0+1 container statuses recorded) May 28 21:35:42.900: INFO: Container nginx ready: false, restart count 0 May 28 21:35:42.900: INFO: kube-flannel-2tjjt started at 2021-05-28 19:59:00 +0000 UTC (1+1 container statuses recorded) May 28 21:35:42.900: INFO: Init container install-cni ready: true, restart count 0 May 28 21:35:42.900: INFO: Container kube-flannel ready: true, restart count 2 May 28 21:35:42.900: INFO: kube-multus-ds-amd64-x7826 started at 2021-05-28 19:59:08 +0000 UTC (0+1 container statuses recorded) May 28 21:35:42.900: INFO: Container kube-multus ready: true, restart count 1 May 28 21:35:42.900: INFO: collectd-qw9nd started at 2021-05-28 20:16:29 +0000 UTC (0+3 container statuses recorded) May 28 21:35:42.900: INFO: Container collectd ready: true, restart count 0 May 28 21:35:42.900: INFO: Container collectd-exporter ready: true, restart count 0 May 28 21:35:42.900: INFO: Container rbac-proxy ready: true, restart count 0 May 28 21:35:42.900: INFO: downward-api-ae51b8b3-41dd-40dd-83d5-3705e223dbce started at 2021-05-28 21:34:46 +0000 UTC (0+1 container statuses recorded) May 28 21:35:42.900: INFO: Container dapi-container ready: false, restart count 0 May 28 21:35:42.900: INFO: busybox-host-aliases83919f30-981f-4af3-9806-93a0489bacba started at 2021-05-28 21:32:07 +0000 UTC (0+1 container statuses recorded) May 28 21:35:42.900: INFO: Container busybox-host-aliases83919f30-981f-4af3-9806-93a0489bacba ready: false, restart count 0 May 28 21:35:42.900: INFO: server-envvars-fbef63c6-af61-4aec-aa68-26ecb9d3cbcd started at 2021-05-28 21:32:57 +0000 UTC (0+1 container statuses recorded) May 28 21:35:42.900: INFO: Container srv ready: true, restart count 0 May 28 21:35:42.900: INFO: nginx-proxy-node1 started at 2021-05-28 20:05:21 +0000 UTC (0+1 container statuses recorded) May 28 21:35:42.900: INFO: Container nginx-proxy ready: true, restart count 1 May 28 21:35:42.900: INFO: kubernetes-metrics-scraper-678c97765c-wblkm started at 2021-05-28 19:59:33 +0000 UTC (0+1 container statuses recorded) May 28 21:35:42.900: INFO: Container kubernetes-metrics-scraper ready: true, restart count 1 May 28 21:35:42.900: INFO: kubernetes-dashboard-86c6f9df5b-c5sbq started at 2021-05-28 19:59:33 +0000 UTC (0+1 container statuses recorded) May 28 21:35:42.900: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 28 21:35:42.900: INFO: node-feature-discovery-worker-5x4qg started at 2021-05-28 20:05:52 +0000 UTC (0+1 container statuses recorded) May 28 21:35:42.900: INFO: Container nfd-worker ready: true, restart count 0 May 28 21:35:42.900: INFO: affinity-nodeport-pq5hr started at 2021-05-28 21:33:36 +0000 UTC (0+1 container statuses recorded) May 28 21:35:42.900: INFO: Container affinity-nodeport ready: true, restart count 0 May 28 21:35:42.900: INFO: execpod-affinityjgw6m started at 2021-05-28 21:33:42 +0000 UTC (0+1 container statuses recorded) May 28 21:35:42.900: INFO: Container agnhost-container ready: true, restart count 0 W0528 21:35:42.913616 37 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 28 21:35:43.037: INFO: Latency metrics for node node1 May 28 21:35:43.037: INFO: Logging node info for node node2 May 28 21:35:43.040: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 3cc89580-b568-4c82-bd1f-200d0823da3b 35544 0 2021-05-28 19:58:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"62:22:2c:ae:14:ae"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-05-28 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubeadm Update v1 2021-05-28 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-28 19:59:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-28 20:06:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-28 20:08:58 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-05-28 20:09:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269633024 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884632576 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-28 20:01:05 +0000 UTC,LastTransitionTime:2021-05-28 20:01:05 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-28 21:35:40 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-28 21:35:40 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-28 21:35:40 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-28 21:35:40 +0000 UTC,LastTransitionTime:2021-05-28 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b2730c4b09814ab9a78e7bc62c820fbb,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:f1459072-d21d-46de-a5d9-46ec9349aae0,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:d731a0fc49b9ad6125b8d5dcb29da2b60bc940b48eacb6f5a9eb2a55c10598db localhost:30500/barometer-collectd:stable],SizeBytes:1464395058,},ContainerImage{Names:[localhost:30500/cmk@sha256:97953d03767e4c2eb5d156394aeaf4bb0b74f3fd1ad08c303cb7561e272a00ff localhost:30500/cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:2bec7a43da8efe70cb7cb14020a6b10aecd02c87e020d394de84e6807e2cf620 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392623,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:7f3d9945acdf5d86edd89b2b16fe1f6d63ba8bdb4cab50e66f9bce162df9e388 localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:9af6075c93013910787a4e97973da6e0739a86dee1186d7965a5d00b1ac35636 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 28 21:35:43.041: INFO: Logging kubelet events for node node2 May 28 21:35:43.044: INFO: Logging pods the kubelet thinks is on node node2 May 28 21:35:43.057: INFO: pod-configmaps-86fdeb65-d48e-477b-a9ac-0c6515e94435 started at 2021-05-28 21:34:10 +0000 UTC (0+1 container statuses recorded) May 28 21:35:43.057: INFO: Container configmap-volume-test ready: true, restart count 0 May 28 21:35:43.057: INFO: nginx-proxy-node2 started at 2021-05-28 20:05:21 +0000 UTC (0+1 container statuses recorded) May 28 21:35:43.057: INFO: Container nginx-proxy ready: true, restart count 2 May 28 21:35:43.057: INFO: collectd-b5dgp started at 2021-05-28 20:16:29 +0000 UTC (0+3 container statuses recorded) May 28 21:35:43.057: INFO: Container collectd ready: true, restart count 0 May 28 21:35:43.057: INFO: Container collectd-exporter ready: true, restart count 0 May 28 21:35:43.057: INFO: Container rbac-proxy ready: true, restart count 0 May 28 21:35:43.057: INFO: affinity-nodeport-9hfzp started at 2021-05-28 21:33:36 +0000 UTC (0+1 container statuses recorded) May 28 21:35:43.057: INFO: Container affinity-nodeport ready: true, restart count 0 May 28 21:35:43.057: INFO: pod-init-f9fc6c0e-0b50-4950-bded-4154253e44cd started at 2021-05-28 21:33:55 +0000 UTC (2+1 container statuses recorded) May 28 21:35:43.057: INFO: Init container init1 ready: false, restart count 0 May 28 21:35:43.057: INFO: Init container init2 ready: false, restart count 0 May 28 21:35:43.057: INFO: Container run1 ready: false, restart count 0 May 28 21:35:43.057: INFO: kube-proxy-z5czn started at 2021-05-28 19:58:24 +0000 UTC (0+1 container statuses recorded) May 28 21:35:43.057: INFO: Container kube-proxy ready: true, restart count 2 May 28 21:35:43.057: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-29vc6 started at 2021-05-28 20:06:47 +0000 UTC (0+1 container statuses recorded) May 28 21:35:43.057: INFO: Container kube-sriovdp ready: true, restart count 0 May 28 21:35:43.057: INFO: cmk-init-discover-node2-95cbr started at 2021-05-28 20:08:52 +0000 UTC (0+3 container statuses recorded) May 28 21:35:43.057: INFO: Container discover ready: false, restart count 0 May 28 21:35:43.057: INFO: Container init ready: false, restart count 0 May 28 21:35:43.057: INFO: Container install ready: false, restart count 0 May 28 21:35:43.057: INFO: pod-adoption started at 2021-05-28 21:33:32 +0000 UTC (0+1 container statuses recorded) May 28 21:35:43.057: INFO: Container pod-adoption ready: false, restart count 0 May 28 21:35:43.057: INFO: client-envvars-93f265fc-7cad-42e9-855d-cf2307eac934 started at 2021-05-28 21:33:01 +0000 UTC (0+1 container statuses recorded) May 28 21:35:43.057: INFO: Container env3cont ready: false, restart count 0 May 28 21:35:43.057: INFO: kube-flannel-d9wsg started at 2021-05-28 19:59:00 +0000 UTC (1+1 container statuses recorded) May 28 21:35:43.057: INFO: Init container install-cni ready: true, restart count 2 May 28 21:35:43.057: INFO: Container kube-flannel ready: true, restart count 2 May 28 21:35:43.057: INFO: kube-multus-ds-amd64-c9cj2 started at 2021-05-28 19:59:08 +0000 UTC (0+1 container statuses recorded) May 28 21:35:43.057: INFO: Container kube-multus ready: true, restart count 1 May 28 21:35:43.057: INFO: termination-message-containere6aa6f27-bc3a-4aab-add1-0aa8a502a109 started at 2021-05-28 21:33:41 +0000 UTC (0+1 container statuses recorded) May 28 21:35:43.057: INFO: Container termination-message-container ready: false, restart count 0 May 28 21:35:43.057: INFO: affinity-nodeport-lvhg2 started at 2021-05-28 21:33:36 +0000 UTC (0+1 container statuses recorded) May 28 21:35:43.057: INFO: Container affinity-nodeport ready: true, restart count 0 May 28 21:35:43.057: INFO: node-feature-discovery-worker-j2wnf started at 2021-05-28 20:05:52 +0000 UTC (0+1 container statuses recorded) May 28 21:35:43.057: INFO: Container nfd-worker ready: true, restart count 0 May 28 21:35:43.057: INFO: cmk-5vxwj started at 2021-05-28 20:09:16 +0000 UTC (0+2 container statuses recorded) May 28 21:35:43.057: INFO: Container nodereport ready: true, restart count 0 May 28 21:35:43.057: INFO: Container reconcile ready: true, restart count 0 May 28 21:35:43.057: INFO: cmk-webhook-6c9d5f8578-6cpp6 started at 2021-05-28 20:09:16 +0000 UTC (0+1 container statuses recorded) May 28 21:35:43.057: INFO: Container cmk-webhook ready: true, restart count 0 May 28 21:35:43.057: INFO: node-exporter-sjsht started at 2021-05-28 20:10:09 +0000 UTC (0+2 container statuses recorded) May 28 21:35:43.057: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:35:43.057: INFO: Container node-exporter ready: true, restart count 0 May 28 21:35:43.057: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-csd8v started at 2021-05-28 20:13:00 +0000 UTC (0+2 container statuses recorded) May 28 21:35:43.057: INFO: Container tas-controller ready: true, restart count 0 May 28 21:35:43.057: INFO: Container tas-extender ready: true, restart count 0 W0528 21:35:43.070856 37 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 28 21:35:43.104: INFO: Latency metrics for node node2 May 28 21:35:43.104: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "container-runtime-7021" for this suite. /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 May 28 21:40:55.490: Unexpected error: <*errors.errorString | 0xc0002c6210>: { s: "timed out waiting for the condition", } timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/pods.go:103 [BeforeEach] [k8s.io] Kubelet /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 28 21:35:55.443: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename kubelet-test STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [k8s.io] Kubelet /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/kubelet.go:38 [It] should print the output to logs [NodeConformance] [Conformance] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 May 28 21:40:55.490: FAIL: Unexpected error: <*errors.errorString | 0xc0002c6210>: { s: "timed out waiting for the condition", } timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/framework.(*PodClient).CreateSync(0xc004aecf40, 0xc004c96c00, 0x211) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/pods.go:103 +0xfe k8s.io/kubernetes/test/e2e/common.glob..func12.2.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/kubelet.go:50 +0x1af k8s.io/kubernetes/test/e2e.RunE2ETests(0xc00254f980) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc00254f980) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc00254f980, 0x4de5140) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [k8s.io] Kubelet /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "kubelet-test-7353". STEP: Found 9 events. May 28 21:40:55.494: INFO: At 2021-05-28 21:35:55 +0000 UTC - event for busybox-scheduling-004d2aeb-20b8-4a74-b96b-eec6d4ab8eac: {default-scheduler } Scheduled: Successfully assigned kubelet-test-7353/busybox-scheduling-004d2aeb-20b8-4a74-b96b-eec6d4ab8eac to node2 May 28 21:40:55.494: INFO: At 2021-05-28 21:35:56 +0000 UTC - event for busybox-scheduling-004d2aeb-20b8-4a74-b96b-eec6d4ab8eac: {multus } AddedInterface: Add eth0 [10.244.3.169/24] May 28 21:40:55.494: INFO: At 2021-05-28 21:35:56 +0000 UTC - event for busybox-scheduling-004d2aeb-20b8-4a74-b96b-eec6d4ab8eac: {kubelet node2} Pulling: Pulling image "docker.io/library/busybox:1.29" May 28 21:40:55.494: INFO: At 2021-05-28 21:35:57 +0000 UTC - event for busybox-scheduling-004d2aeb-20b8-4a74-b96b-eec6d4ab8eac: {kubelet node2} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 28 21:40:55.494: INFO: At 2021-05-28 21:35:57 +0000 UTC - event for busybox-scheduling-004d2aeb-20b8-4a74-b96b-eec6d4ab8eac: {kubelet node2} Failed: Error: ErrImagePull May 28 21:40:55.494: INFO: At 2021-05-28 21:35:58 +0000 UTC - event for busybox-scheduling-004d2aeb-20b8-4a74-b96b-eec6d4ab8eac: {kubelet node2} SandboxChanged: Pod sandbox changed, it will be killed and re-created. May 28 21:40:55.494: INFO: At 2021-05-28 21:36:00 +0000 UTC - event for busybox-scheduling-004d2aeb-20b8-4a74-b96b-eec6d4ab8eac: {multus } AddedInterface: Add eth0 [10.244.3.170/24] May 28 21:40:55.494: INFO: At 2021-05-28 21:36:00 +0000 UTC - event for busybox-scheduling-004d2aeb-20b8-4a74-b96b-eec6d4ab8eac: {kubelet node2} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 28 21:40:55.494: INFO: At 2021-05-28 21:36:00 +0000 UTC - event for busybox-scheduling-004d2aeb-20b8-4a74-b96b-eec6d4ab8eac: {kubelet node2} Failed: Error: ImagePullBackOff May 28 21:40:55.497: INFO: POD NODE PHASE GRACE CONDITIONS May 28 21:40:55.497: INFO: busybox-scheduling-004d2aeb-20b8-4a74-b96b-eec6d4ab8eac node2 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-28 21:35:55 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-28 21:35:55 +0000 UTC ContainersNotReady containers with unready status: [busybox-scheduling-004d2aeb-20b8-4a74-b96b-eec6d4ab8eac]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-28 21:35:55 +0000 UTC ContainersNotReady containers with unready status: [busybox-scheduling-004d2aeb-20b8-4a74-b96b-eec6d4ab8eac]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-28 21:35:55 +0000 UTC }] May 28 21:40:55.497: INFO: May 28 21:40:55.501: INFO: Logging node info for node master1 May 28 21:40:55.504: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 0aa78934-442a-44a3-8c5c-f827e18dd3d7 38745 0 2021-05-28 19:56:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:41:0b:9d:15:5a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-28 19:56:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-28 19:56:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-28 19:59:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-28 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-28 20:02:03 +0000 UTC,LastTransitionTime:2021-05-28 20:02:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-28 21:40:51 +0000 UTC,LastTransitionTime:2021-05-28 19:56:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-28 21:40:51 +0000 UTC,LastTransitionTime:2021-05-28 19:56:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-28 21:40:51 +0000 UTC,LastTransitionTime:2021-05-28 19:56:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-28 21:40:51 +0000 UTC,LastTransitionTime:2021-05-28 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f7fb2c462cae4b9c990ab2e5c72f7816,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:24c06694-15ae-4da4-9143-144d98afdd8d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:7f3d9945acdf5d86edd89b2b16fe1f6d63ba8bdb4cab50e66f9bce162df9e388 tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:9af6075c93013910787a4e97973da6e0739a86dee1186d7965a5d00b1ac35636 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 28 21:40:55.504: INFO: Logging kubelet events for node master1 May 28 21:40:55.507: INFO: Logging pods the kubelet thinks is on node master1 May 28 21:40:55.519: INFO: kube-controller-manager-master1 started at 2021-05-28 19:57:39 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.519: INFO: Container kube-controller-manager ready: true, restart count 2 May 28 21:40:55.519: INFO: kube-multus-ds-amd64-n9j8k started at 2021-05-28 19:59:08 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.519: INFO: Container kube-multus ready: true, restart count 1 May 28 21:40:55.519: INFO: docker-registry-docker-registry-56cbc7bc58-rbghz started at 2021-05-28 20:02:55 +0000 UTC (0+2 container statuses recorded) May 28 21:40:55.519: INFO: Container docker-registry ready: true, restart count 0 May 28 21:40:55.519: INFO: Container nginx ready: true, restart count 0 May 28 21:40:55.519: INFO: prometheus-operator-5bb8cb9d8f-7wdtq started at 2021-05-28 20:10:02 +0000 UTC (0+2 container statuses recorded) May 28 21:40:55.519: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:40:55.519: INFO: Container prometheus-operator ready: true, restart count 0 May 28 21:40:55.519: INFO: kube-scheduler-master1 started at 2021-05-28 19:57:39 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.519: INFO: Container kube-scheduler ready: true, restart count 0 May 28 21:40:55.519: INFO: kube-apiserver-master1 started at 2021-05-28 20:05:21 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.519: INFO: Container kube-apiserver ready: true, restart count 0 May 28 21:40:55.519: INFO: kube-proxy-994p2 started at 2021-05-28 19:58:24 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.519: INFO: Container kube-proxy ready: true, restart count 1 May 28 21:40:55.519: INFO: kube-flannel-d54gm started at 2021-05-28 19:59:00 +0000 UTC (1+1 container statuses recorded) May 28 21:40:55.519: INFO: Init container install-cni ready: true, restart count 0 May 28 21:40:55.519: INFO: Container kube-flannel ready: true, restart count 2 May 28 21:40:55.519: INFO: coredns-7677f9bb54-zb7h8 started at 2021-05-28 19:59:33 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.519: INFO: Container coredns ready: true, restart count 1 May 28 21:40:55.519: INFO: node-exporter-9b7pq started at 2021-05-28 20:10:09 +0000 UTC (0+2 container statuses recorded) May 28 21:40:55.520: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:40:55.520: INFO: Container node-exporter ready: true, restart count 0 W0528 21:40:55.532502 37 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 28 21:40:55.559: INFO: Latency metrics for node master1 May 28 21:40:55.559: INFO: Logging node info for node master2 May 28 21:40:55.562: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 b80f32b6-a396-4f09-a110-345a08d762ee 38720 0 2021-05-28 19:57:04 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"b2:be:c9:d8:cf:bb"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-28 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-28 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-28 19:59:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-28 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-28 20:06:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-28 20:00:49 +0000 UTC,LastTransitionTime:2021-05-28 20:00:49 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-28 21:40:47 +0000 UTC,LastTransitionTime:2021-05-28 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-28 21:40:47 +0000 UTC,LastTransitionTime:2021-05-28 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-28 21:40:47 +0000 UTC,LastTransitionTime:2021-05-28 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-28 21:40:47 +0000 UTC,LastTransitionTime:2021-05-28 20:00:43 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:2746caf91c53460599f165aa716150cd,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:b63b522f-706f-4e28-a104-c73edcd04319,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 28 21:40:55.562: INFO: Logging kubelet events for node master2 May 28 21:40:55.564: INFO: Logging pods the kubelet thinks is on node master2 May 28 21:40:55.574: INFO: kube-proxy-jkbl8 started at 2021-05-28 19:58:24 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.574: INFO: Container kube-proxy ready: true, restart count 1 May 28 21:40:55.575: INFO: dns-autoscaler-5b7b5c9b6f-r797x started at 2021-05-28 19:59:31 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.575: INFO: Container autoscaler ready: true, restart count 1 May 28 21:40:55.575: INFO: node-exporter-frch9 started at 2021-05-28 20:10:09 +0000 UTC (0+2 container statuses recorded) May 28 21:40:55.575: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:40:55.575: INFO: Container node-exporter ready: true, restart count 0 May 28 21:40:55.575: INFO: kube-scheduler-master2 started at 2021-05-28 20:05:21 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.575: INFO: Container kube-scheduler ready: true, restart count 3 May 28 21:40:55.575: INFO: kube-controller-manager-master2 started at 2021-05-28 20:05:41 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.575: INFO: Container kube-controller-manager ready: true, restart count 3 May 28 21:40:55.575: INFO: kube-flannel-xvtkj started at 2021-05-28 19:59:00 +0000 UTC (1+1 container statuses recorded) May 28 21:40:55.575: INFO: Init container install-cni ready: true, restart count 0 May 28 21:40:55.575: INFO: Container kube-flannel ready: true, restart count 2 May 28 21:40:55.575: INFO: kube-multus-ds-amd64-qjwcz started at 2021-05-28 19:59:08 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.575: INFO: Container kube-multus ready: true, restart count 1 May 28 21:40:55.575: INFO: node-feature-discovery-controller-5bf5c49849-n9ncl started at 2021-05-28 20:05:52 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.575: INFO: Container nfd-controller ready: true, restart count 0 May 28 21:40:55.575: INFO: kube-apiserver-master2 started at 2021-05-28 20:05:31 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.575: INFO: Container kube-apiserver ready: true, restart count 0 W0528 21:40:55.589212 37 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 28 21:40:55.612: INFO: Latency metrics for node master2 May 28 21:40:55.612: INFO: Logging node info for node master3 May 28 21:40:55.615: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 301b0b5b-fc42-4c78-adb7-75baf6e0cc7e 38739 0 2021-05-28 19:57:14 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"52:fa:ab:49:88:02"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-28 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-28 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-28 19:59:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-28 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-28 20:02:12 +0000 UTC,LastTransitionTime:2021-05-28 20:02:12 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-28 21:40:50 +0000 UTC,LastTransitionTime:2021-05-28 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-28 21:40:50 +0000 UTC,LastTransitionTime:2021-05-28 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-28 21:40:50 +0000 UTC,LastTransitionTime:2021-05-28 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-28 21:40:50 +0000 UTC,LastTransitionTime:2021-05-28 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a0cb6c0eb1d842469076fff344213c13,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:c6adcff4-8bf7-40d7-9d14-54b1c6a87bc8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 28 21:40:55.615: INFO: Logging kubelet events for node master3 May 28 21:40:55.617: INFO: Logging pods the kubelet thinks is on node master3 May 28 21:40:55.627: INFO: kube-flannel-zrskq started at 2021-05-28 19:59:00 +0000 UTC (1+1 container statuses recorded) May 28 21:40:55.627: INFO: Init container install-cni ready: true, restart count 0 May 28 21:40:55.627: INFO: Container kube-flannel ready: true, restart count 1 May 28 21:40:55.627: INFO: kube-multus-ds-amd64-wqgf7 started at 2021-05-28 19:59:08 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.627: INFO: Container kube-multus ready: true, restart count 1 May 28 21:40:55.627: INFO: coredns-7677f9bb54-8v554 started at 2021-05-28 19:59:28 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.627: INFO: Container coredns ready: true, restart count 1 May 28 21:40:55.627: INFO: node-exporter-w42s5 started at 2021-05-28 20:10:09 +0000 UTC (0+2 container statuses recorded) May 28 21:40:55.627: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:40:55.627: INFO: Container node-exporter ready: true, restart count 0 May 28 21:40:55.627: INFO: kube-apiserver-master3 started at 2021-05-28 20:05:21 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.627: INFO: Container kube-apiserver ready: true, restart count 0 May 28 21:40:55.627: INFO: kube-controller-manager-master3 started at 2021-05-28 20:06:02 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.627: INFO: Container kube-controller-manager ready: true, restart count 1 May 28 21:40:55.627: INFO: kube-scheduler-master3 started at 2021-05-28 20:01:23 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.627: INFO: Container kube-scheduler ready: true, restart count 1 May 28 21:40:55.627: INFO: kube-proxy-t5bh6 started at 2021-05-28 19:58:24 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.627: INFO: Container kube-proxy ready: true, restart count 1 W0528 21:40:55.640794 37 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 28 21:40:55.663: INFO: Latency metrics for node master3 May 28 21:40:55.663: INFO: Logging node info for node node1 May 28 21:40:55.666: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 43e51cb4-5acb-42b5-8f26-cd5e977f3829 38746 0 2021-05-28 19:58:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"d2:9d:b7:73:58:07"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-05-28 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubeadm Update v1 2021-05-28 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-28 19:59:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-28 20:06:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-28 20:08:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-05-28 20:08:43 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269628928 0} {<nil>} 196552372Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884628480 0} {<nil>} 174692020Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-28 20:01:58 +0000 UTC,LastTransitionTime:2021-05-28 20:01:58 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-28 21:40:51 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-28 21:40:51 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-28 21:40:51 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-28 21:40:51 +0000 UTC,LastTransitionTime:2021-05-28 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:abe6e95dbfa24a9abd34d8fa2abe7655,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:17719d1f-7df5-4d95-81f3-7d3ac5110ba2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:d731a0fc49b9ad6125b8d5dcb29da2b60bc940b48eacb6f5a9eb2a55c10598db localhost:30500/barometer-collectd:stable],SizeBytes:1464395058,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:1002495332,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:97953d03767e4c2eb5d156394aeaf4bb0b74f3fd1ad08c303cb7561e272a00ff cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:aa24a0a337084e0747e7c8e97e1131270ae38150e691314f1fa19f4b2f9093c0 golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:2bec7a43da8efe70cb7cb14020a6b10aecd02c87e020d394de84e6807e2cf620 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392623,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 28 21:40:55.667: INFO: Logging kubelet events for node node1 May 28 21:40:55.669: INFO: Logging pods the kubelet thinks is on node node1 May 28 21:40:55.683: INFO: node-feature-discovery-worker-5x4qg started at 2021-05-28 20:05:52 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.684: INFO: Container nfd-worker ready: true, restart count 0 May 28 21:40:55.684: INFO: server-envvars-fbef63c6-af61-4aec-aa68-26ecb9d3cbcd started at 2021-05-28 21:32:57 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.684: INFO: Container srv ready: true, restart count 0 May 28 21:40:55.684: INFO: nginx-proxy-node1 started at 2021-05-28 20:05:21 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.684: INFO: Container nginx-proxy ready: true, restart count 1 May 28 21:40:55.684: INFO: kubernetes-metrics-scraper-678c97765c-wblkm started at 2021-05-28 19:59:33 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.684: INFO: Container kubernetes-metrics-scraper ready: true, restart count 1 May 28 21:40:55.684: INFO: kubernetes-dashboard-86c6f9df5b-c5sbq started at 2021-05-28 19:59:33 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.684: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 28 21:40:55.684: INFO: prometheus-k8s-0 started at 2021-05-28 20:10:26 +0000 UTC (0+5 container statuses recorded) May 28 21:40:55.684: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 28 21:40:55.684: INFO: Container grafana ready: true, restart count 0 May 28 21:40:55.684: INFO: Container prometheus ready: true, restart count 1 May 28 21:40:55.684: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 28 21:40:55.684: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 28 21:40:55.684: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-zk2pt started at 2021-05-28 20:06:47 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.684: INFO: Container kube-sriovdp ready: true, restart count 0 May 28 21:40:55.684: INFO: cmk-init-discover-node1-rvqxm started at 2021-05-28 20:08:32 +0000 UTC (0+3 container statuses recorded) May 28 21:40:55.684: INFO: Container discover ready: false, restart count 0 May 28 21:40:55.684: INFO: Container init ready: false, restart count 0 May 28 21:40:55.684: INFO: Container install ready: false, restart count 0 May 28 21:40:55.684: INFO: cmk-jhzjr started at 2021-05-28 20:09:15 +0000 UTC (0+2 container statuses recorded) May 28 21:40:55.684: INFO: Container nodereport ready: true, restart count 0 May 28 21:40:55.684: INFO: Container reconcile ready: true, restart count 0 May 28 21:40:55.684: INFO: var-expansion-0631260e-be64-437a-abc3-0067ec6056fc started at 2021-05-28 21:39:51 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.684: INFO: Container dapi-container ready: false, restart count 0 May 28 21:40:55.684: INFO: kube-proxy-lsngv started at 2021-05-28 19:58:24 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.684: INFO: Container kube-proxy ready: true, restart count 2 May 28 21:40:55.684: INFO: node-exporter-khdpg started at 2021-05-28 20:10:09 +0000 UTC (0+2 container statuses recorded) May 28 21:40:55.684: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:40:55.684: INFO: Container node-exporter ready: true, restart count 0 May 28 21:40:55.684: INFO: downward-api-db7c4220-02d1-4605-8c6d-a366680de904 started at 2021-05-28 21:37:26 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.684: INFO: Container dapi-container ready: false, restart count 0 May 28 21:40:55.684: INFO: kube-flannel-2tjjt started at 2021-05-28 19:59:00 +0000 UTC (1+1 container statuses recorded) May 28 21:40:55.684: INFO: Init container install-cni ready: true, restart count 0 May 28 21:40:55.684: INFO: Container kube-flannel ready: true, restart count 2 May 28 21:40:55.684: INFO: kube-multus-ds-amd64-x7826 started at 2021-05-28 19:59:08 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.684: INFO: Container kube-multus ready: true, restart count 1 May 28 21:40:55.684: INFO: collectd-qw9nd started at 2021-05-28 20:16:29 +0000 UTC (0+3 container statuses recorded) May 28 21:40:55.684: INFO: Container collectd ready: true, restart count 0 May 28 21:40:55.684: INFO: Container collectd-exporter ready: true, restart count 0 May 28 21:40:55.684: INFO: Container rbac-proxy ready: true, restart count 0 W0528 21:40:55.695918 37 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 28 21:40:55.744: INFO: Latency metrics for node node1 May 28 21:40:55.744: INFO: Logging node info for node node2 May 28 21:40:55.747: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 3cc89580-b568-4c82-bd1f-200d0823da3b 38751 0 2021-05-28 19:58:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"62:22:2c:ae:14:ae"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-05-28 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubeadm Update v1 2021-05-28 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-28 19:59:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-28 20:06:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-28 20:08:58 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-05-28 20:09:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269633024 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884632576 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-28 20:01:05 +0000 UTC,LastTransitionTime:2021-05-28 20:01:05 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-28 21:40:52 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-28 21:40:52 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-28 21:40:52 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-28 21:40:52 +0000 UTC,LastTransitionTime:2021-05-28 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b2730c4b09814ab9a78e7bc62c820fbb,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:f1459072-d21d-46de-a5d9-46ec9349aae0,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:d731a0fc49b9ad6125b8d5dcb29da2b60bc940b48eacb6f5a9eb2a55c10598db localhost:30500/barometer-collectd:stable],SizeBytes:1464395058,},ContainerImage{Names:[localhost:30500/cmk@sha256:97953d03767e4c2eb5d156394aeaf4bb0b74f3fd1ad08c303cb7561e272a00ff localhost:30500/cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:2bec7a43da8efe70cb7cb14020a6b10aecd02c87e020d394de84e6807e2cf620 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392623,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:7f3d9945acdf5d86edd89b2b16fe1f6d63ba8bdb4cab50e66f9bce162df9e388 localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:9af6075c93013910787a4e97973da6e0739a86dee1186d7965a5d00b1ac35636 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 28 21:40:55.748: INFO: Logging kubelet events for node node2 May 28 21:40:55.750: INFO: Logging pods the kubelet thinks is on node node2 May 28 21:40:55.763: INFO: node-exporter-sjsht started at 2021-05-28 20:10:09 +0000 UTC (0+2 container statuses recorded) May 28 21:40:55.763: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:40:55.763: INFO: Container node-exporter ready: true, restart count 0 May 28 21:40:55.763: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-csd8v started at 2021-05-28 20:13:00 +0000 UTC (0+2 container statuses recorded) May 28 21:40:55.763: INFO: Container tas-controller ready: true, restart count 0 May 28 21:40:55.763: INFO: Container tas-extender ready: true, restart count 0 May 28 21:40:55.763: INFO: pod-configmaps-2180dc2a-6106-4fa8-9e3b-493758e4f21e started at 2021-05-28 21:38:41 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.763: INFO: Container env-test ready: false, restart count 0 May 28 21:40:55.763: INFO: cmk-webhook-6c9d5f8578-6cpp6 started at 2021-05-28 20:09:16 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.763: INFO: Container cmk-webhook ready: true, restart count 0 May 28 21:40:55.763: INFO: collectd-b5dgp started at 2021-05-28 20:16:29 +0000 UTC (0+3 container statuses recorded) May 28 21:40:55.764: INFO: Container collectd ready: true, restart count 0 May 28 21:40:55.764: INFO: Container collectd-exporter ready: true, restart count 0 May 28 21:40:55.764: INFO: Container rbac-proxy ready: true, restart count 0 May 28 21:40:55.764: INFO: termination-message-containerbabe6730-3b42-4dce-b45b-bc206cbfe303 started at 2021-05-28 21:36:01 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.764: INFO: Container termination-message-container ready: false, restart count 0 May 28 21:40:55.764: INFO: nginx-proxy-node2 started at 2021-05-28 20:05:21 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.764: INFO: Container nginx-proxy ready: true, restart count 2 May 28 21:40:55.764: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-29vc6 started at 2021-05-28 20:06:47 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.764: INFO: Container kube-sriovdp ready: true, restart count 0 May 28 21:40:55.764: INFO: cmk-init-discover-node2-95cbr started at 2021-05-28 20:08:52 +0000 UTC (0+3 container statuses recorded) May 28 21:40:55.764: INFO: Container discover ready: false, restart count 0 May 28 21:40:55.764: INFO: Container init ready: false, restart count 0 May 28 21:40:55.764: INFO: Container install ready: false, restart count 0 May 28 21:40:55.764: INFO: client-envvars-93f265fc-7cad-42e9-855d-cf2307eac934 started at 2021-05-28 21:38:04 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.764: INFO: Container env3cont ready: false, restart count 0 May 28 21:40:55.764: INFO: kube-proxy-z5czn started at 2021-05-28 19:58:24 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.764: INFO: Container kube-proxy ready: true, restart count 2 May 28 21:40:55.764: INFO: test-rolling-update-controller-d85k8 started at 2021-05-28 21:39:08 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.764: INFO: Container httpd ready: false, restart count 0 May 28 21:40:55.764: INFO: kube-multus-ds-amd64-c9cj2 started at 2021-05-28 19:59:08 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.764: INFO: Container kube-multus ready: true, restart count 1 May 28 21:40:55.764: INFO: tester started at 2021-05-28 21:39:24 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.764: INFO: Container tester ready: false, restart count 0 May 28 21:40:55.764: INFO: kube-flannel-d9wsg started at 2021-05-28 19:59:00 +0000 UTC (1+1 container statuses recorded) May 28 21:40:55.764: INFO: Init container install-cni ready: true, restart count 2 May 28 21:40:55.764: INFO: Container kube-flannel ready: true, restart count 2 May 28 21:40:55.764: INFO: server started at 2021-05-28 21:39:20 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.764: INFO: Container agnhost-container ready: true, restart count 0 May 28 21:40:55.764: INFO: liveness-88e0c64f-821c-4dfd-b7a7-fc00a470ef39 started at 2021-05-28 21:40:00 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.764: INFO: Container liveness ready: true, restart count 3 May 28 21:40:55.764: INFO: busybox-scheduling-004d2aeb-20b8-4a74-b96b-eec6d4ab8eac started at 2021-05-28 21:35:55 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.764: INFO: Container busybox-scheduling-004d2aeb-20b8-4a74-b96b-eec6d4ab8eac ready: false, restart count 0 May 28 21:40:55.764: INFO: test-rollover-controller-thm8c started at 2021-05-28 21:39:51 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.764: INFO: Container httpd ready: false, restart count 0 May 28 21:40:55.764: INFO: cmk-5vxwj started at 2021-05-28 20:09:16 +0000 UTC (0+2 container statuses recorded) May 28 21:40:55.764: INFO: Container nodereport ready: true, restart count 0 May 28 21:40:55.764: INFO: Container reconcile ready: true, restart count 0 May 28 21:40:55.764: INFO: node-feature-discovery-worker-j2wnf started at 2021-05-28 20:05:52 +0000 UTC (0+1 container statuses recorded) May 28 21:40:55.764: INFO: Container nfd-worker ready: true, restart count 0 W0528 21:40:55.779177 37 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 28 21:40:55.828: INFO: Latency metrics for node node2 May 28 21:40:55.828: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "kubelet-test-7353" for this suite. /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 May 28 21:52:42.135: Unexpected error: <*errors.errorString | 0xc000d8e1c0>: { s: "expected pod \"projected-volume-02e11916-3473-4936-a738-961e5c91bbd5\" success: Gave up after waiting 5m0s for pod \"projected-volume-02e11916-3473-4936-a738-961e5c91bbd5\" to be \"Succeeded or Failed\"", } expected pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5" success: Gave up after waiting 5m0s for pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5" to be "Succeeded or Failed" occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:725 [BeforeEach] [sig-storage] Projected combined /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 28 21:47:39.560: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename projected STEP: Waiting for a default service account to be provisioned in namespace [It] should project all components that make up the projection API [Projection][NodeConformance] [Conformance] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 STEP: Creating configMap with name configmap-projected-all-test-volume-2e45b7c5-b504-41b2-abd2-7d2db4ba76a4 STEP: Creating secret with name secret-projected-all-test-volume-55bc7308-2b1e-4a2c-868a-fe96b681907f STEP: Creating a pod to test Check all projections for projected volume plugin May 28 21:47:39.597: INFO: Waiting up to 5m0s for pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5" in namespace "projected-8312" to be "Succeeded or Failed" May 28 21:47:39.603: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 6.134796ms May 28 21:47:41.606: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 2.00920006s May 28 21:47:43.610: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 4.013119105s May 28 21:47:45.614: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 6.016421043s May 28 21:47:47.617: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 8.019289567s May 28 21:47:49.620: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 10.022947952s May 28 21:47:51.625: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 12.027410781s May 28 21:47:53.627: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 14.030244932s May 28 21:47:55.631: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 16.033842587s May 28 21:47:57.634: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 18.036409515s May 28 21:47:59.637: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 20.03977779s May 28 21:48:01.640: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 22.042932758s May 28 21:48:03.643: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 24.045864181s May 28 21:48:05.647: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 26.049380796s May 28 21:48:07.650: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 28.052307859s May 28 21:48:09.653: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 30.055571819s May 28 21:48:11.656: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 32.058465486s May 28 21:48:13.659: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 34.061393097s May 28 21:48:15.662: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 36.064281645s May 28 21:48:17.665: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 38.067628979s May 28 21:48:19.668: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 40.070561843s May 28 21:48:21.672: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 42.074571697s May 28 21:48:23.675: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 44.077442896s May 28 21:48:25.677: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 46.079985431s May 28 21:48:27.681: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 48.083442279s May 28 21:48:29.684: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 50.08637273s May 28 21:48:31.686: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 52.088936118s May 28 21:48:33.689: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 54.091795861s May 28 21:48:35.692: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 56.094576881s May 28 21:48:37.695: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 58.097293293s May 28 21:48:39.698: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.100514698s May 28 21:48:41.700: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.10311611s May 28 21:48:43.703: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.105943431s May 28 21:48:45.706: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.109165448s May 28 21:48:47.710: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.112898029s May 28 21:48:49.713: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.116194319s May 28 21:48:51.717: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.119423667s May 28 21:48:53.719: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.121773706s May 28 21:48:55.722: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.125138842s May 28 21:48:57.725: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.128197859s May 28 21:48:59.730: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.132698324s May 28 21:49:01.734: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.137001056s May 28 21:49:03.739: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.141680277s May 28 21:49:05.743: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.146065409s May 28 21:49:07.746: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.14916259s May 28 21:49:09.751: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.153477852s May 28 21:49:11.754: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.156552735s May 28 21:49:13.757: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.160130603s May 28 21:49:15.761: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.164166142s May 28 21:49:17.765: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.167445102s May 28 21:49:19.768: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.171090897s May 28 21:49:21.771: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.174101452s May 28 21:49:23.775: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.177882409s May 28 21:49:25.778: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.181135703s May 28 21:49:27.782: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.184286661s May 28 21:49:29.785: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.188062221s May 28 21:49:31.789: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.191711668s May 28 21:49:33.792: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.19515125s May 28 21:49:35.800: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.202669199s May 28 21:49:37.803: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.206082054s May 28 21:49:39.807: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.209776014s May 28 21:49:41.812: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.214427419s May 28 21:49:43.817: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.219393951s May 28 21:49:45.820: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.223189713s May 28 21:49:47.824: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.226918787s May 28 21:49:49.828: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.231035666s May 28 21:49:51.832: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.234449524s May 28 21:49:53.835: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.238072307s May 28 21:49:55.838: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.240741226s May 28 21:49:57.842: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.244751121s May 28 21:49:59.846: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.249096953s May 28 21:50:01.850: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.253180884s May 28 21:50:03.856: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.259130977s May 28 21:50:05.860: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.263160559s May 28 21:50:07.865: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.267874533s May 28 21:50:09.869: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.272027692s May 28 21:50:11.872: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.27476899s May 28 21:50:13.875: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.278258999s May 28 21:50:15.879: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.281743238s May 28 21:50:17.882: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.284595094s May 28 21:50:19.885: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.287748533s May 28 21:50:21.889: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.291493643s May 28 21:50:23.891: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.293943769s May 28 21:50:25.894: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.296349778s May 28 21:50:27.896: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.299016197s May 28 21:50:29.900: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.302664529s May 28 21:50:31.903: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.305658769s May 28 21:50:33.906: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.309000154s May 28 21:50:35.910: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.312277716s May 28 21:50:37.913: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.315377448s May 28 21:50:39.916: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.318682728s May 28 21:50:41.919: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.321365427s May 28 21:50:43.922: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.324758943s May 28 21:50:45.927: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.329637216s May 28 21:50:47.930: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.332888618s May 28 21:50:49.934: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.337100944s May 28 21:50:51.937: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.340247064s May 28 21:50:53.942: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.344648974s May 28 21:50:55.946: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.348904549s May 28 21:50:57.950: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.352947053s May 28 21:50:59.954: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.356929974s May 28 21:51:01.958: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.360777974s May 28 21:51:03.961: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.364149494s May 28 21:51:05.964: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.366996387s May 28 21:51:07.968: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.370733491s May 28 21:51:09.972: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.374431029s May 28 21:51:11.975: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.37763337s May 28 21:51:13.978: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.381003756s May 28 21:51:15.981: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.384271097s May 28 21:51:17.985: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.38737057s May 28 21:51:19.988: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.390651712s May 28 21:51:21.993: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.396153292s May 28 21:51:23.997: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.400077124s May 28 21:51:26.000: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.402951911s May 28 21:51:28.004: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.406680092s May 28 21:51:30.007: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.40962114s May 28 21:51:32.010: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.413161282s May 28 21:51:34.015: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.417536047s May 28 21:51:36.018: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.421222687s May 28 21:51:38.021: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.423640533s May 28 21:51:40.025: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.42817047s May 28 21:51:42.030: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.43233969s May 28 21:51:44.032: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.435216887s May 28 21:51:46.037: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.439337006s May 28 21:51:48.039: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.442017336s May 28 21:51:50.043: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.446020502s May 28 21:51:52.046: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.44866253s May 28 21:51:54.049: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.451457913s May 28 21:51:56.051: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.453674444s May 28 21:51:58.055: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.457645529s May 28 21:52:00.059: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.461571644s May 28 21:52:02.062: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.464961173s May 28 21:52:04.066: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.468522695s May 28 21:52:06.069: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.471498215s May 28 21:52:08.072: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.474416193s May 28 21:52:10.075: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.477718768s May 28 21:52:12.078: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.481152617s May 28 21:52:14.081: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.483755778s May 28 21:52:16.084: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.486685707s May 28 21:52:18.087: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.490035381s May 28 21:52:20.090: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.493033056s May 28 21:52:22.094: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.496410292s May 28 21:52:24.096: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.499087613s May 28 21:52:26.099: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.501895708s May 28 21:52:28.103: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.506156302s May 28 21:52:30.106: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.509209693s May 28 21:52:32.109: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.512173139s May 28 21:52:34.112: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.514838065s May 28 21:52:36.115: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.517887482s May 28 21:52:38.118: INFO: Pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.520659321s May 28 21:52:40.125: INFO: Failed to get logs from node "node2" pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5" container "projected-all-volume-test": the server rejected our request for an unknown reason (get pods projected-volume-02e11916-3473-4936-a738-961e5c91bbd5) STEP: delete the pod May 28 21:52:40.129: INFO: Waiting for pod projected-volume-02e11916-3473-4936-a738-961e5c91bbd5 to disappear May 28 21:52:40.131: INFO: Pod projected-volume-02e11916-3473-4936-a738-961e5c91bbd5 still exists May 28 21:52:42.132: INFO: Waiting for pod projected-volume-02e11916-3473-4936-a738-961e5c91bbd5 to disappear May 28 21:52:42.134: INFO: Pod projected-volume-02e11916-3473-4936-a738-961e5c91bbd5 no longer exists May 28 21:52:42.135: FAIL: Unexpected error: <*errors.errorString | 0xc000d8e1c0>: { s: "expected pod \"projected-volume-02e11916-3473-4936-a738-961e5c91bbd5\" success: Gave up after waiting 5m0s for pod \"projected-volume-02e11916-3473-4936-a738-961e5c91bbd5\" to be \"Succeeded or Failed\"", } expected pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5" success: Gave up after waiting 5m0s for pod "projected-volume-02e11916-3473-4936-a738-961e5c91bbd5" to be "Succeeded or Failed" occurred Full Stack Trace k8s.io/kubernetes/test/e2e/framework.(*Framework).testContainerOutputMatcher(0xc000428420, 0x4cdb14e, 0x31, 0xc0016c2400, 0x0, 0xc0011c11a8, 0x3, 0x3, 0x4de8e28) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:725 +0x1ee k8s.io/kubernetes/test/e2e/framework.(*Framework).TestContainerOutput(...) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:525 k8s.io/kubernetes/test/e2e/common.glob..func21.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/projected_combined.go:89 +0xca7 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc00254f980) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc00254f980) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc00254f980, 0x4de5140) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [sig-storage] Projected combined /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "projected-8312". STEP: Found 9 events. May 28 21:52:42.139: INFO: At 2021-05-28 21:47:39 +0000 UTC - event for projected-volume-02e11916-3473-4936-a738-961e5c91bbd5: {default-scheduler } Scheduled: Successfully assigned projected-8312/projected-volume-02e11916-3473-4936-a738-961e5c91bbd5 to node2 May 28 21:52:42.139: INFO: At 2021-05-28 21:47:41 +0000 UTC - event for projected-volume-02e11916-3473-4936-a738-961e5c91bbd5: {multus } AddedInterface: Add eth0 [10.244.3.246/24] May 28 21:52:42.139: INFO: At 2021-05-28 21:47:41 +0000 UTC - event for projected-volume-02e11916-3473-4936-a738-961e5c91bbd5: {kubelet node2} Pulling: Pulling image "docker.io/library/busybox:1.29" May 28 21:52:42.139: INFO: At 2021-05-28 21:47:42 +0000 UTC - event for projected-volume-02e11916-3473-4936-a738-961e5c91bbd5: {kubelet node2} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 28 21:52:42.139: INFO: At 2021-05-28 21:47:42 +0000 UTC - event for projected-volume-02e11916-3473-4936-a738-961e5c91bbd5: {kubelet node2} Failed: Error: ErrImagePull May 28 21:52:42.139: INFO: At 2021-05-28 21:47:43 +0000 UTC - event for projected-volume-02e11916-3473-4936-a738-961e5c91bbd5: {kubelet node2} SandboxChanged: Pod sandbox changed, it will be killed and re-created. May 28 21:52:42.139: INFO: At 2021-05-28 21:47:44 +0000 UTC - event for projected-volume-02e11916-3473-4936-a738-961e5c91bbd5: {multus } AddedInterface: Add eth0 [10.244.3.247/24] May 28 21:52:42.139: INFO: At 2021-05-28 21:47:44 +0000 UTC - event for projected-volume-02e11916-3473-4936-a738-961e5c91bbd5: {kubelet node2} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 28 21:52:42.139: INFO: At 2021-05-28 21:47:44 +0000 UTC - event for projected-volume-02e11916-3473-4936-a738-961e5c91bbd5: {kubelet node2} Failed: Error: ImagePullBackOff May 28 21:52:42.141: INFO: POD NODE PHASE GRACE CONDITIONS May 28 21:52:42.141: INFO: May 28 21:52:42.145: INFO: Logging node info for node master1 May 28 21:52:42.148: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 0aa78934-442a-44a3-8c5c-f827e18dd3d7 45954 0 2021-05-28 19:56:25 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"0a:41:0b:9d:15:5a"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-28 19:56:25 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-28 19:56:26 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-28 19:59:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-28 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-28 20:02:03 +0000 UTC,LastTransitionTime:2021-05-28 20:02:03 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-28 21:52:33 +0000 UTC,LastTransitionTime:2021-05-28 19:56:25 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-28 21:52:33 +0000 UTC,LastTransitionTime:2021-05-28 19:56:25 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-28 21:52:33 +0000 UTC,LastTransitionTime:2021-05-28 19:56:25 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-28 21:52:33 +0000 UTC,LastTransitionTime:2021-05-28 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f7fb2c462cae4b9c990ab2e5c72f7816,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:24c06694-15ae-4da4-9143-144d98afdd8d,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:7f3d9945acdf5d86edd89b2b16fe1f6d63ba8bdb4cab50e66f9bce162df9e388 tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:9af6075c93013910787a4e97973da6e0739a86dee1186d7965a5d00b1ac35636 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 28 21:52:42.148: INFO: Logging kubelet events for node master1 May 28 21:52:42.150: INFO: Logging pods the kubelet thinks is on node master1 May 28 21:52:42.159: INFO: prometheus-operator-5bb8cb9d8f-7wdtq started at 2021-05-28 20:10:02 +0000 UTC (0+2 container statuses recorded) May 28 21:52:42.159: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:52:42.159: INFO: Container prometheus-operator ready: true, restart count 0 May 28 21:52:42.159: INFO: kube-scheduler-master1 started at 2021-05-28 19:57:39 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.159: INFO: Container kube-scheduler ready: true, restart count 0 May 28 21:52:42.159: INFO: kube-apiserver-master1 started at 2021-05-28 20:05:21 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.159: INFO: Container kube-apiserver ready: true, restart count 0 May 28 21:52:42.159: INFO: kube-controller-manager-master1 started at 2021-05-28 19:57:39 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.159: INFO: Container kube-controller-manager ready: true, restart count 2 May 28 21:52:42.159: INFO: kube-multus-ds-amd64-n9j8k started at 2021-05-28 19:59:08 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.159: INFO: Container kube-multus ready: true, restart count 1 May 28 21:52:42.159: INFO: docker-registry-docker-registry-56cbc7bc58-rbghz started at 2021-05-28 20:02:55 +0000 UTC (0+2 container statuses recorded) May 28 21:52:42.159: INFO: Container docker-registry ready: true, restart count 0 May 28 21:52:42.159: INFO: Container nginx ready: true, restart count 0 May 28 21:52:42.159: INFO: node-exporter-9b7pq started at 2021-05-28 20:10:09 +0000 UTC (0+2 container statuses recorded) May 28 21:52:42.159: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:52:42.159: INFO: Container node-exporter ready: true, restart count 0 May 28 21:52:42.159: INFO: kube-proxy-994p2 started at 2021-05-28 19:58:24 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.159: INFO: Container kube-proxy ready: true, restart count 1 May 28 21:52:42.160: INFO: kube-flannel-d54gm started at 2021-05-28 19:59:00 +0000 UTC (1+1 container statuses recorded) May 28 21:52:42.160: INFO: Init container install-cni ready: true, restart count 0 May 28 21:52:42.160: INFO: Container kube-flannel ready: true, restart count 2 May 28 21:52:42.160: INFO: coredns-7677f9bb54-zb7h8 started at 2021-05-28 19:59:33 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.160: INFO: Container coredns ready: true, restart count 1 W0528 21:52:42.171676 37 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 28 21:52:42.194: INFO: Latency metrics for node master1 May 28 21:52:42.194: INFO: Logging node info for node master2 May 28 21:52:42.197: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 b80f32b6-a396-4f09-a110-345a08d762ee 46003 0 2021-05-28 19:57:04 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"b2:be:c9:d8:cf:bb"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-28 19:57:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-28 19:57:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-28 19:59:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-28 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-28 20:06:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-28 20:00:49 +0000 UTC,LastTransitionTime:2021-05-28 20:00:49 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-28 21:52:39 +0000 UTC,LastTransitionTime:2021-05-28 19:57:04 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-28 21:52:39 +0000 UTC,LastTransitionTime:2021-05-28 19:57:04 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-28 21:52:39 +0000 UTC,LastTransitionTime:2021-05-28 19:57:04 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-28 21:52:39 +0000 UTC,LastTransitionTime:2021-05-28 20:00:43 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:2746caf91c53460599f165aa716150cd,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:b63b522f-706f-4e28-a104-c73edcd04319,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 28 21:52:42.197: INFO: Logging kubelet events for node master2 May 28 21:52:42.199: INFO: Logging pods the kubelet thinks is on node master2 May 28 21:52:42.207: INFO: node-feature-discovery-controller-5bf5c49849-n9ncl started at 2021-05-28 20:05:52 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.207: INFO: Container nfd-controller ready: true, restart count 0 May 28 21:52:42.207: INFO: kube-apiserver-master2 started at 2021-05-28 20:05:31 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.207: INFO: Container kube-apiserver ready: true, restart count 0 May 28 21:52:42.207: INFO: kube-controller-manager-master2 started at 2021-05-28 20:05:41 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.207: INFO: Container kube-controller-manager ready: true, restart count 3 May 28 21:52:42.207: INFO: kube-flannel-xvtkj started at 2021-05-28 19:59:00 +0000 UTC (1+1 container statuses recorded) May 28 21:52:42.207: INFO: Init container install-cni ready: true, restart count 0 May 28 21:52:42.207: INFO: Container kube-flannel ready: true, restart count 2 May 28 21:52:42.207: INFO: kube-multus-ds-amd64-qjwcz started at 2021-05-28 19:59:08 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.207: INFO: Container kube-multus ready: true, restart count 1 May 28 21:52:42.207: INFO: kube-scheduler-master2 started at 2021-05-28 20:05:21 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.207: INFO: Container kube-scheduler ready: true, restart count 3 May 28 21:52:42.207: INFO: kube-proxy-jkbl8 started at 2021-05-28 19:58:24 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.207: INFO: Container kube-proxy ready: true, restart count 1 May 28 21:52:42.207: INFO: dns-autoscaler-5b7b5c9b6f-r797x started at 2021-05-28 19:59:31 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.207: INFO: Container autoscaler ready: true, restart count 1 May 28 21:52:42.207: INFO: node-exporter-frch9 started at 2021-05-28 20:10:09 +0000 UTC (0+2 container statuses recorded) May 28 21:52:42.207: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:52:42.207: INFO: Container node-exporter ready: true, restart count 0 W0528 21:52:42.220616 37 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 28 21:52:42.249: INFO: Latency metrics for node master2 May 28 21:52:42.249: INFO: Logging node info for node master3 May 28 21:52:42.251: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 301b0b5b-fc42-4c78-adb7-75baf6e0cc7e 45953 0 2021-05-28 19:57:14 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"52:fa:ab:49:88:02"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-28 19:57:14 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-28 19:57:15 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-28 19:59:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-28 19:59:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-28 20:02:12 +0000 UTC,LastTransitionTime:2021-05-28 20:02:12 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-28 21:52:32 +0000 UTC,LastTransitionTime:2021-05-28 19:57:14 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-28 21:52:32 +0000 UTC,LastTransitionTime:2021-05-28 19:57:14 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-28 21:52:32 +0000 UTC,LastTransitionTime:2021-05-28 19:57:14 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-28 21:52:32 +0000 UTC,LastTransitionTime:2021-05-28 19:59:09 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:a0cb6c0eb1d842469076fff344213c13,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:c6adcff4-8bf7-40d7-9d14-54b1c6a87bc8,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 28 21:52:42.252: INFO: Logging kubelet events for node master3 May 28 21:52:42.254: INFO: Logging pods the kubelet thinks is on node master3 May 28 21:52:42.262: INFO: kube-scheduler-master3 started at 2021-05-28 20:01:23 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.262: INFO: Container kube-scheduler ready: true, restart count 1 May 28 21:52:42.262: INFO: kube-proxy-t5bh6 started at 2021-05-28 19:58:24 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.262: INFO: Container kube-proxy ready: true, restart count 1 May 28 21:52:42.262: INFO: kube-flannel-zrskq started at 2021-05-28 19:59:00 +0000 UTC (1+1 container statuses recorded) May 28 21:52:42.262: INFO: Init container install-cni ready: true, restart count 0 May 28 21:52:42.262: INFO: Container kube-flannel ready: true, restart count 1 May 28 21:52:42.262: INFO: kube-multus-ds-amd64-wqgf7 started at 2021-05-28 19:59:08 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.262: INFO: Container kube-multus ready: true, restart count 1 May 28 21:52:42.262: INFO: coredns-7677f9bb54-8v554 started at 2021-05-28 19:59:28 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.262: INFO: Container coredns ready: true, restart count 1 May 28 21:52:42.262: INFO: node-exporter-w42s5 started at 2021-05-28 20:10:09 +0000 UTC (0+2 container statuses recorded) May 28 21:52:42.262: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:52:42.262: INFO: Container node-exporter ready: true, restart count 0 May 28 21:52:42.262: INFO: kube-apiserver-master3 started at 2021-05-28 20:05:21 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.262: INFO: Container kube-apiserver ready: true, restart count 0 May 28 21:52:42.262: INFO: kube-controller-manager-master3 started at 2021-05-28 20:06:02 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.262: INFO: Container kube-controller-manager ready: true, restart count 1 W0528 21:52:42.273544 37 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 28 21:52:42.304: INFO: Latency metrics for node master3 May 28 21:52:42.304: INFO: Logging node info for node node1 May 28 21:52:42.308: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 43e51cb4-5acb-42b5-8f26-cd5e977f3829 45991 0 2021-05-28 19:58:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"d2:9d:b7:73:58:07"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-05-28 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubeadm Update v1 2021-05-28 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-28 19:59:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-28 20:06:07 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-28 20:08:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-05-28 20:08:43 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269628928 0} {<nil>} 196552372Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884628480 0} {<nil>} 174692020Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-28 20:01:58 +0000 UTC,LastTransitionTime:2021-05-28 20:01:58 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-28 21:52:37 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-28 21:52:37 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-28 21:52:37 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-28 21:52:37 +0000 UTC,LastTransitionTime:2021-05-28 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:abe6e95dbfa24a9abd34d8fa2abe7655,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:17719d1f-7df5-4d95-81f3-7d3ac5110ba2,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:d731a0fc49b9ad6125b8d5dcb29da2b60bc940b48eacb6f5a9eb2a55c10598db localhost:30500/barometer-collectd:stable],SizeBytes:1464395058,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:1002495332,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:97953d03767e4c2eb5d156394aeaf4bb0b74f3fd1ad08c303cb7561e272a00ff cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:aa24a0a337084e0747e7c8e97e1131270ae38150e691314f1fa19f4b2f9093c0 golang:alpine3.12],SizeBytes:301156062,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:2bec7a43da8efe70cb7cb14020a6b10aecd02c87e020d394de84e6807e2cf620 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392623,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 28 21:52:42.309: INFO: Logging kubelet events for node node1 May 28 21:52:42.310: INFO: Logging pods the kubelet thinks is on node node1 May 28 21:52:42.325: INFO: kube-flannel-2tjjt started at 2021-05-28 19:59:00 +0000 UTC (1+1 container statuses recorded) May 28 21:52:42.325: INFO: Init container install-cni ready: true, restart count 0 May 28 21:52:42.325: INFO: Container kube-flannel ready: true, restart count 2 May 28 21:52:42.325: INFO: kube-multus-ds-amd64-x7826 started at 2021-05-28 19:59:08 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.325: INFO: Container kube-multus ready: true, restart count 1 May 28 21:52:42.325: INFO: collectd-qw9nd started at 2021-05-28 20:16:29 +0000 UTC (0+3 container statuses recorded) May 28 21:52:42.325: INFO: Container collectd ready: true, restart count 0 May 28 21:52:42.325: INFO: Container collectd-exporter ready: true, restart count 0 May 28 21:52:42.325: INFO: Container rbac-proxy ready: true, restart count 0 May 28 21:52:42.325: INFO: busybox-f17fb482-4f7e-4929-934e-76bc10d5eb21 started at 2021-05-28 21:52:24 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.325: INFO: Container busybox ready: false, restart count 0 May 28 21:52:42.325: INFO: foo-5v6j9 started at 2021-05-28 21:48:48 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.325: INFO: Container c ready: false, restart count 0 May 28 21:52:42.325: INFO: nginx-proxy-node1 started at 2021-05-28 20:05:21 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.325: INFO: Container nginx-proxy ready: true, restart count 1 May 28 21:52:42.325: INFO: kubernetes-metrics-scraper-678c97765c-wblkm started at 2021-05-28 19:59:33 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.325: INFO: Container kubernetes-metrics-scraper ready: true, restart count 1 May 28 21:52:42.325: INFO: kubernetes-dashboard-86c6f9df5b-c5sbq started at 2021-05-28 19:59:33 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.325: INFO: Container kubernetes-dashboard ready: true, restart count 2 May 28 21:52:42.325: INFO: node-feature-discovery-worker-5x4qg started at 2021-05-28 20:05:52 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.325: INFO: Container nfd-worker ready: true, restart count 0 May 28 21:52:42.325: INFO: downward-api-f4fc3a56-7fae-4d59-9213-e1b490e4e6ba started at 2021-05-28 21:50:51 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.325: INFO: Container dapi-container ready: false, restart count 0 May 28 21:52:42.325: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-zk2pt started at 2021-05-28 20:06:47 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.325: INFO: Container kube-sriovdp ready: true, restart count 0 May 28 21:52:42.325: INFO: cmk-init-discover-node1-rvqxm started at 2021-05-28 20:08:32 +0000 UTC (0+3 container statuses recorded) May 28 21:52:42.325: INFO: Container discover ready: false, restart count 0 May 28 21:52:42.325: INFO: Container init ready: false, restart count 0 May 28 21:52:42.325: INFO: Container install ready: false, restart count 0 May 28 21:52:42.325: INFO: cmk-jhzjr started at 2021-05-28 20:09:15 +0000 UTC (0+2 container statuses recorded) May 28 21:52:42.325: INFO: Container nodereport ready: true, restart count 0 May 28 21:52:42.325: INFO: Container reconcile ready: true, restart count 0 May 28 21:52:42.325: INFO: prometheus-k8s-0 started at 2021-05-28 20:10:26 +0000 UTC (0+5 container statuses recorded) May 28 21:52:42.325: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 28 21:52:42.325: INFO: Container grafana ready: true, restart count 0 May 28 21:52:42.325: INFO: Container prometheus ready: true, restart count 1 May 28 21:52:42.325: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 28 21:52:42.325: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 28 21:52:42.325: INFO: busybox-privileged-false-c15b3720-f23c-48e2-8f8e-a669d903af94 started at 2021-05-28 21:48:55 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.325: INFO: Container busybox-privileged-false-c15b3720-f23c-48e2-8f8e-a669d903af94 ready: false, restart count 0 May 28 21:52:42.325: INFO: kube-proxy-lsngv started at 2021-05-28 19:58:24 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.325: INFO: Container kube-proxy ready: true, restart count 2 May 28 21:52:42.325: INFO: node-exporter-khdpg started at 2021-05-28 20:10:09 +0000 UTC (0+2 container statuses recorded) May 28 21:52:42.325: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:52:42.325: INFO: Container node-exporter ready: true, restart count 0 May 28 21:52:42.325: INFO: var-expansion-f16a0136-bd86-4eae-8c50-bbf3a0c19b1e started at 2021-05-28 21:52:09 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.325: INFO: Container dapi-container ready: false, restart count 0 W0528 21:52:42.338713 37 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 28 21:52:42.377: INFO: Latency metrics for node node1 May 28 21:52:42.377: INFO: Logging node info for node node2 May 28 21:52:42.379: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 3cc89580-b568-4c82-bd1f-200d0823da3b 45989 0 2021-05-28 19:58:22 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"62:22:2c:ae:14:ae"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-05-28 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubeadm Update v1 2021-05-28 19:58:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-28 19:59:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-28 20:06:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-28 20:08:58 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-05-28 20:09:00 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269633024 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884632576 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-28 20:01:05 +0000 UTC,LastTransitionTime:2021-05-28 20:01:05 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-28 21:52:36 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-28 21:52:36 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-28 21:52:36 +0000 UTC,LastTransitionTime:2021-05-28 19:58:22 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-28 21:52:36 +0000 UTC,LastTransitionTime:2021-05-28 19:59:04 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b2730c4b09814ab9a78e7bc62c820fbb,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:f1459072-d21d-46de-a5d9-46ec9349aae0,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:d731a0fc49b9ad6125b8d5dcb29da2b60bc940b48eacb6f5a9eb2a55c10598db localhost:30500/barometer-collectd:stable],SizeBytes:1464395058,},ContainerImage{Names:[localhost:30500/cmk@sha256:97953d03767e4c2eb5d156394aeaf4bb0b74f3fd1ad08c303cb7561e272a00ff localhost:30500/cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726715672,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[nginx@sha256:df13abe416e37eb3db4722840dd479b00ba193ac6606e7902331dcea50f4f1f2 nginx:1.19],SizeBytes:133122553,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:2bec7a43da8efe70cb7cb14020a6b10aecd02c87e020d394de84e6807e2cf620 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44392623,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:7f3d9945acdf5d86edd89b2b16fe1f6d63ba8bdb4cab50e66f9bce162df9e388 localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:9af6075c93013910787a4e97973da6e0739a86dee1186d7965a5d00b1ac35636 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 28 21:52:42.380: INFO: Logging kubelet events for node node2 May 28 21:52:42.382: INFO: Logging pods the kubelet thinks is on node node2 May 28 21:52:42.390: INFO: kube-multus-ds-amd64-c9cj2 started at 2021-05-28 19:59:08 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.390: INFO: Container kube-multus ready: true, restart count 1 May 28 21:52:42.391: INFO: liveness-2f89ae2d-24a9-44a3-a05b-a78c24a0e3ac started at 2021-05-28 21:52:42 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.391: INFO: Container liveness ready: false, restart count 0 May 28 21:52:42.391: INFO: foo-hf8r8 started at 2021-05-28 21:48:48 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.391: INFO: Container c ready: false, restart count 0 May 28 21:52:42.391: INFO: downward-api-059e80da-9086-4b41-b07c-d238e07a5774 started at 2021-05-28 21:49:44 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.391: INFO: Container dapi-container ready: false, restart count 0 May 28 21:52:42.391: INFO: kube-flannel-d9wsg started at 2021-05-28 19:59:00 +0000 UTC (1+1 container statuses recorded) May 28 21:52:42.391: INFO: Init container install-cni ready: true, restart count 2 May 28 21:52:42.391: INFO: Container kube-flannel ready: true, restart count 2 May 28 21:52:42.391: INFO: ss-0 started at 2021-05-28 21:48:11 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.391: INFO: Container webserver ready: false, restart count 0 May 28 21:52:42.391: INFO: cmk-5vxwj started at 2021-05-28 20:09:16 +0000 UTC (0+2 container statuses recorded) May 28 21:52:42.391: INFO: Container nodereport ready: true, restart count 0 May 28 21:52:42.391: INFO: Container reconcile ready: true, restart count 0 May 28 21:52:42.391: INFO: node-feature-discovery-worker-j2wnf started at 2021-05-28 20:05:52 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.391: INFO: Container nfd-worker ready: true, restart count 0 May 28 21:52:42.391: INFO: node-exporter-sjsht started at 2021-05-28 20:10:09 +0000 UTC (0+2 container statuses recorded) May 28 21:52:42.391: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 28 21:52:42.391: INFO: Container node-exporter ready: true, restart count 0 May 28 21:52:42.391: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-csd8v started at 2021-05-28 20:13:00 +0000 UTC (0+2 container statuses recorded) May 28 21:52:42.391: INFO: Container tas-controller ready: true, restart count 0 May 28 21:52:42.391: INFO: Container tas-extender ready: true, restart count 0 May 28 21:52:42.391: INFO: cmk-webhook-6c9d5f8578-6cpp6 started at 2021-05-28 20:09:16 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.391: INFO: Container cmk-webhook ready: true, restart count 0 May 28 21:52:42.391: INFO: collectd-b5dgp started at 2021-05-28 20:16:29 +0000 UTC (0+3 container statuses recorded) May 28 21:52:42.391: INFO: Container collectd ready: true, restart count 0 May 28 21:52:42.391: INFO: Container collectd-exporter ready: true, restart count 0 May 28 21:52:42.391: INFO: Container rbac-proxy ready: true, restart count 0 May 28 21:52:42.391: INFO: ss2-0 started at 2021-05-28 21:51:01 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.391: INFO: Container webserver ready: false, restart count 0 May 28 21:52:42.391: INFO: nginx-proxy-node2 started at 2021-05-28 20:05:21 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.391: INFO: Container nginx-proxy ready: true, restart count 2 May 28 21:52:42.391: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-29vc6 started at 2021-05-28 20:06:47 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.391: INFO: Container kube-sriovdp ready: true, restart count 0 May 28 21:52:42.391: INFO: cmk-init-discover-node2-95cbr started at 2021-05-28 20:08:52 +0000 UTC (0+3 container statuses recorded) May 28 21:52:42.391: INFO: Container discover ready: false, restart count 0 May 28 21:52:42.391: INFO: Container init ready: false, restart count 0 May 28 21:52:42.391: INFO: Container install ready: false, restart count 0 May 28 21:52:42.391: INFO: kube-proxy-z5czn started at 2021-05-28 19:58:24 +0000 UTC (0+1 container statuses recorded) May 28 21:52:42.391: INFO: Container kube-proxy ready: true, restart count 2 W0528 21:52:42.402201 37 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 28 21:52:42.438: INFO: Latency metrics for node node2 May 28 21:52:42.438: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "projected-8312" for this suite.