/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 May 4 16:08:46.900: Unexpected error: <*errors.errorString | 0xc0050d09a0>: { s: "service is not reachable within 2m0s timeout on endpoint 10.10.190.207:32103 over TCP protocol", } service is not reachable within 2m0s timeout on endpoint 10.10.190.207:32103 over TCP protocol occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/network/service.go:1242 [BeforeEach] [sig-network] Services /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 4 16:06:34.484: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename services STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [sig-network] Services /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/network/service.go:782 [It] should be able to create a functioning NodePort service [Conformance] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 STEP: creating service nodeport-test with type=NodePort in namespace services-6457 STEP: creating replication controller nodeport-test in namespace services-6457 I0504 16:06:34.517725 27 runners.go:190] Created replication controller with name: nodeport-test, namespace: services-6457, replica count: 2 I0504 16:06:37.568828 27 runners.go:190] nodeport-test Pods: 2 out of 2 created, 1 running, 1 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady I0504 16:06:40.570102 27 runners.go:190] nodeport-test Pods: 2 out of 2 created, 2 running, 0 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady May 4 16:06:40.570: INFO: Creating new exec pod May 4 16:06:45.594: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 nodeport-test 80' May 4 16:06:45.867: INFO: stderr: "+ nc -zv -t -w 2 nodeport-test 80\nConnection to nodeport-test 80 port [tcp/http] succeeded!\n" May 4 16:06:45.867: INFO: stdout: "" May 4 16:06:45.868: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.233.9.225 80' May 4 16:06:46.114: INFO: stderr: "+ nc -zv -t -w 2 10.233.9.225 80\nConnection to 10.233.9.225 80 port [tcp/http] succeeded!\n" May 4 16:06:46.114: INFO: stdout: "" May 4 16:06:46.114: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:06:46.366: INFO: rc: 1 May 4 16:06:46.366: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:06:47.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:06:48.007: INFO: rc: 1 May 4 16:06:48.007: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:06:48.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:06:48.820: INFO: rc: 1 May 4 16:06:48.820: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:06:49.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:06:49.615: INFO: rc: 1 May 4 16:06:49.615: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:06:50.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:06:50.615: INFO: rc: 1 May 4 16:06:50.615: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:06:51.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:06:51.686: INFO: rc: 1 May 4 16:06:51.686: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:06:52.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:06:52.628: INFO: rc: 1 May 4 16:06:52.628: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:06:53.367: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:06:53.622: INFO: rc: 1 May 4 16:06:53.622: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:06:54.367: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:06:54.829: INFO: rc: 1 May 4 16:06:54.829: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:06:55.367: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:06:55.867: INFO: rc: 1 May 4 16:06:55.867: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:06:56.367: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:06:56.609: INFO: rc: 1 May 4 16:06:56.610: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:06:57.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:06:57.625: INFO: rc: 1 May 4 16:06:57.626: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:06:58.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:06:58.846: INFO: rc: 1 May 4 16:06:58.847: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:06:59.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:06:59.631: INFO: rc: 1 May 4 16:06:59.631: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:00.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:00.620: INFO: rc: 1 May 4 16:07:00.621: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:01.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:01.631: INFO: rc: 1 May 4 16:07:01.631: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:02.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:02.624: INFO: rc: 1 May 4 16:07:02.624: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:03.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:03.634: INFO: rc: 1 May 4 16:07:03.634: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:04.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:05.309: INFO: rc: 1 May 4 16:07:05.309: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:05.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:05.833: INFO: rc: 1 May 4 16:07:05.833: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:06.367: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:07.300: INFO: rc: 1 May 4 16:07:07.300: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:07.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:07.720: INFO: rc: 1 May 4 16:07:07.720: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:08.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:08.994: INFO: rc: 1 May 4 16:07:08.994: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:09.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:09.815: INFO: rc: 1 May 4 16:07:09.815: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:10.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:11.331: INFO: rc: 1 May 4 16:07:11.331: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:11.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:11.756: INFO: rc: 1 May 4 16:07:11.757: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:12.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:12.849: INFO: rc: 1 May 4 16:07:12.849: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:13.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:13.750: INFO: rc: 1 May 4 16:07:13.750: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:14.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:15.098: INFO: rc: 1 May 4 16:07:15.098: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:15.367: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:15.617: INFO: rc: 1 May 4 16:07:15.618: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:16.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:16.695: INFO: rc: 1 May 4 16:07:16.695: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:17.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:17.630: INFO: rc: 1 May 4 16:07:17.630: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:18.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:18.617: INFO: rc: 1 May 4 16:07:18.617: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:19.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:19.637: INFO: rc: 1 May 4 16:07:19.637: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:20.368: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:20.652: INFO: rc: 1 May 4 16:07:20.652: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:21.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:21.753: INFO: rc: 1 May 4 16:07:21.753: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:22.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:22.630: INFO: rc: 1 May 4 16:07:22.631: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:23.368: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:23.640: INFO: rc: 1 May 4 16:07:23.640: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:24.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:24.968: INFO: rc: 1 May 4 16:07:24.968: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:25.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:25.725: INFO: rc: 1 May 4 16:07:25.725: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:26.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:26.659: INFO: rc: 1 May 4 16:07:26.659: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:27.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:27.783: INFO: rc: 1 May 4 16:07:27.783: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:28.367: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:28.653: INFO: rc: 1 May 4 16:07:28.653: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:29.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:29.628: INFO: rc: 1 May 4 16:07:29.628: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:30.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:30.648: INFO: rc: 1 May 4 16:07:30.648: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:31.367: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:31.631: INFO: rc: 1 May 4 16:07:31.631: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:32.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:32.645: INFO: rc: 1 May 4 16:07:32.645: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:33.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:33.647: INFO: rc: 1 May 4 16:07:33.647: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:34.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:34.624: INFO: rc: 1 May 4 16:07:34.624: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:35.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:35.645: INFO: rc: 1 May 4 16:07:35.645: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:36.367: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:36.647: INFO: rc: 1 May 4 16:07:36.647: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:37.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:37.629: INFO: rc: 1 May 4 16:07:37.629: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:38.367: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:38.627: INFO: rc: 1 May 4 16:07:38.627: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:39.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:39.636: INFO: rc: 1 May 4 16:07:39.636: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:40.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:40.638: INFO: rc: 1 May 4 16:07:40.638: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:41.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:41.621: INFO: rc: 1 May 4 16:07:41.621: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:42.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:42.703: INFO: rc: 1 May 4 16:07:42.703: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:43.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:43.638: INFO: rc: 1 May 4 16:07:43.638: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:44.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:44.626: INFO: rc: 1 May 4 16:07:44.626: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:45.367: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:45.622: INFO: rc: 1 May 4 16:07:45.622: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:46.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:46.642: INFO: rc: 1 May 4 16:07:46.642: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:47.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:47.620: INFO: rc: 1 May 4 16:07:47.620: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:48.367: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:48.601: INFO: rc: 1 May 4 16:07:48.602: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:49.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:49.658: INFO: rc: 1 May 4 16:07:49.658: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:50.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:50.588: INFO: rc: 1 May 4 16:07:50.589: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:51.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:51.655: INFO: rc: 1 May 4 16:07:51.655: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:52.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:52.644: INFO: rc: 1 May 4 16:07:52.644: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:53.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:53.800: INFO: rc: 1 May 4 16:07:53.800: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:54.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:54.723: INFO: rc: 1 May 4 16:07:54.723: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:55.367: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:55.631: INFO: rc: 1 May 4 16:07:55.631: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:56.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:56.627: INFO: rc: 1 May 4 16:07:56.627: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:57.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:57.607: INFO: rc: 1 May 4 16:07:57.607: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:58.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:58.630: INFO: rc: 1 May 4 16:07:58.630: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:07:59.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:07:59.611: INFO: rc: 1 May 4 16:07:59.612: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:00.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:00.609: INFO: rc: 1 May 4 16:08:00.609: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:01.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:01.617: INFO: rc: 1 May 4 16:08:01.617: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:02.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:02.616: INFO: rc: 1 May 4 16:08:02.616: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:03.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:03.621: INFO: rc: 1 May 4 16:08:03.621: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:04.367: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:04.633: INFO: rc: 1 May 4 16:08:04.633: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:05.367: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:05.619: INFO: rc: 1 May 4 16:08:05.619: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:06.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:06.884: INFO: rc: 1 May 4 16:08:06.884: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:07.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:07.625: INFO: rc: 1 May 4 16:08:07.625: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:08.368: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:08.637: INFO: rc: 1 May 4 16:08:08.637: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:09.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:09.638: INFO: rc: 1 May 4 16:08:09.638: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:10.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:10.615: INFO: rc: 1 May 4 16:08:10.616: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:11.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:11.800: INFO: rc: 1 May 4 16:08:11.800: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:12.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:12.641: INFO: rc: 1 May 4 16:08:12.641: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:13.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:13.628: INFO: rc: 1 May 4 16:08:13.628: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:14.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:14.632: INFO: rc: 1 May 4 16:08:14.632: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:15.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:15.624: INFO: rc: 1 May 4 16:08:15.624: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:16.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:16.632: INFO: rc: 1 May 4 16:08:16.632: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:17.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:17.685: INFO: rc: 1 May 4 16:08:17.685: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:18.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:18.669: INFO: rc: 1 May 4 16:08:18.669: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:19.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:19.642: INFO: rc: 1 May 4 16:08:19.642: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:20.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:20.667: INFO: rc: 1 May 4 16:08:20.667: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:21.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:21.639: INFO: rc: 1 May 4 16:08:21.639: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:22.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:22.626: INFO: rc: 1 May 4 16:08:22.626: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:23.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:23.625: INFO: rc: 1 May 4 16:08:23.625: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:24.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:24.797: INFO: rc: 1 May 4 16:08:24.797: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:25.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:25.699: INFO: rc: 1 May 4 16:08:25.699: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:26.367: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:26.650: INFO: rc: 1 May 4 16:08:26.650: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:27.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:27.630: INFO: rc: 1 May 4 16:08:27.630: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:28.367: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:28.627: INFO: rc: 1 May 4 16:08:28.627: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:29.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:29.651: INFO: rc: 1 May 4 16:08:29.651: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:30.367: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:30.646: INFO: rc: 1 May 4 16:08:30.646: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:31.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:31.660: INFO: rc: 1 May 4 16:08:31.661: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:32.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:32.952: INFO: rc: 1 May 4 16:08:32.952: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:33.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:33.627: INFO: rc: 1 May 4 16:08:33.628: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:34.367: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:34.616: INFO: rc: 1 May 4 16:08:34.616: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:35.368: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:35.611: INFO: rc: 1 May 4 16:08:35.611: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:36.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:36.639: INFO: rc: 1 May 4 16:08:36.639: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:37.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:37.889: INFO: rc: 1 May 4 16:08:37.889: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:38.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:38.629: INFO: rc: 1 May 4 16:08:38.629: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:39.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:40.057: INFO: rc: 1 May 4 16:08:40.057: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:40.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:40.681: INFO: rc: 1 May 4 16:08:40.681: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:41.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:41.613: INFO: rc: 1 May 4 16:08:41.613: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:42.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:42.632: INFO: rc: 1 May 4 16:08:42.632: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:43.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:43.660: INFO: rc: 1 May 4 16:08:43.660: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:44.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:44.640: INFO: rc: 1 May 4 16:08:44.640: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:45.367: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:45.640: INFO: rc: 1 May 4 16:08:45.640: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:46.366: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:46.632: INFO: rc: 1 May 4 16:08:46.632: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:46.632: INFO: Running '/usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103' May 4 16:08:46.900: INFO: rc: 1 May 4 16:08:46.900: INFO: Service reachability failing with error: error running /usr/local/bin/kubectl --kubeconfig=/root/.kube/config --namespace=services-6457 exec execpodn9wsj -- /bin/sh -x -c nc -zv -t -w 2 10.10.190.207 32103: Command stdout: stderr: + nc -zv -t -w 2 10.10.190.207 32103 nc: connect to 10.10.190.207 port 32103 (tcp) failed: Connection refused command terminated with exit code 1 error: exit status 1 Retrying... May 4 16:08:46.900: FAIL: Unexpected error: <*errors.errorString | 0xc0050d09a0>: { s: "service is not reachable within 2m0s timeout on endpoint 10.10.190.207:32103 over TCP protocol", } service is not reachable within 2m0s timeout on endpoint 10.10.190.207:32103 over TCP protocol occurred Full Stack Trace k8s.io/kubernetes/test/e2e/network.glob..func24.11() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/network/service.go:1242 +0x265 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc002947080) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc002947080) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc002947080, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [sig-network] Services /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "services-6457". STEP: Found 20 events. May 4 16:08:46.916: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for execpodn9wsj: { } Scheduled: Successfully assigned services-6457/execpodn9wsj to node1 May 4 16:08:46.916: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for nodeport-test-lxckq: { } Scheduled: Successfully assigned services-6457/nodeport-test-lxckq to node1 May 4 16:08:46.916: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for nodeport-test-qt2pt: { } Scheduled: Successfully assigned services-6457/nodeport-test-qt2pt to node2 May 4 16:08:46.916: INFO: At 2021-05-04 16:06:34 +0000 UTC - event for nodeport-test: {replication-controller } SuccessfulCreate: Created pod: nodeport-test-qt2pt May 4 16:08:46.916: INFO: At 2021-05-04 16:06:34 +0000 UTC - event for nodeport-test: {replication-controller } SuccessfulCreate: Created pod: nodeport-test-lxckq May 4 16:08:46.916: INFO: At 2021-05-04 16:06:35 +0000 UTC - event for nodeport-test-qt2pt: {multus } AddedInterface: Add eth0 [10.244.3.124/24] May 4 16:08:46.916: INFO: At 2021-05-04 16:06:35 +0000 UTC - event for nodeport-test-qt2pt: {kubelet node2} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 4 16:08:46.916: INFO: At 2021-05-04 16:06:36 +0000 UTC - event for nodeport-test-lxckq: {multus } AddedInterface: Add eth0 [10.244.4.95/24] May 4 16:08:46.916: INFO: At 2021-05-04 16:06:36 +0000 UTC - event for nodeport-test-lxckq: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 4 16:08:46.916: INFO: At 2021-05-04 16:06:36 +0000 UTC - event for nodeport-test-qt2pt: {kubelet node2} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 480.798163ms May 4 16:08:46.916: INFO: At 2021-05-04 16:06:36 +0000 UTC - event for nodeport-test-qt2pt: {kubelet node2} Created: Created container nodeport-test May 4 16:08:46.916: INFO: At 2021-05-04 16:06:36 +0000 UTC - event for nodeport-test-qt2pt: {kubelet node2} Started: Started container nodeport-test May 4 16:08:46.916: INFO: At 2021-05-04 16:06:37 +0000 UTC - event for nodeport-test-lxckq: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 470.078716ms May 4 16:08:46.916: INFO: At 2021-05-04 16:06:37 +0000 UTC - event for nodeport-test-lxckq: {kubelet node1} Started: Started container nodeport-test May 4 16:08:46.916: INFO: At 2021-05-04 16:06:37 +0000 UTC - event for nodeport-test-lxckq: {kubelet node1} Created: Created container nodeport-test May 4 16:08:46.916: INFO: At 2021-05-04 16:06:42 +0000 UTC - event for execpodn9wsj: {kubelet node1} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.20" May 4 16:08:46.916: INFO: At 2021-05-04 16:06:42 +0000 UTC - event for execpodn9wsj: {multus } AddedInterface: Add eth0 [10.244.4.98/24] May 4 16:08:46.916: INFO: At 2021-05-04 16:06:43 +0000 UTC - event for execpodn9wsj: {kubelet node1} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.20" in 664.132475ms May 4 16:08:46.916: INFO: At 2021-05-04 16:06:43 +0000 UTC - event for execpodn9wsj: {kubelet node1} Created: Created container agnhost-container May 4 16:08:46.916: INFO: At 2021-05-04 16:06:43 +0000 UTC - event for execpodn9wsj: {kubelet node1} Started: Started container agnhost-container May 4 16:08:46.919: INFO: POD NODE PHASE GRACE CONDITIONS May 4 16:08:46.919: INFO: execpodn9wsj node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-04 16:06:40 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-04 16:06:44 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-04 16:06:44 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-04 16:06:40 +0000 UTC }] May 4 16:08:46.919: INFO: nodeport-test-lxckq node1 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-04 16:06:34 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-04 16:06:37 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-04 16:06:37 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-04 16:06:34 +0000 UTC }] May 4 16:08:46.919: INFO: nodeport-test-qt2pt node2 Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-04 16:06:34 +0000 UTC } {Ready True 0001-01-01 00:00:00 +0000 UTC 2021-05-04 16:06:37 +0000 UTC } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2021-05-04 16:06:37 +0000 UTC } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-04 16:06:34 +0000 UTC }] May 4 16:08:46.919: INFO: May 4 16:08:46.924: INFO: Logging node info for node master1 May 4 16:08:46.927: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 db982204-549e-4532-90a7-a4410878cfc9 29805 0 2021-05-04 14:43:01 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"3e:f0:43:cb:66:52"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-04 14:43:01 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-04 14:43:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-04 14:45:41 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-04 14:45:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-04 14:51:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-04 14:47:46 +0000 UTC,LastTransitionTime:2021-05-04 14:47:46 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-04 16:08:46 +0000 UTC,LastTransitionTime:2021-05-04 14:43:01 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-04 16:08:46 +0000 UTC,LastTransitionTime:2021-05-04 14:43:01 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-04 16:08:46 +0000 UTC,LastTransitionTime:2021-05-04 14:43:01 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-04 16:08:46 +0000 UTC,LastTransitionTime:2021-05-04 14:47:24 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:88a0771919594d4187f6704fc7592bf8,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:8e0a253b-2aa4-4467-879e-567e7ba1ffa4,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726615179,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:c2fabacbc4e42f3db70f9508e00158b1dce4cf96d91cabaa2eca24e5a0900b66 tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:e074a505e2d62b5119460ab724b2e1df10c8419ef2457f9ce9f3a0f75be3e959 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 4 16:08:46.928: INFO: Logging kubelet events for node master1 May 4 16:08:46.930: INFO: Logging pods the kubelet thinks is on node master1 May 4 16:08:46.956: INFO: node-exporter-jckjs started at 2021-05-04 14:56:10 +0000 UTC (0+2 container statuses recorded) May 4 16:08:46.956: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 4 16:08:46.956: INFO: Container node-exporter ready: true, restart count 0 May 4 16:08:46.956: INFO: kube-scheduler-master1 started at 2021-05-04 14:44:16 +0000 UTC (0+1 container statuses recorded) May 4 16:08:46.956: INFO: Container kube-scheduler ready: true, restart count 0 May 4 16:08:46.956: INFO: kube-apiserver-master1 started at 2021-05-04 14:47:26 +0000 UTC (0+1 container statuses recorded) May 4 16:08:46.956: INFO: Container kube-apiserver ready: true, restart count 0 May 4 16:08:46.956: INFO: kube-controller-manager-master1 started at 2021-05-04 14:47:26 +0000 UTC (0+1 container statuses recorded) May 4 16:08:46.956: INFO: Container kube-controller-manager ready: true, restart count 2 May 4 16:08:46.956: INFO: kube-proxy-8j6ch started at 2021-05-04 14:45:01 +0000 UTC (0+1 container statuses recorded) May 4 16:08:46.956: INFO: Container kube-proxy ready: true, restart count 1 May 4 16:08:46.956: INFO: docker-registry-docker-registry-56cbc7bc58-zhf8t started at 2021-05-04 14:48:42 +0000 UTC (0+2 container statuses recorded) May 4 16:08:46.956: INFO: Container docker-registry ready: true, restart count 0 May 4 16:08:46.956: INFO: Container nginx ready: true, restart count 0 May 4 16:08:46.956: INFO: node-feature-discovery-controller-5bf5c49849-72rn6 started at 2021-05-04 14:51:52 +0000 UTC (0+1 container statuses recorded) May 4 16:08:46.956: INFO: Container nfd-controller ready: true, restart count 0 May 4 16:08:46.956: INFO: kube-flannel-qspzk started at 2021-05-04 14:45:37 +0000 UTC (1+1 container statuses recorded) May 4 16:08:46.956: INFO: Init container install-cni ready: true, restart count 0 May 4 16:08:46.956: INFO: Container kube-flannel ready: true, restart count 3 May 4 16:08:46.956: INFO: kube-multus-ds-amd64-jflvf started at 2021-05-04 14:45:46 +0000 UTC (0+1 container statuses recorded) May 4 16:08:46.956: INFO: Container kube-multus ready: true, restart count 1 May 4 16:08:46.956: INFO: coredns-7677f9bb54-qvcd2 started at 2021-05-04 14:46:11 +0000 UTC (0+1 container statuses recorded) May 4 16:08:46.956: INFO: Container coredns ready: true, restart count 1 W0504 16:08:46.967823 27 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 4 16:08:46.993: INFO: Latency metrics for node master1 May 4 16:08:46.993: INFO: Logging node info for node master2 May 4 16:08:46.995: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e2c15170-247b-4e7b-b818-abc807948bf8 29788 0 2021-05-04 14:43:40 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"16:e0:10:a0:e0:62"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-04 14:43:40 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-04 14:43:41 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-04 14:45:41 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-04 14:45:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-04 14:47:32 +0000 UTC,LastTransitionTime:2021-05-04 14:47:32 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-04 16:08:46 +0000 UTC,LastTransitionTime:2021-05-04 14:43:40 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-04 16:08:46 +0000 UTC,LastTransitionTime:2021-05-04 14:43:40 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-04 16:08:46 +0000 UTC,LastTransitionTime:2021-05-04 14:43:40 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-04 16:08:46 +0000 UTC,LastTransitionTime:2021-05-04 14:47:24 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:6af568f56589422a9bd68e0270ce0f8c,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:bf27bb77-fad2-4b52-85c3-acb5113fc512,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726615179,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 4 16:08:46.996: INFO: Logging kubelet events for node master2 May 4 16:08:46.998: INFO: Logging pods the kubelet thinks is on node master2 May 4 16:08:47.013: INFO: node-exporter-9c6qf started at 2021-05-04 14:56:10 +0000 UTC (0+2 container statuses recorded) May 4 16:08:47.013: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 4 16:08:47.013: INFO: Container node-exporter ready: true, restart count 0 May 4 16:08:47.013: INFO: kube-apiserver-master2 started at 2021-05-04 14:47:26 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.013: INFO: Container kube-apiserver ready: true, restart count 0 May 4 16:08:47.013: INFO: kube-controller-manager-master2 started at 2021-05-04 14:47:26 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.013: INFO: Container kube-controller-manager ready: true, restart count 2 May 4 16:08:47.013: INFO: kube-scheduler-master2 started at 2021-05-04 14:44:16 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.013: INFO: Container kube-scheduler ready: true, restart count 2 May 4 16:08:47.013: INFO: kube-proxy-6b5t8 started at 2021-05-04 14:45:01 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.013: INFO: Container kube-proxy ready: true, restart count 2 May 4 16:08:47.013: INFO: kube-flannel-cxdfr started at 2021-05-04 14:45:37 +0000 UTC (1+1 container statuses recorded) May 4 16:08:47.013: INFO: Init container install-cni ready: true, restart count 0 May 4 16:08:47.013: INFO: Container kube-flannel ready: true, restart count 1 May 4 16:08:47.013: INFO: kube-multus-ds-amd64-dw8tg started at 2021-05-04 14:45:46 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.013: INFO: Container kube-multus ready: true, restart count 1 May 4 16:08:47.013: INFO: dns-autoscaler-5b7b5c9b6f-zbrsq started at 2021-05-04 14:46:08 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.013: INFO: Container autoscaler ready: true, restart count 1 W0504 16:08:47.026350 27 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 4 16:08:47.054: INFO: Latency metrics for node master2 May 4 16:08:47.054: INFO: Logging node info for node master3 May 4 16:08:47.057: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 b533a646-667e-403c-944b-71dec9cc4851 29782 0 2021-05-04 14:43:51 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"16:b0:53:14:f6:c9"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-04 14:43:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-04 14:43:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-04 14:45:42 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-04 14:45:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-04 14:48:50 +0000 UTC,LastTransitionTime:2021-05-04 14:48:50 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-04 16:08:45 +0000 UTC,LastTransitionTime:2021-05-04 14:43:51 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-04 16:08:45 +0000 UTC,LastTransitionTime:2021-05-04 14:43:51 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-04 16:08:45 +0000 UTC,LastTransitionTime:2021-05-04 14:43:51 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-04 16:08:45 +0000 UTC,LastTransitionTime:2021-05-04 14:45:46 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:959373dcda56494486f0c2bb0bb496cc,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:0714ca81-c21e-40d6-a288-48d597238e54,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726615179,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 4 16:08:47.057: INFO: Logging kubelet events for node master3 May 4 16:08:47.059: INFO: Logging pods the kubelet thinks is on node master3 May 4 16:08:47.075: INFO: kube-apiserver-master3 started at 2021-05-04 14:44:16 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.075: INFO: Container kube-apiserver ready: true, restart count 0 May 4 16:08:47.075: INFO: kube-controller-manager-master3 started at 2021-05-04 14:44:16 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.075: INFO: Container kube-controller-manager ready: true, restart count 2 May 4 16:08:47.075: INFO: kube-scheduler-master3 started at 2021-05-04 14:44:16 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.075: INFO: Container kube-scheduler ready: true, restart count 2 May 4 16:08:47.075: INFO: kube-proxy-2p5b6 started at 2021-05-04 14:45:01 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.075: INFO: Container kube-proxy ready: true, restart count 2 May 4 16:08:47.075: INFO: kube-flannel-wznt8 started at 2021-05-04 14:45:37 +0000 UTC (1+1 container statuses recorded) May 4 16:08:47.075: INFO: Init container install-cni ready: true, restart count 0 May 4 16:08:47.075: INFO: Container kube-flannel ready: true, restart count 1 May 4 16:08:47.075: INFO: kube-multus-ds-amd64-cgwz2 started at 2021-05-04 14:45:46 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.075: INFO: Container kube-multus ready: true, restart count 1 May 4 16:08:47.075: INFO: coredns-7677f9bb54-pshfb started at 2021-05-04 14:46:06 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.075: INFO: Container coredns ready: true, restart count 1 May 4 16:08:47.075: INFO: node-exporter-wvppn started at 2021-05-04 14:56:10 +0000 UTC (0+2 container statuses recorded) May 4 16:08:47.075: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 4 16:08:47.075: INFO: Container node-exporter ready: true, restart count 0 W0504 16:08:47.087346 27 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 4 16:08:47.116: INFO: Latency metrics for node master3 May 4 16:08:47.116: INFO: Logging node info for node node1 May 4 16:08:47.119: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 b8cf7e16-d5c7-4e2c-996a-93d93bd4fa1c 29806 0 2021-05-04 14:44:58 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"e2:50:df:03:d2:13"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-05-04 14:44:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubeadm Update v1 2021-05-04 14:44:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-04 14:45:42 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-04 14:52:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-04 14:54:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-05-04 14:54:41 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269633024 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884632576 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-04 14:47:29 +0000 UTC,LastTransitionTime:2021-05-04 14:47:29 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-04 16:08:46 +0000 UTC,LastTransitionTime:2021-05-04 14:44:58 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-04 16:08:46 +0000 UTC,LastTransitionTime:2021-05-04 14:44:58 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-04 16:08:46 +0000 UTC,LastTransitionTime:2021-05-04 14:44:58 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-04 16:08:46 +0000 UTC,LastTransitionTime:2021-05-04 14:47:24 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:bffc023a4ab84df0b0181bc7b8f509e2,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:dc08af72-abca-4f1d-bd0f-0e8d8eb97de5,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:511d36b62d758304a5adb26b5996ed37211ab226beb7de4e67994cbecb0279a7 localhost:30500/barometer-collectd:stable],SizeBytes:1464048999,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:1002569035,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f417461c5e0283b5f2ba8e34dc073a15fe1f9ff6b542330c536c86aa72f7141f cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726615179,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:371dc6bf7e0c7ce112a29341b000c40d840aef1dbb4fdcb3ae5c0597e28f3061 golang:alpine3.12],SizeBytes:301097267,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:75a55d33ecc73c2a242450a9f1cc858499d468f077ea942867e662c247b5e412 nginx:1.19],SizeBytes:133117205,},ContainerImage{Names:[httpd@sha256:addd70e4ee83f3bc9a4c1c7c41e37927ba47faf639312fc936df3afad7926f5a httpd:2.4.39-alpine],SizeBytes:126894770,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:25502c57570a2143842478595be9c2a2a3cba2df60b673aef79d6ca80e3eac06 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44395488,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 4 16:08:47.120: INFO: Logging kubelet events for node node1 May 4 16:08:47.123: INFO: Logging pods the kubelet thinks is on node node1 May 4 16:08:47.140: INFO: nginx-proxy-node1 started at 2021-05-04 14:51:11 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.140: INFO: Container nginx-proxy ready: true, restart count 2 May 4 16:08:47.140: INFO: kube-proxy-t2mbn started at 2021-05-04 14:45:01 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.140: INFO: Container kube-proxy ready: true, restart count 1 May 4 16:08:47.140: INFO: liveness-http started at 2021-05-04 15:33:56 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.140: INFO: Container liveness-http ready: true, restart count 15 May 4 16:08:47.140: INFO: cmk-init-discover-node1-m8vvw started at 2021-05-04 14:54:32 +0000 UTC (0+3 container statuses recorded) May 4 16:08:47.140: INFO: Container discover ready: false, restart count 0 May 4 16:08:47.140: INFO: Container init ready: false, restart count 0 May 4 16:08:47.140: INFO: Container install ready: false, restart count 0 May 4 16:08:47.140: INFO: affinity-nodeport-timeout-ksvxx started at 2021-05-04 16:08:04 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.140: INFO: Container affinity-nodeport-timeout ready: true, restart count 0 May 4 16:08:47.140: INFO: execpod-affinityz74hp started at 2021-05-04 16:08:10 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.140: INFO: Container agnhost-container ready: true, restart count 0 May 4 16:08:47.140: INFO: kube-multus-ds-amd64-pkmbz started at 2021-05-04 14:45:46 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.140: INFO: Container kube-multus ready: true, restart count 1 May 4 16:08:47.140: INFO: ss2-0 started at 2021-05-04 16:08:36 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.140: INFO: Container webserver ready: true, restart count 0 May 4 16:08:47.140: INFO: kubernetes-metrics-scraper-678c97765c-6qwqq started at 2021-05-04 14:46:10 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.140: INFO: Container kubernetes-metrics-scraper ready: true, restart count 1 May 4 16:08:47.140: INFO: cmk-slg76 started at 2021-05-04 14:55:14 +0000 UTC (0+2 container statuses recorded) May 4 16:08:47.140: INFO: Container nodereport ready: true, restart count 0 May 4 16:08:47.140: INFO: Container reconcile ready: true, restart count 0 May 4 16:08:47.140: INFO: prometheus-k8s-0 started at 2021-05-04 14:56:12 +0000 UTC (0+5 container statuses recorded) May 4 16:08:47.140: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 4 16:08:47.140: INFO: Container grafana ready: true, restart count 0 May 4 16:08:47.140: INFO: Container prometheus ready: true, restart count 1 May 4 16:08:47.140: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 4 16:08:47.140: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 4 16:08:47.140: INFO: nodeport-test-lxckq started at 2021-05-04 16:06:34 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.140: INFO: Container nodeport-test ready: true, restart count 0 May 4 16:08:47.140: INFO: execpodn9wsj started at 2021-05-04 16:06:40 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.140: INFO: Container agnhost-container ready: true, restart count 0 May 4 16:08:47.140: INFO: affinity-nodeport-timeout-ncbmb started at 2021-05-04 16:08:04 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.140: INFO: Container affinity-nodeport-timeout ready: true, restart count 0 May 4 16:08:47.140: INFO: ss2-2 started at 2021-05-04 16:08:29 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.140: INFO: Container webserver ready: true, restart count 0 May 4 16:08:47.140: INFO: netserver-0 started at 2021-05-04 16:08:46 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.140: INFO: Container webserver ready: false, restart count 0 May 4 16:08:47.140: INFO: kube-flannel-d6pbl started at 2021-05-04 14:45:37 +0000 UTC (1+1 container statuses recorded) May 4 16:08:47.140: INFO: Init container install-cni ready: true, restart count 2 May 4 16:08:47.140: INFO: Container kube-flannel ready: true, restart count 2 May 4 16:08:47.140: INFO: node-feature-discovery-worker-wfgl5 started at 2021-05-04 14:51:40 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.140: INFO: Container nfd-worker ready: true, restart count 0 May 4 16:08:47.140: INFO: prometheus-operator-5bb8cb9d8f-rrrhf started at 2021-05-04 14:56:03 +0000 UTC (0+2 container statuses recorded) May 4 16:08:47.140: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 4 16:08:47.140: INFO: Container prometheus-operator ready: true, restart count 0 May 4 16:08:47.140: INFO: node-exporter-k8qd9 started at 2021-05-04 14:56:10 +0000 UTC (0+2 container statuses recorded) May 4 16:08:47.140: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 4 16:08:47.140: INFO: Container node-exporter ready: true, restart count 0 May 4 16:08:47.140: INFO: collectd-4755t started at 2021-05-04 15:01:51 +0000 UTC (0+3 container statuses recorded) May 4 16:08:47.140: INFO: Container collectd ready: true, restart count 0 May 4 16:08:47.140: INFO: Container collectd-exporter ready: true, restart count 0 May 4 16:08:47.140: INFO: Container rbac-proxy ready: true, restart count 0 May 4 16:08:47.140: INFO: pfpod started at 2021-05-04 16:08:31 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.140: INFO: Container pause ready: false, restart count 0 May 4 16:08:47.140: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-hvrmt started at 2021-05-04 14:52:50 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.140: INFO: Container kube-sriovdp ready: true, restart count 0 May 4 16:08:47.140: INFO: pod-adoption-release-9sdfn started at 2021-05-04 16:07:10 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.140: INFO: Container pod-adoption-release ready: true, restart count 0 W0504 16:08:47.154562 27 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 4 16:08:47.569: INFO: Latency metrics for node node1 May 4 16:08:47.569: INFO: Logging node info for node node2 May 4 16:08:47.572: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 41567fa7-bb24-4381-9387-e4115195037d 29767 0 2021-05-04 14:44:58 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"32:98:f5:3b:98:5c"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-05-04 14:44:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubeadm Update v1 2021-05-04 14:44:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-04 14:45:42 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-04 14:52:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-04 14:54:58 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-05-04 14:55:01 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269628928 0} {<nil>} 196552372Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884628480 0} {<nil>} 174692020Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-04 14:47:29 +0000 UTC,LastTransitionTime:2021-05-04 14:47:29 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-04 16:08:45 +0000 UTC,LastTransitionTime:2021-05-04 14:44:58 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-04 16:08:45 +0000 UTC,LastTransitionTime:2021-05-04 14:44:58 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-04 16:08:45 +0000 UTC,LastTransitionTime:2021-05-04 14:44:58 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-04 16:08:45 +0000 UTC,LastTransitionTime:2021-05-04 14:45:42 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:d47a1c7ab17f44f2ae7ff788700a8d74,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:df3523a1-a74f-4f8b-beb1-29f5ed8699f3,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:511d36b62d758304a5adb26b5996ed37211ab226beb7de4e67994cbecb0279a7 localhost:30500/barometer-collectd:stable],SizeBytes:1464048999,},ContainerImage{Names:[localhost:30500/cmk@sha256:f417461c5e0283b5f2ba8e34dc073a15fe1f9ff6b542330c536c86aa72f7141f localhost:30500/cmk:v1.5.1],SizeBytes:726615179,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726615179,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[nginx@sha256:75a55d33ecc73c2a242450a9f1cc858499d468f077ea942867e662c247b5e412 nginx:1.19],SizeBytes:133117205,},ContainerImage{Names:[httpd@sha256:addd70e4ee83f3bc9a4c1c7c41e37927ba47faf639312fc936df3afad7926f5a httpd:2.4.39-alpine],SizeBytes:126894770,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:25502c57570a2143842478595be9c2a2a3cba2df60b673aef79d6ca80e3eac06 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44395488,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:c2fabacbc4e42f3db70f9508e00158b1dce4cf96d91cabaa2eca24e5a0900b66 localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:e074a505e2d62b5119460ab724b2e1df10c8419ef2457f9ce9f3a0f75be3e959 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 4 16:08:47.573: INFO: Logging kubelet events for node node2 May 4 16:08:47.574: INFO: Logging pods the kubelet thinks is on node node2 May 4 16:08:47.590: INFO: ss2-0 started at 2021-05-04 16:07:49 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.590: INFO: Container webserver ready: false, restart count 0 May 4 16:08:47.590: INFO: ss2-1 started at 2021-05-04 16:08:40 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.590: INFO: Container webserver ready: true, restart count 0 May 4 16:08:47.590: INFO: kube-flannel-lnwkk started at 2021-05-04 14:45:37 +0000 UTC (1+1 container statuses recorded) May 4 16:08:47.590: INFO: Init container install-cni ready: true, restart count 2 May 4 16:08:47.590: INFO: Container kube-flannel ready: true, restart count 2 May 4 16:08:47.590: INFO: cmk-webhook-6c9d5f8578-fr595 started at 2021-05-04 14:55:15 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.590: INFO: Container cmk-webhook ready: true, restart count 0 May 4 16:08:47.590: INFO: liveness-407c14ad-7a20-4e46-8321-7d673d64b89e started at 2021-05-04 16:06:36 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.590: INFO: Container liveness ready: false, restart count 4 May 4 16:08:47.590: INFO: ss2-2 started at 2021-05-04 16:08:47 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.590: INFO: Container webserver ready: false, restart count 0 May 4 16:08:47.590: INFO: test-pod started at 2021-05-04 16:03:57 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.590: INFO: Container webserver ready: true, restart count 0 May 4 16:08:47.590: INFO: liveness-exec started at 2021-05-04 15:33:56 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.590: INFO: Container liveness-exec ready: false, restart count 5 May 4 16:08:47.590: INFO: affinity-nodeport-timeout-l62pm started at 2021-05-04 16:08:04 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.590: INFO: Container affinity-nodeport-timeout ready: true, restart count 0 May 4 16:08:47.590: INFO: kube-proxy-rfjjf started at 2021-05-04 14:45:01 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.590: INFO: Container kube-proxy ready: true, restart count 2 May 4 16:08:47.590: INFO: node-feature-discovery-worker-jzjqs started at 2021-05-04 14:51:40 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.590: INFO: Container nfd-worker ready: true, restart count 0 May 4 16:08:47.590: INFO: node-exporter-5lghf started at 2021-05-04 14:56:10 +0000 UTC (0+2 container statuses recorded) May 4 16:08:47.590: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 4 16:08:47.590: INFO: Container node-exporter ready: true, restart count 0 May 4 16:08:47.590: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-4nd7x started at 2021-05-04 14:59:02 +0000 UTC (0+2 container statuses recorded) May 4 16:08:47.590: INFO: Container tas-controller ready: true, restart count 0 May 4 16:08:47.590: INFO: Container tas-extender ready: true, restart count 0 May 4 16:08:47.590: INFO: kube-multus-ds-amd64-7r2s4 started at 2021-05-04 14:45:46 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.590: INFO: Container kube-multus ready: true, restart count 1 May 4 16:08:47.590: INFO: cmk-init-discover-node2-zlxzj started at 2021-05-04 14:54:52 +0000 UTC (0+3 container statuses recorded) May 4 16:08:47.590: INFO: Container discover ready: false, restart count 0 May 4 16:08:47.590: INFO: Container init ready: false, restart count 0 May 4 16:08:47.590: INFO: Container install ready: false, restart count 0 May 4 16:08:47.591: INFO: collectd-dhwfp started at 2021-05-04 15:01:51 +0000 UTC (0+3 container statuses recorded) May 4 16:08:47.591: INFO: Container collectd ready: true, restart count 0 May 4 16:08:47.591: INFO: Container collectd-exporter ready: true, restart count 0 May 4 16:08:47.591: INFO: Container rbac-proxy ready: true, restart count 0 May 4 16:08:47.591: INFO: nodeport-test-qt2pt started at 2021-05-04 16:06:34 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.591: INFO: Container nodeport-test ready: true, restart count 0 May 4 16:08:47.591: INFO: pod-init-1999be94-f44c-495a-9753-5adddc9d351e started at 2021-05-04 16:08:42 +0000 UTC (2+1 container statuses recorded) May 4 16:08:47.591: INFO: Init container init1 ready: false, restart count 0 May 4 16:08:47.591: INFO: Init container init2 ready: false, restart count 0 May 4 16:08:47.591: INFO: Container run1 ready: false, restart count 0 May 4 16:08:47.591: INFO: liveness-2eec7a00-c1bb-43a0-8c2e-0a8c35203695 started at 2021-05-04 16:06:09 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.591: INFO: Container liveness ready: true, restart count 0 May 4 16:08:47.591: INFO: nginx-proxy-node2 started at 2021-05-04 14:51:11 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.591: INFO: Container nginx-proxy ready: true, restart count 2 May 4 16:08:47.591: INFO: test-rollover-deployment-5797c7764-xldpw started at 2021-05-04 16:08:32 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.591: INFO: Container agnhost ready: true, restart count 0 May 4 16:08:47.591: INFO: kubernetes-dashboard-86c6f9df5b-hwbpb started at 2021-05-04 14:46:10 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.591: INFO: Container kubernetes-dashboard ready: true, restart count 1 May 4 16:08:47.591: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-wt4b2 started at 2021-05-04 14:52:50 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.591: INFO: Container kube-sriovdp ready: true, restart count 0 May 4 16:08:47.591: INFO: cmk-2fmbx started at 2021-05-04 14:55:14 +0000 UTC (0+2 container statuses recorded) May 4 16:08:47.591: INFO: Container nodereport ready: true, restart count 0 May 4 16:08:47.591: INFO: Container reconcile ready: true, restart count 0 May 4 16:08:47.591: INFO: ss2-1 started at 2021-05-04 16:08:39 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.591: INFO: Container webserver ready: true, restart count 0 May 4 16:08:47.591: INFO: netserver-1 started at 2021-05-04 16:08:46 +0000 UTC (0+1 container statuses recorded) May 4 16:08:47.591: INFO: Container webserver ready: false, restart count 0 W0504 16:08:47.604108 27 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 4 16:08:48.075: INFO: Latency metrics for node node2 May 4 16:08:48.075: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "services-6457" for this suite. [AfterEach] [sig-network] Services /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/network/service.go:786 /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 May 4 16:16:43.910: Unexpected error: <*errors.errorString | 0xc004fb75f0>: { s: "expected pod \"var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c\" success: Gave up after waiting 5m0s for pod \"var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c\" to be \"Succeeded or Failed\"", } expected pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c" success: Gave up after waiting 5m0s for pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c" to be "Succeeded or Failed" occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:725 {"msg":"PASSED [k8s.io] Variable Expansion should fail substituting values in a volume subpath with backticks [sig-storage][Slow] [Conformance]","total":-1,"completed":15,"skipped":410,"failed":1,"failures":["[sig-network] Services should be able to create a functioning NodePort service [Conformance]"]} [BeforeEach] [k8s.io] Variable Expansion /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 4 16:11:37.282: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename var-expansion STEP: Waiting for a default service account to be provisioned in namespace [It] should allow composing env vars into new env vars [NodeConformance] [Conformance] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 STEP: Creating a pod to test env composition May 4 16:11:37.323: INFO: Waiting up to 5m0s for pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c" in namespace "var-expansion-4466" to be "Succeeded or Failed" May 4 16:11:37.328: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 4.829441ms May 4 16:11:39.331: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 2.007501016s May 4 16:11:41.335: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 4.01128742s May 4 16:11:43.338: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 6.014301796s May 4 16:11:45.341: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 8.017181232s May 4 16:11:47.343: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 10.019929856s May 4 16:11:49.346: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 12.022667189s May 4 16:11:51.349: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 14.025444545s May 4 16:11:53.352: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 16.028945404s May 4 16:11:55.360: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 18.036870036s May 4 16:11:57.364: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 20.040544925s May 4 16:11:59.369: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 22.045426684s May 4 16:12:01.373: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 24.049884779s May 4 16:12:03.376: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 26.052713902s May 4 16:12:05.380: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 28.056695091s May 4 16:12:07.384: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 30.060398642s May 4 16:12:09.387: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 32.064029619s May 4 16:12:11.391: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 34.067928279s May 4 16:12:13.395: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 36.071950606s May 4 16:12:15.398: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 38.074712433s May 4 16:12:17.401: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 40.077947343s May 4 16:12:19.405: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 42.081297169s May 4 16:12:21.410: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 44.086163926s May 4 16:12:23.416: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 46.09274328s May 4 16:12:25.419: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 48.096024184s May 4 16:12:27.422: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 50.098954777s May 4 16:12:29.427: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 52.103110338s May 4 16:12:31.431: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 54.107500631s May 4 16:12:33.435: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 56.111230965s May 4 16:12:35.441: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 58.117667238s May 4 16:12:37.444: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m0.120334972s May 4 16:12:39.450: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m2.126360859s May 4 16:12:41.452: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m4.12894724s May 4 16:12:43.455: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m6.132016932s May 4 16:12:45.459: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m8.135519817s May 4 16:12:47.462: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m10.138941089s May 4 16:12:49.466: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m12.142478995s May 4 16:12:51.472: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m14.148335263s May 4 16:12:53.474: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.150948986s May 4 16:12:55.477: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.153903255s May 4 16:12:57.480: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.156670894s May 4 16:12:59.483: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.159527071s May 4 16:13:01.486: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m24.162954455s May 4 16:13:03.492: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.168227513s May 4 16:13:05.496: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.172710077s May 4 16:13:07.500: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.176214839s May 4 16:13:09.502: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.179010069s May 4 16:13:11.506: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m34.182084621s May 4 16:13:13.509: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m36.185379234s May 4 16:13:15.513: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m38.18949735s May 4 16:13:17.516: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m40.192696243s May 4 16:13:19.524: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m42.200290611s May 4 16:13:21.528: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.204376458s May 4 16:13:23.532: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.208187614s May 4 16:13:25.534: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.211003969s May 4 16:13:27.537: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.213998688s May 4 16:13:29.541: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m52.217926728s May 4 16:13:31.547: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m54.223266718s May 4 16:13:33.551: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m56.227338994s May 4 16:13:35.553: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 1m58.230012161s May 4 16:13:37.557: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m0.233641626s May 4 16:13:39.562: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m2.238768605s May 4 16:13:41.566: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m4.242542084s May 4 16:13:43.569: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m6.245427408s May 4 16:13:45.573: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m8.249451301s May 4 16:13:47.577: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m10.25332167s May 4 16:13:49.580: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m12.256814459s May 4 16:13:51.586: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m14.262374897s May 4 16:13:53.590: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m16.266311074s May 4 16:13:55.596: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m18.272985504s May 4 16:13:57.599: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m20.276000694s May 4 16:13:59.603: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m22.279937503s May 4 16:14:01.606: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m24.282998341s May 4 16:14:03.610: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m26.2867587s May 4 16:14:05.614: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m28.290190553s May 4 16:14:07.617: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.29321938s May 4 16:14:09.620: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.296244812s May 4 16:14:11.622: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m34.298900737s May 4 16:14:13.626: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m36.302995147s May 4 16:14:15.630: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m38.306986318s May 4 16:14:17.633: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m40.310015235s May 4 16:14:19.638: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m42.314330393s May 4 16:14:21.642: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m44.3182959s May 4 16:14:23.647: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m46.323143107s May 4 16:14:25.651: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m48.327546962s May 4 16:14:27.654: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m50.330987146s May 4 16:14:29.658: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m52.334367674s May 4 16:14:31.662: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m54.338110376s May 4 16:14:33.665: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m56.341884855s May 4 16:14:35.672: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 2m58.34817387s May 4 16:14:37.676: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m0.352400869s May 4 16:14:39.679: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m2.355458131s May 4 16:14:41.682: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m4.359029168s May 4 16:14:43.690: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m6.366247556s May 4 16:14:45.694: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m8.370994512s May 4 16:14:47.699: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m10.375878532s May 4 16:14:49.704: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m12.380132916s May 4 16:14:51.707: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m14.383501437s May 4 16:14:53.712: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m16.388293884s May 4 16:14:55.720: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m18.396324406s May 4 16:14:57.723: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m20.399324723s May 4 16:14:59.727: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m22.403796891s May 4 16:15:01.731: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m24.407110278s May 4 16:15:03.735: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m26.411838256s May 4 16:15:05.739: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m28.415095225s May 4 16:15:07.742: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m30.418906519s May 4 16:15:09.745: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m32.421842522s May 4 16:15:11.749: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m34.425178561s May 4 16:15:13.753: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m36.429067315s May 4 16:15:15.755: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m38.431948369s May 4 16:15:17.759: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m40.435482204s May 4 16:15:19.762: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m42.439033086s May 4 16:15:21.767: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m44.443427426s May 4 16:15:23.770: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m46.446574103s May 4 16:15:25.773: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m48.449674224s May 4 16:15:27.777: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m50.45329568s May 4 16:15:29.781: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m52.457439311s May 4 16:15:31.783: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m54.460044233s May 4 16:15:33.786: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m56.46300427s May 4 16:15:35.790: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 3m58.466056842s May 4 16:15:37.793: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m0.469105517s May 4 16:15:39.796: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m2.472556178s May 4 16:15:41.799: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m4.47583169s May 4 16:15:43.802: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m6.478987322s May 4 16:15:45.806: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m8.482927311s May 4 16:15:47.810: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m10.486319175s May 4 16:15:49.813: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m12.489796021s May 4 16:15:51.816: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m14.49223551s May 4 16:15:53.818: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m16.494967916s May 4 16:15:55.822: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m18.498861142s May 4 16:15:57.825: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m20.501622892s May 4 16:15:59.828: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m22.504185083s May 4 16:16:01.832: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m24.50902551s May 4 16:16:03.835: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m26.511928947s May 4 16:16:05.840: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m28.516905749s May 4 16:16:07.844: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m30.520152188s May 4 16:16:09.847: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m32.523239633s May 4 16:16:11.850: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m34.52664938s May 4 16:16:13.853: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m36.529662016s May 4 16:16:15.856: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m38.532758507s May 4 16:16:17.859: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m40.535988486s May 4 16:16:19.862: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m42.538398991s May 4 16:16:21.865: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m44.541280771s May 4 16:16:23.869: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m46.545383886s May 4 16:16:25.871: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m48.548046313s May 4 16:16:27.875: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m50.551647884s May 4 16:16:29.878: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m52.554615498s May 4 16:16:31.882: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m54.558886577s May 4 16:16:33.886: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m56.562462197s May 4 16:16:35.890: INFO: Pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c": Phase="Pending", Reason="", readiness=false. Elapsed: 4m58.566308979s May 4 16:16:37.898: INFO: Failed to get logs from node "node2" pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c" container "dapi-container": the server rejected our request for an unknown reason (get pods var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c) STEP: delete the pod May 4 16:16:37.903: INFO: Waiting for pod var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c to disappear May 4 16:16:37.906: INFO: Pod var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c still exists May 4 16:16:39.907: INFO: Waiting for pod var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c to disappear May 4 16:16:39.909: INFO: Pod var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c still exists May 4 16:16:41.906: INFO: Waiting for pod var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c to disappear May 4 16:16:41.909: INFO: Pod var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c still exists May 4 16:16:43.907: INFO: Waiting for pod var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c to disappear May 4 16:16:43.909: INFO: Pod var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c no longer exists May 4 16:16:43.910: FAIL: Unexpected error: <*errors.errorString | 0xc004fb75f0>: { s: "expected pod \"var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c\" success: Gave up after waiting 5m0s for pod \"var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c\" to be \"Succeeded or Failed\"", } expected pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c" success: Gave up after waiting 5m0s for pod "var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c" to be "Succeeded or Failed" occurred Full Stack Trace k8s.io/kubernetes/test/e2e/framework.(*Framework).testContainerOutputMatcher(0xc000f062c0, 0x4c09bc7, 0xf, 0xc00380c400, 0x0, 0xc0046171a8, 0x3, 0x3, 0x4de7488) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:725 +0x1ee k8s.io/kubernetes/test/e2e/framework.(*Framework).TestContainerOutput(...) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:525 k8s.io/kubernetes/test/e2e/common.glob..func9.1() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/expansion.go:58 +0x22a k8s.io/kubernetes/test/e2e.RunE2ETests(0xc002947080) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc002947080) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc002947080, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [k8s.io] Variable Expansion /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "var-expansion-4466". STEP: Found 7 events. May 4 16:16:43.915: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c: { } Scheduled: Successfully assigned var-expansion-4466/var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c to node2 May 4 16:16:43.915: INFO: At 2021-05-04 16:11:38 +0000 UTC - event for var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c: {multus } AddedInterface: Add eth0 [10.244.3.196/24] May 4 16:16:43.915: INFO: At 2021-05-04 16:11:38 +0000 UTC - event for var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c: {kubelet node2} Pulling: Pulling image "docker.io/library/busybox:1.29" May 4 16:16:43.915: INFO: At 2021-05-04 16:11:39 +0000 UTC - event for var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c: {kubelet node2} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 4 16:16:43.915: INFO: At 2021-05-04 16:11:39 +0000 UTC - event for var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c: {kubelet node2} Failed: Error: ErrImagePull May 4 16:16:43.915: INFO: At 2021-05-04 16:11:40 +0000 UTC - event for var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c: {kubelet node2} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 4 16:16:43.915: INFO: At 2021-05-04 16:11:40 +0000 UTC - event for var-expansion-f40503a0-123d-4f48-a90a-ab9f4afa468c: {kubelet node2} Failed: Error: ImagePullBackOff May 4 16:16:43.917: INFO: POD NODE PHASE GRACE CONDITIONS May 4 16:16:43.917: INFO: May 4 16:16:43.922: INFO: Logging node info for node master1 May 4 16:16:43.924: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 db982204-549e-4532-90a7-a4410878cfc9 36704 0 2021-05-04 14:43:01 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"3e:f0:43:cb:66:52"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-04 14:43:01 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-04 14:43:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-04 14:45:41 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-04 14:45:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-04 14:51:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-04 14:47:46 +0000 UTC,LastTransitionTime:2021-05-04 14:47:46 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-04 16:16:38 +0000 UTC,LastTransitionTime:2021-05-04 14:43:01 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-04 16:16:38 +0000 UTC,LastTransitionTime:2021-05-04 14:43:01 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-04 16:16:38 +0000 UTC,LastTransitionTime:2021-05-04 14:43:01 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-04 16:16:38 +0000 UTC,LastTransitionTime:2021-05-04 14:47:24 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:88a0771919594d4187f6704fc7592bf8,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:8e0a253b-2aa4-4467-879e-567e7ba1ffa4,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726615179,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:c2fabacbc4e42f3db70f9508e00158b1dce4cf96d91cabaa2eca24e5a0900b66 tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:e074a505e2d62b5119460ab724b2e1df10c8419ef2457f9ce9f3a0f75be3e959 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 4 16:16:43.925: INFO: Logging kubelet events for node master1 May 4 16:16:43.927: INFO: Logging pods the kubelet thinks is on node master1 May 4 16:16:43.936: INFO: kube-apiserver-master1 started at 2021-05-04 14:47:26 +0000 UTC (0+1 container statuses recorded) May 4 16:16:43.936: INFO: Container kube-apiserver ready: true, restart count 0 May 4 16:16:43.936: INFO: kube-controller-manager-master1 started at 2021-05-04 14:47:26 +0000 UTC (0+1 container statuses recorded) May 4 16:16:43.936: INFO: Container kube-controller-manager ready: true, restart count 2 May 4 16:16:43.936: INFO: kube-proxy-8j6ch started at 2021-05-04 14:45:01 +0000 UTC (0+1 container statuses recorded) May 4 16:16:43.936: INFO: Container kube-proxy ready: true, restart count 1 May 4 16:16:43.936: INFO: docker-registry-docker-registry-56cbc7bc58-zhf8t started at 2021-05-04 14:48:42 +0000 UTC (0+2 container statuses recorded) May 4 16:16:43.936: INFO: Container docker-registry ready: true, restart count 0 May 4 16:16:43.936: INFO: Container nginx ready: true, restart count 0 May 4 16:16:43.936: INFO: node-exporter-jckjs started at 2021-05-04 14:56:10 +0000 UTC (0+2 container statuses recorded) May 4 16:16:43.936: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 4 16:16:43.936: INFO: Container node-exporter ready: true, restart count 0 May 4 16:16:43.936: INFO: kube-scheduler-master1 started at 2021-05-04 14:44:16 +0000 UTC (0+1 container statuses recorded) May 4 16:16:43.936: INFO: Container kube-scheduler ready: true, restart count 0 May 4 16:16:43.936: INFO: kube-flannel-qspzk started at 2021-05-04 14:45:37 +0000 UTC (1+1 container statuses recorded) May 4 16:16:43.936: INFO: Init container install-cni ready: true, restart count 0 May 4 16:16:43.936: INFO: Container kube-flannel ready: true, restart count 3 May 4 16:16:43.936: INFO: kube-multus-ds-amd64-jflvf started at 2021-05-04 14:45:46 +0000 UTC (0+1 container statuses recorded) May 4 16:16:43.936: INFO: Container kube-multus ready: true, restart count 1 May 4 16:16:43.936: INFO: coredns-7677f9bb54-qvcd2 started at 2021-05-04 14:46:11 +0000 UTC (0+1 container statuses recorded) May 4 16:16:43.936: INFO: Container coredns ready: true, restart count 1 May 4 16:16:43.936: INFO: node-feature-discovery-controller-5bf5c49849-72rn6 started at 2021-05-04 14:51:52 +0000 UTC (0+1 container statuses recorded) May 4 16:16:43.936: INFO: Container nfd-controller ready: true, restart count 0 W0504 16:16:43.949603 27 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 4 16:16:43.981: INFO: Latency metrics for node master1 May 4 16:16:43.981: INFO: Logging node info for node master2 May 4 16:16:43.983: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e2c15170-247b-4e7b-b818-abc807948bf8 36702 0 2021-05-04 14:43:40 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"16:e0:10:a0:e0:62"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-04 14:43:40 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-04 14:43:41 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-04 14:45:41 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-04 14:45:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-04 14:47:32 +0000 UTC,LastTransitionTime:2021-05-04 14:47:32 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-04 16:16:37 +0000 UTC,LastTransitionTime:2021-05-04 14:43:40 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-04 16:16:37 +0000 UTC,LastTransitionTime:2021-05-04 14:43:40 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-04 16:16:37 +0000 UTC,LastTransitionTime:2021-05-04 14:43:40 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-04 16:16:37 +0000 UTC,LastTransitionTime:2021-05-04 14:47:24 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:6af568f56589422a9bd68e0270ce0f8c,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:bf27bb77-fad2-4b52-85c3-acb5113fc512,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726615179,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 4 16:16:43.984: INFO: Logging kubelet events for node master2 May 4 16:16:43.986: INFO: Logging pods the kubelet thinks is on node master2 May 4 16:16:43.992: INFO: kube-flannel-cxdfr started at 2021-05-04 14:45:37 +0000 UTC (1+1 container statuses recorded) May 4 16:16:43.992: INFO: Init container install-cni ready: true, restart count 0 May 4 16:16:43.992: INFO: Container kube-flannel ready: true, restart count 1 May 4 16:16:43.992: INFO: kube-multus-ds-amd64-dw8tg started at 2021-05-04 14:45:46 +0000 UTC (0+1 container statuses recorded) May 4 16:16:43.992: INFO: Container kube-multus ready: true, restart count 1 May 4 16:16:43.992: INFO: dns-autoscaler-5b7b5c9b6f-zbrsq started at 2021-05-04 14:46:08 +0000 UTC (0+1 container statuses recorded) May 4 16:16:43.992: INFO: Container autoscaler ready: true, restart count 1 May 4 16:16:43.992: INFO: node-exporter-9c6qf started at 2021-05-04 14:56:10 +0000 UTC (0+2 container statuses recorded) May 4 16:16:43.992: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 4 16:16:43.992: INFO: Container node-exporter ready: true, restart count 0 May 4 16:16:43.992: INFO: kube-apiserver-master2 started at 2021-05-04 14:47:26 +0000 UTC (0+1 container statuses recorded) May 4 16:16:43.992: INFO: Container kube-apiserver ready: true, restart count 0 May 4 16:16:43.992: INFO: kube-controller-manager-master2 started at 2021-05-04 14:47:26 +0000 UTC (0+1 container statuses recorded) May 4 16:16:43.992: INFO: Container kube-controller-manager ready: true, restart count 2 May 4 16:16:43.993: INFO: kube-scheduler-master2 started at 2021-05-04 14:44:16 +0000 UTC (0+1 container statuses recorded) May 4 16:16:43.993: INFO: Container kube-scheduler ready: true, restart count 2 May 4 16:16:43.993: INFO: kube-proxy-6b5t8 started at 2021-05-04 14:45:01 +0000 UTC (0+1 container statuses recorded) May 4 16:16:43.993: INFO: Container kube-proxy ready: true, restart count 2 W0504 16:16:44.003803 27 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 4 16:16:44.032: INFO: Latency metrics for node master2 May 4 16:16:44.032: INFO: Logging node info for node master3 May 4 16:16:44.036: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 b533a646-667e-403c-944b-71dec9cc4851 36701 0 2021-05-04 14:43:51 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"16:b0:53:14:f6:c9"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-04 14:43:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-04 14:43:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-04 14:45:42 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-04 14:45:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-04 14:48:50 +0000 UTC,LastTransitionTime:2021-05-04 14:48:50 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-04 16:16:37 +0000 UTC,LastTransitionTime:2021-05-04 14:43:51 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-04 16:16:37 +0000 UTC,LastTransitionTime:2021-05-04 14:43:51 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-04 16:16:37 +0000 UTC,LastTransitionTime:2021-05-04 14:43:51 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-04 16:16:37 +0000 UTC,LastTransitionTime:2021-05-04 14:45:46 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:959373dcda56494486f0c2bb0bb496cc,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:0714ca81-c21e-40d6-a288-48d597238e54,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726615179,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 4 16:16:44.037: INFO: Logging kubelet events for node master3 May 4 16:16:44.039: INFO: Logging pods the kubelet thinks is on node master3 May 4 16:16:44.047: INFO: kube-apiserver-master3 started at 2021-05-04 14:44:16 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.047: INFO: Container kube-apiserver ready: true, restart count 0 May 4 16:16:44.047: INFO: kube-controller-manager-master3 started at 2021-05-04 14:44:16 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.047: INFO: Container kube-controller-manager ready: true, restart count 2 May 4 16:16:44.047: INFO: kube-scheduler-master3 started at 2021-05-04 14:44:16 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.047: INFO: Container kube-scheduler ready: true, restart count 2 May 4 16:16:44.047: INFO: kube-proxy-2p5b6 started at 2021-05-04 14:45:01 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.047: INFO: Container kube-proxy ready: true, restart count 2 May 4 16:16:44.047: INFO: kube-flannel-wznt8 started at 2021-05-04 14:45:37 +0000 UTC (1+1 container statuses recorded) May 4 16:16:44.047: INFO: Init container install-cni ready: true, restart count 0 May 4 16:16:44.047: INFO: Container kube-flannel ready: true, restart count 1 May 4 16:16:44.047: INFO: kube-multus-ds-amd64-cgwz2 started at 2021-05-04 14:45:46 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.047: INFO: Container kube-multus ready: true, restart count 1 May 4 16:16:44.047: INFO: coredns-7677f9bb54-pshfb started at 2021-05-04 14:46:06 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.047: INFO: Container coredns ready: true, restart count 1 May 4 16:16:44.047: INFO: node-exporter-wvppn started at 2021-05-04 14:56:10 +0000 UTC (0+2 container statuses recorded) May 4 16:16:44.047: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 4 16:16:44.047: INFO: Container node-exporter ready: true, restart count 0 W0504 16:16:44.060398 27 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 4 16:16:44.083: INFO: Latency metrics for node master3 May 4 16:16:44.083: INFO: Logging node info for node node1 May 4 16:16:44.086: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 b8cf7e16-d5c7-4e2c-996a-93d93bd4fa1c 36718 0 2021-05-04 14:44:58 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"e2:50:df:03:d2:13"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-05-04 14:44:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubeadm Update v1 2021-05-04 14:44:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-04 14:45:42 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-04 14:52:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-04 14:54:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-05-04 14:54:41 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269633024 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884632576 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-04 14:47:29 +0000 UTC,LastTransitionTime:2021-05-04 14:47:29 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-04 16:16:40 +0000 UTC,LastTransitionTime:2021-05-04 14:44:58 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-04 16:16:40 +0000 UTC,LastTransitionTime:2021-05-04 14:44:58 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-04 16:16:40 +0000 UTC,LastTransitionTime:2021-05-04 14:44:58 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-04 16:16:40 +0000 UTC,LastTransitionTime:2021-05-04 14:47:24 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:bffc023a4ab84df0b0181bc7b8f509e2,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:dc08af72-abca-4f1d-bd0f-0e8d8eb97de5,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:511d36b62d758304a5adb26b5996ed37211ab226beb7de4e67994cbecb0279a7 localhost:30500/barometer-collectd:stable],SizeBytes:1464048999,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:1002569035,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f417461c5e0283b5f2ba8e34dc073a15fe1f9ff6b542330c536c86aa72f7141f cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726615179,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:371dc6bf7e0c7ce112a29341b000c40d840aef1dbb4fdcb3ae5c0597e28f3061 golang:alpine3.12],SizeBytes:301097267,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:75a55d33ecc73c2a242450a9f1cc858499d468f077ea942867e662c247b5e412 nginx:1.19],SizeBytes:133117205,},ContainerImage{Names:[httpd@sha256:addd70e4ee83f3bc9a4c1c7c41e37927ba47faf639312fc936df3afad7926f5a httpd:2.4.39-alpine],SizeBytes:126894770,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:25502c57570a2143842478595be9c2a2a3cba2df60b673aef79d6ca80e3eac06 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44395488,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 4 16:16:44.087: INFO: Logging kubelet events for node node1 May 4 16:16:44.089: INFO: Logging pods the kubelet thinks is on node node1 May 4 16:16:44.104: INFO: kube-multus-ds-amd64-pkmbz started at 2021-05-04 14:45:46 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.104: INFO: Container kube-multus ready: true, restart count 1 May 4 16:16:44.104: INFO: ss2-0 started at 2021-05-04 16:09:26 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.104: INFO: Container webserver ready: false, restart count 0 May 4 16:16:44.104: INFO: kubernetes-metrics-scraper-678c97765c-6qwqq started at 2021-05-04 14:46:10 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.104: INFO: Container kubernetes-metrics-scraper ready: true, restart count 1 May 4 16:16:44.104: INFO: cmk-slg76 started at 2021-05-04 14:55:14 +0000 UTC (0+2 container statuses recorded) May 4 16:16:44.104: INFO: Container nodereport ready: true, restart count 0 May 4 16:16:44.104: INFO: Container reconcile ready: true, restart count 0 May 4 16:16:44.104: INFO: prometheus-k8s-0 started at 2021-05-04 14:56:12 +0000 UTC (0+5 container statuses recorded) May 4 16:16:44.104: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 4 16:16:44.104: INFO: Container grafana ready: true, restart count 0 May 4 16:16:44.104: INFO: Container prometheus ready: true, restart count 1 May 4 16:16:44.104: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 4 16:16:44.104: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 4 16:16:44.104: INFO: kube-flannel-d6pbl started at 2021-05-04 14:45:37 +0000 UTC (1+1 container statuses recorded) May 4 16:16:44.104: INFO: Init container install-cni ready: true, restart count 2 May 4 16:16:44.104: INFO: Container kube-flannel ready: true, restart count 2 May 4 16:16:44.104: INFO: node-feature-discovery-worker-wfgl5 started at 2021-05-04 14:51:40 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.105: INFO: Container nfd-worker ready: true, restart count 0 May 4 16:16:44.105: INFO: prometheus-operator-5bb8cb9d8f-rrrhf started at 2021-05-04 14:56:03 +0000 UTC (0+2 container statuses recorded) May 4 16:16:44.105: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 4 16:16:44.105: INFO: Container prometheus-operator ready: true, restart count 0 May 4 16:16:44.105: INFO: node-exporter-k8qd9 started at 2021-05-04 14:56:10 +0000 UTC (0+2 container statuses recorded) May 4 16:16:44.105: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 4 16:16:44.105: INFO: Container node-exporter ready: true, restart count 0 May 4 16:16:44.105: INFO: collectd-4755t started at 2021-05-04 15:01:51 +0000 UTC (0+3 container statuses recorded) May 4 16:16:44.105: INFO: Container collectd ready: true, restart count 0 May 4 16:16:44.105: INFO: Container collectd-exporter ready: true, restart count 0 May 4 16:16:44.105: INFO: Container rbac-proxy ready: true, restart count 0 May 4 16:16:44.105: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-hvrmt started at 2021-05-04 14:52:50 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.105: INFO: Container kube-sriovdp ready: true, restart count 0 May 4 16:16:44.105: INFO: affinity-nodeport-transition-hn44d started at 2021-05-04 16:15:06 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.105: INFO: Container affinity-nodeport-transition ready: true, restart count 0 May 4 16:16:44.105: INFO: server started at 2021-05-04 16:15:55 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.105: INFO: Container agnhost-container ready: true, restart count 0 May 4 16:16:44.105: INFO: nginx-proxy-node1 started at 2021-05-04 14:51:11 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.105: INFO: Container nginx-proxy ready: true, restart count 2 May 4 16:16:44.105: INFO: kube-proxy-t2mbn started at 2021-05-04 14:45:01 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.105: INFO: Container kube-proxy ready: true, restart count 1 May 4 16:16:44.105: INFO: liveness-http started at 2021-05-04 15:33:56 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.105: INFO: Container liveness-http ready: false, restart count 17 May 4 16:16:44.105: INFO: server-envvars-e2e8d4b8-6525-4f40-9a98-8cccf5c227b4 started at 2021-05-04 16:10:40 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.105: INFO: Container srv ready: true, restart count 0 May 4 16:16:44.105: INFO: cmk-init-discover-node1-m8vvw started at 2021-05-04 14:54:32 +0000 UTC (0+3 container statuses recorded) May 4 16:16:44.105: INFO: Container discover ready: false, restart count 0 May 4 16:16:44.105: INFO: Container init ready: false, restart count 0 May 4 16:16:44.105: INFO: Container install ready: false, restart count 0 May 4 16:16:44.105: INFO: netserver-0 started at 2021-05-04 16:16:20 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.105: INFO: Container webserver ready: true, restart count 0 W0504 16:16:44.118056 27 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 4 16:16:44.149: INFO: Latency metrics for node node1 May 4 16:16:44.149: INFO: Logging node info for node node2 May 4 16:16:44.152: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 41567fa7-bb24-4381-9387-e4115195037d 36717 0 2021-05-04 14:44:58 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"32:98:f5:3b:98:5c"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-05-04 14:44:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubeadm Update v1 2021-05-04 14:44:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-04 14:45:42 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-04 14:52:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-04 14:54:58 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-05-04 14:55:01 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269628928 0} {<nil>} 196552372Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884628480 0} {<nil>} 174692020Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-04 14:47:29 +0000 UTC,LastTransitionTime:2021-05-04 14:47:29 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-04 16:16:40 +0000 UTC,LastTransitionTime:2021-05-04 14:44:58 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-04 16:16:40 +0000 UTC,LastTransitionTime:2021-05-04 14:44:58 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-04 16:16:40 +0000 UTC,LastTransitionTime:2021-05-04 14:44:58 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-04 16:16:40 +0000 UTC,LastTransitionTime:2021-05-04 14:45:42 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:d47a1c7ab17f44f2ae7ff788700a8d74,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:df3523a1-a74f-4f8b-beb1-29f5ed8699f3,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:511d36b62d758304a5adb26b5996ed37211ab226beb7de4e67994cbecb0279a7 localhost:30500/barometer-collectd:stable],SizeBytes:1464048999,},ContainerImage{Names:[localhost:30500/cmk@sha256:f417461c5e0283b5f2ba8e34dc073a15fe1f9ff6b542330c536c86aa72f7141f localhost:30500/cmk:v1.5.1],SizeBytes:726615179,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726615179,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[nginx@sha256:75a55d33ecc73c2a242450a9f1cc858499d468f077ea942867e662c247b5e412 nginx:1.19],SizeBytes:133117205,},ContainerImage{Names:[httpd@sha256:addd70e4ee83f3bc9a4c1c7c41e37927ba47faf639312fc936df3afad7926f5a httpd:2.4.39-alpine],SizeBytes:126894770,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:25502c57570a2143842478595be9c2a2a3cba2df60b673aef79d6ca80e3eac06 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44395488,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:c2fabacbc4e42f3db70f9508e00158b1dce4cf96d91cabaa2eca24e5a0900b66 localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:e074a505e2d62b5119460ab724b2e1df10c8419ef2457f9ce9f3a0f75be3e959 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 4 16:16:44.153: INFO: Logging kubelet events for node node2 May 4 16:16:44.155: INFO: Logging pods the kubelet thinks is on node node2 May 4 16:16:44.170: INFO: ss2-1 started at 2021-05-04 16:08:40 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.170: INFO: Container webserver ready: true, restart count 0 May 4 16:16:44.170: INFO: kube-flannel-lnwkk started at 2021-05-04 14:45:37 +0000 UTC (1+1 container statuses recorded) May 4 16:16:44.170: INFO: Init container install-cni ready: true, restart count 2 May 4 16:16:44.170: INFO: Container kube-flannel ready: true, restart count 2 May 4 16:16:44.170: INFO: cmk-webhook-6c9d5f8578-fr595 started at 2021-05-04 14:55:15 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.170: INFO: Container cmk-webhook ready: true, restart count 0 May 4 16:16:44.170: INFO: node-exporter-5lghf started at 2021-05-04 14:56:10 +0000 UTC (0+2 container statuses recorded) May 4 16:16:44.170: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 4 16:16:44.170: INFO: Container node-exporter ready: true, restart count 0 May 4 16:16:44.170: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-4nd7x started at 2021-05-04 14:59:02 +0000 UTC (0+2 container statuses recorded) May 4 16:16:44.170: INFO: Container tas-controller ready: true, restart count 0 May 4 16:16:44.170: INFO: Container tas-extender ready: true, restart count 0 May 4 16:16:44.170: INFO: affinity-nodeport-transition-kqrgt started at 2021-05-04 16:15:06 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.170: INFO: Container affinity-nodeport-transition ready: true, restart count 0 May 4 16:16:44.170: INFO: client-envvars-3aca01f8-0d22-4951-b441-fa131ddecb49 started at 2021-05-04 16:15:46 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.170: INFO: Container env3cont ready: false, restart count 0 May 4 16:16:44.170: INFO: liveness-exec started at 2021-05-04 15:33:56 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.170: INFO: Container liveness-exec ready: false, restart count 6 May 4 16:16:44.170: INFO: netserver-1 started at 2021-05-04 16:16:20 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.170: INFO: Container webserver ready: true, restart count 0 May 4 16:16:44.170: INFO: kube-proxy-rfjjf started at 2021-05-04 14:45:01 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.170: INFO: Container kube-proxy ready: true, restart count 2 May 4 16:16:44.170: INFO: node-feature-discovery-worker-jzjqs started at 2021-05-04 14:51:40 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.170: INFO: Container nfd-worker ready: true, restart count 0 May 4 16:16:44.170: INFO: test-container-pod started at 2021-05-04 16:16:42 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.170: INFO: Container webserver ready: false, restart count 0 May 4 16:16:44.170: INFO: kube-multus-ds-amd64-7r2s4 started at 2021-05-04 14:45:46 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.170: INFO: Container kube-multus ready: true, restart count 1 May 4 16:16:44.170: INFO: pod-update-58c100c1-80db-41aa-82d8-3e236dfc5b91 started at 2021-05-04 16:16:21 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.170: INFO: Container nginx ready: false, restart count 0 May 4 16:16:44.170: INFO: cmk-init-discover-node2-zlxzj started at 2021-05-04 14:54:52 +0000 UTC (0+3 container statuses recorded) May 4 16:16:44.170: INFO: Container discover ready: false, restart count 0 May 4 16:16:44.170: INFO: Container init ready: false, restart count 0 May 4 16:16:44.170: INFO: Container install ready: false, restart count 0 May 4 16:16:44.170: INFO: collectd-dhwfp started at 2021-05-04 15:01:51 +0000 UTC (0+3 container statuses recorded) May 4 16:16:44.170: INFO: Container collectd ready: true, restart count 0 May 4 16:16:44.170: INFO: Container collectd-exporter ready: true, restart count 0 May 4 16:16:44.170: INFO: Container rbac-proxy ready: true, restart count 0 May 4 16:16:44.170: INFO: execpod-affinityp2lx7 started at 2021-05-04 16:15:12 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.170: INFO: Container agnhost-container ready: true, restart count 0 May 4 16:16:44.170: INFO: e2e-test-httpd-pod started at 2021-05-04 16:11:06 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.170: INFO: Container e2e-test-httpd-pod ready: false, restart count 0 May 4 16:16:44.170: INFO: pod-configmaps-8d1ee320-8a2b-40a2-bfda-64f8d254cc59 started at 2021-05-04 16:14:12 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.170: INFO: Container env-test ready: false, restart count 0 May 4 16:16:44.170: INFO: tester started at 2021-05-04 16:15:59 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.170: INFO: Container tester ready: false, restart count 0 May 4 16:16:44.170: INFO: nginx-proxy-node2 started at 2021-05-04 14:51:11 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.170: INFO: Container nginx-proxy ready: true, restart count 2 May 4 16:16:44.170: INFO: cmk-2fmbx started at 2021-05-04 14:55:14 +0000 UTC (0+2 container statuses recorded) May 4 16:16:44.170: INFO: Container nodereport ready: true, restart count 0 May 4 16:16:44.170: INFO: Container reconcile ready: true, restart count 0 May 4 16:16:44.170: INFO: affinity-nodeport-transition-qr9hq started at 2021-05-04 16:15:06 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.170: INFO: Container affinity-nodeport-transition ready: true, restart count 0 May 4 16:16:44.170: INFO: kubernetes-dashboard-86c6f9df5b-hwbpb started at 2021-05-04 14:46:10 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.170: INFO: Container kubernetes-dashboard ready: true, restart count 1 May 4 16:16:44.170: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-wt4b2 started at 2021-05-04 14:52:50 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.170: INFO: Container kube-sriovdp ready: true, restart count 0 May 4 16:16:44.170: INFO: test-rolling-update-controller-9v9w8 started at 2021-05-04 16:11:51 +0000 UTC (0+1 container statuses recorded) May 4 16:16:44.170: INFO: Container httpd ready: false, restart count 0 W0504 16:16:44.182795 27 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 4 16:16:44.214: INFO: Latency metrics for node node2 May 4 16:16:44.214: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "var-expansion-4466" for this suite. /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 May 4 16:21:44.330: Timed out after 300.003s. Expected <v1.PodPhase>: Pending to equal <v1.PodPhase>: Succeeded /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/runtime.go:154 [BeforeEach] [k8s.io] Container Runtime /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 4 16:16:44.287: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename container-runtime STEP: Waiting for a default service account to be provisioned in namespace [It] should report termination message [LinuxOnly] if TerminationMessagePath is set as non-root user and at a non-default path [NodeConformance] [Conformance] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 STEP: create the container STEP: wait for the container to reach Succeeded May 4 16:21:44.330: FAIL: Timed out after 300.003s. Expected <v1.PodPhase>: Pending to equal <v1.PodPhase>: Succeeded Full Stack Trace k8s.io/kubernetes/test/e2e/common.glob..func25.1.2.1(0x4c5d8a0, 0x1d, 0xc0004ec0c0, 0x1e, 0xc0064bc000, 0x2, 0x2, 0xc0064ae040, 0x1, 0x1, ...) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/runtime.go:154 +0x3f1 k8s.io/kubernetes/test/e2e/common.glob..func25.1.2.3() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/runtime.go:211 +0x292 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc002947080) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc002947080) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc002947080, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [k8s.io] Container Runtime /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "container-runtime-5315". STEP: Found 9 events. May 4 16:21:44.342: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for termination-message-container10779dbf-3a4f-48c1-86c0-3b0ea708da7c: { } Scheduled: Successfully assigned container-runtime-5315/termination-message-container10779dbf-3a4f-48c1-86c0-3b0ea708da7c to node2 May 4 16:21:44.342: INFO: At 2021-05-04 16:16:45 +0000 UTC - event for termination-message-container10779dbf-3a4f-48c1-86c0-3b0ea708da7c: {multus } AddedInterface: Add eth0 [10.244.3.211/24] May 4 16:21:44.342: INFO: At 2021-05-04 16:16:45 +0000 UTC - event for termination-message-container10779dbf-3a4f-48c1-86c0-3b0ea708da7c: {kubelet node2} Pulling: Pulling image "docker.io/library/busybox:1.29" May 4 16:21:44.342: INFO: At 2021-05-04 16:16:46 +0000 UTC - event for termination-message-container10779dbf-3a4f-48c1-86c0-3b0ea708da7c: {kubelet node2} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 4 16:21:44.342: INFO: At 2021-05-04 16:16:46 +0000 UTC - event for termination-message-container10779dbf-3a4f-48c1-86c0-3b0ea708da7c: {kubelet node2} Failed: Error: ErrImagePull May 4 16:21:44.342: INFO: At 2021-05-04 16:16:48 +0000 UTC - event for termination-message-container10779dbf-3a4f-48c1-86c0-3b0ea708da7c: {kubelet node2} SandboxChanged: Pod sandbox changed, it will be killed and re-created. May 4 16:21:44.342: INFO: At 2021-05-04 16:16:50 +0000 UTC - event for termination-message-container10779dbf-3a4f-48c1-86c0-3b0ea708da7c: {multus } AddedInterface: Add eth0 [10.244.3.213/24] May 4 16:21:44.342: INFO: At 2021-05-04 16:16:50 +0000 UTC - event for termination-message-container10779dbf-3a4f-48c1-86c0-3b0ea708da7c: {kubelet node2} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 4 16:21:44.342: INFO: At 2021-05-04 16:16:50 +0000 UTC - event for termination-message-container10779dbf-3a4f-48c1-86c0-3b0ea708da7c: {kubelet node2} Failed: Error: ImagePullBackOff May 4 16:21:44.344: INFO: POD NODE PHASE GRACE CONDITIONS May 4 16:21:44.344: INFO: May 4 16:21:44.348: INFO: Logging node info for node master1 May 4 16:21:44.350: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 db982204-549e-4532-90a7-a4410878cfc9 40528 0 2021-05-04 14:43:01 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"3e:f0:43:cb:66:52"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-04 14:43:01 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-04 14:43:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-04 14:45:41 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-04 14:45:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-04 14:51:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-04 14:47:46 +0000 UTC,LastTransitionTime:2021-05-04 14:47:46 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-04 16:21:39 +0000 UTC,LastTransitionTime:2021-05-04 14:43:01 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-04 16:21:39 +0000 UTC,LastTransitionTime:2021-05-04 14:43:01 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-04 16:21:39 +0000 UTC,LastTransitionTime:2021-05-04 14:43:01 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-04 16:21:39 +0000 UTC,LastTransitionTime:2021-05-04 14:47:24 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:88a0771919594d4187f6704fc7592bf8,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:8e0a253b-2aa4-4467-879e-567e7ba1ffa4,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726615179,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:c2fabacbc4e42f3db70f9508e00158b1dce4cf96d91cabaa2eca24e5a0900b66 tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:e074a505e2d62b5119460ab724b2e1df10c8419ef2457f9ce9f3a0f75be3e959 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 4 16:21:44.351: INFO: Logging kubelet events for node master1 May 4 16:21:44.353: INFO: Logging pods the kubelet thinks is on node master1 May 4 16:21:44.362: INFO: kube-apiserver-master1 started at 2021-05-04 14:47:26 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.362: INFO: Container kube-apiserver ready: true, restart count 0 May 4 16:21:44.362: INFO: kube-controller-manager-master1 started at 2021-05-04 14:47:26 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.362: INFO: Container kube-controller-manager ready: true, restart count 2 May 4 16:21:44.362: INFO: kube-proxy-8j6ch started at 2021-05-04 14:45:01 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.362: INFO: Container kube-proxy ready: true, restart count 1 May 4 16:21:44.362: INFO: docker-registry-docker-registry-56cbc7bc58-zhf8t started at 2021-05-04 14:48:42 +0000 UTC (0+2 container statuses recorded) May 4 16:21:44.362: INFO: Container docker-registry ready: true, restart count 0 May 4 16:21:44.362: INFO: Container nginx ready: true, restart count 0 May 4 16:21:44.362: INFO: node-exporter-jckjs started at 2021-05-04 14:56:10 +0000 UTC (0+2 container statuses recorded) May 4 16:21:44.362: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 4 16:21:44.362: INFO: Container node-exporter ready: true, restart count 0 May 4 16:21:44.362: INFO: kube-scheduler-master1 started at 2021-05-04 14:44:16 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.362: INFO: Container kube-scheduler ready: true, restart count 0 May 4 16:21:44.362: INFO: kube-flannel-qspzk started at 2021-05-04 14:45:37 +0000 UTC (1+1 container statuses recorded) May 4 16:21:44.362: INFO: Init container install-cni ready: true, restart count 0 May 4 16:21:44.362: INFO: Container kube-flannel ready: true, restart count 3 May 4 16:21:44.362: INFO: kube-multus-ds-amd64-jflvf started at 2021-05-04 14:45:46 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.362: INFO: Container kube-multus ready: true, restart count 1 May 4 16:21:44.362: INFO: coredns-7677f9bb54-qvcd2 started at 2021-05-04 14:46:11 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.362: INFO: Container coredns ready: true, restart count 1 May 4 16:21:44.362: INFO: node-feature-discovery-controller-5bf5c49849-72rn6 started at 2021-05-04 14:51:52 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.362: INFO: Container nfd-controller ready: true, restart count 0 W0504 16:21:44.375622 27 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 4 16:21:44.407: INFO: Latency metrics for node master1 May 4 16:21:44.407: INFO: Logging node info for node master2 May 4 16:21:44.410: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e2c15170-247b-4e7b-b818-abc807948bf8 40494 0 2021-05-04 14:43:40 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"16:e0:10:a0:e0:62"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-04 14:43:40 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-04 14:43:41 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-04 14:45:41 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-04 14:45:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-04 14:47:32 +0000 UTC,LastTransitionTime:2021-05-04 14:47:32 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-04 16:21:38 +0000 UTC,LastTransitionTime:2021-05-04 14:43:40 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-04 16:21:38 +0000 UTC,LastTransitionTime:2021-05-04 14:43:40 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-04 16:21:38 +0000 UTC,LastTransitionTime:2021-05-04 14:43:40 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-04 16:21:38 +0000 UTC,LastTransitionTime:2021-05-04 14:47:24 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:6af568f56589422a9bd68e0270ce0f8c,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:bf27bb77-fad2-4b52-85c3-acb5113fc512,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726615179,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 4 16:21:44.410: INFO: Logging kubelet events for node master2 May 4 16:21:44.412: INFO: Logging pods the kubelet thinks is on node master2 May 4 16:21:44.421: INFO: kube-flannel-cxdfr started at 2021-05-04 14:45:37 +0000 UTC (1+1 container statuses recorded) May 4 16:21:44.421: INFO: Init container install-cni ready: true, restart count 0 May 4 16:21:44.421: INFO: Container kube-flannel ready: true, restart count 1 May 4 16:21:44.421: INFO: kube-multus-ds-amd64-dw8tg started at 2021-05-04 14:45:46 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.421: INFO: Container kube-multus ready: true, restart count 1 May 4 16:21:44.421: INFO: dns-autoscaler-5b7b5c9b6f-zbrsq started at 2021-05-04 14:46:08 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.421: INFO: Container autoscaler ready: true, restart count 1 May 4 16:21:44.421: INFO: node-exporter-9c6qf started at 2021-05-04 14:56:10 +0000 UTC (0+2 container statuses recorded) May 4 16:21:44.421: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 4 16:21:44.421: INFO: Container node-exporter ready: true, restart count 0 May 4 16:21:44.421: INFO: kube-apiserver-master2 started at 2021-05-04 14:47:26 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.421: INFO: Container kube-apiserver ready: true, restart count 0 May 4 16:21:44.421: INFO: kube-controller-manager-master2 started at 2021-05-04 14:47:26 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.421: INFO: Container kube-controller-manager ready: true, restart count 2 May 4 16:21:44.421: INFO: kube-scheduler-master2 started at 2021-05-04 14:44:16 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.421: INFO: Container kube-scheduler ready: true, restart count 2 May 4 16:21:44.421: INFO: kube-proxy-6b5t8 started at 2021-05-04 14:45:01 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.421: INFO: Container kube-proxy ready: true, restart count 2 W0504 16:21:44.434820 27 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 4 16:21:44.465: INFO: Latency metrics for node master2 May 4 16:21:44.465: INFO: Logging node info for node master3 May 4 16:21:44.468: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 b533a646-667e-403c-944b-71dec9cc4851 40493 0 2021-05-04 14:43:51 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"16:b0:53:14:f6:c9"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-04 14:43:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-04 14:43:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-04 14:45:42 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-04 14:45:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-04 14:48:50 +0000 UTC,LastTransitionTime:2021-05-04 14:48:50 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-04 16:21:38 +0000 UTC,LastTransitionTime:2021-05-04 14:43:51 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-04 16:21:38 +0000 UTC,LastTransitionTime:2021-05-04 14:43:51 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-04 16:21:38 +0000 UTC,LastTransitionTime:2021-05-04 14:43:51 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-04 16:21:38 +0000 UTC,LastTransitionTime:2021-05-04 14:45:46 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:959373dcda56494486f0c2bb0bb496cc,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:0714ca81-c21e-40d6-a288-48d597238e54,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726615179,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 4 16:21:44.468: INFO: Logging kubelet events for node master3 May 4 16:21:44.470: INFO: Logging pods the kubelet thinks is on node master3 May 4 16:21:44.478: INFO: coredns-7677f9bb54-pshfb started at 2021-05-04 14:46:06 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.478: INFO: Container coredns ready: true, restart count 1 May 4 16:21:44.478: INFO: node-exporter-wvppn started at 2021-05-04 14:56:10 +0000 UTC (0+2 container statuses recorded) May 4 16:21:44.478: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 4 16:21:44.478: INFO: Container node-exporter ready: true, restart count 0 May 4 16:21:44.478: INFO: kube-apiserver-master3 started at 2021-05-04 14:44:16 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.478: INFO: Container kube-apiserver ready: true, restart count 0 May 4 16:21:44.478: INFO: kube-controller-manager-master3 started at 2021-05-04 14:44:16 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.478: INFO: Container kube-controller-manager ready: true, restart count 2 May 4 16:21:44.478: INFO: kube-scheduler-master3 started at 2021-05-04 14:44:16 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.479: INFO: Container kube-scheduler ready: true, restart count 2 May 4 16:21:44.479: INFO: kube-proxy-2p5b6 started at 2021-05-04 14:45:01 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.479: INFO: Container kube-proxy ready: true, restart count 2 May 4 16:21:44.479: INFO: kube-flannel-wznt8 started at 2021-05-04 14:45:37 +0000 UTC (1+1 container statuses recorded) May 4 16:21:44.479: INFO: Init container install-cni ready: true, restart count 0 May 4 16:21:44.479: INFO: Container kube-flannel ready: true, restart count 1 May 4 16:21:44.479: INFO: kube-multus-ds-amd64-cgwz2 started at 2021-05-04 14:45:46 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.479: INFO: Container kube-multus ready: true, restart count 1 W0504 16:21:44.490737 27 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 4 16:21:44.515: INFO: Latency metrics for node master3 May 4 16:21:44.515: INFO: Logging node info for node node1 May 4 16:21:44.517: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 b8cf7e16-d5c7-4e2c-996a-93d93bd4fa1c 40571 0 2021-05-04 14:44:58 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"e2:50:df:03:d2:13"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-05-04 14:44:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubeadm Update v1 2021-05-04 14:44:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-04 14:45:42 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-04 14:52:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-04 14:54:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-05-04 14:54:41 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269633024 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884632576 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-04 14:47:29 +0000 UTC,LastTransitionTime:2021-05-04 14:47:29 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-04 16:21:43 +0000 UTC,LastTransitionTime:2021-05-04 14:44:58 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-04 16:21:43 +0000 UTC,LastTransitionTime:2021-05-04 14:44:58 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-04 16:21:43 +0000 UTC,LastTransitionTime:2021-05-04 14:44:58 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-04 16:21:43 +0000 UTC,LastTransitionTime:2021-05-04 14:47:24 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:bffc023a4ab84df0b0181bc7b8f509e2,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:dc08af72-abca-4f1d-bd0f-0e8d8eb97de5,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:511d36b62d758304a5adb26b5996ed37211ab226beb7de4e67994cbecb0279a7 localhost:30500/barometer-collectd:stable],SizeBytes:1464048999,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:1002569035,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f417461c5e0283b5f2ba8e34dc073a15fe1f9ff6b542330c536c86aa72f7141f cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726615179,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:371dc6bf7e0c7ce112a29341b000c40d840aef1dbb4fdcb3ae5c0597e28f3061 golang:alpine3.12],SizeBytes:301097267,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:75a55d33ecc73c2a242450a9f1cc858499d468f077ea942867e662c247b5e412 nginx:1.19],SizeBytes:133117205,},ContainerImage{Names:[httpd@sha256:addd70e4ee83f3bc9a4c1c7c41e37927ba47faf639312fc936df3afad7926f5a httpd:2.4.39-alpine],SizeBytes:126894770,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:25502c57570a2143842478595be9c2a2a3cba2df60b673aef79d6ca80e3eac06 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44395488,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 4 16:21:44.518: INFO: Logging kubelet events for node node1 May 4 16:21:44.519: INFO: Logging pods the kubelet thinks is on node node1 May 4 16:21:44.535: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-hvrmt started at 2021-05-04 14:52:50 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.535: INFO: Container kube-sriovdp ready: true, restart count 0 May 4 16:21:44.535: INFO: nginx-proxy-node1 started at 2021-05-04 14:51:11 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.535: INFO: Container nginx-proxy ready: true, restart count 2 May 4 16:21:44.535: INFO: kube-proxy-t2mbn started at 2021-05-04 14:45:01 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.535: INFO: Container kube-proxy ready: true, restart count 1 May 4 16:21:44.535: INFO: liveness-http started at 2021-05-04 15:33:56 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.535: INFO: Container liveness-http ready: true, restart count 19 May 4 16:21:44.535: INFO: server-envvars-e2e8d4b8-6525-4f40-9a98-8cccf5c227b4 started at 2021-05-04 16:10:40 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.535: INFO: Container srv ready: true, restart count 0 May 4 16:21:44.535: INFO: pod-projected-configmaps-2f4387b8-0930-47f9-8f86-6498d10cab39 started at 2021-05-04 16:21:39 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.535: INFO: Container projected-configmap-volume-test ready: true, restart count 0 May 4 16:21:44.535: INFO: host-test-container-pod started at 2021-05-04 16:21:42 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.535: INFO: Container agnhost-container ready: true, restart count 0 May 4 16:21:44.535: INFO: cmk-init-discover-node1-m8vvw started at 2021-05-04 14:54:32 +0000 UTC (0+3 container statuses recorded) May 4 16:21:44.535: INFO: Container discover ready: false, restart count 0 May 4 16:21:44.535: INFO: Container init ready: false, restart count 0 May 4 16:21:44.535: INFO: Container install ready: false, restart count 0 May 4 16:21:44.535: INFO: client-envvars-3aca01f8-0d22-4951-b441-fa131ddecb49 started at 2021-05-04 16:20:49 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.535: INFO: Container env3cont ready: false, restart count 0 May 4 16:21:44.535: INFO: busybox-readonly-false-12e191e5-ff02-4f5c-8645-7b0e6b4f0c01 started at 2021-05-04 16:21:18 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.535: INFO: Container busybox-readonly-false-12e191e5-ff02-4f5c-8645-7b0e6b4f0c01 ready: false, restart count 0 May 4 16:21:44.535: INFO: kube-multus-ds-amd64-pkmbz started at 2021-05-04 14:45:46 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.535: INFO: Container kube-multus ready: true, restart count 1 May 4 16:21:44.535: INFO: kubernetes-metrics-scraper-678c97765c-6qwqq started at 2021-05-04 14:46:10 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.535: INFO: Container kubernetes-metrics-scraper ready: true, restart count 1 May 4 16:21:44.535: INFO: cmk-slg76 started at 2021-05-04 14:55:14 +0000 UTC (0+2 container statuses recorded) May 4 16:21:44.535: INFO: Container nodereport ready: true, restart count 0 May 4 16:21:44.535: INFO: Container reconcile ready: true, restart count 0 May 4 16:21:44.535: INFO: prometheus-k8s-0 started at 2021-05-04 14:56:12 +0000 UTC (0+5 container statuses recorded) May 4 16:21:44.535: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 4 16:21:44.535: INFO: Container grafana ready: true, restart count 0 May 4 16:21:44.535: INFO: Container prometheus ready: true, restart count 1 May 4 16:21:44.535: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 4 16:21:44.535: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 4 16:21:44.535: INFO: netserver-0 started at 2021-05-04 16:21:19 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.535: INFO: Container webserver ready: true, restart count 0 May 4 16:21:44.535: INFO: test-container-pod started at 2021-05-04 16:21:42 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.535: INFO: Container webserver ready: true, restart count 0 May 4 16:21:44.535: INFO: kube-flannel-d6pbl started at 2021-05-04 14:45:37 +0000 UTC (1+1 container statuses recorded) May 4 16:21:44.535: INFO: Init container install-cni ready: true, restart count 2 May 4 16:21:44.535: INFO: Container kube-flannel ready: true, restart count 2 May 4 16:21:44.535: INFO: node-feature-discovery-worker-wfgl5 started at 2021-05-04 14:51:40 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.535: INFO: Container nfd-worker ready: true, restart count 0 May 4 16:21:44.535: INFO: prometheus-operator-5bb8cb9d8f-rrrhf started at 2021-05-04 14:56:03 +0000 UTC (0+2 container statuses recorded) May 4 16:21:44.535: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 4 16:21:44.535: INFO: Container prometheus-operator ready: true, restart count 0 May 4 16:21:44.535: INFO: node-exporter-k8qd9 started at 2021-05-04 14:56:10 +0000 UTC (0+2 container statuses recorded) May 4 16:21:44.535: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 4 16:21:44.535: INFO: Container node-exporter ready: true, restart count 0 May 4 16:21:44.535: INFO: collectd-4755t started at 2021-05-04 15:01:51 +0000 UTC (0+3 container statuses recorded) May 4 16:21:44.535: INFO: Container collectd ready: true, restart count 0 May 4 16:21:44.535: INFO: Container collectd-exporter ready: true, restart count 0 May 4 16:21:44.535: INFO: Container rbac-proxy ready: true, restart count 0 May 4 16:21:44.535: INFO: fail-once-local-ltx4r started at 2021-05-04 16:18:38 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.536: INFO: Container c ready: false, restart count 0 May 4 16:21:44.536: INFO: pod-adoption started at 2021-05-04 16:21:10 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.536: INFO: Container pod-adoption ready: false, restart count 0 W0504 16:21:44.548367 27 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 4 16:21:44.598: INFO: Latency metrics for node node1 May 4 16:21:44.598: INFO: Logging node info for node node2 May 4 16:21:44.601: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 41567fa7-bb24-4381-9387-e4115195037d 40555 0 2021-05-04 14:44:58 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"32:98:f5:3b:98:5c"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-05-04 14:44:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubeadm Update v1 2021-05-04 14:44:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-04 14:45:42 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-04 14:52:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-04 14:54:58 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-05-04 14:55:01 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269628928 0} {<nil>} 196552372Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884628480 0} {<nil>} 174692020Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-04 14:47:29 +0000 UTC,LastTransitionTime:2021-05-04 14:47:29 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-04 16:21:42 +0000 UTC,LastTransitionTime:2021-05-04 14:44:58 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-04 16:21:42 +0000 UTC,LastTransitionTime:2021-05-04 14:44:58 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-04 16:21:42 +0000 UTC,LastTransitionTime:2021-05-04 14:44:58 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-04 16:21:42 +0000 UTC,LastTransitionTime:2021-05-04 14:45:42 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:d47a1c7ab17f44f2ae7ff788700a8d74,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:df3523a1-a74f-4f8b-beb1-29f5ed8699f3,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:511d36b62d758304a5adb26b5996ed37211ab226beb7de4e67994cbecb0279a7 localhost:30500/barometer-collectd:stable],SizeBytes:1464048999,},ContainerImage{Names:[localhost:30500/cmk@sha256:f417461c5e0283b5f2ba8e34dc073a15fe1f9ff6b542330c536c86aa72f7141f localhost:30500/cmk:v1.5.1],SizeBytes:726615179,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726615179,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[nginx@sha256:75a55d33ecc73c2a242450a9f1cc858499d468f077ea942867e662c247b5e412 nginx:1.19],SizeBytes:133117205,},ContainerImage{Names:[httpd@sha256:addd70e4ee83f3bc9a4c1c7c41e37927ba47faf639312fc936df3afad7926f5a httpd:2.4.39-alpine],SizeBytes:126894770,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:25502c57570a2143842478595be9c2a2a3cba2df60b673aef79d6ca80e3eac06 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44395488,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:c2fabacbc4e42f3db70f9508e00158b1dce4cf96d91cabaa2eca24e5a0900b66 localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:e074a505e2d62b5119460ab724b2e1df10c8419ef2457f9ce9f3a0f75be3e959 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 4 16:21:44.602: INFO: Logging kubelet events for node node2 May 4 16:21:44.604: INFO: Logging pods the kubelet thinks is on node node2 May 4 16:21:44.633: INFO: cmk-init-discover-node2-zlxzj started at 2021-05-04 14:54:52 +0000 UTC (0+3 container statuses recorded) May 4 16:21:44.633: INFO: Container discover ready: false, restart count 0 May 4 16:21:44.633: INFO: Container init ready: false, restart count 0 May 4 16:21:44.633: INFO: Container install ready: false, restart count 0 May 4 16:21:44.633: INFO: collectd-dhwfp started at 2021-05-04 15:01:51 +0000 UTC (0+3 container statuses recorded) May 4 16:21:44.633: INFO: Container collectd ready: true, restart count 0 May 4 16:21:44.633: INFO: Container collectd-exporter ready: true, restart count 0 May 4 16:21:44.633: INFO: Container rbac-proxy ready: true, restart count 0 May 4 16:21:44.633: INFO: fail-once-local-bkr6m started at 2021-05-04 16:18:38 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.633: INFO: Container c ready: false, restart count 0 May 4 16:21:44.633: INFO: nginx-proxy-node2 started at 2021-05-04 14:51:11 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.633: INFO: Container nginx-proxy ready: true, restart count 2 May 4 16:21:44.633: INFO: kubernetes-dashboard-86c6f9df5b-hwbpb started at 2021-05-04 14:46:10 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.633: INFO: Container kubernetes-dashboard ready: true, restart count 1 May 4 16:21:44.633: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-wt4b2 started at 2021-05-04 14:52:50 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.633: INFO: Container kube-sriovdp ready: true, restart count 0 May 4 16:21:44.633: INFO: cmk-2fmbx started at 2021-05-04 14:55:14 +0000 UTC (0+2 container statuses recorded) May 4 16:21:44.633: INFO: Container nodereport ready: true, restart count 0 May 4 16:21:44.633: INFO: Container reconcile ready: true, restart count 0 May 4 16:21:44.633: INFO: busybox-fe2bb9a9-1bbd-4e3b-bdc3-65746a06d3c0 started at 2021-05-04 16:20:23 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.633: INFO: Container busybox ready: false, restart count 0 May 4 16:21:44.633: INFO: kube-flannel-lnwkk started at 2021-05-04 14:45:37 +0000 UTC (1+1 container statuses recorded) May 4 16:21:44.633: INFO: Init container install-cni ready: true, restart count 2 May 4 16:21:44.633: INFO: Container kube-flannel ready: true, restart count 2 May 4 16:21:44.633: INFO: cmk-webhook-6c9d5f8578-fr595 started at 2021-05-04 14:55:15 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.633: INFO: Container cmk-webhook ready: true, restart count 0 May 4 16:21:44.633: INFO: ss-0 started at 2021-05-04 16:17:34 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.633: INFO: Container webserver ready: false, restart count 0 May 4 16:21:44.633: INFO: liveness-exec started at 2021-05-04 15:33:56 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.633: INFO: Container liveness-exec ready: false, restart count 6 May 4 16:21:44.633: INFO: kube-proxy-rfjjf started at 2021-05-04 14:45:01 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.633: INFO: Container kube-proxy ready: true, restart count 2 May 4 16:21:44.633: INFO: node-feature-discovery-worker-jzjqs started at 2021-05-04 14:51:40 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.633: INFO: Container nfd-worker ready: true, restart count 0 May 4 16:21:44.633: INFO: node-exporter-5lghf started at 2021-05-04 14:56:10 +0000 UTC (0+2 container statuses recorded) May 4 16:21:44.633: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 4 16:21:44.633: INFO: Container node-exporter ready: true, restart count 0 May 4 16:21:44.633: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-4nd7x started at 2021-05-04 14:59:02 +0000 UTC (0+2 container statuses recorded) May 4 16:21:44.633: INFO: Container tas-controller ready: true, restart count 0 May 4 16:21:44.633: INFO: Container tas-extender ready: true, restart count 0 May 4 16:21:44.633: INFO: netserver-1 started at 2021-05-04 16:21:20 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.633: INFO: Container webserver ready: true, restart count 0 May 4 16:21:44.633: INFO: kube-multus-ds-amd64-7r2s4 started at 2021-05-04 14:45:46 +0000 UTC (0+1 container statuses recorded) May 4 16:21:44.633: INFO: Container kube-multus ready: true, restart count 1 May 4 16:21:44.633: INFO: pod-init-485103d2-8ff5-4cc8-93a4-a2bc5ba380ee started at 2021-05-04 16:17:49 +0000 UTC (2+1 container statuses recorded) May 4 16:21:44.633: INFO: Init container init1 ready: false, restart count 0 May 4 16:21:44.633: INFO: Init container init2 ready: false, restart count 0 May 4 16:21:44.633: INFO: Container run1 ready: false, restart count 0 W0504 16:21:44.646379 27 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 4 16:21:44.676: INFO: Latency metrics for node node2 May 4 16:21:44.676: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "container-runtime-5315" for this suite. /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 May 4 16:27:03.163: Timed out after 300.000s. Expected <v1.PodPhase>: Pending to equal <v1.PodPhase>: Succeeded /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/runtime.go:154 {"msg":"PASSED [sig-storage] Downward API volume should provide podname only [NodeConformance] [Conformance]","total":-1,"completed":17,"skipped":464,"failed":3,"failures":["[sig-network] Services should be able to create a functioning NodePort service [Conformance]","[k8s.io] Variable Expansion should allow composing env vars into new env vars [NodeConformance] [Conformance]","[k8s.io] Container Runtime blackbox test on terminated container should report termination message [LinuxOnly] if TerminationMessagePath is set as non-root user and at a non-default path [NodeConformance] [Conformance]"]} [BeforeEach] [k8s.io] Container Runtime /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 4 16:22:03.129: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename container-runtime STEP: Waiting for a default service account to be provisioned in namespace [It] should report termination message [LinuxOnly] from file when pod succeeds and TerminationMessagePolicy FallbackToLogsOnError is set [NodeConformance] [Conformance] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 STEP: create the container STEP: wait for the container to reach Succeeded May 4 16:27:03.163: FAIL: Timed out after 300.000s. Expected <v1.PodPhase>: Pending to equal <v1.PodPhase>: Succeeded Full Stack Trace k8s.io/kubernetes/test/e2e/common.glob..func25.1.2.1(0x4c5d8a0, 0x1d, 0xc0004ec0c0, 0x1e, 0xc0046deae0, 0x2, 0x2, 0xc004706570, 0x1, 0x1, ...) /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/runtime.go:154 +0x3f1 k8s.io/kubernetes/test/e2e/common.glob..func25.1.2.6() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/runtime.go:262 +0x1d2 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc002947080) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc002947080) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc002947080, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [k8s.io] Container Runtime /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "container-runtime-7490". STEP: Found 9 events. May 4 16:27:03.175: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for termination-message-containere34b0019-bcb9-4fa0-9c1b-5eb0017d80c4: { } Scheduled: Successfully assigned container-runtime-7490/termination-message-containere34b0019-bcb9-4fa0-9c1b-5eb0017d80c4 to node2 May 4 16:27:03.175: INFO: At 2021-05-04 16:22:04 +0000 UTC - event for termination-message-containere34b0019-bcb9-4fa0-9c1b-5eb0017d80c4: {multus } AddedInterface: Add eth0 [10.244.3.247/24] May 4 16:27:03.175: INFO: At 2021-05-04 16:22:04 +0000 UTC - event for termination-message-containere34b0019-bcb9-4fa0-9c1b-5eb0017d80c4: {kubelet node2} Pulling: Pulling image "docker.io/library/busybox:1.29" May 4 16:27:03.175: INFO: At 2021-05-04 16:22:05 +0000 UTC - event for termination-message-containere34b0019-bcb9-4fa0-9c1b-5eb0017d80c4: {kubelet node2} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 4 16:27:03.175: INFO: At 2021-05-04 16:22:05 +0000 UTC - event for termination-message-containere34b0019-bcb9-4fa0-9c1b-5eb0017d80c4: {kubelet node2} Failed: Error: ErrImagePull May 4 16:27:03.175: INFO: At 2021-05-04 16:22:06 +0000 UTC - event for termination-message-containere34b0019-bcb9-4fa0-9c1b-5eb0017d80c4: {kubelet node2} SandboxChanged: Pod sandbox changed, it will be killed and re-created. May 4 16:27:03.175: INFO: At 2021-05-04 16:22:08 +0000 UTC - event for termination-message-containere34b0019-bcb9-4fa0-9c1b-5eb0017d80c4: {multus } AddedInterface: Add eth0 [10.244.3.248/24] May 4 16:27:03.175: INFO: At 2021-05-04 16:22:08 +0000 UTC - event for termination-message-containere34b0019-bcb9-4fa0-9c1b-5eb0017d80c4: {kubelet node2} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 4 16:27:03.175: INFO: At 2021-05-04 16:22:08 +0000 UTC - event for termination-message-containere34b0019-bcb9-4fa0-9c1b-5eb0017d80c4: {kubelet node2} Failed: Error: ImagePullBackOff May 4 16:27:03.177: INFO: POD NODE PHASE GRACE CONDITIONS May 4 16:27:03.177: INFO: May 4 16:27:03.181: INFO: Logging node info for node master1 May 4 16:27:03.184: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 db982204-549e-4532-90a7-a4410878cfc9 43150 0 2021-05-04 14:43:01 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"3e:f0:43:cb:66:52"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-04 14:43:01 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-04 14:43:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-04 14:45:41 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-04 14:45:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-04 14:51:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-04 14:47:46 +0000 UTC,LastTransitionTime:2021-05-04 14:47:46 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-04 16:27:00 +0000 UTC,LastTransitionTime:2021-05-04 14:43:01 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-04 16:27:00 +0000 UTC,LastTransitionTime:2021-05-04 14:43:01 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-04 16:27:00 +0000 UTC,LastTransitionTime:2021-05-04 14:43:01 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-04 16:27:00 +0000 UTC,LastTransitionTime:2021-05-04 14:47:24 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:88a0771919594d4187f6704fc7592bf8,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:8e0a253b-2aa4-4467-879e-567e7ba1ffa4,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726615179,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:c2fabacbc4e42f3db70f9508e00158b1dce4cf96d91cabaa2eca24e5a0900b66 tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:e074a505e2d62b5119460ab724b2e1df10c8419ef2457f9ce9f3a0f75be3e959 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 4 16:27:03.184: INFO: Logging kubelet events for node master1 May 4 16:27:03.186: INFO: Logging pods the kubelet thinks is on node master1 May 4 16:27:03.198: INFO: kube-controller-manager-master1 started at 2021-05-04 14:47:26 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.198: INFO: Container kube-controller-manager ready: true, restart count 2 May 4 16:27:03.198: INFO: kube-proxy-8j6ch started at 2021-05-04 14:45:01 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.198: INFO: Container kube-proxy ready: true, restart count 1 May 4 16:27:03.198: INFO: docker-registry-docker-registry-56cbc7bc58-zhf8t started at 2021-05-04 14:48:42 +0000 UTC (0+2 container statuses recorded) May 4 16:27:03.198: INFO: Container docker-registry ready: true, restart count 0 May 4 16:27:03.198: INFO: Container nginx ready: true, restart count 0 May 4 16:27:03.198: INFO: node-exporter-jckjs started at 2021-05-04 14:56:10 +0000 UTC (0+2 container statuses recorded) May 4 16:27:03.198: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 4 16:27:03.198: INFO: Container node-exporter ready: true, restart count 0 May 4 16:27:03.198: INFO: kube-scheduler-master1 started at 2021-05-04 14:44:16 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.198: INFO: Container kube-scheduler ready: true, restart count 0 May 4 16:27:03.198: INFO: kube-apiserver-master1 started at 2021-05-04 14:47:26 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.198: INFO: Container kube-apiserver ready: true, restart count 0 May 4 16:27:03.198: INFO: kube-flannel-qspzk started at 2021-05-04 14:45:37 +0000 UTC (1+1 container statuses recorded) May 4 16:27:03.198: INFO: Init container install-cni ready: true, restart count 0 May 4 16:27:03.198: INFO: Container kube-flannel ready: true, restart count 3 May 4 16:27:03.198: INFO: kube-multus-ds-amd64-jflvf started at 2021-05-04 14:45:46 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.198: INFO: Container kube-multus ready: true, restart count 1 May 4 16:27:03.198: INFO: coredns-7677f9bb54-qvcd2 started at 2021-05-04 14:46:11 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.198: INFO: Container coredns ready: true, restart count 1 May 4 16:27:03.198: INFO: node-feature-discovery-controller-5bf5c49849-72rn6 started at 2021-05-04 14:51:52 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.198: INFO: Container nfd-controller ready: true, restart count 0 W0504 16:27:03.211255 27 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 4 16:27:03.234: INFO: Latency metrics for node master1 May 4 16:27:03.234: INFO: Logging node info for node master2 May 4 16:27:03.236: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e2c15170-247b-4e7b-b818-abc807948bf8 43147 0 2021-05-04 14:43:40 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"16:e0:10:a0:e0:62"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-04 14:43:40 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-04 14:43:41 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-04 14:45:41 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-04 14:45:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-04 14:47:32 +0000 UTC,LastTransitionTime:2021-05-04 14:47:32 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-04 16:27:00 +0000 UTC,LastTransitionTime:2021-05-04 14:43:40 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-04 16:27:00 +0000 UTC,LastTransitionTime:2021-05-04 14:43:40 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-04 16:27:00 +0000 UTC,LastTransitionTime:2021-05-04 14:43:40 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-04 16:27:00 +0000 UTC,LastTransitionTime:2021-05-04 14:47:24 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:6af568f56589422a9bd68e0270ce0f8c,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:bf27bb77-fad2-4b52-85c3-acb5113fc512,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726615179,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 4 16:27:03.237: INFO: Logging kubelet events for node master2 May 4 16:27:03.239: INFO: Logging pods the kubelet thinks is on node master2 May 4 16:27:03.246: INFO: kube-controller-manager-master2 started at 2021-05-04 14:47:26 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.246: INFO: Container kube-controller-manager ready: true, restart count 2 May 4 16:27:03.246: INFO: kube-scheduler-master2 started at 2021-05-04 14:44:16 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.246: INFO: Container kube-scheduler ready: true, restart count 2 May 4 16:27:03.246: INFO: kube-proxy-6b5t8 started at 2021-05-04 14:45:01 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.246: INFO: Container kube-proxy ready: true, restart count 2 May 4 16:27:03.246: INFO: kube-flannel-cxdfr started at 2021-05-04 14:45:37 +0000 UTC (1+1 container statuses recorded) May 4 16:27:03.246: INFO: Init container install-cni ready: true, restart count 0 May 4 16:27:03.246: INFO: Container kube-flannel ready: true, restart count 1 May 4 16:27:03.246: INFO: kube-multus-ds-amd64-dw8tg started at 2021-05-04 14:45:46 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.246: INFO: Container kube-multus ready: true, restart count 1 May 4 16:27:03.246: INFO: dns-autoscaler-5b7b5c9b6f-zbrsq started at 2021-05-04 14:46:08 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.246: INFO: Container autoscaler ready: true, restart count 1 May 4 16:27:03.246: INFO: node-exporter-9c6qf started at 2021-05-04 14:56:10 +0000 UTC (0+2 container statuses recorded) May 4 16:27:03.246: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 4 16:27:03.246: INFO: Container node-exporter ready: true, restart count 0 May 4 16:27:03.246: INFO: kube-apiserver-master2 started at 2021-05-04 14:47:26 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.246: INFO: Container kube-apiserver ready: true, restart count 0 W0504 16:27:03.258604 27 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 4 16:27:03.282: INFO: Latency metrics for node master2 May 4 16:27:03.282: INFO: Logging node info for node master3 May 4 16:27:03.285: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 b533a646-667e-403c-944b-71dec9cc4851 43146 0 2021-05-04 14:43:51 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"16:b0:53:14:f6:c9"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-04 14:43:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-04 14:43:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-04 14:45:42 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-04 14:45:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-04 14:48:50 +0000 UTC,LastTransitionTime:2021-05-04 14:48:50 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-04 16:27:00 +0000 UTC,LastTransitionTime:2021-05-04 14:43:51 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-04 16:27:00 +0000 UTC,LastTransitionTime:2021-05-04 14:43:51 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-04 16:27:00 +0000 UTC,LastTransitionTime:2021-05-04 14:43:51 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-04 16:27:00 +0000 UTC,LastTransitionTime:2021-05-04 14:45:46 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:959373dcda56494486f0c2bb0bb496cc,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:0714ca81-c21e-40d6-a288-48d597238e54,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726615179,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 4 16:27:03.286: INFO: Logging kubelet events for node master3 May 4 16:27:03.288: INFO: Logging pods the kubelet thinks is on node master3 May 4 16:27:03.296: INFO: kube-flannel-wznt8 started at 2021-05-04 14:45:37 +0000 UTC (1+1 container statuses recorded) May 4 16:27:03.296: INFO: Init container install-cni ready: true, restart count 0 May 4 16:27:03.296: INFO: Container kube-flannel ready: true, restart count 1 May 4 16:27:03.296: INFO: kube-multus-ds-amd64-cgwz2 started at 2021-05-04 14:45:46 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.296: INFO: Container kube-multus ready: true, restart count 1 May 4 16:27:03.296: INFO: coredns-7677f9bb54-pshfb started at 2021-05-04 14:46:06 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.296: INFO: Container coredns ready: true, restart count 1 May 4 16:27:03.296: INFO: node-exporter-wvppn started at 2021-05-04 14:56:10 +0000 UTC (0+2 container statuses recorded) May 4 16:27:03.296: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 4 16:27:03.296: INFO: Container node-exporter ready: true, restart count 0 May 4 16:27:03.296: INFO: kube-apiserver-master3 started at 2021-05-04 14:44:16 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.296: INFO: Container kube-apiserver ready: true, restart count 0 May 4 16:27:03.296: INFO: kube-controller-manager-master3 started at 2021-05-04 14:44:16 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.296: INFO: Container kube-controller-manager ready: true, restart count 2 May 4 16:27:03.296: INFO: kube-scheduler-master3 started at 2021-05-04 14:44:16 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.296: INFO: Container kube-scheduler ready: true, restart count 2 May 4 16:27:03.296: INFO: kube-proxy-2p5b6 started at 2021-05-04 14:45:01 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.296: INFO: Container kube-proxy ready: true, restart count 2 W0504 16:27:03.310317 27 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 4 16:27:03.334: INFO: Latency metrics for node master3 May 4 16:27:03.334: INFO: Logging node info for node node1 May 4 16:27:03.337: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 b8cf7e16-d5c7-4e2c-996a-93d93bd4fa1c 43131 0 2021-05-04 14:44:58 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"e2:50:df:03:d2:13"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-05-04 14:44:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubeadm Update v1 2021-05-04 14:44:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-04 14:45:42 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-04 14:52:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-04 14:54:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-05-04 14:54:41 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269633024 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884632576 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-04 14:47:29 +0000 UTC,LastTransitionTime:2021-05-04 14:47:29 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-04 16:26:55 +0000 UTC,LastTransitionTime:2021-05-04 14:44:58 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-04 16:26:55 +0000 UTC,LastTransitionTime:2021-05-04 14:44:58 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-04 16:26:55 +0000 UTC,LastTransitionTime:2021-05-04 14:44:58 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-04 16:26:55 +0000 UTC,LastTransitionTime:2021-05-04 14:47:24 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:bffc023a4ab84df0b0181bc7b8f509e2,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:dc08af72-abca-4f1d-bd0f-0e8d8eb97de5,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:511d36b62d758304a5adb26b5996ed37211ab226beb7de4e67994cbecb0279a7 localhost:30500/barometer-collectd:stable],SizeBytes:1464048999,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:1002569035,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f417461c5e0283b5f2ba8e34dc073a15fe1f9ff6b542330c536c86aa72f7141f cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726615179,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:371dc6bf7e0c7ce112a29341b000c40d840aef1dbb4fdcb3ae5c0597e28f3061 golang:alpine3.12],SizeBytes:301097267,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:75a55d33ecc73c2a242450a9f1cc858499d468f077ea942867e662c247b5e412 nginx:1.19],SizeBytes:133117205,},ContainerImage{Names:[httpd@sha256:addd70e4ee83f3bc9a4c1c7c41e37927ba47faf639312fc936df3afad7926f5a httpd:2.4.39-alpine],SizeBytes:126894770,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:25502c57570a2143842478595be9c2a2a3cba2df60b673aef79d6ca80e3eac06 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44395488,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 4 16:27:03.338: INFO: Logging kubelet events for node node1 May 4 16:27:03.340: INFO: Logging pods the kubelet thinks is on node node1 May 4 16:27:03.353: INFO: kubernetes-metrics-scraper-678c97765c-6qwqq started at 2021-05-04 14:46:10 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.353: INFO: Container kubernetes-metrics-scraper ready: true, restart count 1 May 4 16:27:03.353: INFO: cmk-slg76 started at 2021-05-04 14:55:14 +0000 UTC (0+2 container statuses recorded) May 4 16:27:03.353: INFO: Container nodereport ready: true, restart count 0 May 4 16:27:03.353: INFO: Container reconcile ready: true, restart count 0 May 4 16:27:03.353: INFO: prometheus-k8s-0 started at 2021-05-04 14:56:12 +0000 UTC (0+5 container statuses recorded) May 4 16:27:03.353: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 4 16:27:03.353: INFO: Container grafana ready: true, restart count 0 May 4 16:27:03.353: INFO: Container prometheus ready: true, restart count 1 May 4 16:27:03.353: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 4 16:27:03.353: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 4 16:27:03.353: INFO: pod-sharedvolume-86936850-dee1-46bf-8b03-52287eae813c started at 2021-05-04 16:22:19 +0000 UTC (0+2 container statuses recorded) May 4 16:27:03.353: INFO: Container busybox-main-container ready: false, restart count 0 May 4 16:27:03.353: INFO: Container busybox-sub-container ready: false, restart count 0 May 4 16:27:03.353: INFO: kube-flannel-d6pbl started at 2021-05-04 14:45:37 +0000 UTC (1+1 container statuses recorded) May 4 16:27:03.353: INFO: Init container install-cni ready: true, restart count 2 May 4 16:27:03.353: INFO: Container kube-flannel ready: true, restart count 2 May 4 16:27:03.353: INFO: node-feature-discovery-worker-wfgl5 started at 2021-05-04 14:51:40 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.353: INFO: Container nfd-worker ready: true, restart count 0 May 4 16:27:03.353: INFO: prometheus-operator-5bb8cb9d8f-rrrhf started at 2021-05-04 14:56:03 +0000 UTC (0+2 container statuses recorded) May 4 16:27:03.353: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 4 16:27:03.353: INFO: Container prometheus-operator ready: true, restart count 0 May 4 16:27:03.353: INFO: node-exporter-k8qd9 started at 2021-05-04 14:56:10 +0000 UTC (0+2 container statuses recorded) May 4 16:27:03.353: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 4 16:27:03.353: INFO: Container node-exporter ready: true, restart count 0 May 4 16:27:03.353: INFO: collectd-4755t started at 2021-05-04 15:01:51 +0000 UTC (0+3 container statuses recorded) May 4 16:27:03.353: INFO: Container collectd ready: true, restart count 0 May 4 16:27:03.353: INFO: Container collectd-exporter ready: true, restart count 0 May 4 16:27:03.353: INFO: Container rbac-proxy ready: true, restart count 0 May 4 16:27:03.353: INFO: fail-once-local-ltx4r started at 2021-05-04 16:18:38 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.353: INFO: Container c ready: false, restart count 0 May 4 16:27:03.353: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-hvrmt started at 2021-05-04 14:52:50 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.353: INFO: Container kube-sriovdp ready: true, restart count 0 May 4 16:27:03.353: INFO: busybox-user-65534-68ff21a5-e8a6-436a-b96d-8961d5f7a8a0 started at 2021-05-04 16:26:47 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.353: INFO: Container busybox-user-65534-68ff21a5-e8a6-436a-b96d-8961d5f7a8a0 ready: false, restart count 0 May 4 16:27:03.353: INFO: nginx-proxy-node1 started at 2021-05-04 14:51:11 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.353: INFO: Container nginx-proxy ready: true, restart count 2 May 4 16:27:03.353: INFO: kube-proxy-t2mbn started at 2021-05-04 14:45:01 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.353: INFO: Container kube-proxy ready: true, restart count 1 May 4 16:27:03.353: INFO: liveness-http started at 2021-05-04 15:33:56 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.353: INFO: Container liveness-http ready: false, restart count 19 May 4 16:27:03.353: INFO: cmk-init-discover-node1-m8vvw started at 2021-05-04 14:54:32 +0000 UTC (0+3 container statuses recorded) May 4 16:27:03.353: INFO: Container discover ready: false, restart count 0 May 4 16:27:03.353: INFO: Container init ready: false, restart count 0 May 4 16:27:03.353: INFO: Container install ready: false, restart count 0 May 4 16:27:03.353: INFO: kube-multus-ds-amd64-pkmbz started at 2021-05-04 14:45:46 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.353: INFO: Container kube-multus ready: true, restart count 1 W0504 16:27:03.366396 27 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 4 16:27:03.397: INFO: Latency metrics for node node1 May 4 16:27:03.397: INFO: Logging node info for node node2 May 4 16:27:03.400: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 41567fa7-bb24-4381-9387-e4115195037d 43124 0 2021-05-04 14:44:58 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"32:98:f5:3b:98:5c"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-05-04 14:44:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubeadm Update v1 2021-05-04 14:44:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-04 14:45:42 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-04 14:52:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-04 14:54:58 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-05-04 14:55:01 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269628928 0} {<nil>} 196552372Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884628480 0} {<nil>} 174692020Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-04 14:47:29 +0000 UTC,LastTransitionTime:2021-05-04 14:47:29 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-04 16:26:53 +0000 UTC,LastTransitionTime:2021-05-04 14:44:58 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-04 16:26:53 +0000 UTC,LastTransitionTime:2021-05-04 14:44:58 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-04 16:26:53 +0000 UTC,LastTransitionTime:2021-05-04 14:44:58 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-04 16:26:53 +0000 UTC,LastTransitionTime:2021-05-04 14:45:42 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:d47a1c7ab17f44f2ae7ff788700a8d74,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:df3523a1-a74f-4f8b-beb1-29f5ed8699f3,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:511d36b62d758304a5adb26b5996ed37211ab226beb7de4e67994cbecb0279a7 localhost:30500/barometer-collectd:stable],SizeBytes:1464048999,},ContainerImage{Names:[localhost:30500/cmk@sha256:f417461c5e0283b5f2ba8e34dc073a15fe1f9ff6b542330c536c86aa72f7141f localhost:30500/cmk:v1.5.1],SizeBytes:726615179,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726615179,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[nginx@sha256:75a55d33ecc73c2a242450a9f1cc858499d468f077ea942867e662c247b5e412 nginx:1.19],SizeBytes:133117205,},ContainerImage{Names:[httpd@sha256:addd70e4ee83f3bc9a4c1c7c41e37927ba47faf639312fc936df3afad7926f5a httpd:2.4.39-alpine],SizeBytes:126894770,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:25502c57570a2143842478595be9c2a2a3cba2df60b673aef79d6ca80e3eac06 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44395488,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:c2fabacbc4e42f3db70f9508e00158b1dce4cf96d91cabaa2eca24e5a0900b66 localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:e074a505e2d62b5119460ab724b2e1df10c8419ef2457f9ce9f3a0f75be3e959 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 4 16:27:03.401: INFO: Logging kubelet events for node node2 May 4 16:27:03.403: INFO: Logging pods the kubelet thinks is on node node2 May 4 16:27:03.416: INFO: kube-flannel-lnwkk started at 2021-05-04 14:45:37 +0000 UTC (1+1 container statuses recorded) May 4 16:27:03.416: INFO: Init container install-cni ready: true, restart count 2 May 4 16:27:03.416: INFO: Container kube-flannel ready: true, restart count 2 May 4 16:27:03.416: INFO: cmk-webhook-6c9d5f8578-fr595 started at 2021-05-04 14:55:15 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.416: INFO: Container cmk-webhook ready: true, restart count 0 May 4 16:27:03.416: INFO: ss-0 started at 2021-05-04 16:17:34 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.416: INFO: Container webserver ready: false, restart count 0 May 4 16:27:03.416: INFO: var-expansion-792ff743-a8c5-4f3a-94b8-4968bd4cf720 started at 2021-05-04 16:25:54 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.416: INFO: Container dapi-container ready: false, restart count 0 May 4 16:27:03.416: INFO: foo-9dkvq started at 2021-05-04 16:25:24 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.416: INFO: Container c ready: false, restart count 0 May 4 16:27:03.416: INFO: liveness-exec started at 2021-05-04 15:33:56 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.416: INFO: Container liveness-exec ready: false, restart count 6 May 4 16:27:03.416: INFO: kube-proxy-rfjjf started at 2021-05-04 14:45:01 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.416: INFO: Container kube-proxy ready: true, restart count 2 May 4 16:27:03.416: INFO: node-feature-discovery-worker-jzjqs started at 2021-05-04 14:51:40 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.416: INFO: Container nfd-worker ready: true, restart count 0 May 4 16:27:03.416: INFO: node-exporter-5lghf started at 2021-05-04 14:56:10 +0000 UTC (0+2 container statuses recorded) May 4 16:27:03.416: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 4 16:27:03.416: INFO: Container node-exporter ready: true, restart count 0 May 4 16:27:03.416: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-4nd7x started at 2021-05-04 14:59:02 +0000 UTC (0+2 container statuses recorded) May 4 16:27:03.416: INFO: Container tas-controller ready: true, restart count 0 May 4 16:27:03.416: INFO: Container tas-extender ready: true, restart count 0 May 4 16:27:03.416: INFO: kube-multus-ds-amd64-7r2s4 started at 2021-05-04 14:45:46 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.416: INFO: Container kube-multus ready: true, restart count 1 May 4 16:27:03.416: INFO: cmk-init-discover-node2-zlxzj started at 2021-05-04 14:54:52 +0000 UTC (0+3 container statuses recorded) May 4 16:27:03.416: INFO: Container discover ready: false, restart count 0 May 4 16:27:03.416: INFO: Container init ready: false, restart count 0 May 4 16:27:03.416: INFO: Container install ready: false, restart count 0 May 4 16:27:03.416: INFO: collectd-dhwfp started at 2021-05-04 15:01:51 +0000 UTC (0+3 container statuses recorded) May 4 16:27:03.416: INFO: Container collectd ready: true, restart count 0 May 4 16:27:03.416: INFO: Container collectd-exporter ready: true, restart count 0 May 4 16:27:03.416: INFO: Container rbac-proxy ready: true, restart count 0 May 4 16:27:03.416: INFO: fail-once-local-bkr6m started at 2021-05-04 16:18:38 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.416: INFO: Container c ready: false, restart count 0 May 4 16:27:03.416: INFO: foo-sxtvr started at 2021-05-04 16:25:24 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.416: INFO: Container c ready: false, restart count 0 May 4 16:27:03.416: INFO: nginx-proxy-node2 started at 2021-05-04 14:51:11 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.416: INFO: Container nginx-proxy ready: true, restart count 2 May 4 16:27:03.416: INFO: pod-service-account-55278b67-f302-4d69-b992-0113a6bbdd84 started at 2021-05-04 16:26:40 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.416: INFO: Container test ready: false, restart count 0 May 4 16:27:03.416: INFO: kubernetes-dashboard-86c6f9df5b-hwbpb started at 2021-05-04 14:46:10 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.416: INFO: Container kubernetes-dashboard ready: true, restart count 1 May 4 16:27:03.416: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-wt4b2 started at 2021-05-04 14:52:50 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.416: INFO: Container kube-sriovdp ready: true, restart count 0 May 4 16:27:03.416: INFO: cmk-2fmbx started at 2021-05-04 14:55:14 +0000 UTC (0+2 container statuses recorded) May 4 16:27:03.416: INFO: Container nodereport ready: true, restart count 0 May 4 16:27:03.416: INFO: Container reconcile ready: true, restart count 0 May 4 16:27:03.416: INFO: pod-exec-websocket-2863438c-c2df-4c3c-9cd1-2b53e8002946 started at 2021-05-04 16:22:49 +0000 UTC (0+1 container statuses recorded) May 4 16:27:03.416: INFO: Container main ready: false, restart count 0 W0504 16:27:03.428317 27 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 4 16:27:03.499: INFO: Latency metrics for node node2 May 4 16:27:03.499: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "container-runtime-7490" for this suite. /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 May 4 16:31:04.113: while waiting for pod to be running Unexpected error: <*errors.errorString | 0xc000342200>: { s: "timed out waiting for the condition", } timed out waiting for the condition occurred /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/expansion.go:270 [BeforeEach] [k8s.io] Variable Expansion /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:174 STEP: Creating a kubernetes client May 4 16:27:03.556: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object, basename var-expansion STEP: Waiting for a default service account to be provisioned in namespace [It] should verify that a failing subpath expansion can be modified during the lifecycle of a container [sig-storage][Slow] [Conformance] /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:597 STEP: creating the pod with failed condition STEP: updating the pod May 4 16:29:04.105: INFO: Successfully updated pod "var-expansion-98b73d79-7107-4138-a06e-af820041f2eb" STEP: waiting for pod running May 4 16:31:04.113: FAIL: while waiting for pod to be running Unexpected error: <*errors.errorString | 0xc000342200>: { s: "timed out waiting for the condition", } timed out waiting for the condition occurred Full Stack Trace k8s.io/kubernetes/test/e2e/common.glob..func9.7() /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/common/expansion.go:270 +0x5d1 k8s.io/kubernetes/test/e2e.RunE2ETests(0xc002947080) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:130 +0x345 k8s.io/kubernetes/test/e2e.TestE2E(0xc002947080) _output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:145 +0x2b testing.tRunner(0xc002947080, 0x4de37a0) /usr/local/go/src/testing/testing.go:1123 +0xef created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1168 +0x2b3 [AfterEach] [k8s.io] Variable Expansion /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:175 STEP: Collecting events from namespace "var-expansion-5782". STEP: Found 9 events. May 4 16:31:04.119: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for var-expansion-98b73d79-7107-4138-a06e-af820041f2eb: { } Scheduled: Successfully assigned var-expansion-5782/var-expansion-98b73d79-7107-4138-a06e-af820041f2eb to node1 May 4 16:31:04.119: INFO: At 2021-05-04 16:27:05 +0000 UTC - event for var-expansion-98b73d79-7107-4138-a06e-af820041f2eb: {multus } AddedInterface: Add eth0 [10.244.4.212/24] May 4 16:31:04.119: INFO: At 2021-05-04 16:27:05 +0000 UTC - event for var-expansion-98b73d79-7107-4138-a06e-af820041f2eb: {kubelet node1} Pulling: Pulling image "docker.io/library/busybox:1.29" May 4 16:31:04.119: INFO: At 2021-05-04 16:27:06 +0000 UTC - event for var-expansion-98b73d79-7107-4138-a06e-af820041f2eb: {kubelet node1} Failed: Failed to pull image "docker.io/library/busybox:1.29": rpc error: code = Unknown desc = Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit May 4 16:31:04.119: INFO: At 2021-05-04 16:27:06 +0000 UTC - event for var-expansion-98b73d79-7107-4138-a06e-af820041f2eb: {kubelet node1} Failed: Error: ErrImagePull May 4 16:31:04.119: INFO: At 2021-05-04 16:27:07 +0000 UTC - event for var-expansion-98b73d79-7107-4138-a06e-af820041f2eb: {kubelet node1} SandboxChanged: Pod sandbox changed, it will be killed and re-created. May 4 16:31:04.119: INFO: At 2021-05-04 16:27:08 +0000 UTC - event for var-expansion-98b73d79-7107-4138-a06e-af820041f2eb: {multus } AddedInterface: Add eth0 [10.244.4.213/24] May 4 16:31:04.119: INFO: At 2021-05-04 16:27:08 +0000 UTC - event for var-expansion-98b73d79-7107-4138-a06e-af820041f2eb: {kubelet node1} BackOff: Back-off pulling image "docker.io/library/busybox:1.29" May 4 16:31:04.119: INFO: At 2021-05-04 16:27:08 +0000 UTC - event for var-expansion-98b73d79-7107-4138-a06e-af820041f2eb: {kubelet node1} Failed: Error: ImagePullBackOff May 4 16:31:04.121: INFO: POD NODE PHASE GRACE CONDITIONS May 4 16:31:04.121: INFO: var-expansion-98b73d79-7107-4138-a06e-af820041f2eb node1 Pending [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2021-05-04 16:27:03 +0000 UTC } {Ready False 0001-01-01 00:00:00 +0000 UTC 2021-05-04 16:27:03 +0000 UTC ContainersNotReady containers with unready status: [dapi-container]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2021-05-04 16:27:03 +0000 UTC ContainersNotReady containers with unready status: [dapi-container]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2021-05-04 16:27:03 +0000 UTC }] May 4 16:31:04.121: INFO: May 4 16:31:04.125: INFO: Logging node info for node master1 May 4 16:31:04.127: INFO: Node Info: &Node{ObjectMeta:{master1 /api/v1/nodes/master1 db982204-549e-4532-90a7-a4410878cfc9 44937 0 2021-05-04 14:43:01 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master1 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"3e:f0:43:cb:66:52"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.202 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/master.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-04 14:43:01 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-04 14:43:02 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-04 14:45:41 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-04 14:45:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}},"f:taints":{}}}} {nfd-master Update v1 2021-05-04 14:51:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/master.version":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-04 14:47:46 +0000 UTC,LastTransitionTime:2021-05-04 14:47:46 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-04 16:31:01 +0000 UTC,LastTransitionTime:2021-05-04 14:43:01 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-04 16:31:01 +0000 UTC,LastTransitionTime:2021-05-04 14:43:01 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-04 16:31:01 +0000 UTC,LastTransitionTime:2021-05-04 14:43:01 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-04 16:31:01 +0000 UTC,LastTransitionTime:2021-05-04 14:47:24 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.202,},NodeAddress{Type:Hostname,Address:master1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:88a0771919594d4187f6704fc7592bf8,SystemUUID:00ACFB60-0631-E711-906E-0017A4403562,BootID:8e0a253b-2aa4-4467-879e-567e7ba1ffa4,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726615179,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[registry@sha256:1cd9409a311350c3072fe510b52046f104416376c126a479cef9a4dfe692cf57 registry:2.7.0],SizeBytes:24191168,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:c2fabacbc4e42f3db70f9508e00158b1dce4cf96d91cabaa2eca24e5a0900b66 tas-controller:latest localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[nginx@sha256:a97eb9ecc708c8aa715ccfb5e9338f5456e4b65575daf304f108301f3b497314 nginx:1.19.2-alpine],SizeBytes:22052669,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:e074a505e2d62b5119460ab724b2e1df10c8419ef2457f9ce9f3a0f75be3e959 tas-extender:latest localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:5577654,},ContainerImage{Names:[alpine@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a alpine:3.12.1],SizeBytes:5573013,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 4 16:31:04.127: INFO: Logging kubelet events for node master1 May 4 16:31:04.129: INFO: Logging pods the kubelet thinks is on node master1 May 4 16:31:04.144: INFO: kube-flannel-qspzk started at 2021-05-04 14:45:37 +0000 UTC (1+1 container statuses recorded) May 4 16:31:04.144: INFO: Init container install-cni ready: true, restart count 0 May 4 16:31:04.144: INFO: Container kube-flannel ready: true, restart count 3 May 4 16:31:04.144: INFO: kube-multus-ds-amd64-jflvf started at 2021-05-04 14:45:46 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.144: INFO: Container kube-multus ready: true, restart count 1 May 4 16:31:04.144: INFO: coredns-7677f9bb54-qvcd2 started at 2021-05-04 14:46:11 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.144: INFO: Container coredns ready: true, restart count 1 May 4 16:31:04.144: INFO: node-feature-discovery-controller-5bf5c49849-72rn6 started at 2021-05-04 14:51:52 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.144: INFO: Container nfd-controller ready: true, restart count 0 May 4 16:31:04.144: INFO: kube-apiserver-master1 started at 2021-05-04 14:47:26 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.144: INFO: Container kube-apiserver ready: true, restart count 0 May 4 16:31:04.144: INFO: kube-controller-manager-master1 started at 2021-05-04 14:47:26 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.144: INFO: Container kube-controller-manager ready: true, restart count 2 May 4 16:31:04.144: INFO: kube-proxy-8j6ch started at 2021-05-04 14:45:01 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.144: INFO: Container kube-proxy ready: true, restart count 1 May 4 16:31:04.144: INFO: docker-registry-docker-registry-56cbc7bc58-zhf8t started at 2021-05-04 14:48:42 +0000 UTC (0+2 container statuses recorded) May 4 16:31:04.144: INFO: Container docker-registry ready: true, restart count 0 May 4 16:31:04.144: INFO: Container nginx ready: true, restart count 0 May 4 16:31:04.144: INFO: node-exporter-jckjs started at 2021-05-04 14:56:10 +0000 UTC (0+2 container statuses recorded) May 4 16:31:04.144: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 4 16:31:04.144: INFO: Container node-exporter ready: true, restart count 0 May 4 16:31:04.144: INFO: kube-scheduler-master1 started at 2021-05-04 14:44:16 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.144: INFO: Container kube-scheduler ready: true, restart count 0 W0504 16:31:04.158239 27 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 4 16:31:04.191: INFO: Latency metrics for node master1 May 4 16:31:04.191: INFO: Logging node info for node master2 May 4 16:31:04.193: INFO: Node Info: &Node{ObjectMeta:{master2 /api/v1/nodes/master2 e2c15170-247b-4e7b-b818-abc807948bf8 44936 0 2021-05-04 14:43:40 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master2 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"16:e0:10:a0:e0:62"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.203 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-04 14:43:40 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-04 14:43:41 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-04 14:45:41 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-04 14:45:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.1.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.1.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.1.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234763776 0} {<nil>} 196518324Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324599808 0} {<nil>} 195629492Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-04 14:47:32 +0000 UTC,LastTransitionTime:2021-05-04 14:47:32 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-04 16:31:01 +0000 UTC,LastTransitionTime:2021-05-04 14:43:40 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-04 16:31:01 +0000 UTC,LastTransitionTime:2021-05-04 14:43:40 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-04 16:31:01 +0000 UTC,LastTransitionTime:2021-05-04 14:43:40 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-04 16:31:01 +0000 UTC,LastTransitionTime:2021-05-04 14:47:24 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.203,},NodeAddress{Type:Hostname,Address:master2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:6af568f56589422a9bd68e0270ce0f8c,SystemUUID:00A0DE53-E51D-E711-906E-0017A4403562,BootID:bf27bb77-fad2-4b52-85c3-acb5113fc512,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726615179,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 4 16:31:04.194: INFO: Logging kubelet events for node master2 May 4 16:31:04.196: INFO: Logging pods the kubelet thinks is on node master2 May 4 16:31:04.211: INFO: node-exporter-9c6qf started at 2021-05-04 14:56:10 +0000 UTC (0+2 container statuses recorded) May 4 16:31:04.212: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 4 16:31:04.212: INFO: Container node-exporter ready: true, restart count 0 May 4 16:31:04.212: INFO: kube-apiserver-master2 started at 2021-05-04 14:47:26 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.212: INFO: Container kube-apiserver ready: true, restart count 0 May 4 16:31:04.212: INFO: kube-controller-manager-master2 started at 2021-05-04 14:47:26 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.212: INFO: Container kube-controller-manager ready: true, restart count 2 May 4 16:31:04.212: INFO: kube-scheduler-master2 started at 2021-05-04 14:44:16 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.212: INFO: Container kube-scheduler ready: true, restart count 2 May 4 16:31:04.212: INFO: kube-proxy-6b5t8 started at 2021-05-04 14:45:01 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.212: INFO: Container kube-proxy ready: true, restart count 2 May 4 16:31:04.212: INFO: kube-flannel-cxdfr started at 2021-05-04 14:45:37 +0000 UTC (1+1 container statuses recorded) May 4 16:31:04.212: INFO: Init container install-cni ready: true, restart count 0 May 4 16:31:04.212: INFO: Container kube-flannel ready: true, restart count 1 May 4 16:31:04.212: INFO: kube-multus-ds-amd64-dw8tg started at 2021-05-04 14:45:46 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.212: INFO: Container kube-multus ready: true, restart count 1 May 4 16:31:04.212: INFO: dns-autoscaler-5b7b5c9b6f-zbrsq started at 2021-05-04 14:46:08 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.212: INFO: Container autoscaler ready: true, restart count 1 W0504 16:31:04.223214 27 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 4 16:31:04.248: INFO: Latency metrics for node master2 May 4 16:31:04.248: INFO: Logging node info for node master3 May 4 16:31:04.251: INFO: Node Info: &Node{ObjectMeta:{master3 /api/v1/nodes/master3 b533a646-667e-403c-944b-71dec9cc4851 44934 0 2021-05-04 14:43:51 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:master3 kubernetes.io/os:linux node-role.kubernetes.io/master:] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"16:b0:53:14:f6:c9"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.204 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2021-05-04 14:43:51 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}} {kubeadm Update v1 2021-05-04 14:43:52 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/master":{}}}}} {flanneld Update v1 2021-05-04 14:45:42 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {kube-controller-manager Update v1 2021-05-04 14:45:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.2.0/24\"":{}},"f:taints":{}}}}]},Spec:NodeSpec{PodCIDR:10.244.2.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{Taint{Key:node-role.kubernetes.io/master,Value:,Effect:NoSchedule,TimeAdded:<nil>,},},ConfigSource:nil,PodCIDRs:[10.244.2.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{201234767872 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{79550 -3} {<nil>} 79550m DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{200324603904 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-04 14:48:50 +0000 UTC,LastTransitionTime:2021-05-04 14:48:50 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-04 16:31:00 +0000 UTC,LastTransitionTime:2021-05-04 14:43:51 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-04 16:31:00 +0000 UTC,LastTransitionTime:2021-05-04 14:43:51 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-04 16:31:00 +0000 UTC,LastTransitionTime:2021-05-04 14:43:51 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-04 16:31:00 +0000 UTC,LastTransitionTime:2021-05-04 14:45:46 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.204,},NodeAddress{Type:Hostname,Address:master3,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:959373dcda56494486f0c2bb0bb496cc,SystemUUID:008B1444-141E-E711-906E-0017A4403562,BootID:0714ca81-c21e-40d6-a288-48d597238e54,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726615179,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[quay.io/coreos/etcd@sha256:04833b601fa130512450afa45c4fe484fee1293634f34c7ddc231bd193c74017 quay.io/coreos/etcd:v3.4.13],SizeBytes:83790470,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[k8s.gcr.io/coredns@sha256:73ca82b4ce829766d4f1f10947c3a338888f876fbed0540dc849c89ff256e90c k8s.gcr.io/coredns:1.7.0],SizeBytes:45227747,},ContainerImage{Names:[k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64@sha256:dce43068853ad396b0fb5ace9a56cc14114e31979e241342d12d04526be1dfcc k8s.gcr.io/cpa/cluster-proportional-autoscaler-amd64:1.8.3],SizeBytes:40647382,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 4 16:31:04.251: INFO: Logging kubelet events for node master3 May 4 16:31:04.254: INFO: Logging pods the kubelet thinks is on node master3 May 4 16:31:04.268: INFO: kube-flannel-wznt8 started at 2021-05-04 14:45:37 +0000 UTC (1+1 container statuses recorded) May 4 16:31:04.268: INFO: Init container install-cni ready: true, restart count 0 May 4 16:31:04.268: INFO: Container kube-flannel ready: true, restart count 1 May 4 16:31:04.268: INFO: kube-multus-ds-amd64-cgwz2 started at 2021-05-04 14:45:46 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.268: INFO: Container kube-multus ready: true, restart count 1 May 4 16:31:04.268: INFO: coredns-7677f9bb54-pshfb started at 2021-05-04 14:46:06 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.268: INFO: Container coredns ready: true, restart count 1 May 4 16:31:04.268: INFO: node-exporter-wvppn started at 2021-05-04 14:56:10 +0000 UTC (0+2 container statuses recorded) May 4 16:31:04.268: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 4 16:31:04.268: INFO: Container node-exporter ready: true, restart count 0 May 4 16:31:04.268: INFO: kube-apiserver-master3 started at 2021-05-04 14:44:16 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.268: INFO: Container kube-apiserver ready: true, restart count 0 May 4 16:31:04.268: INFO: kube-controller-manager-master3 started at 2021-05-04 14:44:16 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.268: INFO: Container kube-controller-manager ready: true, restart count 2 May 4 16:31:04.268: INFO: kube-scheduler-master3 started at 2021-05-04 14:44:16 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.268: INFO: Container kube-scheduler ready: true, restart count 2 May 4 16:31:04.269: INFO: kube-proxy-2p5b6 started at 2021-05-04 14:45:01 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.269: INFO: Container kube-proxy ready: true, restart count 2 W0504 16:31:04.281926 27 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 4 16:31:04.306: INFO: Latency metrics for node master3 May 4 16:31:04.306: INFO: Logging node info for node node1 May 4 16:31:04.309: INFO: Node Info: &Node{ObjectMeta:{node1 /api/v1/nodes/node1 b8cf7e16-d5c7-4e2c-996a-93d93bd4fa1c 44921 0 2021-05-04 14:44:58 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node1 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"e2:50:df:03:d2:13"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.207 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-05-04 14:44:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.4.0/24\"":{}}}}} {kubeadm Update v1 2021-05-04 14:44:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-04 14:45:42 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-04 14:52:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-04 14:54:35 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-05-04 14:54:41 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.4.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.4.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269633024 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884632576 0} {<nil>} BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-04 14:47:29 +0000 UTC,LastTransitionTime:2021-05-04 14:47:29 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-04 16:30:57 +0000 UTC,LastTransitionTime:2021-05-04 14:44:58 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-04 16:30:57 +0000 UTC,LastTransitionTime:2021-05-04 14:44:58 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-04 16:30:57 +0000 UTC,LastTransitionTime:2021-05-04 14:44:58 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-04 16:30:57 +0000 UTC,LastTransitionTime:2021-05-04 14:47:24 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.207,},NodeAddress{Type:Hostname,Address:node1,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:bffc023a4ab84df0b0181bc7b8f509e2,SystemUUID:00CDA902-D022-E711-906E-0017A4403562,BootID:dc08af72-abca-4f1d-bd0f-0e8d8eb97de5,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:511d36b62d758304a5adb26b5996ed37211ab226beb7de4e67994cbecb0279a7 localhost:30500/barometer-collectd:stable],SizeBytes:1464048999,},ContainerImage{Names:[<none>@<none> <none>:<none>],SizeBytes:1002569035,},ContainerImage{Names:[opnfv/barometer-collectd@sha256:ed5c574f653e2a39e784ff322033a2319aafde7366c803a88f20f7a2a8bc1efb opnfv/barometer-collectd:stable],SizeBytes:825413035,},ContainerImage{Names:[localhost:30500/cmk@sha256:f417461c5e0283b5f2ba8e34dc073a15fe1f9ff6b542330c536c86aa72f7141f cmk:v1.5.1 localhost:30500/cmk:v1.5.1],SizeBytes:726615179,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[golang@sha256:371dc6bf7e0c7ce112a29341b000c40d840aef1dbb4fdcb3ae5c0597e28f3061 golang:alpine3.12],SizeBytes:301097267,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/jessie-dnsutils@sha256:ad583e33cb284f7ef046673809b146ec4053cda19b54a85d2b180a86169715eb gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0],SizeBytes:195659796,},ContainerImage{Names:[grafana/grafana@sha256:89304bc2335f4976618548d7b93d165ed67369d3a051d2f627fc4e0aa3d0aff1 grafana/grafana:7.1.0],SizeBytes:179601493,},ContainerImage{Names:[quay.io/prometheus/prometheus@sha256:d4ba4dd1a9ebb90916d0bfed3c204adcb118ed24546bf8dd2e6b30fc0fd2009e quay.io/prometheus/prometheus:v2.20.0],SizeBytes:144886595,},ContainerImage{Names:[nginx@sha256:75a55d33ecc73c2a242450a9f1cc858499d468f077ea942867e662c247b5e412 nginx:1.19],SizeBytes:133117205,},ContainerImage{Names:[httpd@sha256:addd70e4ee83f3bc9a4c1c7c41e37927ba47faf639312fc936df3afad7926f5a httpd:2.4.39-alpine],SizeBytes:126894770,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[directxman12/k8s-prometheus-adapter-amd64@sha256:b63dc612e3cb73f79d2401a4516f794f9f0a83002600ca72e675e41baecff437 directxman12/k8s-prometheus-adapter-amd64:v0.6.0],SizeBytes:53267842,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:25502c57570a2143842478595be9c2a2a3cba2df60b673aef79d6ca80e3eac06 nfvpe/sriov-device-plugin:latest localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44395488,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:9d07c391aeb1a9d02eb4343c113ed01825227c70c32b3cae861711f90191b0fd quay.io/coreos/kube-rbac-proxy:v0.4.1],SizeBytes:41317870,},ContainerImage{Names:[quay.io/coreos/prometheus-operator@sha256:a54e806fb27d2fb0251da4f3b2a3bb5320759af63a54a755788304775f2384a7 quay.io/coreos/prometheus-operator:v0.40.0],SizeBytes:38238457,},ContainerImage{Names:[kubernetesui/metrics-scraper@sha256:1f977343873ed0e2efd4916a6b2f3075f310ff6fe42ee098f54fc58aa7a28ab7 kubernetesui/metrics-scraper:v1.0.6],SizeBytes:34548789,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[quay.io/coreos/prometheus-config-reloader@sha256:c679a143b24b7731ad1577a9865aa3805426cbf1b25e30807b951dff68466ffd quay.io/coreos/prometheus-config-reloader:v0.40.0],SizeBytes:10131705,},ContainerImage{Names:[jimmidyson/configmap-reload@sha256:d107c7a235c266273b1c3502a391fec374430e5625539403d0de797fa9c556a2 jimmidyson/configmap-reload:v0.3.0],SizeBytes:9700438,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[appropriate/curl@sha256:027a0ad3c69d085fea765afca9984787b780c172cead6502fec989198b98d8bb appropriate/curl:edge],SizeBytes:5654234,},ContainerImage{Names:[alpine@sha256:36553b10a4947067b9fbb7d532951066293a68eae893beba1d9235f7d11a20ad alpine:3.12],SizeBytes:5581415,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 4 16:31:04.310: INFO: Logging kubelet events for node node1 May 4 16:31:04.312: INFO: Logging pods the kubelet thinks is on node node1 May 4 16:31:04.333: INFO: kube-multus-ds-amd64-pkmbz started at 2021-05-04 14:45:46 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.333: INFO: Container kube-multus ready: true, restart count 1 May 4 16:31:04.333: INFO: cmk-slg76 started at 2021-05-04 14:55:14 +0000 UTC (0+2 container statuses recorded) May 4 16:31:04.333: INFO: Container nodereport ready: true, restart count 0 May 4 16:31:04.333: INFO: Container reconcile ready: true, restart count 0 May 4 16:31:04.333: INFO: prometheus-k8s-0 started at 2021-05-04 14:56:12 +0000 UTC (0+5 container statuses recorded) May 4 16:31:04.333: INFO: Container custom-metrics-apiserver ready: true, restart count 0 May 4 16:31:04.333: INFO: Container grafana ready: true, restart count 0 May 4 16:31:04.333: INFO: Container prometheus ready: true, restart count 1 May 4 16:31:04.333: INFO: Container prometheus-config-reloader ready: true, restart count 0 May 4 16:31:04.333: INFO: Container rules-configmap-reloader ready: true, restart count 0 May 4 16:31:04.333: INFO: kubernetes-metrics-scraper-678c97765c-6qwqq started at 2021-05-04 14:46:10 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.333: INFO: Container kubernetes-metrics-scraper ready: true, restart count 1 May 4 16:31:04.333: INFO: node-feature-discovery-worker-wfgl5 started at 2021-05-04 14:51:40 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.333: INFO: Container nfd-worker ready: true, restart count 0 May 4 16:31:04.333: INFO: prometheus-operator-5bb8cb9d8f-rrrhf started at 2021-05-04 14:56:03 +0000 UTC (0+2 container statuses recorded) May 4 16:31:04.333: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 4 16:31:04.333: INFO: Container prometheus-operator ready: true, restart count 0 May 4 16:31:04.333: INFO: node-exporter-k8qd9 started at 2021-05-04 14:56:10 +0000 UTC (0+2 container statuses recorded) May 4 16:31:04.333: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 4 16:31:04.333: INFO: Container node-exporter ready: true, restart count 0 May 4 16:31:04.333: INFO: kube-flannel-d6pbl started at 2021-05-04 14:45:37 +0000 UTC (1+1 container statuses recorded) May 4 16:31:04.333: INFO: Init container install-cni ready: true, restart count 2 May 4 16:31:04.333: INFO: Container kube-flannel ready: true, restart count 2 May 4 16:31:04.333: INFO: fail-once-local-ltx4r started at 2021-05-04 16:18:38 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.333: INFO: Container c ready: false, restart count 0 May 4 16:31:04.333: INFO: collectd-4755t started at 2021-05-04 15:01:51 +0000 UTC (0+3 container statuses recorded) May 4 16:31:04.333: INFO: Container collectd ready: true, restart count 0 May 4 16:31:04.333: INFO: Container collectd-exporter ready: true, restart count 0 May 4 16:31:04.333: INFO: Container rbac-proxy ready: true, restart count 0 May 4 16:31:04.333: INFO: busybox-user-65534-68ff21a5-e8a6-436a-b96d-8961d5f7a8a0 started at 2021-05-04 16:26:47 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.333: INFO: Container busybox-user-65534-68ff21a5-e8a6-436a-b96d-8961d5f7a8a0 ready: false, restart count 0 May 4 16:31:04.333: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-hvrmt started at 2021-05-04 14:52:50 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.333: INFO: Container kube-sriovdp ready: true, restart count 0 May 4 16:31:04.333: INFO: kube-proxy-t2mbn started at 2021-05-04 14:45:01 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.333: INFO: Container kube-proxy ready: true, restart count 1 May 4 16:31:04.333: INFO: liveness-http started at 2021-05-04 15:33:56 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.333: INFO: Container liveness-http ready: false, restart count 21 May 4 16:31:04.333: INFO: nginx-proxy-node1 started at 2021-05-04 14:51:11 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.333: INFO: Container nginx-proxy ready: true, restart count 2 May 4 16:31:04.333: INFO: var-expansion-98b73d79-7107-4138-a06e-af820041f2eb started at 2021-05-04 16:27:03 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.333: INFO: Container dapi-container ready: false, restart count 0 May 4 16:31:04.333: INFO: cmk-init-discover-node1-m8vvw started at 2021-05-04 14:54:32 +0000 UTC (0+3 container statuses recorded) May 4 16:31:04.333: INFO: Container discover ready: false, restart count 0 May 4 16:31:04.333: INFO: Container init ready: false, restart count 0 May 4 16:31:04.333: INFO: Container install ready: false, restart count 0 W0504 16:31:04.345166 27 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 4 16:31:04.393: INFO: Latency metrics for node node1 May 4 16:31:04.393: INFO: Logging node info for node node2 May 4 16:31:04.396: INFO: Node Info: &Node{ObjectMeta:{node2 /api/v1/nodes/node2 41567fa7-bb24-4381-9387-e4115195037d 44914 0 2021-05-04 14:44:58 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux cmk.intel.com/cmk-node:true feature.node.kubernetes.io/cpu-cpuid.ADX:true feature.node.kubernetes.io/cpu-cpuid.AESNI:true feature.node.kubernetes.io/cpu-cpuid.AVX:true feature.node.kubernetes.io/cpu-cpuid.AVX2:true feature.node.kubernetes.io/cpu-cpuid.AVX512BW:true feature.node.kubernetes.io/cpu-cpuid.AVX512CD:true feature.node.kubernetes.io/cpu-cpuid.AVX512DQ:true feature.node.kubernetes.io/cpu-cpuid.AVX512F:true feature.node.kubernetes.io/cpu-cpuid.AVX512VL:true feature.node.kubernetes.io/cpu-cpuid.FMA3:true feature.node.kubernetes.io/cpu-cpuid.HLE:true feature.node.kubernetes.io/cpu-cpuid.IBPB:true feature.node.kubernetes.io/cpu-cpuid.MPX:true feature.node.kubernetes.io/cpu-cpuid.RTM:true feature.node.kubernetes.io/cpu-cpuid.STIBP:true feature.node.kubernetes.io/cpu-cpuid.VMX:true feature.node.kubernetes.io/cpu-hardware_multithreading:true feature.node.kubernetes.io/cpu-pstate.turbo:true feature.node.kubernetes.io/cpu-rdt.RDTCMT:true feature.node.kubernetes.io/cpu-rdt.RDTL3CA:true feature.node.kubernetes.io/cpu-rdt.RDTMBA:true feature.node.kubernetes.io/cpu-rdt.RDTMBM:true feature.node.kubernetes.io/cpu-rdt.RDTMON:true feature.node.kubernetes.io/kernel-config.NO_HZ:true feature.node.kubernetes.io/kernel-config.NO_HZ_FULL:true feature.node.kubernetes.io/kernel-selinux.enabled:true feature.node.kubernetes.io/kernel-version.full:3.10.0-1160.25.1.el7.x86_64 feature.node.kubernetes.io/kernel-version.major:3 feature.node.kubernetes.io/kernel-version.minor:10 feature.node.kubernetes.io/kernel-version.revision:0 feature.node.kubernetes.io/memory-numa:true feature.node.kubernetes.io/network-sriov.capable:true feature.node.kubernetes.io/network-sriov.configured:true feature.node.kubernetes.io/pci-0300_1a03.present:true feature.node.kubernetes.io/storage-nonrotationaldisk:true feature.node.kubernetes.io/system-os_release.ID:centos feature.node.kubernetes.io/system-os_release.VERSION_ID:7 feature.node.kubernetes.io/system-os_release.VERSION_ID.major:7 kubernetes.io/arch:amd64 kubernetes.io/hostname:node2 kubernetes.io/os:linux] map[flannel.alpha.coreos.com/backend-data:{"VtepMAC":"32:98:f5:3b:98:5c"} flannel.alpha.coreos.com/backend-type:vxlan flannel.alpha.coreos.com/kube-subnet-manager:true flannel.alpha.coreos.com/public-ip:10.10.190.208 kubeadm.alpha.kubernetes.io/cri-socket:/var/run/dockershim.sock nfd.node.kubernetes.io/extended-resources: nfd.node.kubernetes.io/feature-labels:cpu-cpuid.ADX,cpu-cpuid.AESNI,cpu-cpuid.AVX,cpu-cpuid.AVX2,cpu-cpuid.AVX512BW,cpu-cpuid.AVX512CD,cpu-cpuid.AVX512DQ,cpu-cpuid.AVX512F,cpu-cpuid.AVX512VL,cpu-cpuid.FMA3,cpu-cpuid.HLE,cpu-cpuid.IBPB,cpu-cpuid.MPX,cpu-cpuid.RTM,cpu-cpuid.STIBP,cpu-cpuid.VMX,cpu-hardware_multithreading,cpu-pstate.turbo,cpu-rdt.RDTCMT,cpu-rdt.RDTL3CA,cpu-rdt.RDTMBA,cpu-rdt.RDTMBM,cpu-rdt.RDTMON,kernel-config.NO_HZ,kernel-config.NO_HZ_FULL,kernel-selinux.enabled,kernel-version.full,kernel-version.major,kernel-version.minor,kernel-version.revision,memory-numa,network-sriov.capable,network-sriov.configured,pci-0300_1a03.present,storage-nonrotationaldisk,system-os_release.ID,system-os_release.VERSION_ID,system-os_release.VERSION_ID.major nfd.node.kubernetes.io/worker.version:v0.7.0 node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kube-controller-manager Update v1 2021-05-04 14:44:58 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}}} {kubeadm Update v1 2021-05-04 14:44:59 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}}}}} {flanneld Update v1 2021-05-04 14:45:42 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:flannel.alpha.coreos.com/backend-data":{},"f:flannel.alpha.coreos.com/backend-type":{},"f:flannel.alpha.coreos.com/kube-subnet-manager":{},"f:flannel.alpha.coreos.com/public-ip":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}}} {nfd-master Update v1 2021-05-04 14:52:04 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:nfd.node.kubernetes.io/extended-resources":{},"f:nfd.node.kubernetes.io/feature-labels":{},"f:nfd.node.kubernetes.io/worker.version":{}},"f:labels":{"f:feature.node.kubernetes.io/cpu-cpuid.ADX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AESNI":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX2":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512BW":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512CD":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512F":{},"f:feature.node.kubernetes.io/cpu-cpuid.AVX512VL":{},"f:feature.node.kubernetes.io/cpu-cpuid.FMA3":{},"f:feature.node.kubernetes.io/cpu-cpuid.HLE":{},"f:feature.node.kubernetes.io/cpu-cpuid.IBPB":{},"f:feature.node.kubernetes.io/cpu-cpuid.MPX":{},"f:feature.node.kubernetes.io/cpu-cpuid.RTM":{},"f:feature.node.kubernetes.io/cpu-cpuid.STIBP":{},"f:feature.node.kubernetes.io/cpu-cpuid.VMX":{},"f:feature.node.kubernetes.io/cpu-hardware_multithreading":{},"f:feature.node.kubernetes.io/cpu-pstate.turbo":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTCMT":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTL3CA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBA":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMBM":{},"f:feature.node.kubernetes.io/cpu-rdt.RDTMON":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ":{},"f:feature.node.kubernetes.io/kernel-config.NO_HZ_FULL":{},"f:feature.node.kubernetes.io/kernel-selinux.enabled":{},"f:feature.node.kubernetes.io/kernel-version.full":{},"f:feature.node.kubernetes.io/kernel-version.major":{},"f:feature.node.kubernetes.io/kernel-version.minor":{},"f:feature.node.kubernetes.io/kernel-version.revision":{},"f:feature.node.kubernetes.io/memory-numa":{},"f:feature.node.kubernetes.io/network-sriov.capable":{},"f:feature.node.kubernetes.io/network-sriov.configured":{},"f:feature.node.kubernetes.io/pci-0300_1a03.present":{},"f:feature.node.kubernetes.io/storage-nonrotationaldisk":{},"f:feature.node.kubernetes.io/system-os_release.ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID":{},"f:feature.node.kubernetes.io/system-os_release.VERSION_ID.major":{}}}}} {Swagger-Codegen Update v1 2021-05-04 14:54:58 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:cmk.intel.com/cmk-node":{}}},"f:status":{"f:capacity":{"f:cmk.intel.com/exclusive-cores":{}}}}} {kubelet Update v1 2021-05-04 14:55:01 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cmk.intel.com/exclusive-cores":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:intel.com/intel_sriov_netdevice":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}}}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{80 0} {<nil>} 80 DecimalSI},ephemeral-storage: {{450471260160 0} {<nil>} 439913340Ki BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{201269628928 0} {<nil>} 196552372Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cmk.intel.com/exclusive-cores: {{3 0} {<nil>} 3 DecimalSI},cpu: {{77 0} {<nil>} 77 DecimalSI},ephemeral-storage: {{405424133473 0} {<nil>} 405424133473 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{21474836480 0} {<nil>} 20Gi BinarySI},intel.com/intel_sriov_netdevice: {{4 0} {<nil>} 4 DecimalSI},memory: {{178884628480 0} {<nil>} 174692020Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2021-05-04 14:47:29 +0000 UTC,LastTransitionTime:2021-05-04 14:47:29 +0000 UTC,Reason:FlannelIsUp,Message:Flannel is running on this node,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-05-04 16:30:54 +0000 UTC,LastTransitionTime:2021-05-04 14:44:58 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-05-04 16:30:54 +0000 UTC,LastTransitionTime:2021-05-04 14:44:58 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-05-04 16:30:54 +0000 UTC,LastTransitionTime:2021-05-04 14:44:58 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-05-04 16:30:54 +0000 UTC,LastTransitionTime:2021-05-04 14:45:42 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.10.190.208,},NodeAddress{Type:Hostname,Address:node2,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:d47a1c7ab17f44f2ae7ff788700a8d74,SystemUUID:80B3CD56-852F-E711-906E-0017A4403562,BootID:df3523a1-a74f-4f8b-beb1-29f5ed8699f3,KernelVersion:3.10.0-1160.25.1.el7.x86_64,OSImage:CentOS Linux 7 (Core),ContainerRuntimeVersion:docker://19.3.14,KubeletVersion:v1.19.8,KubeProxyVersion:v1.19.8,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[localhost:30500/barometer-collectd@sha256:511d36b62d758304a5adb26b5996ed37211ab226beb7de4e67994cbecb0279a7 localhost:30500/barometer-collectd:stable],SizeBytes:1464048999,},ContainerImage{Names:[localhost:30500/cmk@sha256:f417461c5e0283b5f2ba8e34dc073a15fe1f9ff6b542330c536c86aa72f7141f localhost:30500/cmk:v1.5.1],SizeBytes:726615179,},ContainerImage{Names:[cmk:v1.5.1],SizeBytes:726615179,},ContainerImage{Names:[centos/python-36-centos7@sha256:ac50754646f0d37616515fb30467d8743fb12954260ec36c9ecb5a94499447e0 centos/python-36-centos7:latest],SizeBytes:650061677,},ContainerImage{Names:[nfvpe/multus@sha256:ac1266b87ba44c09dc2a336f0d5dad968fccd389ce1944a85e87b32cd21f7224 nfvpe/multus:v3.4.2],SizeBytes:276587882,},ContainerImage{Names:[k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2 k8s.gcr.io/etcd:3.4.13-0],SizeBytes:253392289,},ContainerImage{Names:[kubernetesui/dashboard-amd64@sha256:3af248961c56916aeca8eb4000c15d6cf6a69641ea92f0540865bb37b495932f kubernetesui/dashboard-amd64:v2.1.0],SizeBytes:225733746,},ContainerImage{Names:[nginx@sha256:75a55d33ecc73c2a242450a9f1cc858499d468f077ea942867e662c247b5e412 nginx:1.19],SizeBytes:133117205,},ContainerImage{Names:[httpd@sha256:addd70e4ee83f3bc9a4c1c7c41e37927ba47faf639312fc936df3afad7926f5a httpd:2.4.39-alpine],SizeBytes:126894770,},ContainerImage{Names:[httpd@sha256:eb8ccf084cf3e80eece1add239effefd171eb39adbc154d33c14260d905d4060 httpd:2.4.38-alpine],SizeBytes:123781643,},ContainerImage{Names:[k8s.gcr.io/kube-apiserver@sha256:82e0ce4e1d08f3749d05c584fd60986197bfcdf9ce71d4666c71674221d53135 k8s.gcr.io/kube-apiserver:v1.19.8],SizeBytes:118813022,},ContainerImage{Names:[k8s.gcr.io/kube-proxy@sha256:8ed30419d9cf8965854f9ed501159e15deb30c42c3d2a60a278ae169320d140e k8s.gcr.io/kube-proxy:v1.19.8],SizeBytes:117674285,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:17e61a0b9e498b6c73ed97670906be3d5a3ae394739c1bd5b619e1a004885cf0 k8s.gcr.io/e2e-test-images/agnhost:2.20],SizeBytes:113869866,},ContainerImage{Names:[k8s.gcr.io/kube-controller-manager@sha256:2769005fb667dbb936009894d01fe35f5ce1bce45eee80a9ce3c139b9be4080e k8s.gcr.io/kube-controller-manager:v1.19.8],SizeBytes:110805342,},ContainerImage{Names:[gcr.io/k8s-staging-nfd/node-feature-discovery@sha256:5d116c2c340be665a2c8adc9aca7f91396bd5cbde4add4fdc8dab95d8db43425 gcr.io/k8s-staging-nfd/node-feature-discovery:v0.7.0],SizeBytes:108309584,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/sample-apiserver@sha256:ff02aacd9766d597883fabafc7ad604c719a57611db1bcc1564c69a45b000a55 gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.17],SizeBytes:60684726,},ContainerImage{Names:[quay.io/coreos/flannel@sha256:34860ea294a018d392e61936f19a7862d5e92039d196cac9176da14b2bbd0fe3 quay.io/coreos/flannel@sha256:ac5322604bcab484955e6dbc507f45a906bde79046667322e3918a8578ab08c8 quay.io/coreos/flannel:v0.13.0 quay.io/coreos/flannel:v0.13.0-amd64],SizeBytes:57156911,},ContainerImage{Names:[quay.io/coreos/kube-rbac-proxy@sha256:e10d1d982dd653db74ca87a1d1ad017bc5ef1aeb651bdea089debf16485b080b quay.io/coreos/kube-rbac-proxy:v0.5.0],SizeBytes:46626428,},ContainerImage{Names:[k8s.gcr.io/kube-scheduler@sha256:bb66135ce9a25ac405e43bbae6a2ac766e0efcac0a6a73ef9d1fbb4cf4732c9b k8s.gcr.io/kube-scheduler:v1.19.8],SizeBytes:46510430,},ContainerImage{Names:[localhost:30500/sriov-device-plugin@sha256:25502c57570a2143842478595be9c2a2a3cba2df60b673aef79d6ca80e3eac06 localhost:30500/sriov-device-plugin:v3.3.1],SizeBytes:44395488,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonroot@sha256:4bd7ae247de5c988700233c5a4b55e804ffe90f8c66ae64853f1dae37b847213 gcr.io/kubernetes-e2e-test-images/nonroot:1.0],SizeBytes:42321438,},ContainerImage{Names:[quay.io/prometheus/node-exporter@sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee quay.io/prometheus/node-exporter:v0.18.1],SizeBytes:22933477,},ContainerImage{Names:[localhost:30500/tas-controller@sha256:c2fabacbc4e42f3db70f9508e00158b1dce4cf96d91cabaa2eca24e5a0900b66 localhost:30500/tas-controller:0.1],SizeBytes:22922439,},ContainerImage{Names:[localhost:30500/tas-extender@sha256:e074a505e2d62b5119460ab724b2e1df10c8419ef2457f9ce9f3a0f75be3e959 localhost:30500/tas-extender:0.1],SizeBytes:21320903,},ContainerImage{Names:[prom/collectd-exporter@sha256:73fbda4d24421bff3b741c27efc36f1b6fbe7c57c378d56d4ff78101cd556654],SizeBytes:17463681,},ContainerImage{Names:[nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7 nginx:1.14-alpine],SizeBytes:16032814,},ContainerImage{Names:[gcr.io/google-samples/hello-go-gke@sha256:4ea9cd3d35f81fc91bdebca3fae50c180a1048be0613ad0f811595365040396e gcr.io/google-samples/hello-go-gke:1.0],SizeBytes:11443478,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nonewprivs@sha256:10066e9039219449fe3c81f38fe01928f87914150768ab81b62a468e51fa7411 gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0],SizeBytes:6757579,},ContainerImage{Names:[gcr.io/kubernetes-e2e-test-images/nautilus@sha256:33a732d4c42a266912a5091598a0f07653c9134db4b8d571690d8afd509e0bfc gcr.io/kubernetes-e2e-test-images/nautilus:1.0],SizeBytes:4753501,},ContainerImage{Names:[gcr.io/authenticated-image-pulling/alpine@sha256:7ff177862cb50c602bfe81f805969412e619c054a2bbead977d0c276988aa4a0 gcr.io/authenticated-image-pulling/alpine:3.7],SizeBytes:4206620,},ContainerImage{Names:[busybox@sha256:8ccbac733d19c0dd4d70b4f0c1e12245b5fa3ad24758a11035ee505c629c0796 busybox:1.29],SizeBytes:1154361,},ContainerImage{Names:[busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 busybox:1.28],SizeBytes:1146369,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:927d98197ec1141a368550822d18fa1c60bdae27b78b0c004f705f548c07814f k8s.gcr.io/pause:3.2],SizeBytes:682696,},ContainerImage{Names:[k8s.gcr.io/pause@sha256:a319ac2280eb7e3a59e252e54b76327cb4a33cf8389053b0d78277f22bbca2fa k8s.gcr.io/pause:3.3],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} May 4 16:31:04.397: INFO: Logging kubelet events for node node2 May 4 16:31:04.398: INFO: Logging pods the kubelet thinks is on node node2 May 4 16:31:04.419: INFO: cmk-webhook-6c9d5f8578-fr595 started at 2021-05-04 14:55:15 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.419: INFO: Container cmk-webhook ready: true, restart count 0 May 4 16:31:04.419: INFO: kube-flannel-lnwkk started at 2021-05-04 14:45:37 +0000 UTC (1+1 container statuses recorded) May 4 16:31:04.419: INFO: Init container install-cni ready: true, restart count 2 May 4 16:31:04.419: INFO: Container kube-flannel ready: true, restart count 2 May 4 16:31:04.419: INFO: node-feature-discovery-worker-jzjqs started at 2021-05-04 14:51:40 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.419: INFO: Container nfd-worker ready: true, restart count 0 May 4 16:31:04.419: INFO: node-exporter-5lghf started at 2021-05-04 14:56:10 +0000 UTC (0+2 container statuses recorded) May 4 16:31:04.419: INFO: Container kube-rbac-proxy ready: true, restart count 0 May 4 16:31:04.419: INFO: Container node-exporter ready: true, restart count 0 May 4 16:31:04.419: INFO: tas-telemetry-aware-scheduling-575ccbc9d4-4nd7x started at 2021-05-04 14:59:02 +0000 UTC (0+2 container statuses recorded) May 4 16:31:04.419: INFO: Container tas-controller ready: true, restart count 0 May 4 16:31:04.419: INFO: Container tas-extender ready: true, restart count 0 May 4 16:31:04.419: INFO: foo-9dkvq started at 2021-05-04 16:25:24 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.419: INFO: Container c ready: false, restart count 0 May 4 16:31:04.419: INFO: liveness-exec started at 2021-05-04 15:33:56 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.419: INFO: Container liveness-exec ready: false, restart count 6 May 4 16:31:04.419: INFO: kube-proxy-rfjjf started at 2021-05-04 14:45:01 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.419: INFO: Container kube-proxy ready: true, restart count 2 May 4 16:31:04.419: INFO: kube-multus-ds-amd64-7r2s4 started at 2021-05-04 14:45:46 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.419: INFO: Container kube-multus ready: true, restart count 1 May 4 16:31:04.419: INFO: collectd-dhwfp started at 2021-05-04 15:01:51 +0000 UTC (0+3 container statuses recorded) May 4 16:31:04.419: INFO: Container collectd ready: true, restart count 0 May 4 16:31:04.419: INFO: Container collectd-exporter ready: true, restart count 0 May 4 16:31:04.419: INFO: Container rbac-proxy ready: true, restart count 0 May 4 16:31:04.419: INFO: fail-once-local-bkr6m started at 2021-05-04 16:18:38 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.419: INFO: Container c ready: false, restart count 0 May 4 16:31:04.419: INFO: cmk-init-discover-node2-zlxzj started at 2021-05-04 14:54:52 +0000 UTC (0+3 container statuses recorded) May 4 16:31:04.419: INFO: Container discover ready: false, restart count 0 May 4 16:31:04.419: INFO: Container init ready: false, restart count 0 May 4 16:31:04.419: INFO: Container install ready: false, restart count 0 May 4 16:31:04.419: INFO: downward-api-6bd37b02-f45d-43d9-bbeb-bb1f5f8a4813 started at 2021-05-04 16:27:45 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.419: INFO: Container dapi-container ready: false, restart count 0 May 4 16:31:04.419: INFO: foo-sxtvr started at 2021-05-04 16:25:24 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.419: INFO: Container c ready: false, restart count 0 May 4 16:31:04.419: INFO: nginx-proxy-node2 started at 2021-05-04 14:51:11 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.419: INFO: Container nginx-proxy ready: true, restart count 2 May 4 16:31:04.419: INFO: sriov-net-dp-kube-sriov-device-plugin-amd64-wt4b2 started at 2021-05-04 14:52:50 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.419: INFO: Container kube-sriovdp ready: true, restart count 0 May 4 16:31:04.419: INFO: cmk-2fmbx started at 2021-05-04 14:55:14 +0000 UTC (0+2 container statuses recorded) May 4 16:31:04.419: INFO: Container nodereport ready: true, restart count 0 May 4 16:31:04.419: INFO: Container reconcile ready: true, restart count 0 May 4 16:31:04.419: INFO: pod-service-account-55278b67-f302-4d69-b992-0113a6bbdd84 started at 2021-05-04 16:26:40 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.419: INFO: Container test ready: false, restart count 0 May 4 16:31:04.419: INFO: kubernetes-dashboard-86c6f9df5b-hwbpb started at 2021-05-04 14:46:10 +0000 UTC (0+1 container statuses recorded) May 4 16:31:04.419: INFO: Container kubernetes-dashboard ready: true, restart count 1 W0504 16:31:04.433211 27 metrics_grabber.go:105] Did not receive an external client interface. Grabbing metrics from ClusterAutoscaler is disabled. May 4 16:31:04.463: INFO: Latency metrics for node node2 May 4 16:31:04.463: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "var-expansion-5782" for this suite.